2024-09-15T21:03:56.878 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:03:56.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:03:56.986 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675 branch: main description: rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} email: ceph-qa@ceph.com first_in_suite: false job_id: '7905675' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos service trim min: 5 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client ping interval: 4 mon client ping timeout: 12 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(AUTH_BAD_CAPS\) sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 7498493279982dfe87f00616198a5967475f6169 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 2855 sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 sleep_before_teardown: 0 subset: 17936/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7498493279982dfe87f00616198a5967475f6169 targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9Yo+w+W6sb6hFPrFEIaobsqMfFObtb91VA+VNbixtwmP+LS1efET5s65NmZbeAdiCx69YfmpPZoau9vR+zWio= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJsV6O8rkqJqvpx0qPSJo8KWU+nYdBN/h8k05lym81Q1G7t0oEObYBQHPdmP9qlAF6qZEbXOmuctCKOULywGS7o= tasks: - install: null - ceph: null - mon_thrash: freeze_mon_duration: 20 freeze_mon_probability: 10 revive_delay: 20 thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - mon/pool_ops.sh - mon/crush_ops.sh - mon/osd.sh - mon/caps.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_20:00:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:03:56.986 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa; will attempt to use it 2024-09-15T21:03:56.987 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa/tasks 2024-09-15T21:03:56.987 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:03:56.989 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:03:57.010 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'fc3e0bff7468fd55478adbae7f75508b749124b6' 2024-09-15T21:03:57.010 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:03:57.011 INFO:teuthology.packaging:ref: None 2024-09-15T21:03:57.011 INFO:teuthology.packaging:tag: None 2024-09-15T21:03:57.011 INFO:teuthology.packaging:branch: main 2024-09-15T21:03:57.011 INFO:teuthology.packaging:sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:03:57.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-15T21:03:57.246 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:03:57.248 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:03:57.255 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:03:57.255 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:03:57.261 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:03:57.275 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:03:57.282 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:03:57.307 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:58:22.459170', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICdV7g1fPWs9ZB210AEPSWOqV0Gn++IpHr7PWiO3u2pT'} 2024-09-15T21:03:57.330 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:58:22.457269', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILrH4Luybk94WclYB9uaXLegoKpF6lAyDkhFkFjRQaor'} 2024-09-15T21:03:57.331 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:03:57.337 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-15T21:03:57.337 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-09-15T21:03:57.337 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:03:57.401 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f730726f0a0>, signals=[15]) 2024-09-15T21:03:57.401 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:03:57.406 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:03:57.406 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:03:57.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:03:57.486 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:03:57.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:03:57.564 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:03:57.570 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-09-15T21:03:57.578 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-09-15T21:03:57.578 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-09-15T21:03:57.625 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T21:03:57.625 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Ubuntu" 2024-09-15T21:03:57.625 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="22.04" 2024-09-15T21:03:57.625 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T21:03:57.625 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_CODENAME=jammy 2024-09-15T21:03:57.625 INFO:teuthology.orchestra.run.smithi017.stdout:ID=ubuntu 2024-09-15T21:03:57.626 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE=debian 2024-09-15T21:03:57.626 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T21:03:57.626 INFO:teuthology.orchestra.run.smithi017.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T21:03:57.626 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T21:03:57.626 INFO:teuthology.orchestra.run.smithi017.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T21:03:57.626 INFO:teuthology.orchestra.run.smithi017.stdout:UBUNTU_CODENAME=jammy 2024-09-15T21:03:57.627 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-09-15T21:03:57.645 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-09-15T21:03:57.650 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-09-15T21:03:57.651 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu" 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04" 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T21:03:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T21:03:57.700 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T21:03:57.700 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T21:03:57.700 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy 2024-09-15T21:03:57.700 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-09-15T21:03:57.719 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:03:57.727 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:03:57.733 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:03:57.733 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:03:57.736 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:03:57.746 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:03:57.751 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:03:57.752 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:03:57.783 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:03:57.792 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:03:57.846 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:03:57.861 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:03:57.861 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T21:03:57.862 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:03:57.862 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:03:57.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:03:57.867 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-09-15T21:03:57.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:03:57.868 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-09-15T21:03:57.914 INFO:teuthology.orchestra.run.smithi017.stdout:5.15.0-56-generic 2024-09-15T21:03:57.914 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.15.0-56-generic 2024-09-15T21:03:57.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get clean 2024-09-15T21:03:57.915 INFO:teuthology.orchestra.run.smithi119.stdout:5.15.0-56-generic 2024-09-15T21:03:57.915 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.15.0-56-generic 2024-09-15T21:03:57.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2024-09-15T21:03:58.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-09-15T21:03:58.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-09-15T21:03:58.181 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:03:58.204 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T21:03:58.266 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T21:03:58.297 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T21:03:58.322 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:03:58.324 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T21:03:58.377 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T21:03:58.409 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T21:03:58.481 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T21:03:58.518 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T21:03:58.524 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T21:03:58.525 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T21:03:58.526 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T21:03:58.555 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T21:03:58.559 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T21:03:58.560 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T21:03:58.569 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T21:03:58.575 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T21:03:58.577 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T21:03:58.577 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T21:03:58.578 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T21:03:58.578 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T21:03:58.616 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T21:03:58.617 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T21:03:58.617 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T21:03:58.617 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T21:03:58.618 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T21:03:58.618 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T21:03:58.618 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T21:03:58.619 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T21:03:58.633 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T21:03:58.734 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T21:03:58.760 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T21:03:58.771 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T21:03:58.874 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T21:03:59.191 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T21:03:59.191 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T21:03:59.196 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T21:03:59.207 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T21:03:59.213 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T21:03:59.215 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T21:03:59.216 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T21:03:59.217 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T21:03:59.222 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T21:03:59.222 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T21:03:59.226 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T21:03:59.240 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T21:03:59.241 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T21:03:59.311 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T21:03:59.312 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T21:03:59.313 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T21:03:59.319 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T21:03:59.320 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T21:03:59.325 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T21:03:59.325 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T21:03:59.327 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T21:03:59.347 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T21:03:59.364 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T21:03:59.364 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T21:03:59.364 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T21:03:59.369 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T21:03:59.371 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T21:03:59.375 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T21:03:59.376 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T21:03:59.376 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T21:03:59.376 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T21:03:59.377 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T21:03:59.392 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T21:03:59.395 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T21:03:59.396 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T21:03:59.396 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T21:03:59.397 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T21:03:59.401 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T21:03:59.402 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T21:03:59.403 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T21:03:59.436 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T21:03:59.436 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T21:03:59.451 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T21:03:59.463 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T21:03:59.464 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T21:03:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T21:03:59.487 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T21:03:59.488 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T21:03:59.488 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T21:03:59.488 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T21:03:59.517 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T21:03:59.518 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T21:03:59.518 INFO:teuthology.orchestra.run.smithi017.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T21:04:04.639 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 16.0 MB in 2s (7,552 kB/s) 2024-09-15T21:04:05.881 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 16.0 MB in 3s (5,620 kB/s) 2024-09-15T21:04:06.164 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-15T21:04:06.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T21:04:06.248 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-15T21:04:06.460 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-09-15T21:04:06.461 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-09-15T21:04:06.640 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:04:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:04:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:04:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:04:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T21:04:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: python2.7-minimal 2024-09-15T21:04:06.641 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:04:06.642 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2024-09-15T21:04:06.642 INFO:teuthology.orchestra.run.smithi119.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T21:04:06.642 INFO:teuthology.orchestra.run.smithi119.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T21:04:06.642 INFO:teuthology.orchestra.run.smithi119.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T21:04:06.644 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2024-09-15T21:04:06.644 INFO:teuthology.orchestra.run.smithi119.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T21:04:06.644 INFO:teuthology.orchestra.run.smithi119.stdout:Recommended packages: 2024-09-15T21:04:06.644 INFO:teuthology.orchestra.run.smithi119.stdout: thermald 2024-09-15T21:04:06.693 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2024-09-15T21:04:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T21:04:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T21:04:06.693 INFO:teuthology.orchestra.run.smithi119.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T21:04:06.694 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages will be upgraded: 2024-09-15T21:04:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T21:04:06.875 INFO:teuthology.orchestra.run.smithi119.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T21:04:06.875 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 113 MB of archives. 2024-09-15T21:04:06.875 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T21:04:06.875 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T21:04:07.270 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-15T21:04:07.288 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T21:04:07.352 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-15T21:04:07.563 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-09-15T21:04:07.564 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-09-15T21:04:07.762 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:04:07.762 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:04:07.762 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:04:07.763 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:04:07.763 INFO:teuthology.orchestra.run.smithi017.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T21:04:07.763 INFO:teuthology.orchestra.run.smithi017.stdout: python2.7-minimal 2024-09-15T21:04:07.763 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:04:07.764 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-09-15T21:04:07.764 INFO:teuthology.orchestra.run.smithi017.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T21:04:07.764 INFO:teuthology.orchestra.run.smithi017.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T21:04:07.765 INFO:teuthology.orchestra.run.smithi017.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T21:04:07.766 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-09-15T21:04:07.766 INFO:teuthology.orchestra.run.smithi017.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T21:04:07.767 INFO:teuthology.orchestra.run.smithi017.stdout:Recommended packages: 2024-09-15T21:04:07.767 INFO:teuthology.orchestra.run.smithi017.stdout: thermald 2024-09-15T21:04:07.812 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-09-15T21:04:07.813 INFO:teuthology.orchestra.run.smithi017.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T21:04:07.813 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T21:04:07.813 INFO:teuthology.orchestra.run.smithi017.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T21:04:07.814 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages will be upgraded: 2024-09-15T21:04:07.815 INFO:teuthology.orchestra.run.smithi017.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T21:04:07.867 INFO:teuthology.orchestra.run.smithi017.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T21:04:07.868 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 113 MB of archives. 2024-09-15T21:04:07.868 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T21:04:07.868 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T21:04:08.319 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T21:04:08.336 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T21:04:08.429 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T21:04:08.687 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T21:04:09.276 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T21:04:09.276 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T21:04:09.277 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T21:04:09.407 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T21:04:09.430 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T21:04:09.904 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 113 MB in 2s (67.7 MB/s) 2024-09-15T21:04:10.067 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T21:04:10.891 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T21:04:10.891 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T21:04:10.891 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T21:04:11.236 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T21:04:11.314 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T21:04:11.721 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 113 MB in 5s (24.6 MB/s) 2024-09-15T21:04:11.905 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T21:04:11.995 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T21:04:11.999 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:12.163 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:13.927 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T21:04:13.931 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:14.011 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:14.347 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T21:04:14.368 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:14.434 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:14.875 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T21:04:14.896 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:14.933 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:16.622 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T21:04:16.643 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:16.723 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:17.183 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T21:04:17.204 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:17.233 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:21.639 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:21.757 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:22.081 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:22.227 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:22.530 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T21:04:22.552 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T21:04:22.596 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T21:04:23.657 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:23.784 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:24.100 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:24.221 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T21:04:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T21:04:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T21:04:28.501 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T21:04:28.539 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:28.576 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:30.479 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T21:04:30.516 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:30.554 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:31.208 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:31.329 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:31.680 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T21:04:31.801 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:31.927 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T21:04:32.038 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:33.224 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:33.349 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:33.390 INFO:teuthology.orchestra.run.smithi017.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:33.390 INFO:teuthology.orchestra.run.smithi017.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:33.390 INFO:teuthology.orchestra.run.smithi017.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:33.390 INFO:teuthology.orchestra.run.smithi017.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:33.550 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:33.697 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T21:04:34.654 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:34.780 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T21:04:34.908 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:35.076 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:36.224 INFO:teuthology.orchestra.run.smithi119.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:36.224 INFO:teuthology.orchestra.run.smithi119.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:36.224 INFO:teuthology.orchestra.run.smithi119.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:36.224 INFO:teuthology.orchestra.run.smithi119.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:36.394 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:36.477 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T21:04:36.586 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T21:04:36.688 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:36.746 INFO:teuthology.orchestra.run.smithi017.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T21:04:36.746 INFO:teuthology.orchestra.run.smithi017.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:37.769 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:39.171 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T21:04:39.297 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T21:04:39.448 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:39.517 INFO:teuthology.orchestra.run.smithi119.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T21:04:39.517 INFO:teuthology.orchestra.run.smithi119.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:51.417 INFO:teuthology.orchestra.run.smithi017.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T21:04:51.418 INFO:teuthology.orchestra.run.smithi017.stdout:Sourcing file `/etc/default/grub' 2024-09-15T21:04:51.430 INFO:teuthology.orchestra.run.smithi017.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T21:04:51.445 INFO:teuthology.orchestra.run.smithi017.stdout:Generating grub configuration file ... 2024-09-15T21:04:51.704 INFO:teuthology.orchestra.run.smithi017.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:51.724 INFO:teuthology.orchestra.run.smithi017.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:52.008 INFO:teuthology.orchestra.run.smithi017.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:52.014 INFO:teuthology.orchestra.run.smithi017.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:52.090 INFO:teuthology.orchestra.run.smithi017.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T21:04:52.096 INFO:teuthology.orchestra.run.smithi017.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T21:04:52.267 INFO:teuthology.orchestra.run.smithi017.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T21:04:52.267 INFO:teuthology.orchestra.run.smithi017.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T21:04:52.267 INFO:teuthology.orchestra.run.smithi017.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T21:04:52.306 INFO:teuthology.orchestra.run.smithi017.stdout:done 2024-09-15T21:04:52.763 DEBUG:teuthology.orchestra.run.smithi017:> dpkg -s linux-image-generic 2024-09-15T21:04:52.789 INFO:teuthology.orchestra.run.smithi017.stdout:Package: linux-image-generic 2024-09-15T21:04:52.789 INFO:teuthology.orchestra.run.smithi017.stdout:Status: install ok installed 2024-09-15T21:04:52.789 INFO:teuthology.orchestra.run.smithi017.stdout:Priority: optional 2024-09-15T21:04:52.789 INFO:teuthology.orchestra.run.smithi017.stdout:Section: kernel 2024-09-15T21:04:52.789 INFO:teuthology.orchestra.run.smithi017.stdout:Installed-Size: 21 2024-09-15T21:04:52.789 INFO:teuthology.orchestra.run.smithi017.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: amd64 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Source: linux-meta 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Version: 5.15.0.121.121 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Recommends: thermald 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout:Description: Generic Linux kernel image 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: This package will always depend on the latest generic kernel image 2024-09-15T21:04:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: available. 2024-09-15T21:04:52.790 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T21:04:52.791 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:04:52.791 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:04:52.791 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T21:04:52.791 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:04:52.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get clean 2024-09-15T21:04:53.083 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-09-15T21:04:53.214 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:04:53.344 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:04:53.416 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:04:53.499 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:04:53.940 INFO:teuthology.orchestra.run.smithi119.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T21:04:53.941 INFO:teuthology.orchestra.run.smithi119.stdout:Sourcing file `/etc/default/grub' 2024-09-15T21:04:53.969 INFO:teuthology.orchestra.run.smithi119.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T21:04:53.992 INFO:teuthology.orchestra.run.smithi119.stdout:Generating grub configuration file ... 2024-09-15T21:04:54.235 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:54.257 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:54.532 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:54.537 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:54.608 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T21:04:54.613 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T21:04:54.718 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-15T21:04:54.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T21:04:54.773 INFO:teuthology.orchestra.run.smithi119.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T21:04:54.773 INFO:teuthology.orchestra.run.smithi119.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T21:04:54.773 INFO:teuthology.orchestra.run.smithi119.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T21:04:54.803 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-15T21:04:54.825 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-09-15T21:04:55.012 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-09-15T21:04:55.013 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-09-15T21:04:55.212 INFO:teuthology.orchestra.run.smithi017.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T21:04:55.212 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:04:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:04:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:04:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:04:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:04:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:04:55.213 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:04:55.259 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T21:04:55.261 DEBUG:teuthology.orchestra.run.smithi017:> dpkg -s linux-image-generic 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Package: linux-image-generic 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Status: install ok installed 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Priority: optional 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Section: kernel 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Installed-Size: 21 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: amd64 2024-09-15T21:04:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Source: linux-meta 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout:Version: 5.15.0.121.121 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout:Recommends: thermald 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout:Description: Generic Linux kernel image 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout: This package will always depend on the latest generic kernel image 2024-09-15T21:04:55.276 INFO:teuthology.orchestra.run.smithi017.stdout: available. 2024-09-15T21:04:55.276 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-15T21:04:55.294 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 21 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2024-09-15T21:04:55.314 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 5.15.0.121.121 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2024-09-15T21:04:55.315 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2024-09-15T21:04:55.315 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T21:04:55.315 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:04:55.316 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:04:55.316 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T21:04:55.316 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:04:55.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2024-09-15T21:04:55.321 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.RY20sdBhEM 2024-09-15T21:04:55.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub/grub.cfg /tmp/tmp.RY20sdBhEM 2024-09-15T21:04:55.382 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.RY20sdBhEM 2024-09-15T21:04:55.505 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.RY20sdBhEM is 10KB 2024-09-15T21:04:55.520 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.RY20sdBhEM 2024-09-15T21:04:55.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T21:04:55.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-09-15T21:04:55.582 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:04:55.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T21:04:55.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T21:04:55.649 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T21:04:55.649 DEBUG:teuthology.orchestra.run.smithi017:> sudo update-grub 2024-09-15T21:04:55.790 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:04:55.790 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:04:55.861 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:04:55.942 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:04:56.968 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub' 2024-09-15T21:04:56.969 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T21:04:56.975 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-09-15T21:04:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:57.236 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-15T21:04:57.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T21:04:57.315 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-15T21:04:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:57.425 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T21:04:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T21:04:57.518 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-09-15T21:04:57.519 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-09-15T21:04:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T21:04:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T21:04:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T21:04:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-09-15T21:04:57.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2024-09-15T21:04:57.698 INFO:teuthology.orchestra.run.smithi119.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T21:04:57.698 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:04:57.698 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:04:57.698 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:04:57.698 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:04:57.699 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:04:57.699 INFO:teuthology.orchestra.run.smithi119.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:04:57.699 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:04:57.739 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T21:04:57.741 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 21 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T21:04:57.756 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 5.15.0.121.121 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2024-09-15T21:04:57.757 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2024-09-15T21:04:57.757 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-15T21:04:57.801 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.zBPrTURPq3 2024-09-15T21:04:57.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub/grub.cfg /tmp/tmp.zBPrTURPq3 2024-09-15T21:04:57.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.zBPrTURPq3 2024-09-15T21:04:57.975 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.zBPrTURPq3 is 10KB 2024-09-15T21:04:57.988 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.zBPrTURPq3 2024-09-15T21:04:57.993 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T21:04:58.044 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:04:58.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T21:04:58.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T21:04:58.139 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T21:04:58.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo update-grub 2024-09-15T21:04:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:Sourcing file `/etc/default/grub' 2024-09-15T21:04:59.363 INFO:teuthology.orchestra.run.smithi119.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T21:04:59.365 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-09-15T21:04:59.555 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:59.560 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:59.805 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:59.810 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T21:04:59.853 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T21:04:59.967 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T21:04:59.967 INFO:teuthology.orchestra.run.smithi119.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T21:04:59.967 INFO:teuthology.orchestra.run.smithi119.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T21:04:59.988 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-09-15T21:04:59.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-09-15T21:05:27.619 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T21:05:27.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-15T21:05:27.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:05:30.001 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T21:05:30.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:05:30.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:05:46.045 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-09-15T21:05:48.353 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-15T21:05:55.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-15T21:05:55.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:05:57.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:05:57.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:06:00.413 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-15T21:06:12.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:06:12.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:06:30.845 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-15T21:06:45.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:06:45.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:06:46.187 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-15T21:06:46.778 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:06:46.778 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-15T21:06:46.778 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-09-15T21:06:46.824 INFO:teuthology.orchestra.run.smithi119.stdout:5.15.0-121-generic 2024-09-15T21:06:46.824 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T21:06:46.824 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:06:46.824 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T21:06:47.825 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T21:06:47.825 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:06:47.902 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-09-15T21:06:47.926 DEBUG:teuthology.parallel:result is None 2024-09-15T21:06:55.056 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T21:07:07.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-15T21:07:07.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:07:07.328 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-09-15T21:07:07.936 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-15T21:07:07.937 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-15T21:07:07.937 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-09-15T21:07:07.983 INFO:teuthology.orchestra.run.smithi017.stdout:5.15.0-121-generic 2024-09-15T21:07:07.983 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T21:07:07.983 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:07:07.983 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T21:07:08.984 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T21:07:08.984 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:07:09.044 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2024-09-15T21:07:09.077 DEBUG:teuthology.parallel:result is None 2024-09-15T21:07:09.077 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:07:09.085 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:07:09.085 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:07:09.087 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:07:09.093 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:07:09.099 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:07:09.109 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:07:09.109 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:07:09.136 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:07:09.161 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:07:09.168 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:07:09.168 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:07:09.192 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:07:09.209 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:07:09.217 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:07:09.217 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:07:09.226 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:07:09.227 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:07:09.235 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:07:09.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:07:09.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:07:09.284 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:07:09.294 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:07:09.294 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:07:09.320 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:07:09.332 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:07:09.371 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:07:09.419 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:07:09.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:07:09.479 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:07:09.485 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:07:09.532 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:07:09.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:07:09.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-09-15T21:07:09.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-09-15T21:07:09.670 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:07:09.677 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:07:09.677 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:07:09.743 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:07:09.751 DEBUG:teuthology.task.selinux:Excluding smithi017: OS 'ubuntu' does not support SELinux 2024-09-15T21:07:09.752 DEBUG:teuthology.task.selinux:Excluding smithi119: OS 'ubuntu' does not support SELinux 2024-09-15T21:07:09.752 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:07:09.752 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T21:07:09.752 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:07:09.752 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:07:09.760 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:07:09.773 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:07:09.775 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi119.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:14:46.366 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com')] 2024-09-15T21:14:46.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-15T21:14:46.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:14:46.449 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-09-15T21:14:46.532 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-15T21:14:46.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:14:46.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:14:46.613 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-15T21:14:46.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-15T21:14:46.689 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:14:46.700 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:14:46.700 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:14:46.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:14:46.703 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:14:46.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:14:46.730 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T21:14:46.730 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Command line: ntpd -gq 2024-09-15T21:14:46.730 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: ---------------------------------------------------- 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: corporation. Support and training for ntp-4 are 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: available at https://www.nwtime.org/support 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: ---------------------------------------------------- 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: proto: precision = 0.046 usec (-24) 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: basedate set to 2022-02-04 2024-09-15T21:14:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: gps base set to 2022-02-06 (week 2196) 2024-09-15T21:14:46.732 INFO:teuthology.orchestra.run.smithi017.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:14:46.732 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:14:46.732 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:14:46.732 INFO:teuthology.orchestra.run.smithi017.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:14:46.733 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T21:14:46.733 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T21:14:46.733 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T21:14:46.734 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listen normally on 3 ens1f0 172.21.15.17:123 2024-09-15T21:14:46.734 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listen normally on 4 lo [::1]:123 2024-09-15T21:14:46.734 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:d992%4]:123 2024-09-15T21:14:46.734 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:46 ntpd[18380]: Listening on routing socket on fd #22 for interface updates 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Command line: ntpd -gq 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: ---------------------------------------------------- 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: corporation. Support and training for ntp-4 are 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: available at https://www.nwtime.org/support 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: ---------------------------------------------------- 2024-09-15T21:14:46.759 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: proto: precision = 0.061 usec (-24) 2024-09-15T21:14:46.760 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: basedate set to 2022-02-04 2024-09-15T21:14:46.760 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: gps base set to 2022-02-06 (week 2196) 2024-09-15T21:14:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:14:46.761 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:14:46.761 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:14:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:14:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T21:14:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T21:14:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T21:14:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listen normally on 3 enp3s0f1 172.21.15.119:123 2024-09-15T21:14:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listen normally on 4 lo [::1]:123 2024-09-15T21:14:46.763 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5ef%5]:123 2024-09-15T21:14:46.764 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:46 ntpd[18465]: Listening on routing socket on fd #22 for interface updates 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:47.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:47 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:47 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:47 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:47 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:47 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:47 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:47 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:49.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:49 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:49.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:49 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:50.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:50 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:50.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:50 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:50.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:50 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:50.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:50 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:50.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:50 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:50.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:50 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:50 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:50 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:50 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:50 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:50.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:50 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:50.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:50 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:51.732 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:51 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:51 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:52.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:52 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:52.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:52 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:52.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:52 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:52.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:52 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:52.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:52 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:52.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:52 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:52.759 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:52 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:52 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:52 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stdout:15 Sep 21:14:53 ntpd[18380]: ntpd: time slew -0.000825 s 2024-09-15T21:14:53.731 INFO:teuthology.orchestra.run.smithi017.stdout:ntpd: time slew -0.000825s 2024-09-15T21:14:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T21:14:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:15 Sep 21:14:53 ntpd[18380]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:53.760 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:53.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:53.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:53.761 INFO:teuthology.orchestra.run.smithi119.stdout:15 Sep 21:14:53 ntpd[18465]: ntpd: time slew -0.001547 s 2024-09-15T21:14:53.761 INFO:teuthology.orchestra.run.smithi119.stdout:ntpd: time slew -0.001547s 2024-09-15T21:14:53.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T21:14:53.761 INFO:teuthology.orchestra.run.smithi119.stderr:15 Sep 21:14:53 ntpd[18465]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T21:14:53.787 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:14:53.787 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2024-09-15T21:14:53.787 INFO:teuthology.orchestra.run.smithi017.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.787 INFO:teuthology.orchestra.run.smithi017.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.787 INFO:teuthology.orchestra.run.smithi017.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.787 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.811 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:14:53.811 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2024-09-15T21:14:53.811 INFO:teuthology.orchestra.run.smithi119.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.811 INFO:teuthology.orchestra.run.smithi119.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.811 INFO:teuthology.orchestra.run.smithi119.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.811 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:53.812 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:14:53.820 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:14:53.820 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}} 2024-09-15T21:14:53.820 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'} 2024-09-15T21:14:53.820 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:14:53.828 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:14:53.828 INFO:teuthology.task.install:extra packages: [] 2024-09-15T21:14:53.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-key list | grep Ceph 2024-09-15T21:14:53.829 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-key list | grep Ceph 2024-09-15T21:14:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T21:14:53.912 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T21:14:53.914 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T21:14:53.915 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T21:14:53.916 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T21:14:53.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:14:53.938 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T21:14:53.939 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T21:14:53.939 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T21:14:53.939 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:14:54.100 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T21:14:54.100 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:14:54.124 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T21:14:54.124 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:14:54.245 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:14:54.245 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T21:14:54.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-09-15T21:14:54.269 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:14:54.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T21:14:54.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-09-15T21:14:54.450 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:14:54.451 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:14:54.468 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:14:54.468 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:14:54.469 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:14:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:14:54.494 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:14:54.514 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:14:54.525 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T21:14:54.544 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T21:14:54.555 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T21:14:54.575 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T21:14:54.585 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T21:14:54.606 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T21:14:54.617 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T21:14:54.641 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T21:14:54.654 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T21:14:54.676 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T21:14:54.814 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 29.2 kB in 0s (78.8 kB/s) 2024-09-15T21:14:54.828 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 29.2 kB in 0s (81.5 kB/s) 2024-09-15T21:14:55.849 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-15T21:14:55.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:14:55.873 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-15T21:14:55.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:14:55.933 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-15T21:14:55.950 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-09-15T21:14:56.144 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-09-15T21:14:56.144 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-09-15T21:14:56.160 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-09-15T21:14:56.160 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-09-15T21:14:56.352 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:14:56.352 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:14:56.352 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:14:56.353 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:14:56.353 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:14:56.353 INFO:teuthology.orchestra.run.smithi119.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:14:56.353 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:14:56.354 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2024-09-15T21:14:56.354 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T21:14:56.354 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T21:14:56.354 INFO:teuthology.orchestra.run.smithi119.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T21:14:56.354 INFO:teuthology.orchestra.run.smithi119.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T21:14:56.354 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T21:14:56.356 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T21:14:56.357 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet zip 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T21:14:56.358 INFO:teuthology.orchestra.run.smithi119.stdout:Recommended packages: 2024-09-15T21:14:56.359 INFO:teuthology.orchestra.run.smithi119.stdout: btrfs-tools 2024-09-15T21:14:56.365 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:14:56.365 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:14:56.365 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:14:56.366 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:14:56.366 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:14:56.366 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:14:56.366 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:14:56.367 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-09-15T21:14:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T21:14:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T21:14:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T21:14:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T21:14:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T21:14:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T21:14:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet zip 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout:Recommended packages: 2024-09-15T21:14:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: btrfs-tools 2024-09-15T21:14:56.459 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2024-09-15T21:14:56.459 INFO:teuthology.orchestra.run.smithi119.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T21:14:56.459 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T21:14:56.460 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T21:14:56.461 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T21:14:56.462 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T21:14:56.462 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T21:14:56.462 INFO:teuthology.orchestra.run.smithi119.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T21:14:56.481 INFO:teuthology.orchestra.run.smithi017.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T21:14:56.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T21:14:56.484 INFO:teuthology.orchestra.run.smithi017.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T21:14:56.560 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T21:14:56.561 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 244 MB of archives. 2024-09-15T21:14:56.561 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T21:14:56.561 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T21:14:56.644 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T21:14:56.644 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 244 MB of archives. 2024-09-15T21:14:56.644 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T21:14:56.645 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T21:14:56.664 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T21:14:56.671 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T21:14:56.691 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T21:14:56.694 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T21:14:56.723 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T21:14:56.803 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T21:14:56.809 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T21:14:56.845 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T21:14:56.850 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T21:14:56.852 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T21:14:56.852 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T21:14:56.854 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T21:14:56.865 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T21:14:56.868 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T21:14:56.871 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T21:14:56.873 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T21:14:56.892 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T21:14:56.896 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T21:14:56.899 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T21:14:56.902 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T21:14:56.902 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T21:14:56.923 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T21:14:56.925 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T21:14:56.925 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T21:14:56.926 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T21:14:56.926 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T21:14:56.933 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T21:14:56.949 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T21:14:56.950 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T21:14:56.950 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T21:14:56.955 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T21:14:56.955 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T21:14:56.972 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T21:14:56.973 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T21:14:56.974 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T21:14:56.975 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T21:14:56.976 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T21:14:56.990 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T21:14:56.997 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T21:14:56.998 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T21:14:56.998 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T21:14:57.007 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T21:14:57.007 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T21:14:57.023 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T21:14:57.023 INFO:teuthology.orchestra.run.smithi017.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T21:14:57.025 INFO:teuthology.orchestra.run.smithi017.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T21:14:57.028 INFO:teuthology.orchestra.run.smithi017.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T21:14:57.029 INFO:teuthology.orchestra.run.smithi017.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T21:14:57.040 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T21:14:57.047 INFO:teuthology.orchestra.run.smithi017.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T21:14:57.048 INFO:teuthology.orchestra.run.smithi017.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T21:14:57.052 INFO:teuthology.orchestra.run.smithi017.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T21:14:57.053 INFO:teuthology.orchestra.run.smithi017.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T21:14:57.054 INFO:teuthology.orchestra.run.smithi017.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T21:14:57.065 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T21:14:57.071 INFO:teuthology.orchestra.run.smithi017.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T21:14:57.072 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T21:14:57.073 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T21:14:57.089 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T21:14:57.090 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T21:14:57.098 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T21:14:57.103 INFO:teuthology.orchestra.run.smithi017.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T21:14:57.122 INFO:teuthology.orchestra.run.smithi017.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T21:14:57.131 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T21:14:57.139 INFO:teuthology.orchestra.run.smithi017.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T21:14:57.140 INFO:teuthology.orchestra.run.smithi017.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T21:14:57.143 INFO:teuthology.orchestra.run.smithi017.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T21:14:57.144 INFO:teuthology.orchestra.run.smithi017.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T21:14:57.145 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T21:14:57.145 INFO:teuthology.orchestra.run.smithi017.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T21:14:57.146 INFO:teuthology.orchestra.run.smithi017.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T21:14:57.146 INFO:teuthology.orchestra.run.smithi017.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T21:14:57.146 INFO:teuthology.orchestra.run.smithi017.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T21:14:57.147 INFO:teuthology.orchestra.run.smithi017.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T21:14:57.154 INFO:teuthology.orchestra.run.smithi017.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T21:14:57.165 INFO:teuthology.orchestra.run.smithi017.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T21:14:57.167 INFO:teuthology.orchestra.run.smithi017.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T21:14:57.168 INFO:teuthology.orchestra.run.smithi017.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T21:14:57.185 INFO:teuthology.orchestra.run.smithi017.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T21:14:57.185 INFO:teuthology.orchestra.run.smithi017.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T21:14:57.197 INFO:teuthology.orchestra.run.smithi017.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T21:14:57.197 INFO:teuthology.orchestra.run.smithi017.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T21:14:57.197 INFO:teuthology.orchestra.run.smithi017.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T21:14:57.198 INFO:teuthology.orchestra.run.smithi017.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T21:14:57.198 INFO:teuthology.orchestra.run.smithi017.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T21:14:57.199 INFO:teuthology.orchestra.run.smithi017.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T21:14:57.202 INFO:teuthology.orchestra.run.smithi017.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T21:14:57.221 INFO:teuthology.orchestra.run.smithi017.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T21:14:57.222 INFO:teuthology.orchestra.run.smithi017.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T21:14:57.222 INFO:teuthology.orchestra.run.smithi017.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T21:14:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T21:14:57.246 INFO:teuthology.orchestra.run.smithi017.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T21:14:57.248 INFO:teuthology.orchestra.run.smithi017.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T21:14:57.249 INFO:teuthology.orchestra.run.smithi017.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T21:14:57.249 INFO:teuthology.orchestra.run.smithi017.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T21:14:57.253 INFO:teuthology.orchestra.run.smithi017.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T21:14:57.253 INFO:teuthology.orchestra.run.smithi017.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T21:14:57.256 INFO:teuthology.orchestra.run.smithi017.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T21:14:57.272 INFO:teuthology.orchestra.run.smithi017.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T21:14:57.272 INFO:teuthology.orchestra.run.smithi017.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T21:14:57.273 INFO:teuthology.orchestra.run.smithi017.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T21:14:57.297 INFO:teuthology.orchestra.run.smithi017.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T21:14:57.297 INFO:teuthology.orchestra.run.smithi017.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T21:14:57.304 INFO:teuthology.orchestra.run.smithi017.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T21:14:57.304 INFO:teuthology.orchestra.run.smithi017.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T21:14:57.304 INFO:teuthology.orchestra.run.smithi017.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T21:14:57.306 INFO:teuthology.orchestra.run.smithi017.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T21:14:57.307 INFO:teuthology.orchestra.run.smithi017.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T21:14:57.383 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T21:14:57.386 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T21:14:57.395 INFO:teuthology.orchestra.run.smithi017.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T21:14:57.396 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T21:14:57.397 INFO:teuthology.orchestra.run.smithi017.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T21:14:57.404 INFO:teuthology.orchestra.run.smithi017.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T21:14:57.486 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T21:14:57.497 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T21:14:57.532 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T21:14:57.541 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T21:14:57.544 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T21:14:57.545 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T21:14:57.546 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T21:14:57.566 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T21:14:57.571 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T21:14:57.575 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T21:14:57.664 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T21:14:57.665 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T21:14:57.669 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T21:14:57.673 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T21:14:57.676 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T21:14:57.677 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T21:14:57.678 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T21:14:57.680 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T21:14:57.752 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T21:14:57.753 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T21:14:57.839 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T21:14:57.839 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T21:14:57.840 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T21:14:57.841 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T21:14:57.846 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T21:14:57.847 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T21:14:57.848 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T21:14:57.849 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T21:14:57.925 INFO:teuthology.orchestra.run.smithi119.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T21:14:57.926 INFO:teuthology.orchestra.run.smithi119.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T21:14:58.011 INFO:teuthology.orchestra.run.smithi119.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T21:14:58.014 INFO:teuthology.orchestra.run.smithi119.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T21:14:58.015 INFO:teuthology.orchestra.run.smithi119.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T21:14:58.016 INFO:teuthology.orchestra.run.smithi119.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T21:14:58.027 INFO:teuthology.orchestra.run.smithi119.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T21:14:58.027 INFO:teuthology.orchestra.run.smithi119.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T21:14:58.028 INFO:teuthology.orchestra.run.smithi119.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T21:14:58.029 INFO:teuthology.orchestra.run.smithi119.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T21:14:58.097 INFO:teuthology.orchestra.run.smithi119.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T21:14:58.101 INFO:teuthology.orchestra.run.smithi119.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T21:14:58.120 INFO:teuthology.orchestra.run.smithi119.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T21:14:58.122 INFO:teuthology.orchestra.run.smithi119.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T21:14:58.124 INFO:teuthology.orchestra.run.smithi119.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T21:14:58.128 INFO:teuthology.orchestra.run.smithi119.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T21:14:58.129 INFO:teuthology.orchestra.run.smithi119.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T21:14:58.131 INFO:teuthology.orchestra.run.smithi119.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T21:14:58.189 INFO:teuthology.orchestra.run.smithi119.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T21:14:58.378 INFO:teuthology.orchestra.run.smithi119.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T21:14:58.438 INFO:teuthology.orchestra.run.smithi017.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T21:14:58.469 INFO:teuthology.orchestra.run.smithi119.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T21:14:58.470 INFO:teuthology.orchestra.run.smithi119.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T21:14:58.471 INFO:teuthology.orchestra.run.smithi119.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T21:14:58.472 INFO:teuthology.orchestra.run.smithi119.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T21:14:58.475 INFO:teuthology.orchestra.run.smithi119.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T21:14:58.521 INFO:teuthology.orchestra.run.smithi119.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T21:14:58.525 INFO:teuthology.orchestra.run.smithi119.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T21:14:58.526 INFO:teuthology.orchestra.run.smithi119.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T21:14:58.529 INFO:teuthology.orchestra.run.smithi017.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T21:14:58.550 INFO:teuthology.orchestra.run.smithi017.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T21:14:58.552 INFO:teuthology.orchestra.run.smithi119.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T21:14:58.554 INFO:teuthology.orchestra.run.smithi017.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T21:14:58.554 INFO:teuthology.orchestra.run.smithi119.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T21:14:58.556 INFO:teuthology.orchestra.run.smithi119.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T21:14:58.577 INFO:teuthology.orchestra.run.smithi119.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T21:14:58.577 INFO:teuthology.orchestra.run.smithi119.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T21:14:58.597 INFO:teuthology.orchestra.run.smithi017.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T21:14:58.608 INFO:teuthology.orchestra.run.smithi119.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T21:14:58.642 INFO:teuthology.orchestra.run.smithi119.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T21:14:58.643 INFO:teuthology.orchestra.run.smithi119.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T21:14:58.644 INFO:teuthology.orchestra.run.smithi119.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T21:14:58.645 INFO:teuthology.orchestra.run.smithi119.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T21:14:58.646 INFO:teuthology.orchestra.run.smithi119.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T21:14:58.647 INFO:teuthology.orchestra.run.smithi119.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T21:14:58.654 INFO:teuthology.orchestra.run.smithi119.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T21:14:58.729 INFO:teuthology.orchestra.run.smithi119.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T21:14:58.732 INFO:teuthology.orchestra.run.smithi119.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T21:14:58.734 INFO:teuthology.orchestra.run.smithi119.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T21:14:58.805 INFO:teuthology.orchestra.run.smithi017.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T21:14:58.812 INFO:teuthology.orchestra.run.smithi119.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T21:14:58.814 INFO:teuthology.orchestra.run.smithi119.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T21:14:58.818 INFO:teuthology.orchestra.run.smithi119.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T21:14:58.820 INFO:teuthology.orchestra.run.smithi119.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T21:14:58.822 INFO:teuthology.orchestra.run.smithi119.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T21:14:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T21:14:58.834 INFO:teuthology.orchestra.run.smithi119.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T21:14:58.842 INFO:teuthology.orchestra.run.smithi119.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T21:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T21:14:58.896 INFO:teuthology.orchestra.run.smithi119.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T21:14:58.897 INFO:teuthology.orchestra.run.smithi119.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T21:14:58.978 INFO:teuthology.orchestra.run.smithi119.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T21:14:58.980 INFO:teuthology.orchestra.run.smithi119.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T21:14:58.997 INFO:teuthology.orchestra.run.smithi119.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T21:14:58.998 INFO:teuthology.orchestra.run.smithi119.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T21:14:58.999 INFO:teuthology.orchestra.run.smithi119.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T21:14:59.004 INFO:teuthology.orchestra.run.smithi119.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T21:14:59.006 INFO:teuthology.orchestra.run.smithi119.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T21:14:59.806 INFO:teuthology.orchestra.run.smithi017.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T21:14:59.806 INFO:teuthology.orchestra.run.smithi017.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T21:14:59.823 INFO:teuthology.orchestra.run.smithi017.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T21:14:59.866 INFO:teuthology.orchestra.run.smithi119.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T21:14:59.866 INFO:teuthology.orchestra.run.smithi017.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T21:14:59.868 INFO:teuthology.orchestra.run.smithi119.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T21:14:59.892 INFO:teuthology.orchestra.run.smithi017.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T21:14:59.894 INFO:teuthology.orchestra.run.smithi017.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T21:14:59.927 INFO:teuthology.orchestra.run.smithi119.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T21:14:59.976 INFO:teuthology.orchestra.run.smithi119.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T21:14:59.994 INFO:teuthology.orchestra.run.smithi119.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T21:14:59.999 INFO:teuthology.orchestra.run.smithi119.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T21:15:03.615 INFO:teuthology.orchestra.run.smithi017.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T21:15:03.925 INFO:teuthology.orchestra.run.smithi017.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T21:15:03.976 INFO:teuthology.orchestra.run.smithi017.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T21:15:04.552 INFO:teuthology.orchestra.run.smithi119.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T21:15:04.956 INFO:teuthology.orchestra.run.smithi119.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T21:15:04.957 INFO:teuthology.orchestra.run.smithi119.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T21:15:06.945 INFO:teuthology.orchestra.run.smithi017.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T21:15:06.947 INFO:teuthology.orchestra.run.smithi017.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T21:15:06.949 INFO:teuthology.orchestra.run.smithi017.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T21:15:07.799 INFO:teuthology.orchestra.run.smithi017.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T21:15:08.391 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 244 MB in 11s (21.5 MB/s) 2024-09-15T21:15:08.512 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T21:15:08.570 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T21:15:08.575 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T21:15:08.665 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:15:08.675 INFO:teuthology.orchestra.run.smithi119.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T21:15:08.678 INFO:teuthology.orchestra.run.smithi119.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T21:15:08.679 INFO:teuthology.orchestra.run.smithi119.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T21:15:09.041 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T21:15:09.068 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T21:15:09.365 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:15:09.398 INFO:teuthology.orchestra.run.smithi119.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T21:15:09.701 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T21:15:09.728 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T21:15:09.768 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:15:09.974 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 244 MB in 13s (18.8 MB/s) 2024-09-15T21:15:10.138 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T21:15:10.171 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T21:15:10.198 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:15:10.238 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T21:15:10.244 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T21:15:10.246 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:10.328 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:15:10.658 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T21:15:10.673 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T21:15:10.684 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:15:10.700 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T21:15:10.724 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:10.740 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:15:11.093 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T21:15:11.120 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T21:15:11.127 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T21:15:11.153 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:15:11.159 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:15:11.202 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:11.547 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T21:15:11.574 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:15:11.605 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T21:15:11.613 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:11.632 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T21:15:11.671 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:15:11.991 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librados2. 2024-09-15T21:15:12.008 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T21:15:12.018 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:12.035 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:15:12.057 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:12.076 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:12.428 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T21:15:12.455 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:15:12.494 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:12.585 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libnbd0. 2024-09-15T21:15:12.612 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T21:15:12.660 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T21:15:12.823 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T21:15:12.838 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T21:15:12.881 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:15:13.015 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librbd1. 2024-09-15T21:15:13.044 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:13.082 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:13.201 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librados2. 2024-09-15T21:15:13.227 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:13.267 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:13.535 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T21:15:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:13.601 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:13.737 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libnbd0. 2024-09-15T21:15:13.763 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T21:15:13.807 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T21:15:13.903 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rados. 2024-09-15T21:15:13.919 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:13.970 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:14.065 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librbd1. 2024-09-15T21:15:14.091 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:14.131 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:14.365 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T21:15:14.391 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:14.439 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:14.576 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T21:15:14.602 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:14.642 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:14.700 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T21:15:14.716 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:14.758 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:15.069 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T21:15:15.096 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:15.136 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:15.170 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rados. 2024-09-15T21:15:15.197 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:15.236 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:15.473 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T21:15:15.499 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T21:15:15.506 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T21:15:15.521 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:15.547 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:15:15.556 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:15.783 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T21:15:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:15.900 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T21:15:15.927 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T21:15:15.983 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T21:15:16.169 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T21:15:16.196 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:16.236 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:16.269 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T21:15:16.285 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:16.327 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:16.573 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T21:15:16.599 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T21:15:16.639 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:15:16.672 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T21:15:16.699 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T21:15:16.738 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:15:16.917 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T21:15:16.932 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T21:15:16.967 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T21:15:17.041 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T21:15:17.068 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T21:15:17.116 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:15:17.194 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T21:15:17.221 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:17.261 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:17.427 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T21:15:17.443 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T21:15:17.485 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:15:17.623 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T21:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T21:15:17.689 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:15:17.805 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua5.1. 2024-09-15T21:15:17.832 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T21:15:17.872 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:15:17.967 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T21:15:18.013 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T21:15:18.025 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:15:18.241 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-any. 2024-09-15T21:15:18.268 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T21:15:18.311 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T21:15:18.316 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T21:15:18.327 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T21:15:18.361 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:15:18.602 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package zip. 2024-09-15T21:15:18.629 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T21:15:18.669 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T21:15:18.698 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua5.1. 2024-09-15T21:15:18.725 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T21:15:18.764 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:15:18.980 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package unzip. 2024-09-15T21:15:19.007 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T21:15:19.055 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:15:19.101 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-any. 2024-09-15T21:15:19.127 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T21:15:19.167 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T21:15:19.403 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package zip. 2024-09-15T21:15:19.429 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T21:15:19.461 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T21:15:19.547 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package luarocks. 2024-09-15T21:15:19.574 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T21:15:19.692 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:15:19.798 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package unzip. 2024-09-15T21:15:19.825 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T21:15:19.865 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:15:20.125 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librgw2. 2024-09-15T21:15:20.152 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:20.199 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package luarocks. 2024-09-15T21:15:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T21:15:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:15:20.702 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librgw2. 2024-09-15T21:15:20.728 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:20.768 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:20.835 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T21:15:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:20.893 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:21.238 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T21:15:21.265 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T21:15:21.304 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:15:21.320 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T21:15:21.335 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:21.370 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:21.641 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T21:15:21.668 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:21.707 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:21.723 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T21:15:21.750 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T21:15:21.790 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:15:22.061 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-common. 2024-09-15T21:15:22.088 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:22.110 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T21:15:22.127 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:22.136 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:22.176 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:22.529 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-common. 2024-09-15T21:15:22.556 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:22.587 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:23.649 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-base. 2024-09-15T21:15:23.675 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:23.725 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:24.160 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-base. 2024-09-15T21:15:24.186 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:24.235 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:24.319 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T21:15:24.346 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T21:15:24.386 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:15:24.655 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T21:15:24.682 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T21:15:24.722 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:15:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T21:15:24.841 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T21:15:24.881 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:15:25.058 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T21:15:25.085 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T21:15:25.125 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:15:25.183 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T21:15:25.210 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T21:15:25.250 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:15:25.444 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T21:15:25.471 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T21:15:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:15:25.753 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T21:15:25.779 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T21:15:25.814 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T21:15:25.819 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:15:25.841 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T21:15:25.880 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:15:26.089 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T21:15:26.116 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T21:15:26.156 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:15:26.158 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T21:15:26.185 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T21:15:26.224 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T21:15:26.459 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T21:15:26.485 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T21:15:26.525 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:15:26.527 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-portend. 2024-09-15T21:15:26.554 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T21:15:26.594 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T21:15:26.828 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T21:15:26.854 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T21:15:26.896 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T21:15:26.896 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T21:15:26.923 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T21:15:26.963 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T21:15:27.181 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-portend. 2024-09-15T21:15:27.207 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T21:15:27.239 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T21:15:27.241 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T21:15:27.268 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T21:15:27.299 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:15:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T21:15:27.560 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T21:15:27.600 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T21:15:27.602 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T21:15:27.629 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T21:15:27.661 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T21:15:27.945 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T21:15:27.964 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T21:15:27.971 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T21:15:27.990 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T21:15:28.011 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:15:28.030 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T21:15:28.358 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T21:15:28.365 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T21:15:28.385 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T21:15:28.391 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T21:15:28.425 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:15:28.432 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T21:15:28.727 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-mako. 2024-09-15T21:15:28.735 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T21:15:28.754 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T21:15:28.761 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T21:15:28.794 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:15:28.801 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T21:15:29.046 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T21:15:29.061 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T21:15:29.096 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:15:29.122 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T21:15:29.138 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T21:15:29.181 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:15:29.407 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-mako. 2024-09-15T21:15:29.434 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T21:15:29.458 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T21:15:29.474 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:15:29.485 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T21:15:29.516 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:15:30.328 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webob. 2024-09-15T21:15:30.355 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T21:15:30.553 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:15:30.569 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T21:15:30.596 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T21:15:30.635 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:15:30.905 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T21:15:30.913 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T21:15:30.932 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T21:15:30.940 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T21:15:30.975 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:15:30.980 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:15:31.250 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webob. 2024-09-15T21:15:31.277 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T21:15:31.308 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T21:15:31.316 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:15:31.335 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T21:15:31.375 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:15:31.652 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T21:15:31.669 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-paste. 2024-09-15T21:15:31.679 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T21:15:31.685 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T21:15:31.719 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:15:31.722 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:15:32.005 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T21:15:32.022 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T21:15:32.031 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T21:15:32.038 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T21:15:32.072 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:15:32.080 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:15:32.317 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-paste. 2024-09-15T21:15:32.343 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T21:15:32.383 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:15:32.383 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T21:15:32.410 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T21:15:32.450 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:15:32.694 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T21:15:32.710 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T21:15:32.753 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:15:32.753 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T21:15:32.780 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T21:15:32.819 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T21:15:32.997 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T21:15:33.013 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T21:15:33.055 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:15:33.089 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T21:15:33.116 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T21:15:33.147 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:15:33.359 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T21:15:33.385 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T21:15:33.425 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T21:15:33.460 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T21:15:33.476 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T21:15:33.510 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T21:15:33.720 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T21:15:33.746 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T21:15:33.786 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:15:33.838 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-idna. 2024-09-15T21:15:33.865 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T21:15:33.904 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:15:34.166 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T21:15:34.193 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T21:15:34.224 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T21:15:34.233 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T21:15:34.250 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T21:15:34.299 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:15:34.544 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-idna. 2024-09-15T21:15:34.571 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T21:15:34.610 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:15:34.635 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests. 2024-09-15T21:15:34.662 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T21:15:34.701 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:15:34.905 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T21:15:34.932 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T21:15:34.971 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:15:35.205 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T21:15:35.231 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T21:15:35.271 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:15:35.283 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-requests. 2024-09-15T21:15:35.313 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T21:15:35.350 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:15:35.557 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T21:15:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:35.661 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T21:15:35.688 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T21:15:35.728 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:15:35.987 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T21:15:36.014 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:36.053 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:36.364 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T21:15:36.372 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T21:15:36.387 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:36.400 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T21:15:36.439 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:15:36.439 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:36.737 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T21:15:36.761 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T21:15:36.794 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T21:15:36.800 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T21:15:36.809 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:36.844 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:37.070 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T21:15:37.097 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:37.122 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T21:15:37.136 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:37.148 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T21:15:37.188 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:15:37.493 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T21:15:37.520 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T21:15:37.559 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T21:15:37.573 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T21:15:37.600 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:37.639 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:37.888 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T21:15:37.914 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:37.946 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:38.251 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T21:15:38.278 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:38.317 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:38.324 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T21:15:38.351 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:38.390 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:38.978 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T21:15:39.004 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:39.044 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:39.354 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph. 2024-09-15T21:15:39.370 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:39.404 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:39.706 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T21:15:39.733 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:39.773 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:40.105 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph. 2024-09-15T21:15:40.121 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:40.151 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T21:15:40.155 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:40.178 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:40.217 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:40.408 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T21:15:40.434 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:40.483 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:40.628 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephadm. 2024-09-15T21:15:40.645 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:40.678 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:40.861 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T21:15:40.887 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:40.927 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:40.973 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T21:15:40.988 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T21:15:41.022 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T21:15:41.322 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package cephadm. 2024-09-15T21:15:41.349 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:41.375 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T21:15:41.389 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:41.402 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:15:41.433 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:41.658 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T21:15:41.684 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T21:15:41.724 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T21:15:42.036 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T21:15:42.051 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:15:42.094 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:42.446 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-babel. 2024-09-15T21:15:42.475 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:15:42.513 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:42.841 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T21:15:42.868 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T21:15:42.908 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:15:43.149 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-babel. 2024-09-15T21:15:43.176 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:15:43.215 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:43.252 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T21:15:43.280 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:43.318 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:43.535 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T21:15:43.562 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T21:15:43.602 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:15:43.680 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T21:15:43.707 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T21:15:43.746 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T21:15:43.946 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T21:15:43.973 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:44.013 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:44.049 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-routes. 2024-09-15T21:15:44.077 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T21:15:44.116 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:15:44.350 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T21:15:44.376 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T21:15:44.416 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T21:15:44.461 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T21:15:44.488 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:44.527 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:44.727 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-routes. 2024-09-15T21:15:44.754 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T21:15:44.794 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:15:45.172 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T21:15:45.199 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:45.238 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:47.440 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T21:15:47.470 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T21:15:47.515 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:15:47.953 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T21:15:47.982 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T21:15:48.010 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T21:15:48.027 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:15:48.038 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T21:15:48.068 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:15:48.438 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T21:15:48.466 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T21:15:48.504 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:15:48.572 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T21:15:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T21:15:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:15:48.815 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T21:15:48.844 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T21:15:48.968 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:15:48.984 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T21:15:49.012 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T21:15:49.042 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:15:49.312 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T21:15:49.339 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T21:15:49.464 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:15:49.627 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T21:15:49.655 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:49.693 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:50.165 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T21:15:50.193 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:50.231 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:50.938 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T21:15:50.966 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T21:15:51.071 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T21:15:51.284 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T21:15:51.312 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T21:15:51.351 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T21:15:51.424 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T21:15:51.451 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T21:15:51.490 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:15:51.662 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T21:15:51.690 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T21:15:51.729 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:15:51.820 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T21:15:51.848 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T21:15:51.887 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:15:52.082 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T21:15:52.110 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T21:15:52.149 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:15:52.198 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T21:15:52.226 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T21:15:52.264 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T21:15:52.469 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T21:15:52.496 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T21:15:52.535 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T21:15:52.585 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T21:15:52.613 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T21:15:52.660 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T21:15:52.814 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T21:15:52.842 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T21:15:52.872 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T21:15:53.005 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T21:15:53.032 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T21:15:53.071 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:15:53.175 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T21:15:53.203 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T21:15:53.242 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:15:53.349 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T21:15:53.377 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T21:15:53.432 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T21:15:53.561 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T21:15:53.589 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T21:15:53.628 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T21:15:53.710 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T21:15:53.735 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T21:15:53.785 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:15:53.939 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T21:15:53.968 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T21:15:54.006 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:15:54.499 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T21:15:54.516 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:54.557 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:54.678 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T21:15:54.695 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:54.737 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:54.868 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T21:15:54.896 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T21:15:54.934 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:15:55.023 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T21:15:55.051 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T21:15:55.089 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:15:55.254 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T21:15:55.271 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:15:55.304 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:15:55.443 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T21:15:55.471 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:15:55.509 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:15:55.540 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package jq. 2024-09-15T21:15:55.568 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:15:55.609 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T21:15:55.804 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package jq. 2024-09-15T21:15:55.820 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:15:55.862 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T21:15:55.882 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package socat. 2024-09-15T21:15:55.910 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T21:15:55.951 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:15:56.089 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package socat. 2024-09-15T21:15:56.105 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T21:15:56.139 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:15:56.304 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T21:15:56.333 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T21:15:56.379 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:15:56.426 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T21:15:56.442 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T21:15:56.476 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:15:56.725 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-test. 2024-09-15T21:15:56.753 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:56.791 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:56.838 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-test. 2024-09-15T21:15:56.855 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:56.896 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.063 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T21:15:59.091 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:59.129 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.260 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T21:15:59.288 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:59.317 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.449 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T21:15:59.478 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:59.524 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.580 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T21:15:59.596 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:59.629 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.885 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T21:15:59.914 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T21:15:59.916 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T21:15:59.944 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T21:15:59.959 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:15:59.982 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:16:00.320 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T21:16:00.339 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T21:16:00.349 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T21:16:00.368 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T21:16:00.387 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:16:00.406 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:16:00.724 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T21:16:00.752 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T21:16:00.790 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:16:00.793 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T21:16:00.822 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T21:16:00.859 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:16:01.221 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package pkg-config. 2024-09-15T21:16:01.250 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T21:16:01.264 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package pkg-config. 2024-09-15T21:16:01.287 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:16:01.293 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T21:16:01.322 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:16:01.534 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T21:16:01.548 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T21:16:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T21:16:01.565 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T21:16:01.592 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:16:01.606 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:16:01.987 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T21:16:02.004 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T21:16:02.044 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T21:16:02.053 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T21:16:02.072 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T21:16:02.110 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T21:16:02.356 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T21:16:02.384 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T21:16:02.413 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T21:16:02.422 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T21:16:02.440 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T21:16:02.479 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T21:16:02.742 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T21:16:02.771 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T21:16:02.808 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T21:16:02.808 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:16:02.836 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T21:16:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:16:03.120 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T21:16:03.149 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T21:16:03.186 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T21:16:03.202 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T21:16:03.231 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T21:16:03.269 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T21:16:03.531 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-py. 2024-09-15T21:16:03.546 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-py. 2024-09-15T21:16:03.560 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T21:16:03.560 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T21:16:03.605 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T21:16:03.605 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T21:16:03.908 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T21:16:03.934 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T21:16:03.937 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T21:16:03.963 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T21:16:03.975 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:16:04.000 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:16:04.421 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T21:16:04.429 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T21:16:04.438 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T21:16:04.458 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T21:16:04.471 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:16:04.495 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:16:04.774 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-toml. 2024-09-15T21:16:04.803 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T21:16:04.815 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-toml. 2024-09-15T21:16:04.843 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T21:16:04.857 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T21:16:04.881 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T21:16:05.168 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T21:16:05.184 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T21:16:05.197 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T21:16:05.213 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T21:16:05.226 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:16:05.251 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:16:05.538 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T21:16:05.555 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T21:16:05.563 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T21:16:05.588 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:16:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T21:16:05.629 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:16:05.807 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T21:16:05.824 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T21:16:05.856 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:16:05.891 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T21:16:05.907 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T21:16:05.948 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:16:06.619 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package radosgw. 2024-09-15T21:16:06.648 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:16:06.653 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package radosgw. 2024-09-15T21:16:06.681 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:16:06.685 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:06.719 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:07.580 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T21:16:07.597 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:16:07.622 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T21:16:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:07.639 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:16:07.680 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:07.931 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T21:16:07.985 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T21:16:08.251 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:16:08.402 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:16:08.410 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:16:08.536 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:16:08.678 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T21:16:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T21:16:09.029 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T21:16:09.148 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T21:16:09.474 INFO:teuthology.orchestra.run.smithi017.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T21:16:09.537 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:09.569 INFO:teuthology.orchestra.run.smithi119.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T21:16:09.638 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:10.087 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user cephadm....done 2024-09-15T21:16:10.164 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user cephadm....done 2024-09-15T21:16:10.276 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:16:10.320 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:16:10.553 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:16:10.596 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:16:10.813 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:16:10.923 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:16:10.938 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:16:11.158 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T21:16:11.206 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T21:16:11.426 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:16:11.490 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:16:11.552 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T21:16:11.592 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T21:16:11.862 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:16:11.893 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:16:12.205 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T21:16:12.270 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T21:16:12.457 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T21:16:12.538 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T21:16:12.724 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:16:12.823 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:16:12.985 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:16:13.082 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:16:13.244 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:16:13.317 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:16:13.524 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:16:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:16:13.763 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:13.920 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:14.005 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:16:14.115 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T21:16:14.195 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:16:14.216 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:16:14.342 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:16:14.346 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T21:16:14.472 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:16:14.476 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:16:14.598 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:16:14.735 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:16:14.740 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:16:15.017 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:16:15.124 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T21:16:15.159 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T21:16:15.214 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:16:15.340 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T21:16:15.362 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T21:16:15.406 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T21:16:15.470 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:16:15.596 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T21:16:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:16:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T21:16:15.972 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:16:16.098 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T21:16:16.110 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:16:16.369 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:16:16.580 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:16:16.671 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T21:16:16.771 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:16:16.983 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:16:17.309 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T21:16:17.415 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:16:17.452 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:16:17.658 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:16:17.784 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:16:17.910 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:16:18.188 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:16:18.211 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T21:16:18.464 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:16:18.590 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:16:18.595 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T21:16:18.724 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:16:18.830 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:16:19.041 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T21:16:19.198 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:16:19.435 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T21:16:19.491 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:16:19.702 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:16:19.834 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:16:20.094 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:16:20.095 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:16:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:20.397 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:16:20.513 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:16:20.798 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:16:20.822 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:16:21.049 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:16:21.200 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:21.518 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:16:21.861 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:16:22.022 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:16:22.282 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:16:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T21:16:22.675 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:16:22.801 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:16:22.936 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:16:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:16:23.220 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:16:23.301 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:16:23.461 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T21:16:23.472 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T21:16:23.714 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:16:23.762 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:16:23.832 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:16:23.888 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:16:24.014 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:16:24.117 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:16:24.315 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:16:24.376 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T21:16:24.611 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T21:16:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T21:16:24.867 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:16:24.887 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:16:24.993 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:16:25.129 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T21:16:25.303 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:16:25.405 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:16:25.506 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T21:16:25.562 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T21:16:25.782 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:16:25.830 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T21:16:25.909 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:16:26.090 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:16:26.210 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:16:26.341 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T21:16:26.503 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:16:26.633 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:16:26.737 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:16:26.759 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T21:16:26.863 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:16:26.972 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:16:27.044 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:16:27.046 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T21:16:27.153 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:16:27.290 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:16:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:16:27.437 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:16:27.747 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:16:27.801 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:16:28.015 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:16:28.095 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T21:16:28.141 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:16:28.220 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T21:16:28.266 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:16:28.357 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T21:16:28.496 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:16:28.606 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:16:28.618 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:16:28.744 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:16:28.915 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:16:29.137 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:16:29.233 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:16:29.397 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T21:16:29.517 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T21:16:29.522 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T21:16:29.643 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T21:16:29.790 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:16:29.911 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T21:16:29.916 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:16:30.234 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:16:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:16:30.499 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:16:30.544 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:16:30.820 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T21:16:30.876 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T21:16:30.945 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T21:16:31.169 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:16:31.221 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T21:16:31.505 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:16:31.574 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:16:31.614 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:31.715 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:31.825 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:16:31.834 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:16:32.170 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:32.211 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T21:16:32.295 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:32.421 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T21:16:32.496 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:16:32.698 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:32.799 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:32.856 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:16:32.917 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:32.965 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:33.066 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:33.132 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:16:33.132 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:16:33.176 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:16:33.537 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:33.634 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:33.663 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:33.769 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:33.789 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T21:16:33.895 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:34.038 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:16:34.065 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:34.191 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:34.317 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:34.540 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:16:34.540 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:16:35.027 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.128 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.183 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.237 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.309 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.363 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:16:35.427 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.537 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:35.637 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:36.163 INFO:teuthology.orchestra.run.smithi119.stdout:Adding group ceph....done 2024-09-15T21:16:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user ceph....done 2024-09-15T21:16:36.625 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:36.650 INFO:teuthology.orchestra.run.smithi119.stdout:Setting system user ceph properties....done 2024-09-15T21:16:36.665 INFO:teuthology.orchestra.run.smithi119.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T21:16:36.743 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:36.811 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T21:16:36.869 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:37.003 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:37.129 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:37.131 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T21:16:37.605 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:37.646 INFO:teuthology.orchestra.run.smithi017.stdout:Adding group ceph....done 2024-09-15T21:16:37.714 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:37.982 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user ceph....done 2024-09-15T21:16:38.119 INFO:teuthology.orchestra.run.smithi017.stdout:Setting system user ceph properties....done 2024-09-15T21:16:38.136 INFO:teuthology.orchestra.run.smithi017.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T21:16:38.163 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:16:38.163 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:16:38.283 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T21:16:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T21:16:38.658 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:39.009 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T21:16:39.082 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:39.183 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:39.498 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:39.624 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:16:39.625 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:16:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:16:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:16:40.068 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:40.228 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:40.426 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T21:16:40.432 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:16:40.432 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:16:40.886 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:40.921 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:41.129 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:16:41.129 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:16:41.190 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:16:41.191 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:16:41.596 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:41.656 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:41.757 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:41.823 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:16:41.823 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:16:41.891 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:42.103 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:16:42.103 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:16:42.323 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:42.609 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:42.640 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:16:42.640 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:16:42.760 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:42.886 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:43.021 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:43.200 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:43.318 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:43.373 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T21:16:43.444 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:43.507 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T21:16:43.641 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:16:43.641 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:16:43.660 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T21:16:44.087 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:44.238 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:44.372 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:44.523 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:44.703 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T21:16:44.893 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T21:16:45.002 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T21:16:45.162 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T21:16:45.354 INFO:teuthology.orchestra.run.smithi119.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T21:16:45.361 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:46.288 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T21:16:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T21:16:46.938 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:46.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:16:47.092 DEBUG:teuthology.orchestra.run.smithi017:> dpkg-query -W -f '${Version}' ceph 2024-09-15T21:16:47.111 INFO:teuthology.orchestra.run.smithi017.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:16:47.111 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:16:47.112 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T21:16:47.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:16:47.253 DEBUG:teuthology.orchestra.run.smithi119:> dpkg-query -W -f '${Version}' ceph 2024-09-15T21:16:47.272 INFO:teuthology.orchestra.run.smithi119.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:16:47.273 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:16:47.273 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T21:16:47.276 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:16:47.276 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:47.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:16:47.291 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:47.291 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:16:47.327 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:16:47.327 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:47.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:16:47.345 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:16:47.401 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:47.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:16:47.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:16:47.469 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:16:47.470 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:47.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:16:47.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:16:47.537 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:47.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:16:47.550 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:16:47.606 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:16:47.606 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:47.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:16:47.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:16:47.677 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:47.677 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:16:47.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:16:47.745 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T21:16:47.830 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T21:16:47.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2024-09-15T21:16:47.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-09-15T21:16:47.845 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T21:16:47.845 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:16:47.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:16:47.898 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T21:16:47.898 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:16:47.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:16:47.964 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T21:16:47.964 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client ping interval': 4, 'mon client ping timeout': 12, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T21:16:47.964 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675', 'branch': 'main', 'description': 'rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7905675', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client ping interval': 4, 'mon client ping timeout': 12, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 2855, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'sleep_before_teardown': 0, 'subset': '17936/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7498493279982dfe87f00616198a5967475f6169', 'targets': {'smithi017.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9Yo+w+W6sb6hFPrFEIaobsqMfFObtb91VA+VNbixtwmP+LS1efET5s65NmZbeAdiCx69YfmpPZoau9vR+zWio=', 'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJsV6O8rkqJqvpx0qPSJo8KWU+nYdBN/h8k05lym81Q1G7t0oEObYBQHPdmP9qlAF6qZEbXOmuctCKOULywGS7o='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'freeze_mon_duration': 20, 'freeze_mon_probability': 10, 'revive_delay': 20, 'thrash_delay': 1, 'thrash_many': True}}, {'workunit': {'clients': {'client.0': ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_20:00:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T21:16:47.964 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:16:47.998 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:16:48.004 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:16:48.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:16:48.060 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:48.060 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:16:48.110 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:16:48.110 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-15 21:11:12.100955102 +0000 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-15 21:11:11.856957283 +0000 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-15 21:11:11.856957283 +0000 2024-09-15T21:16:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-09-15T21:16:48.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:16:48.216 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-15T21:16:48.216 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-15T21:16:48.216 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000310125 s, 1.7 MB/s 2024-09-15T21:16:48.218 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:16:48.267 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-09-15T21:16:48.313 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-15 21:11:12.608950562 +0000 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-15 21:11:12.340952957 +0000 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-15 21:11:12.340952957 +0000 2024-09-15T21:16:48.314 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-09-15T21:16:48.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:16:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-15T21:16:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-15T21:16:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000278236 s, 1.8 MB/s 2024-09-15T21:16:48.369 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:16:48.419 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-09-15T21:16:48.465 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-15 21:11:13.140945806 +0000 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-15 21:11:12.868948238 +0000 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-15 21:11:12.868948238 +0000 2024-09-15T21:16:48.466 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-09-15T21:16:48.466 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:16:48.520 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-15T21:16:48.520 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-15T21:16:48.520 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000282535 s, 1.8 MB/s 2024-09-15T21:16:48.521 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:16:48.571 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-15 21:11:13.644941302 +0000 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-15 21:11:13.380943662 +0000 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-15 21:11:13.380943662 +0000 2024-09-15T21:16:48.618 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-09-15T21:16:48.618 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:16:48.673 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-15T21:16:48.673 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-15T21:16:48.673 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000313024 s, 1.6 MB/s 2024-09-15T21:16:48.674 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:16:48.723 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:48.723 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:48.723 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:16:48.729 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:16:48.729 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-15 21:11:07.718692994 +0000 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-15 21:11:07.478695137 +0000 2024-09-15T21:16:48.775 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-15 21:11:07.478695137 +0000 2024-09-15T21:16:48.776 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-15T21:16:48.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:16:48.829 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-15T21:16:48.829 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-15T21:16:48.829 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000441279 s, 1.2 MB/s 2024-09-15T21:16:48.830 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:16:48.880 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-09-15T21:16:48.926 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:16:48.926 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:48.926 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-09-15T21:16:48.927 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:48.927 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-15 21:11:08.214688564 +0000 2024-09-15T21:16:48.927 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-15 21:11:07.962690814 +0000 2024-09-15T21:16:48.927 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-15 21:11:07.962690814 +0000 2024-09-15T21:16:48.927 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-15T21:16:48.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:16:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-15T21:16:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-15T21:16:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000415707 s, 1.2 MB/s 2024-09-15T21:16:48.982 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:16:49.031 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-09-15T21:16:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-15 21:11:08.718684061 +0000 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-15 21:11:08.474686241 +0000 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-15 21:11:08.474686241 +0000 2024-09-15T21:16:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-15T21:16:49.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:16:49.133 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-15T21:16:49.133 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-15T21:16:49.133 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000307648 s, 1.7 MB/s 2024-09-15T21:16:49.134 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:16:49.183 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-09-15T21:16:49.231 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:16:49.231 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:49.232 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-09-15T21:16:49.232 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:49.232 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-15 21:11:09.194679810 +0000 2024-09-15T21:16:49.232 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-15 21:11:08.946682025 +0000 2024-09-15T21:16:49.232 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-15 21:11:08.946682025 +0000 2024-09-15T21:16:49.232 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-09-15T21:16:49.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:16:49.285 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-15T21:16:49.285 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-15T21:16:49.285 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00029498 s, 1.7 MB/s 2024-09-15T21:16:49.287 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:16:49.335 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:49.336 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T21:16:49.336 INFO:tasks.ceph:Generating config... 2024-09-15T21:16:49.337 INFO:tasks.ceph:[client] debug monc = 20 2024-09-15T21:16:49.337 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T21:16:49.337 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T21:16:49.337 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-15T21:16:49.338 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T21:16:49.338 INFO:tasks.ceph:[global] ms type = async 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-15T21:16:49.338 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-15T21:16:49.339 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-15T21:16:49.339 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-09-15T21:16:49.339 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-09-15T21:16:49.339 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T21:16:49.339 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] bluestore write v2 = True 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T21:16:49.339 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] mon client ping interval = 4 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] mon client ping timeout = 12 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T21:16:49.340 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T21:16:49.341 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T21:16:49.341 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T21:16:49.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T21:16:49.376 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T21:16:49.393 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T21:16:49.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:16:49.530 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17'), ('mon.c', '[v2:172.21.15.17:3301,v1:172.21.15.17:6790]'), ('mon.b', '172.21.15.119')] 2024-09-15T21:16:49.530 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.119', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client ping interval': 4, 'mon client ping timeout': 12, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-15T21:16:49.531 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:49.531 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:16:49.578 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.17 --addv c '[v2:172.21.15.17:3301,v1:172.21.15.17:6790]' --add b 172.21.15.119 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:49.666 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:49.666 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:16:49.666 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-09-15T21:16:49.666 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2024-09-15T21:16:49.666 INFO:teuthology.orchestra.run.smithi017.stdout:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 17 (quincy) 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:election_strategy: 1 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:16:49.667 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T21:16:49.668 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:16:49.674 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bbfd2e88-7522-4bd5-ae16-c665761d9528... 2024-09-15T21:16:49.674 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:16:49.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:16:49.744 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2024-09-15T21:16:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2024-09-15T21:16:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon_allow_pool_size_one = true 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:16:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.119" 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: mon client directed command retry = 5 2024-09-15T21:16:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: mon election default strategy = 1 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: ms inject socket failures = 5000 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: ms type = async 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug trim objects = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2024-09-15T21:16:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression algorithm = zstd 2024-09-15T21:16:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore write v2 = True 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore zero block detection = True 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: mon client ping interval = 4 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: mon client ping timeout = 12 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2024-09-15T21:16:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify cached snaps = True 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify missing on start = True 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd mclock override recovery settings = True 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd op queue = debug_random 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: osd op queue cut off = debug_random 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-09-15T21:16:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:16:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: # 1m isn't quite enough 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:16:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon min osdmap epochs = 25 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon osdmap full prune interval = 2 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon osdmap full prune min = 15 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon osdmap full prune txsize = 2 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon scrub inject crc mismatch = 0.01 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon scrub inject missing keys = 0.05 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: mon scrub interval = 300 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: paxos service trim min = 5 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2024-09-15T21:16:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: debug monc = 20 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2024-09-15T21:16:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.758 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.759 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.760 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.119" 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: mon client directed command retry = 5 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: mon election default strategy = 1 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: ms inject socket failures = 5000 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: ms type = async 2024-09-15T21:16:49.761 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-09-15T21:16:49.762 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:16:49.763 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore compression algorithm = zstd 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore compression mode = aggressive 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore write v2 = True 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore zero block detection = True 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-15T21:16:49.764 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: mon client ping interval = 4 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: mon client ping timeout = 12 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify cached snaps = True 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify missing on start = True 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock override recovery settings = True 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd op queue = debug_random 2024-09-15T21:16:49.765 INFO:teuthology.orchestra.run.smithi119.stdout: osd op queue cut off = debug_random 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-09-15T21:16:49.766 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.767 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon min osdmap epochs = 25 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon osdmap full prune interval = 2 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon osdmap full prune min = 15 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon osdmap full prune txsize = 2 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon scrub inject crc mismatch = 0.01 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon scrub inject missing keys = 0.05 2024-09-15T21:16:49.768 INFO:teuthology.orchestra.run.smithi119.stdout: mon scrub interval = 300 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: paxos service trim min = 5 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: debug monc = 20 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-09-15T21:16:49.769 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-09-15T21:16:49.771 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T21:16:49.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T21:16:49.864 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T21:16:49.864 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:49.864 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T21:16:49.870 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:49.870 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T21:16:49.917 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:16:49.917 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:49.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:16:49.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:16:49.983 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:16:49.984 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:50.030 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:16:50.030 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:50.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:16:50.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:16:50.055 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:16:50.055 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:50.102 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T21:16:50.102 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T21:16:50.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:16:50.187 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:16:50.202 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T21:16:50.202 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T21:16:50.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T21:16:50.280 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T21:16:50.291 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T21:16:50.291 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T21:16:50.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:50.305 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:50.306 INFO:tasks.ceph:role: osd.0 2024-09-15T21:16:50.306 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:16:50.306 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:16:50.361 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:16:50.361 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:16:50.361 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:16:50.361 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:16:50.361 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:16:50.361 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:16:50.362 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:16:50.362 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:16:50.362 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:16:50.362 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:16:50.385 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-15T21:16:50.387 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-15T21:16:50.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:50.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:50.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:16:50.501 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:16:50.501 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:50.557 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:50.558 INFO:tasks.ceph:role: osd.1 2024-09-15T21:16:50.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:16:50.558 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:16:50.613 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:16:50.613 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:16:50.613 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:16:50.613 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:16:50.614 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:16:50.614 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:16:50.614 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:16:50.614 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:16:50.614 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:16:50.614 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:16:50.637 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-15T21:16:50.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-15T21:16:50.639 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:50.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:50.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:16:50.751 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:16:50.751 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:50.805 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:50.805 INFO:tasks.ceph:role: osd.2 2024-09-15T21:16:50.805 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:16:50.805 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:16:50.864 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:16:50.864 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:16:50.865 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:16:50.888 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-15T21:16:50.890 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-15T21:16:50.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:50.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:50.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:16:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:16:50.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:51.102+0000 7f5a316197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T21:16:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:51.102+0000 7f5a316197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T21:16:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:51.102+0000 7f5a316197c0 -1 bdev(0x5621c96cf800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:51.102+0000 7f5a316197c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T21:16:54.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:54.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:54.374+0000 7fde494ef7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T21:16:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:54.374+0000 7fde494ef7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T21:16:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:54.374+0000 7fde494ef7c0 -1 bdev(0x55fc3b5a3800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:54.374+0000 7fde494ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T21:16:57.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:57.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:57.642+0000 7fa630f0b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T21:16:57.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:57.642+0000 7fa630f0b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T21:16:57.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:57.642+0000 7fa630f0b7c0 -1 bdev(0x55ad12e8f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:57.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:16:57.642+0000 7fa630f0b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T21:17:00.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T21:17:00.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T21:17:00.870 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T21:17:00.870 INFO:tasks.ceph:role: osd.3 2024-09-15T21:17:00.870 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:17:00.870 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:17:00.935 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:17:01.390 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-15T21:17:01.392 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-15T21:17:01.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-15T21:17:01.528 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T21:17:01.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:17:01.537 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:17:01.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T21:17:01.595 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T21:17:01.595 INFO:tasks.ceph:role: osd.4 2024-09-15T21:17:01.595 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:17:01.595 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:17:01.657 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:17:02.086 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-15T21:17:02.088 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-15T21:17:02.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-15T21:17:02.152 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T21:17:02.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:17:02.207 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:17:02.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T21:17:02.263 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T21:17:02.263 INFO:tasks.ceph:role: osd.5 2024-09-15T21:17:02.263 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:17:02.264 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:17:02.326 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:17:02.326 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:17:02.326 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:17:02.326 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:17:02.327 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:17:02.327 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:17:02.327 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:17:02.327 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:17:02.327 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:17:02.327 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:17:02.801 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-15T21:17:02.803 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-15T21:17:02.803 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-15T21:17:02.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T21:17:02.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:17:02.923 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:17:02.923 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:17:03.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:03.036+0000 7fdd565057c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T21:17:03.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:03.040+0000 7fdd565057c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T21:17:03.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:03.040+0000 7fdd565057c0 -1 bdev(0x55bc16a41800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T21:17:03.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:03.040+0000 7fdd565057c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T21:17:06.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T21:17:06.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:17:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:06.308+0000 7fdc507007c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T21:17:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:06.308+0000 7fdc507007c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T21:17:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:06.308+0000 7fdc507007c0 -1 bdev(0x5624d62cb800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T21:17:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:06.308+0000 7fdc507007c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T21:17:09.453 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T21:17:09.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:17:09.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:09.584+0000 7fad59f3f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T21:17:09.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:09.588+0000 7fad59f3f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T21:17:09.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:09.588+0000 7fad59f3f7c0 -1 bdev(0x55b565953800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T21:17:09.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:09.588+0000 7fad59f3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T21:17:12.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T21:17:12.811 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T21:17:12.811 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:17:12.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T21:17:12.826 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:17:12.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T21:17:12.885 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:17:12.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T21:17:12.941 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:12.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T21:17:12.954 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:12.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T21:17:13.009 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:13.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T21:17:13.062 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:13.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T21:17:13.118 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:13.118 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T21:17:13.167 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T21:17:13.167 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:17:13.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:17:13.212 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2024-09-15T21:17:13.212 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDDTudmKDx4BhAAJ3EyxG1xGWaLDpUY26rhlQ== 2024-09-15T21:17:13.212 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2024-09-15T21:17:13.212 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDGTudmzJ6fFhAA/amycbRZtE4i2k3vQCRlng== 2024-09-15T21:17:13.212 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2024-09-15T21:17:13.212 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDJTudmfXKnJhAAPE4HSOH+HyH0CjHD+5CTuQ== 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDCTudm0okcCxAAPCD7AReeY0eaFL0fRDXz8g== 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDPTudmPsRHAhAA+oBJ5XdGjd2DGUflOkhY4w== 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.4] 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDSTudmGLyBEhAA4bghx9Yo0aAgaZ5i3sm8kg== 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.5] 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDVTudmhtkQIxAAWur9Z1epxgrcZ5OFPEp5Ww== 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-09-15T21:17:13.213 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDCTudmPHWdEBAAPtodrPbJmOVSlvLB8c5fFg== 2024-09-15T21:17:13.220 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-09-15T21:17:13.220 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDDTudmKDx4BhAAJ3EyxG1xGWaLDpUY26rhlQ== 2024-09-15T21:17:13.220 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDGTudmzJ6fFhAA/amycbRZtE4i2k3vQCRlng== 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDJTudmfXKnJhAAPE4HSOH+HyH0CjHD+5CTuQ== 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDCTudm0okcCxAAPCD7AReeY0eaFL0fRDXz8g== 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDPTudmPsRHAhAA+oBJ5XdGjd2DGUflOkhY4w== 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDSTudmGLyBEhAA4bghx9Yo0aAgaZ5i3sm8kg== 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDVTudmhtkQIxAAWur9Z1epxgrcZ5OFPEp5Ww== 2024-09-15T21:17:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-09-15T21:17:13.222 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDCTudmPHWdEBAAPtodrPbJmOVSlvLB8c5fFg== 2024-09-15T21:17:13.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:17:13.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:17:13.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.742 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:17:13.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:17:13.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:17:13.878 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T21:17:13.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T21:17:13.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:17:13.981 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T21:17:13.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T21:17:14.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:17:14.130 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T21:17:14.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T21:17:14.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:17:14.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T21:17:14.260 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:17:14.262 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:17:14.308 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T21:17:14.308 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:17:14.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:17:14.311 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:17:14.311 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:17:14.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:17:14.313 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:17:14.313 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:17:14.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:17:14.351 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:17:14.351 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T21:17:14.352 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:17:14.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:17:14.354 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:17:14.354 DEBUG:tasks.ceph:set 0 configs 2024-09-15T21:17:14.354 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T21:17:14.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T21:17:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient: get_monmap_and_config 2024-09-15T21:17:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient: build_initial_monmap 2024-09-15T21:17:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient: monmap: 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:14.472447+0000 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:14.472447+0000 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:14.476 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 1 Processor -- start 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 1 -- start start 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient: init 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient(hunting): picked mon.noname-c con 0x7f61840a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f61840a53b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f61840a5800 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61840a2c30 con 0x7f61840a53b0 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6184138670 con 0x7f61840a5800 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f61840a0f00 con 0x7f61840a4a90 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618bc6f640 10 monclient(hunting): authenticate will time out at 957.867698s 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618920a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f61840a53b0 0x7f6184137850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51208/0 (socket says 172.21.15.17:51208) 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.466+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 learned_addr learned my addr 172.21.15.17:0/101506139 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f61840a4a90 auth_method 0 2024-09-15T21:17:14.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:14.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:14.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:14.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:14.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:14.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:14.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f61840a4a90 auth_method 0 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:14.674+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:17:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f61840a4a90 auth_method 0 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.078+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:17:15.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f61840a4a90 auth_method 0 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:15.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:15.882+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:17:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): tick 2024-09-15T21:17:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:17:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:17:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:17:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 msgr2=0x7f61840a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.119:3300/0 conn(0x7f61840a4a90 0x7f61840a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 >> v1:172.21.15.119:6789/0 conn(0x7f61840a5800 legacy=0x7f6184137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 >> v1:172.21.15.17:6789/0 conn(0x7f61840a53b0 legacy=0x7f6184137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient: _add_conns ranks=[3,2] 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f6168003720 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f61680061c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f61840a0f00 con 0x7f6168003720 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6168008db0 con 0x7f61680061c0 2024-09-15T21:17:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f617b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f6168003720 auth_method 0 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:17.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:17.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:17.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:17.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:17.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:17.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.470+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f6168003720 auth_method 0 2024-09-15T21:17:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:17:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:17.674+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:17:18.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:18.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:18.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:18.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f6168003720 auth_method 0 2024-09-15T21:17:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:17:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.078+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:17:18.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:18.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f6168003720 auth_method 0 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:18.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:18.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:18.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:17:18.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:18.882+0000 7f618920a640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:17:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): tick 2024-09-15T21:17:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:17:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:17:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:17:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 msgr2=0x7f6168005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f6168003720 0x7f6168005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 1 -- 172.21.15.17:0/101506139 reap_dead start 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f61680061c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6168003b30 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f616800a8d0 0x7f616800ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f616800a8d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61840a19b0 con 0x7f6168003b30 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f61840a0f00 con 0x7f616800a8d0 2024-09-15T21:17:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 1 -- 172.21.15.17:0/101506139 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6168008db0 con 0x7f61680061c0 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f617b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f616800a8d0 0x7f616800ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): get_auth_request con 0x7f616800a8d0 auth_method 0 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6188a09640 1 -- 172.21.15.17:0/101506139 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910703322 0 0) 0x7f61840a19b0 con 0x7f6168003b30 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6188a09640 10 monclient(hunting): _negotiate 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6188a09640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6188a09640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6188a09640 10 monclient(hunting): my global_id is 4108 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6188a09640 1 -- 172.21.15.17:0/101506139 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6170003530 con 0x7f6168003b30 2024-09-15T21:17:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient(hunting): handle_auth_done global_id 4103 payload 274 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f616800a8d0 msgr2=0x7f616800ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f616800a8d0 0x7f616800ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 >> v1:172.21.15.119:6789/0 conn(0x7f6168003b30 legacy=0x7f61680091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 1 monclient: found mon.noname-b 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f6189a0b640 1 -- 172.21.15.17:0/101506139 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616800e4c0 con 0x7f61680061c0 2024-09-15T21:17:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.470+0000 7f618a20c640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f616800a8d0 0x7f616800ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6189a0b640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6180000ee0 tx=0x7f618001dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba3a17f98f2418a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 1 -- 172.21.15.17:0/101506139 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61800b3070 con 0x7f61680061c0 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: dump: 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: _finish_auth 0 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: _check_auth_tickets 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f618bc6f640 5 monclient: authenticate success, global_id 4103 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 1 -- 172.21.15.17:0/101506139 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61800a6b50 con 0x7f61680061c0 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f618bc6f640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f618bc6f640 4 set_mon_vals no callback set 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 1 -- 172.21.15.17:0/101506139 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618000f040 con 0x7f61680061c0 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.474+0000 7f6188a09640 10 monclient: dump: 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:20.488 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 10 monclient: shutdown 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 -- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f6168008660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6180000ee0 tx=0x7f618001dd50 comp rx=0 tx=0).stop 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 -- 172.21.15.17:0/101506139 shutdown_connections 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 --2- 172.21.15.17:0/101506139 >> v2:172.21.15.17:3300/0 conn(0x7f616800a8d0 0x7f616800ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 --2- 172.21.15.17:0/101506139 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f6168008660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 -- 172.21.15.17:0/101506139 >> 172.21.15.17:0/101506139 conn(0x7f61840aabb0 msgr2=0x7f61840a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 -- 172.21.15.17:0/101506139 shutdown_connections 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 -- 172.21.15.17:0/101506139 wait complete. 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 10 monclient: build_initial_monmap 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 10 monclient: monmap: 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:20.490 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.478+0000 7f618bc6f640 1 Processor -- start 2024-09-15T21:17:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 -- start start 2024-09-15T21:17:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient: init 2024-09-15T21:17:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f61841cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): picked mon.noname-b con 0x7f6168003b30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f61841d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): picked mon.noname-c con 0x7f61680061c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): picked mon.noname-a con 0x7f616800a8d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f618406f2d0 con 0x7f616800a8d0 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f618406f150 con 0x7f6168003b30 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6184138060 con 0x7f61680061c0 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618bc6f640 10 monclient(hunting): authenticate will time out at 963.881195s 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f61841d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48206/0 (socket says 172.21.15.17:48206) 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 learned_addr learned my addr 172.21.15.17:0/1551854175 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): get_auth_request con 0x7f616800a8d0 auth_method 0 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f61841cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f6168003b30 auth_method 0 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f6189a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient(hunting): handle_auth_done global_id 4107 payload 824 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f61841d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f61841d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 msgr2=0x7f61841cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f61841cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient: _finish_hunting 0 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 monclient: found mon.noname-a 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61840a1280 con 0x7f616800a8d0 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618920a640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f61841d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 msgr2=0x7f61841251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:17:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 msgr2=0x7f61841251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:17:20.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f61840a2420 tx=0x7f617c00ff70 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:20.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.482+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient: get_auth_request con 0x7f616800a8d0 auth_method 0 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 0 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: ms_handle_reset current mon [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 msgr2=0x7f61841251c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f616800a8d0 0x7f61841251c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f6168008640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): picked mon.noname-b con 0x7f6168003b30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f616800ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): picked mon.noname-c con 0x7f61680061c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6168005670 0x7f6168002b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): picked mon.noname-a con 0x7f6168005670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6168005b80 con 0x7f6168005670 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6168005d00 con 0x7f6168003b30 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f616800d3c0 con 0x7f61680061c0 2024-09-15T21:17:20.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6168005670 0x7f6168002b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f616800ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): get_auth_request con 0x7f6168005670 auth_method 0 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): get_auth_request con 0x7f61680061c0 auth_method 0 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f6168008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 10 monclient(hunting): get_auth_request con 0x7f6168003b30 auth_method 0 2024-09-15T21:17:20.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 msgr2=0x7f6168008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 msgr2=0x7f6168008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f6168008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618920a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f6189a0b640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f6168008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient(hunting): handle_auth_done global_id 4107 payload 293 2024-09-15T21:17:20.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 msgr2=0x7f616800ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f616800ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 msgr2=0x7f6168008640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f6168008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient: _finish_hunting 0 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 monclient: found mon.noname-a 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616800d680 con 0x7f6168005670 2024-09-15T21:17:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618a20c640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6168005670 0x7f6168002b80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61840a2420 tx=0x7f617c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:20.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00d760 con 0x7f6168005670 2024-09-15T21:17:20.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: dump: 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:20.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c005d60 con 0x7f6168005670 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00dbc0 con 0x7f6168005670 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 10 monclient: dump: 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617bfff640 4 set_mon_vals no callback set 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 5 monclient: authenticate success, global_id 4107 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 monclient: _renew_subs 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6184126c80 con 0x7f6168005670 2024-09-15T21:17:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 monclient: _renew_subs 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841277e0 con 0x7f6168005670 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f617c007460 con 0x7f6168005670 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f617c020610 con 0x7f6168005670 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.686+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6184137080 con 0x7f6168005670 2024-09-15T21:17:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.706+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6184137080 con 0x7f6168005670 2024-09-15T21:17:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.706+0000 7f617a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.706+0000 7f617a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.790+0000 7f618bc6f640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T21:17:20.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.790+0000 7f618bc6f640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T21:17:20.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.790+0000 7f618bc6f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:20.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.790+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f61841383f0 con 0x7f6168005670 2024-09-15T21:17:20.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f617a7fc640 1 -- 172.21.15.17:0/1551854175 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f617c02a020 con 0x7f6168005670 2024-09-15T21:17:20.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f617a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T21:17:20.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f617a7fc640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-09-15T21:17:20.804 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 10 monclient: shutdown 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6168005670 msgr2=0x7f6168002b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6168005670 0x7f6168002b80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61840a2420 tx=0x7f617c009040 comp rx=0 tx=0).stop 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 shutdown_connections 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6168005670 0x7f6168002b80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61680061c0 0x7f616800ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 --2- 172.21.15.17:0/1551854175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6168003b30 0x7f6168008640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 >> 172.21.15.17:0/1551854175 conn(0x7f61840aabb0 msgr2=0x7f618406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 shutdown_connections 2024-09-15T21:17:20.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:20.794+0000 7f618bc6f640 1 -- 172.21.15.17:0/1551854175 wait complete. 2024-09-15T21:17:20.823 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T21:17:20.824 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T21:17:20.824 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:17:20.824 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T21:17:20.836 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:17:20.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T21:17:20.892 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-15T21:17:20.892 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T21:17:20.946 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:20.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T21:17:20.958 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:20.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T21:17:21.014 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:17:21.014 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T21:17:21.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new ccf68244-48f9-4748-b69c-942fb9065051 0 2024-09-15T21:17:21.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient: get_monmap_and_config 2024-09-15T21:17:21.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient: monmap: 2024-09-15T21:17:21.227 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.227 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:17:21.226430+0000 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:17:21.226430+0000 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:21.228 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 Processor -- start 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 -- start start 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient: init 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff9a40a27b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): picked mon.noname-b con 0x7ff9a409c340 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff9a4074460 0x7ff9a413bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): picked mon.noname-c con 0x7ff9a4074460 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a409c7c0 con 0x7ff9a40a27b0 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a409deb0 con 0x7ff9a4074460 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a406efe0 con 0x7ff9a409c340 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9aa480640 10 monclient(hunting): authenticate will time out at 960.023223s 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a27b0 0x7ff9a40a2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52754/0 (socket says 172.21.15.119:52754) 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff9a4074460 0x7ff9a413bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3669422409 learned_addr learned my addr 172.21.15.119:0/3669422409 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): get_auth_request con 0x7ff9a4074460 auth_method 0 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a37fe640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient(hunting): handle_auth_done global_id 4114 payload 274 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 1 -- 172.21.15.119:0/3669422409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 msgr2=0x7ff9a4139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 1 -- 172.21.15.119:0/3669422409 >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a27b0 legacy=0x7ff9a40a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 1 monclient: found mon.noname-c 2024-09-15T21:17:21.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a2ffd640 1 -- 172.21.15.119:0/3669422409 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322727377 0 0) 0x7ff9a409c7c0 con 0x7ff9a40a27b0 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a8a1d640 1 -- 172.21.15.119:0/3669422409 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a409ecf0 con 0x7ff9a4074460 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a37fe640 10 monclient: get_auth_request con 0x7ff9a409c340 auth_method 0 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a2ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.224+0000 7ff9a37fe640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a8a1d640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a413bf70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff994009d90 tx=0x7ff99401c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58aa6df9578bd82b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 1 -- 172.21.15.119:0/3669422409 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a8660 con 0x7ff9a4074460 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: dump: 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.242 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 1 -- 172.21.15.119:0/3669422409 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9940a8800 con 0x7ff9a4074460 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 5 monclient: authenticate success, global_id 4114 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 1 -- 172.21.15.119:0/3669422409 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a8bd0 con 0x7ff9a4074460 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 4 set_mon_vals no callback set 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9a2ffd640 10 monclient: dump: 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.244 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: shutdown 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- 172.21.15.119:0/3669422409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 msgr2=0x7ff9a413bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a413bf70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff994009d90 tx=0x7ff99401c830 comp rx=0 tx=0).stop 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- 172.21.15.119:0/3669422409 shutdown_connections 2024-09-15T21:17:21.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a413bf70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3669422409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- 172.21.15.119:0/3669422409 >> 172.21.15.119:0/3669422409 conn(0x7ff9a406dcb0 msgr2=0x7ff9a406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- 172.21.15.119:0/3669422409 shutdown_connections 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- 172.21.15.119:0/3669422409 wait complete. 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: monmap: 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:21.246 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 0.000000 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:created 0.000000 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 Processor -- start 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- start start 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: init 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:21.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): picked mon.noname-b con 0x7ff9a4074460 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4134430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): picked mon.noname-c con 0x7ff9a409c340 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff9a4139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): picked mon.noname-a con 0x7ff9a40a27b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a406d740 con 0x7ff9a40a27b0 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a406d5c0 con 0x7ff9a4074460 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a406d8c0 con 0x7ff9a409c340 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.228+0000 7ff9aa480640 10 monclient(hunting): authenticate will time out at 960.027990s 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46532/0 (socket says 172.21.15.119:46532) 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 learned_addr learned my addr 172.21.15.119:0/3290878761 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): get_auth_request con 0x7ff9a4074460 auth_method 0 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_done global_id 4120 payload 824 2024-09-15T21:17:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a8a1d640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff9a4139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 msgr2=0x7ff9a4134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4134430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 msgr2=0x7ff9a4139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff9a4139890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 monclient: found mon.noname-b 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a411e920 con 0x7ff9a4074460 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 10 monclient: get_auth_request con 0x7ff9a409c340 auth_method 0 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff9a4134430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c490 tx=0x7ff99c00c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c018070 con 0x7ff9a4074460 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: dump: 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c007500 con 0x7ff9a4074460 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c004a30 con 0x7ff9a4074460 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 5 monclient: authenticate success, global_id 4120 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 monclient: _renew_subs 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a411ec00 con 0x7ff9a4074460 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a27fc640 4 set_mon_vals no callback set 2024-09-15T21:17:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: dump: 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 monclient: _renew_subs 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a40a4f50 con 0x7ff9a4074460 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff99c01b070 con 0x7ff9a4074460 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff99c012740 con 0x7ff9a4074460 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a409c7c0 con 0x7ff9a4074460 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 msgr2=0x7ff9a4133ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 msgr2=0x7ff9a4133ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c490 tx=0x7ff99c00c950 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c490 tx=0x7ff99c00c950 comp rx=0 tx=0).stop 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 msgr2=0x7ff9a4133ef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4074460 0x7ff9a4133ef0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff984005280 0x7ff984007670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): picked mon.b con 0x7ff984005280 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff98400a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): picked mon.a con 0x7ff9a40a27b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff98400cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): picked mon.c con 0x7ff9a409c340 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff98400d5d0 con 0x7ff9a40a27b0 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff98400d750 con 0x7ff984005280 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff98400d920 con 0x7ff9a409c340 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a0ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff984005280 0x7ff984007670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): get_auth_request con 0x7ff984005280 auth_method 0 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:17:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff98400a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 10 monclient(hunting): get_auth_request con 0x7ff9a40a27b0 auth_method 0 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a8a1d640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff98400cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a8a1d640 10 monclient(hunting): get_auth_request con 0x7ff9a409c340 auth_method 0 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a8a1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a8a1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a8a1d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient(hunting): handle_auth_done global_id 4120 payload 293 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 msgr2=0x7ff98400cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff98400cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 msgr2=0x7ff98400a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff98400a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 monclient: found mon.b 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7ff98400db20 con 0x7ff984005280 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a409c7c0 con 0x7ff984005280 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.232+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff984005280 0x7ff984007670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff99c012b30 tx=0x7ff99c007db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c02d080 con 0x7ff984005280 2024-09-15T21:17:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 10 monclient: dump: 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c012d60 con 0x7ff984005280 2024-09-15T21:17:21.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff99c029470 con 0x7ff984005280 2024-09-15T21:17:21.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a1ffb640 4 set_mon_vals no callback set 2024-09-15T21:17:21.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff99c028600 con 0x7ff984005280 2024-09-15T21:17:21.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.240+0000 7ff9a0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:21.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.312+0000 7ff9aa480640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ccf68244-48f9-4748-b69c-942fb9065051", "id": 0}] 2024-09-15T21:17:21.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.312+0000 7ff9aa480640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ccf68244-48f9-4748-b69c-942fb9065051", "id": 0}] 2024-09-15T21:17:21.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.312+0000 7ff9aa480640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.312+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ccf68244-48f9-4748-b69c-942fb9065051", "id": 0} v 0) -- 0x7ff9a4076450 con 0x7ff984005280 2024-09-15T21:17:21.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.316+0000 7ff9a0ff9640 1 -- 172.21.15.119:0/3290878761 <== mon.1 v2:172.21.15.119:3300/0 5 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ccf68244-48f9-4748-b69c-942fb9065051", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff99c0287e0 con 0x7ff984005280 2024-09-15T21:17:21.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.316+0000 7ff9a0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ccf68244-48f9-4748-b69c-942fb9065051", "id": 0}] 2024-09-15T21:17:21.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.316+0000 7ff9a0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:21.321 INFO:teuthology.orchestra.run.smithi119.stdout:0 2024-09-15T21:17:21.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 10 monclient: shutdown 2024-09-15T21:17:21.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff984005280 msgr2=0x7ff984007670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff984005280 0x7ff984007670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff99c012b30 tx=0x7ff99c007db0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 shutdown_connections 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a409c340 0x7ff98400cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a40a27b0 0x7ff98400a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 --2- 172.21.15.119:0/3290878761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff984005280 0x7ff984007670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 >> 172.21.15.119:0/3290878761 conn(0x7ff9a406dcb0 msgr2=0x7ff9a409fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 shutdown_connections 2024-09-15T21:17:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.320+0000 7ff9aa480640 1 -- 172.21.15.119:0/3290878761 wait complete. 2024-09-15T21:17:21.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 7cf58353-b02d-4f6d-9170-d06191c75695 1 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 10 monclient: get_monmap_and_config 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 10 monclient: monmap: 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:17:21.447310+0000 2024-09-15T21:17:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:17:21.447310+0000 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:21.449 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 1 Processor -- start 2024-09-15T21:17:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 1 -- start start 2024-09-15T21:17:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 10 monclient: init 2024-09-15T21:17:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.444+0000 7f3a93bdf640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a8c09cad0 0x7f3a8c0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): picked mon.noname-c con 0x7f3a8c09cad0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3a8c09ceb0 0x7f3a8c09d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): picked mon.noname-a con 0x7f3a8c09ceb0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c1345d0 0x7f3a8c134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): picked mon.noname-b con 0x7f3a8c1345d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8c09bd80 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8c09bff0 con 0x7f3a8c09cad0 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c09c2d0 con 0x7f3a8c1345d0 2024-09-15T21:17:21.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient(hunting): authenticate will time out at 960.244137s 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a8c09cad0 0x7f3a8c0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a8c09cad0 0x7f3a8c0a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46558/0 (socket says 172.21.15.119:46558) 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 -- 172.21.15.119:0/80163377 learned_addr learned my addr 172.21.15.119:0/80163377 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): get_auth_request con 0x7f3a8c09cad0 auth_method 0 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9217c640 1 --2- 172.21.15.119:0/80163377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c1345d0 0x7f3a8c134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9117a640 1 --2- 172.21.15.119:0/80163377 >> v2:172.21.15.17:3300/0 conn(0x7f3a8c09ceb0 0x7f3a8c09d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9217c640 10 monclient(hunting): get_auth_request con 0x7f3a8c1345d0 auth_method 0 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9217c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9217c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9217c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient(hunting): handle_auth_done global_id 4123 payload 274 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 -- 172.21.15.119:0/80163377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c1345d0 msgr2=0x7f3a8c134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 --2- 172.21.15.119:0/80163377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c1345d0 0x7f3a8c134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 -- 172.21.15.119:0/80163377 >> v2:172.21.15.17:3300/0 conn(0x7f3a8c09ceb0 msgr2=0x7f3a8c09d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 --2- 172.21.15.119:0/80163377 >> v2:172.21.15.17:3300/0 conn(0x7f3a8c09ceb0 0x7f3a8c09d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 monclient: found mon.noname-c 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 -- 172.21.15.119:0/80163377 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c09d800 con 0x7f3a8c09cad0 2024-09-15T21:17:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9117a640 10 monclient: get_auth_request con 0x7f3a8c09ceb0 auth_method 0 2024-09-15T21:17:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9117a640 1 --2- 172.21.15.119:0/80163377 >> v2:172.21.15.17:3300/0 conn(0x7f3a8c09ceb0 0x7f3a8c09d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a9197b640 1 --2- 172.21.15.119:0/80163377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09cad0 0x7f3a8c0a2dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c09de30 tx=0x7f3a7c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7d42fc557e7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 1 -- 172.21.15.119:0/80163377 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0b0070 con 0x7f3a8c09cad0 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: dump: 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 1 -- 172.21.15.119:0/80163377 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0ab600 con 0x7f3a8c09cad0 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 1 -- 172.21.15.119:0/80163377 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0ab9d0 con 0x7f3a8c09cad0 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 5 monclient: authenticate success, global_id 4123 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 4 set_mon_vals no callback set 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a90979640 10 monclient: dump: 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 10 monclient: shutdown 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- 172.21.15.119:0/80163377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09cad0 msgr2=0x7f3a8c0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/80163377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09cad0 0x7f3a8c0a2dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c09de30 tx=0x7f3a7c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- 172.21.15.119:0/80163377 shutdown_connections 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/80163377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c1345d0 0x7f3a8c134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/80163377 >> v2:172.21.15.17:3300/0 conn(0x7f3a8c09ceb0 0x7f3a8c09d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/80163377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09cad0 0x7f3a8c0a2dc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- 172.21.15.119:0/80163377 >> 172.21.15.119:0/80163377 conn(0x7f3a8c0a75d0 msgr2=0x7f3a8c0a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- 172.21.15.119:0/80163377 shutdown_connections 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.448+0000 7f3a93bdf640 1 -- 172.21.15.119:0/80163377 wait complete. 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: monmap: 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 0.000000 2024-09-15T21:17:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:created 0.000000 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 Processor -- start 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- start start 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: init 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c09cad0 0x7f3a8c1cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): picked mon.noname-c con 0x7f3a8c09cad0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 0x7f3a8c131810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): picked mon.noname-b con 0x7f3a8c09ceb0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a8c1345d0 0x7f3a8c131d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): picked mon.noname-a con 0x7f3a8c1345d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1415d0 con 0x7f3a8c1345d0 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c141750 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c1418d0 con 0x7f3a8c09cad0 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient(hunting): authenticate will time out at 960.249250s 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 0x7f3a8c131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 0x7f3a8c131810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46570/0 (socket says 172.21.15.119:46570) 2024-09-15T21:17:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 -- 172.21.15.119:0/3710534789 learned_addr learned my addr 172.21.15.119:0/3710534789 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): get_auth_request con 0x7f3a8c09ceb0 auth_method 0 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9217c640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a8c1345d0 0x7f3a8c131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient(hunting): handle_auth_done global_id 4126 payload 824 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9197b640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c09cad0 0x7f3a8c1cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 -- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c09cad0 msgr2=0x7f3a8c1cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c09cad0 0x7f3a8c1cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 -- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a8c1345d0 msgr2=0x7f3a8c131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a8c1345d0 0x7f3a8c131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 monclient: found mon.noname-b 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 -- 172.21.15.119:0/3710534789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c09e530 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9217c640 10 monclient: get_auth_request con 0x7f3a8c1345d0 auth_method 0 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9197b640 10 monclient: get_auth_request con 0x7f3a8c09cad0 auth_method 0 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9217c640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a8c1345d0 0x7f3a8c131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9197b640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c09cad0 0x7f3a8c1cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a9117a640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 0x7f3a8c131810 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800c7f0 tx=0x7f3a8800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880079d0 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: dump: 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.470 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a88007b70 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 5 monclient: authenticate success, global_id 4126 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _renew_subs 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c1387f0 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880113f0 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 10 monclient: dump: 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a83fff640 4 set_mon_vals no callback set 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _renew_subs 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c13c820 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a88011920 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f3a88021700 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.452+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c09d2c0 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.456+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3a8c1387f0 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.456+0000 7f3a827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.456+0000 7f3a827fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.528+0000 7f3a93bdf640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "7cf58353-b02d-4f6d-9170-d06191c75695", "id": 1}] 2024-09-15T21:17:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.528+0000 7f3a93bdf640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "7cf58353-b02d-4f6d-9170-d06191c75695", "id": 1}] 2024-09-15T21:17:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.528+0000 7f3a93bdf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.528+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7cf58353-b02d-4f6d-9170-d06191c75695", "id": 1} v 0) -- 0x7f3a8c06dfe0 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a827fc640 1 -- 172.21.15.119:0/3710534789 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7cf58353-b02d-4f6d-9170-d06191c75695", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f3a8c09e530 con 0x7f3a8c09ceb0 2024-09-15T21:17:21.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "7cf58353-b02d-4f6d-9170-d06191c75695", "id": 1}] 2024-09-15T21:17:21.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a827fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:21.538 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-09-15T21:17:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 10 monclient: shutdown 2024-09-15T21:17:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 msgr2=0x7f3a8c131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 0x7f3a8c131810 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800c7f0 tx=0x7f3a8800ccb0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 shutdown_connections 2024-09-15T21:17:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a8c1345d0 0x7f3a8c131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c09ceb0 0x7f3a8c131810 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 --2- 172.21.15.119:0/3710534789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a8c09cad0 0x7f3a8c1cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 >> 172.21.15.119:0/3710534789 conn(0x7f3a8c0a75d0 msgr2=0x7f3a8c0a5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 shutdown_connections 2024-09-15T21:17:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.536+0000 7f3a93bdf640 1 -- 172.21.15.119:0/3710534789 wait complete. 2024-09-15T21:17:21.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new d6e7cff2-5a15-4a7e-9a15-ff26517c535a 2 2024-09-15T21:17:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.660+0000 7f03ec3f1640 10 monclient: get_monmap_and_config 2024-09-15T21:17:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.660+0000 7f03ec3f1640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.660+0000 7f03ec3f1640 10 monclient: monmap: 2024-09-15T21:17:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:17:21.663485+0000 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:17:21.663485+0000 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:21.665 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 Processor -- start 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 -- start start 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient: init 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03e4133b20 0x7f03e4133f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): picked mon.noname-c con 0x7f03e4133b20 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03e409df50 0x7f03e4134440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): picked mon.noname-a con 0x7f03e409df50 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f03e40a11d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e409eff0 con 0x7f03e40a11d0 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f03e4134980 con 0x7f03e409df50 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f03e409f2d0 con 0x7f03e4133b20 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient(hunting): authenticate will time out at 960.460814s 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03e4133b20 0x7f03e4133f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03e4133b20 0x7f03e4133f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46580/0 (socket says 172.21.15.119:46580) 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 -- 172.21.15.119:0/1759518572 learned_addr learned my addr 172.21.15.119:0/1759518572 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): get_auth_request con 0x7f03e4133b20 auth_method 0 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e998c640 1 --2- 172.21.15.119:0/1759518572 >> v2:172.21.15.17:3300/0 conn(0x7f03e409df50 0x7f03e4134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e998c640 10 monclient(hunting): get_auth_request con 0x7f03e409df50 auth_method 0 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e998c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e998c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e998c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient(hunting): handle_auth_done global_id 4129 payload 274 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 -- 172.21.15.119:0/1759518572 >> v2:172.21.15.17:3300/0 conn(0x7f03e409df50 msgr2=0x7f03e4134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 --2- 172.21.15.119:0/1759518572 >> v2:172.21.15.17:3300/0 conn(0x7f03e409df50 0x7f03e4134440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 -- 172.21.15.119:0/1759518572 >> v1:172.21.15.17:6789/0 conn(0x7f03e40a11d0 legacy=0x7f03e40990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 monclient: found mon.noname-c 2024-09-15T21:17:21.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 -- 172.21.15.119:0/1759518572 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e409f5b0 con 0x7f03e4133b20 2024-09-15T21:17:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ea18d640 1 --2- 172.21.15.119:0/1759518572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e4133f00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03e40a4050 tx=0x7f03e001d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b24bfd7a6f62d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 1 -- 172.21.15.119:0/1759518572 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e000b040 con 0x7f03e4133b20 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: dump: 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 1 -- 172.21.15.119:0/1759518572 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e00a76e0 con 0x7f03e4133b20 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 5 monclient: authenticate success, global_id 4129 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03ec3f1640 4 set_mon_vals no callback set 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 1 -- 172.21.15.119:0/1759518572 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e00a7ab0 con 0x7f03e4133b20 2024-09-15T21:17:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.664+0000 7f03e918b640 10 monclient: dump: 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: shutdown 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1759518572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 msgr2=0x7f03e4133f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1759518572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e4133f00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03e40a4050 tx=0x7f03e001d7b0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1759518572 shutdown_connections 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1759518572 >> v2:172.21.15.17:3300/0 conn(0x7f03e409df50 0x7f03e4134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1759518572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e4133f00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1759518572 >> 172.21.15.119:0/1759518572 conn(0x7f03e40a66b0 msgr2=0x7f03e40a6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1759518572 shutdown_connections 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1759518572 wait complete. 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: monmap: 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.679 INFO:teuthology.orchestra.run.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 0.000000 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:created 0.000000 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 Processor -- start 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- start start 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: init 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03e409df50 0x7f03e41cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): picked mon.noname-c con 0x7f03e409df50 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e40a11d0 0x7f03e413a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): picked mon.noname-a con 0x7f03e40a11d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): picked mon.noname-b con 0x7f03e4133b20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f03e413e470 con 0x7f03e40a11d0 2024-09-15T21:17:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f03e413e2f0 con 0x7f03e4133b20 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f03e413e5f0 con 0x7f03e409df50 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient(hunting): authenticate will time out at 960.466187s 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e413ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46594/0 (socket says 172.21.15.119:46594) 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 -- 172.21.15.119:0/1987990598 learned_addr learned my addr 172.21.15.119:0/1987990598 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): get_auth_request con 0x7f03e4133b20 auth_method 0 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea18d640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03e409df50 0x7f03e41cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03e998c640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e40a11d0 0x7f03e413a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea18d640 10 monclient(hunting): get_auth_request con 0x7f03e409df50 auth_method 0 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea18d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea18d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea18d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03e998c640 10 monclient(hunting): get_auth_request con 0x7f03e40a11d0 auth_method 0 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03e998c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03e998c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03e998c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient(hunting): handle_auth_done global_id 4132 payload 824 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 -- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03e409df50 msgr2=0x7f03e41cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03e409df50 0x7f03e41cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 -- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e40a11d0 msgr2=0x7f03e413a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e40a11d0 0x7f03e413a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 monclient: found mon.noname-b 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 -- 172.21.15.119:0/1987990598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e4136e00 con 0x7f03e4133b20 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea18d640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03e409df50 0x7f03e41cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ea98e640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e413ab70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c780 tx=0x7f03dc00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc016020 con 0x7f03e4133b20 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: dump: 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.685 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc00a9f0 con 0x7f03e4133b20 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 5 monclient: authenticate success, global_id 4132 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: _renew_subs 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e41370e0 con 0x7f03e4133b20 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00ae20 con 0x7f03e4133b20 2024-09-15T21:17:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 10 monclient: dump: 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03e898a640 4 set_mon_vals no callback set 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: _renew_subs 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e4075080 con 0x7f03e4133b20 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03dc019020 con 0x7f03e4133b20 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.668+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f03dc012840 con 0x7f03e4133b20 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03ec3f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03ec3f1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03ec3f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e40990b0 con 0x7f03e4133b20 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f03dc020070 con 0x7f03e4133b20 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.672+0000 7f03daffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:21.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.748+0000 7f03ec3f1640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "d6e7cff2-5a15-4a7e-9a15-ff26517c535a", "id": 2}] 2024-09-15T21:17:21.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.748+0000 7f03ec3f1640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "d6e7cff2-5a15-4a7e-9a15-ff26517c535a", "id": 2}] 2024-09-15T21:17:21.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.748+0000 7f03ec3f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.748+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d6e7cff2-5a15-4a7e-9a15-ff26517c535a", "id": 2} v 0) -- 0x7f03e41cc1a0 con 0x7f03e4133b20 2024-09-15T21:17:21.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03daffd640 1 -- 172.21.15.119:0/1987990598 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6e7cff2-5a15-4a7e-9a15-ff26517c535a", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f03dc0124f0 con 0x7f03e4133b20 2024-09-15T21:17:21.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "d6e7cff2-5a15-4a7e-9a15-ff26517c535a", "id": 2}] 2024-09-15T21:17:21.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03daffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:21.759 INFO:teuthology.orchestra.run.smithi119.stdout:2 2024-09-15T21:17:21.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 10 monclient: shutdown 2024-09-15T21:17:21.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 msgr2=0x7f03e413ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e413ab70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c780 tx=0x7f03dc00cc40 comp rx=0 tx=0).stop 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 shutdown_connections 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03e4133b20 0x7f03e413ab70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e40a11d0 0x7f03e413a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 --2- 172.21.15.119:0/1987990598 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03e409df50 0x7f03e41cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 >> 172.21.15.119:0/1987990598 conn(0x7f03e40a66b0 msgr2=0x7f03e413b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 shutdown_connections 2024-09-15T21:17:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.756+0000 7f03ec3f1640 1 -- 172.21.15.119:0/1987990598 wait complete. 2024-09-15T21:17:21.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new e95cbd18-104f-4bf8-8496-096427e7459b 3 2024-09-15T21:17:21.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient: get_monmap_and_config 2024-09-15T21:17:21.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient: monmap: 2024-09-15T21:17:21.878 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:17:21.877138+0000 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:17:21.877138+0000 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 Processor -- start 2024-09-15T21:17:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 -- start start 2024-09-15T21:17:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient: init 2024-09-15T21:17:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:17:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc0a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): picked mon.noname-b con 0x7f45dc0a28b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f45dc0a31d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45dc134d70 0x7f45dc13c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): picked mon.noname-c con 0x7f45dc134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc09dd50 con 0x7f45dc0a31d0 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f45dc09d790 con 0x7f45dc134d70 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc09d4b0 con 0x7f45dc0a28b0 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e2afa640 10 monclient(hunting): authenticate will time out at 960.673857s 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45dc134d70 0x7f45dc13c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45dc134d70 0x7f45dc13c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46610/0 (socket says 172.21.15.119:46610) 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 -- 172.21.15.119:0/4081288108 learned_addr learned my addr 172.21.15.119:0/4081288108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): get_auth_request con 0x7f45dc134d70 auth_method 0 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e0896640 1 --2- 172.21.15.119:0/4081288108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc0a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e0896640 10 monclient(hunting): get_auth_request con 0x7f45dc0a28b0 auth_method 0 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e0896640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e0896640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e0896640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient(hunting): handle_auth_done global_id 4135 payload 274 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 -- 172.21.15.119:0/4081288108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 msgr2=0x7f45dc0a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 --2- 172.21.15.119:0/4081288108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc0a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 -- 172.21.15.119:0/4081288108 >> v1:172.21.15.17:6789/0 conn(0x7f45dc0a31d0 legacy=0x7f45dc134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 monclient: found mon.noname-c 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.876+0000 7f45e1097640 1 -- 172.21.15.119:0/4081288108 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc09c410 con 0x7f45dc134d70 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e1097640 1 --2- 172.21.15.119:0/4081288108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc134d70 0x7f45dc13c8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009e50 tx=0x7f45d401c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64da2c7fcbd7cb0c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 1 -- 172.21.15.119:0/4081288108 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40a5650 con 0x7f45dc134d70 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: dump: 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 1 -- 172.21.15.119:0/4081288108 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d40a57f0 con 0x7f45dc134d70 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 5 monclient: authenticate success, global_id 4135 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 4 set_mon_vals no callback set 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 1 -- 172.21.15.119:0/4081288108 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40a5bc0 con 0x7f45dc134d70 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45db7fe640 10 monclient: dump: 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: shutdown 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/4081288108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc134d70 msgr2=0x7f45dc13c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 --2- 172.21.15.119:0/4081288108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc134d70 0x7f45dc13c8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009e50 tx=0x7f45d401c8f0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/4081288108 shutdown_connections 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 --2- 172.21.15.119:0/4081288108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc134d70 0x7f45dc13c8c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 --2- 172.21.15.119:0/4081288108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc0a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/4081288108 >> 172.21.15.119:0/4081288108 conn(0x7f45dc0a74a0 msgr2=0x7f45dc0a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/4081288108 shutdown_connections 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/4081288108 wait complete. 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: build_initial_monmap 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: monmap: 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 0.000000 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:created 0.000000 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 Processor -- start 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- start start 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: init 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): picked mon.noname-c con 0x7f45dc0a28b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 0x7f45dc1cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): picked mon.noname-b con 0x7f45dc0a31d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45dc134d70 0x7f45dc1ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): picked mon.noname-a con 0x7f45dc134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc09ebc0 con 0x7f45dc134d70 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc09ea40 con 0x7f45dc0a31d0 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc13f620 con 0x7f45dc0a28b0 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient(hunting): authenticate will time out at 960.678762s 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 0x7f45dc1cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 0x7f45dc1cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46616/0 (socket says 172.21.15.119:46616) 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 -- 172.21.15.119:0/258043868 learned_addr learned my addr 172.21.15.119:0/258043868 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): get_auth_request con 0x7f45dc0a31d0 auth_method 0 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e0896640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e1097640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45dc134d70 0x7f45dc1ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e0896640 10 monclient(hunting): get_auth_request con 0x7f45dc0a28b0 auth_method 0 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e0896640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e0896640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e0896640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient(hunting): handle_auth_done global_id 4138 payload 824 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 -- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 msgr2=0x7f45dc06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc06cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 -- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45dc134d70 msgr2=0x7f45dc1ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45dc134d70 0x7f45dc1ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 monclient: found mon.noname-b 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 -- 172.21.15.119:0/258043868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc09f1b0 con 0x7f45dc0a31d0 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e1097640 10 monclient: get_auth_request con 0x7f45dc134d70 auth_method 0 2024-09-15T21:17:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e1097640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45dc134d70 0x7f45dc1ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45dbfff640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 0x7f45dc1cdd00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c450 tx=0x7f45c400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4012830 con 0x7f45dc0a31d0 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: dump: 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: _finish_auth 0 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4017070 con 0x7f45dc0a31d0 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 5 monclient: authenticate success, global_id 4138 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4012c30 con 0x7f45dc0a31d0 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: _renew_subs 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc09e100 con 0x7f45dc0a31d0 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45daffd640 4 set_mon_vals no callback set 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 10 monclient: dump: 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:21.907 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: _renew_subs 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc137a90 con 0x7f45dc0a31d0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f45c401c460 con 0x7f45dc0a31d0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.880+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f45c4012dd0 con 0x7f45dc0a31d0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45e2afa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45e2afa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45e2afa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc0a2c90 con 0x7f45dc0a31d0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f45c401b730 con 0x7f45dc0a31d0 2024-09-15T21:17:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45d97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:21.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.884+0000 7f45d97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.956+0000 7f45e2afa640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "e95cbd18-104f-4bf8-8496-096427e7459b", "id": 3}] 2024-09-15T21:17:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.956+0000 7f45e2afa640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "e95cbd18-104f-4bf8-8496-096427e7459b", "id": 3}] 2024-09-15T21:17:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.956+0000 7f45e2afa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.956+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e95cbd18-104f-4bf8-8496-096427e7459b", "id": 3} v 0) -- 0x7f45dc137df0 con 0x7f45dc0a31d0 2024-09-15T21:17:21.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.960+0000 7f45d97fa640 1 -- 172.21.15.119:0/258043868 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e95cbd18-104f-4bf8-8496-096427e7459b", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f45c401bae0 con 0x7f45dc0a31d0 2024-09-15T21:17:21.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.960+0000 7f45d97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "e95cbd18-104f-4bf8-8496-096427e7459b", "id": 3}] 2024-09-15T21:17:21.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.960+0000 7f45d97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:21.965 INFO:teuthology.orchestra.run.smithi119.stdout:3 2024-09-15T21:17:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 10 monclient: shutdown 2024-09-15T21:17:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 msgr2=0x7f45dc1cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 0x7f45dc1cdd00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c450 tx=0x7f45c400c910 comp rx=0 tx=0).stop 2024-09-15T21:17:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 shutdown_connections 2024-09-15T21:17:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45dc134d70 0x7f45dc1ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc0a31d0 0x7f45dc1cdd00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 --2- 172.21.15.119:0/258043868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45dc0a28b0 0x7f45dc06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 >> 172.21.15.119:0/258043868 conn(0x7f45dc0a74a0 msgr2=0x7f45dc09fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 shutdown_connections 2024-09-15T21:17:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:21.964+0000 7f45e2afa640 1 -- 172.21.15.119:0/258043868 wait complete. 2024-09-15T21:17:21.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 79502179-8d0c-49fa-86e7-2afe7bfee508 4 2024-09-15T21:17:22.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient: get_monmap_and_config 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient: build_initial_monmap 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient: monmap: 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:17:22.086882+0000 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:17:22.086882+0000 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:22.088 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:22.089 INFO:teuthology.orchestra.run.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:22.089 INFO:teuthology.orchestra.run.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:22.089 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 Processor -- start 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 -- start start 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient: init 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4634137710 0x7f46340a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): picked mon.noname-c con 0x7f4634137710 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46340a1ae0 0x7f4634137af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): picked mon.noname-a con 0x7f46340a1ae0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4634138030 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:22.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463409dae0 con 0x7f4634138030 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f463409e410 con 0x7f46340a1ae0 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f463409ddc0 con 0x7f4634137710 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f463bac5640 10 monclient(hunting): authenticate will time out at 960.883694s 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4634137710 0x7f46340a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4634137710 0x7f46340a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46630/0 (socket says 172.21.15.119:46630) 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 1 -- 172.21.15.119:0/1465990070 learned_addr learned my addr 172.21.15.119:0/1465990070 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): get_auth_request con 0x7f4634137710 auth_method 0 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.084+0000 7f4639861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639060640 1 --2- 172.21.15.119:0/1465990070 >> v2:172.21.15.17:3300/0 conn(0x7f46340a1ae0 0x7f4634137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 10 monclient(hunting): handle_auth_done global_id 4141 payload 274 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 1 -- 172.21.15.119:0/1465990070 >> v2:172.21.15.17:3300/0 conn(0x7f46340a1ae0 msgr2=0x7f4634137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 1 --2- 172.21.15.119:0/1465990070 >> v2:172.21.15.17:3300/0 conn(0x7f46340a1ae0 0x7f4634137af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 1 -- 172.21.15.119:0/1465990070 >> v1:172.21.15.17:6789/0 conn(0x7f4634138030 legacy=0x7f46340990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 1 monclient: found mon.noname-c 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 1 -- 172.21.15.119:0/1465990070 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463409e0a0 con 0x7f4634137710 2024-09-15T21:17:22.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f4639861640 1 --2- 172.21.15.119:0/1465990070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4634137710 0x7f46340a15a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46340a7530 tx=0x7f462401c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7342455e40ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 1 -- 172.21.15.119:0/1465990070 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240b1070 con 0x7f4634137710 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: dump: 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 5 monclient: authenticate success, global_id 4141 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 1 -- 172.21.15.119:0/1465990070 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46240a47c0 con 0x7f4634137710 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 4 set_mon_vals no callback set 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 1 -- 172.21.15.119:0/1465990070 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240a4b90 con 0x7f4634137710 2024-09-15T21:17:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463885f640 10 monclient: dump: 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 10 monclient: shutdown 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 -- 172.21.15.119:0/1465990070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4634137710 msgr2=0x7f46340a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 --2- 172.21.15.119:0/1465990070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4634137710 0x7f46340a15a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46340a7530 tx=0x7f462401c0c0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 -- 172.21.15.119:0/1465990070 shutdown_connections 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 --2- 172.21.15.119:0/1465990070 >> v2:172.21.15.17:3300/0 conn(0x7f46340a1ae0 0x7f4634137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 --2- 172.21.15.119:0/1465990070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4634137710 0x7f46340a15a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 -- 172.21.15.119:0/1465990070 >> 172.21.15.119:0/1465990070 conn(0x7f46340a66b0 msgr2=0x7f46340a6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 -- 172.21.15.119:0/1465990070 shutdown_connections 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 -- 172.21.15.119:0/1465990070 wait complete. 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 10 monclient: build_initial_monmap 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 10 monclient: monmap: 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 0.000000 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:created 0.000000 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.088+0000 7f463bac5640 1 Processor -- start 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- start start 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: init 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:17:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4634137710 0x7f4634136560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): picked mon.noname-a con 0x7f4634137710 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4634138030 0x7f463412b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): picked mon.noname-c con 0x7f4634138030 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 0x7f463412bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): picked mon.noname-b con 0x7f463412bab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4634142cc0 con 0x7f4634137710 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4634142b40 con 0x7f463412bab0 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4634142e40 con 0x7f4634138030 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient(hunting): authenticate will time out at 960.888488s 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 0x7f463412bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 0x7f463412bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46644/0 (socket says 172.21.15.119:46644) 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 -- 172.21.15.119:0/1906604838 learned_addr learned my addr 172.21.15.119:0/1906604838 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): get_auth_request con 0x7f463412bab0 auth_method 0 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f4639060640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4634138030 0x7f463412b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f4639861640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4634137710 0x7f4634136560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient(hunting): handle_auth_done global_id 4144 payload 824 2024-09-15T21:17:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 -- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4634138030 msgr2=0x7f463412b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4634138030 0x7f463412b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 -- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4634137710 msgr2=0x7f4634136560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4634137710 0x7f4634136560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 monclient: found mon.noname-b 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 -- 172.21.15.119:0/1906604838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46340a4ac0 con 0x7f463412bab0 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f4639060640 10 monclient: get_auth_request con 0x7f4634138030 auth_method 0 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f4639861640 10 monclient: get_auth_request con 0x7f4634137710 auth_method 0 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f4639060640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4634138030 0x7f463412b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f4639861640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4634137710 0x7f4634136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463a062640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 0x7f463412bf20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c7b0 tx=0x7f462c00cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c004b70 con 0x7f463412bab0 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: dump: 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.106 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c004d10 con 0x7f463412bab0 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 5 monclient: authenticate success, global_id 4144 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _renew_subs 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463412d4a0 con 0x7f463412bab0 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c00aa90 con 0x7f463412bab0 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:22.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 10 monclient: dump: 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462bfff640 4 set_mon_vals no callback set 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _renew_subs 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463412c460 con 0x7f463412bab0 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f462c005030 con 0x7f463412bab0 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f462c01b400 con 0x7f463412bab0 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.092+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4634137af0 con 0x7f463412bab0 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.096+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f462c012420 con 0x7f463412bab0 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.096+0000 7f462a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.096+0000 7f462a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:22.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.160+0000 7f463bac5640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "79502179-8d0c-49fa-86e7-2afe7bfee508", "id": 4}] 2024-09-15T21:17:22.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.160+0000 7f463bac5640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "79502179-8d0c-49fa-86e7-2afe7bfee508", "id": 4}] 2024-09-15T21:17:22.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.160+0000 7f463bac5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.160+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "79502179-8d0c-49fa-86e7-2afe7bfee508", "id": 4} v 0) -- 0x7f463413a330 con 0x7f463412bab0 2024-09-15T21:17:22.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f462a7fc640 1 -- 172.21.15.119:0/1906604838 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "79502179-8d0c-49fa-86e7-2afe7bfee508", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f462c005500 con 0x7f463412bab0 2024-09-15T21:17:22.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f462a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "79502179-8d0c-49fa-86e7-2afe7bfee508", "id": 4}] 2024-09-15T21:17:22.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f462a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:22.170 INFO:teuthology.orchestra.run.smithi119.stdout:4 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 10 monclient: shutdown 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 msgr2=0x7f463412bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 0x7f463412bf20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c7b0 tx=0x7f462c00cc70 comp rx=0 tx=0).stop 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 shutdown_connections 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f463412bab0 0x7f463412bf20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4634138030 0x7f463412b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 --2- 172.21.15.119:0/1906604838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4634137710 0x7f4634136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 >> 172.21.15.119:0/1906604838 conn(0x7f46340a66b0 msgr2=0x7f463409cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 shutdown_connections 2024-09-15T21:17:22.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.168+0000 7f463bac5640 1 -- 172.21.15.119:0/1906604838 wait complete. 2024-09-15T21:17:22.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 10352789-0e34-46f3-be2f-a929f23a8799 5 2024-09-15T21:17:22.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient: get_monmap_and_config 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient: build_initial_monmap 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient: monmap: 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:17:22.297902+0000 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:17:22.297902+0000 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:22.299 INFO:teuthology.orchestra.run.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:22.300 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 Processor -- start 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 -- start start 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient: init 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f399c09c7c0 0x7f399c137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): picked mon.noname-a con 0x7f399c09c7c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f399c09cba0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 0x7f399c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): picked mon.noname-b con 0x7f399c0a2560 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399c09a630 con 0x7f399c09cba0 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f399c074810 con 0x7f399c09c7c0 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f399c09a8f0 con 0x7f399c0a2560 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f39a153a640 10 monclient(hunting): authenticate will time out at 961.094926s 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399a7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f399c09cba0 0x7f399c0a1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43182/0 (socket says 172.21.15.119:43182) 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399a7fc640 1 -- 172.21.15.119:0/2224036851 learned_addr learned my addr 172.21.15.119:0/2224036851 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:17:22.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107871499 0 0) 0x7f399c09a630 con 0x7f399c09cba0 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 0x7f399c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 1 --2- 172.21.15.119:0/2224036851 >> v2:172.21.15.17:3300/0 conn(0x7f399c09c7c0 0x7f399c137bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): my global_id is 4147 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3984003530 con 0x7f399c09cba0 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): get_auth_request con 0x7f399c09c7c0 auth_method 0 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): get_auth_request con 0x7f399c0a2560 auth_method 0 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1579676760 0 0) 0x7f3984003530 con 0x7f399c09cba0 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 msgr2=0x7f399c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 0x7f399c13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 >> v2:172.21.15.17:3300/0 conn(0x7f399c09c7c0 msgr2=0x7f399c137bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 --2- 172.21.15.119:0/2224036851 >> v2:172.21.15.17:3300/0 conn(0x7f399c09c7c0 0x7f399c137bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c068f70 con 0x7f399c09cba0 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f398c001700 con 0x7f399c09cba0 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: dump: 2024-09-15T21:17:22.308 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 >> v1:172.21.15.119:6789/0 conn(0x7f399c09cba0 legacy=0x7f399c0a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3984003a60 0x7f3984003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): picked mon.b con 0x7f3984003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3984004810 0x7f3984004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): picked mon.a con 0x7f3984004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 0x7f39840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): picked mon.c con 0x7f399c0a2560 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f399c074810 con 0x7f3984004810 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f399c09a8f0 con 0x7f3984003a60 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f398400bb80 con 0x7f399c0a2560 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f3999ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3984003a60 0x7f3984003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): get_auth_request con 0x7f3984003a60 auth_method 0 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.296+0000 7f399affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient(hunting): handle_auth_done global_id 4147 payload 293 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 -- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 msgr2=0x7f39840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f399c0a2560 0x7f39840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 -- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3984004810 msgr2=0x7f3984004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3984004810 0x7f3984004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 monclient: found mon.b 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 -- 172.21.15.119:0/2224036851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3984003530 con 0x7f3984003a60 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 5 monclient: authenticate success, global_id 4147 2024-09-15T21:17:22.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399affd640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3984003a60 0x7f3984003e80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3990009ae0 tx=0x7f399001ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6e7e8da0b3a0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900096b0 con 0x7f3984003a60 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: dump: 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39900a64a0 con 0x7f3984003a60 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 1 -- 172.21.15.119:0/2224036851 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900a6890 con 0x7f3984003a60 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 4 set_mon_vals no callback set 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f3999ffb640 10 monclient: dump: 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.314 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.315 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 10 monclient: shutdown 2024-09-15T21:17:22.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 1 -- 172.21.15.119:0/2224036851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3984003a60 msgr2=0x7f3984003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 1 --2- 172.21.15.119:0/2224036851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3984003a60 0x7f3984003e80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3990009ae0 tx=0x7f399001ca90 comp rx=0 tx=0).stop 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f399b7fe640 1 -- 172.21.15.119:0/2224036851 reap_dead start 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 1 -- 172.21.15.119:0/2224036851 shutdown_connections 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 1 -- 172.21.15.119:0/2224036851 >> 172.21.15.119:0/2224036851 conn(0x7f399c0a2f40 msgr2=0x7f399c0a3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 1 -- 172.21.15.119:0/2224036851 shutdown_connections 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 1 -- 172.21.15.119:0/2224036851 wait complete. 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 10 monclient: build_initial_monmap 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.300+0000 7f39a153a640 10 monclient: monmap: 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 0 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 0.000000 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:created 0.000000 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 Processor -- start 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- start start 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: init 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3984003a60 0x7f399c08eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): picked mon.noname-c con 0x7f3984003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f399c13d000 0x7f399c08cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): picked mon.noname-a con 0x7f399c13d000 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 0x7f399c08d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): picked mon.noname-b con 0x7f399c08d530 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f399c140610 con 0x7f399c13d000 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f399c140490 con 0x7f399c08d530 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f399c140790 con 0x7f3984003a60 2024-09-15T21:17:22.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient(hunting): authenticate will time out at 961.100754s 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 0x7f399c08d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 0x7f399c08d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41032/0 (socket says 172.21.15.119:41032) 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 -- 172.21.15.119:0/3951048171 learned_addr learned my addr 172.21.15.119:0/3951048171 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): get_auth_request con 0x7f399c08d530 auth_method 0 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399a7fc640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f399c13d000 0x7f399c08cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399affd640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3984003a60 0x7f399c08eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399affd640 10 monclient(hunting): get_auth_request con 0x7f3984003a60 auth_method 0 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient(hunting): handle_auth_done global_id 4150 payload 824 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 -- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3984003a60 msgr2=0x7f399c08eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3984003a60 0x7f399c08eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 -- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f399c13d000 msgr2=0x7f399c08cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f399c13d000 0x7f399c08cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 -- 172.21.15.119:0/3951048171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c0a53e0 con 0x7f399c08d530 2024-09-15T21:17:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399a7fc640 10 monclient: get_auth_request con 0x7f399c13d000 auth_method 0 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399a7fc640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f399c13d000 0x7f399c08cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f399b7fe640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 0x7f399c08d9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f399c09ade0 tx=0x7f399400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994018020 con 0x7f399c08d530 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: dump: 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399400bae0 con 0x7f399c08d530 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 5 monclient: authenticate success, global_id 4150 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _renew_subs 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994014400 con 0x7f399c08d530 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c09d630 con 0x7f399c08d530 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 10 monclient: dump: 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:epoch 1 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:election_strategy: 1 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39997fa640 4 set_mon_vals no callback set 2024-09-15T21:17:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _renew_subs 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c08dee0 con 0x7f399c08d530 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f399401b020 con 0x7f399c08d530 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f399401dbb0 con 0x7f399c08d530 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.304+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399c137bf0 con 0x7f399c08d530 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.308+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3994022030 con 0x7f399c08d530 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.308+0000 7f39837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.308+0000 7f39837fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.372+0000 7f39a153a640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "10352789-0e34-46f3-be2f-a929f23a8799", "id": 5}] 2024-09-15T21:17:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.372+0000 7f39a153a640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "10352789-0e34-46f3-be2f-a929f23a8799", "id": 5}] 2024-09-15T21:17:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.372+0000 7f39a153a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.372+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "10352789-0e34-46f3-be2f-a929f23a8799", "id": 5} v 0) -- 0x7f399c139f00 con 0x7f399c08d530 2024-09-15T21:17:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39837fe640 1 -- 172.21.15.119:0/3951048171 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10352789-0e34-46f3-be2f-a929f23a8799", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f399401d780 con 0x7f399c08d530 2024-09-15T21:17:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "10352789-0e34-46f3-be2f-a929f23a8799", "id": 5}] 2024-09-15T21:17:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39837fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:22.383 INFO:teuthology.orchestra.run.smithi119.stdout:5 2024-09-15T21:17:22.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 10 monclient: shutdown 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 msgr2=0x7f399c08d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 0x7f399c08d9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f399c09ade0 tx=0x7f399400b480 comp rx=0 tx=0).stop 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 shutdown_connections 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c08d530 0x7f399c08d9a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f399c13d000 0x7f399c08cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 --2- 172.21.15.119:0/3951048171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3984003a60 0x7f399c08eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 >> 172.21.15.119:0/3951048171 conn(0x7f399c0a2f40 msgr2=0x7f399c099560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 shutdown_connections 2024-09-15T21:17:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-15T21:17:22.380+0000 7f39a153a640 1 -- 172.21.15.119:0/3951048171 wait complete. 2024-09-15T21:17:22.398 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:17:22.398 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:17:22.401 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:17:22.401 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:17:22.401 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:17:22.404 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:17:22.405 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:17:22.405 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:17:22.408 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:17:22.408 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T21:17:22.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T21:17:22.410 INFO:tasks.ceph.osd.3:Started 2024-09-15T21:17:22.411 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T21:17:22.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T21:17:22.413 INFO:tasks.ceph.osd.4:Started 2024-09-15T21:17:22.413 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T21:17:22.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T21:17:22.416 INFO:tasks.ceph.osd.5:Started 2024-09-15T21:17:22.416 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:22.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient: get_monmap_and_config 2024-09-15T21:17:22.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient: build_initial_monmap 2024-09-15T21:17:22.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient: monmap: 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:22.535085+0000 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:22.535085+0000 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 Processor -- start 2024-09-15T21:17:22.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 -- start start 2024-09-15T21:17:22.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient: init 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee180a4a90 0x7fee180a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): picked mon.noname-c con 0x7fee180a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee180a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): picked mon.noname-b con 0x7fee180a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fee18137850 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:22.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee180a2c30 con 0x7fee18137850 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fee180a0f00 con 0x7fee180a4a90 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fee180a19b0 con 0x7fee180a53b0 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1cd87640 10 monclient(hunting): authenticate will time out at 965.930035s 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee180a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee180a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47624/0 (socket says 172.21.15.17:47624) 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 -- 172.21.15.17:0/4119233908 learned_addr learned my addr 172.21.15.17:0/4119233908 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): get_auth_request con 0x7fee180a53b0 auth_method 0 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1659c640 1 --2- 172.21.15.17:0/4119233908 >> v2:172.21.15.119:3300/0 conn(0x7fee180a4a90 0x7fee180a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1659c640 10 monclient(hunting): get_auth_request con 0x7fee180a4a90 auth_method 0 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient(hunting): handle_auth_done global_id 4145 payload 274 2024-09-15T21:17:22.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 -- 172.21.15.17:0/4119233908 >> v2:172.21.15.119:3300/0 conn(0x7fee180a4a90 msgr2=0x7fee180a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 --2- 172.21.15.17:0/4119233908 >> v2:172.21.15.119:3300/0 conn(0x7fee180a4a90 0x7fee180a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 -- 172.21.15.17:0/4119233908 >> v1:172.21.15.119:6789/0 conn(0x7fee18137850 legacy=0x7fee18137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 monclient: found mon.noname-b 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:22.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee15d9b640 1 -- 172.21.15.17:0/4119233908 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee180a1c90 con 0x7fee180a53b0 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.530+0000 7fee1659c640 1 --2- 172.21.15.17:0/4119233908 >> v2:172.21.15.119:3300/0 conn(0x7fee180a4a90 0x7fee180a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 1 --2- 172.21.15.17:0/4119233908 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee180a57d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009870 tx=0x7fee0c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c902eef19b2204fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 1 -- 172.21.15.17:0/4119233908 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0b1070 con 0x7fee180a53b0 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: dump: 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.546 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 1 -- 172.21.15.17:0/4119233908 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c0ac640 con 0x7fee180a53b0 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 1 -- 172.21.15.17:0/4119233908 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0aca10 con 0x7fee180a53b0 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 5 monclient: authenticate success, global_id 4145 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:22.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 4 set_mon_vals no callback set 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1559a640 10 monclient: dump: 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.548 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.549 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.549 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: shutdown 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- 172.21.15.17:0/4119233908 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 msgr2=0x7fee180a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 --2- 172.21.15.17:0/4119233908 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee180a57d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009870 tx=0x7fee0c01c3f0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- 172.21.15.17:0/4119233908 shutdown_connections 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 --2- 172.21.15.17:0/4119233908 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee180a57d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 --2- 172.21.15.17:0/4119233908 >> v2:172.21.15.119:3300/0 conn(0x7fee180a4a90 0x7fee180a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- 172.21.15.17:0/4119233908 >> 172.21.15.17:0/4119233908 conn(0x7fee180aabb0 msgr2=0x7fee180a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- 172.21.15.17:0/4119233908 shutdown_connections 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- 172.21.15.17:0/4119233908 wait complete. 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: build_initial_monmap 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: monmap: 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:22.554 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 Processor -- start 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- start start 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: init 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee180a4a90 0x7fee181cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): picked mon.noname-b con 0x7fee180a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee18136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): picked mon.noname-c con 0x7fee180a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 0x7fee1813cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): picked mon.noname-a con 0x7fee18137850 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fee181426d0 con 0x7fee18137850 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee18142550 con 0x7fee180a4a90 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fee18142850 con 0x7fee180a53b0 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1cd87640 10 monclient(hunting): authenticate will time out at 965.935109s 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 0x7fee1813cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee18136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 0x7fee1813cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48236/0 (socket says 172.21.15.17:48236) 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 -- 172.21.15.17:0/2638451186 learned_addr learned my addr 172.21.15.17:0/2638451186 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): get_auth_request con 0x7fee18137850 auth_method 0 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee180a4a90 0x7fee181cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): get_auth_request con 0x7fee180a53b0 auth_method 0 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): get_auth_request con 0x7fee180a4a90 auth_method 0 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee1659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient(hunting): handle_auth_done global_id 4134 payload 824 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 -- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 msgr2=0x7fee18136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee18136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 -- 172.21.15.17:0/2638451186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee180a4a90 msgr2=0x7fee181cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee180a4a90 0x7fee181cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 monclient: found mon.noname-a 2024-09-15T21:17:22.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 -- 172.21.15.17:0/2638451186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee181d0a10 con 0x7fee18137850 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee15d9b640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee18136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.534+0000 7fee16d9d640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 0x7fee1813cb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee1000c7f0 tx=0x7fee1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10003b80 con 0x7fee18137850 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: dump: 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee10003d20 con 0x7fee18137850 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 5 monclient: authenticate success, global_id 4134 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 monclient: _renew_subs 2024-09-15T21:17:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee18121010 con 0x7fee18137850 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1000aa30 con 0x7fee18137850 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: dump: 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 monclient: _renew_subs 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee18120d30 con 0x7fee18137850 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee07fff640 4 set_mon_vals no callback set 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee1000abd0 con 0x7fee18137850 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fee10012b50 con 0x7fee18137850 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee180a4e70 con 0x7fee18137850 2024-09-15T21:17:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fee181d0a10 con 0x7fee18137850 2024-09-15T21:17:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.538+0000 7fee06ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:22.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.598+0000 7fee1cd87640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:22.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.598+0000 7fee1cd87640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:22.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.598+0000 7fee1cd87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:22.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.598+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee18069200 con 0x7fee18137850 2024-09-15T21:17:22.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee06ffd640 1 -- 172.21.15.17:0/2638451186 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7fee180a2c30 con 0x7fee18137850 2024-09-15T21:17:22.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee06ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:22.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee06ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:22.610 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:22.610 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:22.374306+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 10 monclient: shutdown 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 msgr2=0x7fee1813cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 0x7fee1813cb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee1000c7f0 tx=0x7fee1000ccb0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 shutdown_connections 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee18137850 0x7fee1813cb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee180a53b0 0x7fee18136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 --2- 172.21.15.17:0/2638451186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee180a4a90 0x7fee181cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:22.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 >> 172.21.15.17:0/2638451186 conn(0x7fee180aabb0 msgr2=0x7fee180a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:22.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 shutdown_connections 2024-09-15T21:17:22.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:22.602+0000 7fee1cd87640 1 -- 172.21.15.17:0/2638451186 wait complete. 2024-09-15T21:17:22.628 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T21:17:22.628 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T21:17:23.232 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:23.335 INFO:tasks.ceph.mgr.x.smithi119.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-15T21:17:23.335 INFO:tasks.ceph.mgr.x.smithi119.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-15T21:17:23.335 INFO:tasks.ceph.mgr.x.smithi119.stderr: from numpy import show_config as show_numpy_config 2024-09-15T21:17:23.339 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-15T21:17:23.336+0000 7f38882357c0 -1 Falling back to public interface 2024-09-15T21:17:23.355 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient: get_monmap_and_config 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient: build_initial_monmap 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient: monmap: 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:epoch 0 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:17:23.352612+0000 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:17:23.352612+0000 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:23.356 INFO:teuthology.misc.health.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:23.357 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:23.358 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 Processor -- start 2024-09-15T21:17:23.358 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 -- start start 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient: init 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae70137680 0x7fae700a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): picked mon.noname-c con 0x7fae70137680 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70137e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): picked mon.noname-b con 0x7fae70137a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 0x7fae70136a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:23.359 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): picked mon.noname-a con 0x7fae70136600 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fae700a1990 con 0x7fae70136600 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fae700a25c0 con 0x7fae70137680 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fae700a16b0 con 0x7fae70137a60 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.346+0000 7fae74d83640 10 monclient(hunting): authenticate will time out at 966.747984s 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70137e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 0x7fae70136a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:23.360 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70137e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47648/0 (socket says 172.21.15.17:47648) 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 0x7fae70136a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48250/0 (socket says 172.21.15.17:48250) 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 -- 172.21.15.17:0/4281185535 learned_addr learned my addr 172.21.15.17:0/4281185535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): get_auth_request con 0x7fae70137a60 auth_method 0 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): get_auth_request con 0x7fae70136600 auth_method 0 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:23.361 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6e59c640 1 --2- 172.21.15.17:0/4281185535 >> v2:172.21.15.119:3300/0 conn(0x7fae70137680 0x7fae700a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6e59c640 10 monclient(hunting): get_auth_request con 0x7fae70137680 auth_method 0 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_done global_id 4151 payload 274 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 -- 172.21.15.17:0/4281185535 >> v2:172.21.15.119:3300/0 conn(0x7fae70137680 msgr2=0x7fae700a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 --2- 172.21.15.17:0/4281185535 >> v2:172.21.15.119:3300/0 conn(0x7fae70137680 0x7fae700a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 -- 172.21.15.17:0/4281185535 >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 msgr2=0x7fae70136a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 --2- 172.21.15.17:0/4281185535 >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 0x7fae70136a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.362 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:23.363 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 monclient: found mon.noname-b 2024-09-15T21:17:23.363 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:23.363 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:23.363 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 -- 172.21.15.17:0/4281185535 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae700a1c70 con 0x7fae70137a60 2024-09-15T21:17:23.363 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6ed9d640 1 --2- 172.21.15.17:0/4281185535 >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 0x7fae70136a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6dd9b640 1 --2- 172.21.15.17:0/4281185535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70137e40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae7006e580 tx=0x7fae6401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6437fa641d513bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 1 -- 172.21.15.17:0/4281185535 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640b0070 con 0x7fae70137a60 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: dump: 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:23.367 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: _finish_auth 0 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 1 -- 172.21.15.17:0/4281185535 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae640ab600 con 0x7fae70137a60 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 5 monclient: authenticate success, global_id 4151 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 1 -- 172.21.15.17:0/4281185535 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640ab9d0 con 0x7fae70137a60 2024-09-15T21:17:23.368 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 4 set_mon_vals no callback set 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae6d59a640 10 monclient: dump: 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:23.369 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 10 monclient: shutdown 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 -- 172.21.15.17:0/4281185535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 msgr2=0x7fae70137e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 --2- 172.21.15.17:0/4281185535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70137e40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae7006e580 tx=0x7fae6401c470 comp rx=0 tx=0).stop 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 -- 172.21.15.17:0/4281185535 shutdown_connections 2024-09-15T21:17:23.370 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 --2- 172.21.15.17:0/4281185535 >> v2:172.21.15.17:3300/0 conn(0x7fae70136600 0x7fae70136a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 --2- 172.21.15.17:0/4281185535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70137e40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 --2- 172.21.15.17:0/4281185535 >> v2:172.21.15.119:3300/0 conn(0x7fae70137680 0x7fae700a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 -- 172.21.15.17:0/4281185535 >> 172.21.15.17:0/4281185535 conn(0x7fae700a6750 msgr2=0x7fae700a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 -- 172.21.15.17:0/4281185535 shutdown_connections 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.350+0000 7fae74d83640 1 -- 172.21.15.17:0/4281185535 wait complete. 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: build_initial_monmap 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: monmap: 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:epoch 0 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:created 0.000000 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:23.371 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 Processor -- start 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- start start 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: init 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae70136600 0x7fae7011d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:23.372 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): picked mon.noname-b con 0x7fae70136600 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 0x7fae7011fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): picked mon.noname-a con 0x7fae70137680 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70135860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): picked mon.noname-c con 0x7fae70137a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fae701415a0 con 0x7fae70137680 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae70141420 con 0x7fae70136600 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fae70141720 con 0x7fae70137a60 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient(hunting): authenticate will time out at 966.752993s 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 0x7fae7011fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:23.373 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70135860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 0x7fae7011fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48254/0 (socket says 172.21.15.17:48254) 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 -- 172.21.15.17:0/1085656341 learned_addr learned my addr 172.21.15.17:0/1085656341 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): get_auth_request con 0x7fae70137680 auth_method 0 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): get_auth_request con 0x7fae70137a60 auth_method 0 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:23.374 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6e59c640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae70136600 0x7fae7011d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient(hunting): handle_auth_done global_id 4140 payload 824 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 -- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 msgr2=0x7fae70135860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70135860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 -- 172.21.15.17:0/1085656341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae70136600 msgr2=0x7fae7011d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae70136600 0x7fae7011d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.375 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 -- 172.21.15.17:0/1085656341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7011eea0 con 0x7fae70137680 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6ed9d640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6e59c640 10 monclient: get_auth_request con 0x7fae70136600 auth_method 0 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6e59c640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae70136600 0x7fae7011d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae6dd9b640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 0x7fae7011fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae640ae040 tx=0x7fae64019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640ba020 con 0x7fae70137680 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: dump: 2024-09-15T21:17:23.376 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: _finish_auth 0 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:23.377 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae640b6420 con 0x7fae70137680 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 5 monclient: authenticate success, global_id 4140 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _renew_subs 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7009c620 con 0x7fae70137680 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:23.378 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640b0040 con 0x7fae70137680 2024-09-15T21:17:23.379 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:23.379 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:23.379 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 10 monclient: dump: 2024-09-15T21:17:23.379 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:23.379 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:23.379 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.380 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:23.380 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:23.380 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:23.380 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:23.380 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:23.380 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:23.381 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:23.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _renew_subs 2024-09-15T21:17:23.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5ffff640 4 set_mon_vals no callback set 2024-09-15T21:17:23.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:23.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae70121200 con 0x7fae70137680 2024-09-15T21:17:23.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fae640bd020 con 0x7fae70137680 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fae640bfa90 con 0x7fae70137680 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.354+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae700a0cd0 con 0x7fae70137680 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.358+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fae640c4030 con 0x7fae70137680 2024-09-15T21:17:23.382 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.358+0000 7fae5effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:23.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.358+0000 7fae5effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:23.426 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:23.426 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:23.426 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:23.426 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae70120a20 con 0x7fae70137680 2024-09-15T21:17:23.426 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae5effd640 1 -- 172.21.15.17:0/1085656341 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7fae640160f0 con 0x7fae70137680 2024-09-15T21:17:23.427 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae5effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:23.427 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae5effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:23.427 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-15T21:17:23.427 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":8,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:22.374306+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:23.428 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 10 monclient: shutdown 2024-09-15T21:17:23.428 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:23.428 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 msgr2=0x7fae7011fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:23.428 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 0x7fae7011fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae640ae040 tx=0x7fae64019210 comp rx=0 tx=0).stop 2024-09-15T21:17:23.428 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 shutdown_connections 2024-09-15T21:17:23.429 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae70137a60 0x7fae70135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.429 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae70137680 0x7fae7011fa00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.429 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 --2- 172.21.15.17:0/1085656341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae70136600 0x7fae7011d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:23.429 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 >> 172.21.15.17:0/1085656341 conn(0x7fae700a6750 msgr2=0x7fae7009d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:23.429 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 shutdown_connections 2024-09-15T21:17:23.429 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:23.418+0000 7fae74d83640 1 -- 172.21.15.17:0/1085656341 wait complete. 2024-09-15T21:17:23.443 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-15T21:17:23.535 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-15T21:17:23.526+0000 7fe5cd8cf7c0 -1 Falling back to public interface 2024-09-15T21:17:23.555 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-15T21:17:23.552+0000 7f23aded67c0 -1 Falling back to public interface 2024-09-15T21:17:23.755 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-15T21:17:23.746+0000 7f62893cf7c0 -1 Falling back to public interface 2024-09-15T21:17:24.030 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-15T21:17:24.028+0000 7f06618ca7c0 -1 Falling back to public interface 2024-09-15T21:17:24.055 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-15T21:17:24.046+0000 7f3176c157c0 -1 Falling back to public interface 2024-09-15T21:17:26.175 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-15T21:17:26.172+0000 7f6da8dd8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:17:26.177 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-09-15T21:17:26.176+0000 7f6da8dd8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:17:27.350 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-15T21:17:27.348+0000 7f38882357c0 -1 osd.3 0 log_to_monitors true 2024-09-15T21:17:27.533 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-15T21:17:27.532+0000 7f23aded67c0 -1 osd.5 0 log_to_monitors true 2024-09-15T21:17:27.746 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-15T21:17:27.738+0000 7f62893cf7c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:17:27.856 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-15T21:17:27.846+0000 7fe5cd8cf7c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:17:28.034 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-15T21:17:28.032+0000 7f06618ca7c0 -1 osd.4 0 log_to_monitors true 2024-09-15T21:17:28.221 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-15T21:17:28.214+0000 7f3176c157c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:17:29.230 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-15T21:17:29.228+0000 7f38841de640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T21:17:29.230 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-15T21:17:29.222+0000 7f6285378640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T21:17:29.239 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-15T21:17:29.236+0000 7f065d873640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T21:17:29.253 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-15T21:17:29.242+0000 7f62811b2640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:29.263 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-09-15T21:17:29.260+0000 7f06596ad640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:29.263 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-15T21:17:29.260+0000 7f3880018640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:29.910 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-15T21:17:29.908+0000 7f23a9e7f640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T21:17:29.938 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-09-15T21:17:29.936+0000 7f23a1cb1640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:29.939 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-15T21:17:29.930+0000 7fe5c9878640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T21:17:29.962 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-15T21:17:29.954+0000 7fe5c16aa640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:30.049 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:30.170 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient: get_monmap_and_config 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient: build_initial_monmap 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient: monmap: 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:epoch 0 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:17:30.167661+0000 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:17:30.167661+0000 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:30.171 INFO:teuthology.misc.health.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:30.172 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:30.173 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 1 Processor -- start 2024-09-15T21:17:30.173 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 1 -- start start 2024-09-15T21:17:30.173 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient: init 2024-09-15T21:17:30.173 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f900009d310 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f900009de00 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f900013b150 0x7f900013d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient(hunting): picked mon.noname-a con 0x7f900013b150 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900009e7d0 con 0x7f900009de00 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900009eab0 con 0x7f900009d310 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f90000a1990 con 0x7f900013b150 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9007d21640 10 monclient(hunting): authenticate will time out at 973.563234s 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f900013b150 0x7f900013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:30.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90052bc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f900009de00 0x7f9000139a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42988/0 (socket says 172.21.15.17:42988) 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90052bc640 1 -- 172.21.15.17:0/879958330 learned_addr learned my addr 172.21.15.17:0/879958330 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): get_auth_request con 0x7f900013b150 auth_method 0 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient(hunting): handle_auth_done global_id 4164 payload 274 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 1 -- 172.21.15.17:0/879958330 >> v1:172.21.15.119:6789/0 conn(0x7f900009d310 legacy=0x7f900009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 1 -- 172.21.15.17:0/879958330 >> v1:172.21.15.17:6789/0 conn(0x7f900009de00 legacy=0x7f9000139a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient: _finish_hunting 0 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 1 monclient: found mon.noname-a 2024-09-15T21:17:30.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:30.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:30.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f90062be640 1 -- 172.21.15.17:0/879958330 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900009e230 con 0x7f900013b150 2024-09-15T21:17:30.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9004abb640 1 -- 172.21.15.17:0/879958330 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142819741 0 0) 0x7f900009e7d0 con 0x7f900009de00 2024-09-15T21:17:30.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.162+0000 7f9004abb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:30.180 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f90062be640 1 --2- 172.21.15.17:0/879958330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900013b150 0x7f900013d540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009d90 tx=0x7f8ff801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554f626a48505064 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:30.180 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 1 -- 172.21.15.17:0/879958330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80b2070 con 0x7f900013b150 2024-09-15T21:17:30.180 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:30.180 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:30.180 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: dump: 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: _finish_auth 0 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:30.181 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 1 -- 172.21.15.17:0/879958330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff80a5800 con 0x7f900013b150 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 1 -- 172.21.15.17:0/879958330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80a5bd0 con 0x7f900013b150 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 5 monclient: authenticate success, global_id 4164 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 4 set_mon_vals no callback set 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9004abb640 10 monclient: dump: 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:30.182 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 10 monclient: shutdown 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 -- 172.21.15.17:0/879958330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900013b150 msgr2=0x7f900013d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 --2- 172.21.15.17:0/879958330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900013b150 0x7f900013d540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009d90 tx=0x7f8ff801c830 comp rx=0 tx=0).stop 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 -- 172.21.15.17:0/879958330 shutdown_connections 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 --2- 172.21.15.17:0/879958330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900013b150 0x7f900013d540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 -- 172.21.15.17:0/879958330 >> 172.21.15.17:0/879958330 conn(0x7f90000a6750 msgr2=0x7f90000a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 -- 172.21.15.17:0/879958330 shutdown_connections 2024-09-15T21:17:30.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 -- 172.21.15.17:0/879958330 wait complete. 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 10 monclient: build_initial_monmap 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 10 monclient: monmap: 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:epoch 0 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:created 0.000000 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 Processor -- start 2024-09-15T21:17:30.184 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 1 -- start start 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.166+0000 7f9007d21640 10 monclient: init 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f900009d310 0x7f9000133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): picked mon.noname-b con 0x7f900009d310 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 0x7f90001344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): picked mon.noname-a con 0x7f900009de00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f900013b150 0x7f9000071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): picked mon.noname-c con 0x7f900013b150 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9000140830 con 0x7f900009de00 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f90001406b0 con 0x7f900009d310 2024-09-15T21:17:30.185 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90001409b0 con 0x7f900013b150 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient(hunting): authenticate will time out at 973.568379s 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 0x7f90001344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 0x7f90001344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47830/0 (socket says 172.21.15.17:47830) 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f900013b150 0x7f9000071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 -- 172.21.15.17:0/4180383437 learned_addr learned my addr 172.21.15.17:0/4180383437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): get_auth_request con 0x7f900009de00 auth_method 0 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): get_auth_request con 0x7f900013b150 auth_method 0 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:30.186 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f900009d310 0x7f9000133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): get_auth_request con 0x7f900009d310 auth_method 0 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:30.187 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient(hunting): handle_auth_done global_id 4170 payload 824 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 -- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f900013b150 msgr2=0x7f9000071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f900013b150 0x7f9000071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 -- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f900009d310 msgr2=0x7f9000133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f900009d310 0x7f9000133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient: _finish_hunting 0 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 monclient: found mon.noname-a 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 -- 172.21.15.17:0/4180383437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900011ca20 con 0x7f900009de00 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90062be640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f900013b150 0x7f9000071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:30.188 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f90052bc640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 0x7f90001344b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c480 tx=0x7f8ff000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0018070 con 0x7f900009de00 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: dump: 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:30.189 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff00074e0 con 0x7f900009de00 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0004a30 con 0x7f900009de00 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 10 monclient: dump: 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:30.190 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 5 monclient: authenticate success, global_id 4170 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _renew_subs 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff7fff640 4 set_mon_vals no callback set 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900011dc70 con 0x7f900009de00 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _renew_subs 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900009d100 con 0x7f900009de00 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:30.191 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900009d6f0 con 0x7f900009de00 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) ==== 48811+0+0 (secure 0 0 0) 0x7f8ff0004bd0 con 0x7f900009de00 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fe403b930 0x7f8fe403ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1911+0+0 (secure 0 0 0) 0x7f8ff004e1d0 con 0x7f900009de00 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fe403b930 0x7f8fe403ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:30.192 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 10 monclient: get_auth_request con 0x7f8fe403b930 auth_method 0 2024-09-15T21:17:30.193 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.170+0000 7f9005abd640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fe403b930 0x7f8fe403ddf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc007fd0 tx=0x7f8ffc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:30.193 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.174+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8ff001bcf0 con 0x7f900009de00 2024-09-15T21:17:30.193 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.178+0000 7f8ff67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:30.193 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.178+0000 7f8ff67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:30.380 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f9007d21640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:30.380 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f9007d21640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:30.380 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f9007d21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:30.380 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f900011d860 con 0x7f900009de00 2024-09-15T21:17:30.380 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f8ff67fc640 1 -- 172.21.15.17:0/4180383437 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+8011 (secure 0 0 0) 0x7f8ff000e280 con 0x7f900009de00 2024-09-15T21:17:30.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f8ff67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:30.381 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.370+0000 7f8ff67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:30.381 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-15T21:17:30.381 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":11,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:30.217174+0000","last_up_change":"2024-09-15T21:17:30.217174+0000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6809","nonce":3272576910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6811","nonce":3272576910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6815","nonce":3272576910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6813","nonce":3272576910}]},"public_addr":"172.21.15.17:6809/3272576910","cluster_addr":"172.21.15.17:6811/3272576910","heartbeat_back_addr":"172.21.15.17:6815/3272576910","heartbeat_front_addr":"172.21.15.17:6813/3272576910","state":["exists","up"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6801","nonce":671096025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6803","nonce":671096025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6807","nonce":671096025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6805","nonce":671096025}]},"public_addr":"172.21.15.17:6801/671096025","cluster_addr":"172.21.15.17:6803/671096025","heartbeat_back_addr":"172.21.15.17:6807/671096025","heartbeat_front_addr":"172.21.15.17:6805/671096025","state":["exists","up"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6801","nonce":3353277098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6803","nonce":3353277098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6807","nonce":3353277098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6805","nonce":3353277098}]},"public_addr":"172.21.15.119:6801/3353277098","cluster_addr":"172.21.15.119:6803/3353277098","heartbeat_back_addr":"172.21.15.119:6807/3353277098","heartbeat_front_addr":"172.21.15.119:6805/3353277098","state":["exists","up"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6817","nonce":2086984234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6819","nonce":2086984234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6823","nonce":2086984234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6821","nonce":2086984234}]},"public_addr":"172.21.15.119:6817/2086984234","cluster_addr":"172.21.15.119:6819/2086984234","heartbeat_back_addr":"172.21.15.119:6823/2086984234","heartbeat_front_addr":"172.21.15.119:6821/2086984234","state":["exists","up"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6809","nonce":444741843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6811","nonce":444741843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6815","nonce":444741843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6813","nonce":444741843}]},"public_addr":"172.21.15.119:6809/444741843","cluster_addr":"172.21.15.119:6811/444741843","heartbeat_back_addr":"172.21.15.119:6815/444741843","heartbeat_front_addr":"172.21.15.119:6813/444741843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fe403b930 msgr2=0x7f8fe403ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fe403b930 0x7f8fe403ddf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc007fd0 tx=0x7f8ffc009040 comp rx=0 tx=0).stop 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 10 monclient: shutdown 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 msgr2=0x7f90001344b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 0x7f90001344b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c480 tx=0x7f8ff000c940 comp rx=0 tx=0).stop 2024-09-15T21:17:30.383 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 shutdown_connections 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fe403b930 0x7f8fe403ddf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f900013b150 0x7f9000071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f900009de00 0x7f90001344b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 --2- 172.21.15.17:0/4180383437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f900009d310 0x7f9000133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 >> 172.21.15.17:0/4180383437 conn(0x7f90000a6750 msgr2=0x7f900013b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 shutdown_connections 2024-09-15T21:17:30.384 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:30.374+0000 7f9007d21640 1 -- 172.21.15.17:0/4180383437 wait complete. 2024-09-15T21:17:30.402 DEBUG:teuthology.misc:5 of 6 OSDs are up 2024-09-15T21:17:30.945 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-15T21:17:30.938+0000 7f31733d0640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T21:17:30.978 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-15T21:17:30.970+0000 7f316a9f0640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:37.008 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:37.132 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.122+0000 7f1289b27640 10 monclient: get_monmap_and_config 2024-09-15T21:17:37.132 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.122+0000 7f1289b27640 10 monclient: build_initial_monmap 2024-09-15T21:17:37.132 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.122+0000 7f1289b27640 10 monclient: monmap: 2024-09-15T21:17:37.132 INFO:teuthology.misc.health.smithi017.stderr:epoch 0 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:17:37.129278+0000 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:17:37.129278+0000 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:37.133 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:37.135 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 Processor -- start 2024-09-15T21:17:37.135 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 -- start start 2024-09-15T21:17:37.135 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient: init 2024-09-15T21:17:37.135 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:37.135 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f12840a1ed0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12840a29c0 0x7f128409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): picked mon.noname-a con 0x7f12840a29c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f128413aa90 0x7f128413ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): picked mon.noname-c con 0x7f128413aa90 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128409e260 con 0x7f12840a1ed0 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f128409c6d0 con 0x7f12840a29c0 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f128409dd30 con 0x7f128413aa90 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1289b27640 10 monclient(hunting): authenticate will time out at 980.525559s 2024-09-15T21:17:37.136 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12840a29c0 0x7f128409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12840a29c0 0x7f128409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47846/0 (socket says 172.21.15.17:47846) 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 -- 172.21.15.17:0/1502777059 learned_addr learned my addr 172.21.15.17:0/1502777059 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): get_auth_request con 0x7f12840a29c0 auth_method 0 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1283fff640 1 --2- 172.21.15.17:0/1502777059 >> v2:172.21.15.119:3300/0 conn(0x7f128413aa90 0x7f128413ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1283fff640 10 monclient(hunting): get_auth_request con 0x7f128413aa90 auth_method 0 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1283fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.137 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1283fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f1283fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient(hunting): handle_auth_done global_id 4182 payload 274 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 -- 172.21.15.17:0/1502777059 >> v2:172.21.15.119:3300/0 conn(0x7f128413aa90 msgr2=0x7f128413ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 --2- 172.21.15.17:0/1502777059 >> v2:172.21.15.119:3300/0 conn(0x7f128413aa90 0x7f128413ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 -- 172.21.15.17:0/1502777059 >> v1:172.21.15.119:6789/0 conn(0x7f12840a1ed0 legacy=0x7f12840a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 monclient: found mon.noname-a 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.138 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.126+0000 7f127bfff640 1 -- 172.21.15.17:0/1502777059 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1284068a40 con 0x7f12840a29c0 2024-09-15T21:17:37.142 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 1 --2- 172.21.15.17:0/1502777059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a29c0 0x7f128409ec20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f128409e710 tx=0x7f126c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f321c36e908c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 1 -- 172.21.15.17:0/1502777059 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c0b0070 con 0x7f12840a29c0 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: dump: 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.143 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 1 -- 172.21.15.17:0/1502777059 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f126c0ab600 con 0x7f12840a29c0 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:37.144 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 1 -- 172.21.15.17:0/1502777059 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c0ab9d0 con 0x7f12840a29c0 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 5 monclient: authenticate success, global_id 4182 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 4 set_mon_vals no callback set 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1282ffd640 10 monclient: dump: 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.145 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: shutdown 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- 172.21.15.17:0/1502777059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a29c0 msgr2=0x7f128409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 --2- 172.21.15.17:0/1502777059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a29c0 0x7f128409ec20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f128409e710 tx=0x7f126c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- 172.21.15.17:0/1502777059 shutdown_connections 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 --2- 172.21.15.17:0/1502777059 >> v2:172.21.15.119:3300/0 conn(0x7f128413aa90 0x7f128413ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 --2- 172.21.15.17:0/1502777059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a29c0 0x7f128409ec20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- 172.21.15.17:0/1502777059 >> 172.21.15.17:0/1502777059 conn(0x7f12840a2fc0 msgr2=0x7f12840a33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- 172.21.15.17:0/1502777059 shutdown_connections 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- 172.21.15.17:0/1502777059 wait complete. 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: build_initial_monmap 2024-09-15T21:17:37.146 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: monmap: 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:epoch 0 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:created 0.000000 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 Processor -- start 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- start start 2024-09-15T21:17:37.147 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: init 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 0x7f12841cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): picked mon.noname-a con 0x7f12840a1ed0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f12840a29c0 0x7f12841cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): picked mon.noname-c con 0x7f12840a29c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f128413aa90 0x7f12841d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): picked mon.noname-b con 0x7f128413aa90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1284140030 con 0x7f12840a1ed0 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f128413feb0 con 0x7f128413aa90 2024-09-15T21:17:37.148 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f12841401b0 con 0x7f12840a29c0 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1289b27640 10 monclient(hunting): authenticate will time out at 980.531515s 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 0x7f12841cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 0x7f12841cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47856/0 (socket says 172.21.15.17:47856) 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f12840a29c0 0x7f12841cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 1 -- 172.21.15.17:0/589537848 learned_addr learned my addr 172.21.15.17:0/589537848 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): get_auth_request con 0x7f12840a1ed0 auth_method 0 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): get_auth_request con 0x7f12840a29c0 auth_method 0 2024-09-15T21:17:37.149 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f127bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1283fff640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f128413aa90 0x7f12841d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1283fff640 10 monclient(hunting): get_auth_request con 0x7f128413aa90 auth_method 0 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1283fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1283fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.150 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f1283fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 10 monclient(hunting): handle_auth_done global_id 4185 payload 824 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.130+0000 7f12837fe640 1 -- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f12840a29c0 msgr2=0x7f12841cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f12840a29c0 0x7f12841cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 1 -- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f128413aa90 msgr2=0x7f12841d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f128413aa90 0x7f12841d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 1 monclient: found mon.noname-a 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 1 -- 172.21.15.17:0/589537848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128412f750 con 0x7f12840a1ed0 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12837fe640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 0x7f12841cf770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f127400c450 tx=0x7f127400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274012830 con 0x7f12840a1ed0 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.151 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: dump: 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:37.152 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: _finish_auth 0 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1274017070 con 0x7f12840a1ed0 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274012c30 con 0x7f12840a1ed0 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 10 monclient: dump: 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:epoch 1 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.153 INFO:teuthology.misc.health.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr: 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f12827fc640 4 set_mon_vals no callback set 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 5 monclient: authenticate success, global_id 4185 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 monclient: _renew_subs 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128412eeb0 con 0x7f12840a1ed0 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 monclient: _renew_subs 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128413a380 con 0x7f12840a1ed0 2024-09-15T21:17:37.154 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:37.155 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:37.155 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.155 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128409ec20 con 0x7f12840a1ed0 2024-09-15T21:17:37.155 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f127401c460 con 0x7f12840a1ed0 2024-09-15T21:17:37.155 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f126403b9c0 0x7f126403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.155 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3993+0+0 (secure 0 0 0) 0x7f127404f540 con 0x7f12840a1ed0 2024-09-15T21:17:37.156 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f127bfff640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f126403b9c0 0x7f126403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.156 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f127bfff640 10 monclient: get_auth_request con 0x7f126403b9c0 auth_method 0 2024-09-15T21:17:37.156 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.134+0000 7f127bfff640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f126403b9c0 0x7f126403de80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f126c0ad000 tx=0x7f126c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:37.156 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.138+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f127401b3c0 con 0x7f12840a1ed0 2024-09-15T21:17:37.156 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.138+0000 7f1280ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:37.156 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.138+0000 7f1280ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:37.322 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1289b27640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:37.322 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1289b27640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:37.322 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1289b27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.322 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f128409d6b0 con 0x7f12840a1ed0 2024-09-15T21:17:37.323 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1280ff9640 1 -- 172.21.15.17:0/589537848 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10485 (secure 0 0 0) 0x7f127401ac80 con 0x7f12840a1ed0 2024-09-15T21:17:37.323 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1280ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:37.323 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.314+0000 7f1280ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:37.323 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-15T21:17:37.324 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":14,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:33.253682+0000","last_up_change":"2024-09-15T21:17:31.232299+0000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:17:31.217943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6809","nonce":3272576910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6811","nonce":3272576910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6815","nonce":3272576910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6813","nonce":3272576910}]},"public_addr":"172.21.15.17:6809/3272576910","cluster_addr":"172.21.15.17:6811/3272576910","heartbeat_back_addr":"172.21.15.17:6815/3272576910","heartbeat_front_addr":"172.21.15.17:6813/3272576910","state":["exists","up"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6817","nonce":2555706928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6819","nonce":2555706928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6823","nonce":2555706928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6821","nonce":2555706928}]},"public_addr":"172.21.15.17:6817/2555706928","cluster_addr":"172.21.15.17:6819/2555706928","heartbeat_back_addr":"172.21.15.17:6823/2555706928","heartbeat_front_addr":"172.21.15.17:6821/2555706928","state":["exists","up"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6801","nonce":671096025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6803","nonce":671096025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6807","nonce":671096025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6805","nonce":671096025}]},"public_addr":"172.21.15.17:6801/671096025","cluster_addr":"172.21.15.17:6803/671096025","heartbeat_back_addr":"172.21.15.17:6807/671096025","heartbeat_front_addr":"172.21.15.17:6805/671096025","state":["exists","up"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6801","nonce":3353277098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6803","nonce":3353277098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6807","nonce":3353277098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6805","nonce":3353277098}]},"public_addr":"172.21.15.119:6801/3353277098","cluster_addr":"172.21.15.119:6803/3353277098","heartbeat_back_addr":"172.21.15.119:6807/3353277098","heartbeat_front_addr":"172.21.15.119:6805/3353277098","state":["exists","up"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6817","nonce":2086984234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6819","nonce":2086984234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6823","nonce":2086984234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6821","nonce":2086984234}]},"public_addr":"172.21.15.119:6817/2086984234","cluster_addr":"172.21.15.119:6819/2086984234","heartbeat_back_addr":"172.21.15.119:6823/2086984234","heartbeat_front_addr":"172.21.15.119:6821/2086984234","state":["exists","up"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6809","nonce":444741843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6811","nonce":444741843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6815","nonce":444741843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6813","nonce":444741843}]},"public_addr":"172.21.15.119:6809/444741843","cluster_addr":"172.21.15.119:6811/444741843","heartbeat_back_addr":"172.21.15.119:6815/444741843","heartbeat_front_addr":"172.21.15.119:6813/444741843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.768611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.237742+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.809091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.353555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.041478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.487022+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f126403b9c0 msgr2=0x7f126403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f126403b9c0 0x7f126403de80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f126c0ad000 tx=0x7f126c008040 comp rx=0 tx=0).stop 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 10 monclient: shutdown 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 msgr2=0x7f12841cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 0x7f12841cf770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f127400c450 tx=0x7f127400c910 comp rx=0 tx=0).stop 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 shutdown_connections 2024-09-15T21:17:37.326 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f126403b9c0 0x7f126403de80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.327 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f128413aa90 0x7f12841d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.327 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f12840a29c0 0x7f12841cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.327 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 --2- 172.21.15.17:0/589537848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12840a1ed0 0x7f12841cf770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.327 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 >> 172.21.15.17:0/589537848 conn(0x7f12840a2fc0 msgr2=0x7f1284074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:37.327 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 shutdown_connections 2024-09-15T21:17:37.327 INFO:teuthology.misc.health.smithi017.stderr:2024-09-15T21:17:37.318+0000 7f1289b27640 1 -- 172.21.15.17:0/589537848 wait complete. 2024-09-15T21:17:37.340 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-15T21:17:37.340 INFO:tasks.ceph:Creating RBD pool 2024-09-15T21:17:37.340 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.454+0000 7f470f11a640 10 monclient: get_monmap_and_config 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.454+0000 7f470f11a640 10 monclient: build_initial_monmap 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient: monmap: 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:37.462200+0000 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:37.462200+0000 2024-09-15T21:17:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:37.466 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 Processor -- start 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 -- start start 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient: init 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47080a0890 0x7f47080a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): picked mon.noname-b con 0x7f47080a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:37.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f470809ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47080a11b0 0x7f4708135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): picked mon.noname-a con 0x7f47080a11b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4708073df0 con 0x7f470809ffc0 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f47080a3650 con 0x7f47080a11b0 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f470806f2a0 con 0x7f47080a0890 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470f11a640 10 monclient(hunting): authenticate will time out at 980.857556s 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47080a0890 0x7f47080a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47080a11b0 0x7f4708135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f4707fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f470809ffc0 0x7f4708135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42994/0 (socket says 172.21.15.17:42994) 2024-09-15T21:17:37.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f4707fff640 1 -- 172.21.15.17:0/2257116240 learned_addr learned my addr 172.21.15.17:0/2257116240 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): get_auth_request con 0x7f47080a0890 auth_method 0 2024-09-15T21:17:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): get_auth_request con 0x7f47080a11b0 auth_method 0 2024-09-15T21:17:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient(hunting): handle_auth_done global_id 4188 payload 274 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 1 -- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47080a0890 msgr2=0x7f47080a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 1 --2- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47080a0890 0x7f47080a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 1 -- 172.21.15.17:0/2257116240 >> v1:172.21.15.17:6789/0 conn(0x7f470809ffc0 legacy=0x7f4708135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient: _finish_hunting 0 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 1 monclient: found mon.noname-a 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f47077fe640 1 -- 172.21.15.17:0/2257116240 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389464773 0 0) 0x7f4708073df0 con 0x7f470809ffc0 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470d6b7640 1 -- 172.21.15.17:0/2257116240 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080a2490 con 0x7f47080a11b0 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f470ceb6640 1 --2- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47080a0890 0x7f47080a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:37.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.458+0000 7f47077fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 --2- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f4708135d80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f470000f370 tx=0x7f4700020c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f25044eb1fe3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 1 -- 172.21.15.17:0/2257116240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000d0d0 con 0x7f47080a11b0 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: dump: 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 1 -- 172.21.15.17:0/2257116240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000d270 con 0x7f47080a11b0 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 1 -- 172.21.15.17:0/2257116240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47000a9900 con 0x7f47080a11b0 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 5 monclient: authenticate success, global_id 4188 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 4 set_mon_vals no callback set 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47077fe640 10 monclient: dump: 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.479 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: shutdown 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 msgr2=0x7f4708135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 --2- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f4708135d80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f470000f370 tx=0x7f4700020c30 comp rx=0 tx=0).stop 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2257116240 shutdown_connections 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 --2- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f4708135d80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 --2- 172.21.15.17:0/2257116240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47080a0890 0x7f47080a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2257116240 >> 172.21.15.17:0/2257116240 conn(0x7f47080a6690 msgr2=0x7f47080a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2257116240 shutdown_connections 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2257116240 wait complete. 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: build_initial_monmap 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: monmap: 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 Processor -- start 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- start start 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: init 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 0x7f470812e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): picked mon.noname-c con 0x7f470809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47080a0890 0x7f470812eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): picked mon.noname-b con 0x7f47080a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f470812f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): picked mon.noname-a con 0x7f47080a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f470809f830 con 0x7f47080a11b0 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47081364d0 con 0x7f47080a0890 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f470813e2f0 con 0x7f470809ffc0 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient(hunting): authenticate will time out at 980.862544s 2024-09-15T21:17:37.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 0x7f470812e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f470812f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f470812f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47868/0 (socket says 172.21.15.17:47868) 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 0x7f470812e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56454/0 (socket says 172.21.15.17:56454) 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 -- 172.21.15.17:0/2578674159 learned_addr learned my addr 172.21.15.17:0/2578674159 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): get_auth_request con 0x7f47080a11b0 auth_method 0 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): get_auth_request con 0x7f470809ffc0 auth_method 0 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:37.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f4707fff640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47080a0890 0x7f470812eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient(hunting): handle_auth_done global_id 4194 payload 824 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 -- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 msgr2=0x7f470812e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 0x7f470812e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 -- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47080a0890 msgr2=0x7f470812eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47080a0890 0x7f470812eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient: _finish_hunting 0 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 monclient: found mon.noname-a 2024-09-15T21:17:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 -- 172.21.15.17:0/2578674159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4708139470 con 0x7f47080a11b0 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470ceb6640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 0x7f470812e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470d6b7640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f470812f080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47000b3040 tx=0x7f470001d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47000c0020 con 0x7f47080a11b0 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: dump: 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: _finish_auth 0 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: _check_auth_tickets 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000b940 con 0x7f47080a11b0 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000d860 con 0x7f47080a11b0 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 5 monclient: authenticate success, global_id 4194 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: _renew_subs 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470806f420 con 0x7f47080a11b0 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f47057fa640 10 monclient: dump: 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f4706ffd640 4 set_mon_vals no callback set 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: _renew_subs 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.462+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4708135920 con 0x7f47080a11b0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470f11a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470f11a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470f11a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47080a0c70 con 0x7f47080a11b0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f47000c3020 con 0x7f47080a11b0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f47057fa640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46f403b9c0 0x7f46f403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3993+0+0 (secure 0 0 0) 0x7f47000ee9a0 con 0x7f47080a11b0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470ceb6640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46f403b9c0 0x7f46f403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470ceb6640 10 monclient: get_auth_request con 0x7f46f403b9c0 auth_method 0 2024-09-15T21:17:37.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.466+0000 7f470ceb6640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46f403b9c0 0x7f46f403de80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009580 tx=0x7f46fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:37.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.470+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f47000ca030 con 0x7f47080a11b0 2024-09-15T21:17:37.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.470+0000 7f47057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:37.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.470+0000 7f47057fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:37.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.642+0000 7f470f11a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T21:17:37.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.642+0000 7f470f11a640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T21:17:37.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.642+0000 7f470f11a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:37.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:37.642+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f47080a5240 con 0x7f47080a11b0 2024-09-15T21:17:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.274+0000 7f47057fa640 1 -- 172.21.15.17:0/2578674159 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) ==== 109+0+0 (secure 0 0 0) 0x7f470001a940 con 0x7f47080a11b0 2024-09-15T21:17:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.274+0000 7f47057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T21:17:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.274+0000 7f47057fa640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-09-15T21:17:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'rbd' created 2024-09-15T21:17:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46f403b9c0 msgr2=0x7f46f403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46f403b9c0 0x7f46f403de80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009580 tx=0x7f46fc009190 comp rx=0 tx=0).stop 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 10 monclient: shutdown 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 msgr2=0x7f470812f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f470812f080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47000b3040 tx=0x7f470001d9d0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 shutdown_connections 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46f403b9c0 0x7f46f403de80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47080a11b0 0x7f470812f080 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47080a0890 0x7f470812eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 --2- 172.21.15.17:0/2578674159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f470809ffc0 0x7f470812e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 >> 172.21.15.17:0/2578674159 conn(0x7f47080a6690 msgr2=0x7f470806faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 shutdown_connections 2024-09-15T21:17:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.278+0000 7f470f11a640 1 -- 172.21.15.17:0/2578674159 wait complete. 2024-09-15T21:17:38.308 DEBUG:teuthology.orchestra.run.smithi017:> rbd --cluster ceph pool init rbd 2024-09-15T21:17:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient: get_monmap_and_config 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient: build_initial_monmap 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient: monmap: 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:38.355901+0000 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:38.355901+0000 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:38.359 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:38.360 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:38.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 Processor -- start 2024-09-15T21:17:38.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 -- start start 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient: init 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient: _reopen_session rank -1 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x5588214c6710 0x55882155ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): picked mon.noname-b con 0x5588214c6710 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 --2- >> v2:172.21.15.17:3300/0 conn(0x5588214c6af0 0x5588214c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): picked mon.noname-a con 0x5588214c6af0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): picked mon.noname-c-legacy con 0x558821557510 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x558821310c80 con 0x558821557510 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x558821392160 con 0x5588214c6af0 2024-09-15T21:17:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x5588213a2d30 con 0x5588214c6710 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): _renew_subs 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2553381100 10 monclient(hunting): authenticate will time out at 981.751228s 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x5588214c6af0 0x5588214c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x5588214c6af0 0x5588214c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43996/0 (socket says 172.21.15.17:43996) 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 -- 172.21.15.17:0/1840844888 learned_addr learned my addr 172.21.15.17:0/1840844888 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): get_auth_request con 0x5588214c6af0 auth_method 0 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 1 --2- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x5588214c6710 0x55882155ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): get_auth_request con 0x5588214c6710 auth_method 0 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:38.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient(hunting): handle_auth_done global_id 4197 payload 274 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 -- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x5588214c6710 msgr2=0x55882155ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 --2- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x5588214c6710 0x55882155ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 -- 172.21.15.17:0/1840844888 >> v1:172.21.15.119:6789/0 conn(0x558821557510 legacy=0x558821557960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient: _finish_hunting 0 2024-09-15T21:17:38.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 monclient: found mon.noname-a 2024-09-15T21:17:38.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:38.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:38.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.350+0000 7f2551689640 1 -- 172.21.15.17:0/1840844888 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588214bf670 con 0x5588214c6af0 2024-09-15T21:17:38.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2551689640 1 --2- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x5588214c6ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f253c00a5c0 tx=0x7f253c01b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e79a1a60ce66f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:38.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 1 -- 172.21.15.17:0/1840844888 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0af070 con 0x5588214c6af0 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: dump: 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:38.370 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: _finish_auth 0 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: _check_auth_tickets 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 1 -- 172.21.15.17:0/1840844888 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c0aa600 con 0x5588214c6af0 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 5 monclient: authenticate success, global_id 4197 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 10 monclient: get_monmap_and_config success 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 4 set_mon_vals no callback set 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 1 -- 172.21.15.17:0/1840844888 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0aa9d0 con 0x5588214c6af0 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2550e88640 10 monclient: dump: 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 10 monclient: shutdown 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 -- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 msgr2=0x5588214c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 --2- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x5588214c6ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f253c00a5c0 tx=0x7f253c01b330 comp rx=0 tx=0).stop 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 -- 172.21.15.17:0/1840844888 shutdown_connections 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 --2- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x5588214c6ed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 --2- 172.21.15.17:0/1840844888 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x5588214c6710 0x55882155ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 -- 172.21.15.17:0/1840844888 >> 172.21.15.17:0/1840844888 conn(0x5588214cb580 msgr2=0x5588214a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 -- 172.21.15.17:0/1840844888 shutdown_connections 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.354+0000 7f2553381100 1 -- 172.21.15.17:0/1840844888 wait complete. 2024-09-15T21:17:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: build_initial_monmap 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: monmap: 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:38.374 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 Processor -- start 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 -- start start 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: init 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: _reopen_session rank -1 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588214c6710 0x55882156c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): picked mon.noname-b con 0x5588214c6710 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x55882156cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): picked mon.noname-a con 0x5588214c6af0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x558821557510 0x55882156d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): picked mon.noname-c con 0x558821557510 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x558821337a70 con 0x5588214c6af0 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558821395f70 con 0x5588214c6710 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x55882155b0b0 con 0x558821557510 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): _renew_subs 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient(hunting): authenticate will time out at 981.757476s 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x55882156cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x558821557510 0x55882156d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x55882156cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44012/0 (socket says 172.21.15.17:44012) 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 -- 172.21.15.17:0/375139469 learned_addr learned my addr 172.21.15.17:0/375139469 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): get_auth_request con 0x5588214c6af0 auth_method 0 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): get_auth_request con 0x558821557510 auth_method 0 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:38.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f25530ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551e8a640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588214c6710 0x55882156c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient(hunting): handle_auth_done global_id 4200 payload 824 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 -- 172.21.15.17:0/375139469 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x558821557510 msgr2=0x55882156d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x558821557510 0x55882156d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 -- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588214c6710 msgr2=0x55882156c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588214c6710 0x55882156c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient: _finish_hunting 0 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 monclient: found mon.noname-a 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:38.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588215617a0 con 0x5588214c6af0 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551e8a640 10 monclient: get_auth_request con 0x5588214c6710 auth_method 0 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551e8a640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588214c6710 0x55882156c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2551689640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x55882156cec0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f253c0ad040 tx=0x7f253c0180d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0ba020 con 0x5588214c6af0 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: dump: 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c0b5470 con 0x5588214c6af0 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 5 monclient: authenticate success, global_id 4200 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: _renew_subs 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558821561a80 con 0x5588214c6af0 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0af040 con 0x5588214c6af0 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 10 monclient: dump: 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:38.381 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2543fff640 4 set_mon_vals no callback set 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: _renew_subs 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558821560560 con 0x5588214c6af0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f253c0bd020 con 0x5588214c6af0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f253003a2b0 0x7f253003c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.358+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4335+0+0 (secure 0 0 0) 0x7f253c0e9840 con 0x5588214c6af0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2553381100 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2553381100 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2553381100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x5588215608a0 con 0x5588214c6af0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2551e8a640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f253003a2b0 0x7f253003c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2551e8a640 10 monclient: get_auth_request con 0x7f253003a2b0 auth_method 0 2024-09-15T21:17:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:38.362+0000 7f2551e8a640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f253003a2b0 0x7f253003c770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2548005fd0 tx=0x7f2548005a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.282+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v16) ==== 141+0+0 (secure 0 0 0) 0x7f253c0c4030 con 0x5588214c6af0 2024-09-15T21:17:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.282+0000 7f2541ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T21:17:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.282+0000 7f2541ffb640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-09-15T21:17:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.282+0000 7f2553381100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.282+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f253c0aa600 con 0x5588214c6af0 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f253c0b6b30 con 0x5588214c6af0 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2541ffb640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2543fff640 10 monclient: _renew_subs 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2543fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2543fff640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=16}) -- 0x7f252c002af0 con 0x5588214c6af0 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 8 ==== osd_map(16..16 src has 1..16) ==== 658+0+0 (secure 0 0 0) 0x7f253c0b4dc0 con 0x5588214c6af0 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2553381100 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x558821557110 0x558821575f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) -- 0x558821576470 con 0x558821557110 2024-09-15T21:17:39.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f25530ec640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x558821557110 0x558821575f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:39.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.286+0000 7f25530ec640 10 monclient: get_auth_request con 0x558821557110 auth_method 0 2024-09-15T21:17:39.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.290+0000 7f25530ec640 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x558821557110 0x558821575f30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:39.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.302+0000 7f25530ec640 1 -- 172.21.15.17:0/375139469 <== osd.5 v2:172.21.15.119:6808/444741843 1 ==== osd_op_reply(1 rbd_trash [create] v16'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f2544002620 con 0x558821557110 2024-09-15T21:17:39.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.306+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) -- 0x5588215c90a0 con 0x558821557110 2024-09-15T21:17:39.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.306+0000 7f25530ec640 1 -- 172.21.15.17:0/375139469 <== osd.5 v2:172.21.15.119:6808/444741843 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2544002620 con 0x558821557110 2024-09-15T21:17:39.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.306+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) -- 0x5588215c81b0 con 0x558821557110 2024-09-15T21:17:39.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.306+0000 7f25530ec640 1 -- 172.21.15.17:0/375139469 <== osd.5 v2:172.21.15.119:6808/444741843 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2544002620 con 0x558821557110 2024-09-15T21:17:39.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.306+0000 7f25437fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:39.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:39.306+0000 7f25437fe640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f2524001f10 con 0x5588214c6af0 2024-09-15T21:17:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.294+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v17) ==== 55+0+0 (secure 0 0 0) 0x7f253c015040 con 0x5588214c6af0 2024-09-15T21:17:40.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.294+0000 7f2541ffb640 10 monclient: _renew_subs 2024-09-15T21:17:40.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.294+0000 7f2541ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:40.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.294+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=17}) -- 0x7f2530040510 con 0x5588214c6af0 2024-09-15T21:17:40.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.294+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 10 ==== osd_map(17..17 src has 1..17) ==== 626+0+0 (secure 0 0 0) 0x7f253c0b67a0 con 0x5588214c6af0 2024-09-15T21:17:40.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.294+0000 7f2543fff640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) -- 0x7f252c0057a0 con 0x558821557110 2024-09-15T21:17:40.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.302+0000 7f25530ec640 1 -- 172.21.15.17:0/375139469 <== osd.5 v2:172.21.15.119:6808/444741843 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v17'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f2544002620 con 0x558821557110 2024-09-15T21:17:40.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.302+0000 7f25437fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:40.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:40.302+0000 7f25437fe640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v17) -- 0x7f2524003360 con 0x5588214c6af0 2024-09-15T21:17:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.306+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v18) ==== 43+0+0 (secure 0 0 0) 0x7f253c0be3b0 con 0x5588214c6af0 2024-09-15T21:17:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.306+0000 7f2541ffb640 10 monclient: _renew_subs 2024-09-15T21:17:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.306+0000 7f2541ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.306+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=18}) -- 0x7f25300405c0 con 0x5588214c6af0 2024-09-15T21:17:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.306+0000 7f2541ffb640 1 -- 172.21.15.17:0/375139469 <== mon.0 v2:172.21.15.17:3300/0 12 ==== osd_map(18..18 src has 1..18) ==== 654+0+0 (secure 0 0 0) 0x7f253c0d1030 con 0x5588214c6af0 2024-09-15T21:17:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.306+0000 7f2543fff640 1 -- 172.21.15.17:0/375139469 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) -- 0x7f252c005b60 con 0x558821557110 2024-09-15T21:17:41.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f25530ec640 1 -- 172.21.15.17:0/375139469 <== osd.5 v2:172.21.15.119:6808/444741843 5 ==== osd_op_reply(7 rbd_info [write 0~19] v18'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2544002620 con 0x558821557110 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x558821557110 msgr2=0x558821575f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x558821557110 0x558821575f30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f253003a2b0 msgr2=0x7f253003c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f253003a2b0 0x7f253003c770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2548005fd0 tx=0x7f2548005a10 comp rx=0 tx=0).stop 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 10 monclient: shutdown 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 msgr2=0x55882156cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 --2- 172.21.15.17:0/375139469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5588214c6af0 0x55882156cec0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f253c0ad040 tx=0x7f253c0180d0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f25530ec640 1 -- 172.21.15.17:0/375139469 reap_dead start 2024-09-15T21:17:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 shutdown_connections 2024-09-15T21:17:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.314+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 >> 172.21.15.17:0/375139469 conn(0x5588214cb580 msgr2=0x5588214c0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.318+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 shutdown_connections 2024-09-15T21:17:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.318+0000 7f2553381100 1 -- 172.21.15.17:0/375139469 wait complete. 2024-09-15T21:17:41.335 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:17:41.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-15T21:17:41.336 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:17:41.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient: get_monmap_and_config 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient: build_initial_monmap 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient: monmap: 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:41.470613+0000 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:41.470613+0000 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:41.475 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 Processor -- start 2024-09-15T21:17:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 -- start start 2024-09-15T21:17:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient: init 2024-09-15T21:17:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:17:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 0x7f1f6c09ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): picked mon.noname-b con 0x7f1f6c099600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1f6c0999e0 0x7f1f6c0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): picked mon.noname-a con 0x7f1f6c0999e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1f6c099de0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6c075200 con 0x7f1f6c099de0 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0748c0 con 0x7f1f6c0999e0 2024-09-15T21:17:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c074d80 con 0x7f1f6c099600 2024-09-15T21:17:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f73b7c640 10 monclient(hunting): authenticate will time out at 984.866154s 2024-09-15T21:17:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 0x7f1f6c09ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1f6c0999e0 0x7f1f6c0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 0x7f1f6c09ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45312/0 (socket says 172.21.15.17:45312) 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 -- 172.21.15.17:0/2274511075 learned_addr learned my addr 172.21.15.17:0/2274511075 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): get_auth_request con 0x7f1f6c099600 auth_method 0 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): get_auth_request con 0x7f1f6c0999e0 auth_method 0 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71117640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient(hunting): handle_auth_done global_id 4205 payload 274 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 -- 172.21.15.17:0/2274511075 >> v2:172.21.15.17:3300/0 conn(0x7f1f6c0999e0 msgr2=0x7f1f6c0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 --2- 172.21.15.17:0/2274511075 >> v2:172.21.15.17:3300/0 conn(0x7f1f6c0999e0 0x7f1f6c0a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 -- 172.21.15.17:0/2274511075 >> v1:172.21.15.119:6789/0 conn(0x7f1f6c099de0 legacy=0x7f1f6c0a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient: _finish_hunting 0 2024-09-15T21:17:41.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 monclient: found mon.noname-b 2024-09-15T21:17:41.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:41.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:41.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.466+0000 7f1f71918640 1 -- 172.21.15.17:0/2274511075 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c09dbc0 con 0x7f1f6c099600 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71918640 1 --2- 172.21.15.17:0/2274511075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 0x7f1f6c09ee60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009e50 tx=0x7f1f6401c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa9abf71a0cd52e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 1 -- 172.21.15.17:0/2274511075 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640a56e0 con 0x7f1f6c099600 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: dump: 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: _finish_auth 0 2024-09-15T21:17:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: _check_auth_tickets 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 1 -- 172.21.15.17:0/2274511075 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f640a5880 con 0x7f1f6c099600 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 1 -- 172.21.15.17:0/2274511075 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640a5c50 con 0x7f1f6c099600 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f70916640 10 monclient: dump: 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 5 monclient: authenticate success, global_id 4205 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 4 set_mon_vals no callback set 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: shutdown 2024-09-15T21:17:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- 172.21.15.17:0/2274511075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 msgr2=0x7f1f6c09ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/2274511075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 0x7f1f6c09ee60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009e50 tx=0x7f1f6401c8c0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- 172.21.15.17:0/2274511075 shutdown_connections 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/2274511075 >> v2:172.21.15.17:3300/0 conn(0x7f1f6c0999e0 0x7f1f6c0a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/2274511075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c099600 0x7f1f6c09ee60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- 172.21.15.17:0/2274511075 >> 172.21.15.17:0/2274511075 conn(0x7f1f6c0a6820 msgr2=0x7f1f6c0a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- 172.21.15.17:0/2274511075 shutdown_connections 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- 172.21.15.17:0/2274511075 wait complete. 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: build_initial_monmap 2024-09-15T21:17:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: monmap: 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 Processor -- start 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- start start 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: init 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f6c099600 0x7f1f6c091630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): picked mon.noname-b con 0x7f1f6c099600 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 0x7f1f6c091b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): picked mon.noname-c con 0x7f1f6c0999e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f6c099de0 0x7f1f6c0920b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): picked mon.noname-a con 0x7f1f6c099de0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c068d10 con 0x7f1f6c099de0 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c0a2150 con 0x7f1f6c099600 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c142300 con 0x7f1f6c0999e0 2024-09-15T21:17:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f73b7c640 10 monclient(hunting): authenticate will time out at 984.871155s 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f6c099de0 0x7f1f6c0920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 0x7f1f6c091b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 0x7f1f6c091b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45326/0 (socket says 172.21.15.17:45326) 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 1 -- 172.21.15.17:0/3375546294 learned_addr learned my addr 172.21.15.17:0/3375546294 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): get_auth_request con 0x7f1f6c0999e0 auth_method 0 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): get_auth_request con 0x7f1f6c099de0 auth_method 0 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71918640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f6c099600 0x7f1f6c091630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient(hunting): handle_auth_done global_id 4206 payload 824 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 -- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 msgr2=0x7f1f6c091b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 0x7f1f6c091b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 -- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f6c099600 msgr2=0x7f1f6c091630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f6c099600 0x7f1f6c091630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient: _finish_hunting 0 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 monclient: found mon.noname-a 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 -- 172.21.15.17:0/3375546294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0950a0 con 0x7f1f6c099de0 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71918640 10 monclient: get_auth_request con 0x7f1f6c099600 auth_method 0 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71117640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 0x7f1f6c091b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f71918640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f6c099600 0x7f1f6c091630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.470+0000 7f1f72119640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f6c099de0 0x7f1f6c0920b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c440 tx=0x7f1f5c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011790 con 0x7f1f6c099de0 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: dump: 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c017070 con 0x7f1f6c099de0 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 5 monclient: authenticate success, global_id 4206 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 monclient: _renew_subs 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c09d810 con 0x7f1f6c099de0 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011b70 con 0x7f1f6c099de0 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 10 monclient: dump: 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 monclient: _renew_subs 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6bfff640 4 set_mon_vals no callback set 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c0a2a50 con 0x7f1f6c099de0 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1f5c011d10 con 0x7f1f6c099de0 2024-09-15T21:17:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c09ee60 con 0x7f1f6c099de0 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1f5803b9f0 0x7f1f5803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f1f5c04f600 con 0x7f1f6c099de0 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f71918640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1f5803b9f0 0x7f1f5803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f71918640 10 monclient: get_auth_request con 0x7f1f5803b9f0 auth_method 0 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.474+0000 7f1f71918640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1f5803b9f0 0x7f1f5803deb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f640ae000 tx=0x7f1f640a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.478+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1f5c010cf0 con 0x7f1f6c099de0 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.478+0000 7f1f6a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.478+0000 7f1f6a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:41.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.650+0000 7f1f73b7c640 10 start_mon_command cmd=[{"prefix": "config log", "num": 1, "format": "json"}] 2024-09-15T21:17:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.650+0000 7f1f73b7c640 10 monclient: _send_command 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2024-09-15T21:17:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.650+0000 7f1f73b7c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.650+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f1f6c097950 con 0x7f1f6c099de0 2024-09-15T21:17:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f6a7fc640 1 -- 172.21.15.17:0/3375546294 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v4) ==== 86+0+159 (secure 0 0 0) 0x7f1f5c04f280 con 0x7f1f6c099de0 2024-09-15T21:17:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f6a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2024-09-15T21:17:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f6a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:41.662 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1f5803b9f0 msgr2=0x7f1f5803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1f5803b9f0 0x7f1f5803deb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f640ae000 tx=0x7f1f640a7070 comp rx=0 tx=0).stop 2024-09-15T21:17:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 10 monclient: shutdown 2024-09-15T21:17:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f6c099de0 msgr2=0x7f1f6c0920b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f6c099de0 0x7f1f6c0920b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c440 tx=0x7f1f5c00c900 comp rx=0 tx=0).stop 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 shutdown_connections 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1f5803b9f0 0x7f1f5803deb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f6c099de0 0x7f1f6c0920b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1f6c0999e0 0x7f1f6c091b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 --2- 172.21.15.17:0/3375546294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f6c099600 0x7f1f6c091630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 >> 172.21.15.17:0/3375546294 conn(0x7f1f6c0a6820 msgr2=0x7f1f6c0a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 shutdown_connections 2024-09-15T21:17:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.654+0000 7f1f73b7c640 1 -- 172.21.15.17:0/3375546294 wait complete. 2024-09-15T21:17:41.681 INFO:teuthology.orchestra.run.smithi017.stdout:[{"version":4,"timestamp":"2024-09-15T21:17:30.943746+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"7222.680366"}]}] 2024-09-15T21:17:41.681 INFO:tasks.ceph_manager:config epoch is 4 2024-09-15T21:17:41.681 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:17:41.681 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:17:41.682 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient: get_monmap_and_config 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient: build_initial_monmap 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient: monmap: 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:41.804094+0000 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:41.804094+0000 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:41.808 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 Processor -- start 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 -- start start 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient: init 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f77e40a1250 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77e40a1630 0x7f77e40a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): picked mon.noname-a con 0x7f77e40a1630 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e413f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): picked mon.noname-b con 0x7f77e413ccd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e4075220 con 0x7f77e40a1250 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f77e4074da0 con 0x7f77e40a1630 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f77e409dbc0 con 0x7f77e413ccd0 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.798+0000 7f77eba10640 10 monclient(hunting): authenticate will time out at 985.199915s 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e413f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77e40a1630 0x7f77e40a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e413f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45336/0 (socket says 172.21.15.17:45336) 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 -- 172.21.15.17:0/1813411260 learned_addr learned my addr 172.21.15.17:0/1813411260 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): get_auth_request con 0x7f77e413ccd0 auth_method 0 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): get_auth_request con 0x7f77e40a1630 auth_method 0 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e8fab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient(hunting): handle_auth_done global_id 4211 payload 274 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 -- 172.21.15.17:0/1813411260 >> v2:172.21.15.17:3300/0 conn(0x7f77e40a1630 msgr2=0x7f77e40a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 --2- 172.21.15.17:0/1813411260 >> v2:172.21.15.17:3300/0 conn(0x7f77e40a1630 0x7f77e40a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 -- 172.21.15.17:0/1813411260 >> v1:172.21.15.119:6789/0 conn(0x7f77e40a1250 legacy=0x7f77e409ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient: _finish_hunting 0 2024-09-15T21:17:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 monclient: found mon.noname-b 2024-09-15T21:17:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 -- 172.21.15.17:0/1813411260 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e40a1fc0 con 0x7f77e413ccd0 2024-09-15T21:17:41.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77e9fad640 1 --2- 172.21.15.17:0/1813411260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e413f0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009e50 tx=0x7f77dc01c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9767c63d5e732d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:41.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 1 -- 172.21.15.17:0/1813411260 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0a5640 con 0x7f77e413ccd0 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: dump: 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.819 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: _finish_auth 0 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 1 -- 172.21.15.17:0/1813411260 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc0a57e0 con 0x7f77e413ccd0 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 5 monclient: authenticate success, global_id 4211 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 1 -- 172.21.15.17:0/1813411260 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0a5bb0 con 0x7f77e413ccd0 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 4 set_mon_vals no callback set 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77dbfff640 10 monclient: dump: 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 10 monclient: shutdown 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 -- 172.21.15.17:0/1813411260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 msgr2=0x7f77e413f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 --2- 172.21.15.17:0/1813411260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e413f0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009e50 tx=0x7f77dc01c980 comp rx=0 tx=0).stop 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 -- 172.21.15.17:0/1813411260 shutdown_connections 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 --2- 172.21.15.17:0/1813411260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e413f0c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 --2- 172.21.15.17:0/1813411260 >> v2:172.21.15.17:3300/0 conn(0x7f77e40a1630 0x7f77e40a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 -- 172.21.15.17:0/1813411260 >> 172.21.15.17:0/1813411260 conn(0x7f77e40a6820 msgr2=0x7f77e40a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 -- 172.21.15.17:0/1813411260 shutdown_connections 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.802+0000 7f77eba10640 1 -- 172.21.15.17:0/1813411260 wait complete. 2024-09-15T21:17:41.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: build_initial_monmap 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: monmap: 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 Processor -- start 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- start start 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: init 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77e40a1250 0x7f77e408d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): picked mon.noname-a con 0x7f77e40a1250 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40a1630 0x7f77e408dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): picked mon.noname-b con 0x7f77e40a1630 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e408e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): picked mon.noname-c con 0x7f77e413ccd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f77e41421b0 con 0x7f77e40a1250 2024-09-15T21:17:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4142030 con 0x7f77e40a1630 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f77e4142330 con 0x7f77e413ccd0 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient(hunting): authenticate will time out at 985.204997s 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77e40a1250 0x7f77e408d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e408e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e408e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45346/0 (socket says 172.21.15.17:45346) 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77e40a1250 0x7f77e408d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44038/0 (socket says 172.21.15.17:44038) 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 -- 172.21.15.17:0/3926347327 learned_addr learned my addr 172.21.15.17:0/3926347327 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): get_auth_request con 0x7f77e413ccd0 auth_method 0 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): get_auth_request con 0x7f77e40a1250 auth_method 0 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e8fab640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40a1630 0x7f77e408dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e8fab640 10 monclient(hunting): get_auth_request con 0x7f77e40a1630 auth_method 0 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e8fab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e8fab640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e8fab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient(hunting): handle_auth_done global_id 4214 payload 824 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 -- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40a1630 msgr2=0x7f77e408dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40a1630 0x7f77e408dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 -- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77e40a1250 msgr2=0x7f77e408d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77e40a1250 0x7f77e408d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient: _finish_hunting 0 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 monclient: found mon.noname-c 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 -- 172.21.15.17:0/3926347327 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e4090ed0 con 0x7f77e413ccd0 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e9fad640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e408e010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f77dc01a5b0 tx=0x7f77dc019720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0bc020 con 0x7f77e413ccd0 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: dump: 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 5 monclient: authenticate success, global_id 4214 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _renew_subs 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc0b8880 con 0x7f77e413ccd0 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e4090910 con 0x7f77e413ccd0 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0a5640 con 0x7f77e413ccd0 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 10 monclient: dump: 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77db7fe640 4 set_mon_vals no callback set 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _renew_subs 2024-09-15T21:17:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e408ef50 con 0x7f77e413ccd0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e40a1a80 con 0x7f77e413ccd0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f77dc0bf020 con 0x7f77e413ccd0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f77c803b9c0 0x7f77c803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f77dc0eab70 con 0x7f77e413ccd0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f77c803b9c0 0x7f77c803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.806+0000 7f77e97ac640 10 monclient: get_auth_request con 0x7f77c803b9c0 auth_method 0 2024-09-15T21:17:41.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.810+0000 7f77e97ac640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f77c803b9c0 0x7f77c803de80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f77d4007fd0 tx=0x7f77d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:41.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.810+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f77dc0c6030 con 0x7f77e413ccd0 2024-09-15T21:17:41.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.810+0000 7f77d9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:41.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:41.810+0000 7f77d9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.018+0000 7f77eba10640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-09-15T21:17:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.018+0000 7f77eba10640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-15T21:17:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.018+0000 7f77eba10640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.018+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f77e4091210 con 0x7f77e413ccd0 2024-09-15T21:17:42.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77d9ffb640 1 -- 172.21.15.17:0/3926347327 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+93778 (secure 0 0 0) 0x7f77dc016600 con 0x7f77e413ccd0 2024-09-15T21:17:42.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77d9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-15T21:17:42.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77d9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:42.031 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f77c803b9c0 msgr2=0x7f77c803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f77c803b9c0 0x7f77c803de80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f77d4007fd0 tx=0x7f77d4009040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 10 monclient: shutdown 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 msgr2=0x7f77e408e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e408e010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f77dc01a5b0 tx=0x7f77dc019720 comp rx=0 tx=0).stop 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 shutdown_connections 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f77c803b9c0 0x7f77c803de80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f77e413ccd0 0x7f77e408e010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40a1630 0x7f77e408dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 --2- 172.21.15.17:0/3926347327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77e40a1250 0x7f77e408d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 >> 172.21.15.17:0/3926347327 conn(0x7f77e40a6820 msgr2=0x7f77e413bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 shutdown_connections 2024-09-15T21:17:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.022+0000 7f77eba10640 1 -- 172.21.15.17:0/3926347327 wait complete. 2024-09-15T21:17:42.048 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1238844171},{"type":"v1","addr":"172.21.15.119:6825","nonce":1238844171}]},"active_addr":"172.21.15.119:6825/1238844171","active_change":"2024-09-15T21:17:25.154215+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3265418294}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":394289560}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1556682753}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1937938728}]}]} 2024-09-15T21:17:42.052 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:17:42.052 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:17:42.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:42.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.162+0000 7f41909c4640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.162+0000 7f41909c4640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.162+0000 7f41909c4640 10 monclient: monmap: 2024-09-15T21:17:42.172 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.172 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.169378+0000 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.169378+0000 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.173 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 Processor -- start 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 -- start start 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient: init 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f418c073ad0 0x7f418c073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): picked mon.noname-c con 0x7f418c073ad0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c0a2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): picked mon.noname-b con 0x7f418c0a4200 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f418c09f570 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418c0a1490 con 0x7f418c09f570 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f418c0713c0 con 0x7f418c073ad0 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f418c071540 con 0x7f418c0a4200 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient(hunting): authenticate will time out at 985.564818s 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c0a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c0a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45348/0 (socket says 172.21.15.17:45348) 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 -- 172.21.15.17:0/4196891676 learned_addr learned my addr 172.21.15.17:0/4196891676 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): get_auth_request con 0x7f418c0a4200 auth_method 0 2024-09-15T21:17:42.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418a59c640 1 --2- 172.21.15.17:0/4196891676 >> v2:172.21.15.119:3300/0 conn(0x7f418c073ad0 0x7f418c073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient(hunting): handle_auth_done global_id 4217 payload 274 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 -- 172.21.15.17:0/4196891676 >> v2:172.21.15.119:3300/0 conn(0x7f418c073ad0 msgr2=0x7f418c073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 --2- 172.21.15.17:0/4196891676 >> v2:172.21.15.119:3300/0 conn(0x7f418c073ad0 0x7f418c073eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 -- 172.21.15.17:0/4196891676 >> v1:172.21.15.119:6789/0 conn(0x7f418c09f570 legacy=0x7f418c09f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 -- 172.21.15.17:0/4196891676 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418c0a3ce0 con 0x7f418c0a4200 2024-09-15T21:17:42.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f4189d9b640 1 --2- 172.21.15.17:0/4196891676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c0a2150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4174009870 tx=0x7f417401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33605ef12aa5cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 1 -- 172.21.15.17:0/4196891676 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41740b0070 con 0x7f418c0a4200 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: dump: 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 1 -- 172.21.15.17:0/4196891676 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41740ab600 con 0x7f418c0a4200 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 5 monclient: authenticate success, global_id 4217 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 1 -- 172.21.15.17:0/4196891676 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41740ab9d0 con 0x7f418c0a4200 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f41909c4640 4 set_mon_vals no callback set 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.166+0000 7f418959a640 10 monclient: dump: 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: shutdown 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/4196891676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 msgr2=0x7f418c0a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 --2- 172.21.15.17:0/4196891676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c0a2150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4174009870 tx=0x7f417401c4b0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/4196891676 shutdown_connections 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 --2- 172.21.15.17:0/4196891676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c0a2150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 --2- 172.21.15.17:0/4196891676 >> v2:172.21.15.119:3300/0 conn(0x7f418c073ad0 0x7f418c073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/4196891676 >> 172.21.15.17:0/4196891676 conn(0x7f418c0a4d40 msgr2=0x7f418c0a5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/4196891676 shutdown_connections 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/4196891676 wait complete. 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: monmap: 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.186 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 Processor -- start 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- start start 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: init 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:42.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c073ad0 0x7f418c13c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): picked mon.noname-b con 0x7f418c073ad0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 0x7f418c13cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): picked mon.noname-a con 0x7f418c09f570 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c125940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): picked mon.noname-c con 0x7f418c0a4200 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f418c1402b0 con 0x7f418c09f570 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f418c0a3f10 con 0x7f418c073ad0 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f418c140430 con 0x7f418c0a4200 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient(hunting): authenticate will time out at 985.570012s 2024-09-15T21:17:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 0x7f418c13cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c125940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 0x7f418c13cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44054/0 (socket says 172.21.15.17:44054) 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 1 -- 172.21.15.17:0/6470356 learned_addr learned my addr 172.21.15.17:0/6470356 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c125940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45364/0 (socket says 172.21.15.17:45364) 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): get_auth_request con 0x7f418c09f570 auth_method 0 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): get_auth_request con 0x7f418c0a4200 auth_method 0 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418a59c640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c073ad0 0x7f418c13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418a59c640 10 monclient(hunting): get_auth_request con 0x7f418c073ad0 auth_method 0 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient(hunting): handle_auth_done global_id 4220 payload 824 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 -- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c073ad0 msgr2=0x7f418c13c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c073ad0 0x7f418c13c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 -- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 msgr2=0x7f418c13cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 0x7f418c13cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 monclient: found mon.noname-c 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 -- 172.21.15.17:0/6470356 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418c127f10 con 0x7f418c0a4200 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4189d9b640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 0x7f418c13cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f418ad9d640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c125940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f418400c410 tx=0x7f418400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184011790 con 0x7f418c0a4200 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:42.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: dump: 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4184017070 con 0x7f418c0a4200 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184011b70 con 0x7f418c0a4200 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 5 monclient: authenticate success, global_id 4220 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _renew_subs 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4188d99640 4 set_mon_vals no callback set 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418c128fa0 con 0x7f418c0a4200 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 10 monclient: dump: 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.193 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _renew_subs 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f418c13d0e0 con 0x7f418c0a4200 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:42.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418c128dd0 con 0x7f418c0a4200 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.170+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4184011d10 con 0x7f418c0a4200 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.174+0000 7f4182ffd640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f416c03b9c0 0x7f416c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.174+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f418404f490 con 0x7f418c0a4200 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.174+0000 7f418a59c640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f416c03b9c0 0x7f416c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.174+0000 7f418a59c640 10 monclient: get_auth_request con 0x7f416c03b9c0 auth_method 0 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.174+0000 7f418a59c640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f416c03b9c0 0x7f416c03de80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f417c0070b0 tx=0x7f417c007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.178+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4184021020 con 0x7f418c0a4200 2024-09-15T21:17:42.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.178+0000 7f4182ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:42.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.178+0000 7f4182ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:42.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.350+0000 7f41909c4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:42.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.350+0000 7f41909c4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:42.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.350+0000 7f41909c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.350+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f418c09f040 con 0x7f418c0a4200 2024-09-15T21:17:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f4182ffd640 1 -- 172.21.15.17:0/6470356 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12389 (secure 0 0 0) 0x7f4184010cf0 con 0x7f418c0a4200 2024-09-15T21:17:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f4182ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f4182ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:42.361 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:41.296979+0000","last_up_change":"2024-09-15T21:17:31.232299+0000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:17:31.217943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:17:37.652178+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6809","nonce":3272576910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6811","nonce":3272576910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6815","nonce":3272576910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6813","nonce":3272576910}]},"public_addr":"172.21.15.17:6809/3272576910","cluster_addr":"172.21.15.17:6811/3272576910","heartbeat_back_addr":"172.21.15.17:6815/3272576910","heartbeat_front_addr":"172.21.15.17:6813/3272576910","state":["exists","up"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6817","nonce":2555706928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6819","nonce":2555706928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6823","nonce":2555706928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6821","nonce":2555706928}]},"public_addr":"172.21.15.17:6817/2555706928","cluster_addr":"172.21.15.17:6819/2555706928","heartbeat_back_addr":"172.21.15.17:6823/2555706928","heartbeat_front_addr":"172.21.15.17:6821/2555706928","state":["exists","up"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6801","nonce":671096025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6803","nonce":671096025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6807","nonce":671096025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6805","nonce":671096025}]},"public_addr":"172.21.15.17:6801/671096025","cluster_addr":"172.21.15.17:6803/671096025","heartbeat_back_addr":"172.21.15.17:6807/671096025","heartbeat_front_addr":"172.21.15.17:6805/671096025","state":["exists","up"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6801","nonce":3353277098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6803","nonce":3353277098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6807","nonce":3353277098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6805","nonce":3353277098}]},"public_addr":"172.21.15.119:6801/3353277098","cluster_addr":"172.21.15.119:6803/3353277098","heartbeat_back_addr":"172.21.15.119:6807/3353277098","heartbeat_front_addr":"172.21.15.119:6805/3353277098","state":["exists","up"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6817","nonce":2086984234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6819","nonce":2086984234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6823","nonce":2086984234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6821","nonce":2086984234}]},"public_addr":"172.21.15.119:6817/2086984234","cluster_addr":"172.21.15.119:6819/2086984234","heartbeat_back_addr":"172.21.15.119:6823/2086984234","heartbeat_front_addr":"172.21.15.119:6821/2086984234","state":["exists","up"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6809","nonce":444741843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6811","nonce":444741843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6815","nonce":444741843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6813","nonce":444741843}]},"public_addr":"172.21.15.119:6809/444741843","cluster_addr":"172.21.15.119:6811/444741843","heartbeat_back_addr":"172.21.15.119:6815/444741843","heartbeat_front_addr":"172.21.15.119:6813/444741843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.768611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.237742+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.809091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.353555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.041478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.487022+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:42.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f416c03b9c0 msgr2=0x7f416c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f416c03b9c0 0x7f416c03de80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f417c0070b0 tx=0x7f417c007040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 10 monclient: shutdown 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 msgr2=0x7f418c125940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c125940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f418400c410 tx=0x7f418400c8d0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 shutdown_connections 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f416c03b9c0 0x7f416c03de80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f418c0a4200 0x7f418c125940 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f418c09f570 0x7f418c13cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 --2- 172.21.15.17:0/6470356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c073ad0 0x7f418c13c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 >> 172.21.15.17:0/6470356 conn(0x7f418c0a4d40 msgr2=0x7f418c0a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 shutdown_connections 2024-09-15T21:17:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.354+0000 7f41909c4640 1 -- 172.21.15.17:0/6470356 wait complete. 2024-09-15T21:17:42.381 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:17:42.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:42.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient: monmap: 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.496694+0000 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.496694+0000 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.500 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.501 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.501 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 1 Processor -- start 2024-09-15T21:17:42.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 1 -- start start 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient: init 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 0x7f9c8809b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 10 monclient(hunting): picked mon.noname-b con 0x7f9c880a1330 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.490+0000 7f9c90159640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9c880a1710 0x7f9c880a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient(hunting): picked mon.noname-c con 0x7f9c880a1710 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9c88099240 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880a01f0 con 0x7f9c88099240 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8809b700 con 0x7f9c880a1710 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9c880a2bb0 con 0x7f9c880a1330 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient(hunting): authenticate will time out at 985.892072s 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 0x7f9c8809b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 0x7f9c8809b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45378/0 (socket says 172.21.15.17:45378) 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 -- 172.21.15.17:0/2954848698 learned_addr learned my addr 172.21.15.17:0/2954848698 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): get_auth_request con 0x7f9c880a1330 auth_method 0 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8d6f4640 1 --2- 172.21.15.17:0/2954848698 >> v2:172.21.15.119:3300/0 conn(0x7f9c880a1710 0x7f9c880a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient(hunting): handle_auth_done global_id 4223 payload 274 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 -- 172.21.15.17:0/2954848698 >> v2:172.21.15.119:3300/0 conn(0x7f9c880a1710 msgr2=0x7f9c880a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 --2- 172.21.15.17:0/2954848698 >> v2:172.21.15.119:3300/0 conn(0x7f9c880a1710 0x7f9c880a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 -- 172.21.15.17:0/2954848698 >> v1:172.21.15.119:6789/0 conn(0x7f9c88099240 legacy=0x7f9c880996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 -- 172.21.15.17:0/2954848698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c880a2ea0 con 0x7f9c880a1330 2024-09-15T21:17:42.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8def5640 1 --2- 172.21.15.17:0/2954848698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 0x7f9c8809b1c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009870 tx=0x7f9c8401c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e21b95b8799a6ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 1 -- 172.21.15.17:0/2954848698 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b0070 con 0x7f9c880a1330 2024-09-15T21:17:42.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: dump: 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 1 -- 172.21.15.17:0/2954848698 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840ab600 con 0x7f9c880a1330 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 5 monclient: authenticate success, global_id 4223 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 1 -- 172.21.15.17:0/2954848698 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840ab9d0 con 0x7f9c880a1330 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 4 set_mon_vals no callback set 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c8cef3640 10 monclient: dump: 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.511 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 10 monclient: shutdown 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- 172.21.15.17:0/2954848698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 msgr2=0x7f9c8809b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 --2- 172.21.15.17:0/2954848698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 0x7f9c8809b1c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009870 tx=0x7f9c8401c240 comp rx=0 tx=0).stop 2024-09-15T21:17:42.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- 172.21.15.17:0/2954848698 shutdown_connections 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 --2- 172.21.15.17:0/2954848698 >> v2:172.21.15.119:3300/0 conn(0x7f9c880a1710 0x7f9c880a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 --2- 172.21.15.17:0/2954848698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c880a1330 0x7f9c8809b1c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- 172.21.15.17:0/2954848698 >> 172.21.15.17:0/2954848698 conn(0x7f9c880a6820 msgr2=0x7f9c880a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- 172.21.15.17:0/2954848698 shutdown_connections 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.494+0000 7f9c90159640 1 -- 172.21.15.17:0/2954848698 wait complete. 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: monmap: 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.513 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 Processor -- start 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- start start 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: init 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c88099240 0x7f9c881d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): picked mon.noname-c con 0x7f9c88099240 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c880a1330 0x7f9c881d0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): picked mon.noname-b con 0x7f9c880a1330 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 0x7f9c8812b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): picked mon.noname-a con 0x7f9c880a1710 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88140600 con 0x7f9c880a1710 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88140480 con 0x7f9c880a1330 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9c88140780 con 0x7f9c88099240 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient(hunting): authenticate will time out at 985.897013s 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c88099240 0x7f9c881d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 0x7f9c8812b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 0x7f9c8812b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44056/0 (socket says 172.21.15.17:44056) 2024-09-15T21:17:42.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 1 -- 172.21.15.17:0/4174634135 learned_addr learned my addr 172.21.15.17:0/4174634135 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): get_auth_request con 0x7f9c880a1710 auth_method 0 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): get_auth_request con 0x7f9c88099240 auth_method 0 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c880a1330 0x7f9c881d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 10 monclient(hunting): get_auth_request con 0x7f9c880a1330 auth_method 0 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient(hunting): handle_auth_done global_id 4226 payload 824 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 -- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c880a1330 msgr2=0x7f9c881d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c880a1330 0x7f9c881d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 -- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 msgr2=0x7f9c8812b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 0x7f9c8812b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 monclient: found mon.noname-c 2024-09-15T21:17:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 -- 172.21.15.17:0/4174634135 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c880a5010 con 0x7f9c88099240 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8e6f6640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 0x7f9c8812b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8def5640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c88099240 0x7f9c881d00e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9c880a0c30 tx=0x7f9c84018fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b8020 con 0x7f9c88099240 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: dump: 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840b4640 con 0x7f9c88099240 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 5 monclient: authenticate success, global_id 4226 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _renew_subs 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8809d810 con 0x7f9c88099240 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b0040 con 0x7f9c88099240 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 10 monclient: dump: 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _renew_subs 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7ffff640 4 set_mon_vals no callback set 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8812dd50 con 0x7f9c88099240 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8809b1c0 con 0x7f9c88099240 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9c840bb070 con 0x7f9c88099240 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c6c03ba20 0x7f9c6c03dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f9c840e8c20 con 0x7f9c88099240 2024-09-15T21:17:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c6c03ba20 0x7f9c6c03dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.498+0000 7f9c8d6f4640 10 monclient: get_auth_request con 0x7f9c6c03ba20 auth_method 0 2024-09-15T21:17:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.502+0000 7f9c8d6f4640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c6c03ba20 0x7f9c6c03dee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9c88075430 tx=0x7f9c78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.502+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9c880a01f0 con 0x7f9c88099240 2024-09-15T21:17:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.502+0000 7f9c7e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.502+0000 7f9c7e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:42.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c90159640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:42.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c90159640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:42.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c90159640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c8813ca50 con 0x7f9c88099240 2024-09-15T21:17:42.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c7e7fc640 1 -- 172.21.15.17:0/4174634135 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12389 (secure 0 0 0) 0x7f9c840ab4c0 con 0x7f9c88099240 2024-09-15T21:17:42.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c7e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:42.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.670+0000 7f9c7e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:42.681 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:42.681 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:41.296979+0000","last_up_change":"2024-09-15T21:17:31.232299+0000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:17:31.217943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:17:37.652178+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6809","nonce":3272576910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6811","nonce":3272576910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6815","nonce":3272576910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6813","nonce":3272576910}]},"public_addr":"172.21.15.17:6809/3272576910","cluster_addr":"172.21.15.17:6811/3272576910","heartbeat_back_addr":"172.21.15.17:6815/3272576910","heartbeat_front_addr":"172.21.15.17:6813/3272576910","state":["exists","up"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6817","nonce":2555706928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6819","nonce":2555706928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6823","nonce":2555706928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6821","nonce":2555706928}]},"public_addr":"172.21.15.17:6817/2555706928","cluster_addr":"172.21.15.17:6819/2555706928","heartbeat_back_addr":"172.21.15.17:6823/2555706928","heartbeat_front_addr":"172.21.15.17:6821/2555706928","state":["exists","up"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6801","nonce":671096025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6803","nonce":671096025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6807","nonce":671096025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6805","nonce":671096025}]},"public_addr":"172.21.15.17:6801/671096025","cluster_addr":"172.21.15.17:6803/671096025","heartbeat_back_addr":"172.21.15.17:6807/671096025","heartbeat_front_addr":"172.21.15.17:6805/671096025","state":["exists","up"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6801","nonce":3353277098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6803","nonce":3353277098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6807","nonce":3353277098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6805","nonce":3353277098}]},"public_addr":"172.21.15.119:6801/3353277098","cluster_addr":"172.21.15.119:6803/3353277098","heartbeat_back_addr":"172.21.15.119:6807/3353277098","heartbeat_front_addr":"172.21.15.119:6805/3353277098","state":["exists","up"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6817","nonce":2086984234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6819","nonce":2086984234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6823","nonce":2086984234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6821","nonce":2086984234}]},"public_addr":"172.21.15.119:6817/2086984234","cluster_addr":"172.21.15.119:6819/2086984234","heartbeat_back_addr":"172.21.15.119:6823/2086984234","heartbeat_front_addr":"172.21.15.119:6821/2086984234","state":["exists","up"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6809","nonce":444741843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6811","nonce":444741843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6815","nonce":444741843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6813","nonce":444741843}]},"public_addr":"172.21.15.119:6809/444741843","cluster_addr":"172.21.15.119:6811/444741843","heartbeat_back_addr":"172.21.15.119:6815/444741843","heartbeat_front_addr":"172.21.15.119:6813/444741843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.768611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.237742+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.809091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.353555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.041478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.487022+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:42.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c6c03ba20 msgr2=0x7f9c6c03dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c6c03ba20 0x7f9c6c03dee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9c88075430 tx=0x7f9c78008040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 10 monclient: shutdown 2024-09-15T21:17:42.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c88099240 msgr2=0x7f9c881d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c88099240 0x7f9c881d00e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9c880a0c30 tx=0x7f9c84018fe0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 shutdown_connections 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c6c03ba20 0x7f9c6c03dee0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c880a1710 0x7f9c8812b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c880a1330 0x7f9c881d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 --2- 172.21.15.17:0/4174634135 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c88099240 0x7f9c881d00e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 >> 172.21.15.17:0/4174634135 conn(0x7f9c880a6820 msgr2=0x7f9c8813cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 shutdown_connections 2024-09-15T21:17:42.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.674+0000 7f9c90159640 1 -- 172.21.15.17:0/4174634135 wait complete. 2024-09-15T21:17:42.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:17:42.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:17:42.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:17:42.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:17:42.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:17:42.701 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:17:42.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient: monmap: 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.808687+0000 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.808687+0000 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.812 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 Processor -- start 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 -- start start 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient: init 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d81378c0 0x7f52d8137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): picked mon.noname-b con 0x7f52d81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f52d81381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52d80a5210 0x7f52d80a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): picked mon.noname-a con 0x7f52d80a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:42.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52dec23640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52d80a5210 0x7f52d80a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52dec23640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52d80a5210 0x7f52d80a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44070/0 (socket says 172.21.15.17:44070) 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52dec23640 1 -- 172.21.15.17:0/1430218241 learned_addr learned my addr 172.21.15.17:0/1430218241 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d80a1d00 con 0x7f52d81381e0 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f52d80a1fe0 con 0x7f52d80a5210 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52de422640 1 --2- 172.21.15.17:0/1430218241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d81378c0 0x7f52d8137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f52d80a22c0 con 0x7f52d81378c0 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.802+0000 7f52e0686640 10 monclient(hunting): authenticate will time out at 986.203997s 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): get_auth_request con 0x7f52d81378c0 auth_method 0 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dec23640 10 monclient(hunting): get_auth_request con 0x7f52d80a5210 auth_method 0 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dec23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dec23640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dec23640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient(hunting): handle_auth_done global_id 4229 payload 274 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 1 -- 172.21.15.17:0/1430218241 >> v2:172.21.15.17:3300/0 conn(0x7f52d80a5210 msgr2=0x7f52d80a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 1 --2- 172.21.15.17:0/1430218241 >> v2:172.21.15.17:3300/0 conn(0x7f52d80a5210 0x7f52d80a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 1 -- 172.21.15.17:0/1430218241 >> v1:172.21.15.17:6789/0 conn(0x7f52d81381e0 legacy=0x7f52d80a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 1 -- 172.21.15.17:0/1430218241 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d80a5c00 con 0x7f52d81378c0 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52de422640 1 --2- 172.21.15.17:0/1430218241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d81378c0 0x7f52d8137ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097c0 tx=0x7f52d401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f0c657eb1bcc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 1 -- 172.21.15.17:0/1430218241 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40b0070 con 0x7f52d81378c0 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 10 monclient: dump: 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.820 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.806+0000 7f52dd420640 1 -- 172.21.15.17:0/1430218241 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d40ab600 con 0x7f52d81378c0 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dd420640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dd420640 1 -- 172.21.15.17:0/1430218241 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40ab9d0 con 0x7f52d81378c0 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dd420640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dd420640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dd420640 10 monclient: dump: 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 5 monclient: authenticate success, global_id 4229 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 4 set_mon_vals no callback set 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: shutdown 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d81378c0 msgr2=0x7f52d8137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 --2- 172.21.15.17:0/1430218241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d81378c0 0x7f52d8137ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097c0 tx=0x7f52d401c360 comp rx=0 tx=0).stop 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 shutdown_connections 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 --2- 172.21.15.17:0/1430218241 >> v2:172.21.15.17:3300/0 conn(0x7f52d80a5210 0x7f52d80a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 --2- 172.21.15.17:0/1430218241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d81378c0 0x7f52d8137ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 >> 172.21.15.17:0/1430218241 conn(0x7f52d80aac40 msgr2=0x7f52d80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 shutdown_connections 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- 172.21.15.17:0/1430218241 wait complete. 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: monmap: 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.824 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient: monmap: 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.815244+0000 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.815244+0000 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 Processor -- start 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 -- start start 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient: init 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 Processor -- start 2024-09-15T21:17:42.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- start start 2024-09-15T21:17:42.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: init 2024-09-15T21:17:42.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa39409d400 0x7fa39409fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): picked mon.noname-c con 0x7fa39409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 0x7fa39409dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): picked mon.noname-b con 0x7fa39409d7e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa394134660 0x7fa394134b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): picked mon.noname-a con 0x7fa394134660 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa39409eaf0 con 0x7fa394134660 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa394074760 con 0x7fa39409d400 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa39409e810 con 0x7fa39409d7e0 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa398cbc640 10 monclient(hunting): authenticate will time out at 986.210626s 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 0x7fa39409dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa392d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa394134660 0x7fa394134b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 0x7fa39409dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45404/0 (socket says 172.21.15.17:45404) 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 -- 172.21.15.17:0/1906000610 learned_addr learned my addr 172.21.15.17:0/1906000610 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): get_auth_request con 0x7fa39409d7e0 auth_method 0 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa392d9d640 10 monclient(hunting): get_auth_request con 0x7fa394134660 auth_method 0 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa392d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa392d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa392d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa39259c640 1 --2- 172.21.15.17:0/1906000610 >> v2:172.21.15.119:3300/0 conn(0x7fa39409d400 0x7fa39409fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa39259c640 10 monclient(hunting): get_auth_request con 0x7fa39409d400 auth_method 0 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa39259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa39259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d80a5210 0x7f52d81cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): picked mon.noname-c con 0x7f52d80a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d81381e0 0x7f52d8136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): picked mon.noname-b con 0x7f52d81381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52d813cd70 0x7f52d813d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): picked mon.noname-a con 0x7f52d813cd70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f52d806f340 con 0x7f52d813cd70 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52d806f1c0 con 0x7f52d81381e0 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f52d81425c0 con 0x7f52d80a5210 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52e0686640 10 monclient(hunting): authenticate will time out at 986.210982s 2024-09-15T21:17:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa39259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient(hunting): handle_auth_done global_id 4232 payload 274 2024-09-15T21:17:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 -- 172.21.15.17:0/1906000610 >> v2:172.21.15.119:3300/0 conn(0x7fa39409d400 msgr2=0x7fa39409fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 --2- 172.21.15.17:0/1906000610 >> v2:172.21.15.119:3300/0 conn(0x7fa39409d400 0x7fa39409fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d80a5210 0x7f52d81cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 -- 172.21.15.17:0/1906000610 >> v2:172.21.15.17:3300/0 conn(0x7fa394134660 msgr2=0x7fa394134b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 --2- 172.21.15.17:0/1906000610 >> v2:172.21.15.17:3300/0 conn(0x7fa394134660 0x7fa394134b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa391d9b640 1 -- 172.21.15.17:0/1906000610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3940a1c20 con 0x7fa39409d7e0 2024-09-15T21:17:42.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7fa39259c640 1 --2- 172.21.15.17:0/1906000610 >> v2:172.21.15.119:3300/0 conn(0x7fa39409d400 0x7fa39409fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d80a5210 0x7f52d81cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45412/0 (socket says 172.21.15.17:45412) 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 1 -- 172.21.15.17:0/1096894773 learned_addr learned my addr 172.21.15.17:0/1096894773 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): get_auth_request con 0x7f52d80a5210 auth_method 0 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d81381e0 0x7f52d8136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): get_auth_request con 0x7f52d81381e0 auth_method 0 2024-09-15T21:17:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52d813cd70 0x7f52d813d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): get_auth_request con 0x7f52d813cd70 auth_method 0 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52de422640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52ddc21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.810+0000 7f52dec23640 10 monclient(hunting): handle_auth_done global_id 4230 payload 824 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 -- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d80a5210 msgr2=0x7f52d81cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52d80a5210 0x7f52d81cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 -- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d81381e0 msgr2=0x7f52d8136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d81381e0 0x7f52d8136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 monclient: found mon.noname-a 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 -- 172.21.15.17:0/1096894773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81215a0 con 0x7f52d813cd70 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ddc21640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d81381e0 0x7f52d8136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dec23640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52d813cd70 0x7f52d813d1c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52d0007aa0 tx=0x7f52d0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d001a070 con 0x7f52d813cd70 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: dump: 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d000dc70 con 0x7f52d813cd70 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 5 monclient: authenticate success, global_id 4230 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0015400 con 0x7f52d813cd70 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52dcc1f640 4 set_mon_vals no callback set 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: dump: 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 10 monclient: _renew_subs 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d80a12f0 con 0x7f52d813cd70 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 10 monclient: _renew_subs 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81212c0 con 0x7f52d813cd70 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f52d001d070 con 0x7f52d813cd70 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52d8137ca0 con 0x7f52d813cd70 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f52bc03b9c0 0x7f52bc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f52d004fad0 con 0x7f52d813cd70 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7f52bc041830 0x7f52bc043c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 --> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52bc044340 con 0x7f52bc041830 2024-09-15T21:17:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f52d00143a0 con 0x7f52d813cd70 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ceffd640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52de422640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f52bc03b9c0 0x7f52bc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52de422640 10 monclient: get_auth_request con 0x7f52bc03b9c0 auth_method 0 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ddc21640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7f52bc041830 0x7f52bc043c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ddc21640 10 monclient: get_auth_request con 0x7f52bc041830 auth_method 0 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52ddc21640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7f52bc041830 0x7f52bc043c70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.814+0000 7f52de422640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f52bc03b9c0 0x7f52bc03de80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f52d401aa10 tx=0x7f52d40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.818+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== osd.2 v2:172.21.15.17:6800/671096025 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52bc044340 con 0x7f52bc041830 2024-09-15T21:17:42.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 1 --2- 172.21.15.17:0/1906000610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 0x7fa39409dbc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3940a6ed0 tx=0x7fa38401c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a48a42ae502241a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 1 -- 172.21.15.17:0/1906000610 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3840b2070 con 0x7fa39409d7e0 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: dump: 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 5 monclient: authenticate success, global_id 4232 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 1 -- 172.21.15.17:0/1906000610 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3840a5810 con 0x7fa39409d7e0 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 4 set_mon_vals no callback set 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 1 -- 172.21.15.17:0/1906000610 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3840a5be0 con 0x7fa39409d7e0 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39159a640 10 monclient: dump: 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: shutdown 2024-09-15T21:17:42.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- 172.21.15.17:0/1906000610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 msgr2=0x7fa39409dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- 172.21.15.17:0/1906000610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 0x7fa39409dbc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3940a6ed0 tx=0x7fa38401c870 comp rx=0 tx=0).stop 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- 172.21.15.17:0/1906000610 shutdown_connections 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- 172.21.15.17:0/1906000610 >> v2:172.21.15.17:3300/0 conn(0x7fa394134660 0x7fa394134b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- 172.21.15.17:0/1906000610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa39409d7e0 0x7fa39409dbc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- 172.21.15.17:0/1906000610 >> v2:172.21.15.119:3300/0 conn(0x7fa39409d400 0x7fa39409fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- 172.21.15.17:0/1906000610 >> 172.21.15.17:0/1906000610 conn(0x7fa3940a6490 msgr2=0x7fa3940a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- 172.21.15.17:0/1906000610 shutdown_connections 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- 172.21.15.17:0/1906000610 wait complete. 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: monmap: 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.850 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 Processor -- start 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- start start 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: init 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:42.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa39409d400 0x7fa394132da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): picked mon.noname-a con 0x7fa39409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa394134660 0x7fa3941332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): picked mon.noname-b con 0x7fa394134660 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa394133820 0x7fa39411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): picked mon.noname-c con 0x7fa394133820 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa3940a6c80 con 0x7fa39409d400 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3940a6b00 con 0x7fa394134660 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa3940a10a0 con 0x7fa394133820 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa398cbc640 10 monclient(hunting): authenticate will time out at 986.222820s 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa39409d400 0x7fa394132da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa39409d400 0x7fa394132da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44094/0 (socket says 172.21.15.17:44094) 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 1 -- 172.21.15.17:0/3792968714 learned_addr learned my addr 172.21.15.17:0/3792968714 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): get_auth_request con 0x7fa39409d400 auth_method 0 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa394133820 0x7fa39411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): get_auth_request con 0x7fa394133820 auth_method 0 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa394134660 0x7fa3941332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 10 monclient(hunting): get_auth_request con 0x7fa394134660 auth_method 0 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient(hunting): handle_auth_done global_id 4238 payload 824 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 -- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa394134660 msgr2=0x7fa3941332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa394134660 0x7fa3941332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 -- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa39409d400 msgr2=0x7fa394132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa39409d400 0x7fa394132da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 monclient: found mon.noname-c 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 -- 172.21.15.17:0/3792968714 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39411eba0 con 0x7fa394133820 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa39259c640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa39409d400 0x7fa394132da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa391d9b640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa394134660 0x7fa3941332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa392d9d640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa394133820 0x7fa39411c4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa38c008590 tx=0x7fa38c008a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c01e070 con 0x7fa394133820 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.822+0000 7fa382ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: dump: 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 5 monclient: authenticate success, global_id 4238 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 10 monclient: _renew_subs 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39413b5b0 con 0x7fa394133820 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c003bc0 con 0x7fa394133820 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00be20 con 0x7fa394133820 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: dump: 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.857 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 10 monclient: _renew_subs 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39409c320 con 0x7fa394133820 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa390d99640 4 set_mon_vals no callback set 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa39409d0f0 con 0x7fa394133820 2024-09-15T21:17:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa38c0188d0 con 0x7fa394133820 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa37403b9c0 0x7fa37403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7fa38c0044d0 con 0x7fa394133820 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fa3740417c0 0x7fa374043c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 --> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3740442d0 con 0x7fa3740417c0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa38c05d060 con 0x7fa394133820 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa391d9b640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fa3740417c0 0x7fa374043c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa391d9b640 10 monclient: get_auth_request con 0x7fa3740417c0 auth_method 0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa39259c640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa37403b9c0 0x7fa37403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa39259c640 10 monclient: get_auth_request con 0x7fa37403b9c0 auth_method 0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa391d9b640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fa3740417c0 0x7fa374043c00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa39259c640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa37403b9c0 0x7fa37403de80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa388009e00 tx=0x7fa388008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.826+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== osd.0 v2:172.21.15.17:6808/3272576910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3740442d0 con 0x7fa3740417c0 2024-09-15T21:17:42.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 --> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52d8137ca0 con 0x7f52bc041830 2024-09-15T21:17:42.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52ceffd640 1 -- 172.21.15.17:0/1096894773 <== osd.2 v2:172.21.15.17:6800/671096025 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f52d8137ca0 con 0x7f52bc041830 2024-09-15T21:17:42.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7f52bc041830 msgr2=0x7f52bc043c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7f52bc041830 0x7f52bc043c70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f52bc03b9c0 msgr2=0x7f52bc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f52bc03b9c0 0x7f52bc03de80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f52d401aa10 tx=0x7f52d40a6000 comp rx=0 tx=0).stop 2024-09-15T21:17:42.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 10 monclient: shutdown 2024-09-15T21:17:42.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52d813cd70 msgr2=0x7f52d813d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52e0686640 1 --2- 172.21.15.17:0/1096894773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52d813cd70 0x7f52d813d1c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52d0007aa0 tx=0x7f52d0007f60 comp rx=0 tx=0).stop 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.834+0000 7f52dec23640 1 -- 172.21.15.17:0/1096894773 reap_dead start 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.838+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 shutdown_connections 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.838+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 >> 172.21.15.17:0/1096894773 conn(0x7f52d80aac40 msgr2=0x7f52d80a8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.838+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 shutdown_connections 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.838+0000 7f52e0686640 1 -- 172.21.15.17:0/1096894773 wait complete. 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 --> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3940a1490 con 0x7fa3740417c0 2024-09-15T21:17:42.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa382ffd640 1 -- 172.21.15.17:0/3792968714 <== osd.0 v2:172.21.15.17:6808/3272576910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3940a1490 con 0x7fa3740417c0 2024-09-15T21:17:42.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fa3740417c0 msgr2=0x7fa374043c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fa3740417c0 0x7fa374043c00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa37403b9c0 msgr2=0x7fa37403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa37403b9c0 0x7fa37403de80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa388009e00 tx=0x7fa388008040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 10 monclient: shutdown 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa394133820 msgr2=0x7fa39411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 --2- 172.21.15.17:0/3792968714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa394133820 0x7fa39411c4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa38c008590 tx=0x7fa38c008a50 comp rx=0 tx=0).stop 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa392d9d640 1 -- 172.21.15.17:0/3792968714 reap_dead start 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 shutdown_connections 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.842+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 >> 172.21.15.17:0/3792968714 conn(0x7fa3940a6490 msgr2=0x7fa3940a5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.846+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 shutdown_connections 2024-09-15T21:17:42.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.846+0000 7fa398cbc640 1 -- 172.21.15.17:0/3792968714 wait complete. 2024-09-15T21:17:42.866 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:42.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:17:42.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.862+0000 7f08642a6640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.862+0000 7f08642a6640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.862+0000 7f08642a6640 10 monclient: monmap: 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.870026+0000 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.870026+0000 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.873 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.874 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 1 Processor -- start 2024-09-15T21:17:42.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 1 -- start start 2024-09-15T21:17:42.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient: init 2024-09-15T21:17:42.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f085c1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f085c1383b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f085c13e4a0 0x7f085c140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient(hunting): picked mon.noname-c con 0x7f085c13e4a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0a1a20 con 0x7f085c1378c0 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0a1d00 con 0x7f085c1383b0 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f085c0a1fe0 con 0x7f085c13e4a0 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f08642a6640 10 monclient(hunting): authenticate will time out at 986.265241s 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862042640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f085c1378c0 0x7f085c137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54620/0 (socket says 172.21.15.17:54620) 2024-09-15T21:17:42.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862042640 1 -- 172.21.15.17:0/1865819764 learned_addr learned my addr 172.21.15.17:0/1865819764 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 1 --2- 172.21.15.17:0/1865819764 >> v2:172.21.15.119:3300/0 conn(0x7f085c13e4a0 0x7f085c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): get_auth_request con 0x7f085c13e4a0 auth_method 0 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694113202 0 0) 0x7f0858001120 con 0x7f085c1378c0 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): _negotiate 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): my global_id is 4236 2024-09-15T21:17:42.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0848003530 con 0x7f085c1378c0 2024-09-15T21:17:42.878 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:42.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:42.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290731103 0 0) 0x7f085c0a1d00 con 0x7f085c1383b0 2024-09-15T21:17:42.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): _negotiate 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 10 monclient(hunting): my global_id is 4267 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0858001120 con 0x7f085c1383b0 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient(hunting): handle_auth_done global_id 4264 payload 274 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 1 -- 172.21.15.17:0/1865819764 >> v1:172.21.15.119:6789/0 conn(0x7f085c1383b0 legacy=0x7f085c13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 1 -- 172.21.15.17:0/1865819764 >> v1:172.21.15.17:6789/0 conn(0x7f085c1378c0 legacy=0x7f085c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 1 monclient: found mon.noname-c 2024-09-15T21:17:42.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:17:42.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7f0862843640 1 -- 172.21.15.17:0/1865819764 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c0a22c0 con 0x7f085c13e4a0 2024-09-15T21:17:42.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7fe47f67c640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7fe47f67c640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.866+0000 7fe47f67c640 10 monclient: monmap: 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.873544+0000 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.873544+0000 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.882 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 1 Processor -- start 2024-09-15T21:17:42.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 1 -- start start 2024-09-15T21:17:42.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient: init 2024-09-15T21:17:42.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2024-09-15T21:17:42.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0862843640 1 --2- 172.21.15.17:0/1865819764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 0x7f085c140890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0854009dc0 tx=0x7f085401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb52b645794af47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe4780a28b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient(hunting): picked mon.noname-b con 0x7fe47809ba60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe4780a33a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe47809cc50 con 0x7fe4780a28b0 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe47806d0d0 con 0x7fe4780a33a0 2024-09-15T21:17:42.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe47809dbc0 con 0x7fe47809ba60 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47f67c640 10 monclient(hunting): authenticate will time out at 986.268905s 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47d418640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe4780a28b0 0x7fe4780a2c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54634/0 (socket says 172.21.15.17:54634) 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47d418640 1 -- 172.21.15.17:0/669443649 learned_addr learned my addr 172.21.15.17:0/669443649 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540b1070 con 0x7f085c13e4a0 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:42.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 1 --2- 172.21.15.17:0/669443649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: dump: 2024-09-15T21:17:42.886 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.886 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.886 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 5 monclient: authenticate success, global_id 4264 2024-09-15T21:17:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): get_auth_request con 0x7fe47809ba60 auth_method 0 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08540a4db0 con 0x7f085c13e4a0 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 1 -- 172.21.15.17:0/1865819764 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540ac640 con 0x7f085c13e4a0 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f0861040640 10 monclient: dump: 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.890 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 4 set_mon_vals no callback set 2024-09-15T21:17:42.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe46ffff640 1 -- 172.21.15.17:0/669443649 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207125874 0 0) 0x7fe47809cc50 con 0x7fe4780a28b0 2024-09-15T21:17:42.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe46ffff640 10 monclient(hunting): _negotiate 2024-09-15T21:17:42.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe46ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe46ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe46ffff640 10 monclient(hunting): my global_id is 4239 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe46ffff640 1 -- 172.21.15.17:0/669443649 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe464003530 con 0x7fe4780a28b0 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient(hunting): handle_auth_done global_id 4241 payload 274 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 1 -- 172.21.15.17:0/669443649 >> v1:172.21.15.119:6789/0 conn(0x7fe4780a33a0 legacy=0x7fe478143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 1 -- 172.21.15.17:0/669443649 >> v1:172.21.15.17:6789/0 conn(0x7fe4780a28b0 legacy=0x7fe4780a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 1 -- 172.21.15.17:0/669443649 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47806ce50 con 0x7fe47809ba60 2024-09-15T21:17:42.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: shutdown 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 -- 172.21.15.17:0/1865819764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 msgr2=0x7f085c140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 --2- 172.21.15.17:0/1865819764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 0x7f085c140890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0854009dc0 tx=0x7f085401c4f0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 -- 172.21.15.17:0/1865819764 shutdown_connections 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 --2- 172.21.15.17:0/1865819764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 0x7f085c140890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 -- 172.21.15.17:0/1865819764 >> 172.21.15.17:0/1865819764 conn(0x7f085c0aac40 msgr2=0x7f085c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 -- 172.21.15.17:0/1865819764 shutdown_connections 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 -- 172.21.15.17:0/1865819764 wait complete. 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: monmap: 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.894 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 Processor -- start 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 -- start start 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: init 2024-09-15T21:17:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7fe47cc17640 1 --2- 172.21.15.17:0/669443649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781412b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe4740097f0 tx=0x7fe47401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ecf66fb89a820b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:17:42.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f085c1378c0 0x7f085c09ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient(hunting): picked mon.noname-a con 0x7f085c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f085c1383b0 0x7f085c1368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.870+0000 7f08642a6640 10 monclient(hunting): picked mon.noname-c con 0x7f085c1383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 0x7f085c09c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient(hunting): picked mon.noname-b con 0x7f085c13e4a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f085c144280 con 0x7f085c1378c0 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f085c144100 con 0x7f085c13e4a0 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f085c144400 con 0x7f085c1383b0 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient(hunting): authenticate will time out at 986.272050s 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 1 -- 172.21.15.17:0/669443649 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740b0070 con 0x7fe47809ba60 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: dump: 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.898 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 1 -- 172.21.15.17:0/669443649 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4740ab600 con 0x7fe47809ba60 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 5 monclient: authenticate success, global_id 4241 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 4 set_mon_vals no callback set 2024-09-15T21:17:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 1 -- 172.21.15.17:0/669443649 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740ab9d0 con 0x7fe47809ba60 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe46ffff640 10 monclient: dump: 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.901 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.901 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.901 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.901 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f085c1383b0 0x7f085c1368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f085c1383b0 0x7f085c1368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45430/0 (socket says 172.21.15.17:45430) 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 -- 172.21.15.17:0/1386938576 learned_addr learned my addr 172.21.15.17:0/1386938576 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): get_auth_request con 0x7f085c1383b0 auth_method 0 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f085c1378c0 0x7f085c09ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): get_auth_request con 0x7f085c1378c0 auth_method 0 2024-09-15T21:17:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 0x7f085c09c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): get_auth_request con 0x7f085c13e4a0 auth_method 0 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient(hunting): handle_auth_done global_id 4244 payload 824 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 -- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 msgr2=0x7f085c09c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c13e4a0 0x7f085c09c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 -- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f085c1378c0 msgr2=0x7f085c09ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f085c1378c0 0x7f085c09ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 monclient: found mon.noname-c 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 -- 172.21.15.17:0/1386938576 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c09f7c0 con 0x7f085c1383b0 2024-09-15T21:17:42.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f085c1378c0 0x7f085c09ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:42.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0861841640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f085c1383b0 0x7f085c1368b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f084c00c490 tx=0x7f084c00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: shutdown 2024-09-15T21:17:42.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- 172.21.15.17:0/669443649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 msgr2=0x7fe4781412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 --2- 172.21.15.17:0/669443649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781412b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe4740097f0 tx=0x7fe47401c430 comp rx=0 tx=0).stop 2024-09-15T21:17:42.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- 172.21.15.17:0/669443649 shutdown_connections 2024-09-15T21:17:42.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 --2- 172.21.15.17:0/669443649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781412b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- 172.21.15.17:0/669443649 >> 172.21.15.17:0/669443649 conn(0x7fe4780a74a0 msgr2=0x7fe4780a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- 172.21.15.17:0/669443649 shutdown_connections 2024-09-15T21:17:42.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- 172.21.15.17:0/669443649 wait complete. 2024-09-15T21:17:42.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0052d0 con 0x7f085c1383b0 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: dump: 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.907 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c005490 con 0x7f085c1383b0 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 5 monclient: authenticate success, global_id 4244 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient: _renew_subs 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c0a0630 con 0x7f085c1383b0 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient: _renew_subs 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c0a0350 con 0x7f085c1383b0 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: monmap: 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 Processor -- start 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f07eef20640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f07eef20640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f07eef20640 10 monclient: monmap: 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.880574+0000 2024-09-15T21:17:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.880574+0000 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.911 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- start start 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: init 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): picked mon.noname-c con 0x7fe47809ba60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780a28b0 0x7fe47813bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): picked mon.noname-a con 0x7fe4780a28b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4780a33a0 0x7fe4781400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): picked mon.noname-b con 0x7fe4780a33a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe478147ad0 con 0x7fe4780a28b0 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe478147950 con 0x7fe4780a33a0 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe478147c50 con 0x7fe47809ba60 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47f67c640 10 monclient(hunting): authenticate will time out at 986.274927s 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c012910 con 0x7f085c1383b0 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: dump: 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f086083f640 4 set_mon_vals no callback set 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f085c0a12f0 con 0x7f085c1383b0 2024-09-15T21:17:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f084c012ab0 con 0x7f085c1383b0 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780a28b0 0x7fe47813bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4780a33a0 0x7fe4781400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780a28b0 0x7fe47813bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44110/0 (socket says 172.21.15.17:44110) 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): get_auth_request con 0x7fe47809ba60 auth_method 0 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 1 -- 172.21.15.17:0/2778018851 learned_addr learned my addr 172.21.15.17:0/2778018851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): get_auth_request con 0x7fe4780a33a0 auth_method 0 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f084003b9c0 0x7f084003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): get_auth_request con 0x7fe4780a28b0 auth_method 0 2024-09-15T21:17:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f07eef20640 1 Processor -- start 2024-09-15T21:17:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f084c04f690 con 0x7f085c1383b0 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47dc19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f0840041830 0x7f0840043c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 --> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0840044340 con 0x7f0840041830 2024-09-15T21:17:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f084c056050 con 0x7f085c1383b0 2024-09-15T21:17:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0852ffd640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-09-15T21:17:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient(hunting): handle_auth_done global_id 4247 payload 824 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 -- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4780a33a0 msgr2=0x7fe4781400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4780a33a0 0x7fe4781400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 -- 172.21.15.17:0/2778018851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780a28b0 msgr2=0x7fe47813bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780a28b0 0x7fe47813bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 monclient: found mon.noname-c 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 -- 172.21.15.17:0/2778018851 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe478143070 con 0x7fe47809ba60 2024-09-15T21:17:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47cc17640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780a28b0 0x7fe47813bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f07eef20640 1 -- start start 2024-09-15T21:17:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862042640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f084003b9c0 0x7f084003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f0862843640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f0840041830 0x7f0840043c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f0862843640 10 monclient: get_auth_request con 0x7f0840041830 auth_method 0 2024-09-15T21:17:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7fe47d418640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781426e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe47000cbc0 tx=0x7fe47000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f0862042640 10 monclient: get_auth_request con 0x7f084003b9c0 auth_method 0 2024-09-15T21:17:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.874+0000 7f07eef20640 10 monclient: init 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f00ad0e2640 10 monclient: get_monmap_and_config 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f00ad0e2640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f00ad0e2640 10 monclient: monmap: 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:42.882388+0000 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:42.882388+0000 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:42.925 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47000aca0 con 0x7fe47809ba60 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: dump: 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47000ae40 con 0x7fe47809ba60 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470012600 con 0x7fe47809ba60 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: dump: 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46f7fe640 4 set_mon_vals no callback set 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 5 monclient: authenticate success, global_id 4247 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 10 monclient: _renew_subs 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe478143ee0 con 0x7fe47809ba60 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 10 monclient: _renew_subs 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe478144230 con 0x7fe47809ba60 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f07e80a2820 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eccbc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07e80a2820 0x7f07e80a2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54640/0 (socket says 172.21.15.17:54640) 2024-09-15T21:17:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eccbc640 1 -- 172.21.15.17:0/276989105 learned_addr learned my addr 172.21.15.17:0/276989105 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f0862843640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f0840041830 0x7f0840043c70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f0862042640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f084003b9c0 0x7f084003de80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0858000c10 tx=0x7f0858009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 1 --2- 172.21.15.17:0/276989105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e809baf0 0x7f07e8141290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): picked mon.noname-b con 0x7f07e809baf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 1 --2- 172.21.15.17:0/276989105 >> v2:172.21.15.119:3300/0 conn(0x7f07e80a3310 0x7f07e81437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4780a2c90 con 0x7fe47809ba60 2024-09-15T21:17:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): picked mon.noname-c con 0x7f07e80a3310 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 --2- 172.21.15.17:0/276989105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e809baf0 0x7f07e8141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e806d410 con 0x7f07e80a2820 2024-09-15T21:17:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f07e806ccc0 con 0x7f07e80a3310 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07e809c220 con 0x7f07e809baf0 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07eef20640 10 monclient(hunting): authenticate will time out at 986.277401s 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): get_auth_request con 0x7f07e809baf0 auth_method 0 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07ed4bd640 1 --2- 172.21.15.17:0/276989105 >> v2:172.21.15.119:3300/0 conn(0x7f07e80a3310 0x7f07e81437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe4700127a0 con 0x7fe47809ba60 2024-09-15T21:17:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07ed4bd640 10 monclient(hunting): get_auth_request con 0x7f07e80a3310 auth_method 0 2024-09-15T21:17:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07ed4bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07ed4bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07ed4bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe45c03b9f0 0x7fe45c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7fe4700519a0 con 0x7fe47809ba60 2024-09-15T21:17:42.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7fe45c041860 0x7fe45c043ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 --> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe47001be50 con 0x7fe45c041860 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fe470055060 con 0x7fe47809ba60 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe46dffb640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47cc17640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe45c03b9f0 0x7fe45c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47cc17640 10 monclient: get_auth_request con 0x7fe45c03b9f0 auth_method 0 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47dc19640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7fe45c041860 0x7fe45c043ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47dc19640 10 monclient: get_auth_request con 0x7fe45c041860 auth_method 0 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47cc17640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe45c03b9f0 0x7fe45c03deb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe47401ccd0 tx=0x7fe474008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7fe47dc19640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7fe45c041860 0x7fe45c043ca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== osd.4 v2:172.21.15.119:6816/2086984234 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0840044340 con 0x7f0840041830 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient(hunting): handle_auth_done global_id 4250 payload 274 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 -- 172.21.15.17:0/276989105 >> v2:172.21.15.119:3300/0 conn(0x7f07e80a3310 msgr2=0x7f07e81437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 --2- 172.21.15.17:0/276989105 >> v2:172.21.15.119:3300/0 conn(0x7f07e80a3310 0x7f07e81437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e77fe640 1 -- 172.21.15.17:0/276989105 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819475158 0 0) 0x7f07e806d410 con 0x7f07e80a2820 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 -- 172.21.15.17:0/276989105 >> v1:172.21.15.17:6789/0 conn(0x7f07e80a2820 legacy=0x7f07e80a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 -- 172.21.15.17:0/276989105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e809d4f0 con 0x7f07e809baf0 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e77fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.878+0000 7f07e7fff640 1 --2- 172.21.15.17:0/276989105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e809baf0 0x7f07e8141290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f07d80097c0 tx=0x7f07d801c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d8dd6880012cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 1 Processor -- start 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 1 -- start start 2024-09-15T21:17:42.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient: init 2024-09-15T21:17:42.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 1 -- 172.21.15.17:0/276989105 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b0070 con 0x7f07e809baf0 2024-09-15T21:17:42.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: dump: 2024-09-15T21:17:42.939 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.939 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 1 -- 172.21.15.17:0/276989105 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d80ab600 con 0x7f07e809baf0 2024-09-15T21:17:42.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 5 monclient: authenticate success, global_id 4250 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 4 set_mon_vals no callback set 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 1 -- 172.21.15.17:0/276989105 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80ab9d0 con 0x7f07e809baf0 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e77fe640 10 monclient: dump: 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.942 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== osd.3 v2:172.21.15.119:6800/3353277098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe47001be50 con 0x7fe45c041860 2024-09-15T21:17:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f00a80a3a10 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a80a4500 0x7f00a813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient(hunting): picked mon.noname-b con 0x7f00a80a4500 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f00a813d120 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a806e3d0 con 0x7f00a80a3a10 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a806ea90 con 0x7f00a813d120 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f00a806fb30 con 0x7f00a80a4500 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00ad0e2640 10 monclient(hunting): authenticate will time out at 986.281090s 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: shutdown 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e809baf0 msgr2=0x7f07e8141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 --2- 172.21.15.17:0/276989105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e809baf0 0x7f07e8141290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f07d80097c0 tx=0x7f07d801c360 comp rx=0 tx=0).stop 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 shutdown_connections 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 --2- 172.21.15.17:0/276989105 >> v2:172.21.15.119:3300/0 conn(0x7f07e80a3310 0x7f07e81437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 --2- 172.21.15.17:0/276989105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e809baf0 0x7f07e8141290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 >> 172.21.15.17:0/276989105 conn(0x7f07e80a75d0 msgr2=0x7f07e80a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 shutdown_connections 2024-09-15T21:17:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f00a80a3a10 0x7f00a80a3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54652/0 (socket says 172.21.15.17:54652) 2024-09-15T21:17:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a6d9d640 1 -- 172.21.15.17:0/1507344977 learned_addr learned my addr 172.21.15.17:0/1507344977 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 1 --2- 172.21.15.17:0/1507344977 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a80a4500 0x7f00a813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): get_auth_request con 0x7f00a80a4500 auth_method 0 2024-09-15T21:17:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient(hunting): handle_auth_done global_id 4253 payload 274 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 1 -- 172.21.15.17:0/1507344977 >> v1:172.21.15.119:6789/0 conn(0x7f00a813d120 legacy=0x7f00a813f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 1 -- 172.21.15.17:0/1507344977 >> v1:172.21.15.17:6789/0 conn(0x7f00a80a3a10 legacy=0x7f00a80a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 1 monclient: found mon.noname-b 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f00a659c640 1 -- 172.21.15.17:0/1507344977 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a806ecd0 con 0x7f00a80a4500 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- 172.21.15.17:0/276989105 wait complete. 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: monmap: 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 Processor -- start 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- start start 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: init 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e809baf0 0x7f07e8139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): picked mon.noname-a con 0x7f07e809baf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e80a2820 0x7f07e813a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): picked mon.noname-c con 0x7f07e80a2820 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07e80a3310 0x7f07e813a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): picked mon.noname-b con 0x7f07e80a3310 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f07e81468d0 con 0x7f07e809baf0 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8146750 con 0x7f07e80a3310 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07e8146a50 con 0x7f07e80a2820 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eef20640 10 monclient(hunting): authenticate will time out at 986.282876s 2024-09-15T21:17:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e809baf0 0x7f07e8139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e809baf0 0x7f07e8139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44124/0 (socket says 172.21.15.17:44124) 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 -- 172.21.15.17:0/2994466146 learned_addr learned my addr 172.21.15.17:0/2994466146 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): get_auth_request con 0x7f07e809baf0 auth_method 0 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e80a2820 0x7f07e813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07e80a3310 0x7f07e813a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): get_auth_request con 0x7f07e80a2820 auth_method 0 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): get_auth_request con 0x7f07e80a3310 auth_method 0 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07ed4bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient(hunting): handle_auth_done global_id 4254 payload 824 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 -- 172.21.15.17:0/2994466146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e80a2820 msgr2=0x7f07e813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07e80a2820 0x7f07e813a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 -- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07e80a3310 msgr2=0x7f07e813a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07e80a3310 0x7f07e813a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 monclient: found mon.noname-a 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 -- 172.21.15.17:0/2994466146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e813d810 con 0x7f07e809baf0 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.882+0000 7f07eccbc640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e809baf0 0x7f07e8139bc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c870 tx=0x7f07e000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0018070 con 0x7f07e809baf0 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: dump: 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e000a640 con 0x7f07e809baf0 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e000aa30 con 0x7f07e809baf0 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: dump: 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 5 monclient: authenticate success, global_id 4254 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 10 monclient: _renew_subs 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e813daf0 con 0x7f07e809baf0 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 10 monclient: _renew_subs 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e809e680 con 0x7f07e809baf0 2024-09-15T21:17:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e6ffd640 4 set_mon_vals no callback set 2024-09-15T21:17:42.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a659c640 1 --2- 172.21.15.17:0/1507344977 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a80a4500 0x7f00a813cbe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f009c009820 tx=0x7f009c01bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f569779be8b534f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 1 -- 172.21.15.17:0/1507344977 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c0ab070 con 0x7f00a80a4500 2024-09-15T21:17:42.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:42.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: dump: 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 1 -- 172.21.15.17:0/1507344977 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c0a56a0 con 0x7f00a80a4500 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 5 monclient: authenticate success, global_id 4253 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 1 -- 172.21.15.17:0/1507344977 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c0a5a70 con 0x7f00a80a4500 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 4 set_mon_vals no callback set 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00a5d9b640 10 monclient: dump: 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 10 monclient: shutdown 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 -- 172.21.15.17:0/1507344977 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a80a4500 msgr2=0x7f00a813cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 --2- 172.21.15.17:0/1507344977 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a80a4500 0x7f00a813cbe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f009c009820 tx=0x7f009c01bf20 comp rx=0 tx=0).stop 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 -- 172.21.15.17:0/1507344977 shutdown_connections 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 --2- 172.21.15.17:0/1507344977 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a80a4500 0x7f00a813cbe0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 -- 172.21.15.17:0/1507344977 >> 172.21.15.17:0/1507344977 conn(0x7f00a80a8980 msgr2=0x7f00a80a4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 -- 172.21.15.17:0/1507344977 shutdown_connections 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 1 -- 172.21.15.17:0/1507344977 wait complete. 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 10 monclient: build_initial_monmap 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f00ad0e2640 10 monclient: monmap: 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:42.961 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07c8000f90 con 0x7f07e809baf0 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f07e001b070 con 0x7f07e809baf0 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07d403ba10 0x7f07d403ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f07e0007980 con 0x7f07e809baf0 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f07d4041810 0x7f07d4043c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07e0009eb0 con 0x7f07d4041810 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f07e0054060 con 0x7f07e809baf0 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e57fa640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-09-15T21:17:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e7fff640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07d403ba10 0x7f07d403ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e7fff640 10 monclient: get_auth_request con 0x7f07d403ba10 auth_method 0 2024-09-15T21:17:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07ed4bd640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f07d4041810 0x7f07d4043c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07ed4bd640 10 monclient: get_auth_request con 0x7f07d4041810 auth_method 0 2024-09-15T21:17:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07e7fff640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07d403ba10 0x7f07d403ded0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07d80ad000 tx=0x7f07d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.886+0000 7f07ed4bd640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f07d4041810 0x7f07d4043c50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 Processor -- start 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 -- start start 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient: init 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a80a3a10 0x7f00a813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): picked mon.noname-a con 0x7f00a80a3a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a813d120 0x7f00a813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): picked mon.noname-b con 0x7f00a813d120 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a81d0680 0x7f00a81d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): picked mon.noname-c con 0x7f00a81d0680 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8144030 con 0x7f00a80a3a10 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8143eb0 con 0x7f00a813d120 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f00a81441b0 con 0x7f00a81d0680 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00ad0e2640 10 monclient(hunting): authenticate will time out at 986.289633s 2024-09-15T21:17:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== osd.5 v2:172.21.15.119:6808/444741843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07e0009eb0 con 0x7f07d4041810 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a80a3a10 0x7f00a813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a80a3a10 0x7f00a813c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44132/0 (socket says 172.21.15.17:44132) 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 -- 172.21.15.17:0/3160060887 learned_addr learned my addr 172.21.15.17:0/3160060887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): get_auth_request con 0x7f00a80a3a10 auth_method 0 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a813d120 0x7f00a813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): get_auth_request con 0x7f00a813d120 auth_method 0 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a759e640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a81d0680 0x7f00a81d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2024-09-15T21:17:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 -- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a81d0680 msgr2=0x7f00a81d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a81d0680 0x7f00a81d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 -- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a813d120 msgr2=0x7f00a813c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a813d120 0x7f00a813c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 monclient: found mon.noname-a 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 -- 172.21.15.17:0/3160060887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a81248d0 con 0x7f00a80a3a10 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a759e640 10 monclient: get_auth_request con 0x7f00a81d0680 auth_method 0 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a759e640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00a81d0680 0x7f00a81d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a6d9d640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a80a3a10 0x7f00a813c0b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f009800c780 tx=0x7f009800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098017070 con 0x7f00a80a3a10 2024-09-15T21:17:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: dump: 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0098005ac0 con 0x7f00a80a3a10 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098007070 con 0x7f00a80a3a10 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f008f7fe640 10 monclient: dump: 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.890+0000 7f00a559a640 4 set_mon_vals no callback set 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 5 monclient: authenticate success, global_id 4257 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 10 monclient: _renew_subs 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a8125d00 con 0x7f00a80a3a10 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 10 monclient: _renew_subs 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a809f1f0 con 0x7f00a80a3a10 2024-09-15T21:17:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00a80a3e70 con 0x7f00a80a3a10 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f009801a070 con 0x7f00a80a3a10 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f009403b9c0 0x7f009403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f00980583a0 con 0x7f00a80a3a10 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f0094041830 0x7f0094043c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 --> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0094044340 con 0x7f0094041830 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0098054060 con 0x7f00a80a3a10 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00a759e640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f0094041830 0x7f0094043c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00a759e640 10 monclient: get_auth_request con 0x7f0094041830 auth_method 0 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f00a759e640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f0094041830 0x7f0094043c70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.894+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== osd.1 v2:172.21.15.17:6816/2555706928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0094044340 con 0x7f0094041830 2024-09-15T21:17:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 --> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f085c137ca0 con 0x7f0840041830 2024-09-15T21:17:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f0852ffd640 1 -- 172.21.15.17:0/1386938576 <== osd.4 v2:172.21.15.119:6816/2086984234 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f085c137ca0 con 0x7f0840041830 2024-09-15T21:17:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 --> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4781d4b20 con 0x7fe45c041860 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f0840041830 msgr2=0x7f0840043c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f0840041830 0x7f0840043c70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f084003b9c0 msgr2=0x7f084003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f084003b9c0 0x7f084003de80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0858000c10 tx=0x7f0858009040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 10 monclient: shutdown 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f085c1383b0 msgr2=0x7f085c1368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 --2- 172.21.15.17:0/1386938576 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f085c1383b0 0x7f085c1368b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f084c00c490 tx=0x7f084c00c950 comp rx=0 tx=0).stop 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f0862843640 1 -- 172.21.15.17:0/1386938576 reap_dead start 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 shutdown_connections 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 >> 172.21.15.17:0/1386938576 conn(0x7f085c0aac40 msgr2=0x7f085c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 shutdown_connections 2024-09-15T21:17:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7f08642a6640 1 -- 172.21.15.17:0/1386938576 wait complete. 2024-09-15T21:17:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.898+0000 7fe46dffb640 1 -- 172.21.15.17:0/2778018851 <== osd.3 v2:172.21.15.119:6800/3353277098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe4781d4b20 con 0x7fe45c041860 2024-09-15T21:17:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7fe45c041860 msgr2=0x7fe45c043ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7fe45c041860 0x7fe45c043ca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe45c03b9f0 msgr2=0x7fe45c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe45c03b9f0 0x7fe45c03deb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe47401ccd0 tx=0x7fe474008040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 10 monclient: shutdown 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 msgr2=0x7fe4781426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 --2- 172.21.15.17:0/2778018851 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe47809ba60 0x7fe4781426e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe47000cbc0 tx=0x7fe47000a060 comp rx=0 tx=0).stop 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47dc19640 1 -- 172.21.15.17:0/2778018851 reap_dead start 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 shutdown_connections 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 >> 172.21.15.17:0/2778018851 conn(0x7fe4780a74a0 msgr2=0x7fe47809f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 shutdown_connections 2024-09-15T21:17:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.902+0000 7fe47f67c640 1 -- 172.21.15.17:0/2778018851 wait complete. 2024-09-15T21:17:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07c8002c80 con 0x7f07d4041810 2024-09-15T21:17:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07e57fa640 1 -- 172.21.15.17:0/2994466146 <== osd.5 v2:172.21.15.119:6808/444741843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f07c8002c80 con 0x7f07d4041810 2024-09-15T21:17:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f07d4041810 msgr2=0x7f07d4043c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f07d4041810 0x7f07d4043c50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07d403ba10 msgr2=0x7f07d403ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07d403ba10 0x7f07d403ded0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07d80ad000 tx=0x7f07d8008040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 10 monclient: shutdown 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e809baf0 msgr2=0x7f07e8139bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 --2- 172.21.15.17:0/2994466146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e809baf0 0x7f07e8139bc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c870 tx=0x7f07e000cd30 comp rx=0 tx=0).stop 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07ed4bd640 1 -- 172.21.15.17:0/2994466146 reap_dead start 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 shutdown_connections 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 >> 172.21.15.17:0/2994466146 conn(0x7f07e80a75d0 msgr2=0x7f07e809fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 shutdown_connections 2024-09-15T21:17:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.910+0000 7f07eef20640 1 -- 172.21.15.17:0/2994466146 wait complete. 2024-09-15T21:17:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00a659c640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f009403b9c0 0x7f009403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00a659c640 10 monclient: get_auth_request con 0x7f009403b9c0 auth_method 0 2024-09-15T21:17:42.985 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:42.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:17:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 --> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00a8125980 con 0x7f0094041830 2024-09-15T21:17:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f008f7fe640 1 -- 172.21.15.17:0/3160060887 <== osd.1 v2:172.21.15.17:6816/2555706928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f00a8125980 con 0x7f0094041830 2024-09-15T21:17:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00a659c640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f009403b9c0 0x7f009403de80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f009c0ad000 tx=0x7f009c0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f0094041830 msgr2=0x7f0094043c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f0094041830 0x7f0094043c70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f009403b9c0 msgr2=0x7f009403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f009403b9c0 0x7f009403de80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f009c0ad000 tx=0x7f009c0b2040 comp rx=0 tx=0).stop 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 10 monclient: shutdown 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a80a3a10 msgr2=0x7f00a813c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 --2- 172.21.15.17:0/3160060887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a80a3a10 0x7f00a813c0b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f009800c780 tx=0x7f009800cc40 comp rx=0 tx=0).stop 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00a759e640 1 -- 172.21.15.17:0/3160060887 reap_dead start 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 shutdown_connections 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 >> 172.21.15.17:0/3160060887 conn(0x7f00a80a8980 msgr2=0x7f00a80a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 shutdown_connections 2024-09-15T21:17:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:42.922+0000 7f00ad0e2640 1 -- 172.21.15.17:0/3160060887 wait complete. 2024-09-15T21:17:42.989 INFO:teuthology.orchestra.run.smithi017.stdout:47244640260 2024-09-15T21:17:42.989 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:17:42.990 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:42.990 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:17:42.991 INFO:teuthology.orchestra.run.smithi017.stdout:51539607558 2024-09-15T21:17:42.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:17:43.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.002+0000 7f81b257b640 10 monclient: get_monmap_and_config 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.002+0000 7f81b257b640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.002+0000 7f81b257b640 10 monclient: monmap: 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:43.008977+0000 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:43.008977+0000 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:43.013 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.002+0000 7f81b257b640 1 Processor -- start 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.002+0000 7f81b257b640 1 -- start start 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.002+0000 7f81b257b640 10 monclient: init 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81ac09d9b0 0x7f81ac09dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): picked mon.noname-c con 0x7f81ac09d9b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81ac0a0600 0x7f81ac09e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): picked mon.noname-a con 0x7f81ac0a0600 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f81ac0990b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:43.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81ac09f8d0 con 0x7f81ac0990b0 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f81ac09ff20 con 0x7f81ac0a0600 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f81ac09e810 con 0x7f81ac09d9b0 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b257b640 10 monclient(hunting): authenticate will time out at 986.404630s 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81ac0a0600 0x7f81ac09e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b0b18640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f81ac0990b0 0x7f81ac0994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54662/0 (socket says 172.21.15.17:54662) 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81b0b18640 1 -- 172.21.15.17:0/275985516 learned_addr learned my addr 172.21.15.17:0/275985516 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): get_auth_request con 0x7f81ac0a0600 auth_method 0 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 1 --2- 172.21.15.17:0/275985516 >> v2:172.21.15.119:3300/0 conn(0x7f81ac09d9b0 0x7f81ac09dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): get_auth_request con 0x7f81ac09d9b0 auth_method 0 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_done global_id 4260 payload 274 2024-09-15T21:17:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 1 -- 172.21.15.17:0/275985516 >> v2:172.21.15.119:3300/0 conn(0x7f81ac09d9b0 msgr2=0x7f81ac09dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 1 --2- 172.21.15.17:0/275985516 >> v2:172.21.15.119:3300/0 conn(0x7f81ac09d9b0 0x7f81ac09dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 1 -- 172.21.15.17:0/275985516 >> v1:172.21.15.17:6789/0 conn(0x7f81ac0990b0 legacy=0x7f81ac0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81aaffd640 1 -- 172.21.15.17:0/275985516 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392095719 0 0) 0x7f81ac09f8d0 con 0x7f81ac0990b0 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81ab7fe640 1 -- 172.21.15.17:0/275985516 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac09fbb0 con 0x7f81ac0a0600 2024-09-15T21:17:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.006+0000 7f81aaffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81ab7fe640 1 --2- 172.21.15.17:0/275985516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac0a0600 0x7f81ac09e2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0a4040 tx=0x7f81a401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602eb08db1d91c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 1 -- 172.21.15.17:0/275985516 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40b2070 con 0x7f81ac0a0600 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: dump: 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 5 monclient: authenticate success, global_id 4260 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 1 -- 172.21.15.17:0/275985516 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a40a57c0 con 0x7f81ac0a0600 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 4 set_mon_vals no callback set 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 1 -- 172.21.15.17:0/275985516 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40b2070 con 0x7f81ac0a0600 2024-09-15T21:17:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81aaffd640 10 monclient: dump: 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 10 monclient: shutdown 2024-09-15T21:17:43.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 -- 172.21.15.17:0/275985516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac0a0600 msgr2=0x7f81ac09e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 --2- 172.21.15.17:0/275985516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac0a0600 0x7f81ac09e2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0a4040 tx=0x7f81a401c7f0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 -- 172.21.15.17:0/275985516 shutdown_connections 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 --2- 172.21.15.17:0/275985516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac0a0600 0x7f81ac09e2d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 --2- 172.21.15.17:0/275985516 >> v2:172.21.15.119:3300/0 conn(0x7f81ac09d9b0 0x7f81ac09dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 -- 172.21.15.17:0/275985516 >> 172.21.15.17:0/275985516 conn(0x7f81ac0a66b0 msgr2=0x7f81ac0a6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 -- 172.21.15.17:0/275985516 shutdown_connections 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 -- 172.21.15.17:0/275985516 wait complete. 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 10 monclient: monmap: 2024-09-15T21:17:43.024 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:43.025 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:43.026 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 Processor -- start 2024-09-15T21:17:43.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 1 -- start start 2024-09-15T21:17:43.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.010+0000 7f81b257b640 10 monclient: init 2024-09-15T21:17:43.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0990b0 0x7f81ac1d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): picked mon.noname-b con 0x7f81ac0990b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 0x7f81ac1d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): picked mon.noname-a con 0x7f81ac09d9b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81ac0a0600 0x7f81ac1d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): picked mon.noname-c con 0x7f81ac0a0600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac13e600 con 0x7f81ac09d9b0 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac13e480 con 0x7f81ac0990b0 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f81ac13e780 con 0x7f81ac0a0600 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient(hunting): authenticate will time out at 986.412648s 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 0x7f81ac1d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 0x7f81ac1d43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44154/0 (socket says 172.21.15.17:44154) 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 -- 172.21.15.17:0/2490797546 learned_addr learned my addr 172.21.15.17:0/2490797546 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): get_auth_request con 0x7f81ac09d9b0 auth_method 0 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0990b0 0x7f81ac1d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): get_auth_request con 0x7f81ac0990b0 auth_method 0 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b0b18640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81ac0a0600 0x7f81ac1d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b0b18640 10 monclient(hunting): get_auth_request con 0x7f81ac0a0600 auth_method 0 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b0b18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b0b18640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b0b18640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient(hunting): handle_auth_done global_id 4266 payload 824 2024-09-15T21:17:43.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 -- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81ac0a0600 msgr2=0x7f81ac1d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81ac0a0600 0x7f81ac1d48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 -- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0990b0 msgr2=0x7f81ac1d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0990b0 0x7f81ac1d3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 -- 172.21.15.17:0/2490797546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac1416f0 con 0x7f81ac09d9b0 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81ab7fe640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 0x7f81ac1d43a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f81a40982e0 tx=0x7f81a4019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40bd030 con 0x7f81ac09d9b0 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: dump: 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a40039a0 con 0x7f81ac09d9b0 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40b2070 con 0x7f81ac09d9b0 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 10 monclient: dump: 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.033 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81aa7fc640 4 set_mon_vals no callback set 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 5 monclient: authenticate success, global_id 4266 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _renew_subs 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac09d6a0 con 0x7f81ac09d9b0 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _renew_subs 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac141410 con 0x7f81ac09d9b0 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f81a40b9660 con 0x7f81ac09d9b0 2024-09-15T21:17:43.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f819803b9f0 0x7f819803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f81a40ea470 con 0x7f81ac09d9b0 2024-09-15T21:17:43.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.014+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac0994c0 con 0x7f81ac09d9b0 2024-09-15T21:17:43.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.018+0000 7f81abfff640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f819803b9f0 0x7f819803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.018+0000 7f81abfff640 10 monclient: get_auth_request con 0x7f819803b9f0 auth_method 0 2024-09-15T21:17:43.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.018+0000 7f81abfff640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f819803b9f0 0x7f819803deb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009b60 tx=0x7f81a0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.026+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f81a4016500 con 0x7f81ac09d9b0 2024-09-15T21:17:43.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.026+0000 7f81a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.026+0000 7f81a8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient: get_monmap_and_config 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient: monmap: 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:43.054624+0000 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:43.054624+0000 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:43.059 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 Processor -- start 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 -- start start 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient: init 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f74f81378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 0x7f74f813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): picked mon.noname-b con 0x7f74f81383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f74f813d2c0 0x7f74f813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): picked mon.noname-a con 0x7f74f813d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f80a1a20 con 0x7f74f81378c0 2024-09-15T21:17:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f74f80a1d00 con 0x7f74f813d2c0 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f74f80a1fe0 con 0x7f74f81383b0 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f75004b3640 10 monclient(hunting): authenticate will time out at 986.450301s 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 0x7f74f813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 0x7f74f813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45494/0 (socket says 172.21.15.17:45494) 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 -- 172.21.15.17:0/202512177 learned_addr learned my addr 172.21.15.17:0/202512177 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): get_auth_request con 0x7f74f81383b0 auth_method 0 2024-09-15T21:17:43.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 1 --2- 172.21.15.17:0/202512177 >> v2:172.21.15.17:3300/0 conn(0x7f74f813d2c0 0x7f74f813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): get_auth_request con 0x7f74f813d2c0 auth_method 0 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient(hunting): handle_auth_done global_id 4262 payload 274 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 -- 172.21.15.17:0/202512177 >> v2:172.21.15.17:3300/0 conn(0x7f74f813d2c0 msgr2=0x7f74f813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 --2- 172.21.15.17:0/202512177 >> v2:172.21.15.17:3300/0 conn(0x7f74f813d2c0 0x7f74f813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 -- 172.21.15.17:0/202512177 >> v1:172.21.15.119:6789/0 conn(0x7f74f81378c0 legacy=0x7f74f8137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 monclient: found mon.noname-b 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fda4e640 1 -- 172.21.15.17:0/202512177 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f80a22c0 con 0x7f74f81383b0 2024-09-15T21:17:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.050+0000 7f74fea50640 1 --2- 172.21.15.17:0/202512177 >> v2:172.21.15.17:3300/0 conn(0x7f74f813d2c0 0x7f74f813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:43.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fda4e640 1 --2- 172.21.15.17:0/202512177 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 0x7f74f813cd80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009870 tx=0x7f74e801bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa12a616d4659a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 1 -- 172.21.15.17:0/202512177 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e80ab070 con 0x7f74f81383b0 2024-09-15T21:17:43.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:43.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 10 monclient: dump: 2024-09-15T21:17:43.066 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 5 monclient: authenticate success, global_id 4262 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 1 -- 172.21.15.17:0/202512177 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e80a5660 con 0x7f74f81383b0 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f74fd24d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 4 set_mon_vals no callback set 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 10 monclient: shutdown 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 1 -- 172.21.15.17:0/202512177 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 msgr2=0x7f74f813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 1 --2- 172.21.15.17:0/202512177 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 0x7f74f813cd80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009870 tx=0x7f74e801bf20 comp rx=0 tx=0).stop 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 1 -- 172.21.15.17:0/202512177 shutdown_connections 2024-09-15T21:17:43.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 1 --2- 172.21.15.17:0/202512177 >> v2:172.21.15.17:3300/0 conn(0x7f74f813d2c0 0x7f74f813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 1 --2- 172.21.15.17:0/202512177 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81383b0 0x7f74f813cd80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.054+0000 7f75004b3640 1 -- 172.21.15.17:0/202512177 >> 172.21.15.17:0/202512177 conn(0x7f74f80aac40 msgr2=0x7f74f80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 -- 172.21.15.17:0/202512177 shutdown_connections 2024-09-15T21:17:43.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 -- 172.21.15.17:0/202512177 wait complete. 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient: monmap: 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.073 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 Processor -- start 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 -- start start 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient: init 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 0x7f74f80a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): picked mon.noname-c con 0x7f74f81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f813d2c0 0x7f74f81d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): picked mon.noname-a con 0x7f74f813d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8136ec0 0x7f74f81cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): picked mon.noname-b con 0x7f74f8136ec0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8143070 con 0x7f74f813d2c0 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8142ef0 con 0x7f74f8136ec0 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f74f81431f0 con 0x7f74f81378c0 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f75004b3640 10 monclient(hunting): authenticate will time out at 986.458452s 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 0x7f74f80a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 0x7f74f80a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45498/0 (socket says 172.21.15.17:45498) 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 -- 172.21.15.17:0/369427504 learned_addr learned my addr 172.21.15.17:0/369427504 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): get_auth_request con 0x7f74f81378c0 auth_method 0 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f813d2c0 0x7f74f81d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): get_auth_request con 0x7f74f813d2c0 auth_method 0 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fda4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 -- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8136ec0 msgr2=0x7f74f81cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8136ec0 0x7f74f81cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 -- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f813d2c0 msgr2=0x7f74f81d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f813d2c0 0x7f74f81d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 monclient: found mon.noname-c 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 -- 172.21.15.17:0/369427504 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8121850 con 0x7f74f81378c0 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74fe24f640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 0x7f74f80a0440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74f400c490 tx=0x7f74f4001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4017070 con 0x7f74f81378c0 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74eeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74eeffd640 10 monclient: dump: 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.058+0000 7f74eeffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 5 monclient: authenticate success, global_id 4265 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 monclient: _renew_subs 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f809f2c0 con 0x7f74f81378c0 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 monclient: _renew_subs 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f8120fb0 con 0x7f74f81378c0 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f81214b0 con 0x7f74f81378c0 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f4003600 con 0x7f74f81378c0 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74fca4c640 4 set_mon_vals no callback set 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4003a30 con 0x7f74f81378c0 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 10 monclient: dump: 2024-09-15T21:17:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f74f401a070 con 0x7f74f81378c0 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b9a0 0x7f74dc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f74f404ef40 con 0x7f74f81378c0 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.062+0000 7f74fda4e640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b9a0 0x7f74dc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.066+0000 7f74fda4e640 10 monclient: get_auth_request con 0x7f74dc03b9a0 auth_method 0 2024-09-15T21:17:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.066+0000 7f74fda4e640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b9a0 0x7f74dc03de60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74e80ad000 tx=0x7f74e80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.070+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f74f81214b0 con 0x7f74f81378c0 2024-09-15T21:17:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.070+0000 7f74eeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.070+0000 7f74eeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.110+0000 7fc1ddc50640 10 monclient: get_monmap_and_config 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.110+0000 7fc1ddc50640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.110+0000 7fc1ddc50640 10 monclient: monmap: 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:43.117637+0000 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:43.117637+0000 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 Processor -- start 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 -- start start 2024-09-15T21:17:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient: init 2024-09-15T21:17:43.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc1d81378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc1d81383b0 0x7fc1d813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): picked mon.noname-c con 0x7fc1d81383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1d813d2c0 0x7fc1d813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): picked mon.noname-a con 0x7fc1d813d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d80a1a20 con 0x7fc1d81378c0 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d80a1d00 con 0x7fc1d813d2c0 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d80a1fe0 con 0x7fc1d81383b0 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1d813d2c0 0x7fc1d813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1d813d2c0 0x7fc1d813f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44186/0 (socket says 172.21.15.17:44186) 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc1d81383b0 0x7fc1d813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 -- 172.21.15.17:0/3599648187 learned_addr learned my addr 172.21.15.17:0/3599648187 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1ddc50640 10 monclient(hunting): authenticate will time out at 986.513280s 2024-09-15T21:17:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): get_auth_request con 0x7fc1d813d2c0 auth_method 0 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): get_auth_request con 0x7fc1d81383b0 auth_method 0 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient(hunting): handle_auth_done global_id 4275 payload 274 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 -- 172.21.15.17:0/3599648187 >> v2:172.21.15.119:3300/0 conn(0x7fc1d81383b0 msgr2=0x7fc1d813cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 --2- 172.21.15.17:0/3599648187 >> v2:172.21.15.119:3300/0 conn(0x7fc1d81383b0 0x7fc1d813cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 -- 172.21.15.17:0/3599648187 >> v1:172.21.15.119:6789/0 conn(0x7fc1d81378c0 legacy=0x7fc1d8137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.114+0000 7fc1d7fff640 1 -- 172.21.15.17:0/3599648187 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80a22c0 con 0x7fc1d813d2c0 2024-09-15T21:17:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d7fff640 1 --2- 172.21.15.17:0/3599648187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d813d2c0 0x7fc1d813f6b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009e50 tx=0x7fc1d001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b48a0988d517f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/3599648187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00a4a30 con 0x7fc1d813d2c0 2024-09-15T21:17:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: dump: 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.130 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 5 monclient: authenticate success, global_id 4275 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/3599648187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d00a4bd0 con 0x7fc1d813d2c0 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 4 set_mon_vals no callback set 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/3599648187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00ac3f0 con 0x7fc1d813d2c0 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1d6ffd640 10 monclient: dump: 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 10 monclient: shutdown 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 1 -- 172.21.15.17:0/3599648187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d813d2c0 msgr2=0x7fc1d813f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/3599648187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d813d2c0 0x7fc1d813f6b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009e50 tx=0x7fc1d001c3b0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 1 -- 172.21.15.17:0/3599648187 shutdown_connections 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/3599648187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d813d2c0 0x7fc1d813f6b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/3599648187 >> v2:172.21.15.119:3300/0 conn(0x7fc1d81383b0 0x7fc1d813cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.118+0000 7fc1ddc50640 1 -- 172.21.15.17:0/3599648187 >> 172.21.15.17:0/3599648187 conn(0x7fc1d80aac40 msgr2=0x7fc1d80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- 172.21.15.17:0/3599648187 shutdown_connections 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- 172.21.15.17:0/3599648187 wait complete. 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: monmap: 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.134 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.134 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:43.134 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:43.134 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:43.134 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 Processor -- start 2024-09-15T21:17:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- start start 2024-09-15T21:17:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: init 2024-09-15T21:17:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1d81378c0 0x7fc1d81d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): picked mon.noname-c con 0x7fc1d81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 0x7fc1d8136ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): picked mon.noname-a con 0x7fc1d81383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1d813d2c0 0x7fc1d809dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): picked mon.noname-b con 0x7fc1d813d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8143020 con 0x7fc1d81383b0 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8142ea0 con 0x7fc1d813d2c0 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d81431a0 con 0x7fc1d81378c0 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient(hunting): authenticate will time out at 986.521702s 2024-09-15T21:17:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 0x7fc1d8136ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 0x7fc1d8136ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44194/0 (socket says 172.21.15.17:44194) 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 -- 172.21.15.17:0/2323705394 learned_addr learned my addr 172.21.15.17:0/2323705394 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1d81378c0 0x7fc1d81d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): get_auth_request con 0x7fc1d81383b0 auth_method 0 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): get_auth_request con 0x7fc1d81378c0 auth_method 0 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d7fff640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1d813d2c0 0x7fc1d809dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d7fff640 10 monclient(hunting): get_auth_request con 0x7fc1d813d2c0 auth_method 0 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 -- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1d81378c0 msgr2=0x7fc1d81d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1d81378c0 0x7fc1d81d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 -- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1d813d2c0 msgr2=0x7fc1d809dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1d813d2c0 0x7fc1d809dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 -- 172.21.15.17:0/2323705394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d8120770 con 0x7fc1d81383b0 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d77fe640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1d81378c0 0x7fc1d81d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1cffff640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 0x7fc1d8136ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000e930 tx=0x7fc1c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0017070 con 0x7fc1d81383b0 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: dump: 2024-09-15T21:17:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 5 monclient: authenticate success, global_id 4278 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _renew_subs 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d8121630 con 0x7fc1d81383b0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _renew_subs 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d809f660 con 0x7fc1d81383b0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c000c9a0 con 0x7fc1d81383b0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000cd50 con 0x7fc1d81383b0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d67fc640 4 set_mon_vals no callback set 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1d4ff9640 10 monclient: dump: 2024-09-15T21:17:43.142 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.122+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc005090 con 0x7fc1d81383b0 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient: get_monmap_and_config 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient: monmap: 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:43.130286+0000 2024-09-15T21:17:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:43.130286+0000 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc1c001a070 con 0x7fc1d81383b0 2024-09-15T21:17:43.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 Processor -- start 2024-09-15T21:17:43.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 -- start start 2024-09-15T21:17:43.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient: init 2024-09-15T21:17:43.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7fc1d4ff9640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc1b803b9c0 0x7fc1b803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7fc1c004fb30 con 0x7fc1d81383b0 2024-09-15T21:17:43.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7fc1d77fe640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc1b803b9c0 0x7fc1b803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7fc1d77fe640 10 monclient: get_auth_request con 0x7fc1b803b9c0 auth_method 0 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f41a81378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 0x7f41a813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): picked mon.noname-b con 0x7f41a81383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41a813d2c0 0x7f41a813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): picked mon.noname-a con 0x7f41a813d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a80a1a20 con 0x7f41a81378c0 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f41a80a1d00 con 0x7f41a813d2c0 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f41a80a1fe0 con 0x7f41a81383b0 2024-09-15T21:17:43.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f41aca28640 10 monclient(hunting): authenticate will time out at 986.525414s 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 0x7f41a813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 0x7f41a813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45518/0 (socket says 172.21.15.17:45518) 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 -- 172.21.15.17:0/1334040929 learned_addr learned my addr 172.21.15.17:0/1334040929 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): get_auth_request con 0x7f41a81383b0 auth_method 0 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7fc1d77fe640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc1b803b9c0 0x7fc1b803de80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8007fd0 tx=0x7fc1c8007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient(hunting): handle_auth_done global_id 4271 payload 274 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 -- 172.21.15.17:0/1334040929 >> v2:172.21.15.17:3300/0 conn(0x7f41a813d2c0 msgr2=0x7f41a813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 --2- 172.21.15.17:0/1334040929 >> v2:172.21.15.17:3300/0 conn(0x7f41a813d2c0 0x7f41a813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 -- 172.21.15.17:0/1334040929 >> v1:172.21.15.17:6789/0 conn(0x7f41a81378c0 legacy=0x7f41a8137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 monclient: found mon.noname-b 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.126+0000 7f419ffff640 1 -- 172.21.15.17:0/1334040929 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a80a22c0 con 0x7f41a81383b0 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f419ffff640 1 --2- 172.21.15.17:0/1334040929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 0x7f41a813cd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4190009870 tx=0x7f419001bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a29f752a9b26a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 1 -- 172.21.15.17:0/1334040929 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41900ab070 con 0x7f41a81383b0 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: dump: 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 1 -- 172.21.15.17:0/1334040929 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41900a5680 con 0x7f41a81383b0 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 5 monclient: authenticate success, global_id 4271 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 1 -- 172.21.15.17:0/1334040929 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41900a5a50 con 0x7f41a81383b0 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:43.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 4 set_mon_vals no callback set 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41a5d9b640 10 monclient: dump: 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 10 monclient: shutdown 2024-09-15T21:17:43.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 -- 172.21.15.17:0/1334040929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 msgr2=0x7f41a813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 --2- 172.21.15.17:0/1334040929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 0x7f41a813cd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4190009870 tx=0x7f419001bf40 comp rx=0 tx=0).stop 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 -- 172.21.15.17:0/1334040929 shutdown_connections 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 --2- 172.21.15.17:0/1334040929 >> v2:172.21.15.17:3300/0 conn(0x7f41a813d2c0 0x7f41a813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 --2- 172.21.15.17:0/1334040929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81383b0 0x7f41a813cd80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 -- 172.21.15.17:0/1334040929 >> 172.21.15.17:0/1334040929 conn(0x7f41a80aac40 msgr2=0x7f41a80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 -- 172.21.15.17:0/1334040929 shutdown_connections 2024-09-15T21:17:43.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 -- 172.21.15.17:0/1334040929 wait complete. 2024-09-15T21:17:43.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc1c0013c20 con 0x7fc1d81383b0 2024-09-15T21:17:43.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7fc1d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7fc1d4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 10 monclient: monmap: 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.158 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.130+0000 7f41aca28640 1 Processor -- start 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- start start 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: init 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81378c0 0x7f41a813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): picked mon.noname-c con 0x7f41a81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41a81383b0 0x7f41a80a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): picked mon.noname-b con 0x7f41a81383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:43.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 0x7f41a80a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): picked mon.noname-a con 0x7f41a813d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8143070 con 0x7f41a813d2c0 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8142ef0 con 0x7f41a81383b0 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f41a81431f0 con 0x7f41a81378c0 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient(hunting): authenticate will time out at 986.536911s 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 0x7f41a80a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81378c0 0x7f41a813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 0x7f41a80a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44216/0 (socket says 172.21.15.17:44216) 2024-09-15T21:17:43.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 -- 172.21.15.17:0/2529646716 learned_addr learned my addr 172.21.15.17:0/2529646716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): get_auth_request con 0x7f41a813d2c0 auth_method 0 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81378c0 0x7f41a813c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45524/0 (socket says 172.21.15.17:45524) 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): get_auth_request con 0x7f41a81378c0 auth_method 0 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41a81383b0 0x7f41a80a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): get_auth_request con 0x7f41a81383b0 auth_method 0 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 -- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81378c0 msgr2=0x7f41a813c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81378c0 0x7f41a813c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 -- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41a81383b0 msgr2=0x7f41a80a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41a81383b0 0x7f41a80a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 -- 172.21.15.17:0/2529646716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a813cd00 con 0x7f41a813d2c0 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a6d9d640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 0x7f41a80a05c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f41a000ba10 tx=0x7f41a000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0019070 con 0x7f41a813d2c0 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: dump: 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.163 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a000cb40 con 0x7f41a813d2c0 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 5 monclient: authenticate success, global_id 4281 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _renew_subs 2024-09-15T21:17:43.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a813cfa0 con 0x7f41a813d2c0 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0014440 con 0x7f41a813d2c0 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 10 monclient: dump: 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.165 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41a559a640 4 set_mon_vals no callback set 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _renew_subs 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a809eb50 con 0x7f41a813d2c0 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f41a001c070 con 0x7f41a813d2c0 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f419effd640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f418803b9c0 0x7f418803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.138+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a8137ca0 con 0x7f41a813d2c0 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.142+0000 7f41a659c640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f418803b9c0 0x7f418803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.142+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f41a001dd90 con 0x7f41a813d2c0 2024-09-15T21:17:43.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.142+0000 7f41a659c640 10 monclient: get_auth_request con 0x7f418803b9c0 auth_method 0 2024-09-15T21:17:43.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.142+0000 7f41a659c640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f418803b9c0 0x7f418803de80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f41a81ce400 tx=0x7f41980090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient: get_monmap_and_config 2024-09-15T21:17:43.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient: monmap: 2024-09-15T21:17:43.170 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:43.151898+0000 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:43.151898+0000 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 Processor -- start 2024-09-15T21:17:43.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 -- start start 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient: init 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c1378c0 0x7f273c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): picked mon.noname-b con 0x7f273c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f273c1381e0 0x7f273c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): picked mon.noname-c con 0x7f273c1381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f273c0a5040 0x7f273c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c1378c0 0x7f273c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c1378c0 0x7f273c137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45526/0 (socket says 172.21.15.17:45526) 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f273c0a5040 0x7f273c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f273c0a5040 0x7f273c0a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44222/0 (socket says 172.21.15.17:44222) 2024-09-15T21:17:43.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): picked mon.noname-a con 0x7f273c0a5040 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f273c0a1d00 con 0x7f273c0a5040 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f273c0a1fe0 con 0x7f273c1381e0 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f273c0a22c0 con 0x7f273c1378c0 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2743e23640 10 monclient(hunting): authenticate will time out at 986.547435s 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 1 -- 172.21.15.17:0/1607410154 learned_addr learned my addr 172.21.15.17:0/1607410154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): get_auth_request con 0x7f273c1378c0 auth_method 0 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 1 --2- 172.21.15.17:0/1607410154 >> v2:172.21.15.119:3300/0 conn(0x7f273c1381e0 0x7f273c0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): get_auth_request con 0x7f273c1381e0 auth_method 0 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): get_auth_request con 0x7f273c0a5040 auth_method 0 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27413be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f2741bbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 10 monclient(hunting): handle_auth_done global_id 4284 payload 274 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 1 -- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c1378c0 msgr2=0x7f273c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 1 --2- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c1378c0 0x7f273c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 1 -- 172.21.15.17:0/1607410154 >> v2:172.21.15.119:3300/0 conn(0x7f273c1381e0 msgr2=0x7f273c0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.146+0000 7f27423c0640 1 --2- 172.21.15.17:0/1607410154 >> v2:172.21.15.119:3300/0 conn(0x7f273c1381e0 0x7f273c0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f27423c0640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f27423c0640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f27423c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f27423c0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f27423c0640 1 -- 172.21.15.17:0/1607410154 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c0a4490 con 0x7f273c0a5040 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f41a001bcd0 con 0x7f41a813d2c0 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f419effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f419effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f27423c0640 1 --2- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0a5040 0x7f273c0a54f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f273c0a7b50 tx=0x7f273401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bbd6bcbd2f30f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 1 -- 172.21.15.17:0/1607410154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340b2070 con 0x7f273c0a5040 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: dump: 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.179 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2743e23640 5 monclient: authenticate success, global_id 4284 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2743e23640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 1 -- 172.21.15.17:0/1607410154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27340a5710 con 0x7f273c0a5040 2024-09-15T21:17:43.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 1 -- 172.21.15.17:0/1607410154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340a5ae0 con 0x7f273c0a5040 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2743e23640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2743e23640 4 set_mon_vals no callback set 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.150+0000 7f2740bbd640 10 monclient: dump: 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.181 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient: shutdown 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0a5040 msgr2=0x7f273c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0a5040 0x7f273c0a54f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f273c0a7b50 tx=0x7f273401c880 comp rx=0 tx=0).stop 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1607410154 shutdown_connections 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- 172.21.15.17:0/1607410154 >> v2:172.21.15.119:3300/0 conn(0x7f273c1381e0 0x7f273c0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0a5040 0x7f273c0a54f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- 172.21.15.17:0/1607410154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c1378c0 0x7f273c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1607410154 >> 172.21.15.17:0/1607410154 conn(0x7f273c0aac40 msgr2=0x7f273c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1607410154 shutdown_connections 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1607410154 wait complete. 2024-09-15T21:17:43.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient: monmap: 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 Processor -- start 2024-09-15T21:17:43.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- start start 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient: init 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c0a5040 0x7f273c1cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): picked mon.noname-c con 0x7f273c0a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 0x7f273c136c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): picked mon.noname-a con 0x7f273c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 0x7f273c136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 0x7f273c136c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44238/0 (socket says 172.21.15.17:44238) 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f273c1381e0 0x7f273c13cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 -- 172.21.15.17:0/1721266843 learned_addr learned my addr 172.21.15.17:0/1721266843 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): picked mon.noname-b con 0x7f273c1381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f273c141730 con 0x7f273c1378c0 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1415b0 con 0x7f273c1381e0 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f273c1418b0 con 0x7f273c0a5040 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2743e23640 10 monclient(hunting): authenticate will time out at 986.555394s 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c0a5040 0x7f273c1cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): get_auth_request con 0x7f273c0a5040 auth_method 0 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): get_auth_request con 0x7f273c1378c0 auth_method 0 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f2741bbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f273c1381e0 0x7f273c13cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): get_auth_request con 0x7f273c1381e0 auth_method 0 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27423c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient(hunting): handle_auth_done global_id 4287 payload 824 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 -- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c0a5040 msgr2=0x7f273c1cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c0a5040 0x7f273c1cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 -- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f273c1381e0 msgr2=0x7f273c13cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f273c1381e0 0x7f273c13cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 monclient: found mon.noname-a 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.154+0000 7f27413be640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c13e850 con 0x7f273c1378c0 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27413be640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 0x7f273c136c30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f272c00e930 tx=0x7f272c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c017070 con 0x7f273c1378c0 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: dump: 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c00c9a0 con 0x7f273c1378c0 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c00cd50 con 0x7f273c1378c0 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 10 monclient: dump: 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.190 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 5 monclient: authenticate success, global_id 4287 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 10 monclient: _renew_subs 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c0a8a10 con 0x7f273c1378c0 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2733fff640 4 set_mon_vals no callback set 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 10 monclient: _renew_subs 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2743e23640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c121d30 con 0x7f273c1378c0 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f271ffff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f271ffff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f271ffff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2718005090 con 0x7f273c1378c0 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f272c01a070 con 0x7f273c1378c0 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f27200441d0 0x7f2720046690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f272c04fed0 con 0x7f273c1378c0 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2741bbf640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f27200441d0 0x7f2720046690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.158+0000 7f2741bbf640 10 monclient: get_auth_request con 0x7f27200441d0 auth_method 0 2024-09-15T21:17:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f2741bbf640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f27200441d0 0x7f2720046690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f273c1208e0 tx=0x7f2738009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient: get_monmap_and_config 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient: monmap: 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:43.167652+0000 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:43.167652+0000 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 Processor -- start 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 -- start start 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient: init 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a940736a0 0x7f8a94073a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): picked mon.noname-b con 0x7f8a940736a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a94073fc0 0x7f8a9409f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): picked mon.noname-c con 0x7f8a94073fc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a940a21e0 0x7f8a9409f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): picked mon.noname-a con 0x7f8a940a21e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8a940a1800 con 0x7f8a940a21e0 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a940a1520 con 0x7f8a94073fc0 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a940a1ae0 con 0x7f8a940736a0 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9c1cc640 10 monclient(hunting): authenticate will time out at 986.563419s 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a940a21e0 0x7f8a9409f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a940a21e0 0x7f8a9409f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44254/0 (socket says 172.21.15.17:44254) 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 1 -- 172.21.15.17:0/1771551914 learned_addr learned my addr 172.21.15.17:0/1771551914 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 10 monclient(hunting): get_auth_request con 0x7f8a940a21e0 auth_method 0 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a9a769640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a99f68640 1 --2- 172.21.15.17:0/1771551914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a940736a0 0x7f8a94073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a99f68640 10 monclient(hunting): get_auth_request con 0x7f8a940736a0 auth_method 0 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a99f68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a99f68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.162+0000 7f8a99f68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 10 monclient(hunting): handle_auth_done global_id 4283 payload 274 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 -- 172.21.15.17:0/1771551914 >> v2:172.21.15.119:3300/0 conn(0x7f8a94073fc0 msgr2=0x7f8a9409f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 --2- 172.21.15.17:0/1771551914 >> v2:172.21.15.119:3300/0 conn(0x7f8a94073fc0 0x7f8a9409f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 -- 172.21.15.17:0/1771551914 >> v2:172.21.15.17:3300/0 conn(0x7f8a940a21e0 msgr2=0x7f8a9409f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 --2- 172.21.15.17:0/1771551914 >> v2:172.21.15.17:3300/0 conn(0x7f8a940a21e0 0x7f8a9409f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 monclient: found mon.noname-b 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 -- 172.21.15.17:0/1771551914 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a940a3f80 con 0x7f8a940736a0 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a99f68640 1 --2- 172.21.15.17:0/1771551914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a940736a0 0x7f8a94073a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a94074b30 tx=0x7f8a9001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828a64f28dac9db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f272c04da50 con 0x7f273c1378c0 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f27327fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f27327fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:43.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 1 -- 172.21.15.17:0/1771551914 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900b0070 con 0x7f8a940736a0 2024-09-15T21:17:43.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 10 monclient: dump: 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a9c1cc640 5 monclient: authenticate success, global_id 4283 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a9c1cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 1 -- 172.21.15.17:0/1771551914 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a900ab400 con 0x7f8a940736a0 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a98f66640 1 -- 172.21.15.17:0/1771551914 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900ab7d0 con 0x7f8a940736a0 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a9c1cc640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.166+0000 7f8a9c1cc640 4 set_mon_vals no callback set 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a98f66640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a98f66640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a98f66640 10 monclient: dump: 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.204 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a9c1cc640 10 monclient: shutdown 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a9c1cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/1771551914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a940736a0 msgr2=0x7f8a94073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.170+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/1771551914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a940736a0 0x7f8a94073a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a94074b30 tx=0x7f8a9001c340 comp rx=0 tx=0).stop 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/1771551914 shutdown_connections 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/1771551914 >> v2:172.21.15.119:3300/0 conn(0x7f8a94073fc0 0x7f8a9409f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/1771551914 >> v2:172.21.15.17:3300/0 conn(0x7f8a940a21e0 0x7f8a9409f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/1771551914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a940736a0 0x7f8a94073a80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/1771551914 >> 172.21.15.17:0/1771551914 conn(0x7f8a940a4e90 msgr2=0x7f8a940a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/1771551914 shutdown_connections 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/1771551914 wait complete. 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient: build_initial_monmap 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient: monmap: 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:43.206 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 Processor -- start 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- start start 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient: init 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940736a0 0x7f8a94139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): picked mon.noname-a con 0x7f8a940736a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a94073fc0 0x7f8a94139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): picked mon.noname-c con 0x7f8a94073fc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a940a21e0 0x7f8a941216d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): picked mon.noname-b con 0x7f8a940a21e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94141df0 con 0x7f8a940736a0 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a940a0320 con 0x7f8a940a21e0 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a94141f70 con 0x7f8a94073fc0 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9c1cc640 10 monclient(hunting): authenticate will time out at 986.574581s 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a94073fc0 0x7f8a94139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a94073fc0 0x7f8a94139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45552/0 (socket says 172.21.15.17:45552) 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 1 -- 172.21.15.17:0/817498833 learned_addr learned my addr 172.21.15.17:0/817498833 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 10 monclient(hunting): get_auth_request con 0x7f8a94073fc0 auth_method 0 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99767640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a940a21e0 0x7f8a941216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): get_auth_request con 0x7f8a940a21e0 auth_method 0 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940736a0 0x7f8a94139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): get_auth_request con 0x7f8a940736a0 auth_method 0 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a9a769640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.174+0000 7f8a99f68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 10 monclient(hunting): handle_auth_done global_id 4309 payload 824 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 -- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a94073fc0 msgr2=0x7f8a94139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a94073fc0 0x7f8a94139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 -- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940736a0 msgr2=0x7f8a94139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940736a0 0x7f8a94139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 10 monclient: _finish_hunting 0 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 monclient: found mon.noname-b 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:17:43.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 -- 172.21.15.17:0/817498833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a94123150 con 0x7f8a940a21e0 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9a769640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a940a21e0 0x7f8a941216d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c007a40 tx=0x7f8a8c007f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01a070 con 0x7f8a940a21e0 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: dump: 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.211 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8c00dc80 con 0x7f8a940a21e0 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 5 monclient: authenticate success, global_id 4309 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c015640 con 0x7f8a940a21e0 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8a7fc640 10 monclient: dump: 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:43.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 monclient: _renew_subs 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a940a7b30 con 0x7f8a940a21e0 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 monclient: _renew_subs 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a94124c10 con 0x7f8a940a21e0 2024-09-15T21:17:43.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a8bfff640 4 set_mon_vals no callback set 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.178+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a7c005090 con 0x7f8a940a21e0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.182+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8a8c01e3f0 con 0x7f8a940a21e0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.182+0000 7f8a8a7fc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a7803b9f0 0x7f8a7803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.182+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f8a8c04fe90 con 0x7f8a940a21e0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.182+0000 7f8a99f68640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a7803b9f0 0x7f8a7803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.182+0000 7f8a99f68640 10 monclient: get_auth_request con 0x7f8a7803b9f0 auth_method 0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.182+0000 7f8a99f68640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a7803b9f0 0x7f8a7803deb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8a900ad000 tx=0x7f8a900a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.194+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8a8c023080 con 0x7f8a940a21e0 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.194+0000 7f8a8a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:43.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.194+0000 7f8a8a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f75004b3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f75004b3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f75004b3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f74f8137ca0 con 0x7f74f81378c0 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f74eeffd640 1 -- 172.21.15.17:0/369427504 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74f4003bd0 con 0x7f74f81378c0 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f74eeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T21:17:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.286+0000 7f74eeffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:43.297 INFO:teuthology.orchestra.run.smithi017.stdout:47244640259 2024-09-15T21:17:43.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b9a0 msgr2=0x7f74dc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b9a0 0x7f74dc03de60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74e80ad000 tx=0x7f74e80a6000 comp rx=0 tx=0).stop 2024-09-15T21:17:43.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 10 monclient: shutdown 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 msgr2=0x7f74f80a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 0x7f74f80a0440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74f400c490 tx=0x7f74f4001e80 comp rx=0 tx=0).stop 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 shutdown_connections 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b9a0 0x7f74dc03de60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8136ec0 0x7f74f81cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f813d2c0 0x7f74f81d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 --2- 172.21.15.17:0/369427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f81378c0 0x7f74f80a0440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 >> 172.21.15.17:0/369427504 conn(0x7f74f80aac40 msgr2=0x7f74f806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 shutdown_connections 2024-09-15T21:17:43.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.290+0000 7f75004b3640 1 -- 172.21.15.17:0/369427504 wait complete. 2024-09-15T21:17:43.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.0 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc1bc005380 con 0x7fc1d81383b0 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/2323705394 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc1c001a360 con 0x7fc1d81383b0 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1d4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:43.334 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc1b803b9c0 msgr2=0x7fc1b803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc1b803b9c0 0x7fc1b803de80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8007fd0 tx=0x7fc1c8007e40 comp rx=0 tx=0).stop 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 10 monclient: shutdown 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 msgr2=0x7fc1d8136ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 0x7fc1d8136ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000e930 tx=0x7fc1c000edf0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 shutdown_connections 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc1b803b9c0 0x7fc1b803de80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1d813d2c0 0x7fc1d809dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d81383b0 0x7fc1d8136ca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 --2- 172.21.15.17:0/2323705394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1d81378c0 0x7fc1d81d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 >> 172.21.15.17:0/2323705394 conn(0x7fc1d80aac40 msgr2=0x7fc1d806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 shutdown_connections 2024-09-15T21:17:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.326+0000 7fc1ddc50640 1 -- 172.21.15.17:0/2323705394 wait complete. 2024-09-15T21:17:43.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.5 2024-09-15T21:17:43.352 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:43.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T21:17:43.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T21:17:43.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f81ac13fef0 con 0x7f81ac09d9b0 2024-09-15T21:17:43.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81a8ff9640 1 -- 172.21.15.17:0/2490797546 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f81a40b7d20 con 0x7f81ac09d9b0 2024-09-15T21:17:43.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T21:17:43.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81a8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:43.358 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:43.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f819803b9f0 msgr2=0x7f819803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f819803b9f0 0x7f819803deb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009b60 tx=0x7f81a0009190 comp rx=0 tx=0).stop 2024-09-15T21:17:43.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 10 monclient: shutdown 2024-09-15T21:17:43.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 msgr2=0x7f81ac1d43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 0x7f81ac1d43a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f81a40982e0 tx=0x7f81a4019590 comp rx=0 tx=0).stop 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 shutdown_connections 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f819803b9f0 0x7f819803deb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81ac0a0600 0x7f81ac1d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81ac09d9b0 0x7f81ac1d43a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 --2- 172.21.15.17:0/2490797546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0990b0 0x7f81ac1d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.350+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 >> 172.21.15.17:0/2490797546 conn(0x7f81ac0a66b0 msgr2=0x7f81ac133d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.354+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 shutdown_connections 2024-09-15T21:17:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.354+0000 7f81b257b640 1 -- 172.21.15.17:0/2490797546 wait complete. 2024-09-15T21:17:43.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-09-15T21:17:43.381 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.426+0000 7f41aca28640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.426+0000 7f41aca28640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.426+0000 7f41aca28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.426+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f41a8069270 con 0x7f41a813d2c0 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f419effd640 1 -- 172.21.15.17:0/2529646716 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41a004df40 con 0x7f41a813d2c0 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f419effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T21:17:43.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f419effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:43.438 INFO:teuthology.orchestra.run.smithi017.stdout:51539607558 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f418803b9c0 msgr2=0x7f418803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f418803b9c0 0x7f418803de80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f41a81ce400 tx=0x7f41980090e0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 10 monclient: shutdown 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 msgr2=0x7f41a80a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 0x7f41a80a05c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f41a000ba10 tx=0x7f41a000bed0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 shutdown_connections 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f418803b9c0 0x7f418803de80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41a813d2c0 0x7f41a80a05c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41a81383b0 0x7f41a80a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 --2- 172.21.15.17:0/2529646716 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41a81378c0 0x7f41a813c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 >> 172.21.15.17:0/2529646716 conn(0x7f41a80aac40 msgr2=0x7f41a8136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 shutdown_connections 2024-09-15T21:17:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.430+0000 7f41aca28640 1 -- 172.21.15.17:0/2529646716 wait complete. 2024-09-15T21:17:43.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a9c1cc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T21:17:43.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a9c1cc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T21:17:43.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a9c1cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:17:43.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8a7c005650 con 0x7f8a940a21e0 2024-09-15T21:17:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a8a7fc640 1 -- 172.21.15.17:0/817498833 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a8c010070 con 0x7f8a940a21e0 2024-09-15T21:17:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a8a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T21:17:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.450+0000 7f8a8a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:43.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.1 2024-09-15T21:17:43.461 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:43.462 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2718005650 con 0x7f273c1378c0 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f27327fc640 1 -- 172.21.15.17:0/1721266843 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f272c04ddd0 con 0x7f273c1378c0 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f27327fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T21:17:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f27327fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:43.463 INFO:teuthology.orchestra.run.smithi017.stdout:47244640260 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a7803b9f0 msgr2=0x7f8a7803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a7803b9f0 0x7f8a7803deb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8a900ad000 tx=0x7f8a900a6000 comp rx=0 tx=0).stop 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 10 monclient: shutdown 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a940a21e0 msgr2=0x7f8a941216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a940a21e0 0x7f8a941216d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c007a40 tx=0x7f8a8c007f00 comp rx=0 tx=0).stop 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 shutdown_connections 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a7803b9f0 0x7f8a7803deb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a940a21e0 0x7f8a941216d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a94073fc0 0x7f8a94139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 --2- 172.21.15.17:0/817498833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940736a0 0x7f8a94139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 >> 172.21.15.17:0/817498833 conn(0x7f8a940a4e90 msgr2=0x7f8a9409da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 shutdown_connections 2024-09-15T21:17:43.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f8a9c1cc640 1 -- 172.21.15.17:0/817498833 wait complete. 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f27200441d0 msgr2=0x7f2720046690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f27200441d0 0x7f2720046690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f273c1208e0 tx=0x7f2738009240 comp rx=0 tx=0).stop 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 10 monclient: shutdown 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 msgr2=0x7f273c136c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 0x7f273c136c30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f272c00e930 tx=0x7f272c00edf0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 shutdown_connections 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f27200441d0 0x7f2720046690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f273c1381e0 0x7f273c13cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1378c0 0x7f273c136c30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 --2- 172.21.15.17:0/1721266843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f273c0a5040 0x7f273c1cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:43.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 >> 172.21.15.17:0/1721266843 conn(0x7f273c0aac40 msgr2=0x7f273c0a8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:43.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 shutdown_connections 2024-09-15T21:17:43.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:43.454+0000 7f271ffff640 1 -- 172.21.15.17:0/1721266843 wait complete. 2024-09-15T21:17:43.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2024-09-15T21:17:43.480 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:43.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.4 2024-09-15T21:17:43.481 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:44.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient: get_monmap_and_config 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient: build_initial_monmap 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient: monmap: 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:44.455209+0000 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:44.455209+0000 2024-09-15T21:17:44.458 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:44.459 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 Processor -- start 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 -- start start 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient: init 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53381378c0 0x7f5338137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): picked mon.noname-c con 0x7f53381378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f53381381e0 0x7f53380a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): picked mon.noname-a con 0x7f53381381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53380a5040 0x7f53380a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): picked mon.noname-b con 0x7f53380a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f53380a1d00 con 0x7f53381381e0 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f53380a1fe0 con 0x7f53381378c0 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f53380a22c0 con 0x7f53380a5040 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533ce60640 10 monclient(hunting): authenticate will time out at 987.851235s 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f53381381e0 0x7f53380a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f53381381e0 0x7f53380a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44282/0 (socket says 172.21.15.17:44282) 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 -- 172.21.15.17:0/506297610 learned_addr learned my addr 172.21.15.17:0/506297610 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): get_auth_request con 0x7f53381381e0 auth_method 0 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 1 --2- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53380a5040 0x7f53380a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): get_auth_request con 0x7f53380a5040 auth_method 0 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f5336d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533659c640 1 --2- 172.21.15.17:0/506297610 >> v2:172.21.15.119:3300/0 conn(0x7f53381378c0 0x7f5338137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533659c640 10 monclient(hunting): get_auth_request con 0x7f53381378c0 auth_method 0 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f533659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient(hunting): handle_auth_done global_id 4296 payload 274 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 -- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53380a5040 msgr2=0x7f53380a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 --2- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53380a5040 0x7f53380a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 -- 172.21.15.17:0/506297610 >> v2:172.21.15.119:3300/0 conn(0x7f53381378c0 msgr2=0x7f5338137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 --2- 172.21.15.17:0/506297610 >> v2:172.21.15.119:3300/0 conn(0x7f53381378c0 0x7f5338137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:44.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 monclient: found mon.noname-a 2024-09-15T21:17:44.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:44.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.450+0000 7f532ffff640 1 -- 172.21.15.17:0/506297610 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53380a4490 con 0x7f53381381e0 2024-09-15T21:17:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f532ffff640 1 --2- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381381e0 0x7f53380a4b00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5320009870 tx=0x7f532001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2745e9262121f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 1 -- 172.21.15.17:0/506297610 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200b1070 con 0x7f53381381e0 2024-09-15T21:17:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: dump: 2024-09-15T21:17:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: _finish_auth 0 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 1 -- 172.21.15.17:0/506297610 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200ac620 con 0x7f53381381e0 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 1 -- 172.21.15.17:0/506297610 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200ac9f0 con 0x7f53381381e0 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 5 monclient: authenticate success, global_id 4296 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 4 set_mon_vals no callback set 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f5335d9b640 10 monclient: dump: 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.471 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 10 monclient: shutdown 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 -- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381381e0 msgr2=0x7f53380a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 --2- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381381e0 0x7f53380a4b00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5320009870 tx=0x7f532001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 -- 172.21.15.17:0/506297610 shutdown_connections 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 --2- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53380a5040 0x7f53380a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 --2- 172.21.15.17:0/506297610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381381e0 0x7f53380a4b00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 --2- 172.21.15.17:0/506297610 >> v2:172.21.15.119:3300/0 conn(0x7f53381378c0 0x7f5338137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 -- 172.21.15.17:0/506297610 >> 172.21.15.17:0/506297610 conn(0x7f53380aac40 msgr2=0x7f53380a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 -- 172.21.15.17:0/506297610 shutdown_connections 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 -- 172.21.15.17:0/506297610 wait complete. 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 10 monclient: build_initial_monmap 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 10 monclient: monmap: 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:44.473 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 Processor -- start 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 1 -- start start 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.454+0000 7f533ce60640 10 monclient: init 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53380a5040 0x7f53381cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): picked mon.noname-b con 0x7f53380a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 0x7f5338136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): picked mon.noname-a con 0x7f53381378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53381381e0 0x7f533813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): picked mon.noname-c con 0x7f53381381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5338141730 con 0x7f53381378c0 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53381415b0 con 0x7f53380a5040 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f53381418b0 con 0x7f53381381e0 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient(hunting): authenticate will time out at 987.856397s 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 0x7f5338136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53381381e0 0x7f533813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 0x7f5338136c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44292/0 (socket says 172.21.15.17:44292) 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 -- 172.21.15.17:0/423039985 learned_addr learned my addr 172.21.15.17:0/423039985 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): get_auth_request con 0x7f53381378c0 auth_method 0 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): get_auth_request con 0x7f53381381e0 auth_method 0 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f5336d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533659c640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53380a5040 0x7f53381cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient(hunting): handle_auth_done global_id 4299 payload 824 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 -- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53381381e0 msgr2=0x7f533813cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53381381e0 0x7f533813cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 -- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53380a5040 msgr2=0x7f53381cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53380a5040 0x7f53381cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 monclient: found mon.noname-a 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 -- 172.21.15.17:0/423039985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5338122330 con 0x7f53381378c0 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533659c640 10 monclient: get_auth_request con 0x7f53380a5040 auth_method 0 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533659c640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53380a5040 0x7f53381cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532ffff640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 0x7f5338136c70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53200af040 tx=0x7f5320019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:44.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200b1040 con 0x7f53381378c0 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: dump: 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: _finish_auth 0 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200ac420 con 0x7f53381378c0 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200b5680 con 0x7f53381378c0 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 10 monclient: dump: 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.479 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 5 monclient: authenticate success, global_id 4299 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _renew_subs 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533559a640 4 set_mon_vals no callback set 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5338122610 con 0x7f53381378c0 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _renew_subs 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533806ea50 con 0x7f53381378c0 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:44.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5338137ca0 con 0x7f53381378c0 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f53200bc020 con 0x7f53381378c0 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f531803b9f0 0x7f531803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f53200e8eb0 con 0x7f53381378c0 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533659c640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f531803b9f0 0x7f531803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533659c640 10 monclient: get_auth_request con 0x7f531803b9f0 auth_method 0 2024-09-15T21:17:44.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.458+0000 7f533659c640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f531803b9f0 0x7f531803deb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53280097c0 tx=0x7f5328008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:44.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.462+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f53200c3020 con 0x7f53381378c0 2024-09-15T21:17:44.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.462+0000 7f532effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:44.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.462+0000 7f532effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f533ce60640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T21:17:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f533ce60640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T21:17:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f533ce60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5338121ec0 con 0x7f53381378c0 2024-09-15T21:17:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f532effd640 1 -- 172.21.15.17:0/423039985 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f53200160f0 con 0x7f53381378c0 2024-09-15T21:17:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f532effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T21:17:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.630+0000 7f532effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:44.640 INFO:teuthology.orchestra.run.smithi017.stdout:47244640261 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f531803b9f0 msgr2=0x7f531803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f531803b9f0 0x7f531803deb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53280097c0 tx=0x7f5328008040 comp rx=0 tx=0).stop 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 10 monclient: shutdown 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 msgr2=0x7f5338136c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 0x7f5338136c70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53200af040 tx=0x7f5320019180 comp rx=0 tx=0).stop 2024-09-15T21:17:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 shutdown_connections 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f531803b9f0 0x7f531803deb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f53381381e0 0x7f533813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53381378c0 0x7f5338136c70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 --2- 172.21.15.17:0/423039985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53380a5040 0x7f53381cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 >> 172.21.15.17:0/423039985 conn(0x7f53380aac40 msgr2=0x7f53380a7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 shutdown_connections 2024-09-15T21:17:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.634+0000 7f533ce60640 1 -- 172.21.15.17:0/423039985 wait complete. 2024-09-15T21:17:44.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2024-09-15T21:17:44.657 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:44.657 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:17:44.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:44.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.758+0000 7f7cbc860640 10 monclient: get_monmap_and_config 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.758+0000 7f7cbc860640 10 monclient: build_initial_monmap 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.758+0000 7f7cbc860640 10 monclient: monmap: 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:44.765936+0000 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:44.765936+0000 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:44.769 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:44.770 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:44.770 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:44.770 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 Processor -- start 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 -- start start 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient: init 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cb80a0890 0x7f7cb80a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): picked mon.noname-c con 0x7f7cb80a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7cb809ffc0 0x7f7cb80a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): picked mon.noname-a con 0x7f7cb809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb8135670 0x7f7cb8135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): picked mon.noname-b con 0x7f7cb8135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb80a16f0 con 0x7f7cb809ffc0 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb80a3130 con 0x7f7cb80a0890 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb8073df0 con 0x7f7cb8135670 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cbc860640 10 monclient(hunting): authenticate will time out at 988.161230s 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7cb809ffc0 0x7f7cb80a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb8135670 0x7f7cb8135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7cb809ffc0 0x7f7cb80a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44306/0 (socket says 172.21.15.17:44306) 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 -- 172.21.15.17:0/2489138149 learned_addr learned my addr 172.21.15.17:0/2489138149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): get_auth_request con 0x7f7cb809ffc0 auth_method 0 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): get_auth_request con 0x7f7cb8135670 auth_method 0 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb659c640 1 --2- 172.21.15.17:0/2489138149 >> v2:172.21.15.119:3300/0 conn(0x7f7cb80a0890 0x7f7cb80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb659c640 10 monclient(hunting): get_auth_request con 0x7f7cb80a0890 auth_method 0 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_done global_id 4302 payload 274 2024-09-15T21:17:44.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 -- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb8135670 msgr2=0x7f7cb8135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:44.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 --2- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb8135670 0x7f7cb8135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 -- 172.21.15.17:0/2489138149 >> v2:172.21.15.119:3300/0 conn(0x7f7cb80a0890 msgr2=0x7f7cb80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 --2- 172.21.15.17:0/2489138149 >> v2:172.21.15.119:3300/0 conn(0x7f7cb80a0890 0x7f7cb80a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 monclient: found mon.noname-a 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 -- 172.21.15.17:0/2489138149 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb80a1bf0 con 0x7f7cb809ffc0 2024-09-15T21:17:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb6d9d640 1 --2- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb8135670 0x7f7cb8135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:44.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.762+0000 7f7cb5d9b640 1 --2- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb809ffc0 0x7f7cb80a11b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009870 tx=0x7f7ca801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb91c8a7b1d57ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:44.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 1 -- 172.21.15.17:0/2489138149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80b0070 con 0x7f7cb809ffc0 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: dump: 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.783 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: _finish_auth 0 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: _check_auth_tickets 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 5 monclient: authenticate success, global_id 4302 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 1 -- 172.21.15.17:0/2489138149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca80ab600 con 0x7f7cb809ffc0 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 1 -- 172.21.15.17:0/2489138149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80ab9d0 con 0x7f7cb809ffc0 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb559a640 10 monclient: dump: 2024-09-15T21:17:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 4 set_mon_vals no callback set 2024-09-15T21:17:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: shutdown 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb809ffc0 msgr2=0x7f7cb80a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb809ffc0 0x7f7cb80a11b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009870 tx=0x7f7ca801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/2489138149 shutdown_connections 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb8135670 0x7f7cb8135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- 172.21.15.17:0/2489138149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb809ffc0 0x7f7cb80a11b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- 172.21.15.17:0/2489138149 >> v2:172.21.15.119:3300/0 conn(0x7f7cb80a0890 0x7f7cb80a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/2489138149 >> 172.21.15.17:0/2489138149 conn(0x7f7cb80a66a0 msgr2=0x7f7cb80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/2489138149 shutdown_connections 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/2489138149 wait complete. 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: build_initial_monmap 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: monmap: 2024-09-15T21:17:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 Processor -- start 2024-09-15T21:17:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- start start 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: init 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb809ffc0 0x7f7cb81cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): picked mon.noname-b con 0x7f7cb809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 0x7f7cb8134a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): picked mon.noname-c con 0x7f7cb80a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 0x7f7cb811c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): picked mon.noname-a con 0x7f7cb8135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb813f680 con 0x7f7cb8135670 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb813f500 con 0x7f7cb809ffc0 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb813f800 con 0x7f7cb80a0890 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient(hunting): authenticate will time out at 988.166220s 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 0x7f7cb811c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 0x7f7cb8134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 0x7f7cb8134a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45580/0 (socket says 172.21.15.17:45580) 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 1 -- 172.21.15.17:0/1661843900 learned_addr learned my addr 172.21.15.17:0/1661843900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 0x7f7cb811c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44320/0 (socket says 172.21.15.17:44320) 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): get_auth_request con 0x7f7cb80a0890 auth_method 0 2024-09-15T21:17:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): get_auth_request con 0x7f7cb8135670 auth_method 0 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient(hunting): handle_auth_done global_id 4305 payload 824 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 -- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 msgr2=0x7f7cb8134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 0x7f7cb8134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb659c640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb809ffc0 0x7f7cb81cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 -- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb809ffc0 msgr2=0x7f7cb81cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb809ffc0 0x7f7cb81cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 monclient: found mon.noname-a 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:44.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 -- 172.21.15.17:0/1661843900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb811e0a0 con 0x7f7cb8135670 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb5d9b640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 0x7f7cb8134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cb6d9d640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 0x7f7cb811c900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009e50 tx=0x7f7cb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0018030 con 0x7f7cb8135670 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: dump: 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.792 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb000cd90 con 0x7f7cb8135670 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 5 monclient: authenticate success, global_id 4305 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _renew_subs 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb809d0f0 con 0x7f7cb8135670 2024-09-15T21:17:44.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0003bc0 con 0x7f7cb8135670 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca6ffd640 10 monclient: dump: 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:44.794 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7ca7fff640 4 set_mon_vals no callback set 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _renew_subs 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb811fb60 con 0x7f7cb8135670 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.766+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb8135aa0 con 0x7f7cb8135670 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.770+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7cb001b070 con 0x7f7cb8135670 2024-09-15T21:17:44.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.770+0000 7f7ca6ffd640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7c9803b9c0 0x7f7c9803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.770+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f7cb00502b0 con 0x7f7cb8135670 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.770+0000 7f7cb659c640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7c9803b9c0 0x7f7c9803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.770+0000 7f7cb659c640 10 monclient: get_auth_request con 0x7f7c9803b9c0 auth_method 0 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.770+0000 7f7cb659c640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7c9803b9c0 0x7f7c9803de80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009790 tx=0x7f7cac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.774+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7cb0022070 con 0x7f7cb8135670 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.774+0000 7f7ca6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:44.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.774+0000 7f7ca6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:44.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.938+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cb811ff40 con 0x7f7c9803b9c0 2024-09-15T21:17:44.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.942+0000 7f7ca6ffd640 1 -- 172.21.15.17:0/1661843900 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34220 (secure 0 0 0) 0x7f7cb811ff40 con 0x7f7c9803b9c0 2024-09-15T21:17:44.951 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:17:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7c9803b9c0 msgr2=0x7f7c9803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7c9803b9c0 0x7f7c9803de80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009790 tx=0x7f7cac008040 comp rx=0 tx=0).stop 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 10 monclient: shutdown 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 msgr2=0x7f7cb811c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 0x7f7cb811c900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009e50 tx=0x7f7cb000c490 comp rx=0 tx=0).stop 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 shutdown_connections 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7c9803b9c0 0x7f7c9803de80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb8135670 0x7f7cb811c900 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7cb80a0890 0x7f7cb8134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 --2- 172.21.15.17:0/1661843900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb809ffc0 0x7f7cb81cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 >> 172.21.15.17:0/1661843900 conn(0x7f7cb80a66a0 msgr2=0x7f7cb8136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 shutdown_connections 2024-09-15T21:17:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:44.946+0000 7f7cbc860640 1 -- 172.21.15.17:0/1661843900 wait complete. 2024-09-15T21:17:44.971 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T21:17:44.397237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":171632,"kb_used_data":1108,"kb_used_omap":9,"kb_used_meta":170358,"kb_avail":566059408,"statfs":{"total":579820584960,"available":579644833792,"internally_reserved":0,"allocated":1134592,"data_stored":1396768,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9567,"internal_metadata":174447265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.223117"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276795+0000","last_change":"2024-09-15T21:17:42.277325+0000","last_active":"2024-09-15T21:17:42.276795+0000","last_peered":"2024-09-15T21:17:42.276795+0000","last_clean":"2024-09-15T21:17:42.276795+0000","last_became_active":"2024-09-15T21:17:40.059370+0000","last_became_peered":"2024-09-15T21:17:40.059370+0000","last_unstale":"2024-09-15T21:17:42.276795+0000","last_undegraded":"2024-09-15T21:17:42.276795+0000","last_fullsized":"2024-09-15T21:17:42.276795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:59.298050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.311058+0000","last_change":"2024-09-15T21:17:41.311058+0000","last_active":"2024-09-15T21:17:41.311058+0000","last_peered":"2024-09-15T21:17:41.311058+0000","last_clean":"2024-09-15T21:17:41.311058+0000","last_became_active":"2024-09-15T21:17:39.299252+0000","last_became_peered":"2024-09-15T21:17:39.299252+0000","last_unstale":"2024-09-15T21:17:41.311058+0000","last_undegraded":"2024-09-15T21:17:41.311058+0000","last_fullsized":"2024-09-15T21:17:41.311058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:09.083407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276328+0000","last_change":"2024-09-15T21:17:42.276772+0000","last_active":"2024-09-15T21:17:42.276328+0000","last_peered":"2024-09-15T21:17:42.276328+0000","last_clean":"2024-09-15T21:17:42.276328+0000","last_became_active":"2024-09-15T21:17:39.302570+0000","last_became_peered":"2024-09-15T21:17:39.302570+0000","last_unstale":"2024-09-15T21:17:42.276328+0000","last_undegraded":"2024-09-15T21:17:42.276328+0000","last_fullsized":"2024-09-15T21:17:42.276328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:14.596682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.310852+0000","last_change":"2024-09-15T21:17:41.310852+0000","last_active":"2024-09-15T21:17:41.310852+0000","last_peered":"2024-09-15T21:17:41.310852+0000","last_clean":"2024-09-15T21:17:41.310852+0000","last_became_active":"2024-09-15T21:17:39.297755+0000","last_became_peered":"2024-09-15T21:17:39.297755+0000","last_unstale":"2024-09-15T21:17:41.310852+0000","last_undegraded":"2024-09-15T21:17:41.310852+0000","last_fullsized":"2024-09-15T21:17:41.310852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:03.856073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.320791+0000","last_change":"2024-09-15T21:17:41.320791+0000","last_active":"2024-09-15T21:17:41.320791+0000","last_peered":"2024-09-15T21:17:41.320791+0000","last_clean":"2024-09-15T21:17:41.320791+0000","last_became_active":"2024-09-15T21:17:39.306488+0000","last_became_peered":"2024-09-15T21:17:39.306488+0000","last_unstale":"2024-09-15T21:17:41.320791+0000","last_undegraded":"2024-09-15T21:17:41.320791+0000","last_fullsized":"2024-09-15T21:17:41.320791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:32.809530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.309694+0000","last_change":"2024-09-15T21:17:41.310299+0000","last_active":"2024-09-15T21:17:41.309694+0000","last_peered":"2024-09-15T21:17:41.309694+0000","last_clean":"2024-09-15T21:17:41.309694+0000","last_became_active":"2024-09-15T21:17:39.297311+0000","last_became_peered":"2024-09-15T21:17:39.297311+0000","last_unstale":"2024-09-15T21:17:41.309694+0000","last_undegraded":"2024-09-15T21:17:41.309694+0000","last_fullsized":"2024-09-15T21:17:41.309694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:04.414329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276249+0000","last_change":"2024-09-15T21:17:42.276652+0000","last_active":"2024-09-15T21:17:42.276249+0000","last_peered":"2024-09-15T21:17:42.276249+0000","last_clean":"2024-09-15T21:17:42.276249+0000","last_became_active":"2024-09-15T21:17:39.302172+0000","last_became_peered":"2024-09-15T21:17:39.302172+0000","last_unstale":"2024-09-15T21:17:42.276249+0000","last_undegraded":"2024-09-15T21:17:42.276249+0000","last_fullsized":"2024-09-15T21:17:42.276249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:37.127020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.319475+0000","last_change":"2024-09-15T21:17:41.319914+0000","last_active":"2024-09-15T21:17:41.319475+0000","last_peered":"2024-09-15T21:17:41.319475+0000","last_clean":"2024-09-15T21:17:41.319475+0000","last_became_active":"2024-09-15T21:17:39.306227+0000","last_became_peered":"2024-09-15T21:17:39.306227+0000","last_unstale":"2024-09-15T21:17:41.319475+0000","last_undegraded":"2024-09-15T21:17:41.319475+0000","last_fullsized":"2024-09-15T21:17:41.319475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:19:50.937347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276106+0000","last_change":"2024-09-15T21:17:32.810848+0000","last_active":"2024-09-15T21:17:42.276106+0000","last_peered":"2024-09-15T21:17:42.276106+0000","last_clean":"2024-09-15T21:17:42.276106+0000","last_became_active":"2024-09-15T21:17:32.808947+0000","last_became_peered":"2024-09-15T21:17:32.808947+0000","last_unstale":"2024-09-15T21:17:42.276106+0000","last_undegraded":"2024-09-15T21:17:42.276106+0000","last_fullsized":"2024-09-15T21:17:42.276106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:46:33.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":57307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":581267,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":172032,"data_stored":57307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":586235,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:44.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.086+0000 7ff1a7c62640 10 monclient: get_monmap_and_config 2024-09-15T21:17:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.086+0000 7ff1a7c62640 10 monclient: build_initial_monmap 2024-09-15T21:17:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.086+0000 7ff1a7c62640 10 monclient: monmap: 2024-09-15T21:17:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:45.092974+0000 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:45.092974+0000 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 Processor -- start 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 -- start start 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient: init 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff1a01378c0 0x7ff1a0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): picked mon.noname-c con 0x7ff1a01378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 0x7ff1a00a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): picked mon.noname-b con 0x7ff1a01381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff1a00a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a00a1d00 con 0x7ff1a00a5040 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a00a1fe0 con 0x7ff1a01378c0 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a00a22c0 con 0x7ff1a01381e0 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a7c62640 10 monclient(hunting): authenticate will time out at 988.488985s 2024-09-15T21:17:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 0x7ff1a00a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 0x7ff1a00a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45588/0 (socket says 172.21.15.17:45588) 2024-09-15T21:17:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 -- 172.21.15.17:0/1937794108 learned_addr learned my addr 172.21.15.17:0/1937794108 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): get_auth_request con 0x7ff1a01381e0 auth_method 0 2024-09-15T21:17:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 1 --2- 172.21.15.17:0/1937794108 >> v2:172.21.15.119:3300/0 conn(0x7ff1a01378c0 0x7ff1a0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): get_auth_request con 0x7ff1a01378c0 auth_method 0 2024-09-15T21:17:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient(hunting): handle_auth_done global_id 4301 payload 274 2024-09-15T21:17:45.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 -- 172.21.15.17:0/1937794108 >> v2:172.21.15.119:3300/0 conn(0x7ff1a01378c0 msgr2=0x7ff1a0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:45.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 --2- 172.21.15.17:0/1937794108 >> v2:172.21.15.119:3300/0 conn(0x7ff1a01378c0 0x7ff1a0137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 -- 172.21.15.17:0/1937794108 >> v1:172.21.15.119:6789/0 conn(0x7ff1a00a5040 legacy=0x7ff1a00a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient: _finish_hunting 0 2024-09-15T21:17:45.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 monclient: found mon.noname-b 2024-09-15T21:17:45.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:45.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:45.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.090+0000 7ff1a51fd640 1 -- 172.21.15.17:0/1937794108 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a00a5c00 con 0x7ff1a01381e0 2024-09-15T21:17:45.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a51fd640 1 --2- 172.21.15.17:0/1937794108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 0x7ff1a00a4b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff190009870 tx=0x7ff19001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e830c44a928e9d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 1 -- 172.21.15.17:0/1937794108 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1900b1070 con 0x7ff1a01381e0 2024-09-15T21:17:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: dump: 2024-09-15T21:17:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.117 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.117 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.117 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.117 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.117 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.117 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.118 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.118 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:45.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:45.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:45.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 1 -- 172.21.15.17:0/1937794108 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1900ac410 con 0x7ff1a01381e0 2024-09-15T21:17:45.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 5 monclient: authenticate success, global_id 4301 2024-09-15T21:17:45.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:45.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:45.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:45.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 1 -- 172.21.15.17:0/1937794108 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1900ac7e0 con 0x7ff1a01381e0 2024-09-15T21:17:45.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 4 set_mon_vals no callback set 2024-09-15T21:17:45.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:45.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a49fc640 10 monclient: dump: 2024-09-15T21:17:45.120 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.120 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.122 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: shutdown 2024-09-15T21:17:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- 172.21.15.17:0/1937794108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 msgr2=0x7ff1a00a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/1937794108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 0x7ff1a00a4b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff190009870 tx=0x7ff19001c420 comp rx=0 tx=0).stop 2024-09-15T21:17:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- 172.21.15.17:0/1937794108 shutdown_connections 2024-09-15T21:17:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/1937794108 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a01381e0 0x7ff1a00a4b00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/1937794108 >> v2:172.21.15.119:3300/0 conn(0x7ff1a01378c0 0x7ff1a0137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- 172.21.15.17:0/1937794108 >> 172.21.15.17:0/1937794108 conn(0x7ff1a00aac40 msgr2=0x7ff1a00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- 172.21.15.17:0/1937794108 shutdown_connections 2024-09-15T21:17:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- 172.21.15.17:0/1937794108 wait complete. 2024-09-15T21:17:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: build_initial_monmap 2024-09-15T21:17:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: monmap: 2024-09-15T21:17:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:45.126 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 Processor -- start 2024-09-15T21:17:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- start start 2024-09-15T21:17:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: init 2024-09-15T21:17:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 0x7ff1a01cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): picked mon.noname-c con 0x7ff1a00a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a01378c0 0x7ff1a0136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): picked mon.noname-b con 0x7ff1a01378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1a01381e0 0x7ff1a013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): picked mon.noname-a con 0x7ff1a01381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a006f340 con 0x7ff1a01381e0 2024-09-15T21:17:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a006f1c0 con 0x7ff1a01378c0 2024-09-15T21:17:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a01425c0 con 0x7ff1a00a5040 2024-09-15T21:17:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a7c62640 10 monclient(hunting): authenticate will time out at 988.495300s 2024-09-15T21:17:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 0x7ff1a01cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1a01381e0 0x7ff1a013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 0x7ff1a01cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45604/0 (socket says 172.21.15.17:45604) 2024-09-15T21:17:45.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 -- 172.21.15.17:0/2219939242 learned_addr learned my addr 172.21.15.17:0/2219939242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:45.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): get_auth_request con 0x7ff1a00a5040 auth_method 0 2024-09-15T21:17:45.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): get_auth_request con 0x7ff1a01381e0 auth_method 0 2024-09-15T21:17:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a61ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a51fd640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a01378c0 0x7ff1a0136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a51fd640 10 monclient(hunting): get_auth_request con 0x7ff1a01378c0 auth_method 0 2024-09-15T21:17:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a51fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a51fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a51fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient(hunting): handle_auth_done global_id 4304 payload 824 2024-09-15T21:17:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 -- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a01378c0 msgr2=0x7ff1a0136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a01378c0 0x7ff1a0136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 -- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1a01381e0 msgr2=0x7ff1a013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1a01381e0 0x7ff1a013ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 monclient: found mon.noname-c 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.094+0000 7ff1a59fe640 1 -- 172.21.15.17:0/2219939242 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a0121630 con 0x7ff1a00a5040 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a59fe640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 0x7ff1a01cfba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00e930 tx=0x7ff19c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c017070 con 0x7ff1a00a5040 2024-09-15T21:17:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: dump: 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c00ca00 con 0x7ff1a00a5040 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00cdb0 con 0x7ff1a00a5040 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 5 monclient: authenticate success, global_id 4304 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 monclient: _renew_subs 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a00a12f0 con 0x7ff1a00a5040 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1977fe640 4 set_mon_vals no callback set 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 10 monclient: dump: 2024-09-15T21:17:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 monclient: _renew_subs 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a0120d90 con 0x7ff1a00a5040 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a0137ca0 con 0x7ff1a00a5040 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff19c01a070 con 0x7ff1a00a5040 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff18403b9c0 0x7ff18403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7ff19c04fb80 con 0x7ff1a00a5040 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a51fd640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff18403b9c0 0x7ff18403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a51fd640 10 monclient: get_auth_request con 0x7ff18403b9c0 auth_method 0 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.098+0000 7ff1a51fd640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff18403b9c0 0x7ff18403de80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1900ae000 tx=0x7ff1900a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.102+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff19c013c70 con 0x7ff1a00a5040 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.102+0000 7ff1967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.102+0000 7ff1967fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:45.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.266+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1a0069270 con 0x7ff18403b9c0 2024-09-15T21:17:45.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1967fc640 1 -- 172.21.15.17:0/2219939242 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34220 (secure 0 0 0) 0x7ff1a0069270 con 0x7ff18403b9c0 2024-09-15T21:17:45.279 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff18403b9c0 msgr2=0x7ff18403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff18403b9c0 0x7ff18403de80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1900ae000 tx=0x7ff1900a6040 comp rx=0 tx=0).stop 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 10 monclient: shutdown 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 msgr2=0x7ff1a01cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 0x7ff1a01cfba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00e930 tx=0x7ff19c00edf0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 shutdown_connections 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff18403b9c0 0x7ff18403de80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1a01381e0 0x7ff1a013ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a01378c0 0x7ff1a0136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 --2- 172.21.15.17:0/2219939242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1a00a5040 0x7ff1a01cfba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.270+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 >> 172.21.15.17:0/2219939242 conn(0x7ff1a00aac40 msgr2=0x7ff1a009da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.274+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 shutdown_connections 2024-09-15T21:17:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.274+0000 7ff1a7c62640 1 -- 172.21.15.17:0/2219939242 wait complete. 2024-09-15T21:17:45.296 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T21:17:45.176052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":171632,"kb_used_data":1108,"kb_used_omap":9,"kb_used_meta":170358,"kb_avail":566059408,"statfs":{"total":579820584960,"available":579644833792,"internally_reserved":0,"allocated":1134592,"data_stored":1396768,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9567,"internal_metadata":174447265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878542"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276795+0000","last_change":"2024-09-15T21:17:42.277325+0000","last_active":"2024-09-15T21:17:42.276795+0000","last_peered":"2024-09-15T21:17:42.276795+0000","last_clean":"2024-09-15T21:17:42.276795+0000","last_became_active":"2024-09-15T21:17:40.059370+0000","last_became_peered":"2024-09-15T21:17:40.059370+0000","last_unstale":"2024-09-15T21:17:42.276795+0000","last_undegraded":"2024-09-15T21:17:42.276795+0000","last_fullsized":"2024-09-15T21:17:42.276795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:59.298050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.311058+0000","last_change":"2024-09-15T21:17:41.311058+0000","last_active":"2024-09-15T21:17:41.311058+0000","last_peered":"2024-09-15T21:17:41.311058+0000","last_clean":"2024-09-15T21:17:41.311058+0000","last_became_active":"2024-09-15T21:17:39.299252+0000","last_became_peered":"2024-09-15T21:17:39.299252+0000","last_unstale":"2024-09-15T21:17:41.311058+0000","last_undegraded":"2024-09-15T21:17:41.311058+0000","last_fullsized":"2024-09-15T21:17:41.311058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:09.083407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276328+0000","last_change":"2024-09-15T21:17:42.276772+0000","last_active":"2024-09-15T21:17:42.276328+0000","last_peered":"2024-09-15T21:17:42.276328+0000","last_clean":"2024-09-15T21:17:42.276328+0000","last_became_active":"2024-09-15T21:17:39.302570+0000","last_became_peered":"2024-09-15T21:17:39.302570+0000","last_unstale":"2024-09-15T21:17:42.276328+0000","last_undegraded":"2024-09-15T21:17:42.276328+0000","last_fullsized":"2024-09-15T21:17:42.276328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:14.596682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.310852+0000","last_change":"2024-09-15T21:17:41.310852+0000","last_active":"2024-09-15T21:17:41.310852+0000","last_peered":"2024-09-15T21:17:41.310852+0000","last_clean":"2024-09-15T21:17:41.310852+0000","last_became_active":"2024-09-15T21:17:39.297755+0000","last_became_peered":"2024-09-15T21:17:39.297755+0000","last_unstale":"2024-09-15T21:17:41.310852+0000","last_undegraded":"2024-09-15T21:17:41.310852+0000","last_fullsized":"2024-09-15T21:17:41.310852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:03.856073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.320791+0000","last_change":"2024-09-15T21:17:41.320791+0000","last_active":"2024-09-15T21:17:41.320791+0000","last_peered":"2024-09-15T21:17:41.320791+0000","last_clean":"2024-09-15T21:17:41.320791+0000","last_became_active":"2024-09-15T21:17:39.306488+0000","last_became_peered":"2024-09-15T21:17:39.306488+0000","last_unstale":"2024-09-15T21:17:41.320791+0000","last_undegraded":"2024-09-15T21:17:41.320791+0000","last_fullsized":"2024-09-15T21:17:41.320791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:32.809530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.309694+0000","last_change":"2024-09-15T21:17:41.310299+0000","last_active":"2024-09-15T21:17:41.309694+0000","last_peered":"2024-09-15T21:17:41.309694+0000","last_clean":"2024-09-15T21:17:41.309694+0000","last_became_active":"2024-09-15T21:17:39.297311+0000","last_became_peered":"2024-09-15T21:17:39.297311+0000","last_unstale":"2024-09-15T21:17:41.309694+0000","last_undegraded":"2024-09-15T21:17:41.309694+0000","last_fullsized":"2024-09-15T21:17:41.309694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:04.414329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276249+0000","last_change":"2024-09-15T21:17:42.276652+0000","last_active":"2024-09-15T21:17:42.276249+0000","last_peered":"2024-09-15T21:17:42.276249+0000","last_clean":"2024-09-15T21:17:42.276249+0000","last_became_active":"2024-09-15T21:17:39.302172+0000","last_became_peered":"2024-09-15T21:17:39.302172+0000","last_unstale":"2024-09-15T21:17:42.276249+0000","last_undegraded":"2024-09-15T21:17:42.276249+0000","last_fullsized":"2024-09-15T21:17:42.276249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:37.127020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.319475+0000","last_change":"2024-09-15T21:17:41.319914+0000","last_active":"2024-09-15T21:17:41.319475+0000","last_peered":"2024-09-15T21:17:41.319475+0000","last_clean":"2024-09-15T21:17:41.319475+0000","last_became_active":"2024-09-15T21:17:39.306227+0000","last_became_peered":"2024-09-15T21:17:39.306227+0000","last_unstale":"2024-09-15T21:17:41.319475+0000","last_undegraded":"2024-09-15T21:17:41.319475+0000","last_fullsized":"2024-09-15T21:17:41.319475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:19:50.937347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276106+0000","last_change":"2024-09-15T21:17:32.810848+0000","last_active":"2024-09-15T21:17:42.276106+0000","last_peered":"2024-09-15T21:17:42.276106+0000","last_clean":"2024-09-15T21:17:42.276106+0000","last_became_active":"2024-09-15T21:17:32.808947+0000","last_became_peered":"2024-09-15T21:17:32.808947+0000","last_unstale":"2024-09-15T21:17:42.276106+0000","last_undegraded":"2024-09-15T21:17:42.276106+0000","last_fullsized":"2024-09-15T21:17:42.276106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:46:33.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":57307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":581267,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":172032,"data_stored":57307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":586235,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:45.297 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:17:45.297 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:17:45.298 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:17:45.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:17:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.406+0000 7fc604a22640 10 monclient: get_monmap_and_config 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.406+0000 7fc604a22640 10 monclient: build_initial_monmap 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.406+0000 7fc604a22640 10 monclient: monmap: 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:45.412904+0000 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:45.412904+0000 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:45.417 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.406+0000 7fc604a22640 1 Processor -- start 2024-09-15T21:17:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.406+0000 7fc604a22640 1 -- start start 2024-09-15T21:17:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.406+0000 7fc604a22640 10 monclient: init 2024-09-15T21:17:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:17:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 0x7fc60009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): picked mon.noname-a con 0x7fc60009d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 0x7fc600134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): picked mon.noname-b con 0x7fc60009dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc600134ba0 0x7fc600134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): picked mon.noname-c con 0x7fc600134ba0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc6000a1bc0 con 0x7fc60009d400 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc6000a3490 con 0x7fc600134ba0 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000a3130 con 0x7fc60009dd20 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient(hunting): authenticate will time out at 988.808214s 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 0x7fc60009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 0x7fc600134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 0x7fc60009d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44328/0 (socket says 172.21.15.17:44328) 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 1 -- 172.21.15.17:0/2006242436 learned_addr learned my addr 172.21.15.17:0/2006242436 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): get_auth_request con 0x7fc60009d400 auth_method 0 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2006242436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 0x7fc600134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45620/0 (socket says 172.21.15.17:45620) 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): get_auth_request con 0x7fc60009dd20 auth_method 0 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fed9d640 1 --2- 172.21.15.17:0/2006242436 >> v2:172.21.15.119:3300/0 conn(0x7fc600134ba0 0x7fc600134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fed9d640 10 monclient(hunting): get_auth_request con 0x7fc600134ba0 auth_method 0 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_done global_id 4307 payload 274 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 -- 172.21.15.17:0/2006242436 >> v2:172.21.15.119:3300/0 conn(0x7fc600134ba0 msgr2=0x7fc600134fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2006242436 >> v2:172.21.15.119:3300/0 conn(0x7fc600134ba0 0x7fc600134fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 -- 172.21.15.17:0/2006242436 >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 msgr2=0x7fc60009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2006242436 >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 0x7fc60009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 monclient: found mon.noname-b 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 -- 172.21.15.17:0/2006242436 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60009e7a0 con 0x7fc60009dd20 2024-09-15T21:17:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fe59c640 1 --2- 172.21.15.17:0/2006242436 >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 0x7fc60009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2006242436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 0x7fc600134660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6000a2730 tx=0x7fc5e801c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0ffa4a93b62ded server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 1 -- 172.21.15.17:0/2006242436 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b0070 con 0x7fc60009dd20 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: dump: 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: _finish_auth 0 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: _check_auth_tickets 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 1 -- 172.21.15.17:0/2006242436 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e80ab600 con 0x7fc60009dd20 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 1 -- 172.21.15.17:0/2006242436 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80ab9d0 con 0x7fc60009dd20 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 5 monclient: authenticate success, global_id 4307 2024-09-15T21:17:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 4 set_mon_vals no callback set 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc5fd59a640 10 monclient: dump: 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.431 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 10 monclient: shutdown 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- 172.21.15.17:0/2006242436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 msgr2=0x7fc600134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- 172.21.15.17:0/2006242436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 0x7fc600134660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6000a2730 tx=0x7fc5e801c340 comp rx=0 tx=0).stop 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- 172.21.15.17:0/2006242436 shutdown_connections 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- 172.21.15.17:0/2006242436 >> v2:172.21.15.119:3300/0 conn(0x7fc600134ba0 0x7fc600134fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- 172.21.15.17:0/2006242436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc60009dd20 0x7fc600134660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 --2- 172.21.15.17:0/2006242436 >> v2:172.21.15.17:3300/0 conn(0x7fc60009d400 0x7fc60009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- 172.21.15.17:0/2006242436 >> 172.21.15.17:0/2006242436 conn(0x7fc6000a66a0 msgr2=0x7fc6000a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- 172.21.15.17:0/2006242436 shutdown_connections 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.410+0000 7fc604a22640 1 -- 172.21.15.17:0/2006242436 wait complete. 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: build_initial_monmap 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: monmap: 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:45.433 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 Processor -- start 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- start start 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: init 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc60009d400 0x7fc6001cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): picked mon.noname-b con 0x7fc60009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60009dd20 0x7fc6001337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): picked mon.noname-a con 0x7fc60009dd20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 0x7fc600133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): picked mon.noname-c con 0x7fc600134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc60009fb80 con 0x7fc60009dd20 2024-09-15T21:17:45.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc60009fa00 con 0x7fc60009d400 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc60013e6f0 con 0x7fc600134ba0 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient(hunting): authenticate will time out at 988.813356s 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 0x7fc600133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60009dd20 0x7fc6001337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 0x7fc600133cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45630/0 (socket says 172.21.15.17:45630) 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): get_auth_request con 0x7fc60009dd20 auth_method 0 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 1 -- 172.21.15.17:0/2074235300 learned_addr learned my addr 172.21.15.17:0/2074235300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): get_auth_request con 0x7fc600134ba0 auth_method 0 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fe59c640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc60009d400 0x7fc6001cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fe59c640 10 monclient(hunting): get_auth_request con 0x7fc60009d400 auth_method 0 2024-09-15T21:17:45.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient(hunting): handle_auth_done global_id 4314 payload 824 2024-09-15T21:17:45.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 -- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 msgr2=0x7fc600133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 0x7fc600133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 -- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc60009d400 msgr2=0x7fc6001cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc60009d400 0x7fc6001cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 monclient: found mon.noname-a 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 -- 172.21.15.17:0/2074235300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60011de20 con 0x7fc60009dd20 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fed9d640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 0x7fc600133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fdd9b640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60009dd20 0x7fc6001337b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80ab000 tx=0x7fc5e80190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b8020 con 0x7fc60009dd20 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:45.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: dump: 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:45.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e80b4410 con 0x7fc60009dd20 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b0040 con 0x7fc60009dd20 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 10 monclient: dump: 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.443 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 5 monclient: authenticate success, global_id 4314 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _renew_subs 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5fcd99640 4 set_mon_vals no callback set 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc600120180 con 0x7fc60009dd20 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _renew_subs 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60011e980 con 0x7fc60009dd20 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600134660 con 0x7fc60009dd20 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc5e80bb020 con 0x7fc60009dd20 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc5e003b9c0 0x7fc5e003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.414+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fc5e80e9510 con 0x7fc60009dd20 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.418+0000 7fc5fe59c640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc5e003b9c0 0x7fc5e003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.418+0000 7fc5fe59c640 10 monclient: get_auth_request con 0x7fc5e003b9c0 auth_method 0 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.418+0000 7fc5fe59c640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc5e003b9c0 0x7fc5e003de80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009240 tx=0x7fc5f000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.422+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc5e80c2030 con 0x7fc60009dd20 2024-09-15T21:17:45.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.422+0000 7fc5f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.422+0000 7fc5f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:45.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-09-15T21:17:45.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-09-15T21:17:45.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc60011ec50 con 0x7fc60009dd20 2024-09-15T21:17:45.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc5f6ffd640 1 -- 172.21.15.17:0/2074235300 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc5e8016050 con 0x7fc60009dd20 2024-09-15T21:17:45.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc5f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-09-15T21:17:45.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc5f6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:45.638 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:45.638 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc5e003b9c0 msgr2=0x7fc5e003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc5e003b9c0 0x7fc5e003de80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009240 tx=0x7fc5f000b040 comp rx=0 tx=0).stop 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 10 monclient: shutdown 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60009dd20 msgr2=0x7fc6001337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60009dd20 0x7fc6001337b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80ab000 tx=0x7fc5e80190e0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 shutdown_connections 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc5e003b9c0 0x7fc5e003de80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc600134ba0 0x7fc600133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60009dd20 0x7fc6001337b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 --2- 172.21.15.17:0/2074235300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc60009d400 0x7fc6001cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 >> 172.21.15.17:0/2074235300 conn(0x7fc6000a66a0 msgr2=0x7fc6000a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 shutdown_connections 2024-09-15T21:17:45.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.630+0000 7fc604a22640 1 -- 172.21.15.17:0/2074235300 wait complete. 2024-09-15T21:17:45.657 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:17:45.658 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-15T21:17:45.676 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-15T21:17:45.677 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:45.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient: get_monmap_and_config 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient: build_initial_monmap 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient: monmap: 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:45.794891+0000 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:45.794891+0000 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:45.798 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:45.799 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:45.799 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 Processor -- start 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 -- start start 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient: init 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f609809cad0 0x7f60980a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): picked mon.noname-a con 0x7f609809cad0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f609809ceb0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f609813cda0 0x7f609813f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): picked mon.noname-c con 0x7f609813cda0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f609809bd80 con 0x7f609809ceb0 2024-09-15T21:17:45.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f609809bff0 con 0x7f609809cad0 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f609809c2d0 con 0x7f609813cda0 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f609db56640 10 monclient(hunting): authenticate will time out at 989.190468s 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f609809cad0 0x7f60980a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f609809cad0 0x7f60980a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44358/0 (socket says 172.21.15.17:44358) 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 -- 172.21.15.17:0/3971174917 learned_addr learned my addr 172.21.15.17:0/3971174917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): get_auth_request con 0x7f609809cad0 auth_method 0 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f6097fff640 1 --2- 172.21.15.17:0/3971174917 >> v2:172.21.15.119:3300/0 conn(0x7f609813cda0 0x7f609813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f6097fff640 10 monclient(hunting): get_auth_request con 0x7f609813cda0 auth_method 0 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f6097fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f6097fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f6097fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient(hunting): handle_auth_done global_id 4317 payload 274 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 -- 172.21.15.17:0/3971174917 >> v2:172.21.15.119:3300/0 conn(0x7f609813cda0 msgr2=0x7f609813f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 --2- 172.21.15.17:0/3971174917 >> v2:172.21.15.119:3300/0 conn(0x7f609813cda0 0x7f609813f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 -- 172.21.15.17:0/3971174917 >> v1:172.21.15.119:6789/0 conn(0x7f609809ceb0 legacy=0x7f609809d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 monclient: found mon.noname-a 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:45.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.790+0000 7f60977fe640 1 -- 172.21.15.17:0/3971174917 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60980a4300 con 0x7f609809cad0 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f60977fe640 1 --2- 172.21.15.17:0/3971174917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809cad0 0x7f60980a2dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f609806d880 tx=0x7f608801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ae8245a91f02a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 1 -- 172.21.15.17:0/3971174917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60880b0070 con 0x7f609809cad0 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: dump: 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.809 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 1 -- 172.21.15.17:0/3971174917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60880ab600 con 0x7f609809cad0 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 1 -- 172.21.15.17:0/3971174917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60880ab9d0 con 0x7f609809cad0 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f6096ffd640 10 monclient: dump: 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.811 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 5 monclient: authenticate success, global_id 4317 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 4 set_mon_vals no callback set 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: shutdown 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- 172.21.15.17:0/3971174917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809cad0 msgr2=0x7f60980a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 --2- 172.21.15.17:0/3971174917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809cad0 0x7f60980a2dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f609806d880 tx=0x7f608801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- 172.21.15.17:0/3971174917 shutdown_connections 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 --2- 172.21.15.17:0/3971174917 >> v2:172.21.15.119:3300/0 conn(0x7f609813cda0 0x7f609813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 --2- 172.21.15.17:0/3971174917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809cad0 0x7f60980a2dc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- 172.21.15.17:0/3971174917 >> 172.21.15.17:0/3971174917 conn(0x7f60980a75d0 msgr2=0x7f60980a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- 172.21.15.17:0/3971174917 shutdown_connections 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- 172.21.15.17:0/3971174917 wait complete. 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: build_initial_monmap 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: monmap: 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:45.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 Processor -- start 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- start start 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: init 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 0x7f60981cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient(hunting): picked mon.noname-c con 0x7f609809cad0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809ceb0 0x7f60981cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient(hunting): picked mon.noname-a con 0x7f609809ceb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609813cda0 0x7f60981cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient(hunting): picked mon.noname-b con 0x7f609813cda0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.794+0000 7f609db56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6098142280 con 0x7f609809ceb0 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6098142100 con 0x7f609813cda0 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6098142400 con 0x7f609809cad0 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient(hunting): authenticate will time out at 989.196025s 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 0x7f60981cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809ceb0 0x7f60981cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 0x7f60981cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45638/0 (socket says 172.21.15.17:45638) 2024-09-15T21:17:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 1 -- 172.21.15.17:0/732207730 learned_addr learned my addr 172.21.15.17:0/732207730 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): get_auth_request con 0x7f609809cad0 auth_method 0 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): get_auth_request con 0x7f609809ceb0 auth_method 0 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:45.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6097fff640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609813cda0 0x7f60981cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient(hunting): handle_auth_done global_id 4320 payload 824 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 -- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 msgr2=0x7f60981cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 0x7f60981cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 -- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609813cda0 msgr2=0x7f60981cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609813cda0 0x7f60981cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 monclient: found mon.noname-a 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 -- 172.21.15.17:0/732207730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60980a0040 con 0x7f609809ceb0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6097fff640 10 monclient: get_auth_request con 0x7f609813cda0 auth_method 0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 0x7f60981cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6097fff640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609813cda0 0x7f60981cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f608ffff640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809ceb0 0x7f60981cfa30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f608000c430 tx=0x7f608000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080017070 con 0x7f609809ceb0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: dump: 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: _finish_auth 0 2024-09-15T21:17:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 5 monclient: authenticate success, global_id 4320 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient: _renew_subs 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60800052d0 con 0x7f609809ceb0 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609813d380 con 0x7f609809ceb0 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080003820 con 0x7f609809ceb0 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 10 monclient: dump: 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60967fc640 4 set_mon_vals no callback set 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient: _renew_subs 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609813b7e0 con 0x7f609809ceb0 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609809e100 con 0x7f609809ceb0 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f608001a070 con 0x7f609809ceb0 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f607803b9f0 0x7f607803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f608004ed40 con 0x7f609809ceb0 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f607803b9f0 0x7f607803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 10 monclient: get_auth_request con 0x7f607803b9f0 auth_method 0 2024-09-15T21:17:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.798+0000 7f60977fe640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f607803b9f0 0x7f607803deb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f608801c420 tx=0x7f6088008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.802+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f60800039c0 con 0x7f609809ceb0 2024-09-15T21:17:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.802+0000 7f6094ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.802+0000 7f6094ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:45.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.970+0000 7f609db56640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:45.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.970+0000 7f609db56640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:45.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.970+0000 7f609db56640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:45.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.970+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60980a2da0 con 0x7f609809ceb0 2024-09-15T21:17:45.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f6094ff9640 1 -- 172.21.15.17:0/732207730 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12346 (secure 0 0 0) 0x7f6080011680 con 0x7f609809ceb0 2024-09-15T21:17:45.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f6094ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:17:45.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f6094ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:45.981 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":19,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:17:44.391397+0000","last_up_change":"2024-09-15T21:17:31.232299+0000","last_in_change":"2024-09-15T21:17:22.374306+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:17:31.217943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:17:37.652178+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6809","nonce":3272576910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6811","nonce":3272576910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6815","nonce":3272576910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6813","nonce":3272576910}]},"public_addr":"172.21.15.17:6809/3272576910","cluster_addr":"172.21.15.17:6811/3272576910","heartbeat_back_addr":"172.21.15.17:6815/3272576910","heartbeat_front_addr":"172.21.15.17:6813/3272576910","state":["exists","up"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6817","nonce":2555706928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6819","nonce":2555706928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6823","nonce":2555706928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6821","nonce":2555706928}]},"public_addr":"172.21.15.17:6817/2555706928","cluster_addr":"172.21.15.17:6819/2555706928","heartbeat_back_addr":"172.21.15.17:6823/2555706928","heartbeat_front_addr":"172.21.15.17:6821/2555706928","state":["exists","up"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6801","nonce":671096025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6803","nonce":671096025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6807","nonce":671096025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6805","nonce":671096025}]},"public_addr":"172.21.15.17:6801/671096025","cluster_addr":"172.21.15.17:6803/671096025","heartbeat_back_addr":"172.21.15.17:6807/671096025","heartbeat_front_addr":"172.21.15.17:6805/671096025","state":["exists","up"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6801","nonce":3353277098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6803","nonce":3353277098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6807","nonce":3353277098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6805","nonce":3353277098}]},"public_addr":"172.21.15.119:6801/3353277098","cluster_addr":"172.21.15.119:6803/3353277098","heartbeat_back_addr":"172.21.15.119:6807/3353277098","heartbeat_front_addr":"172.21.15.119:6805/3353277098","state":["exists","up"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6817","nonce":2086984234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6819","nonce":2086984234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6823","nonce":2086984234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6821","nonce":2086984234}]},"public_addr":"172.21.15.119:6817/2086984234","cluster_addr":"172.21.15.119:6819/2086984234","heartbeat_back_addr":"172.21.15.119:6823/2086984234","heartbeat_front_addr":"172.21.15.119:6821/2086984234","state":["exists","up"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6809","nonce":444741843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6811","nonce":444741843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6815","nonce":444741843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6813","nonce":444741843}]},"public_addr":"172.21.15.119:6809/444741843","cluster_addr":"172.21.15.119:6811/444741843","heartbeat_back_addr":"172.21.15.119:6815/444741843","heartbeat_front_addr":"172.21.15.119:6813/444741843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.768611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.237742+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.809091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.353555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.041478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.487022+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f607803b9f0 msgr2=0x7f607803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f607803b9f0 0x7f607803deb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f608801c420 tx=0x7f6088008040 comp rx=0 tx=0).stop 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 10 monclient: shutdown 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809ceb0 msgr2=0x7f60981cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809ceb0 0x7f60981cfa30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f608000c430 tx=0x7f608000c8f0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 shutdown_connections 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f607803b9f0 0x7f607803deb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609813cda0 0x7f60981cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f609809ceb0 0x7f60981cfa30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 --2- 172.21.15.17:0/732207730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f609809cad0 0x7f60981cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:45.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 >> 172.21.15.17:0/732207730 conn(0x7f60980a75d0 msgr2=0x7f609813c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:45.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 shutdown_connections 2024-09-15T21:17:45.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:45.974+0000 7f609db56640 1 -- 172.21.15.17:0/732207730 wait complete. 2024-09-15T21:17:46.000 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:17:31.217943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:17:37.652178+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T21:17:46.000 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T21:17:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient: get_monmap_and_config 2024-09-15T21:17:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient: build_initial_monmap 2024-09-15T21:17:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient: monmap: 2024-09-15T21:17:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:46.122387+0000 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:46.122387+0000 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:46.126 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 Processor -- start 2024-09-15T21:17:46.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 -- start start 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient: init 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24b00a0890 0x7f24b00a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): picked mon.noname-c con 0x7f24b00a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24b009ffc0 0x7f24b00a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): picked mon.noname-a con 0x7f24b009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f24b0135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b0073df0 con 0x7f24b0135670 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f24b00a3650 con 0x7f24b009ffc0 2024-09-15T21:17:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f24b006f2a0 con 0x7f24b00a0890 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b79c4640 10 monclient(hunting): authenticate will time out at 989.517784s 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24b009ffc0 0x7f24b00a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24b009ffc0 0x7f24b00a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44386/0 (socket says 172.21.15.17:44386) 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 -- 172.21.15.17:0/3967014376 learned_addr learned my addr 172.21.15.17:0/3967014376 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): get_auth_request con 0x7f24b009ffc0 auth_method 0 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b5f61640 1 --1- 172.21.15.17:0/3967014376 >> v1:172.21.15.17:6789/0 conn(0x7f24b0135670 0x7f24b0135b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54676/0 (socket says 172.21.15.17:54676) 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b5760640 1 --2- 172.21.15.17:0/3967014376 >> v2:172.21.15.119:3300/0 conn(0x7f24b00a0890 0x7f24b00a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b5760640 10 monclient(hunting): get_auth_request con 0x7f24b00a0890 auth_method 0 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b5760640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b5760640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b5760640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_done global_id 4323 payload 274 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 -- 172.21.15.17:0/3967014376 >> v2:172.21.15.119:3300/0 conn(0x7f24b00a0890 msgr2=0x7f24b00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 --2- 172.21.15.17:0/3967014376 >> v2:172.21.15.119:3300/0 conn(0x7f24b00a0890 0x7f24b00a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 -- 172.21.15.17:0/3967014376 >> v1:172.21.15.17:6789/0 conn(0x7f24b0135670 legacy=0x7f24b0135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient: _finish_hunting 0 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 monclient: found mon.noname-a 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24b4f5f640 1 -- 172.21.15.17:0/3967014376 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b00a2490 con 0x7f24b009ffc0 2024-09-15T21:17:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24a7fff640 1 -- 172.21.15.17:0/3967014376 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195937390 0 0) 0x7f24b0073df0 con 0x7f24b0135670 2024-09-15T21:17:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.118+0000 7f24a7fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 1 --2- 172.21.15.17:0/3967014376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b009ffc0 0x7f24b00a11b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24b009d0a0 tx=0x7f24a001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9dfbaad1770754 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 1 -- 172.21.15.17:0/3967014376 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00b0070 con 0x7f24b009ffc0 2024-09-15T21:17:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: dump: 2024-09-15T21:17:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: _finish_auth 0 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 1 -- 172.21.15.17:0/3967014376 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a00ab600 con 0x7f24b009ffc0 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 1 -- 172.21.15.17:0/3967014376 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00ab9d0 con 0x7f24b009ffc0 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 5 monclient: authenticate success, global_id 4323 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 4 set_mon_vals no callback set 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a7fff640 10 monclient: dump: 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: shutdown 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- 172.21.15.17:0/3967014376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b009ffc0 msgr2=0x7f24b00a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3967014376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b009ffc0 0x7f24b00a11b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24b009d0a0 tx=0x7f24a001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- 172.21.15.17:0/3967014376 shutdown_connections 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3967014376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b009ffc0 0x7f24b00a11b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3967014376 >> v2:172.21.15.119:3300/0 conn(0x7f24b00a0890 0x7f24b00a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- 172.21.15.17:0/3967014376 >> 172.21.15.17:0/3967014376 conn(0x7f24b00a6690 msgr2=0x7f24b00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- 172.21.15.17:0/3967014376 shutdown_connections 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- 172.21.15.17:0/3967014376 wait complete. 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: build_initial_monmap 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: monmap: 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:46.141 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 Processor -- start 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- start start 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: init 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b009ffc0 0x7f24b01342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): picked mon.noname-b con 0x7f24b009ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 0x7f24b01347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): picked mon.noname-c con 0x7f24b00a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 0x7f24b011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): picked mon.noname-a con 0x7f24b0135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f24b009f830 con 0x7f24b0135670 2024-09-15T21:17:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f24b00a16f0 con 0x7f24b009ffc0 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24b0140510 con 0x7f24b00a0890 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient(hunting): authenticate will time out at 989.522979s 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 0x7f24b011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 0x7f24b01347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 0x7f24b011c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44394/0 (socket says 172.21.15.17:44394) 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 0x7f24b01347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45642/0 (socket says 172.21.15.17:45642) 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 -- 172.21.15.17:0/3184017069 learned_addr learned my addr 172.21.15.17:0/3184017069 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): get_auth_request con 0x7f24b0135670 auth_method 0 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): get_auth_request con 0x7f24b00a0890 auth_method 0 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5760640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b009ffc0 0x7f24b01342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient(hunting): handle_auth_done global_id 4329 payload 824 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 -- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 msgr2=0x7f24b01347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 0x7f24b01347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 -- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b009ffc0 msgr2=0x7f24b01342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b009ffc0 0x7f24b01342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient: _finish_hunting 0 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 monclient: found mon.noname-a 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 -- 172.21.15.17:0/3184017069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b011ed00 con 0x7f24b0135670 2024-09-15T21:17:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b4f5f640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 0x7f24b01347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:46.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5760640 10 monclient: get_auth_request con 0x7f24b009ffc0 auth_method 0 2024-09-15T21:17:46.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5760640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b009ffc0 0x7f24b01342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:46.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b5f61640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 0x7f24b011c4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c7a0 tx=0x7f24a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8018070 con 0x7f24b0135670 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: dump: 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a800a640 con 0x7f24b0135670 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 5 monclient: authenticate success, global_id 4329 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: _renew_subs 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b011efe0 con 0x7f24b0135670 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a800aa70 con 0x7f24b0135670 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24a5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24a5ffb640 10 monclient: dump: 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.122+0000 7f24a6ffd640 4 set_mon_vals no callback set 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 10 monclient: _renew_subs 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b0134d20 con 0x7f24b0135670 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b0135ac0 con 0x7f24b0135670 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f24a801b070 con 0x7f24b0135670 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24a5ffb640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f249403b9c0 0x7f249403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f24a804eee0 con 0x7f24b0135670 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b5760640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f249403b9c0 0x7f249403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b5760640 10 monclient: get_auth_request con 0x7f249403b9c0 auth_method 0 2024-09-15T21:17:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.126+0000 7f24b5760640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f249403b9c0 0x7f249403de80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009840 tx=0x7f24ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.130+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f24a800adf0 con 0x7f24b0135670 2024-09-15T21:17:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.130+0000 7f24a5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.130+0000 7f24a5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:46.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24b79c4640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T21:17:46.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24b79c4640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T21:17:46.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24b79c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24b006d020 con 0x7f24b0135670 2024-09-15T21:17:46.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24a5ffb640 1 -- 172.21.15.17:0/3184017069 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f24a8007980 con 0x7f24b0135670 2024-09-15T21:17:46.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24a5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T21:17:46.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.294+0000 7f24a5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:46.304 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f249403b9c0 msgr2=0x7f249403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f249403b9c0 0x7f249403de80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009840 tx=0x7f24ac008040 comp rx=0 tx=0).stop 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 10 monclient: shutdown 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 msgr2=0x7f24b011c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 0x7f24b011c4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c7a0 tx=0x7f24a800cc60 comp rx=0 tx=0).stop 2024-09-15T21:17:46.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 shutdown_connections 2024-09-15T21:17:46.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f249403b9c0 0x7f249403de80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24b0135670 0x7f24b011c4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24b00a0890 0x7f24b01347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 --2- 172.21.15.17:0/3184017069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b009ffc0 0x7f24b01342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 >> 172.21.15.17:0/3184017069 conn(0x7f24b00a6690 msgr2=0x7f24b00a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:46.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 shutdown_connections 2024-09-15T21:17:46.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.298+0000 7f24b79c4640 1 -- 172.21.15.17:0/3184017069 wait complete. 2024-09-15T21:17:46.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-15T21:17:46.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient: get_monmap_and_config 2024-09-15T21:17:46.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient: build_initial_monmap 2024-09-15T21:17:46.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient: monmap: 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:46.446880+0000 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:46.446880+0000 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:46.534 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 Processor -- start 2024-09-15T21:17:46.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 -- start start 2024-09-15T21:17:46.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient: init 2024-09-15T21:17:46.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f285409d400 0x7f285409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): picked mon.noname-c con 0x7f285409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f285409dd20 0x7f2854134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): picked mon.noname-b con 0x7f285409dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2854134ba0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28540a1bc0 con 0x7f2854134ba0 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f28540a35e0 con 0x7f285409d400 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f285409f040 con 0x7f285409dd20 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285c2d7640 10 monclient(hunting): authenticate will time out at 989.843013s 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285a874640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2854134ba0 0x7f285413bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54688/0 (socket says 172.21.15.17:54688) 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f285409dd20 0x7f2854134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285a874640 1 -- 172.21.15.17:0/2429118844 learned_addr learned my addr 172.21.15.17:0/2429118844 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): get_auth_request con 0x7f285409dd20 auth_method 0 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285a073640 1 --2- 172.21.15.17:0/2429118844 >> v2:172.21.15.119:3300/0 conn(0x7f285409d400 0x7f285409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient(hunting): handle_auth_done global_id 4319 payload 274 2024-09-15T21:17:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859071640 1 -- 172.21.15.17:0/2429118844 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686164374 0 0) 0x7f28540a1bc0 con 0x7f2854134ba0 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 1 -- 172.21.15.17:0/2429118844 >> v2:172.21.15.119:3300/0 conn(0x7f285409d400 msgr2=0x7f285409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 1 --2- 172.21.15.17:0/2429118844 >> v2:172.21.15.119:3300/0 conn(0x7f285409d400 0x7f285409d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 1 -- 172.21.15.17:0/2429118844 >> v1:172.21.15.17:6789/0 conn(0x7f2854134ba0 legacy=0x7f285413bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient: _finish_hunting 0 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 1 monclient: found mon.noname-b 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859872640 1 -- 172.21.15.17:0/2429118844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28540a0320 con 0x7f285409dd20 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f2859071640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285a073640 10 monclient: get_auth_request con 0x7f285409d400 auth_method 0 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.442+0000 7f285a073640 1 --2- 172.21.15.17:0/2429118844 >> v2:172.21.15.119:3300/0 conn(0x7f285409d400 0x7f285409d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:17:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859872640 1 --2- 172.21.15.17:0/2429118844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f285409dd20 0x7f2854134660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28440097c0 tx=0x7f284401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a71f4f40d49da2c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 1 -- 172.21.15.17:0/2429118844 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28440b2070 con 0x7f285409dd20 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: dump: 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.545 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: _finish_auth 0 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: _check_auth_tickets 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 1 -- 172.21.15.17:0/2429118844 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28440ad640 con 0x7f285409dd20 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 5 monclient: authenticate success, global_id 4319 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 1 -- 172.21.15.17:0/2429118844 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28440ada10 con 0x7f285409dd20 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 4 set_mon_vals no callback set 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f2859071640 10 monclient: dump: 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.547 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: shutdown 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- 172.21.15.17:0/2429118844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f285409dd20 msgr2=0x7f2854134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 --2- 172.21.15.17:0/2429118844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f285409dd20 0x7f2854134660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28440097c0 tx=0x7f284401c480 comp rx=0 tx=0).stop 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- 172.21.15.17:0/2429118844 shutdown_connections 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 --2- 172.21.15.17:0/2429118844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f285409dd20 0x7f2854134660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 --2- 172.21.15.17:0/2429118844 >> v2:172.21.15.119:3300/0 conn(0x7f285409d400 0x7f285409d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- 172.21.15.17:0/2429118844 >> 172.21.15.17:0/2429118844 conn(0x7f28540a6690 msgr2=0x7f28540a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:46.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- 172.21.15.17:0/2429118844 shutdown_connections 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- 172.21.15.17:0/2429118844 wait complete. 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: build_initial_monmap 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: monmap: 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:46.549 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 Processor -- start 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- start start 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: init 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285409d400 0x7f28541cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): picked mon.noname-a con 0x7f285409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f285409dd20 0x7f285412eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): picked mon.noname-b con 0x7f285409dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 0x7f285412f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): picked mon.noname-c con 0x7f2854134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:46.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f285409fa70 con 0x7f285409d400 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f285409fbf0 con 0x7f285409dd20 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f285409ccd0 con 0x7f2854134ba0 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.446+0000 7f285c2d7640 10 monclient(hunting): authenticate will time out at 989.847996s 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 0x7f285412f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285409d400 0x7f28541cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 0x7f285412f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45660/0 (socket says 172.21.15.17:45660) 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285409d400 0x7f28541cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44396/0 (socket says 172.21.15.17:44396) 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 -- 172.21.15.17:0/3085003763 learned_addr learned my addr 172.21.15.17:0/3085003763 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): get_auth_request con 0x7f2854134ba0 auth_method 0 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): get_auth_request con 0x7f285409d400 auth_method 0 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f2859872640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f285409dd20 0x7f285412eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f2859872640 10 monclient(hunting): get_auth_request con 0x7f285409dd20 auth_method 0 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f2859872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f2859872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f2859872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient(hunting): handle_auth_done global_id 4322 payload 824 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 -- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f285409dd20 msgr2=0x7f285412eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f285409dd20 0x7f285412eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 -- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285409d400 msgr2=0x7f28541cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285409d400 0x7f28541cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient: _finish_hunting 0 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 monclient: found mon.noname-c 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 -- 172.21.15.17:0/3085003763 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2854071100 con 0x7f2854134ba0 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a874640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 0x7f285412f040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c7a0 tx=0x7f284c00cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c018070 con 0x7f2854134ba0 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: dump: 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: _finish_auth 0 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c00a640 con 0x7f2854134ba0 2024-09-15T21:17:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c00aa70 con 0x7f2854134ba0 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 5 monclient: authenticate success, global_id 4322 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 monclient: _renew_subs 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285406fbe0 con 0x7f2854134ba0 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f2858870640 4 set_mon_vals no callback set 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 10 monclient: dump: 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.556 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 monclient: _renew_subs 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2854070e20 con 0x7f2854134ba0 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f284c00ac10 con 0x7f2854134ba0 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285406f9c0 con 0x7f2854134ba0 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283803b9c0 0x7f283803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f284c04f070 con 0x7f2854134ba0 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283803b9c0 0x7f283803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 10 monclient: get_auth_request con 0x7f283803b9c0 auth_method 0 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.450+0000 7f285a073640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283803b9c0 0x7f283803de80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f285000ab60 tx=0x7f2850008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.454+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f285406f9c0 con 0x7f2854134ba0 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.454+0000 7f284affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.454+0000 7f284affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:46.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f285c2d7640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T21:17:46.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f285c2d7640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T21:17:46.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f285c2d7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) -- 0x7f285409d7e0 con 0x7f2854134ba0 2024-09-15T21:17:46.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f284affd640 1 -- 172.21.15.17:0/3085003763 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) ==== 92+0+10 (secure 0 0 0) 0x7f284c01b360 con 0x7f2854134ba0 2024-09-15T21:17:46.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f284affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T21:17:46.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.618+0000 7f284affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:46.629 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283803b9c0 msgr2=0x7f283803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283803b9c0 0x7f283803de80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f285000ab60 tx=0x7f2850008040 comp rx=0 tx=0).stop 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 10 monclient: shutdown 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 msgr2=0x7f285412f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 0x7f285412f040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c7a0 tx=0x7f284c00cc60 comp rx=0 tx=0).stop 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 shutdown_connections 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283803b9c0 0x7f283803de80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2854134ba0 0x7f285412f040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f285409dd20 0x7f285412eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 --2- 172.21.15.17:0/3085003763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285409d400 0x7f28541cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 >> 172.21.15.17:0/3085003763 conn(0x7f28540a6690 msgr2=0x7f28540a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:46.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 shutdown_connections 2024-09-15T21:17:46.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.622+0000 7f285c2d7640 1 -- 172.21.15.17:0/3085003763 wait complete. 2024-09-15T21:17:46.646 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-15T21:17:46.646 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.754+0000 7f2224bc8640 10 monclient: get_monmap_and_config 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.754+0000 7f2224bc8640 10 monclient: build_initial_monmap 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.754+0000 7f2224bc8640 10 monclient: monmap: 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:46.761337+0000 2024-09-15T21:17:46.764 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:46.761337+0000 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:46.765 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 Processor -- start 2024-09-15T21:17:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 -- start start 2024-09-15T21:17:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient: init 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22201340c0 0x7f22201344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): picked mon.noname-c con 0x7f22201340c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f222009d6c0 0x7f22200a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): picked mon.noname-a con 0x7f222009d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f22200a0dd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22201349e0 con 0x7f22200a0dd0 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f222009d520 con 0x7f222009d6c0 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f222009e1b0 con 0x7f22201340c0 2024-09-15T21:17:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient(hunting): authenticate will time out at 990.157222s 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f222009d6c0 0x7f22200a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f222009d6c0 0x7f22200a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44408/0 (socket says 172.21.15.17:44408) 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 -- 172.21.15.17:0/431231004 learned_addr learned my addr 172.21.15.17:0/431231004 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): get_auth_request con 0x7f222009d6c0 auth_method 0 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221e59c640 1 --2- 172.21.15.17:0/431231004 >> v2:172.21.15.119:3300/0 conn(0x7f22201340c0 0x7f22201344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221e59c640 10 monclient(hunting): get_auth_request con 0x7f22201340c0 auth_method 0 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient(hunting): handle_auth_done global_id 4338 payload 274 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 -- 172.21.15.17:0/431231004 >> v2:172.21.15.119:3300/0 conn(0x7f22201340c0 msgr2=0x7f22201344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 --2- 172.21.15.17:0/431231004 >> v2:172.21.15.119:3300/0 conn(0x7f22201340c0 0x7f22201344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 -- 172.21.15.17:0/431231004 >> v1:172.21.15.119:6789/0 conn(0x7f22200a0dd0 legacy=0x7f222013ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 monclient: found mon.noname-a 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 -- 172.21.15.17:0/431231004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222009e490 con 0x7f222009d6c0 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2217fff640 1 --2- 172.21.15.17:0/431231004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f222009d6c0 0x7f22200a0890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f222009d0a0 tx=0x7f220801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ced764a41ab3c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 1 -- 172.21.15.17:0/431231004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080b0070 con 0x7f222009d6c0 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: dump: 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.775 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 1 -- 172.21.15.17:0/431231004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22080ab600 con 0x7f222009d6c0 2024-09-15T21:17:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 5 monclient: authenticate success, global_id 4338 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 1 -- 172.21.15.17:0/431231004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080ab9d0 con 0x7f222009d6c0 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f2224bc8640 4 set_mon_vals no callback set 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.758+0000 7f221dd9b640 10 monclient: dump: 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.777 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: shutdown 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/431231004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f222009d6c0 msgr2=0x7f22200a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 --2- 172.21.15.17:0/431231004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f222009d6c0 0x7f22200a0890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f222009d0a0 tx=0x7f220801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/431231004 shutdown_connections 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 --2- 172.21.15.17:0/431231004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f222009d6c0 0x7f22200a0890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 --2- 172.21.15.17:0/431231004 >> v2:172.21.15.119:3300/0 conn(0x7f22201340c0 0x7f22201344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/431231004 >> 172.21.15.17:0/431231004 conn(0x7f22200a66a0 msgr2=0x7f22200a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:46.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/431231004 shutdown_connections 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/431231004 wait complete. 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: build_initial_monmap 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: monmap: 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 Processor -- start 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- start start 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: init 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f222009d6c0 0x7f2220130c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): picked mon.noname-b con 0x7f222009d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 0x7f2220131150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): picked mon.noname-a con 0x7f22200a0dd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22201340c0 0x7f22201cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): picked mon.noname-c con 0x7f22201340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f22200a2710 con 0x7f22200a0dd0 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22200a2590 con 0x7f222009d6c0 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f22201408b0 con 0x7f22201340c0 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient(hunting): authenticate will time out at 990.162177s 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22201340c0 0x7f22201cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 0x7f2220131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 0x7f2220131150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44420/0 (socket says 172.21.15.17:44420) 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): get_auth_request con 0x7f22201340c0 auth_method 0 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 1 -- 172.21.15.17:0/2730870040 learned_addr learned my addr 172.21.15.17:0/2730870040 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): get_auth_request con 0x7f22200a0dd0 auth_method 0 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221e59c640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f222009d6c0 0x7f2220130c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221e59c640 10 monclient(hunting): get_auth_request con 0x7f222009d6c0 auth_method 0 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient(hunting): handle_auth_done global_id 4325 payload 824 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 -- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f222009d6c0 msgr2=0x7f2220130c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f222009d6c0 0x7f2220130c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 -- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 msgr2=0x7f2220131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 0x7f2220131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 monclient: found mon.noname-c 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 -- 172.21.15.17:0/2730870040 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2220131ad0 con 0x7f22201340c0 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2217fff640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 0x7f2220131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221ed9d640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22201340c0 0x7f22201cd9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f221800c780 tx=0x7f221800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218016020 con 0x7f22201340c0 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: dump: 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.784 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221800aa50 con 0x7f22201340c0 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 5 monclient: authenticate success, global_id 4325 2024-09-15T21:17:46.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: _renew_subs 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222009d0f0 con 0x7f22201340c0 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221800ce30 con 0x7f22201340c0 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2216ffd640 10 monclient: dump: 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:46.786 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f221d59a640 4 set_mon_vals no callback set 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: _renew_subs 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.762+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f222013a820 con 0x7f22201340c0 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2224bc8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2224bc8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2224bc8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2220137400 con 0x7f22201340c0 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2218019020 con 0x7f22201340c0 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2216ffd640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f220003b9c0 0x7f220003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f221804f7d0 con 0x7f22201340c0 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f221e59c640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f220003b9c0 0x7f220003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f221e59c640 10 monclient: get_auth_request con 0x7f220003b9c0 auth_method 0 2024-09-15T21:17:46.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.766+0000 7f221e59c640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f220003b9c0 0x7f220003de80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f221000b9d0 tx=0x7f2210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:46.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.770+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2218020030 con 0x7f22201340c0 2024-09-15T21:17:46.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.770+0000 7f2216ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:46.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.770+0000 7f2216ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:46.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.934+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22200a0890 con 0x7f220003b9c0 2024-09-15T21:17:46.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2216ffd640 1 -- 172.21.15.17:0/2730870040 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34220 (secure 0 0 0) 0x7f22200a0890 con 0x7f220003b9c0 2024-09-15T21:17:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:46.946 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:17:46.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f220003b9c0 msgr2=0x7f220003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f220003b9c0 0x7f220003de80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f221000b9d0 tx=0x7f2210008040 comp rx=0 tx=0).stop 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 10 monclient: shutdown 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22201340c0 msgr2=0x7f22201cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22201340c0 0x7f22201cd9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f221800c780 tx=0x7f221800cc40 comp rx=0 tx=0).stop 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 shutdown_connections 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f220003b9c0 0x7f220003de80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22201340c0 0x7f22201cd9c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22200a0dd0 0x7f2220131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 --2- 172.21.15.17:0/2730870040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f222009d6c0 0x7f2220130c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 >> 172.21.15.17:0/2730870040 conn(0x7f22200a66a0 msgr2=0x7f2220073d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 shutdown_connections 2024-09-15T21:17:46.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:46.938+0000 7f2224bc8640 1 -- 172.21.15.17:0/2730870040 wait complete. 2024-09-15T21:17:46.963 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T21:17:45.176052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":171632,"kb_used_data":1108,"kb_used_omap":9,"kb_used_meta":170358,"kb_avail":566059408,"statfs":{"total":579820584960,"available":579644833792,"internally_reserved":0,"allocated":1134592,"data_stored":1396768,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9567,"internal_metadata":174447265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878542"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276795+0000","last_change":"2024-09-15T21:17:42.277325+0000","last_active":"2024-09-15T21:17:42.276795+0000","last_peered":"2024-09-15T21:17:42.276795+0000","last_clean":"2024-09-15T21:17:42.276795+0000","last_became_active":"2024-09-15T21:17:40.059370+0000","last_became_peered":"2024-09-15T21:17:40.059370+0000","last_unstale":"2024-09-15T21:17:42.276795+0000","last_undegraded":"2024-09-15T21:17:42.276795+0000","last_fullsized":"2024-09-15T21:17:42.276795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:59.298050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.311058+0000","last_change":"2024-09-15T21:17:41.311058+0000","last_active":"2024-09-15T21:17:41.311058+0000","last_peered":"2024-09-15T21:17:41.311058+0000","last_clean":"2024-09-15T21:17:41.311058+0000","last_became_active":"2024-09-15T21:17:39.299252+0000","last_became_peered":"2024-09-15T21:17:39.299252+0000","last_unstale":"2024-09-15T21:17:41.311058+0000","last_undegraded":"2024-09-15T21:17:41.311058+0000","last_fullsized":"2024-09-15T21:17:41.311058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:09.083407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276328+0000","last_change":"2024-09-15T21:17:42.276772+0000","last_active":"2024-09-15T21:17:42.276328+0000","last_peered":"2024-09-15T21:17:42.276328+0000","last_clean":"2024-09-15T21:17:42.276328+0000","last_became_active":"2024-09-15T21:17:39.302570+0000","last_became_peered":"2024-09-15T21:17:39.302570+0000","last_unstale":"2024-09-15T21:17:42.276328+0000","last_undegraded":"2024-09-15T21:17:42.276328+0000","last_fullsized":"2024-09-15T21:17:42.276328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:14.596682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.310852+0000","last_change":"2024-09-15T21:17:41.310852+0000","last_active":"2024-09-15T21:17:41.310852+0000","last_peered":"2024-09-15T21:17:41.310852+0000","last_clean":"2024-09-15T21:17:41.310852+0000","last_became_active":"2024-09-15T21:17:39.297755+0000","last_became_peered":"2024-09-15T21:17:39.297755+0000","last_unstale":"2024-09-15T21:17:41.310852+0000","last_undegraded":"2024-09-15T21:17:41.310852+0000","last_fullsized":"2024-09-15T21:17:41.310852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:03.856073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.320791+0000","last_change":"2024-09-15T21:17:41.320791+0000","last_active":"2024-09-15T21:17:41.320791+0000","last_peered":"2024-09-15T21:17:41.320791+0000","last_clean":"2024-09-15T21:17:41.320791+0000","last_became_active":"2024-09-15T21:17:39.306488+0000","last_became_peered":"2024-09-15T21:17:39.306488+0000","last_unstale":"2024-09-15T21:17:41.320791+0000","last_undegraded":"2024-09-15T21:17:41.320791+0000","last_fullsized":"2024-09-15T21:17:41.320791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:32.809530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.309694+0000","last_change":"2024-09-15T21:17:41.310299+0000","last_active":"2024-09-15T21:17:41.309694+0000","last_peered":"2024-09-15T21:17:41.309694+0000","last_clean":"2024-09-15T21:17:41.309694+0000","last_became_active":"2024-09-15T21:17:39.297311+0000","last_became_peered":"2024-09-15T21:17:39.297311+0000","last_unstale":"2024-09-15T21:17:41.309694+0000","last_undegraded":"2024-09-15T21:17:41.309694+0000","last_fullsized":"2024-09-15T21:17:41.309694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:04.414329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276249+0000","last_change":"2024-09-15T21:17:42.276652+0000","last_active":"2024-09-15T21:17:42.276249+0000","last_peered":"2024-09-15T21:17:42.276249+0000","last_clean":"2024-09-15T21:17:42.276249+0000","last_became_active":"2024-09-15T21:17:39.302172+0000","last_became_peered":"2024-09-15T21:17:39.302172+0000","last_unstale":"2024-09-15T21:17:42.276249+0000","last_undegraded":"2024-09-15T21:17:42.276249+0000","last_fullsized":"2024-09-15T21:17:42.276249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:37.127020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.319475+0000","last_change":"2024-09-15T21:17:41.319914+0000","last_active":"2024-09-15T21:17:41.319475+0000","last_peered":"2024-09-15T21:17:41.319475+0000","last_clean":"2024-09-15T21:17:41.319475+0000","last_became_active":"2024-09-15T21:17:39.306227+0000","last_became_peered":"2024-09-15T21:17:39.306227+0000","last_unstale":"2024-09-15T21:17:41.319475+0000","last_undegraded":"2024-09-15T21:17:41.319475+0000","last_fullsized":"2024-09-15T21:17:41.319475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:19:50.937347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:42.276106+0000","last_change":"2024-09-15T21:17:32.810848+0000","last_active":"2024-09-15T21:17:42.276106+0000","last_peered":"2024-09-15T21:17:42.276106+0000","last_clean":"2024-09-15T21:17:42.276106+0000","last_became_active":"2024-09-15T21:17:32.808947+0000","last_became_peered":"2024-09-15T21:17:32.808947+0000","last_unstale":"2024-09-15T21:17:42.276106+0000","last_undegraded":"2024-09-15T21:17:42.276106+0000","last_fullsized":"2024-09-15T21:17:42.276106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:46:33.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":57307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":581267,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":172032,"data_stored":57307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":586235,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:46.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient: get_monmap_and_config 2024-09-15T21:17:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient: monmap: 2024-09-15T21:17:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:47.075384+0000 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:47.075384+0000 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:47.079 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 Processor -- start 2024-09-15T21:17:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 -- start start 2024-09-15T21:17:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient: init 2024-09-15T21:17:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0ab4073640 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 0x7f0ab413cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): picked mon.noname-b con 0x7f0ab4074130 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ab413d140 0x7f0ab413f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): picked mon.noname-c con 0x7f0ab413d140 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ab40a0890 con 0x7f0ab4073640 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab40a3640 con 0x7f0ab413d140 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab40746e0 con 0x7f0ab4074130 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0abbdec640 10 monclient(hunting): authenticate will time out at 990.470971s 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 0x7f0ab413cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 0x7f0ab413cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45664/0 (socket says 172.21.15.17:45664) 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 -- 172.21.15.17:0/3752257955 learned_addr learned my addr 172.21.15.17:0/3752257955 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): get_auth_request con 0x7f0ab4074130 auth_method 0 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient(hunting): handle_auth_done global_id 4328 payload 274 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0aba389640 1 --2- 172.21.15.17:0/3752257955 >> v2:172.21.15.119:3300/0 conn(0x7f0ab413d140 0x7f0ab413f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 -- 172.21.15.17:0/3752257955 >> v2:172.21.15.119:3300/0 conn(0x7f0ab413d140 msgr2=0x7f0ab413f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 --2- 172.21.15.17:0/3752257955 >> v2:172.21.15.119:3300/0 conn(0x7f0ab413d140 0x7f0ab413f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 -- 172.21.15.17:0/3752257955 >> v1:172.21.15.119:6789/0 conn(0x7f0ab4073640 legacy=0x7f0ab4073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 monclient: found mon.noname-b 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.070+0000 7f0ab9387640 1 -- 172.21.15.17:0/3752257955 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab40a1640 con 0x7f0ab4074130 2024-09-15T21:17:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 1 --2- 172.21.15.17:0/3752257955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 0x7f0ab413cc00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009870 tx=0x7f0aa401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894a212d1e69f660 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 1 -- 172.21.15.17:0/3752257955 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40b0070 con 0x7f0ab4074130 2024-09-15T21:17:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: dump: 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 1 -- 172.21.15.17:0/3752257955 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa40ab600 con 0x7f0ab4074130 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 1 -- 172.21.15.17:0/3752257955 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40ab9d0 con 0x7f0ab4074130 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 5 monclient: authenticate success, global_id 4328 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 4 set_mon_vals no callback set 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab8b86640 10 monclient: dump: 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: shutdown 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- 172.21.15.17:0/3752257955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 msgr2=0x7f0ab413cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 --2- 172.21.15.17:0/3752257955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 0x7f0ab413cc00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009870 tx=0x7f0aa401c3f0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- 172.21.15.17:0/3752257955 shutdown_connections 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 --2- 172.21.15.17:0/3752257955 >> v2:172.21.15.119:3300/0 conn(0x7f0ab413d140 0x7f0ab413f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 --2- 172.21.15.17:0/3752257955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4074130 0x7f0ab413cc00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- 172.21.15.17:0/3752257955 >> 172.21.15.17:0/3752257955 conn(0x7f0ab40a66a0 msgr2=0x7f0ab40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- 172.21.15.17:0/3752257955 shutdown_connections 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- 172.21.15.17:0/3752257955 wait complete. 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: monmap: 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:47.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 Processor -- start 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- start start 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: init 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4073640 0x7f0ab41cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): picked mon.noname-c con 0x7f0ab4073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 0x7f0ab41cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): picked mon.noname-a con 0x7f0ab4074130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab413d140 0x7f0ab41d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): picked mon.noname-b con 0x7f0ab413d140 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab40a0ec0 con 0x7f0ab4074130 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab40a0d40 con 0x7f0ab413d140 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab4142290 con 0x7f0ab4073640 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0abbdec640 10 monclient(hunting): authenticate will time out at 990.475757s 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 0x7f0ab41cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9b88640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4073640 0x7f0ab41cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 0x7f0ab41cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44432/0 (socket says 172.21.15.17:44432) 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 1 -- 172.21.15.17:0/1674684036 learned_addr learned my addr 172.21.15.17:0/1674684036 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 10 monclient(hunting): get_auth_request con 0x7f0ab4074130 auth_method 0 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9387640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.074+0000 7f0ab9b88640 10 monclient(hunting): get_auth_request con 0x7f0ab4073640 auth_method 0 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aba389640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab413d140 0x7f0ab41d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aba389640 10 monclient(hunting): get_auth_request con 0x7f0ab413d140 auth_method 0 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aba389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aba389640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aba389640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 10 monclient(hunting): handle_auth_done global_id 4344 payload 824 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 -- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4073640 msgr2=0x7f0ab41cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4073640 0x7f0ab41cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 -- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab413d140 msgr2=0x7f0ab41d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab413d140 0x7f0ab41d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 monclient: found mon.noname-a 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 -- 172.21.15.17:0/1674684036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab41375a0 con 0x7f0ab4074130 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9387640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 0x7f0ab41cfb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40ae040 tx=0x7f0aa4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40b0040 con 0x7f0ab4074130 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: dump: 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa40b9070 con 0x7f0ab4074130 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40b48a0 con 0x7f0ab4074130 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 5 monclient: authenticate success, global_id 4344 2024-09-15T21:17:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 monclient: _renew_subs 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab40a4860 con 0x7f0ab4074130 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aabfff640 4 set_mon_vals no callback set 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 10 monclient: dump: 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 monclient: _renew_subs 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab41372c0 con 0x7f0ab4074130 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0aa40ab4c0 con 0x7f0ab4074130 2024-09-15T21:17:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab41370c0 con 0x7f0ab4074130 2024-09-15T21:17:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a9803b9c0 0x7f0a9803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f0aa40e98c0 con 0x7f0ab4074130 2024-09-15T21:17:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a9803b9c0 0x7f0a9803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 10 monclient: get_auth_request con 0x7f0a9803b9c0 auth_method 0 2024-09-15T21:17:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.078+0000 7f0ab9b88640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a9803b9c0 0x7f0a9803de80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007fd0 tx=0x7f0ab0007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.082+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0aa4016130 con 0x7f0ab4074130 2024-09-15T21:17:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.082+0000 7f0aaa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.082+0000 7f0aaa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:47.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.250+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ab40a4310 con 0x7f0a9803b9c0 2024-09-15T21:17:47.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0aaa7fc640 1 -- 172.21.15.17:0/1674684036 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34064 (secure 0 0 0) 0x7f0ab40a4310 con 0x7f0a9803b9c0 2024-09-15T21:17:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:17:47.265 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:17:47.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a9803b9c0 msgr2=0x7f0a9803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a9803b9c0 0x7f0a9803de80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007fd0 tx=0x7f0ab0007e40 comp rx=0 tx=0).stop 2024-09-15T21:17:47.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 10 monclient: shutdown 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 msgr2=0x7f0ab41cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 0x7f0ab41cfb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40ae040 tx=0x7f0aa4019190 comp rx=0 tx=0).stop 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 shutdown_connections 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a9803b9c0 0x7f0a9803de80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab413d140 0x7f0ab41d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab4074130 0x7f0ab41cfb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 --2- 172.21.15.17:0/1674684036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ab4073640 0x7f0ab41cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 >> 172.21.15.17:0/1674684036 conn(0x7f0ab40a66a0 msgr2=0x7f0ab409e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 shutdown_connections 2024-09-15T21:17:47.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.258+0000 7f0abbdec640 1 -- 172.21.15.17:0/1674684036 wait complete. 2024-09-15T21:17:47.282 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-15T21:17:47.176698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":171672,"kb_used_data":1180,"kb_used_omap":9,"kb_used_meta":170358,"kb_avail":566059368,"statfs":{"total":579820584960,"available":579644792832,"internally_reserved":0,"allocated":1208320,"data_stored":1424746,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9567,"internal_metadata":174447265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.876396"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.399987+0000","last_change":"2024-09-15T21:17:42.277325+0000","last_active":"2024-09-15T21:17:44.399987+0000","last_peered":"2024-09-15T21:17:44.399987+0000","last_clean":"2024-09-15T21:17:44.399987+0000","last_became_active":"2024-09-15T21:17:40.059370+0000","last_became_peered":"2024-09-15T21:17:40.059370+0000","last_unstale":"2024-09-15T21:17:44.399987+0000","last_undegraded":"2024-09-15T21:17:44.399987+0000","last_fullsized":"2024-09-15T21:17:44.399987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:59.298050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.311058+0000","last_change":"2024-09-15T21:17:41.311058+0000","last_active":"2024-09-15T21:17:41.311058+0000","last_peered":"2024-09-15T21:17:41.311058+0000","last_clean":"2024-09-15T21:17:41.311058+0000","last_became_active":"2024-09-15T21:17:39.299252+0000","last_became_peered":"2024-09-15T21:17:39.299252+0000","last_unstale":"2024-09-15T21:17:41.311058+0000","last_undegraded":"2024-09-15T21:17:41.311058+0000","last_fullsized":"2024-09-15T21:17:41.311058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:09.083407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.399901+0000","last_change":"2024-09-15T21:17:42.276772+0000","last_active":"2024-09-15T21:17:44.399901+0000","last_peered":"2024-09-15T21:17:44.399901+0000","last_clean":"2024-09-15T21:17:44.399901+0000","last_became_active":"2024-09-15T21:17:39.302570+0000","last_became_peered":"2024-09-15T21:17:39.302570+0000","last_unstale":"2024-09-15T21:17:44.399901+0000","last_undegraded":"2024-09-15T21:17:44.399901+0000","last_fullsized":"2024-09-15T21:17:44.399901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:14.596682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:17:41.310852+0000","last_change":"2024-09-15T21:17:41.310852+0000","last_active":"2024-09-15T21:17:41.310852+0000","last_peered":"2024-09-15T21:17:41.310852+0000","last_clean":"2024-09-15T21:17:41.310852+0000","last_became_active":"2024-09-15T21:17:39.297755+0000","last_became_peered":"2024-09-15T21:17:39.297755+0000","last_unstale":"2024-09-15T21:17:41.310852+0000","last_undegraded":"2024-09-15T21:17:41.310852+0000","last_fullsized":"2024-09-15T21:17:41.310852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:03.856073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":24,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.439446+0000","last_change":"2024-09-15T21:17:41.320791+0000","last_active":"2024-09-15T21:17:44.439446+0000","last_peered":"2024-09-15T21:17:44.439446+0000","last_clean":"2024-09-15T21:17:44.439446+0000","last_became_active":"2024-09-15T21:17:39.306488+0000","last_became_peered":"2024-09-15T21:17:39.306488+0000","last_unstale":"2024-09-15T21:17:44.439446+0000","last_undegraded":"2024-09-15T21:17:44.439446+0000","last_fullsized":"2024-09-15T21:17:44.439446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:32.809530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.397073+0000","last_change":"2024-09-15T21:17:41.310299+0000","last_active":"2024-09-15T21:17:44.397073+0000","last_peered":"2024-09-15T21:17:44.397073+0000","last_clean":"2024-09-15T21:17:44.397073+0000","last_became_active":"2024-09-15T21:17:39.297311+0000","last_became_peered":"2024-09-15T21:17:39.297311+0000","last_unstale":"2024-09-15T21:17:44.397073+0000","last_undegraded":"2024-09-15T21:17:44.397073+0000","last_fullsized":"2024-09-15T21:17:44.397073+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:04.414329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.399884+0000","last_change":"2024-09-15T21:17:42.276652+0000","last_active":"2024-09-15T21:17:44.399884+0000","last_peered":"2024-09-15T21:17:44.399884+0000","last_clean":"2024-09-15T21:17:44.399884+0000","last_became_active":"2024-09-15T21:17:39.302172+0000","last_became_peered":"2024-09-15T21:17:39.302172+0000","last_unstale":"2024-09-15T21:17:44.399884+0000","last_undegraded":"2024-09-15T21:17:44.399884+0000","last_fullsized":"2024-09-15T21:17:44.399884+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:37.127020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":23,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.439387+0000","last_change":"2024-09-15T21:17:41.319914+0000","last_active":"2024-09-15T21:17:44.439387+0000","last_peered":"2024-09-15T21:17:44.439387+0000","last_clean":"2024-09-15T21:17:44.439387+0000","last_became_active":"2024-09-15T21:17:39.306227+0000","last_became_peered":"2024-09-15T21:17:39.306227+0000","last_unstale":"2024-09-15T21:17:44.439387+0000","last_undegraded":"2024-09-15T21:17:44.439387+0000","last_fullsized":"2024-09-15T21:17:44.439387+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:19:50.937347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":113,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T21:17:44.399806+0000","last_change":"2024-09-15T21:17:32.810848+0000","last_active":"2024-09-15T21:17:44.399806+0000","last_peered":"2024-09-15T21:17:44.399806+0000","last_clean":"2024-09-15T21:17:44.399806+0000","last_became_active":"2024-09-15T21:17:32.808947+0000","last_became_peered":"2024-09-15T21:17:32.808947+0000","last_unstale":"2024-09-15T21:17:44.399806+0000","last_undegraded":"2024-09-15T21:17:44.399806+0000","last_fullsized":"2024-09-15T21:17:44.399806+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:46:33.906115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":176128,"data_stored":57326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":61928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":590837,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":590837,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:47.282 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-15T21:17:47.283 DEBUG:tasks.mon_thrash:Yielding 2024-09-15T21:17:47.283 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T21:17:47.296 INFO:tasks.workunit:Pulling workunits from ref 7498493279982dfe87f00616198a5967475f6169 2024-09-15T21:17:47.297 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T21:17:47.297 DEBUG:teuthology.orchestra.run.smithi119:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:17:47.297 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-15T21:17:47.297 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726435067, revive delay: 20.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 20.0 2024-09-15T21:17:47.298 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:17:47.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:17:47.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:17:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-15T21:17:47.304 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:17:47.350 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-15T21:17:47.350 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-15T21:17:47.398 INFO:tasks.workunit:timeout=3h 2024-09-15T21:17:47.398 INFO:tasks.workunit:cleanup=True 2024-09-15T21:17:47.398 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7498493279982dfe87f00616198a5967475f6169 2024-09-15T21:17:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.410+0000 7fe31e3c9640 10 monclient: get_monmap_and_config 2024-09-15T21:17:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.410+0000 7fe31e3c9640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.410+0000 7fe31e3c9640 10 monclient: monmap: 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:47.416481+0000 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:47.416481+0000 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:47.420 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.410+0000 7fe31e3c9640 1 Processor -- start 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.410+0000 7fe31e3c9640 1 -- start start 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.410+0000 7fe31e3c9640 10 monclient: init 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe318073ad0 0x7fe318073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): picked mon.noname-a con 0x7fe318073ad0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3180a4200 0x7fe3180a2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): picked mon.noname-b con 0x7fe3180a4200 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe31809f570 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180a1490 con 0x7fe31809f570 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe3180713c0 con 0x7fe318073ad0 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe318071540 con 0x7fe3180a4200 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient(hunting): authenticate will time out at 990.812638s 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3180a4200 0x7fe3180a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe318073ad0 0x7fe318073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31c966640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe31809f570 0x7fe31809f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54696/0 (socket says 172.21.15.17:54696) 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31c966640 1 -- 172.21.15.17:0/3093418272 learned_addr learned my addr 172.21.15.17:0/3093418272 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): get_auth_request con 0x7fe3180a4200 auth_method 0 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): get_auth_request con 0x7fe318073ad0 auth_method 0 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe317fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient(hunting): handle_auth_done global_id 4334 payload 274 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 -- 172.21.15.17:0/3093418272 >> v2:172.21.15.17:3300/0 conn(0x7fe318073ad0 msgr2=0x7fe318073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 1 -- 172.21.15.17:0/3093418272 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626890523 0 0) 0x7fe3180a1490 con 0x7fe31809f570 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 --2- 172.21.15.17:0/3093418272 >> v2:172.21.15.17:3300/0 conn(0x7fe318073ad0 0x7fe318073eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 -- 172.21.15.17:0/3093418272 >> v1:172.21.15.17:6789/0 conn(0x7fe31809f570 legacy=0x7fe31809f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 monclient: found mon.noname-b 2024-09-15T21:17:47.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 -- 172.21.15.17:0/3093418272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180a3ce0 con 0x7fe3180a4200 2024-09-15T21:17:47.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe3177fe640 1 --2- 172.21.15.17:0/3093418272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3180a4200 0x7fe3180a2150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3080097c0 tx=0x7fe30801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d84c9b0dbcdc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 1 -- 172.21.15.17:0/3093418272 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080b0070 con 0x7fe3180a4200 2024-09-15T21:17:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: dump: 2024-09-15T21:17:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 5 monclient: authenticate success, global_id 4334 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 1 -- 172.21.15.17:0/3093418272 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3080ab400 con 0x7fe3180a4200 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 4 set_mon_vals no callback set 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 1 -- 172.21.15.17:0/3093418272 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080ab7d0 con 0x7fe3180a4200 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe316ffd640 10 monclient: dump: 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 10 monclient: shutdown 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 -- 172.21.15.17:0/3093418272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3180a4200 msgr2=0x7fe3180a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/3093418272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3180a4200 0x7fe3180a2150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3080097c0 tx=0x7fe30801c3f0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 -- 172.21.15.17:0/3093418272 shutdown_connections 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/3093418272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3180a4200 0x7fe3180a2150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/3093418272 >> v2:172.21.15.17:3300/0 conn(0x7fe318073ad0 0x7fe318073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.414+0000 7fe31e3c9640 1 -- 172.21.15.17:0/3093418272 >> 172.21.15.17:0/3093418272 conn(0x7fe3180a4d40 msgr2=0x7fe3180a5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- 172.21.15.17:0/3093418272 shutdown_connections 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- 172.21.15.17:0/3093418272 wait complete. 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: monmap: 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:47.434 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 Processor -- start 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- start start 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: init 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 0x7fe3181258b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): picked mon.noname-c con 0x7fe318073ad0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31809f570 0x7fe31813c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): picked mon.noname-a con 0x7fe31809f570 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3180a4200 0x7fe3181233d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): picked mon.noname-b con 0x7fe3180a4200 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe3181402b0 con 0x7fe31809f570 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180a3f10 con 0x7fe3180a4200 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe318140430 con 0x7fe318073ad0 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient(hunting): authenticate will time out at 990.817435s 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 0x7fe3181258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31809f570 0x7fe31813c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 0x7fe3181258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45686/0 (socket says 172.21.15.17:45686) 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 -- 172.21.15.17:0/2166548543 learned_addr learned my addr 172.21.15.17:0/2166548543 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): get_auth_request con 0x7fe318073ad0 auth_method 0 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): get_auth_request con 0x7fe31809f570 auth_method 0 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient(hunting): handle_auth_done global_id 4337 payload 824 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 -- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3180a4200 msgr2=0x7fe3181233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3180a4200 0x7fe3181233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 -- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31809f570 msgr2=0x7fe31813c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31809f570 0x7fe31813c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 monclient: found mon.noname-c 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 -- 172.21.15.17:0/2166548543 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181270b0 con 0x7fe318073ad0 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe317fff640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 0x7fe3181258b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00e930 tx=0x7fe30c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c017070 con 0x7fe318073ad0 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: dump: 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 5 monclient: authenticate success, global_id 4337 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _renew_subs 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c00ca00 con 0x7fe318073ad0 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3180a33e0 con 0x7fe318073ad0 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00cdb0 con 0x7fe318073ad0 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 10 monclient: dump: 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _renew_subs 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe318126240 con 0x7fe318073ad0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3167fc640 4 set_mon_vals no callback set 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe30c01a070 con 0x7fe318073ad0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe318073eb0 con 0x7fe318073ad0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30403b9f0 0x7fe30403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fe30c04fa70 con 0x7fe318073ad0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.418+0000 7fe3177fe640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30403b9f0 0x7fe30403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.422+0000 7fe3177fe640 10 monclient: get_auth_request con 0x7fe30403b9f0 auth_method 0 2024-09-15T21:17:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.422+0000 7fe3177fe640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30403b9f0 0x7fe30403deb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3080ad000 tx=0x7fe3080a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.426+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe30c013c70 con 0x7fe318073ad0 2024-09-15T21:17:47.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.426+0000 7fe314ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.426+0000 7fe314ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:47.453 INFO:tasks.workunit.client.0.smithi119.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T21:17:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:17:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe31809f040 con 0x7fe318073ad0 2024-09-15T21:17:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe314ff9640 1 -- 172.21.15.17:0/2166548543 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fe30c04fe20 con 0x7fe318073ad0 2024-09-15T21:17:47.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe314ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:47.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe314ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:47.667 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":27,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:17:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30403b9f0 msgr2=0x7fe30403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30403b9f0 0x7fe30403deb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3080ad000 tx=0x7fe3080a6000 comp rx=0 tx=0).stop 2024-09-15T21:17:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 10 monclient: shutdown 2024-09-15T21:17:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 msgr2=0x7fe3181258b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.658+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 0x7fe3181258b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00e930 tx=0x7fe30c00edf0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 shutdown_connections 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30403b9f0 0x7fe30403deb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3180a4200 0x7fe3181233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31809f570 0x7fe31813c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 --2- 172.21.15.17:0/2166548543 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe318073ad0 0x7fe3181258b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 >> 172.21.15.17:0/2166548543 conn(0x7fe3180a4d40 msgr2=0x7fe31809da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 shutdown_connections 2024-09-15T21:17:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.662+0000 7fe31e3c9640 1 -- 172.21.15.17:0/2166548543 wait complete. 2024-09-15T21:17:47.685 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:17:47.685 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:17:47.685 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:17:47.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:17:47.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.794+0000 7f8adbc8e640 10 monclient: get_monmap_and_config 2024-09-15T21:17:47.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.794+0000 7f8adbc8e640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.794+0000 7f8adbc8e640 10 monclient: monmap: 2024-09-15T21:17:47.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:47.800777+0000 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:47.800777+0000 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:47.805 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.794+0000 7f8adbc8e640 1 Processor -- start 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.794+0000 7f8adbc8e640 1 -- start start 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.794+0000 7f8adbc8e640 10 monclient: init 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ad409f490 0x7f8ad40a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): picked mon.noname-a con 0x7f8ad409f490 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:47.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad409f870 0x7f8ad409fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): picked mon.noname-b con 0x7f8ad409f870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ad4135750 0x7f8ad4135b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): picked mon.noname-c con 0x7f8ad4135750 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad409efe0 con 0x7f8ad409f490 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40a4870 con 0x7f8ad4135750 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40a4140 con 0x7f8ad409f870 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient(hunting): authenticate will time out at 991.196246s 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad409f870 0x7f8ad409fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ad409f490 0x7f8ad40a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad409f870 0x7f8ad409fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33142/0 (socket says 172.21.15.17:33142) 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 1 -- 172.21.15.17:0/486306638 learned_addr learned my addr 172.21.15.17:0/486306638 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): get_auth_request con 0x7f8ad409f490 auth_method 0 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): get_auth_request con 0x7f8ad409f870 auth_method 0 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ada22b640 1 --2- 172.21.15.17:0/486306638 >> v2:172.21.15.119:3300/0 conn(0x7f8ad4135750 0x7f8ad4135b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ada22b640 10 monclient(hunting): get_auth_request con 0x7f8ad4135750 auth_method 0 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ada22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ada22b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ada22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_done global_id 4356 payload 274 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad409f870 msgr2=0x7f8ad409fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 --2- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad409f870 0x7f8ad409fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/486306638 >> v2:172.21.15.119:3300/0 conn(0x7f8ad4135750 msgr2=0x7f8ad4135b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 --2- 172.21.15.17:0/486306638 >> v2:172.21.15.119:3300/0 conn(0x7f8ad4135750 0x7f8ad4135b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 monclient: found mon.noname-a 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/486306638 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40a1320 con 0x7f8ad409f490 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad9a2a640 1 --2- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad40a39a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40a5450 tx=0x7f8ad001cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f39c011f43ae6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 1 -- 172.21.15.17:0/486306638 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b0070 con 0x7f8ad409f490 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: dump: 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.817 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 1 -- 172.21.15.17:0/486306638 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad00ab600 con 0x7f8ad409f490 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 5 monclient: authenticate success, global_id 4356 2024-09-15T21:17:47.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 4 set_mon_vals no callback set 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 1 -- 172.21.15.17:0/486306638 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00ab9d0 con 0x7f8ad409f490 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8ad8a28640 10 monclient: dump: 2024-09-15T21:17:47.819 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 10 monclient: shutdown 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 msgr2=0x7f8ad40a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad40a39a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40a5450 tx=0x7f8ad001cda0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- 172.21.15.17:0/486306638 shutdown_connections 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/486306638 >> v2:172.21.15.119:3300/0 conn(0x7f8ad4135750 0x7f8ad4135b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad409f870 0x7f8ad409fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/486306638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad40a39a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- 172.21.15.17:0/486306638 >> 172.21.15.17:0/486306638 conn(0x7f8ad40a8910 msgr2=0x7f8ad40a4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.798+0000 7f8adbc8e640 1 -- 172.21.15.17:0/486306638 shutdown_connections 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- 172.21.15.17:0/486306638 wait complete. 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: monmap: 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:47.822 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 Processor -- start 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- start start 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: init 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad41cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): picked mon.noname-a con 0x7f8ad409f490 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad409f870 0x7f8ad4134b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): picked mon.noname-b con 0x7f8ad409f870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad4135750 0x7f8ad4135090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): picked mon.noname-c con 0x7f8ad4135750 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40a24c0 con 0x7f8ad409f490 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40a2340 con 0x7f8ad409f870 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad413f3d0 con 0x7f8ad4135750 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): authenticate will time out at 991.201621s 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad41cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad41cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51858/0 (socket says 172.21.15.17:51858) 2024-09-15T21:17:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad4135750 0x7f8ad4135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/2343174155 learned_addr learned my addr 172.21.15.17:0/2343174155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): get_auth_request con 0x7f8ad409f490 auth_method 0 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): get_auth_request con 0x7f8ad4135750 auth_method 0 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ada22b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad409f870 0x7f8ad4134b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): get_auth_request con 0x7f8ad409f870 auth_method 0 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient(hunting): handle_auth_done global_id 4359 payload 824 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad4135750 msgr2=0x7f8ad4135090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ad4135750 0x7f8ad4135090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad409f870 msgr2=0x7f8ad4134b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad409f870 0x7f8ad4134b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 monclient: found mon.noname-a 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad41201b0 con 0x7f8ad409f490 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9a2a640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad41cf900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00ae040 tx=0x7f8ad00190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b0040 con 0x7f8ad409f490 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: dump: 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad00b9070 con 0x7f8ad409f490 2024-09-15T21:17:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b4650 con 0x7f8ad409f490 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 10 monclient: dump: 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 5 monclient: authenticate success, global_id 4359 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: _renew_subs 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8acbfff640 4 set_mon_vals no callback set 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad409f180 con 0x7f8ad409f490 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: _renew_subs 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad4121c70 con 0x7f8ad409f490 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 0x7f8ad411ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4073ff0 con 0x7f8ad411e870 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8ad00b47f0 con 0x7f8ad409f490 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 0x7f8ad411ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient: get_auth_request con 0x7f8ad411e870 auth_method 0 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ab803b9c0 0x7f8ab803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 10 monclient(hunting): handle_auth_done global_id 4362 payload 824 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f8ad00e9820 con 0x7f8ad409f490 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.802+0000 7f8ad9229640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ad409fc70 con 0x7f8ad411e870 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8ad9229640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 0x7f8ad411ec50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40115d0 tx=0x7f8ac4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400f900 con 0x7f8ad411e870 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8ada22b640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ab803b9c0 0x7f8ab803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 10 monclient: dump: 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8ada22b640 10 monclient: get_auth_request con 0x7f8ab803b9c0 auth_method 0 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8ac400f900 con 0x7f8ad411e870 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 msgr2=0x7f8ad411ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 0x7f8ad411ec50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40115d0 tx=0x7f8ac4011a90 comp rx=0 tx=0).stop 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 msgr2=0x7f8ad411ec50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8aca7fc640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411e870 0x7f8ad411ec50 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40115d0 tx=0x7f8ac4011a90 comp rx=0 tx=0).stop 2024-09-15T21:17:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.806+0000 7f8ada22b640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ab803b9c0 0x7f8ab803de80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4120700 tx=0x7f8acc009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8adbc8e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:17:47.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 0x7f8ad409ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8adbc8e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad41cfe40 con 0x7f8ad411f190 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 0x7f8ad409ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient: get_auth_request con 0x7f8ad411f190 auth_method 0 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 10 monclient(hunting): handle_auth_done global_id 4365 payload 824 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 1 -- 172.21.15.17:0/2343174155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8ad40693f0 con 0x7f8ad411f190 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8ad9229640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 0x7f8ad409ba60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40115d0 tx=0x7f8ac40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4004460 con 0x7f8ad411f190 2024-09-15T21:17:47.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 10 monclient: dump: 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f8ac4026070 con 0x7f8ad411f190 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 msgr2=0x7f8ad409ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 0x7f8ad409ba60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40115d0 tx=0x7f8ac40046d0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 msgr2=0x7f8ad409ba60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.810+0000 7f8aca7fc640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad411f190 0x7f8ad409ba60 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40115d0 tx=0x7f8ac40046d0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:17:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 8, 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 33431, 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:17:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 28, 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:17:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:17:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:17:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:17:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:17:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:17:47.853 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:17:47.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ab803b9c0 msgr2=0x7f8ab803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ab803b9c0 0x7f8ab803de80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4120700 tx=0x7f8acc009820 comp rx=0 tx=0).stop 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 10 monclient: shutdown 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 msgr2=0x7f8ad41cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 --2- 172.21.15.17:0/2343174155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ad409f490 0x7f8ad41cf900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00ae040 tx=0x7f8ad00190e0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8ada22b640 1 -- 172.21.15.17:0/2343174155 reap_dead start 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8ada22b640 1 -- 172.21.15.17:0/2343174155 reap_dead start 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 shutdown_connections 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 >> 172.21.15.17:0/2343174155 conn(0x7f8ad40a8910 msgr2=0x7f8ad4072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 shutdown_connections 2024-09-15T21:17:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.814+0000 7f8adbc8e640 1 -- 172.21.15.17:0/2343174155 wait complete. 2024-09-15T21:17:47.856 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:17:47.856 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:17:47.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient: get_monmap_and_config 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient: monmap: 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:47.971083+0000 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:47.971083+0000 2024-09-15T21:17:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:47.975 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 1 Processor -- start 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 1 -- start start 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient: init 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f936809c6a0 0x7f936809ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient(hunting): picked mon.noname-a con 0x7f936809c6a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9368137c60 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9368138040 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93680a2980 con 0x7f9368138040 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9368074840 con 0x7f9368137c60 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f936809a860 con 0x7f936809c6a0 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936c80e640 10 monclient(hunting): authenticate will time out at 991.366420s 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f936809c6a0 0x7f936809ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f936809c6a0 0x7f936809ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51886/0 (socket says 172.21.15.17:51886) 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 -- 172.21.15.17:0/3080764350 learned_addr learned my addr 172.21.15.17:0/3080764350 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f9366d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9368138040 0x7f93680a2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41722/0 (socket says 172.21.15.17:41722) 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): get_auth_request con 0x7f936809c6a0 auth_method 0 2024-09-15T21:17:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient(hunting): handle_auth_done global_id 4368 payload 274 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 -- 172.21.15.17:0/3080764350 >> v1:172.21.15.119:6789/0 conn(0x7f9368137c60 legacy=0x7f93680a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 -- 172.21.15.17:0/3080764350 >> v1:172.21.15.17:6789/0 conn(0x7f9368138040 legacy=0x7f93680a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 monclient: found mon.noname-a 2024-09-15T21:17:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f9365d9b640 1 -- 172.21.15.17:0/3080764350 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532403692 0 0) 0x7f93680a2980 con 0x7f9368138040 2024-09-15T21:17:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f936659c640 1 -- 172.21.15.17:0/3080764350 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680742d0 con 0x7f936809c6a0 2024-09-15T21:17:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.966+0000 7f9365d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:17:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936659c640 1 --2- 172.21.15.17:0/3080764350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f936809c6a0 0x7f936809ca80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9358009870 tx=0x7f935801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f1e95860570e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 1 -- 172.21.15.17:0/3080764350 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580ab610 con 0x7f936809c6a0 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: dump: 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: _finish_auth 0 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 5 monclient: authenticate success, global_id 4368 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 1 -- 172.21.15.17:0/3080764350 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93580b0070 con 0x7f936809c6a0 2024-09-15T21:17:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 1 -- 172.21.15.17:0/3080764350 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580ab9d0 con 0x7f936809c6a0 2024-09-15T21:17:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9365d9b640 10 monclient: dump: 2024-09-15T21:17:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 4 set_mon_vals no callback set 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: shutdown 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- 172.21.15.17:0/3080764350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f936809c6a0 msgr2=0x7f936809ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 --2- 172.21.15.17:0/3080764350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f936809c6a0 0x7f936809ca80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9358009870 tx=0x7f935801c3f0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- 172.21.15.17:0/3080764350 shutdown_connections 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 --2- 172.21.15.17:0/3080764350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f936809c6a0 0x7f936809ca80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- 172.21.15.17:0/3080764350 >> 172.21.15.17:0/3080764350 conn(0x7f93680a2c50 msgr2=0x7f93680a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- 172.21.15.17:0/3080764350 shutdown_connections 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- 172.21.15.17:0/3080764350 wait complete. 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: build_initial_monmap 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: monmap: 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 Processor -- start 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- start start 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: init 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936809c6a0 0x7f9368099310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): picked mon.noname-b con 0x7f936809c6a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9368137c60 0x7f9368099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): picked mon.noname-a con 0x7f9368137c60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9368138040 0x7f936813c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): picked mon.noname-c con 0x7f9368138040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f93681438a0 con 0x7f9368137c60 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9368143720 con 0x7f936809c6a0 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9368143a20 con 0x7f9368138040 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936c80e640 10 monclient(hunting): authenticate will time out at 991.371651s 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9368137c60 0x7f9368099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9368137c60 0x7f9368099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51894/0 (socket says 172.21.15.17:51894) 2024-09-15T21:17:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9368138040 0x7f936813c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 1 -- 172.21.15.17:0/47043093 learned_addr learned my addr 172.21.15.17:0/47043093 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): get_auth_request con 0x7f9368137c60 auth_method 0 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): get_auth_request con 0x7f9368138040 auth_method 0 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936659c640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936809c6a0 0x7f9368099310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f935dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f9366d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.970+0000 7f936659c640 10 monclient(hunting): get_auth_request con 0x7f936809c6a0 auth_method 0 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 10 monclient(hunting): handle_auth_done global_id 4374 payload 824 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9368138040 msgr2=0x7f936813c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9368138040 0x7f936813c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936809c6a0 msgr2=0x7f9368099310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936809c6a0 0x7f9368099310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936813fbc0 con 0x7f9368137c60 2024-09-15T21:17:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f9366d9d640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9368138040 0x7f936813c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936809c6a0 0x7f9368099310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935dd9b640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9368137c60 0x7f9368099850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f935000c450 tx=0x7f935000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350011790 con 0x7f9368137c60 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: dump: 2024-09-15T21:17:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9350017070 con 0x7f9368137c60 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350011b90 con 0x7f9368137c60 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: dump: 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 5 monclient: authenticate success, global_id 4374 2024-09-15T21:17:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 10 monclient: _renew_subs 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936559a640 4 set_mon_vals no callback set 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936813fea0 con 0x7f9368137c60 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 10 monclient: _renew_subs 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93680a48e0 con 0x7f9368137c60 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 0x7f936813cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f936809ca80 con 0x7f936813cbd0 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 0x7f936813cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient: get_auth_request con 0x7f936813cbd0 auth_method 0 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9350011d30 con 0x7f9368137c60 2024-09-15T21:17:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 10 monclient(hunting): handle_auth_done global_id 4349 payload 824 2024-09-15T21:17:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f936813e100 con 0x7f936813cbd0 2024-09-15T21:17:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f934803b9c0 0x7f934803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f935004f6d0 con 0x7f9368137c60 2024-09-15T21:17:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f936659c640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 0x7f936813cfb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f936806a5a0 tx=0x7f9358019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358003ab0 con 0x7f936813cbd0 2024-09-15T21:17:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: dump: 2024-09-15T21:17:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.007 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.007 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.007 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.007 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.007 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.007 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f9366d9d640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f934803b9c0 0x7f934803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f9366d9d640 10 monclient: get_auth_request con 0x7f934803b9c0 auth_method 0 2024-09-15T21:17:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f93580b0040 con 0x7f936813cbd0 2024-09-15T21:17:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 msgr2=0x7f936813cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 0x7f936813cfb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f936806a5a0 tx=0x7f9358019190 comp rx=0 tx=0).stop 2024-09-15T21:17:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 msgr2=0x7f936813cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f935f7fe640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813cbd0 0x7f936813cfb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.974+0000 7f9366d9d640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f934803b9c0 0x7f934803de80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93680a57e0 tx=0x7f9360001680 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 0x7f9368138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f936813e9e0 con 0x7f936813d4f0 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 0x7f9368138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient: get_auth_request con 0x7f936813d4f0 auth_method 0 2024-09-15T21:17:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 10 monclient(hunting): handle_auth_done global_id 4352 payload 824 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 1 -- 172.21.15.17:0/47043093 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f936806d420 con 0x7f936813d4f0 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936659c640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 0x7f9368138550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f935801aaa0 tx=0x7f9358009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580c9030 con 0x7f936813d4f0 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 10 monclient: dump: 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f9358003e60 con 0x7f936813d4f0 2024-09-15T21:17:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 msgr2=0x7f9368138550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 0x7f9368138550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f935801aaa0 tx=0x7f9358009240 comp rx=0 tx=0).stop 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 msgr2=0x7f9368138550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f935f7fe640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f936813d4f0 0x7f9368138550 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f935801aaa0 tx=0x7f9358009240 comp rx=0 tx=0).stop 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:17:48.015 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 8, 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 33603, 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 28, 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:17:48.016 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:48.017 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:48.018 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:17:48.019 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:48.020 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:17:48.021 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:48.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:48.026 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.027 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.028 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:17:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:17:48.029 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f934803b9c0 msgr2=0x7f934803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f934803b9c0 0x7f934803de80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93680a57e0 tx=0x7f9360001680 comp rx=0 tx=0).stop 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 10 monclient: shutdown 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9368137c60 msgr2=0x7f9368099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 --2- 172.21.15.17:0/47043093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9368137c60 0x7f9368099850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f935000c450 tx=0x7f935000c910 comp rx=0 tx=0).stop 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f9366d9d640 1 -- 172.21.15.17:0/47043093 reap_dead start 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f9366d9d640 1 -- 172.21.15.17:0/47043093 reap_dead start 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 shutdown_connections 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 >> 172.21.15.17:0/47043093 conn(0x7f93680a2c50 msgr2=0x7f93680a4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 shutdown_connections 2024-09-15T21:17:48.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:47.982+0000 7f936c80e640 1 -- 172.21.15.17:0/47043093 wait complete. 2024-09-15T21:17:48.032 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:17:48.032 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:17:48.032 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.134+0000 7fda3b7a7640 10 monclient: get_monmap_and_config 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.134+0000 7fda3b7a7640 10 monclient: build_initial_monmap 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.134+0000 7fda3b7a7640 10 monclient: monmap: 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:48.141819+0000 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:48.141819+0000 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:48.146 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 Processor -- start 2024-09-15T21:17:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 -- start start 2024-09-15T21:17:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient: init 2024-09-15T21:17:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-09-15T21:17:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fda340a0890 0x7fda340a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): picked mon.noname-c con 0x7fda340a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda3409ffc0 0x7fda340a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): picked mon.noname-a con 0x7fda3409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fda34135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda340a3130 con 0x7fda34135670 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fda340a16f0 con 0x7fda3409ffc0 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fda34073df0 con 0x7fda340a0890 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda3b7a7640 10 monclient(hunting): authenticate will time out at 991.537199s 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda3409ffc0 0x7fda340a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda3409ffc0 0x7fda340a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51900/0 (socket says 172.21.15.17:51900) 2024-09-15T21:17:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 -- 172.21.15.17:0/3831915035 learned_addr learned my addr 172.21.15.17:0/3831915035 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): get_auth_request con 0x7fda3409ffc0 auth_method 0 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda39543640 1 --2- 172.21.15.17:0/3831915035 >> v2:172.21.15.119:3300/0 conn(0x7fda340a0890 0x7fda340a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda39543640 10 monclient(hunting): get_auth_request con 0x7fda340a0890 auth_method 0 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda39543640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda39543640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda39543640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient(hunting): handle_auth_done global_id 4377 payload 274 2024-09-15T21:17:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 -- 172.21.15.17:0/3831915035 >> v2:172.21.15.119:3300/0 conn(0x7fda340a0890 msgr2=0x7fda340a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 --2- 172.21.15.17:0/3831915035 >> v2:172.21.15.119:3300/0 conn(0x7fda340a0890 0x7fda340a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 -- 172.21.15.17:0/3831915035 >> v1:172.21.15.119:6789/0 conn(0x7fda34135670 legacy=0x7fda34135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient: _finish_hunting 0 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 monclient: found mon.noname-a 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 -- 172.21.15.17:0/3831915035 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda340a1bf0 con 0x7fda3409ffc0 2024-09-15T21:17:48.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.138+0000 7fda38d42640 1 --2- 172.21.15.17:0/3831915035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda3409ffc0 0x7fda340a11b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda2401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5efc1dee7864156 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 1 -- 172.21.15.17:0/3831915035 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240b0070 con 0x7fda3409ffc0 2024-09-15T21:17:48.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: dump: 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.158 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: _finish_auth 0 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 1 -- 172.21.15.17:0/3831915035 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda240ab600 con 0x7fda3409ffc0 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 5 monclient: authenticate success, global_id 4377 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 1 -- 172.21.15.17:0/3831915035 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240ab9d0 con 0x7fda3409ffc0 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 4 set_mon_vals no callback set 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2bfff640 10 monclient: dump: 2024-09-15T21:17:48.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: shutdown 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3831915035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda3409ffc0 msgr2=0x7fda340a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3831915035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda3409ffc0 0x7fda340a11b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda24009870 tx=0x7fda2401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3831915035 shutdown_connections 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3831915035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda3409ffc0 0x7fda340a11b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3831915035 >> v2:172.21.15.119:3300/0 conn(0x7fda340a0890 0x7fda340a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3831915035 >> 172.21.15.17:0/3831915035 conn(0x7fda340a66a0 msgr2=0x7fda340a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3831915035 shutdown_connections 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3831915035 wait complete. 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: build_initial_monmap 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: monmap: 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 Processor -- start 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- start start 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: init 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fda3409ffc0 0x7fda34134870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): picked mon.noname-c con 0x7fda3409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda340a0890 0x7fda34134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): picked mon.noname-b con 0x7fda340a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda34135670 0x7fda3411d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): picked mon.noname-a con 0x7fda34135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fda34140690 con 0x7fda34135670 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda34140510 con 0x7fda340a0890 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fda34140810 con 0x7fda3409ffc0 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient(hunting): authenticate will time out at 991.542378s 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fda3409ffc0 0x7fda34134870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda34135670 0x7fda3411d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fda3409ffc0 0x7fda34134870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33176/0 (socket says 172.21.15.17:33176) 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 -- 172.21.15.17:0/3484819937 learned_addr learned my addr 172.21.15.17:0/3484819937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): get_auth_request con 0x7fda34135670 auth_method 0 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): get_auth_request con 0x7fda3409ffc0 auth_method 0 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda340a0890 0x7fda34134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): get_auth_request con 0x7fda340a0890 auth_method 0 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient(hunting): handle_auth_done global_id 4355 payload 824 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda340a0890 msgr2=0x7fda34134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda340a0890 0x7fda34134db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda34135670 msgr2=0x7fda3411d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda34135670 0x7fda3411d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient: _finish_hunting 0 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 monclient: found mon.noname-c 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3413cf80 con 0x7fda3409ffc0 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39d44640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda34135670 0x7fda3411d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda39543640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fda3409ffc0 0x7fda34134870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d800 tx=0x7fda3000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3000b6c0 con 0x7fda3409ffc0 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: dump: 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda30019070 con 0x7fda3409ffc0 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3000bae0 con 0x7fda3409ffc0 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 5 monclient: authenticate success, global_id 4355 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: _renew_subs 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3409d0f0 con 0x7fda3409ffc0 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda2b7fe640 4 set_mon_vals no callback set 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda29ffb640 10 monclient: dump: 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: _renew_subs 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda34120f60 con 0x7fda3409ffc0 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.142+0000 7fda3b7a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 0x7fda3402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda341cde10 con 0x7fda3411e160 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fda3000bc80 con 0x7fda3409ffc0 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 0x7fda3402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient: get_auth_request con 0x7fda3411e160 auth_method 0 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fda1803b9c0 0x7fda1803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fda3004fdf0 con 0x7fda3409ffc0 2024-09-15T21:17:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 10 monclient(hunting): handle_auth_done global_id 4366 payload 824 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda34135aa0 con 0x7fda3411e160 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda39d44640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fda1803b9c0 0x7fda1803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda39d44640 10 monclient: get_auth_request con 0x7fda1803b9c0 auth_method 0 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda38d42640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 0x7fda3402d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda34073b00 tx=0x7fda2401ac20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240bb070 con 0x7fda3411e160 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 10 monclient: dump: 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda39d44640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fda1803b9c0 0x7fda1803de80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fda34121310 tx=0x7fda2c003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fda240b6470 con 0x7fda3411e160 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 msgr2=0x7fda3402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 0x7fda3402d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda34073b00 tx=0x7fda2401ac20 comp rx=0 tx=0).stop 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 msgr2=0x7fda3402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.146+0000 7fda29ffb640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e160 0x7fda3402d450 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda34073b00 tx=0x7fda2401ac20 comp rx=0 tx=0).stop 2024-09-15T21:17:48.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:17:48.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 0x7fda3402d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda34070ad0 con 0x7fda3411e540 2024-09-15T21:17:48.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 0x7fda3402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient: get_auth_request con 0x7fda3411e540 auth_method 0 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 10 monclient(hunting): handle_auth_done global_id 4369 payload 824 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 1 -- 172.21.15.17:0/3484819937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fda3411fa40 con 0x7fda3411e540 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda38d42640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 0x7fda3402d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda2401aa90 tx=0x7fda24019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240b0040 con 0x7fda3411e540 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:17:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 10 monclient: dump: 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7fda240b6620 con 0x7fda3411e540 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:48.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 msgr2=0x7fda3402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 0x7fda3402d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda2401aa90 tx=0x7fda24019180 comp rx=0 tx=0).stop 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 msgr2=0x7fda3402d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda29ffb640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda3411e540 0x7fda3402d990 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda2401aa90 tx=0x7fda24019180 comp rx=0 tx=0).stop 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 8, 2024-09-15T21:17:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 33735, 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 28, 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:17:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:17:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:17:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:17:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:17:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:17:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:17:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:17:48.190 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:17:48.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fda1803b9c0 msgr2=0x7fda1803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fda1803b9c0 0x7fda1803de80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fda34121310 tx=0x7fda2c003d50 comp rx=0 tx=0).stop 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 10 monclient: shutdown 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fda3409ffc0 msgr2=0x7fda34134870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 --2- 172.21.15.17:0/3484819937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fda3409ffc0 0x7fda34134870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d800 tx=0x7fda3000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda39d44640 1 -- 172.21.15.17:0/3484819937 reap_dead start 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda39d44640 1 -- 172.21.15.17:0/3484819937 reap_dead start 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 shutdown_connections 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 >> 172.21.15.17:0/3484819937 conn(0x7fda340a66a0 msgr2=0x7fda340a4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 shutdown_connections 2024-09-15T21:17:48.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.154+0000 7fda3b7a7640 1 -- 172.21.15.17:0/3484819937 wait complete. 2024-09-15T21:17:48.193 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:17:48.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:17:48.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:17:48.193 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:17:48.193 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:17:48.193 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:17:48.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:17:48.216 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:17:48.216 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:17:48.216 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:17:48.317 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-15T21:17:48.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient: get_monmap_and_config 2024-09-15T21:17:48.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient: build_initial_monmap 2024-09-15T21:17:48.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient: monmap: 2024-09-15T21:17:48.328 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:48.328 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.328 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:48.324493+0000 2024-09-15T21:17:48.328 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:48.324493+0000 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:48.329 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:48.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 1 Processor -- start 2024-09-15T21:17:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 1 -- start start 2024-09-15T21:17:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient: init 2024-09-15T21:17:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2024-09-15T21:17:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9cf409d400 0x7f9cf409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient(hunting): picked mon.noname-c con 0x7f9cf409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:17:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9cf409dd20 0x7f9cf4134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.318+0000 7f9cfc071640 10 monclient(hunting): picked mon.noname-a con 0x7f9cf409dd20 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9cf4134ba0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf40a3130 con 0x7f9cf4134ba0 2024-09-15T21:17:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf40a1bc0 con 0x7f9cf409dd20 2024-09-15T21:17:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf40a3490 con 0x7f9cf409d400 2024-09-15T21:17:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient(hunting): authenticate will time out at 991.720086s 2024-09-15T21:17:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf9e0d640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f9cf409d400 msgr2=0x7f9cf409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf9e0d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9cf409d400 0x7f9cf409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9cf409dd20 0x7f9cf4134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9cf409dd20 0x7f9cf4134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51916/0 (socket says 172.21.15.17:51916) 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 -- 172.21.15.17:0/409193148 learned_addr learned my addr 172.21.15.17:0/409193148 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): get_auth_request con 0x7f9cf409dd20 auth_method 0 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient(hunting): handle_auth_done global_id 4383 payload 274 2024-09-15T21:17:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 -- 172.21.15.17:0/409193148 >> v2:172.21.15.119:3300/0 conn(0x7f9cf409d400 msgr2=0x7f9cf409d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 --2- 172.21.15.17:0/409193148 >> v2:172.21.15.119:3300/0 conn(0x7f9cf409d400 0x7f9cf409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 -- 172.21.15.17:0/409193148 >> v1:172.21.15.17:6789/0 conn(0x7f9cf4134ba0 legacy=0x7f9cf413bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient: _finish_hunting 0 2024-09-15T21:17:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 monclient: found mon.noname-a 2024-09-15T21:17:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 -- 172.21.15.17:0/409193148 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf409e7a0 con 0x7f9cf409dd20 2024-09-15T21:17:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf960c640 1 --2- 172.21.15.17:0/409193148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409dd20 0x7f9cf4134660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cf409d0a0 tx=0x7f9cf001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d312b23b962f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 1 -- 172.21.15.17:0/409193148 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00b0070 con 0x7f9cf409dd20 2024-09-15T21:17:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: dump: 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: _finish_auth 0 2024-09-15T21:17:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: _check_auth_tickets 2024-09-15T21:17:48.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:48.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 1 -- 172.21.15.17:0/409193148 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf00ab600 con 0x7f9cf409dd20 2024-09-15T21:17:48.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:48.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 1 -- 172.21.15.17:0/409193148 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00ab9d0 con 0x7f9cf409dd20 2024-09-15T21:17:48.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 5 monclient: authenticate success, global_id 4383 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 4 set_mon_vals no callback set 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cf8e0b640 10 monclient: dump: 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.353 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.354 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.354 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.354 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.354 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.354 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.354 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 10 monclient: shutdown 2024-09-15T21:17:48.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:48.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- 172.21.15.17:0/409193148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409dd20 msgr2=0x7f9cf4134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 --2- 172.21.15.17:0/409193148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409dd20 0x7f9cf4134660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cf409d0a0 tx=0x7f9cf001c400 comp rx=0 tx=0).stop 2024-09-15T21:17:48.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- 172.21.15.17:0/409193148 shutdown_connections 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 --2- 172.21.15.17:0/409193148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409dd20 0x7f9cf4134660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 --2- 172.21.15.17:0/409193148 >> v2:172.21.15.119:3300/0 conn(0x7f9cf409d400 0x7f9cf409d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- 172.21.15.17:0/409193148 >> 172.21.15.17:0/409193148 conn(0x7f9cf40a66a0 msgr2=0x7f9cf40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- 172.21.15.17:0/409193148 shutdown_connections 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.322+0000 7f9cfc071640 1 -- 172.21.15.17:0/409193148 wait complete. 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: build_initial_monmap 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: monmap: 2024-09-15T21:17:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 Processor -- start 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- start start 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: init 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 0x7f9cf41cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): picked mon.noname-a con 0x7f9cf409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf409dd20 0x7f9cf41cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): picked mon.noname-b con 0x7f9cf409dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cf4134ba0 0x7f9cf41d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): picked mon.noname-c con 0x7f9cf4134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf409ce50 con 0x7f9cf409d400 2024-09-15T21:17:48.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf409ccd0 con 0x7f9cf409dd20 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf409fa20 con 0x7f9cf4134ba0 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient(hunting): authenticate will time out at 991.725075s 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf960c640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf409dd20 msgr2=0x7f9cf41cffd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 0x7f9cf41cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf960c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf409dd20 0x7f9cf41cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 0x7f9cf41cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51922/0 (socket says 172.21.15.17:51922) 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 -- 172.21.15.17:0/1313552886 learned_addr learned my addr 172.21.15.17:0/1313552886 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): get_auth_request con 0x7f9cf409d400 auth_method 0 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cf4134ba0 0x7f9cf41d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): get_auth_request con 0x7f9cf4134ba0 auth_method 0 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfa60e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient(hunting): handle_auth_done global_id 4386 payload 824 2024-09-15T21:17:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 -- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cf4134ba0 msgr2=0x7f9cf41d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cf4134ba0 0x7f9cf41d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 -- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf409dd20 msgr2=0x7f9cf41cffd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf409dd20 0x7f9cf41cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient: _finish_hunting 0 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 monclient: found mon.noname-a 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 -- 172.21.15.17:0/1313552886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf413b550 con 0x7f9cf409d400 2024-09-15T21:17:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf9e0d640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 0x7f9cf41cfa90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b530 tx=0x7f9ce400b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4013020 con 0x7f9cf409d400 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: dump: 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.362 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce4003df0 con 0x7f9cf409d400 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 5 monclient: authenticate success, global_id 4386 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _renew_subs 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf413b830 con 0x7f9cf409d400 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40052b0 con 0x7f9cf409d400 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:48.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 10 monclient: dump: 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cebfff640 4 set_mon_vals no callback set 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _renew_subs 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf413b270 con 0x7f9cf409d400 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf413c6e0 con 0x7f9cf409d400 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9ce4016020 con 0x7f9cf409d400 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cd803b970 0x7f9cd803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f9ce404f520 con 0x7f9cf409d400 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf960c640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cd803b970 0x7f9cd803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.326+0000 7f9cf960c640 10 monclient: get_auth_request con 0x7f9cd803b970 auth_method 0 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.330+0000 7f9cf960c640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cd803b970 0x7f9cd803de30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00ad000 tx=0x7f9cf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.330+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9ce401d020 con 0x7f9cf409d400 2024-09-15T21:17:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.330+0000 7f9cea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.330+0000 7f9cea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cfc071640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:17:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cfc071640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cfc071640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9cf409d7e0 con 0x7f9cf409d400 2024-09-15T21:17:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cea7fc640 1 -- 172.21.15.17:0/1313552886 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f9ce400d380 con 0x7f9cf409d400 2024-09-15T21:17:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:48.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.570+0000 7f9cea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":28,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cd803b970 msgr2=0x7f9cd803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cd803b970 0x7f9cd803de30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00ad000 tx=0x7f9cf0008040 comp rx=0 tx=0).stop 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 10 monclient: shutdown 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 msgr2=0x7f9cf41cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 0x7f9cf41cfa90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b530 tx=0x7f9ce400b9f0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 shutdown_connections 2024-09-15T21:17:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cd803b970 0x7f9cd803de30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cf4134ba0 0x7f9cf41d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf409dd20 0x7f9cf41cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 --2- 172.21.15.17:0/1313552886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf409d400 0x7f9cf41cfa90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 >> 172.21.15.17:0/1313552886 conn(0x7f9cf40a66a0 msgr2=0x7f9cf4073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 shutdown_connections 2024-09-15T21:17:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:48.574+0000 7f9cfc071640 1 -- 172.21.15.17:0/1313552886 wait complete. 2024-09-15T21:17:51.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:17:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient: get_monmap_and_config 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient: build_initial_monmap 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient: monmap: 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:51.731592+0000 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:51.731592+0000 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:51.736 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 1 Processor -- start 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 1 -- start start 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient: init 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe640a0890 0x7fbe640a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient(hunting): picked mon.noname-a con 0x7fbe640a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbe6409ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbe640a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe64136490 con 0x7fbe640a11b0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe640a3130 con 0x7fbe6409ffc0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbe64073df0 con 0x7fbe640a0890 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe692c0640 10 monclient(hunting): authenticate will time out at 995.127758s 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe627fc640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fbe6409ffc0 legacy=0x7fbe64135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe62ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe640a0890 0x7fbe640a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe62ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe640a0890 0x7fbe640a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51936/0 (socket says 172.21.15.17:51936) 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe637fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbe640a11b0 0x7fbe64135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41750/0 (socket says 172.21.15.17:41750) 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.726+0000 7fbe637fe640 1 -- 172.21.15.17:0/3240628003 learned_addr learned my addr 172.21.15.17:0/3240628003 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): get_auth_request con 0x7fbe640a0890 auth_method 0 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_done global_id 4389 payload 274 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 1 -- 172.21.15.17:0/3240628003 >> v1:172.21.15.119:6789/0 conn(0x7fbe6409ffc0 legacy=0x7fbe64135670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 1 -- 172.21.15.17:0/3240628003 >> v1:172.21.15.17:6789/0 conn(0x7fbe640a11b0 legacy=0x7fbe64135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:17:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 1 monclient: found mon.noname-a 2024-09-15T21:17:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 1 -- 172.21.15.17:0/3240628003 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640a1bf0 con 0x7fbe640a0890 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe62ffd640 1 --2- 172.21.15.17:0/3240628003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe640a0890 0x7fbe640a0c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe540097f0 tx=0x7fbe5401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cc95aedfbd8f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 1 -- 172.21.15.17:0/3240628003 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540b0070 con 0x7fbe640a0890 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: dump: 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 1 -- 172.21.15.17:0/3240628003 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540ab600 con 0x7fbe640a0890 2024-09-15T21:17:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 5 monclient: authenticate success, global_id 4389 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 4 set_mon_vals no callback set 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 1 -- 172.21.15.17:0/3240628003 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540ab9d0 con 0x7fbe640a0890 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe61ffb640 10 monclient: dump: 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 10 monclient: shutdown 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 -- 172.21.15.17:0/3240628003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe640a0890 msgr2=0x7fbe640a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 --2- 172.21.15.17:0/3240628003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe640a0890 0x7fbe640a0c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe540097f0 tx=0x7fbe5401c360 comp rx=0 tx=0).stop 2024-09-15T21:17:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 -- 172.21.15.17:0/3240628003 shutdown_connections 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 --2- 172.21.15.17:0/3240628003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe640a0890 0x7fbe640a0c70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 -- 172.21.15.17:0/3240628003 >> 172.21.15.17:0/3240628003 conn(0x7fbe640a66a0 msgr2=0x7fbe640a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 -- 172.21.15.17:0/3240628003 shutdown_connections 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.730+0000 7fbe692c0640 1 -- 172.21.15.17:0/3240628003 wait complete. 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: build_initial_monmap 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: monmap: 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 Processor -- start 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- start start 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: init 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 0x7fbe64134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): picked mon.noname-a con 0x7fbe6409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe640a0890 0x7fbe64135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): picked mon.noname-b con 0x7fbe640a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe640a11b0 0x7fbe6411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): picked mon.noname-c con 0x7fbe640a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64141ab0 con 0x7fbe6409ffc0 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64141930 con 0x7fbe640a0890 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbe64141c30 con 0x7fbe640a11b0 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient(hunting): authenticate will time out at 995.133543s 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 0x7fbe64134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 0x7fbe64134f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51942/0 (socket says 172.21.15.17:51942) 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 -- 172.21.15.17:0/641326272 learned_addr learned my addr 172.21.15.17:0/641326272 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): get_auth_request con 0x7fbe6409ffc0 auth_method 0 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe627fc640 1 -- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe640a0890 msgr2=0x7fbe64135490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe627fc640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe640a0890 0x7fbe64135490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe640a11b0 0x7fbe6411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): get_auth_request con 0x7fbe640a11b0 auth_method 0 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient(hunting): handle_auth_done global_id 4395 payload 824 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 -- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe640a11b0 msgr2=0x7fbe6411c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe640a11b0 0x7fbe6411c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 -- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe640a0890 msgr2=0x7fbe64135490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe640a0890 0x7fbe64135490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 monclient: found mon.noname-a 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 -- 172.21.15.17:0/641326272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6413e450 con 0x7fbe6409ffc0 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe62ffd640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 0x7fbe64134f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe540ae040 tx=0x7fbe54019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540b8020 con 0x7fbe6409ffc0 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: dump: 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540b4630 con 0x7fbe6409ffc0 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540b0040 con 0x7fbe6409ffc0 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 5 monclient: authenticate success, global_id 4395 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _renew_subs 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6411ea50 con 0x7fbe6409ffc0 2024-09-15T21:17:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe617fa640 4 set_mon_vals no callback set 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 10 monclient: dump: 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _renew_subs 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe640726a0 con 0x7fbe6409ffc0 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbe540bb020 con 0x7fbe6409ffc0 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6409d0f0 con 0x7fbe6409ffc0 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9a0 0x7fbe5003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.734+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fbe540e9530 con 0x7fbe6409ffc0 2024-09-15T21:17:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.738+0000 7fbe627fc640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9a0 0x7fbe5003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.738+0000 7fbe627fc640 10 monclient: get_auth_request con 0x7fbe5003b9a0 auth_method 0 2024-09-15T21:17:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.738+0000 7fbe627fc640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9a0 0x7fbe5003de60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe6411e380 tx=0x7fbe58006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.742+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbe540c2030 con 0x7fbe6409ffc0 2024-09-15T21:17:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.742+0000 7fbe4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.742+0000 7fbe4b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:51.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:17:51.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:51.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:17:51.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbe640a0c70 con 0x7fbe6409ffc0 2024-09-15T21:17:51.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe4b7fe640 1 -- 172.21.15.17:0/641326272 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fbe54016070 con 0x7fbe6409ffc0 2024-09-15T21:17:51.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:51.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe4b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:51.990 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":32,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9a0 msgr2=0x7fbe5003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9a0 0x7fbe5003de60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe6411e380 tx=0x7fbe58006d90 comp rx=0 tx=0).stop 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 10 monclient: shutdown 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 msgr2=0x7fbe64134f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 0x7fbe64134f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe540ae040 tx=0x7fbe54019100 comp rx=0 tx=0).stop 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 shutdown_connections 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9a0 0x7fbe5003de60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe640a11b0 0x7fbe6411c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe640a0890 0x7fbe64135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 --2- 172.21.15.17:0/641326272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe6409ffc0 0x7fbe64134f50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 >> 172.21.15.17:0/641326272 conn(0x7fbe640a66a0 msgr2=0x7fbe64072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 shutdown_connections 2024-09-15T21:17:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:51.982+0000 7fbe692c0640 1 -- 172.21.15.17:0/641326272 wait complete. 2024-09-15T21:17:54.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-15T21:17:55.012 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient: get_monmap_and_config 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient: build_initial_monmap 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient: monmap: 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:55.140197+0000 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:55.140197+0000 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:55.144 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:55.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 1 Processor -- start 2024-09-15T21:17:55.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 1 -- start start 2024-09-15T21:17:55.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient: init 2024-09-15T21:17:55.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:55.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efcf40a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efcf40a5580 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf4140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient(hunting): picked mon.noname-b con 0x7efcf40a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf40a2c30 con 0x7efcf40a5580 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf40a19b0 con 0x7efcf40a4a90 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efcf40a1c90 con 0x7efcf40a59d0 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcfba5d640 10 monclient(hunting): authenticate will time out at 998.535724s 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf97f9640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7efcf40a4a90 legacy=0x7efcf40a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf9ffa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf4140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf8ff8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efcf40a5580 0x7efcf413cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41764/0 (socket says 172.21.15.17:41764) 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf8ff8640 1 -- 172.21.15.17:0/3338495642 learned_addr learned my addr 172.21.15.17:0/3338495642 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf9ffa640 10 monclient(hunting): get_auth_request con 0x7efcf40a59d0 auth_method 0 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf9ffa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf9ffa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.134+0000 7efcf9ffa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_done global_id 4364 payload 274 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 1 -- 172.21.15.17:0/3338495642 >> v1:172.21.15.119:6789/0 conn(0x7efcf40a4a90 legacy=0x7efcf40a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 1 -- 172.21.15.17:0/3338495642 >> v1:172.21.15.17:6789/0 conn(0x7efcf40a5580 legacy=0x7efcf413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 10 monclient: _finish_hunting 0 2024-09-15T21:17:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 1 monclient: found mon.noname-b 2024-09-15T21:17:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:17:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 1 -- 172.21.15.17:0/3338495642 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf40a1f70 con 0x7efcf40a59d0 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf9ffa640 1 --2- 172.21.15.17:0/3338495642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf4140440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcec009df0 tx=0x7efcec01c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e69c14bc07bdf107 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 1 -- 172.21.15.17:0/3338495642 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0b2070 con 0x7efcf40a59d0 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: dump: 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: _finish_auth 0 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 1 -- 172.21.15.17:0/3338495642 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcec0a57e0 con 0x7efcf40a59d0 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 5 monclient: authenticate success, global_id 4364 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 10 monclient: get_monmap_and_config success 2024-09-15T21:17:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 1 -- 172.21.15.17:0/3338495642 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0a5bb0 con 0x7efcf40a59d0 2024-09-15T21:17:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 4 set_mon_vals no callback set 2024-09-15T21:17:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcf3fff640 10 monclient: dump: 2024-09-15T21:17:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 10 monclient: shutdown 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 -- 172.21.15.17:0/3338495642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 msgr2=0x7efcf4140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 --2- 172.21.15.17:0/3338495642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf4140440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcec009df0 tx=0x7efcec01c850 comp rx=0 tx=0).stop 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 -- 172.21.15.17:0/3338495642 shutdown_connections 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 --2- 172.21.15.17:0/3338495642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf4140440 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 -- 172.21.15.17:0/3338495642 >> 172.21.15.17:0/3338495642 conn(0x7efcf40aabb0 msgr2=0x7efcf40a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 -- 172.21.15.17:0/3338495642 shutdown_connections 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.138+0000 7efcfba5d640 1 -- 172.21.15.17:0/3338495642 wait complete. 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: build_initial_monmap 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: monmap: 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 Processor -- start 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- start start 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: init 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 0x7efcf4137300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): picked mon.noname-a con 0x7efcf40a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40a5580 0x7efcf4137840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): picked mon.noname-b con 0x7efcf40a5580 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf413cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): picked mon.noname-c con 0x7efcf40a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efcf41439d0 con 0x7efcf40a4a90 2024-09-15T21:17:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4143850 con 0x7efcf40a5580 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efcf4143b50 con 0x7efcf40a59d0 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient(hunting): authenticate will time out at 998.541021s 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 0x7efcf4137300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf413cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 0x7efcf4137300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51950/0 (socket says 172.21.15.17:51950) 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 1 -- 172.21.15.17:0/2230738162 learned_addr learned my addr 172.21.15.17:0/2230738162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf413cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33218/0 (socket says 172.21.15.17:33218) 2024-09-15T21:17:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): get_auth_request con 0x7efcf40a4a90 auth_method 0 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf8ff8640 1 -- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40a5580 msgr2=0x7efcf4137840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): get_auth_request con 0x7efcf40a59d0 auth_method 0 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf8ff8640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40a5580 0x7efcf4137840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient(hunting): handle_auth_done global_id 4367 payload 824 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 -- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40a5580 msgr2=0x7efcf4137840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40a5580 0x7efcf4137840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 -- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 msgr2=0x7efcf4137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 0x7efcf4137300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient: _finish_hunting 0 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 monclient: found mon.noname-c 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 -- 172.21.15.17:0/2230738162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf4120c80 con 0x7efcf40a59d0 2024-09-15T21:17:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 0x7efcf4137300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf9ffa640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf413cb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcf40ab0f0 tx=0x7efcec0195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0bd020 con 0x7efcf40a59d0 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: dump: 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcec0b8670 con 0x7efcf40a59d0 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 5 monclient: authenticate success, global_id 4367 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _renew_subs 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf4120f60 con 0x7efcf40a59d0 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0b2040 con 0x7efcf40a59d0 2024-09-15T21:17:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 10 monclient: dump: 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf2ffd640 4 set_mon_vals no callback set 2024-09-15T21:17:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _renew_subs 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf413fe80 con 0x7efcf40a59d0 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf40a4e70 con 0x7efcf40a59d0 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7efcec0c0020 con 0x7efcf40a59d0 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efce003b970 0x7efce003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7efcec0ea830 con 0x7efcf40a59d0 2024-09-15T21:17:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efce003b970 0x7efce003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.142+0000 7efcf97f9640 10 monclient: get_auth_request con 0x7efce003b970 auth_method 0 2024-09-15T21:17:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.146+0000 7efcf97f9640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efce003b970 0x7efce003de30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efce4003fd0 tx=0x7efce4003ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:17:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.146+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7efcec0c7030 con 0x7efcf40a59d0 2024-09-15T21:17:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.146+0000 7efcf1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:17:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.146+0000 7efcf1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:17:55.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.386+0000 7efcfba5d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:17:55.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.386+0000 7efcfba5d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:55.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.386+0000 7efcfba5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:17:55.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efcf4069200 con 0x7efcf40a59d0 2024-09-15T21:17:55.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcf1ffb640 1 -- 172.21.15.17:0/2230738162 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7efcec016560 con 0x7efcf40a59d0 2024-09-15T21:17:55.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcf1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:17:55.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcf1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:17:55.398 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":35,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:17:55.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efce003b970 msgr2=0x7efce003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:55.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efce003b970 0x7efce003de30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efce4003fd0 tx=0x7efce4003ba0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 10 monclient: shutdown 2024-09-15T21:17:55.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:17:55.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 msgr2=0x7efcf413cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf413cb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcf40ab0f0 tx=0x7efcec0195f0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 shutdown_connections 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efce003b970 0x7efce003de30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf40a59d0 0x7efcf413cb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40a5580 0x7efcf4137840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 --2- 172.21.15.17:0/2230738162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a4a90 0x7efcf4137300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 >> 172.21.15.17:0/2230738162 conn(0x7efcf40aabb0 msgr2=0x7efcf40a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 shutdown_connections 2024-09-15T21:17:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:55.390+0000 7efcfba5d640 1 -- 172.21.15.17:0/2230738162 wait complete. 2024-09-15T21:17:58.418 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:17:58.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient: get_monmap_and_config 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient: build_initial_monmap 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient: monmap: 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:17:58.547834+0000 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:17:58.547834+0000 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:17:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:17:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:17:58.552 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:17:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 Processor -- start 2024-09-15T21:17:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 -- start start 2024-09-15T21:17:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient: init 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient: _reopen_session rank -1 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6480a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): picked mon.noname-a con 0x7fe64809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): picked mon.noname-b con 0x7fe6480a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6480a3130 con 0x7fe6480a0890 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe648073df0 con 0x7fe64809ffc0 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe6480a1bf0 con 0x7fe6480a1380 2024-09-15T21:17:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): _renew_subs 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64eaf8640 10 monclient(hunting): authenticate will time out at 1001.943500s 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47524/0 (socket says 172.21.15.17:47524) 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 learned_addr learned my addr 172.21.15.17:0/918858489 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 10 monclient(hunting): get_auth_request con 0x7fe64809ffc0 auth_method 0 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64d095640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64c894640 1 --1- 172.21.15.17:0/918858489 >> v1:172.21.15.17:6789/0 conn(0x7fe6480a0890 0x7fe6480a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56236/0 (socket says 172.21.15.17:56236) 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64d095640 10 monclient(hunting): get_auth_request con 0x7fe6480a1380 auth_method 0 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64d095640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64d095640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe64d095640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.542+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.546+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.546+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.546+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.546+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.546+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:17:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 10 monclient(hunting): get_auth_request con 0x7fe64809ffc0 auth_method 0 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 10 monclient(hunting): get_auth_request con 0x7fe6480a1380 auth_method 0 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:58.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:17:58.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:17:58.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:58.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:58.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:58.750+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:17:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 10 monclient(hunting): get_auth_request con 0x7fe6480a1380 auth_method 0 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 10 monclient(hunting): get_auth_request con 0x7fe64809ffc0 auth_method 0 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:59.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:59.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.154+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:17:59.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:59.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 10 monclient(hunting): get_auth_request con 0x7fe64809ffc0 auth_method 0 2024-09-15T21:17:59.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:17:59.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:59.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 10 monclient(hunting): get_auth_request con 0x7fe6480a1380 auth_method 0 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:17:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:17:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:17:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:17:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:17:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:17:59.958+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:18:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): tick 2024-09-15T21:18:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:18:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:18:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 msgr2=0x7fe64813d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6480a1380 0x7fe64813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 msgr2=0x7fe64813aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe64809ffc0 0x7fe64813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.17:6789/0 conn(0x7fe6480a0890 legacy=0x7fe6480a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient: _add_conns ranks=[4,1] 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 0x7fe628005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): picked mon.noname-c con 0x7fe628003700 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe628006250 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64809ccd0 con 0x7fe628006250 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe648073df0 con 0x7fe628003700 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe6467fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:01.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe64c894640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628006250 legacy=0x7fe628008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:18:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.546+0000 7fe64c894640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:01.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.750+0000 7fe64c894640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:01.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.750+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628006250 legacy=0x7fe628008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:18:01.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:01.750+0000 7fe64c894640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:18:02.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:02.154+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628006250 legacy=0x7fe628008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:18:02.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:02.154+0000 7fe64c894640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:02.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:02.154+0000 7fe64c894640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:18:02.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:02.958+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628006250 legacy=0x7fe628008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:18:02.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:02.958+0000 7fe64c894640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:02.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:02.958+0000 7fe64c894640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): tick 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe628003700 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628006250 legacy=0x7fe628008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 reap_dead start 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe628009f60 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe62800ab60 0x7fe628005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): picked mon.noname-c con 0x7fe62800ab60 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe628009030 0x7fe628008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): picked mon.noname-a con 0x7fe628009030 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62800afa0 con 0x7fe628009f60 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe648073df0 con 0x7fe628009030 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 1 -- 172.21.15.17:0/918858489 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe6480a1bf0 con 0x7fe62800ab60 2024-09-15T21:18:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe6467fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64c894640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628009f60 legacy=0x7fe62800a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe647fff640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe62800ab60 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe647fff640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe62800ab60 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.17:3300/0 conn(0x7fe628009030 0x7fe628008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): get_auth_request con 0x7fe628009030 auth_method 0 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient(hunting): handle_auth_done global_id 4401 payload 274 2024-09-15T21:18:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe62800ab60 msgr2=0x7fe628005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe62800ab60 0x7fe628005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 >> v1:172.21.15.119:6789/0 conn(0x7fe628009f60 legacy=0x7fe62800a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient: _finish_hunting 0 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 monclient: found mon.noname-a 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:04.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.546+0000 7fe64d095640 1 -- 172.21.15.17:0/918858489 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64809ccd0 con 0x7fe628009030 2024-09-15T21:18:04.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe64d095640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe628009030 0x7fe628008690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe64806f010 tx=0x7fe64001d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f775ccdd4fe6607f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 1 -- 172.21.15.17:0/918858489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6400ab070 con 0x7fe628009030 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 10 monclient: dump: 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 1 -- 172.21.15.17:0/918858489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6400a5cc0 con 0x7fe628009030 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe6477fe640 1 -- 172.21.15.17:0/918858489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6400af620 con 0x7fe628009030 2024-09-15T21:18:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.550+0000 7fe64eaf8640 5 monclient: authenticate success, global_id 4401 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 4 set_mon_vals no callback set 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe6477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe6477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe6477fe640 10 monclient: dump: 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.563 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 10 monclient: shutdown 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 -- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe628009030 msgr2=0x7fe628008690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe628009030 0x7fe628008690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe64806f010 tx=0x7fe64001d220 comp rx=0 tx=0).stop 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 -- 172.21.15.17:0/918858489 shutdown_connections 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/918858489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe628009030 0x7fe628008690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/918858489 >> v2:172.21.15.119:3300/0 conn(0x7fe62800ab60 0x7fe628005b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 -- 172.21.15.17:0/918858489 >> 172.21.15.17:0/918858489 conn(0x7fe6480a66a0 msgr2=0x7fe6480a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 -- 172.21.15.17:0/918858489 shutdown_connections 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 -- 172.21.15.17:0/918858489 wait complete. 2024-09-15T21:18:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 10 monclient: build_initial_monmap 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 10 monclient: monmap: 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 Processor -- start 2024-09-15T21:18:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 1 -- start start 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 10 monclient: init 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.554+0000 7fe64eaf8640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 0x7fe648134c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): picked mon.noname-c con 0x7fe628009030 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe628009f60 0x7fe6481ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): picked mon.noname-b con 0x7fe628009f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62800ab60 0x7fe6481ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): picked mon.noname-a con 0x7fe62800ab60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe648136210 con 0x7fe62800ab60 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe648136090 con 0x7fe628009f60 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe648136390 con 0x7fe628009030 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient(hunting): authenticate will time out at 1007.956170s 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe647fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe628009f60 msgr2=0x7fe6481ca170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe647fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe628009f60 0x7fe6481ca170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62800ab60 0x7fe6481ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 0x7fe648134c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62800ab60 0x7fe6481ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47554/0 (socket says 172.21.15.17:47554) 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 0x7fe648134c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57840/0 (socket says 172.21.15.17:57840) 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 1 -- 172.21.15.17:0/164138989 learned_addr learned my addr 172.21.15.17:0/164138989 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): get_auth_request con 0x7fe62800ab60 auth_method 0 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): get_auth_request con 0x7fe628009030 auth_method 0 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64d095640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient(hunting): handle_auth_done global_id 4370 payload 824 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 -- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe628009f60 msgr2=0x7fe6481ca170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe628009f60 0x7fe6481ca170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 -- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62800ab60 msgr2=0x7fe6481ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62800ab60 0x7fe6481ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient: _finish_hunting 0 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 monclient: found mon.noname-c 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 -- 172.21.15.17:0/164138989 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6481212b0 con 0x7fe628009030 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64c894640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 0x7fe648134c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c420 tx=0x7fe63800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380074e0 con 0x7fe628009030 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: dump: 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe638007680 con 0x7fe628009030 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638004a10 con 0x7fe628009030 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 5 monclient: authenticate success, global_id 4370 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _renew_subs 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe648123050 con 0x7fe628009030 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe646ffd640 4 set_mon_vals no callback set 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 10 monclient: dump: 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _renew_subs 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:04.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6481cadc0 con 0x7fe628009030 2024-09-15T21:18:04.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:04.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:04.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:04.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe648138ab0 con 0x7fe628009030 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe63801a020 con 0x7fe628009030 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe62803d410 0x7fe62803f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fe63804f150 con 0x7fe628009030 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe647fff640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe62803d410 0x7fe62803f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.558+0000 7fe647fff640 10 monclient: get_auth_request con 0x7fe62803d410 auth_method 0 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.562+0000 7fe647fff640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe62803d410 0x7fe62803f8d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe630006db0 tx=0x7fe630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.566+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe6380113a0 con 0x7fe628009030 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.566+0000 7fe6457fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:04.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.566+0000 7fe6457fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe64806d020 con 0x7fe628009030 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe6457fa640 1 -- 172.21.15.17:0/164138989 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7fe638004d90 con 0x7fe628009030 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe6457fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe6457fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:04.823 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe62803d410 msgr2=0x7fe62803f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe62803d410 0x7fe62803f8d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe630006db0 tx=0x7fe630008040 comp rx=0 tx=0).stop 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 10 monclient: shutdown 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 msgr2=0x7fe648134c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 0x7fe648134c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c420 tx=0x7fe63800c8e0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 shutdown_connections 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe62803d410 0x7fe62803f8d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62800ab60 0x7fe6481ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe628009f60 0x7fe6481ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 --2- 172.21.15.17:0/164138989 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe628009030 0x7fe648134c40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 >> 172.21.15.17:0/164138989 conn(0x7fe6480a66a0 msgr2=0x7fe6480a3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 shutdown_connections 2024-09-15T21:18:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.814+0000 7fe64eaf8640 1 -- 172.21.15.17:0/164138989 wait complete. 2024-09-15T21:18:04.842 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:18:04.842 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:18:04.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient: get_monmap_and_config 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient: build_initial_monmap 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient: monmap: 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:04.959113+0000 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:04.959113+0000 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:04.963 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 1 Processor -- start 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 1 -- start start 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient: init 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0440a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0440a5580 0x7fe04413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient(hunting): picked mon.noname-a con 0x7fe0440a5580 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:04.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe0440a59d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0440a2c30 con 0x7fe0440a4a90 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0440a19b0 con 0x7fe0440a59d0 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe0440a0f40 con 0x7fe0440a5580 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe048dce640 10 monclient(hunting): authenticate will time out at 1008.354685s 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0440a5580 0x7fe04413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe042d9d640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fe0440a59d0 legacy=0x7fe04413f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0440a5580 0x7fe04413cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47566/0 (socket says 172.21.15.17:47566) 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 -- 172.21.15.17:0/3391856807 learned_addr learned my addr 172.21.15.17:0/3391856807 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): get_auth_request con 0x7fe0440a5580 auth_method 0 2024-09-15T21:18:04.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient(hunting): handle_auth_done global_id 4407 payload 274 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 -- 172.21.15.17:0/3391856807 >> v1:172.21.15.119:6789/0 conn(0x7fe0440a59d0 legacy=0x7fe04413f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 -- 172.21.15.17:0/3391856807 >> v1:172.21.15.17:6789/0 conn(0x7fe0440a4a90 legacy=0x7fe0440a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:18:04.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 monclient: found mon.noname-a 2024-09-15T21:18:04.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:04.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:04.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.954+0000 7fe041d9b640 1 -- 172.21.15.17:0/3391856807 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440a1ce0 con 0x7fe0440a5580 2024-09-15T21:18:04.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe041d9b640 1 --2- 172.21.15.17:0/3391856807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a5580 0x7fe04413cd10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0098a0 tx=0x7fe02c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79959b4b33b84b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 1 -- 172.21.15.17:0/3391856807 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0b2070 con 0x7fe0440a5580 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: dump: 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: _finish_auth 0 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: _check_auth_tickets 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 1 -- 172.21.15.17:0/3391856807 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c0ad680 con 0x7fe0440a5580 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 1 -- 172.21.15.17:0/3391856807 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0ada50 con 0x7fe0440a5580 2024-09-15T21:18:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 5 monclient: authenticate success, global_id 4407 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 4 set_mon_vals no callback set 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe04159a640 10 monclient: dump: 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: shutdown 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- 172.21.15.17:0/3391856807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a5580 msgr2=0x7fe04413cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 --2- 172.21.15.17:0/3391856807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a5580 0x7fe04413cd10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0098a0 tx=0x7fe02c01c460 comp rx=0 tx=0).stop 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- 172.21.15.17:0/3391856807 shutdown_connections 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 --2- 172.21.15.17:0/3391856807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a5580 0x7fe04413cd10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- 172.21.15.17:0/3391856807 >> 172.21.15.17:0/3391856807 conn(0x7fe0440aabb0 msgr2=0x7fe0440a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- 172.21.15.17:0/3391856807 shutdown_connections 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- 172.21.15.17:0/3391856807 wait complete. 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: build_initial_monmap 2024-09-15T21:18:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: monmap: 2024-09-15T21:18:04.978 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:04.978 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:04.978 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:04.978 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:04.978 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:04.978 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 Processor -- start 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- start start 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: init 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0440a4a90 0x7fe0441cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): picked mon.noname-c con 0x7fe0440a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0440a5580 0x7fe04413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): picked mon.noname-b con 0x7fe0440a5580 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 0x7fe04413f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): picked mon.noname-a con 0x7fe0440a59d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441437c0 con 0x7fe0440a59d0 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe04406f2f0 con 0x7fe0440a5580 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe044143940 con 0x7fe0440a4a90 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.958+0000 7fe048dce640 10 monclient(hunting): authenticate will time out at 1008.359984s 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe041d9b640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0440a5580 msgr2=0x7fe04413ef70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe041d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0440a5580 0x7fe04413ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:04.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 0x7fe04413f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0440a4a90 0x7fe0441cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 0x7fe04413f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47574/0 (socket says 172.21.15.17:47574) 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0440a4a90 0x7fe0441cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57848/0 (socket says 172.21.15.17:57848) 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 -- 172.21.15.17:0/2536163360 learned_addr learned my addr 172.21.15.17:0/2536163360 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): get_auth_request con 0x7fe0440a59d0 auth_method 0 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): get_auth_request con 0x7fe0440a4a90 auth_method 0 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient(hunting): handle_auth_done global_id 4413 payload 824 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0440a4a90 msgr2=0x7fe0441cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0440a4a90 0x7fe0441cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0440a5580 msgr2=0x7fe04413ef70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:04.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0440a5580 0x7fe04413ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 monclient: found mon.noname-a 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04413b230 con 0x7fe0440a59d0 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0440a4a90 0x7fe0441cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe042d9d640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 0x7fe04413f4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe03400b400 tx=0x7fe03400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034016070 con 0x7fe0440a59d0 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: dump: 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.983 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: _finish_auth 0 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:04.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 5 monclient: authenticate success, global_id 4413 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe034003da0 con 0x7fe0440a59d0 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0340052b0 con 0x7fe0440a59d0 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: dump: 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe040d99640 4 set_mon_vals no callback set 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 10 monclient: _renew_subs 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04413f9f0 con 0x7fe0440a59d0 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 10 monclient: _renew_subs 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04413ca30 con 0x7fe0440a59d0 2024-09-15T21:18:04.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 0x7fe044138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440a4e70 con 0x7fe04413c190 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 0x7fe044138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe034019070 con 0x7fe0440a59d0 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient: get_auth_request con 0x7fe04413c190 auth_method 0 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 10 monclient(hunting): handle_auth_done global_id 4416 payload 824 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe04413fe50 con 0x7fe04413c190 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe02403b970 0x7fe02403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fe03404ec10 con 0x7fe0440a59d0 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe04259c640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 0x7fe044138cf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe03c001f60 tx=0x7fe03c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00de10 con 0x7fe04413c190 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: dump: 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.988 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe041d9b640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe02403b970 0x7fe02403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe041d9b640 10 monclient: get_auth_request con 0x7fe02403b970 auth_method 0 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe03c018070 con 0x7fe04413c190 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 msgr2=0x7fe044138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 0x7fe044138cf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe03c001f60 tx=0x7fe03c00d520 comp rx=0 tx=0).stop 2024-09-15T21:18:04.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 msgr2=0x7fe044138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:04.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe03affd640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04413c190 0x7fe044138cf0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe03c001f60 tx=0x7fe03c00d520 comp rx=0 tx=0).stop 2024-09-15T21:18:04.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.962+0000 7fe041d9b640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe02403b970 0x7fe02403de30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0af000 tx=0x7fe02c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:04.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 0x7fe044139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:04.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:04.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe044139b80 con 0x7fe044139230 2024-09-15T21:18:04.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 0x7fe044139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:04.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient: get_auth_request con 0x7fe044139230 auth_method 0 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 10 monclient(hunting): handle_auth_done global_id 4419 payload 824 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 1 -- 172.21.15.17:0/2536163360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe044069200 con 0x7fe044139230 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe04259c640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 0x7fe044139640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe03c001f60 tx=0x7fe03c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01ed70 con 0x7fe044139230 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 10 monclient: dump: 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:04.992 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7fe03c0050d0 con 0x7fe044139230 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 msgr2=0x7fe044139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:04.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 0x7fe044139640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe03c001f60 tx=0x7fe03c001260 comp rx=0 tx=0).stop 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 msgr2=0x7fe044139640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe03affd640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe044139230 0x7fe044139640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 12, 2024-09-15T21:18:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 50591, 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:05.008 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe02403b970 msgr2=0x7fe02403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe02403b970 0x7fe02403de30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0af000 tx=0x7fe02c008040 comp rx=0 tx=0).stop 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 10 monclient: shutdown 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 msgr2=0x7fe04413f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe042d9d640 1 -- 172.21.15.17:0/2536163360 reap_dead start 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 0x7fe04413f4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe03400b400 tx=0x7fe03400b8c0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 shutdown_connections 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 --2- 172.21.15.17:0/2536163360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0440a59d0 0x7fe04413f4b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 >> 172.21.15.17:0/2536163360 conn(0x7fe0440aabb0 msgr2=0x7fe0440a86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:05.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 shutdown_connections 2024-09-15T21:18:05.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:04.970+0000 7fe048dce640 1 -- 172.21.15.17:0/2536163360 wait complete. 2024-09-15T21:18:05.011 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:18:05.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:18:05.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:18:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient: get_monmap_and_config 2024-09-15T21:18:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient: build_initial_monmap 2024-09-15T21:18:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient: monmap: 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:05.135495+0000 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:05.135495+0000 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:05.139 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 Processor -- start 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 -- start start 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient: init 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d800a0890 0x7f6d800a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): picked mon.noname-c con 0x7f6d800a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6d8009ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): picked mon.noname-b con 0x7f6d800a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d800a3130 con 0x7f6d8009ffc0 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6d80073df0 con 0x7f6d800a0890 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d800a1bf0 con 0x7f6d800a11b0 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d88345640 10 monclient(hunting): authenticate will time out at 1008.530982s 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d860e1640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f6d800a0890 msgr2=0x7f6d800a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d860e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d800a0890 0x7f6d800a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d858e0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6d8009ffc0 0x7f6d80135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56260/0 (socket says 172.21.15.17:56260) 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d858e0640 1 -- 172.21.15.17:0/3119979495 learned_addr learned my addr 172.21.15.17:0/3119979495 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): get_auth_request con 0x7f6d800a11b0 auth_method 0 2024-09-15T21:18:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient(hunting): handle_auth_done global_id 4376 payload 274 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 1 -- 172.21.15.17:0/3119979495 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a0890 msgr2=0x7f6d800a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 1 --2- 172.21.15.17:0/3119979495 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a0890 0x7f6d800a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 1 -- 172.21.15.17:0/3119979495 >> v1:172.21.15.17:6789/0 conn(0x7f6d8009ffc0 legacy=0x7f6d80135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient: _finish_hunting 0 2024-09-15T21:18:05.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 1 monclient: found mon.noname-b 2024-09-15T21:18:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d850df640 1 -- 172.21.15.17:0/3119979495 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381617844 0 0) 0x7f6d800a3130 con 0x7f6d8009ffc0 2024-09-15T21:18:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d868e2640 1 -- 172.21.15.17:0/3119979495 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d800a15e0 con 0x7f6d800a11b0 2024-09-15T21:18:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.130+0000 7f6d850df640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:05.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d868e2640 1 --2- 172.21.15.17:0/3119979495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80135d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009dc0 tx=0x7f6d7801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e860dae8bc1f98e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:05.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 1 -- 172.21.15.17:0/3119979495 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780b2070 con 0x7f6d800a11b0 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: dump: 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:05.152 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: _finish_auth 0 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: _check_auth_tickets 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 5 monclient: authenticate success, global_id 4376 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 1 -- 172.21.15.17:0/3119979495 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d780a5800 con 0x7f6d800a11b0 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 1 -- 172.21.15.17:0/3119979495 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780a5bd0 con 0x7f6d800a11b0 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:05.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d850df640 10 monclient: dump: 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 4 set_mon_vals no callback set 2024-09-15T21:18:05.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 10 monclient: shutdown 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 -- 172.21.15.17:0/3119979495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 msgr2=0x7f6d80135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 --2- 172.21.15.17:0/3119979495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80135d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009dc0 tx=0x7f6d7801c830 comp rx=0 tx=0).stop 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 -- 172.21.15.17:0/3119979495 shutdown_connections 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 --2- 172.21.15.17:0/3119979495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80135d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 --2- 172.21.15.17:0/3119979495 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a0890 0x7f6d800a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 -- 172.21.15.17:0/3119979495 >> 172.21.15.17:0/3119979495 conn(0x7f6d800a66a0 msgr2=0x7f6d800a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 -- 172.21.15.17:0/3119979495 shutdown_connections 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 -- 172.21.15.17:0/3119979495 wait complete. 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 10 monclient: build_initial_monmap 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 10 monclient: monmap: 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:05.155 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 Processor -- start 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 1 -- start start 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.134+0000 7f6d88345640 10 monclient: init 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:05.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8009ffc0 0x7f6d80136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): picked mon.noname-a con 0x7f6d8009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a0890 0x7f6d80136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): picked mon.noname-b con 0x7f6d800a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80137520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): picked mon.noname-c con 0x7f6d800a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80140510 con 0x7f6d8009ffc0 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d801364d0 con 0x7f6d800a0890 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d80140690 con 0x7f6d800a11b0 2024-09-15T21:18:05.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): authenticate will time out at 1008.536304s 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d858e0640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a0890 msgr2=0x7f6d80136fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d858e0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a0890 0x7f6d80136fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80137520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57862/0 (socket says 172.21.15.17:57862) 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 -- 172.21.15.17:0/2635589993 learned_addr learned my addr 172.21.15.17:0/2635589993 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): get_auth_request con 0x7f6d800a11b0 auth_method 0 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8009ffc0 0x7f6d80136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): get_auth_request con 0x7f6d8009ffc0 auth_method 0 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient(hunting): handle_auth_done global_id 4379 payload 824 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a0890 msgr2=0x7f6d80136fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a0890 0x7f6d80136fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8009ffc0 msgr2=0x7f6d80136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8009ffc0 0x7f6d80136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient: _finish_hunting 0 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 monclient: found mon.noname-c 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d801390f0 con 0x7f6d800a11b0 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d868e2640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80137520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d780b0040 tx=0x7f6d780195d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780bd030 con 0x7f6d800a11b0 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: dump: 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d780b8530 con 0x7f6d800a11b0 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:05.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780b2040 con 0x7f6d800a11b0 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: dump: 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 5 monclient: authenticate success, global_id 4379 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient: _renew_subs 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d848de640 4 set_mon_vals no callback set 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d801393d0 con 0x7f6d800a11b0 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient: _renew_subs 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d80137bd0 con 0x7f6d800a11b0 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 0x7f6d8012e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8013c300 con 0x7f6d80138cc0 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 0x7f6d8012e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:05.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient: get_auth_request con 0x7f6d80138cc0 auth_method 0 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6d780c0070 con 0x7f6d800a11b0 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 10 monclient(hunting): handle_auth_done global_id 4382 payload 824 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d800a0c70 con 0x7f6d80138cc0 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b970 0x7f6d6403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f6d780ea570 con 0x7f6d800a11b0 2024-09-15T21:18:05.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d860e1640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 0x7f6d8012e600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003fd0 tx=0x7f6d7c009a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c016070 con 0x7f6d80138cc0 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: dump: 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:05.165 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d858e0640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b970 0x7f6d6403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d858e0640 10 monclient: get_auth_request con 0x7f6d6403b970 auth_method 0 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6d7c00e680 con 0x7f6d80138cc0 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 msgr2=0x7f6d8012e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 0x7f6d8012e600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003fd0 tx=0x7f6d7c009a40 comp rx=0 tx=0).stop 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 msgr2=0x7f6d8012e600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d76ffd640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d80138cc0 0x7f6d8012e600 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003fd0 tx=0x7f6d7c009a40 comp rx=0 tx=0).stop 2024-09-15T21:18:05.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.138+0000 7f6d858e0640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b970 0x7f6d6403de30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d700010f0 tx=0x7f6d70007a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 0x7f6d8012ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8012f490 con 0x7f6d8012eb40 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 0x7f6d8012ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient: get_auth_request con 0x7f6d8012eb40 auth_method 0 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:05.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 10 monclient(hunting): handle_auth_done global_id 4385 payload 824 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 1 -- 172.21.15.17:0/2635589993 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6d8006d020 con 0x7f6d8012eb40 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d860e1640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 0x7f6d8012ef50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003fd0 tx=0x7f6d7c0059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c016070 con 0x7f6d8012eb40 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 10 monclient: dump: 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.169 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4700 (secure 0 0 0) 0x7f6d7c006380 con 0x7f6d8012eb40 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 msgr2=0x7f6d8012ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 0x7f6d8012ef50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003fd0 tx=0x7f6d7c0059e0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 msgr2=0x7f6d8012ef50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:05.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d76ffd640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8012eb40 0x7f6d8012ef50 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003fd0 tx=0x7f6d7c0059e0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 12, 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 50767, 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:05.183 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b970 msgr2=0x7f6d6403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b970 0x7f6d6403de30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d700010f0 tx=0x7f6d70007a90 comp rx=0 tx=0).stop 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 10 monclient: shutdown 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 msgr2=0x7f6d80137520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 --2- 172.21.15.17:0/2635589993 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a11b0 0x7f6d80137520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d780b0040 tx=0x7f6d780195d0 comp rx=0 tx=0).stop 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d868e2640 1 -- 172.21.15.17:0/2635589993 reap_dead start 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d868e2640 1 -- 172.21.15.17:0/2635589993 reap_dead start 2024-09-15T21:18:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 shutdown_connections 2024-09-15T21:18:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 >> 172.21.15.17:0/2635589993 conn(0x7f6d800a66a0 msgr2=0x7f6d8006fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 shutdown_connections 2024-09-15T21:18:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:05.146+0000 7f6d88345640 1 -- 172.21.15.17:0/2635589993 wait complete. 2024-09-15T21:18:05.186 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:18:05.186 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:18:25.186 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:18:25.187 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:18:25.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:18:25.190 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:18:25.190 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:18:25.190 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient: get_monmap_and_config 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient: build_initial_monmap 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient: monmap: 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:25.323457+0000 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:25.323457+0000 2024-09-15T21:18:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:25.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 1 Processor -- start 2024-09-15T21:18:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 1 -- start start 2024-09-15T21:18:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient: init 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7febe80a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient(hunting): picked mon.noname-b con 0x7febe809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7febe80a1380 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe80a3130 con 0x7febe80a1380 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe8073df0 con 0x7febe80a0890 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febe80a1bf0 con 0x7febe809ffc0 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febf0119640 10 monclient(hunting): authenticate will time out at 1028.718923s 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febee6b6640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7febe80a1380 0x7febe813d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56244/0 (socket says 172.21.15.17:56244) 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febee6b6640 1 -- 172.21.15.17:0/2903307440 learned_addr learned my addr 172.21.15.17:0/2903307440 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): get_auth_request con 0x7febe809ffc0 auth_method 0 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient(hunting): handle_auth_done global_id 4388 payload 274 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 1 -- 172.21.15.17:0/2903307440 >> v1:172.21.15.119:6789/0 conn(0x7febe80a0890 legacy=0x7febe80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 1 -- 172.21.15.17:0/2903307440 >> v1:172.21.15.17:6789/0 conn(0x7febe80a1380 legacy=0x7febe813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febeceb3640 1 -- 172.21.15.17:0/2903307440 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985553468 0 0) 0x7febe80a3130 con 0x7febe80a1380 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient: _finish_hunting 0 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 1 monclient: found mon.noname-b 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febed6b4640 1 -- 172.21.15.17:0/2903307440 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe809ccd0 con 0x7febe809ffc0 2024-09-15T21:18:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.318+0000 7febeceb3640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febed6b4640 1 --2- 172.21.15.17:0/2903307440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe813aac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febd80097f0 tx=0x7febd801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a1909255b18d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 1 -- 172.21.15.17:0/2903307440 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80b0070 con 0x7febe809ffc0 2024-09-15T21:18:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: dump: 2024-09-15T21:18:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: _finish_auth 0 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: _check_auth_tickets 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 5 monclient: authenticate success, global_id 4388 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 1 -- 172.21.15.17:0/2903307440 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd80ab600 con 0x7febe809ffc0 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 4 set_mon_vals no callback set 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 1 -- 172.21.15.17:0/2903307440 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80ab9d0 con 0x7febe809ffc0 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febeceb3640 10 monclient: dump: 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: shutdown 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- 172.21.15.17:0/2903307440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 msgr2=0x7febe813aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 --2- 172.21.15.17:0/2903307440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe813aac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febd80097f0 tx=0x7febd801c3a0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- 172.21.15.17:0/2903307440 shutdown_connections 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 --2- 172.21.15.17:0/2903307440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe813aac0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- 172.21.15.17:0/2903307440 >> 172.21.15.17:0/2903307440 conn(0x7febe80a66a0 msgr2=0x7febe80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- 172.21.15.17:0/2903307440 shutdown_connections 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- 172.21.15.17:0/2903307440 wait complete. 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: build_initial_monmap 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: monmap: 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 Processor -- start 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- start start 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: init 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): picked mon.noname-c con 0x7febe809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 0x7febe809d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): picked mon.noname-a con 0x7febe80a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 0x7febe809de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): picked mon.noname-b con 0x7febe80a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7febe81412e0 con 0x7febe80a0890 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febe8141160 con 0x7febe80a1380 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febe8141460 con 0x7febe809ffc0 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.322+0000 7febf0119640 10 monclient(hunting): authenticate will time out at 1028.723914s 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 0x7febe809d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 0x7febe809d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50882/0 (socket says 172.21.15.17:50882) 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 -- 172.21.15.17:0/372315633 learned_addr learned my addr 172.21.15.17:0/372315633 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): get_auth_request con 0x7febe80a0890 auth_method 0 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe809d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): get_auth_request con 0x7febe809ffc0 auth_method 0 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 0x7febe809de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 10 monclient(hunting): get_auth_request con 0x7febe80a1380 auth_method 0 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 1 -- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 msgr2=0x7febe809de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 1 -- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 msgr2=0x7febe809de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 0x7febe809de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febee6b6640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 0x7febe809de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient(hunting): handle_auth_done global_id 4431 payload 824 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 -- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 msgr2=0x7febe809d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe809d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 -- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 msgr2=0x7febe809de80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 0x7febe809de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient: _finish_hunting 0 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 monclient: found mon.noname-a 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 -- 172.21.15.17:0/372315633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe8120d30 con 0x7febe80a0890 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febed6b4640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 0x7febe809d940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febd8018fb0 tx=0x7febd8019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80b0040 con 0x7febe80a0890 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: dump: 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd80b9070 con 0x7febe80a0890 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 5 monclient: authenticate success, global_id 4431 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 monclient: _renew_subs 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe8121010 con 0x7febe80a0890 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80aac30 con 0x7febe80a0890 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 10 monclient: dump: 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 monclient: _renew_subs 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febdf7fe640 4 set_mon_vals no callback set 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe813d820 con 0x7febe80a0890 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7febd80be460 con 0x7febe80a0890 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe80a0c70 con 0x7febe80a0890 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febcc03b9c0 0x7febcc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7febd80e9b60 con 0x7febe80a0890 2024-09-15T21:18:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febcc03b9c0 0x7febcc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 10 monclient: get_auth_request con 0x7febcc03b9c0 auth_method 0 2024-09-15T21:18:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.326+0000 7febedeb5640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febcc03b9c0 0x7febcc03de80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febe4007fd0 tx=0x7febe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.330+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7febd80160b0 con 0x7febe80a0890 2024-09-15T21:18:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.330+0000 7febde7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.330+0000 7febde7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febf0119640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febf0119640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febf0119640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7febe809d0f0 con 0x7febe80a0890 2024-09-15T21:18:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febde7fc640 1 -- 172.21.15.17:0/372315633 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7febd80aadd0 con 0x7febe80a0890 2024-09-15T21:18:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febde7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.570+0000 7febde7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:25.580 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febcc03b9c0 msgr2=0x7febcc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febcc03b9c0 0x7febcc03de80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febe4007fd0 tx=0x7febe4009040 comp rx=0 tx=0).stop 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 10 monclient: shutdown 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 msgr2=0x7febe809d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 0x7febe809d940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febd8018fb0 tx=0x7febd8019140 comp rx=0 tx=0).stop 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 shutdown_connections 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febcc03b9c0 0x7febcc03de80 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febe4007fd0 tx=0x7febe4009040 comp rx=0 tx=0).stop 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe80a1380 0x7febe809de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe80a0890 0x7febe809d940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 --2- 172.21.15.17:0/372315633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febe809ffc0 0x7febe809d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 >> 172.21.15.17:0/372315633 conn(0x7febe80a66a0 msgr2=0x7febe80a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 shutdown_connections 2024-09-15T21:18:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.574+0000 7febf0119640 1 -- 172.21.15.17:0/372315633 wait complete. 2024-09-15T21:18:25.600 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:18:25.600 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:18:25.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:18:25.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient: get_monmap_and_config 2024-09-15T21:18:25.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient: build_initial_monmap 2024-09-15T21:18:25.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient: monmap: 2024-09-15T21:18:25.721 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:25.718392+0000 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:25.718392+0000 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:25.722 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:25.723 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 Processor -- start 2024-09-15T21:18:25.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 -- start start 2024-09-15T21:18:25.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient: init 2024-09-15T21:18:25.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:25.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 0x7f17800a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): picked mon.noname-c con 0x7f17800a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17800a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): picked mon.noname-b con 0x7f17800a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1780137850 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17800a2c30 con 0x7f1780137850 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f17800a0f00 con 0x7f17800a4a90 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f17800a19b0 con 0x7f17800a53b0 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:25.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f1785d18640 10 monclient(hunting): authenticate will time out at 1029.113762s 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1780137850 0x7f1780137cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56258/0 (socket says 172.21.15.17:56258) 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17800a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177ffff640 1 -- 172.21.15.17:0/3061497375 learned_addr learned my addr 172.21.15.17:0/3061497375 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): get_auth_request con 0x7f17800a53b0 auth_method 0 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 1 --2- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 0x7f17800a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 10 monclient(hunting): get_auth_request con 0x7f17800a4a90 auth_method 0 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 1 -- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 msgr2=0x7f17800a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 1 -- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 msgr2=0x7f17800a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 1 --2- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 0x7f17800a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient(hunting): handle_auth_done global_id 4394 payload 274 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177f7fe640 1 --2- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 0x7f17800a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 1 -- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 msgr2=0x7f17800a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 1 --2- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 0x7f17800a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 1 -- 172.21.15.17:0/3061497375 >> v1:172.21.15.17:6789/0 conn(0x7f1780137850 legacy=0x7f1780137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 1 monclient: found mon.noname-b 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177effd640 1 -- 172.21.15.17:0/3061497375 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17800a1c90 con 0x7f17800a53b0 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177e7fc640 1 -- 172.21.15.17:0/3061497375 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726196418 0 0) 0x7f17780014a0 con 0x7f1780137850 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.714+0000 7f177e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 --2- 172.21.15.17:0/3061497375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17800a57d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17700097c0 tx=0x7f177001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83cfa13fd98a6d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 1 -- 172.21.15.17:0/3061497375 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700b0070 con 0x7f17800a53b0 2024-09-15T21:18:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: dump: 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.730 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 1 -- 172.21.15.17:0/3061497375 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17700ab600 con 0x7f17800a53b0 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 5 monclient: authenticate success, global_id 4394 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 4 set_mon_vals no callback set 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 1 -- 172.21.15.17:0/3061497375 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700ab9d0 con 0x7f17800a53b0 2024-09-15T21:18:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177e7fc640 10 monclient: dump: 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.732 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: shutdown 2024-09-15T21:18:25.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- 172.21.15.17:0/3061497375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 msgr2=0x7f17800a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 --2- 172.21.15.17:0/3061497375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17800a57d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17700097c0 tx=0x7f177001c330 comp rx=0 tx=0).stop 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- 172.21.15.17:0/3061497375 shutdown_connections 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 --2- 172.21.15.17:0/3061497375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17800a57d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 --2- 172.21.15.17:0/3061497375 >> v2:172.21.15.119:3300/0 conn(0x7f17800a4a90 0x7f17800a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- 172.21.15.17:0/3061497375 >> 172.21.15.17:0/3061497375 conn(0x7f17800aabb0 msgr2=0x7f17800a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- 172.21.15.17:0/3061497375 shutdown_connections 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- 172.21.15.17:0/3061497375 wait complete. 2024-09-15T21:18:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: build_initial_monmap 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: monmap: 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 Processor -- start 2024-09-15T21:18:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- start start 2024-09-15T21:18:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: init 2024-09-15T21:18:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:18:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 0x7f17801cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): picked mon.noname-b con 0x7f17800a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:25.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17801368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): picked mon.noname-c con 0x7f17800a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:25.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780137850 0x7f178013ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): picked mon.noname-a con 0x7f1780137850 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:25.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f17801426d0 con 0x7f1780137850 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1780142550 con 0x7f17800a4a90 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1780142850 con 0x7f17800a53b0 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1785d18640 10 monclient(hunting): authenticate will time out at 1029.119079s 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17801368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780137850 0x7f178013ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 0x7f17801cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17801368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58192/0 (socket says 172.21.15.17:58192) 2024-09-15T21:18:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 -- 172.21.15.17:0/2891454263 learned_addr learned my addr 172.21.15.17:0/2891454263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): get_auth_request con 0x7f17800a53b0 auth_method 0 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 10 monclient(hunting): get_auth_request con 0x7f17800a4a90 auth_method 0 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): get_auth_request con 0x7f1780137850 auth_method 0 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 msgr2=0x7f17801cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 msgr2=0x7f17801cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 0x7f17801cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177f7fe640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 0x7f17801cfa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient(hunting): handle_auth_done global_id 4397 payload 824 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 msgr2=0x7f17801cfa20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17800a4a90 0x7f17801cfa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780137850 msgr2=0x7f178013ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780137850 0x7f178013ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 monclient: found mon.noname-c 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1780121240 con 0x7f17800a53b0 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f177effd640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17801368b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1770019f60 tx=0x7f17700190d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.718+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700b8020 con 0x7f17800a53b0 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: dump: 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17700b4410 con 0x7f17800a53b0 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 5 monclient: authenticate success, global_id 4397 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 10 monclient: _renew_subs 2024-09-15T21:18:25.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17800a06a0 con 0x7f17800a53b0 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700b0040 con 0x7f17800a53b0 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: dump: 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 10 monclient: _renew_subs 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177dffb640 4 set_mon_vals no callback set 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f17801206e0 con 0x7f17800a53b0 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 0x7f178013cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f17800a4e70 con 0x7f1780136df0 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 0x7f178013cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient: get_auth_request con 0x7f1780136df0 auth_method 0 2024-09-15T21:18:25.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f17700bb020 con 0x7f17800a53b0 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 10 monclient(hunting): handle_auth_done global_id 4440 payload 824 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17800a84c0 con 0x7f1780136df0 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f176003ba20 0x7f176003dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f17700e94e0 con 0x7f17800a53b0 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177f7fe640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 0x7f178013cfc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1768000ee0 tx=0x7f1768010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177ffff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f176003ba20 0x7f176003dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768019020 con 0x7f1780136df0 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: dump: 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.753 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177ffff640 10 monclient: get_auth_request con 0x7f176003ba20 auth_method 0 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f176800e9c0 con 0x7f1780136df0 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 msgr2=0x7f178013cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 0x7f178013cfc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1768000ee0 tx=0x7f1768010b70 comp rx=0 tx=0).stop 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 msgr2=0x7f178013cfc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f1777fff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1780136df0 0x7f178013cfc0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1768000ee0 tx=0x7f1768010b70 comp rx=0 tx=0).stop 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.722+0000 7f177ffff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f176003ba20 0x7f176003dee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1778004fd0 tx=0x7f1778001870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 0x7f1780138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1780120a20 con 0x7f178013d500 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 0x7f1780138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient: get_auth_request con 0x7f178013d500 auth_method 0 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 10 monclient(hunting): handle_auth_done global_id 4443 payload 824 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 1 -- 172.21.15.17:0/2891454263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1780120be0 con 0x7f178013d500 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177f7fe640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 0x7f1780138cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1768000ee0 tx=0x7f17680046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768019040 con 0x7f178013d500 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 10 monclient: dump: 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7f1768020220 con 0x7f178013d500 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 msgr2=0x7f1780138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 0x7f1780138cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1768000ee0 tx=0x7f17680046d0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 msgr2=0x7f1780138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1777fff640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178013d500 0x7f1780138cf0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1768000ee0 tx=0x7f17680046d0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 71351, 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:25.772 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f176003ba20 msgr2=0x7f176003dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f176003ba20 0x7f176003dee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1778004fd0 tx=0x7f1778001870 comp rx=0 tx=0).stop 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 10 monclient: shutdown 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 msgr2=0x7f17801368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 --2- 172.21.15.17:0/2891454263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17800a53b0 0x7f17801368b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1770019f60 tx=0x7f17700190d0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177ffff640 1 -- 172.21.15.17:0/2891454263 reap_dead start 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f177ffff640 1 -- 172.21.15.17:0/2891454263 reap_dead start 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 shutdown_connections 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 >> 172.21.15.17:0/2891454263 conn(0x7f17800aabb0 msgr2=0x7f17800a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 shutdown_connections 2024-09-15T21:18:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.730+0000 7f1785d18640 1 -- 172.21.15.17:0/2891454263 wait complete. 2024-09-15T21:18:25.775 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:18:25.775 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:18:25.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:18:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.882+0000 7f0618ac2640 10 monclient: get_monmap_and_config 2024-09-15T21:18:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.882+0000 7f0618ac2640 10 monclient: build_initial_monmap 2024-09-15T21:18:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.882+0000 7f0618ac2640 10 monclient: monmap: 2024-09-15T21:18:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:25.889902+0000 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:25.889902+0000 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:25.893 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 Processor -- start 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 -- start start 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient: init 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 0x7f061409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): picked mon.noname-c con 0x7f061409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 0x7f0614134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): picked mon.noname-b con 0x7f061409dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0614134ba0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06140a3490 con 0x7f0614134ba0 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f06140a1bc0 con 0x7f061409d400 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f06140a0320 con 0x7f061409dd20 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0618ac2640 10 monclient(hunting): authenticate will time out at 1029.285271s 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 0x7f0614134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 0x7f0614134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58194/0 (socket says 172.21.15.17:58194) 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 -- 172.21.15.17:0/4150300811 learned_addr learned my addr 172.21.15.17:0/4150300811 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): get_auth_request con 0x7f061409dd20 auth_method 0 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 1 --2- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 0x7f061409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 10 monclient(hunting): get_auth_request con 0x7f061409d400 auth_method 0 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 1 -- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 msgr2=0x7f061409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 1 -- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 msgr2=0x7f061409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 1 --2- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 0x7f061409d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient(hunting): handle_auth_done global_id 4400 payload 274 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f061259c640 1 --2- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 0x7f061409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 -- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 msgr2=0x7f061409d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 --2- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 0x7f061409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 -- 172.21.15.17:0/4150300811 >> v1:172.21.15.119:6789/0 conn(0x7f0614134ba0 legacy=0x7f061413b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 monclient: found mon.noname-b 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.886+0000 7f0611d9b640 1 -- 172.21.15.17:0/4150300811 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061409e7a0 con 0x7f061409dd20 2024-09-15T21:18:25.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0611d9b640 1 --2- 172.21.15.17:0/4150300811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 0x7f0614134660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0608009870 tx=0x7f060801ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1103460097c4d0bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 1 -- 172.21.15.17:0/4150300811 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080ab610 con 0x7f061409dd20 2024-09-15T21:18:25.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:25.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: dump: 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: _finish_auth 0 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: _check_auth_tickets 2024-09-15T21:18:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 1 -- 172.21.15.17:0/4150300811 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06080b0070 con 0x7f061409dd20 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 1 -- 172.21.15.17:0/4150300811 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080ab9d0 con 0x7f061409dd20 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 5 monclient: authenticate success, global_id 4400 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 4 set_mon_vals no callback set 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f061159a640 10 monclient: dump: 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 10 monclient: shutdown 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 -- 172.21.15.17:0/4150300811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 msgr2=0x7f0614134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 --2- 172.21.15.17:0/4150300811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 0x7f0614134660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0608009870 tx=0x7f060801ce50 comp rx=0 tx=0).stop 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 -- 172.21.15.17:0/4150300811 shutdown_connections 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 --2- 172.21.15.17:0/4150300811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f061409dd20 0x7f0614134660 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 --2- 172.21.15.17:0/4150300811 >> v2:172.21.15.119:3300/0 conn(0x7f061409d400 0x7f061409d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 -- 172.21.15.17:0/4150300811 >> 172.21.15.17:0/4150300811 conn(0x7f06140a66a0 msgr2=0x7f06140a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 -- 172.21.15.17:0/4150300811 shutdown_connections 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 -- 172.21.15.17:0/4150300811 wait complete. 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 10 monclient: build_initial_monmap 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 10 monclient: monmap: 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 Processor -- start 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 1 -- start start 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.890+0000 7f0618ac2640 10 monclient: init 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061409d400 0x7f061411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): picked mon.noname-a con 0x7f061409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061409dd20 0x7f061411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): picked mon.noname-b con 0x7f061409dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0614134ba0 0x7f061411cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): picked mon.noname-c con 0x7f0614134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f06140a3130 con 0x7f061409d400 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f061409e130 con 0x7f061409dd20 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f06140a32b0 con 0x7f0614134ba0 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): authenticate will time out at 1029.292187s 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061409d400 0x7f061411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0614134ba0 0x7f061411cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0614134ba0 0x7f061411cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58202/0 (socket says 172.21.15.17:58202) 2024-09-15T21:18:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 1 -- 172.21.15.17:0/2604308082 learned_addr learned my addr 172.21.15.17:0/2604308082 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): get_auth_request con 0x7f0614134ba0 auth_method 0 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061409d400 0x7f061411c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50924/0 (socket says 172.21.15.17:50924) 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061409dd20 0x7f061411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): get_auth_request con 0x7f061409d400 auth_method 0 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): get_auth_request con 0x7f061409dd20 auth_method 0 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient(hunting): handle_auth_done global_id 4446 payload 824 2024-09-15T21:18:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0614134ba0 msgr2=0x7f061411cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0614134ba0 0x7f061411cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061409dd20 msgr2=0x7f061411ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061409dd20 0x7f061411ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient: _finish_hunting 0 2024-09-15T21:18:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 monclient: found mon.noname-a 2024-09-15T21:18:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061411e1e0 con 0x7f061409d400 2024-09-15T21:18:25.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0614134ba0 0x7f061411cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:25.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f061259c640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061409d400 0x7f061411c4e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f060400e7e0 tx=0x7f060400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604017030 con 0x7f061409d400 2024-09-15T21:18:25.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:25.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: dump: 2024-09-15T21:18:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.920 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 5 monclient: authenticate success, global_id 4446 2024-09-15T21:18:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f060400edd0 con 0x7f061409d400 2024-09-15T21:18:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060400f040 con 0x7f061409d400 2024-09-15T21:18:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient: _renew_subs 2024-09-15T21:18:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061411ed70 con 0x7f061409d400 2024-09-15T21:18:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0603fff640 4 set_mon_vals no callback set 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: dump: 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.924 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.924 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.924 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient: _renew_subs 2024-09-15T21:18:25.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:25.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06140a4250 con 0x7f061409d400 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 0x7f06141d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f060401a070 con 0x7f061409d400 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f061409d7e0 con 0x7f06141cffa0 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 0x7f06141d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient: get_auth_request con 0x7f06141cffa0 auth_method 0 2024-09-15T21:18:25.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05f403b9c0 0x7f05f403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f060404f340 con 0x7f061409d400 2024-09-15T21:18:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 10 monclient(hunting): handle_auth_done global_id 4406 payload 824 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06140a1660 con 0x7f06141cffa0 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05f403b9c0 0x7f05f403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 10 monclient: get_auth_request con 0x7f05f403b9c0 auth_method 0 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0611d9b640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 0x7f06141d0380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06140a1e00 tx=0x7f0608019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080abc80 con 0x7f06141cffa0 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: dump: 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.928 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.929 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.929 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0612d9d640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05f403b9c0 0x7f05f403de80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f061411d790 tx=0x7f060c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f06080b0040 con 0x7f06141cffa0 2024-09-15T21:18:25.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:25.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:25.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 msgr2=0x7f06141d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 0x7f06141d0380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06140a1e00 tx=0x7f0608019190 comp rx=0 tx=0).stop 2024-09-15T21:18:25.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 msgr2=0x7f06141d0380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:25.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.894+0000 7f0602ffd640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141cffa0 0x7f06141d0380 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06140a1e00 tx=0x7f0608019190 comp rx=0 tx=0).stop 2024-09-15T21:18:25.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:25.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 0x7f0614137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:25.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:25.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f061406d020 con 0x7f06141d08c0 2024-09-15T21:18:25.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 0x7f0614137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient: get_auth_request con 0x7f06141d08c0 auth_method 0 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 10 monclient(hunting): handle_auth_done global_id 4409 payload 824 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 1 -- 172.21.15.17:0/2604308082 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f06141d0ca0 con 0x7f06141d08c0 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0611d9b640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 0x7f0614137680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f060801a020 tx=0x7f0608009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080c5870 con 0x7f06141d08c0 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:25.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 10 monclient: dump: 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f06080c5870 con 0x7f06141d08c0 2024-09-15T21:18:25.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 msgr2=0x7f0614137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 0x7f0614137680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f060801a020 tx=0x7f0608009240 comp rx=0 tx=0).stop 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 msgr2=0x7f0614137680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0602ffd640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06141d08c0 0x7f0614137680 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f060801a020 tx=0x7f0608009240 comp rx=0 tx=0).stop 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:25.935 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:25.936 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:25.936 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:18:25.936 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-15T21:18:25.936 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 71523, 2024-09-15T21:18:25.936 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:25.936 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:25.937 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:18:25.937 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:25.937 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.937 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:18:25.937 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:25.937 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:25.938 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:25.939 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:25.939 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:25.939 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:25.939 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:25.939 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:25.939 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:25.940 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:25.941 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:25.942 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:25.943 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:25.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:25.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:25.945 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:25.946 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:25.946 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.946 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:25.948 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:25.948 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:25.948 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:25.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.948 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:25.948 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:25.949 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.949 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:25.949 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:25.950 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.950 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.950 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:25.950 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:25.951 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:25.951 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:25.951 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:25.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.951 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:25.952 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:25.952 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:25.952 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:25.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:25.952 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:25.953 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:25.954 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:18:25.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:25.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:25.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:25.956 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:25.956 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:25.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05f403b9c0 msgr2=0x7f05f403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05f403b9c0 0x7f05f403de80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f061411d790 tx=0x7f060c009040 comp rx=0 tx=0).stop 2024-09-15T21:18:25.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 10 monclient: shutdown 2024-09-15T21:18:25.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061409d400 msgr2=0x7f061411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 --2- 172.21.15.17:0/2604308082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061409d400 0x7f061411c4e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f060400e7e0 tx=0x7f060400eca0 comp rx=0 tx=0).stop 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0612d9d640 1 -- 172.21.15.17:0/2604308082 reap_dead start 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0612d9d640 1 -- 172.21.15.17:0/2604308082 reap_dead start 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 shutdown_connections 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 >> 172.21.15.17:0/2604308082 conn(0x7f06140a66a0 msgr2=0x7f06140a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 shutdown_connections 2024-09-15T21:18:25.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:25.902+0000 7f0618ac2640 1 -- 172.21.15.17:0/2604308082 wait complete. 2024-09-15T21:18:25.958 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:18:25.958 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:18:25.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:18:26.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient: get_monmap_and_config 2024-09-15T21:18:26.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient: build_initial_monmap 2024-09-15T21:18:26.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient: monmap: 2024-09-15T21:18:26.082 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:26.082 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.082 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:26.079016+0000 2024-09-15T21:18:26.082 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:26.079016+0000 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:26.083 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 Processor -- start 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 -- start start 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient: init 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e000a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): picked mon.noname-b con 0x7f5e000a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e0009ffc0 0x7f5e000a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): picked mon.noname-a con 0x7f5e0009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e00135670 0x7f5e00135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): picked mon.noname-c con 0x7f5e00135670 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5e000a16f0 con 0x7f5e0009ffc0 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e000a3130 con 0x7f5e00135670 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e00073df0 con 0x7f5e000a0890 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e07478640 10 monclient(hunting): authenticate will time out at 1029.474922s 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e0009ffc0 0x7f5e000a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e0009ffc0 0x7f5e000a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50938/0 (socket says 172.21.15.17:50938) 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 -- 172.21.15.17:0/1521888263 learned_addr learned my addr 172.21.15.17:0/1521888263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): get_auth_request con 0x7f5e0009ffc0 auth_method 0 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 1 --2- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e000a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 1 --2- 172.21.15.17:0/1521888263 >> v2:172.21.15.119:3300/0 conn(0x7f5e00135670 0x7f5e00135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): get_auth_request con 0x7f5e00135670 auth_method 0 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): get_auth_request con 0x7f5e000a0890 auth_method 0 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05a15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient(hunting): handle_auth_done global_id 4449 payload 274 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 -- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 msgr2=0x7f5e000a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:26.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 --2- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e000a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 -- 172.21.15.17:0/1521888263 >> v2:172.21.15.119:3300/0 conn(0x7f5e00135670 msgr2=0x7f5e00135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 --2- 172.21.15.17:0/1521888263 >> v2:172.21.15.119:3300/0 conn(0x7f5e00135670 0x7f5e00135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient: _finish_hunting 0 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 monclient: found mon.noname-a 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.074+0000 7f5e04a13640 1 -- 172.21.15.17:0/1521888263 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e000a1bf0 con 0x7f5e0009ffc0 2024-09-15T21:18:26.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e04a13640 1 --2- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e0009ffc0 0x7f5e000a11b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006f1c0 tx=0x7f5df001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98968ce3259f8c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 1 -- 172.21.15.17:0/1521888263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00b0070 con 0x7f5e0009ffc0 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: dump: 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.092 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 1 -- 172.21.15.17:0/1521888263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df00ab600 con 0x7f5e0009ffc0 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 5 monclient: authenticate success, global_id 4449 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 1 -- 172.21.15.17:0/1521888263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00ab9d0 con 0x7f5e0009ffc0 2024-09-15T21:18:26.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 4 set_mon_vals no callback set 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5df7fff640 10 monclient: dump: 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: shutdown 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 -- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e0009ffc0 msgr2=0x7f5e000a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 --2- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e0009ffc0 0x7f5e000a11b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006f1c0 tx=0x7f5df001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 -- 172.21.15.17:0/1521888263 shutdown_connections 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 --2- 172.21.15.17:0/1521888263 >> v2:172.21.15.119:3300/0 conn(0x7f5e00135670 0x7f5e00135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 --2- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e0009ffc0 0x7f5e000a11b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 --2- 172.21.15.17:0/1521888263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e000a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 -- 172.21.15.17:0/1521888263 >> 172.21.15.17:0/1521888263 conn(0x7f5e000a66a0 msgr2=0x7f5e000a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 -- 172.21.15.17:0/1521888263 shutdown_connections 2024-09-15T21:18:26.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 -- 172.21.15.17:0/1521888263 wait complete. 2024-09-15T21:18:26.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: build_initial_monmap 2024-09-15T21:18:26.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: monmap: 2024-09-15T21:18:26.106 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:26.106 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:26.106 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:26.106 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 Processor -- start 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 -- start start 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: init 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.078+0000 7f5e07478640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e0009ffc0 0x7f5e00134810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): picked mon.noname-b con 0x7f5e0009ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:26.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e00134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): picked mon.noname-c con 0x7f5e000a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e00135670 0x7f5e0009d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): picked mon.noname-a con 0x7f5e00135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0013f680 con 0x7f5e00135670 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0013f500 con 0x7f5e0009ffc0 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0013f800 con 0x7f5e000a0890 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): authenticate will time out at 1029.480056s 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e00135670 0x7f5e0009d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e00134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e00135670 0x7f5e0009d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50944/0 (socket says 172.21.15.17:50944) 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 1 -- 172.21.15.17:0/712774322 learned_addr learned my addr 172.21.15.17:0/712774322 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): get_auth_request con 0x7f5e00135670 auth_method 0 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e00134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58252/0 (socket says 172.21.15.17:58252) 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e0009ffc0 0x7f5e00134810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): get_auth_request con 0x7f5e000a0890 auth_method 0 2024-09-15T21:18:26.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): get_auth_request con 0x7f5e0009ffc0 auth_method 0 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient(hunting): handle_auth_done global_id 4415 payload 824 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e0009ffc0 msgr2=0x7f5e00134810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e0009ffc0 0x7f5e00134810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e00135670 msgr2=0x7f5e0009d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e00135670 0x7f5e0009d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient: _finish_hunting 0 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 monclient: found mon.noname-c 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e00072f50 con 0x7f5e000a0890 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e00135670 0x7f5e0009d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e04a13640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e00134d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e0009f7c0 tx=0x7f5df0019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00b0040 con 0x7f5e000a0890 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: dump: 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:26.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df00b4420 con 0x7f5e000a0890 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 5 monclient: authenticate success, global_id 4415 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient: _renew_subs 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0009d0f0 con 0x7f5e000a0890 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00b47f0 con 0x7f5e000a0890 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: dump: 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.114 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient: _renew_subs 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df6ffd640 4 set_mon_vals no callback set 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001221d0 con 0x7f5e000a0890 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 0x7f5e0009d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00135aa0 con 0x7f5e001208f0 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 0x7f5e0009d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient: get_auth_request con 0x7f5e001208f0 auth_method 0 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5df00b4990 con 0x7f5e000a0890 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5de403b9c0 0x7f5de403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 10 monclient(hunting): handle_auth_done global_id 14107 payload 824 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e001219f0 con 0x7f5e001208f0 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f5df00e94a0 con 0x7f5e000a0890 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5de403b9c0 0x7f5de403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 10 monclient: get_auth_request con 0x7f5de403b9c0 auth_method 0 2024-09-15T21:18:26.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05214640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 0x7f5e0009d940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0119a0 tx=0x7f5dfc011e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc019070 con 0x7f5e001208f0 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: dump: 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.118 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5e05a15640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5de403b9c0 0x7f5de403de80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a40 tx=0x7f5df800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5dfc004300 con 0x7f5e001208f0 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 msgr2=0x7f5e0009d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 0x7f5e0009d940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0119a0 tx=0x7f5dfc011e60 comp rx=0 tx=0).stop 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 msgr2=0x7f5e0009d940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.082+0000 7f5df5ffb640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001208f0 0x7f5e0009d940 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0119a0 tx=0x7f5dfc011e60 comp rx=0 tx=0).stop 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 0x7f5e0009de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001dac90 con 0x7f5e001cfb80 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 0x7f5e0009de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient: get_auth_request con 0x7f5e001cfb80 auth_method 0 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 10 monclient(hunting): handle_auth_done global_id 14110 payload 824 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 1 -- 172.21.15.17:0/712774322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5e001dae40 con 0x7f5e001cfb80 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05214640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 0x7f5e0009de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01b000 tx=0x7f5dfc0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc019040 con 0x7f5e001cfb80 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 10 monclient: dump: 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f5dfc003c40 con 0x7f5e001cfb80 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 msgr2=0x7f5e0009de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 0x7f5e0009de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01b000 tx=0x7f5dfc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 msgr2=0x7f5e0009de80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:26.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5df5ffb640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e001cfb80 0x7f5e0009de80 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01b000 tx=0x7f5dfc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 815, 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:26.123 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:26.124 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:26.125 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:26.126 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.128 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:26.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:26.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:26.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:26.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:26.134 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:26.135 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:26.136 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5de403b9c0 msgr2=0x7f5de403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5de403b9c0 0x7f5de403de80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a40 tx=0x7f5df800b040 comp rx=0 tx=0).stop 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 10 monclient: shutdown 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 msgr2=0x7f5e00134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 --2- 172.21.15.17:0/712774322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e000a0890 0x7f5e00134d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e0009f7c0 tx=0x7f5df0019180 comp rx=0 tx=0).stop 2024-09-15T21:18:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05a15640 1 -- 172.21.15.17:0/712774322 reap_dead start 2024-09-15T21:18:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e05a15640 1 -- 172.21.15.17:0/712774322 reap_dead start 2024-09-15T21:18:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 shutdown_connections 2024-09-15T21:18:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 >> 172.21.15.17:0/712774322 conn(0x7f5e000a66a0 msgr2=0x7f5e000a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 shutdown_connections 2024-09-15T21:18:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.090+0000 7f5e07478640 1 -- 172.21.15.17:0/712774322 wait complete. 2024-09-15T21:18:26.139 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:18:26.139 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:18:26.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:18:26.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.254+0000 7fe900648640 10 monclient: get_monmap_and_config 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.254+0000 7fe900648640 10 monclient: build_initial_monmap 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.254+0000 7fe900648640 10 monclient: monmap: 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:26.261233+0000 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:26.261233+0000 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:26.264 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:26.265 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:26.265 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:26.265 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.254+0000 7fe900648640 1 Processor -- start 2024-09-15T21:18:26.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.254+0000 7fe900648640 1 -- start start 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.254+0000 7fe900648640 10 monclient: init 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f80a0890 0x7fe8f80a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient(hunting): picked mon.noname-b con 0x7fe8f80a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe8f809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe8f80a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f8136490 con 0x7fe8f80a11b0 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f80a3130 con 0x7fe8f809ffc0 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f8073df0 con 0x7fe8f80a0890 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:26.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient(hunting): authenticate will time out at 1029.656612s 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8febe5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8f80a11b0 0x7fe8f8135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56270/0 (socket says 172.21.15.17:56270) 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f80a0890 0x7fe8f80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8febe5640 1 -- 172.21.15.17:0/940599510 learned_addr learned my addr 172.21.15.17:0/940599510 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): get_auth_request con 0x7fe8f80a0890 auth_method 0 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_done global_id 4418 payload 274 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 1 -- 172.21.15.17:0/940599510 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606244493 0 0) 0x7fe8f8136490 con 0x7fe8f80a11b0 2024-09-15T21:18:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 1 -- 172.21.15.17:0/940599510 >> v1:172.21.15.119:6789/0 conn(0x7fe8f809ffc0 legacy=0x7fe8f8135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 1 -- 172.21.15.17:0/940599510 >> v1:172.21.15.17:6789/0 conn(0x7fe8f80a11b0 legacy=0x7fe8f8135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient: _finish_hunting 0 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 1 monclient: found mon.noname-b 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 1 -- 172.21.15.17:0/940599510 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f80a1bf0 con 0x7fe8f80a0890 2024-09-15T21:18:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:26.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fe3e4640 1 --2- 172.21.15.17:0/940599510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f80a0890 0x7fe8f80a0c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00097f0 tx=0x7fe8f001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68807500f10308ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 1 -- 172.21.15.17:0/940599510 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00b0070 con 0x7fe8f80a0890 2024-09-15T21:18:26.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: dump: 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.274 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: _finish_auth 0 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: _check_auth_tickets 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 1 -- 172.21.15.17:0/940599510 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f00ab600 con 0x7fe8f80a0890 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 1 -- 172.21.15.17:0/940599510 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00ab9d0 con 0x7fe8f80a0890 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe8fd3e2640 10 monclient: dump: 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 5 monclient: authenticate success, global_id 4418 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.258+0000 7fe900648640 4 set_mon_vals no callback set 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: shutdown 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:26.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/940599510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f80a0890 msgr2=0x7fe8f80a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 --2- 172.21.15.17:0/940599510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f80a0890 0x7fe8f80a0c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00097f0 tx=0x7fe8f001c3a0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/940599510 shutdown_connections 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 --2- 172.21.15.17:0/940599510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f80a0890 0x7fe8f80a0c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/940599510 >> 172.21.15.17:0/940599510 conn(0x7fe8f80a66a0 msgr2=0x7fe8f80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/940599510 shutdown_connections 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/940599510 wait complete. 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: build_initial_monmap 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: monmap: 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:26.277 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 Processor -- start 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- start start 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: init 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 0x7fe8f811f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): picked mon.noname-c con 0x7fe8f809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f80a0890 0x7fe8f81d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): picked mon.noname-b con 0x7fe8f80a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:26.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f80a11b0 0x7fe8f811cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): picked mon.noname-a con 0x7fe8f80a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8141ab0 con 0x7fe8f80a11b0 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8141930 con 0x7fe8f80a0890 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f8141c30 con 0x7fe8f809ffc0 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient(hunting): authenticate will time out at 1029.661675s 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 0x7fe8f811f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f80a11b0 0x7fe8f811cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 0x7fe8f811f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58266/0 (socket says 172.21.15.17:58266) 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 1 -- 172.21.15.17:0/1022117493 learned_addr learned my addr 172.21.15.17:0/1022117493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:26.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): get_auth_request con 0x7fe8f809ffc0 auth_method 0 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): get_auth_request con 0x7fe8f80a11b0 auth_method 0 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f80a0890 0x7fe8f81d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): get_auth_request con 0x7fe8f80a0890 auth_method 0 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fdbe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient(hunting): handle_auth_done global_id 4458 payload 824 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 -- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 msgr2=0x7fe8f811f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 0x7fe8f811f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 -- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f80a0890 msgr2=0x7fe8f81d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f80a0890 0x7fe8f81d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient: _finish_hunting 0 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 monclient: found mon.noname-a 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 -- 172.21.15.17:0/1022117493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f809d0f0 con 0x7fe8f80a11b0 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8fe3e4640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 0x7fe8f811f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8febe5640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f80a11b0 0x7fe8f811cfc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c690 tx=0x7fe8e800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8018070 con 0x7fe8f80a11b0 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: dump: 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.282 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8004d90 con 0x7fe8f80a11b0 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 5 monclient: authenticate success, global_id 4458 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _renew_subs 2024-09-15T21:18:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f8120820 con 0x7fe8f80a11b0 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800a880 con 0x7fe8f80a11b0 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 10 monclient: dump: 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:26.284 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _renew_subs 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81c6ac0 con 0x7fe8f80a11b0 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f7fff640 4 set_mon_vals no callback set 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe8e801b070 con 0x7fe8f80a11b0 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f80a0c70 con 0x7fe8f80a11b0 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8e403b9c0 0x7fe8e403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:26.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.262+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fe8e804edc0 con 0x7fe8f80a11b0 2024-09-15T21:18:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.266+0000 7fe8fe3e4640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8e403b9c0 0x7fe8e403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.266+0000 7fe8fe3e4640 10 monclient: get_auth_request con 0x7fe8e403b9c0 auth_method 0 2024-09-15T21:18:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.266+0000 7fe8fe3e4640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8e403b9c0 0x7fe8e403de80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009740 tx=0x7fe8f00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.270+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe8f809d0f0 con 0x7fe8f80a11b0 2024-09-15T21:18:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.270+0000 7fe8f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.270+0000 7fe8f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe900648640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:18:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe900648640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:18:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe900648640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fe8f811ea80 con 0x7fe8f80a11b0 2024-09-15T21:18:26.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe8f6ffd640 1 -- 172.21.15.17:0/1022117493 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fe8e8007980 con 0x7fe8f80a11b0 2024-09-15T21:18:26.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe8f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:18:26.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fe8f6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:26.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=8,config=2,health=12,kv=10,logm=68} crc {auth=1290027008,config=3541143443,health=861809967,kv=754178774,logm=1061666178}) 2024-09-15T21:18:26.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=8,config=2,health=12,kv=11,logm=67} crc {auth=1290027008,config=3541143443,health=861809967,kv=301574753,logm=3069263315}) 2024-09-15T21:18:26.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=8,config=2,health=12,kv=10,logm=68} crc {auth=1290027008,config=3541143443,health=861809967,kv=754178774,logm=1061666178}) 2024-09-15T21:18:26.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.510+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=8,config=2,health=12,kv=10,logm=68} crc {auth=1595678963,config=3541143443,health=861809967,kv=359496365,logm=1184270563}) 2024-09-15T21:18:26.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97943862}) 2024-09-15T21:18:26.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1799161754}) 2024-09-15T21:18:26.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97943862}) 2024-09-15T21:18:26.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=93600967}) 2024-09-15T21:18:26.524 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1872227674}) 2024-09-15T21:18:26.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3196060665}) 2024-09-15T21:18:26.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1872227674}) 2024-09-15T21:18:26.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1738947583}) 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8e403b9c0 msgr2=0x7fe8e403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8e403b9c0 0x7fe8e403de80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009740 tx=0x7fe8f00a6000 comp rx=0 tx=0).stop 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 10 monclient: shutdown 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f80a11b0 msgr2=0x7fe8f811cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f80a11b0 0x7fe8f811cfc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c690 tx=0x7fe8e800cb50 comp rx=0 tx=0).stop 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 shutdown_connections 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8e403b9c0 0x7fe8e403de80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f80a11b0 0x7fe8f811cfc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f80a0890 0x7fe8f81d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 --2- 172.21.15.17:0/1022117493 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8f809ffc0 0x7fe8f811f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:26.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 >> 172.21.15.17:0/1022117493 conn(0x7fe8f80a66a0 msgr2=0x7fe8f813e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:26.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 shutdown_connections 2024-09-15T21:18:26.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fe900648640 1 -- 172.21.15.17:0/1022117493 wait complete. 2024-09-15T21:18:26.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=2} crc {logm=4142486714,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1995790778,mon_config_key=80854102}) 2024-09-15T21:18:26.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=3} crc {logm=2838656996,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2862043591,mon_config_key=1565140462}) 2024-09-15T21:18:26.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=2} crc {logm=4142486714,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1995790778,mon_config_key=80854102}) 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.514+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4} crc {logm=1610809778,mdsmap=173658046,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1995790778,mon_config_key=2724965167}) 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.518+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.518+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=39} crc {mon_config_key=3341611962,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3586935393,osd_snap=841190635,osdmap=68872697}) 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.518+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=39} crc {mon_config_key=3341611962,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3586935393,osd_snap=841190635,osdmap=259915937}) 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.518+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.518+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=39} crc {mon_config_key=3341611962,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3586935393,osd_snap=841190635,osdmap=68872697}) 2024-09-15T21:18:26.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:18:26.518+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=39} crc {mon_config_key=3331758901,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3586935393,osd_snap=841190635,osdmap=1989773986}) 2024-09-15T21:18:26.541 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:18:27.542 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:18:27.543 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:18:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.666+0000 7fd563d65640 10 monclient: get_monmap_and_config 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.666+0000 7fd563d65640 10 monclient: build_initial_monmap 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.666+0000 7fd563d65640 10 monclient: monmap: 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:27.672251+0000 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:27.672251+0000 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:27.676 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:27.676 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:27.676 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.666+0000 7fd563d65640 1 Processor -- start 2024-09-15T21:18:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.666+0000 7fd563d65640 1 -- start start 2024-09-15T21:18:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.666+0000 7fd563d65640 10 monclient: init 2024-09-15T21:18:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c1340c0 0x7fd55c1344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): picked mon.noname-b con 0x7fd55c1340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 0x7fd55c0a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): picked mon.noname-a con 0x7fd55c09d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd55c0a0dd0 0x7fd55c0a1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): picked mon.noname-c con 0x7fd55c0a0dd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c09d520 con 0x7fd55c09d6c0 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c1349e0 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c09e1b0 con 0x7fd55c1340c0 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient(hunting): authenticate will time out at 1031.068459s 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 0x7fd55c0a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c1340c0 0x7fd55c1344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 0x7fd55c0a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33402/0 (socket says 172.21.15.17:33402) 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 1 -- 172.21.15.17:0/3431106734 learned_addr learned my addr 172.21.15.17:0/3431106734 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): get_auth_request con 0x7fd55c09d6c0 auth_method 0 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): get_auth_request con 0x7fd55c1340c0 auth_method 0 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 1 --2- 172.21.15.17:0/3431106734 >> v2:172.21.15.119:3300/0 conn(0x7fd55c0a0dd0 0x7fd55c0a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): get_auth_request con 0x7fd55c0a0dd0 auth_method 0 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd562302640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient(hunting): handle_auth_done global_id 4424 payload 274 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 -- 172.21.15.17:0/3431106734 >> v2:172.21.15.119:3300/0 conn(0x7fd55c0a0dd0 msgr2=0x7fd55c0a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 --2- 172.21.15.17:0/3431106734 >> v2:172.21.15.119:3300/0 conn(0x7fd55c0a0dd0 0x7fd55c0a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 -- 172.21.15.17:0/3431106734 >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 msgr2=0x7fd55c0a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 --2- 172.21.15.17:0/3431106734 >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 0x7fd55c0a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient: _finish_hunting 0 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 monclient: found mon.noname-b 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 -- 172.21.15.17:0/3431106734 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c09e490 con 0x7fd55c1340c0 2024-09-15T21:18:27.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561300640 1 --2- 172.21.15.17:0/3431106734 >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 0x7fd55c0a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:27.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd561b01640 1 --2- 172.21.15.17:0/3431106734 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c1340c0 0x7fd55c1344a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd55c06f1c0 tx=0x7fd55801c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c017e49fc39e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:27.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 1 -- 172.21.15.17:0/3431106734 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580b0070 con 0x7fd55c1340c0 2024-09-15T21:18:27.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:27.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: dump: 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: _finish_auth 0 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 1 -- 172.21.15.17:0/3431106734 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5580ab600 con 0x7fd55c1340c0 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 5 monclient: authenticate success, global_id 4424 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 1 -- 172.21.15.17:0/3431106734 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580ab9d0 con 0x7fd55c1340c0 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd560aff640 10 monclient: dump: 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.670+0000 7fd563d65640 4 set_mon_vals no callback set 2024-09-15T21:18:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: shutdown 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/3431106734 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c1340c0 msgr2=0x7fd55c1344a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- 172.21.15.17:0/3431106734 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c1340c0 0x7fd55c1344a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd55c06f1c0 tx=0x7fd55801c340 comp rx=0 tx=0).stop 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/3431106734 shutdown_connections 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- 172.21.15.17:0/3431106734 >> v2:172.21.15.119:3300/0 conn(0x7fd55c0a0dd0 0x7fd55c0a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- 172.21.15.17:0/3431106734 >> v2:172.21.15.17:3300/0 conn(0x7fd55c09d6c0 0x7fd55c0a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- 172.21.15.17:0/3431106734 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c1340c0 0x7fd55c1344a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/3431106734 >> 172.21.15.17:0/3431106734 conn(0x7fd55c0a66a0 msgr2=0x7fd55c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/3431106734 shutdown_connections 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/3431106734 wait complete. 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: build_initial_monmap 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: monmap: 2024-09-15T21:18:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 Processor -- start 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- start start 2024-09-15T21:18:27.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: init 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c09d6c0 0x7fd55c1cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): picked mon.noname-c con 0x7fd55c09d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 0x7fd55c13a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): picked mon.noname-b con 0x7fd55c0a0dd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c1340c0 0x7fd55c13a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): picked mon.noname-a con 0x7fd55c1340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c13ce70 con 0x7fd55c1340c0 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0a27a0 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c13cff0 con 0x7fd55c09d6c0 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient(hunting): authenticate will time out at 1031.073843s 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 0x7fd55c13a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c09d6c0 0x7fd55c1cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c1340c0 0x7fd55c13a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 0x7fd55c13a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:46034/0 (socket says 172.21.15.17:46034) 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 -- 172.21.15.17:0/2447987516 learned_addr learned my addr 172.21.15.17:0/2447987516 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): get_auth_request con 0x7fd55c0a0dd0 auth_method 0 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): get_auth_request con 0x7fd55c1340c0 auth_method 0 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): get_auth_request con 0x7fd55c09d6c0 auth_method 0 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:27.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd562302640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient(hunting): handle_auth_done global_id 14122 payload 824 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 -- 172.21.15.17:0/2447987516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c09d6c0 msgr2=0x7fd55c1cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c09d6c0 0x7fd55c1cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 -- 172.21.15.17:0/2447987516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c1340c0 msgr2=0x7fd55c13a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c1340c0 0x7fd55c13a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient: _finish_hunting 0 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 monclient: found mon.noname-b 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 -- 172.21.15.17:0/2447987516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c09c500 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd561300640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 0x7fd55c13a0c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00bfd0 tx=0x7fd54c00e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c01e070 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: dump: 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 5 monclient: authenticate success, global_id 14122 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c011860 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _renew_subs 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c011c50 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c09d0f0 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd553fff640 4 set_mon_vals no callback set 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd5527fc640 10 monclient: dump: 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:27.699 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _renew_subs 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c1306d0 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.674+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55c0a1260 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.678+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd54c021070 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.678+0000 7fd5527fc640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd54003b9c0 0x7fd54003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.678+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fd54c0523e0 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.678+0000 7fd561b01640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd54003b9c0 0x7fd54003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.678+0000 7fd561b01640 10 monclient: get_auth_request con 0x7fd54003b9c0 auth_method 0 2024-09-15T21:18:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.678+0000 7fd561b01640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd54003b9c0 0x7fd54003de80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5580ad000 tx=0x7fd5580a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.686+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd54c00d350 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.686+0000 7fd5527fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.686+0000 7fd5527fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:27.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:27.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:27.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:27.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd55c131230 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd5527fc640 1 -- 172.21.15.17:0/2447987516 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fd54c011df0 con 0x7fd55c0a0dd0 2024-09-15T21:18:27.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd5527fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:27.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd5527fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:27.930 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:27.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd54003b9c0 msgr2=0x7fd54003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:27.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd54003b9c0 0x7fd54003de80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5580ad000 tx=0x7fd5580a6000 comp rx=0 tx=0).stop 2024-09-15T21:18:27.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 10 monclient: shutdown 2024-09-15T21:18:27.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:27.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 msgr2=0x7fd55c13a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:27.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 0x7fd55c13a0c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00bfd0 tx=0x7fd54c00e730 comp rx=0 tx=0).stop 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 shutdown_connections 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd54003b9c0 0x7fd54003de80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c1340c0 0x7fd55c13a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c0a0dd0 0x7fd55c13a0c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 --2- 172.21.15.17:0/2447987516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c09d6c0 0x7fd55c1cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 >> 172.21.15.17:0/2447987516 conn(0x7fd55c0a66a0 msgr2=0x7fd55c133300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 shutdown_connections 2024-09-15T21:18:27.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:27.922+0000 7fd563d65640 1 -- 172.21.15.17:0/2447987516 wait complete. 2024-09-15T21:18:27.948 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:18:27.948 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:18:27.948 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:18:27.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:18:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.050+0000 7f8952053640 10 monclient: get_monmap_and_config 2024-09-15T21:18:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.050+0000 7f8952053640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.050+0000 7f8952053640 10 monclient: monmap: 2024-09-15T21:18:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:28.057306+0000 2024-09-15T21:18:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:28.057306+0000 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:28.061 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.050+0000 7f8952053640 1 Processor -- start 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 -- start start 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient: init 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f894c0a3a80 0x7f894c0a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): picked mon.noname-a con 0x7f894c0a3a80 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 0x7f894c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): picked mon.noname-b con 0x7f894c06e580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f894c137c20 0x7f894c1380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): picked mon.noname-c con 0x7f894c137c20 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f894c06ea00 con 0x7f894c0a3a80 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f894c09ef70 con 0x7f894c137c20 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f894c0a48e0 con 0x7f894c06e580 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient(hunting): authenticate will time out at 1031.452798s 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f894c0a3a80 0x7f894c0a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 0x7f894c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 0x7f894c0a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59654/0 (socket says 172.21.15.17:59654) 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 -- 172.21.15.17:0/228349556 learned_addr learned my addr 172.21.15.17:0/228349556 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): get_auth_request con 0x7f894c0a3a80 auth_method 0 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): get_auth_request con 0x7f894c06e580 auth_method 0 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894bfff640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.119:3300/0 conn(0x7f894c137c20 0x7f894c1380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient(hunting): handle_auth_done global_id 4430 payload 274 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 -- 172.21.15.17:0/228349556 >> v2:172.21.15.119:3300/0 conn(0x7f894c137c20 msgr2=0x7f894c1380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.119:3300/0 conn(0x7f894c137c20 0x7f894c1380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 -- 172.21.15.17:0/228349556 >> v2:172.21.15.17:3300/0 conn(0x7f894c0a3a80 msgr2=0x7f894c0a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.17:3300/0 conn(0x7f894c0a3a80 0x7f894c0a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 monclient: found mon.noname-b 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 -- 172.21.15.17:0/228349556 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c09f980 con 0x7f894c06e580 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894bfff640 10 monclient: get_auth_request con 0x7f894c137c20 auth_method 0 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894bfff640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.119:3300/0 conn(0x7f894c137c20 0x7f894c1380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894b7fe640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.17:3300/0 conn(0x7f894c0a3a80 0x7f894c0a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894affd640 1 --2- 172.21.15.17:0/228349556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 0x7f894c0a43a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f894c06e990 tx=0x7f894401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4413f82be7038a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 1 -- 172.21.15.17:0/228349556 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89440a5640 con 0x7f894c06e580 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: dump: 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 5 monclient: authenticate success, global_id 4430 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 1 -- 172.21.15.17:0/228349556 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89440a57e0 con 0x7f894c06e580 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 4 set_mon_vals no callback set 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 1 -- 172.21.15.17:0/228349556 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89440a5bb0 con 0x7f894c06e580 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f894a7fc640 10 monclient: dump: 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.054+0000 7f8952053640 10 monclient: shutdown 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/228349556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 msgr2=0x7f894c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- 172.21.15.17:0/228349556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 0x7f894c0a43a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f894c06e990 tx=0x7f894401c8b0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/228349556 shutdown_connections 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.119:3300/0 conn(0x7f894c137c20 0x7f894c1380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- 172.21.15.17:0/228349556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c06e580 0x7f894c0a43a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- 172.21.15.17:0/228349556 >> v2:172.21.15.17:3300/0 conn(0x7f894c0a3a80 0x7f894c0a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/228349556 >> 172.21.15.17:0/228349556 conn(0x7f894c0a8980 msgr2=0x7f894c0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/228349556 shutdown_connections 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/228349556 wait complete. 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: monmap: 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 Processor -- start 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- start start 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: init 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 0x7f894c1cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): picked mon.noname-a con 0x7f894c06e580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c0a3a80 0x7f894c136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): picked mon.noname-c con 0x7f894c0a3a80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894c137c20 0x7f894c139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): picked mon.noname-b con 0x7f894c137c20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0a0470 con 0x7f894c06e580 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0a02f0 con 0x7f894c137c20 2024-09-15T21:18:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f894c141690 con 0x7f894c0a3a80 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): authenticate will time out at 1031.457490s 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 0x7f894c1cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 0x7f894c1cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33422/0 (socket says 172.21.15.17:33422) 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 -- 172.21.15.17:0/597901456 learned_addr learned my addr 172.21.15.17:0/597901456 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): get_auth_request con 0x7f894c06e580 auth_method 0 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c0a3a80 0x7f894c136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894c137c20 0x7f894c139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): get_auth_request con 0x7f894c0a3a80 auth_method 0 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): get_auth_request con 0x7f894c137c20 auth_method 0 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient(hunting): handle_auth_done global_id 4470 payload 824 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c0a3a80 msgr2=0x7f894c136ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f894c0a3a80 0x7f894c136ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894c137c20 msgr2=0x7f894c139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894c137c20 0x7f894c139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 monclient: found mon.noname-a 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c13b8e0 con 0x7f894c06e580 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894b7fe640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 0x7f894c1cf9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f893c00ebe0 tx=0x7f893c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c017070 con 0x7f894c06e580 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: dump: 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893c00cc30 con 0x7f894c06e580 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c013470 con 0x7f894c06e580 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 5 monclient: authenticate success, global_id 4470 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: _renew_subs 2024-09-15T21:18:28.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c0a7960 con 0x7f894c06e580 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: dump: 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8949ffb640 4 set_mon_vals no callback set 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: _renew_subs 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c13d3a0 con 0x7f894c06e580 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 0x7f894c13a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1380d0 con 0x7f894c137430 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 0x7f894c13a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient: get_auth_request con 0x7f894c137430 auth_method 0 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f893c01a070 con 0x7f894c06e580 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 10 monclient(hunting): handle_auth_done global_id 4473 payload 824 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f894c0008e0 con 0x7f894c137430 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f893003b9c0 0x7f893003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f893c04f8b0 con 0x7f894c06e580 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894affd640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 0x7f894c13a3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f894c0a5b90 tx=0x7f8944019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f893003b9c0 0x7f893003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f894bfff640 10 monclient: get_auth_request con 0x7f893003b9c0 auth_method 0 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944009510 con 0x7f894c137430 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.058+0000 7f8943fff640 10 monclient: dump: 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f89440c0660 con 0x7f894c137430 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 msgr2=0x7f894c13a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 0x7f894c13a3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f894c0a5b90 tx=0x7f8944019650 comp rx=0 tx=0).stop 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 msgr2=0x7f894c13a3f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f8943fff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c137430 0x7f894c13a3f0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f894c0a5b90 tx=0x7f8944019650 comp rx=0 tx=0).stop 2024-09-15T21:18:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.062+0000 7f894bfff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f893003b9c0 0x7f893003de80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f893400aa60 tx=0x7f8934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 0x7f894c071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f894c13ad10 con 0x7f894c13a930 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 0x7f894c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient: get_auth_request con 0x7f894c13a930 auth_method 0 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 10 monclient(hunting): handle_auth_done global_id 4476 payload 824 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 1 -- 172.21.15.17:0/597901456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f894c13c440 con 0x7f894c13a930 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894affd640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 0x7f894c071290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f894401d150 tx=0x7f89440a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944009af0 con 0x7f894c13a930 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 10 monclient: dump: 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.097 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7f8944009af0 con 0x7f894c13a930 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 msgr2=0x7f894c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 0x7f894c071290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f894401d150 tx=0x7f89440a7000 comp rx=0 tx=0).stop 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 msgr2=0x7f894c071290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8943fff640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c13a930 0x7f894c071290 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f894401d150 tx=0x7f89440a7000 comp rx=0 tx=0).stop 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 73687, 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:28.099 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.100 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.101 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:28.102 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:28.103 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.104 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:28.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.108 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.108 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:28.109 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:28.109 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.109 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.109 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.110 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:28.110 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:28.110 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.110 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:28.112 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:28.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.114 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.115 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.117 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:28.117 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f893003b9c0 msgr2=0x7f893003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f893003b9c0 0x7f893003de80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f893400aa60 tx=0x7f8934008040 comp rx=0 tx=0).stop 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 10 monclient: shutdown 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 msgr2=0x7f894c1cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 0x7f894c1cf9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f893c00ebe0 tx=0x7f893c00c520 comp rx=0 tx=0).stop 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f894bfff640 1 -- 172.21.15.17:0/597901456 reap_dead start 2024-09-15T21:18:28.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 shutdown_connections 2024-09-15T21:18:28.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 --2- 172.21.15.17:0/597901456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f894c06e580 0x7f894c1cf9c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 >> 172.21.15.17:0/597901456 conn(0x7f894c0a8980 msgr2=0x7f894c09db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 shutdown_connections 2024-09-15T21:18:28.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.066+0000 7f8952053640 1 -- 172.21.15.17:0/597901456 wait complete. 2024-09-15T21:18:28.119 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:18:28.119 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:18:28.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:18:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient: get_monmap_and_config 2024-09-15T21:18:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient: monmap: 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:28.227258+0000 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:28.227258+0000 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:28.231 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 1 Processor -- start 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 1 -- start start 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient: init 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f57800a3c50 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57800a0010 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5780142350 0x7f5780144740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient(hunting): picked mon.noname-c con 0x7f5780142350 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a4530 con 0x7f57800a0010 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a4310 con 0x7f57800a3c50 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5780070ee0 con 0x7f5780142350 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5786f6b640 10 monclient(hunting): authenticate will time out at 1031.622604s 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57800a0010 0x7f5780074ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46254/0 (socket says 172.21.15.17:46254) 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577ffff640 1 -- 172.21.15.17:0/1070981783 learned_addr learned my addr 172.21.15.17:0/1070981783 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 1 --2- 172.21.15.17:0/1070981783 >> v2:172.21.15.119:3300/0 conn(0x7f5780142350 0x7f5780144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): get_auth_request con 0x7f5780142350 auth_method 0 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577f7fe640 1 -- 172.21.15.17:0/1070981783 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915191631 0 0) 0x7f57800a4530 con 0x7f57800a0010 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577f7fe640 10 monclient(hunting): my global_id is 4479 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f577f7fe640 1 -- 172.21.15.17:0/1070981783 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f576c003530 con 0x7f57800a0010 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient(hunting): handle_auth_done global_id 14128 payload 274 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 1 -- 172.21.15.17:0/1070981783 >> v1:172.21.15.119:6789/0 conn(0x7f57800a3c50 legacy=0x7f57800743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 1 -- 172.21.15.17:0/1070981783 >> v1:172.21.15.17:6789/0 conn(0x7f57800a0010 legacy=0x7f5780074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 1 monclient: found mon.noname-c 2024-09-15T21:18:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.222+0000 7f5785508640 1 -- 172.21.15.17:0/1070981783 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57800a0460 con 0x7f5780142350 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5785508640 1 --2- 172.21.15.17:0/1070981783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f5780144740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5780071060 tx=0x7f577001c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b42411f28b6409 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 1 -- 172.21.15.17:0/1070981783 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700b0070 con 0x7f5780142350 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: dump: 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 1 -- 172.21.15.17:0/1070981783 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57700ab600 con 0x7f5780142350 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 1 -- 172.21.15.17:0/1070981783 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700ab9d0 con 0x7f5780142350 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f577f7fe640 10 monclient: dump: 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 5 monclient: authenticate success, global_id 14128 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 4 set_mon_vals no callback set 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: shutdown 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 -- 172.21.15.17:0/1070981783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 msgr2=0x7f5780144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 --2- 172.21.15.17:0/1070981783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f5780144740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5780071060 tx=0x7f577001c340 comp rx=0 tx=0).stop 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 -- 172.21.15.17:0/1070981783 shutdown_connections 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 --2- 172.21.15.17:0/1070981783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f5780144740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 -- 172.21.15.17:0/1070981783 >> 172.21.15.17:0/1070981783 conn(0x7f57800a4e90 msgr2=0x7f57800a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 -- 172.21.15.17:0/1070981783 shutdown_connections 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 -- 172.21.15.17:0/1070981783 wait complete. 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: monmap: 2024-09-15T21:18:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 Processor -- start 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 -- start start 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: init 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57800a0010 0x7f578011e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient(hunting): picked mon.noname-c con 0x7f57800a0010 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57800a3c50 0x7f578011eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 10 monclient(hunting): picked mon.noname-a con 0x7f57800a3c50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.226+0000 7f5786f6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f578011f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): picked mon.noname-b con 0x7f5780142350 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5780147830 con 0x7f57800a3c50 2024-09-15T21:18:28.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57801476b0 con 0x7f5780142350 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57801479b0 con 0x7f57800a0010 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): authenticate will time out at 1031.628041s 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57800a3c50 0x7f578011eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57800a3c50 0x7f578011eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33446/0 (socket says 172.21.15.17:33446) 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 -- 172.21.15.17:0/973069593 learned_addr learned my addr 172.21.15.17:0/973069593 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): get_auth_request con 0x7f57800a3c50 auth_method 0 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57800a0010 0x7f578011e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f578011f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): get_auth_request con 0x7f5780142350 auth_method 0 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): get_auth_request con 0x7f57800a0010 auth_method 0 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient(hunting): handle_auth_done global_id 4482 payload 824 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57800a0010 msgr2=0x7f578011e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57800a0010 0x7f578011e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 msgr2=0x7f578011f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f578011f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 monclient: found mon.noname-a 2024-09-15T21:18:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5780121060 con 0x7f57800a3c50 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5780142350 0x7f578011f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577ffff640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57800a3c50 0x7f578011eb20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f576800c4d0 tx=0x7f576800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768017070 con 0x7f57800a3c50 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: dump: 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5768005d10 con 0x7f57800a3c50 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 5 monclient: authenticate success, global_id 4482 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient: _renew_subs 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5780121ed0 con 0x7f57800a3c50 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680072b0 con 0x7f57800a3c50 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: dump: 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.253 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577e7fc640 4 set_mon_vals no callback set 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient: _renew_subs 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578011fab0 con 0x7f57800a3c50 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 0x7f57801d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f578009f180 con 0x7f57801d3b60 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 0x7f57801d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient: get_auth_request con 0x7f57801d3b60 auth_method 0 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f576801a070 con 0x7f57800a3c50 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 10 monclient(hunting): handle_auth_done global_id 4439 payload 824 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57800743a0 con 0x7f57801d3b60 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f576003b9c0 0x7f576003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f576804ec20 con 0x7f57800a3c50 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5784d07640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 0x7f57801d3f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f577800ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778003b90 con 0x7f57801d3b60 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: dump: 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.256 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f576003b9c0 0x7f576003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 10 monclient: get_auth_request con 0x7f576003b9c0 auth_method 0 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5778018070 con 0x7f57801d3b60 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:28.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 msgr2=0x7f57801d3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 0x7f57801d3f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f577800ff70 comp rx=0 tx=0).stop 2024-09-15T21:18:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 msgr2=0x7f57801d3f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f577d7fa640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d3b60 0x7f57801d3f40 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f577800ff70 comp rx=0 tx=0).stop 2024-09-15T21:18:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.230+0000 7f5785508640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f576003b9c0 0x7f576003de80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57700ad000 tx=0x7f5770008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 0x7f5780142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57800693a0 con 0x7f57801d4480 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 0x7f5780142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient: get_auth_request con 0x7f57801d4480 auth_method 0 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 10 monclient(hunting): handle_auth_done global_id 4442 payload 824 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 1 -- 172.21.15.17:0/973069593 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f57801d4890 con 0x7f57801d4480 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5784d07640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 0x7f5780142730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5778017000 tx=0x7f57780014b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577800d9d0 con 0x7f57801d4480 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 10 monclient: dump: 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.261 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f577800d9d0 con 0x7f57801d4480 2024-09-15T21:18:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 msgr2=0x7f5780142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 0x7f5780142730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5778017000 tx=0x7f57780014b0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 msgr2=0x7f5780142730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f577d7fa640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57801d4480 0x7f5780142730 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5778017000 tx=0x7f57780014b0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.263 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:28.264 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:28.264 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:28.264 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:18:28.264 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-15T21:18:28.264 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 73859, 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.266 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:28.267 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:28.268 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.269 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.271 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.273 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.274 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:28.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:18:28.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.278 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.278 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:28.278 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:28.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f576003b9c0 msgr2=0x7f576003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f576003b9c0 0x7f576003de80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57700ad000 tx=0x7f5770008040 comp rx=0 tx=0).stop 2024-09-15T21:18:28.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 10 monclient: shutdown 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57800a3c50 msgr2=0x7f578011eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 --2- 172.21.15.17:0/973069593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57800a3c50 0x7f578011eb20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f576800c4d0 tx=0x7f576800c990 comp rx=0 tx=0).stop 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5785508640 1 -- 172.21.15.17:0/973069593 reap_dead start 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5785508640 1 -- 172.21.15.17:0/973069593 reap_dead start 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 shutdown_connections 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 >> 172.21.15.17:0/973069593 conn(0x7f57800a4e90 msgr2=0x7f5780073380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 shutdown_connections 2024-09-15T21:18:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.238+0000 7f5786f6b640 1 -- 172.21.15.17:0/973069593 wait complete. 2024-09-15T21:18:28.280 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:18:28.281 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:18:28.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:18:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient: get_monmap_and_config 2024-09-15T21:18:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient: monmap: 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:28.390944+0000 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:28.390944+0000 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:28.394 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 Processor -- start 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 -- start start 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient: init 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c1378c0 0x7fe09c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): picked mon.noname-b con 0x7fe09c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 0x7fe09c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): picked mon.noname-a con 0x7fe09c1381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe09c0a5040 0x7fe09c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): picked mon.noname-c con 0x7fe09c0a5040 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe09c0a1d00 con 0x7fe09c1381e0 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe09c0a1fe0 con 0x7fe09c0a5040 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c0a22c0 con 0x7fe09c1378c0 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a3bb6640 10 monclient(hunting): authenticate will time out at 1031.786987s 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 0x7fe09c0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c1378c0 0x7fe09c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 0x7fe09c0a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33462/0 (socket says 172.21.15.17:33462) 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 1 -- 172.21.15.17:0/4275626042 learned_addr learned my addr 172.21.15.17:0/4275626042 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): get_auth_request con 0x7fe09c1381e0 auth_method 0 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): get_auth_request con 0x7fe09c1378c0 auth_method 0 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 1 --2- 172.21.15.17:0/4275626042 >> v2:172.21.15.119:3300/0 conn(0x7fe09c0a5040 0x7fe09c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): get_auth_request con 0x7fe09c0a5040 auth_method 0 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a2153640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient(hunting): handle_auth_done global_id 4445 payload 274 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 -- 172.21.15.17:0/4275626042 >> v2:172.21.15.119:3300/0 conn(0x7fe09c0a5040 msgr2=0x7fe09c0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 --2- 172.21.15.17:0/4275626042 >> v2:172.21.15.119:3300/0 conn(0x7fe09c0a5040 0x7fe09c0a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 -- 172.21.15.17:0/4275626042 >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 msgr2=0x7fe09c0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 --2- 172.21.15.17:0/4275626042 >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 0x7fe09c0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 monclient: found mon.noname-b 2024-09-15T21:18:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:18:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1952640 1 -- 172.21.15.17:0/4275626042 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c0a4490 con 0x7fe09c1378c0 2024-09-15T21:18:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.386+0000 7fe0a1151640 1 --2- 172.21.15.17:0/4275626042 >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 0x7fe09c0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a1952640 1 --2- 172.21.15.17:0/4275626042 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c1378c0 0x7fe09c137ca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a6150 tx=0x7fe09801c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d647a4c49ada17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 1 -- 172.21.15.17:0/4275626042 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980b0070 con 0x7fe09c1378c0 2024-09-15T21:18:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: dump: 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 1 -- 172.21.15.17:0/4275626042 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0980ab600 con 0x7fe09c1378c0 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 1 -- 172.21.15.17:0/4275626042 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980ab9d0 con 0x7fe09c1378c0 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 5 monclient: authenticate success, global_id 4445 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 4 set_mon_vals no callback set 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a0950640 10 monclient: dump: 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 10 monclient: shutdown 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4275626042 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c1378c0 msgr2=0x7fe09c137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4275626042 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c1378c0 0x7fe09c137ca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a6150 tx=0x7fe09801c340 comp rx=0 tx=0).stop 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4275626042 shutdown_connections 2024-09-15T21:18:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4275626042 >> v2:172.21.15.119:3300/0 conn(0x7fe09c0a5040 0x7fe09c0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4275626042 >> v2:172.21.15.17:3300/0 conn(0x7fe09c1381e0 0x7fe09c0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4275626042 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c1378c0 0x7fe09c137ca0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4275626042 >> 172.21.15.17:0/4275626042 conn(0x7fe09c0aac40 msgr2=0x7fe09c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4275626042 shutdown_connections 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4275626042 wait complete. 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 10 monclient: monmap: 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 Processor -- start 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 1 -- start start 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.390+0000 7fe0a3bb6640 10 monclient: init 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c0a5040 0x7fe09c1cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): picked mon.noname-c con 0x7fe09c0a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe09c1378c0 0x7fe09c136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): picked mon.noname-a con 0x7fe09c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:28.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1381e0 0x7fe09c13ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): picked mon.noname-b con 0x7fe09c1381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c141730 con 0x7fe09c1378c0 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1415b0 con 0x7fe09c1381e0 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c1418b0 con 0x7fe09c0a5040 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): authenticate will time out at 1031.792353s 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c0a5040 0x7fe09c1cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe09c1378c0 0x7fe09c136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c0a5040 0x7fe09c1cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59708/0 (socket says 172.21.15.17:59708) 2024-09-15T21:18:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe09c1378c0 0x7fe09c136c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33464/0 (socket says 172.21.15.17:33464) 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 -- 172.21.15.17:0/4211643896 learned_addr learned my addr 172.21.15.17:0/4211643896 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): get_auth_request con 0x7fe09c0a5040 auth_method 0 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): get_auth_request con 0x7fe09c1378c0 auth_method 0 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1381e0 0x7fe09c13ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): get_auth_request con 0x7fe09c1381e0 auth_method 0 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient(hunting): handle_auth_done global_id 4448 payload 824 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1381e0 msgr2=0x7fe09c13ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1381e0 0x7fe09c13ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe09c1378c0 msgr2=0x7fe09c136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe09c1378c0 0x7fe09c136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 monclient: found mon.noname-c 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c121d70 con 0x7fe09c0a5040 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe09c1378c0 0x7fe09c136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1952640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c0a5040 0x7fe09c1cfce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0980ae040 tx=0x7fe0980190e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980b0040 con 0x7fe09c0a5040 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: dump: 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0980b9070 con 0x7fe09c0a5040 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 5 monclient: authenticate success, global_id 4448 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient: _renew_subs 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c0a12f0 con 0x7fe09c0a5040 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980b4650 con 0x7fe09c0a5040 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: dump: 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.417 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0937fe640 4 set_mon_vals no callback set 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient: _renew_subs 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c122610 con 0x7fe09c0a5040 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 0x7fe09c13d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c06ea70 con 0x7fe09c120e10 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 0x7fe09c13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient: get_auth_request con 0x7fe09c120e10 auth_method 0 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe0980b47f0 con 0x7fe09c0a5040 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08003b9f0 0x7fe08003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 10 monclient(hunting): handle_auth_done global_id 14143 payload 824 2024-09-15T21:18:28.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe09c137ca0 con 0x7fe09c120e10 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fe0980e9900 con 0x7fe09c0a5040 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08003b9f0 0x7fe08003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 10 monclient: get_auth_request con 0x7fe08003b9f0 auth_method 0 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a1151640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 0x7fe09c13d370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe09c06e7b0 tx=0x7fe08c00f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004020 con 0x7fe09c120e10 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: dump: 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0a2153640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08003b9f0 0x7fe08003deb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a0b30 tx=0x7fe09400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe08c004020 con 0x7fe09c120e10 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 msgr2=0x7fe09c13d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 0x7fe09c13d370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe09c06e7b0 tx=0x7fe08c00f610 comp rx=0 tx=0).stop 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 msgr2=0x7fe09c13d370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:28.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.394+0000 7fe0927fc640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c120e10 0x7fe09c13d370 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:28.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 0x7fe09c13d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c069270 con 0x7fe09c1371b0 2024-09-15T21:18:28.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 0x7fe09c13d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient: get_auth_request con 0x7fe09c1371b0 auth_method 0 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 10 monclient(hunting): handle_auth_done global_id 14146 payload 824 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 1 -- 172.21.15.17:0/4211643896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe09c06cba0 con 0x7fe09c1371b0 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a1151640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 0x7fe09c13d8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01b000 tx=0x7fe08c0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c014970 con 0x7fe09c1371b0 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:28.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 10 monclient: dump: 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7fe08c0133a0 con 0x7fe09c1371b0 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:28.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 msgr2=0x7fe09c13d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 0x7fe09c13d8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01b000 tx=0x7fe08c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 msgr2=0x7fe09c13d8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0927fc640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1371b0 0x7fe09c13d8b0 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01b000 tx=0x7fe08c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 3127, 2024-09-15T21:18:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.428 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:28.429 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:28.430 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:28.433 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:28.436 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:28.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:28.438 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:28.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08003b9f0 msgr2=0x7fe08003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08003b9f0 0x7fe08003deb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0a0b30 tx=0x7fe09400b040 comp rx=0 tx=0).stop 2024-09-15T21:18:28.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 10 monclient: shutdown 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c0a5040 msgr2=0x7fe09c1cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 --2- 172.21.15.17:0/4211643896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe09c0a5040 0x7fe09c1cfce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0980ae040 tx=0x7fe0980190e0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a2153640 1 -- 172.21.15.17:0/4211643896 reap_dead start 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a2153640 1 -- 172.21.15.17:0/4211643896 reap_dead start 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 shutdown_connections 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 >> 172.21.15.17:0/4211643896 conn(0x7fe09c0aac40 msgr2=0x7fe09c0a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 shutdown_connections 2024-09-15T21:18:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.402+0000 7fe0a3bb6640 1 -- 172.21.15.17:0/4211643896 wait complete. 2024-09-15T21:18:28.440 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:18:28.441 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T21:18:28.441 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:18:28.441 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T21:18:28.441 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T21:18:28.441 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:18:28.441 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:18:28.453 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:18:28.453 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:18:28.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:18:28.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient: get_monmap_and_config 2024-09-15T21:18:28.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient: monmap: 2024-09-15T21:18:28.614 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.614 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:28.611840+0000 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:28.611840+0000 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:28.615 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 Processor -- start 2024-09-15T21:18:28.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 -- start start 2024-09-15T21:18:28.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient: init 2024-09-15T21:18:28.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:18:28.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32e40a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f32e409ffc0 0x7f32e413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): picked mon.noname-a con 0x7f32e409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e40a1380 0x7f32e413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): picked mon.noname-b con 0x7f32e40a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e40a3130 con 0x7f32e40a0890 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f32e4073df0 con 0x7f32e409ffc0 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f32e40a1bf0 con 0x7f32e40a1380 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32eb39f640 10 monclient(hunting): authenticate will time out at 1032.007278s 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e993c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e40a1380 msgr2=0x7f32e413d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e993c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e40a1380 0x7f32e413d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f32e409ffc0 0x7f32e413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e913b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f32e40a0890 0x7f32e40a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46266/0 (socket says 172.21.15.17:46266) 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e913b640 1 -- 172.21.15.17:0/3142285565 learned_addr learned my addr 172.21.15.17:0/3142285565 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): get_auth_request con 0x7f32e409ffc0 auth_method 0 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient(hunting): handle_auth_done global_id 4491 payload 274 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 1 -- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e40a1380 msgr2=0x7f32e413d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 1 --2- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e40a1380 0x7f32e413d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 1 -- 172.21.15.17:0/3142285565 >> v1:172.21.15.17:6789/0 conn(0x7f32e40a0890 legacy=0x7f32e40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 1 monclient: found mon.noname-a 2024-09-15T21:18:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32dbfff640 1 -- 172.21.15.17:0/3142285565 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69814212 0 0) 0x7f32e40a3130 con 0x7f32e40a0890 2024-09-15T21:18:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32e893a640 1 -- 172.21.15.17:0/3142285565 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e409ccd0 con 0x7f32e409ffc0 2024-09-15T21:18:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.606+0000 7f32dbfff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32e893a640 1 --2- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e409ffc0 0x7f32e413aac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097f0 tx=0x7f32d401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfc6ad801a73f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 1 -- 172.21.15.17:0/3142285565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40b0070 con 0x7f32e409ffc0 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: dump: 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 1 -- 172.21.15.17:0/3142285565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40ab600 con 0x7f32e409ffc0 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 5 monclient: authenticate success, global_id 4491 2024-09-15T21:18:28.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 1 -- 172.21.15.17:0/3142285565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40ab9d0 con 0x7f32e409ffc0 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 4 set_mon_vals no callback set 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32dbfff640 10 monclient: dump: 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.628 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 10 monclient: shutdown 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 -- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e409ffc0 msgr2=0x7f32e413aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e409ffc0 0x7f32e413aac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097f0 tx=0x7f32d401c380 comp rx=0 tx=0).stop 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 -- 172.21.15.17:0/3142285565 shutdown_connections 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e40a1380 0x7f32e413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3142285565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e409ffc0 0x7f32e413aac0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 -- 172.21.15.17:0/3142285565 >> 172.21.15.17:0/3142285565 conn(0x7f32e40a66a0 msgr2=0x7f32e40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 -- 172.21.15.17:0/3142285565 shutdown_connections 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 -- 172.21.15.17:0/3142285565 wait complete. 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 10 monclient: build_initial_monmap 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 10 monclient: monmap: 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.610+0000 7f32eb39f640 1 Processor -- start 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- start start 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: init 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e409ffc0 0x7f32e411e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): picked mon.noname-c con 0x7f32e409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40a0890 0x7f32e41cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): picked mon.noname-b con 0x7f32e40a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 0x7f32e411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): picked mon.noname-a con 0x7f32e40a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4140100 con 0x7f32e40a1380 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32e413ff80 con 0x7f32e40a0890 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f32e4140280 con 0x7f32e409ffc0 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient(hunting): authenticate will time out at 1032.012606s 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e913b640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e409ffc0 msgr2=0x7f32e411e860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e913b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e409ffc0 0x7f32e411e860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 0x7f32e411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 0x7f32e411c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33492/0 (socket says 172.21.15.17:33492) 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 -- 172.21.15.17:0/3609795188 learned_addr learned my addr 172.21.15.17:0/3609795188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): get_auth_request con 0x7f32e40a1380 auth_method 0 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40a0890 0x7f32e41cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): get_auth_request con 0x7f32e40a0890 auth_method 0 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e893a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient(hunting): handle_auth_done global_id 4497 payload 824 2024-09-15T21:18:28.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 -- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e409ffc0 msgr2=0x7f32e411e860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e409ffc0 0x7f32e411e860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 -- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40a0890 msgr2=0x7f32e41cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40a0890 0x7f32e41cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient: _finish_hunting 0 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 monclient: found mon.noname-a 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 -- 172.21.15.17:0/3609795188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e411fd80 con 0x7f32e40a1380 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e993c640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 0x7f32e411c4e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00b440 tx=0x7f32dc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x7f32e40a1380 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: dump: 2024-09-15T21:18:28.634 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 5 monclient: authenticate success, global_id 4497 2024-09-15T21:18:28.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc003df0 con 0x7f32e40a1380 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _renew_subs 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc006310 con 0x7f32e40a1380 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e4120060 con 0x7f32e40a1380 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32db7fe640 4 set_mon_vals no callback set 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 10 monclient: dump: 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:28.636 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _renew_subs 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e411f1f0 con 0x7f32e40a1380 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e40a0c70 con 0x7f32e40a1380 2024-09-15T21:18:28.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f32dc01a070 con 0x7f32e40a1380 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f32c803b970 0x7f32c803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f32dc04eb70 con 0x7f32e40a1380 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e913b640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f32c803b970 0x7f32c803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e913b640 10 monclient: get_auth_request con 0x7f32c803b970 auth_method 0 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.614+0000 7f32e913b640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f32c803b970 0x7f32c803de30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32e0001470 tx=0x7f32e0001830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.618+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f32dc01ad70 con 0x7f32e40a1380 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.618+0000 7f32d9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:28.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.618+0000 7f32d9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:28.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:28.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:28.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:28.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f32e406d020 con 0x7f32e40a1380 2024-09-15T21:18:28.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32d9ffb640 1 -- 172.21.15.17:0/3609795188 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f32dc01b8d0 con 0x7f32e40a1380 2024-09-15T21:18:28.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32d9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:28.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32d9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:28.866 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f32c803b970 msgr2=0x7f32c803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f32c803b970 0x7f32c803de30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32e0001470 tx=0x7f32e0001830 comp rx=0 tx=0).stop 2024-09-15T21:18:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 10 monclient: shutdown 2024-09-15T21:18:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 msgr2=0x7f32e411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 0x7f32e411c4e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00b440 tx=0x7f32dc00b900 comp rx=0 tx=0).stop 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 shutdown_connections 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f32c803b970 0x7f32c803de30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32e40a1380 0x7f32e411c4e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40a0890 0x7f32e41cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 --2- 172.21.15.17:0/3609795188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f32e409ffc0 0x7f32e411e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 >> 172.21.15.17:0/3609795188 conn(0x7f32e40a66a0 msgr2=0x7f32e40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 shutdown_connections 2024-09-15T21:18:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:28.858+0000 7f32eb39f640 1 -- 172.21.15.17:0/3609795188 wait complete. 2024-09-15T21:18:31.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient: get_monmap_and_config 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient: build_initial_monmap 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient: monmap: 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:32.016389+0000 2024-09-15T21:18:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:32.016389+0000 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:32.020 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 Processor -- start 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 -- start start 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient: init 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa57c1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa57c1383b0 0x7fa57c13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): picked mon.noname-a con 0x7fa57c1383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c13d2c0 0x7fa57c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): picked mon.noname-b con 0x7fa57c13d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c0a1a20 con 0x7fa57c1378c0 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa57c0a1d00 con 0x7fa57c1383b0 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c0a1fe0 con 0x7fa57c13d2c0 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.010+0000 7fa580f3e640 10 monclient(hunting): authenticate will time out at 1035.411954s 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa57c1378c0 0x7fa57c137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46278/0 (socket says 172.21.15.17:46278) 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57a59c640 1 -- 172.21.15.17:0/2903656910 learned_addr learned my addr 172.21.15.17:0/2903656910 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57ad9d640 1 -- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c13d2c0 msgr2=0x7fa57c13f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57ad9d640 1 --2- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c13d2c0 0x7fa57c13f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 --2- 172.21.15.17:0/2903656910 >> v2:172.21.15.17:3300/0 conn(0x7fa57c1383b0 0x7fa57c13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): get_auth_request con 0x7fa57c1383b0 auth_method 0 2024-09-15T21:18:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 1 -- 172.21.15.17:0/2903656910 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 135143760 0 0) 0x7fa56c0013d0 con 0x7fa57c1378c0 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient(hunting): _negotiate 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient(hunting): my global_id is 4500 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 1 -- 172.21.15.17:0/2903656910 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa568003530 con 0x7fa57c1378c0 2024-09-15T21:18:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient(hunting): handle_auth_done global_id 4503 payload 274 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 -- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c13d2c0 msgr2=0x7fa57c13f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 --2- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c13d2c0 0x7fa57c13f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 1 -- 172.21.15.17:0/2903656910 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3979846237 0 0) 0x7fa568003530 con 0x7fa57c1378c0 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 -- 172.21.15.17:0/2903656910 >> v1:172.21.15.17:6789/0 conn(0x7fa57c1378c0 legacy=0x7fa57c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 monclient: found mon.noname-a 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 -- 172.21.15.17:0/2903656910 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c0a22c0 con 0x7fa57c1383b0 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa579d9b640 1 --2- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1383b0 0x7fa57c13cd80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5640097f0 tx=0x7fa56401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e09a020fbe4d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 1 -- 172.21.15.17:0/2903656910 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c001b80 con 0x7fa57c1383b0 2024-09-15T21:18:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: dump: 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: _finish_auth 0 2024-09-15T21:18:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: _check_auth_tickets 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 1 -- 172.21.15.17:0/2903656910 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5640a54c0 con 0x7fa57c1383b0 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 5 monclient: authenticate success, global_id 4503 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 1 -- 172.21.15.17:0/2903656910 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a5890 con 0x7fa57c1383b0 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 4 set_mon_vals no callback set 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa57959a640 10 monclient: dump: 2024-09-15T21:18:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 10 monclient: shutdown 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 -- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1383b0 msgr2=0x7fa57c13cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 --2- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1383b0 0x7fa57c13cd80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5640097f0 tx=0x7fa56401bf80 comp rx=0 tx=0).stop 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 -- 172.21.15.17:0/2903656910 shutdown_connections 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 --2- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c13d2c0 0x7fa57c13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 --2- 172.21.15.17:0/2903656910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1383b0 0x7fa57c13cd80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 -- 172.21.15.17:0/2903656910 >> 172.21.15.17:0/2903656910 conn(0x7fa57c0aac40 msgr2=0x7fa57c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 -- 172.21.15.17:0/2903656910 shutdown_connections 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.014+0000 7fa580f3e640 1 -- 172.21.15.17:0/2903656910 wait complete. 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: build_initial_monmap 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: monmap: 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 Processor -- start 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- start start 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: init 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 0x7fa57c09bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): picked mon.noname-a con 0x7fa57c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c1383b0 0x7fa57c09c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): picked mon.noname-c con 0x7fa57c1383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa57c13d2c0 0x7fa57c1d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): picked mon.noname-b con 0x7fa57c13d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c143130 con 0x7fa57c1378c0 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c142fb0 con 0x7fa57c13d2c0 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c1432b0 con 0x7fa57c1383b0 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient(hunting): authenticate will time out at 1035.417154s 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa579d9b640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c1383b0 msgr2=0x7fa57c09c460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa579d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c1383b0 0x7fa57c09c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 0x7fa57c09bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 0x7fa57c09bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33510/0 (socket says 172.21.15.17:33510) 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 -- 172.21.15.17:0/3774349909 learned_addr learned my addr 172.21.15.17:0/3774349909 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): get_auth_request con 0x7fa57c1378c0 auth_method 0 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57ad9d640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa57c13d2c0 0x7fa57c1d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57ad9d640 10 monclient(hunting): get_auth_request con 0x7fa57c13d2c0 auth_method 0 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient(hunting): handle_auth_done global_id 4506 payload 824 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 -- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c1383b0 msgr2=0x7fa57c09c460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c1383b0 0x7fa57c09c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 -- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa57c13d2c0 msgr2=0x7fa57c1d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa57c13d2c0 0x7fa57c1d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:18:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 monclient: found mon.noname-a 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 -- 172.21.15.17:0/3774349909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c13cfa0 con 0x7fa57c1378c0 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57ad9d640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa57c13d2c0 0x7fa57c1d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa57a59c640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 0x7fa57c09bf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00cc20 tx=0x7fa56c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c017070 con 0x7fa57c1378c0 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: dump: 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c00ce30 con 0x7fa57c1378c0 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 5 monclient: authenticate success, global_id 4506 2024-09-15T21:18:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _renew_subs 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c0a12f0 con 0x7fa57c1378c0 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c012600 con 0x7fa57c1378c0 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 10 monclient: dump: 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa578d99640 4 set_mon_vals no callback set 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _renew_subs 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c09e7e0 con 0x7fa57c1378c0 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c137ca0 con 0x7fa57c1378c0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa56c01a070 con 0x7fa57c1378c0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa55c03b970 0x7fa55c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fa56c04f850 con 0x7fa57c1378c0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa579d9b640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa55c03b970 0x7fa55c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.018+0000 7fa579d9b640 10 monclient: get_auth_request con 0x7fa55c03b970 auth_method 0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.022+0000 7fa579d9b640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa55c03b970 0x7fa55c03de30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa5640ae000 tx=0x7fa564008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.022+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa57c0a1a20 con 0x7fa57c1378c0 2024-09-15T21:18:32.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.022+0000 7fa572ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:32.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.022+0000 7fa572ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.254+0000 7fa580f3e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.254+0000 7fa580f3e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.254+0000 7fa580f3e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.254+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa57c069270 con 0x7fa57c1378c0 2024-09-15T21:18:32.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa572ffd640 1 -- 172.21.15.17:0/3774349909 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fa56c007a50 con 0x7fa57c1378c0 2024-09-15T21:18:32.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa572ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:32.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa572ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:32.265 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa55c03b970 msgr2=0x7fa55c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa55c03b970 0x7fa55c03de30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa5640ae000 tx=0x7fa564008040 comp rx=0 tx=0).stop 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 10 monclient: shutdown 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 msgr2=0x7fa57c09bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 0x7fa57c09bf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00cc20 tx=0x7fa56c00a430 comp rx=0 tx=0).stop 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 shutdown_connections 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa55c03b970 0x7fa55c03de30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa57c13d2c0 0x7fa57c1d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa57c1383b0 0x7fa57c09c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 --2- 172.21.15.17:0/3774349909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa57c1378c0 0x7fa57c09bf20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:32.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 >> 172.21.15.17:0/3774349909 conn(0x7fa57c0aac40 msgr2=0x7fa57c0a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:32.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 shutdown_connections 2024-09-15T21:18:32.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:32.258+0000 7fa580f3e640 1 -- 172.21.15.17:0/3774349909 wait complete. 2024-09-15T21:18:35.284 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:18:35.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient: get_monmap_and_config 2024-09-15T21:18:35.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient: build_initial_monmap 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient: monmap: 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:35.412274+0000 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:35.412274+0000 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:35.415 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:35.416 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:35.416 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:35.416 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:35.416 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:35.416 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 Processor -- start 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 -- start start 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient: init 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f384c1340c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f384c09d6c0 0x7f384c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:35.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): picked mon.noname-a con 0x7f384c09d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c13b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): picked mon.noname-b con 0x7f384c139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f384c09e1b0 con 0x7f384c1340c0 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f384c09d520 con 0x7f384c09d6c0 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f384c09e490 con 0x7f384c139390 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f3851d92640 10 monclient(hunting): authenticate will time out at 1038.807650s 2024-09-15T21:18:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384bfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 msgr2=0x7f384c13b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c13b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f384c09d6c0 0x7f384c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f384c1340c0 0x7f384c1344a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46282/0 (socket says 172.21.15.17:46282) 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384b7fe640 1 -- 172.21.15.17:0/3968395002 learned_addr learned my addr 172.21.15.17:0/3968395002 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): get_auth_request con 0x7f384c09d6c0 auth_method 0 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.406+0000 7f384affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 10 monclient(hunting): handle_auth_done global_id 4509 payload 274 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 1 -- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 msgr2=0x7f384c13b780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 1 --2- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c13b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 1 -- 172.21.15.17:0/3968395002 >> v1:172.21.15.17:6789/0 conn(0x7f384c1340c0 legacy=0x7f384c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 1 monclient: found mon.noname-a 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 1 -- 172.21.15.17:0/3968395002 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c09e770 con 0x7f384c09d6c0 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384affd640 1 --2- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c09d6c0 0x7f384c138e50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d90 tx=0x7f384001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d059164dd0367ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 1 -- 172.21.15.17:0/3968395002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400ac070 con 0x7f384c09d6c0 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: dump: 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:35.424 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 1 -- 172.21.15.17:0/3968395002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38400a66a0 con 0x7f384c09d6c0 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 5 monclient: authenticate success, global_id 4509 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 1 -- 172.21.15.17:0/3968395002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400a6a70 con 0x7f384c09d6c0 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 4 set_mon_vals no callback set 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f384a7fc640 10 monclient: dump: 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 10 monclient: shutdown 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 -- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c09d6c0 msgr2=0x7f384c138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 --2- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c09d6c0 0x7f384c138e50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d90 tx=0x7f384001c850 comp rx=0 tx=0).stop 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 -- 172.21.15.17:0/3968395002 shutdown_connections 2024-09-15T21:18:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 --2- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c13b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 --2- 172.21.15.17:0/3968395002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c09d6c0 0x7f384c138e50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 -- 172.21.15.17:0/3968395002 >> 172.21.15.17:0/3968395002 conn(0x7f384c0a66a0 msgr2=0x7f384c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 -- 172.21.15.17:0/3968395002 shutdown_connections 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 1 -- 172.21.15.17:0/3968395002 wait complete. 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 10 monclient: build_initial_monmap 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.410+0000 7f3851d92640 10 monclient: monmap: 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:35.430 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:35.430 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:35.430 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:35.430 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:35.430 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:35.430 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 Processor -- start 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- start start 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: init 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c09d6c0 0x7f384c132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): picked mon.noname-b con 0x7f384c09d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 0x7f384c132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:35.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): picked mon.noname-a con 0x7f384c1340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): picked mon.noname-c con 0x7f384c139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f384c13c050 con 0x7f384c1340c0 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f384c13bed0 con 0x7f384c09d6c0 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f384c13c1d0 con 0x7f384c139390 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient(hunting): authenticate will time out at 1038.812628s 2024-09-15T21:18:35.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384bfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 msgr2=0x7f384c11c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 0x7f384c132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 0x7f384c132840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33534/0 (socket says 172.21.15.17:33534) 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 -- 172.21.15.17:0/82736005 learned_addr learned my addr 172.21.15.17:0/82736005 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): get_auth_request con 0x7f384c1340c0 auth_method 0 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:35.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c09d6c0 0x7f384c132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): get_auth_request con 0x7f384c09d6c0 auth_method 0 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient(hunting): handle_auth_done global_id 4512 payload 824 2024-09-15T21:18:35.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 -- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 msgr2=0x7f384c11c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 -- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c09d6c0 msgr2=0x7f384c132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c09d6c0 0x7f384c132300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 monclient: found mon.noname-a 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 -- 172.21.15.17:0/82736005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c133560 con 0x7f384c1340c0 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384affd640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 0x7f384c132840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d90 tx=0x7f38400195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400c0070 con 0x7f384c1340c0 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: dump: 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:35.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 5 monclient: authenticate success, global_id 4512 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _renew_subs 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38400a7690 con 0x7f384c1340c0 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c120060 con 0x7f384c1340c0 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400ac050 con 0x7f384c1340c0 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:35.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 10 monclient: dump: 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:35.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _renew_subs 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c11fd80 con 0x7f384c1340c0 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3849ffb640 4 set_mon_vals no callback set 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c1344a0 con 0x7f384c1340c0 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f38400af070 con 0x7f384c1340c0 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f383803b970 0x7f383803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f38400e9e90 con 0x7f384c1340c0 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f383803b970 0x7f383803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 10 monclient: get_auth_request con 0x7f383803b970 auth_method 0 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.414+0000 7f384b7fe640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f383803b970 0x7f383803de30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f383c000f70 tx=0x7f383c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.418+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3840016560 con 0x7f384c1340c0 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.418+0000 7f3833fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.418+0000 7f3833fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3851d92640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3851d92640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3851d92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f384c06d020 con 0x7f384c1340c0 2024-09-15T21:18:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3833fff640 1 -- 172.21.15.17:0/82736005 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f38400c3360 con 0x7f384c1340c0 2024-09-15T21:18:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3833fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.654+0000 7f3833fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f383803b970 msgr2=0x7f383803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f383803b970 0x7f383803de30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f383c000f70 tx=0x7f383c009040 comp rx=0 tx=0).stop 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 10 monclient: shutdown 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 msgr2=0x7f384c132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 0x7f384c132840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d90 tx=0x7f38400195f0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 shutdown_connections 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f383803b970 0x7f383803de30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f384c139390 0x7f384c11c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384c1340c0 0x7f384c132840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 --2- 172.21.15.17:0/82736005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c09d6c0 0x7f384c132300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 >> 172.21.15.17:0/82736005 conn(0x7f384c0a66a0 msgr2=0x7f384c0a4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 shutdown_connections 2024-09-15T21:18:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:35.658+0000 7f3851d92640 1 -- 172.21.15.17:0/82736005 wait complete. 2024-09-15T21:18:38.682 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient: get_monmap_and_config 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient: build_initial_monmap 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient: monmap: 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:38.811575+0000 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:38.811575+0000 2024-09-15T21:18:38.814 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:38.815 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 Processor -- start 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 -- start start 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient: init 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): picked mon.noname-c con 0x7ff71c1378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:38.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff71c1381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): picked mon.noname-a con 0x7ff71c0a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff71c0a1d00 con 0x7ff71c1381e0 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff71c0a1fe0 con 0x7ff71c0a5210 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff71c0a22c0 con 0x7ff71c1378c0 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff722020640 10 monclient(hunting): authenticate will time out at 1042.206976s 2024-09-15T21:18:38.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42854/0 (socket says 172.21.15.17:42854) 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 learned_addr learned my addr 172.21.15.17:0/3701103950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 10 monclient(hunting): get_auth_request con 0x7ff71c0a5210 auth_method 0 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request con 0x7ff71c1378c0 auth_method 0 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:38.806+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:39.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.006+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:39.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 10 monclient(hunting): get_auth_request con 0x7ff71c0a5210 auth_method 0 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request con 0x7ff71c1378c0 auth_method 0 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:18:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.010+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:18:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 10 monclient(hunting): get_auth_request con 0x7ff71c0a5210 auth_method 0 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request con 0x7ff71c1378c0 auth_method 0 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:39.410+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:18:40.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:40.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 10 monclient(hunting): get_auth_request con 0x7ff71c0a5210 auth_method 0 2024-09-15T21:18:40.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:40.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request con 0x7ff71c1378c0 auth_method 0 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:18:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:40.214+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:Note: switching to '7498493279982dfe87f00616198a5967475f6169'. 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T21:18:40.953 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr: git switch -c 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr:Or undo this operation with: 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr: git switch - 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:40.954 INFO:tasks.workunit.client.0.smithi119.stderr:HEAD is now at 74984932799 Merge pull request #59797 from zdover23/wip-doc-2024-09-15-install-install-storage-cluster 2024-09-15T21:18:40.955 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T21:18:41.002 INFO:tasks.workunit.client.0.smithi119.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T21:18:41.006 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T21:18:41.006 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T21:18:41.746 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T21:18:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): tick 2024-09-15T21:18:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:18:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:18:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:18:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 msgr2=0x7ff71c137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.119:3300/0 conn(0x7ff71c1378c0 0x7ff71c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 -- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 msgr2=0x7ff71c0a56a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 --2- 172.21.15.17:0/3701103950 >> v2:172.21.15.17:3300/0 conn(0x7ff71c0a5210 0x7ff71c0a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 -- 172.21.15.17:0/3701103950 >> v1:172.21.15.119:6789/0 conn(0x7ff71c1381e0 legacy=0x7ff71c0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient: _add_conns ranks=[2,0] 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff6fc005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): picked mon.noname-b con 0x7ff6fc0036d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff6fc006230 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 -- 172.21.15.17:0/3701103950 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff71c0a4490 con 0x7ff6fc006230 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 1 -- 172.21.15.17:0/3701103950 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff71c0a1fe0 con 0x7ff6fc0036d0 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff7197fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:41.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff6fc005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:41.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:41.810+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff6fc005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:41.830 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T21:18:41.880 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T21:18:41.882 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T21:18:41.882 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T21:18:41.926 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T21:18:41.931 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-15T21:18:41.931 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T21:18:41.978 INFO:tasks.workunit:Running workunits matching mon/pool_ops.sh on client.0... 2024-09-15T21:18:41.979 INFO:tasks.workunit:Running workunit mon/pool_ops.sh... 2024-09-15T21:18:41.980 DEBUG:teuthology.orchestra.run.smithi119:workunit test mon/pool_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7498493279982dfe87f00616198a5967475f6169 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/pool_ops.sh 2024-09-15T21:18:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:42.010+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff6fc005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:42.010+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff6fc005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:18:42.030 INFO:tasks.workunit.client.0.smithi119.stderr:+ TEST_POOL=testpool1234 2024-09-15T21:18:42.031 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create testpool1234 8 --autoscale-mode off 2024-09-15T21:18:42.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient: get_monmap_and_config 2024-09-15T21:18:42.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient: build_initial_monmap 2024-09-15T21:18:42.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient: monmap: 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:42.154308+0000 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:42.154308+0000 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:42.156 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:42.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 Processor -- start 2024-09-15T21:18:42.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 -- start start 2024-09-15T21:18:42.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient: init 2024-09-15T21:18:42.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:42.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:18:42.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): picked mon.noname-c con 0x7f8054135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f80540a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): picked mon.noname-b con 0x7f8054135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8054136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80540a1d80 con 0x7f8054136460 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f80540a3600 con 0x7f8054135bf0 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f80540a2060 con 0x7f8054135fd0 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8059965640 10 monclient(hunting): authenticate will time out at 1040.951012s 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f80537fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8054136460 0x7f80540a0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48454/0 (socket says 172.21.15.119:48454) 2024-09-15T21:18:42.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f80537fe640 1 -- 172.21.15.119:0/3102619357 learned_addr learned my addr 172.21.15.119:0/3102619357 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f80527fc640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 msgr2=0x7f80540a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 10 monclient(hunting): get_auth_request con 0x7f8054135bf0 auth_method 0 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f80527fc640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f80540a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:42.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.149+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:42.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f80527fc640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 msgr2=0x7f80540a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:42.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f80527fc640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f80540a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:18:42.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:42.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 10 monclient(hunting): get_auth_request con 0x7f8054135bf0 auth_method 0 2024-09-15T21:18:42.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:42.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:42.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:42.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:18:42.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:42.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:42.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.353+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:18:42.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:42.410+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff6fc005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:42.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:42.410+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff6fc005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 10 monclient(hunting): get_auth_request con 0x7f8054135bf0 auth_method 0 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:18:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:18:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f80527fc640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 msgr2=0x7f80540a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:42.757+0000 7f80527fc640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f80540a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:18:43.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.214+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff6fc005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.214+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff6fc005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:18:43.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111757269 0 0) 0x7f80540a1d80 con 0x7f8054136460 2024-09-15T21:18:43.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:18:43.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): my global_id is 14158 2024-09-15T21:18:43.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f803c003530 con 0x7f8054136460 2024-09-15T21:18:43.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1332187662 0 0) 0x7f803c003530 con 0x7f8054136460 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 msgr2=0x7f80540a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f80540a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 msgr2=0x7f805409fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 --2- 172.21.15.119:0/3102619357 >> v2:172.21.15.119:3300/0 conn(0x7f8054135bf0 0x7f805409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80540a2340 con 0x7f8054136460 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f804c001a80 con 0x7f8054136460 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: dump: 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.488 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 >> v1:172.21.15.119:6789/0 conn(0x7f8054136460 legacy=0x7f80540a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0039c0 0x7f803c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): picked mon.b con 0x7f803c0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f803c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): picked mon.c con 0x7f8054135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f803c004da0 0x7f803c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): picked mon.a con 0x7f803c004da0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f80540a3600 con 0x7f803c004da0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f80540a2060 con 0x7f803c0039c0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f803c00bce0 con 0x7f8054135fd0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8051ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80527fc640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 msgr2=0x7f803c004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80527fc640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f803c004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0039c0 0x7f803c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): get_auth_request con 0x7f803c0039c0 auth_method 0 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:43.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f803c004da0 0x7f803c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): get_auth_request con 0x7f803c004da0 auth_method 0 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f80537fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient(hunting): handle_auth_done global_id 14158 payload 293 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 msgr2=0x7f803c004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8054135fd0 0x7f803c004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f803c004da0 msgr2=0x7f803c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f803c004da0 0x7f803c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 monclient: found mon.b 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 1 -- 172.21.15.119:0/3102619357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c003530 con 0x7f803c0039c0 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8052ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8059965640 5 monclient: authenticate success, global_id 14158 2024-09-15T21:18:43.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.477+0000 7f8059965640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.478+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192436153 0 0) 0x7ff71c0a4490 con 0x7ff6fc006230 2024-09-15T21:18:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.478+0000 7ff71a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:18:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.478+0000 7ff71a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.478+0000 7ff71a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.478+0000 7ff71a7fc640 10 monclient(hunting): my global_id is 4515 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.478+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff708003530 con 0x7ff6fc006230 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2275706394 0 0) 0x7ff708003530 con 0x7ff6fc006230 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff6fc005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff6fc005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc00b080 con 0x7ff6fc006230 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff704001740 con 0x7ff6fc006230 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: dump: 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 >> v1:172.21.15.17:6789/0 conn(0x7ff6fc006230 legacy=0x7ff6fc0086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 1 -- 172.21.15.17:0/3701103950 reap_dead start 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff7080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): picked mon.c con 0x7ff6fc0036d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff708004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): picked mon.a con 0x7ff7080040d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff70800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71b7fe640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff7080039c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): picked mon.b con 0x7ff708004e70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c0a1fe0 con 0x7ff7080040d0 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff7080039c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff708005690 con 0x7ff708004e70 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff70800bf10 con 0x7ff6fc0036d0 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff708004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): get_auth_request con 0x7ff7080040d0 auth_method 0 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff70800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): get_auth_request con 0x7ff708004e70 auth_method 0 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.482+0000 7ff71affd640 10 monclient(hunting): handle_auth_done global_id 4515 payload 293 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff7080039c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff7080039c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 msgr2=0x7ff70800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff70800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 1 monclient: found mon.a 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 1 -- 172.21.15.17:0/3701103950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff708003530 con 0x7ff7080040d0 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff71affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff722020640 5 monclient: authenticate success, global_id 4515 2024-09-15T21:18:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.486+0000 7ff722020640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:43.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.481+0000 7f8052ffd640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0039c0 0x7f803c003de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8044010950 tx=0x7f804401ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f66805ce757efe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.481+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80440ab070 con 0x7f803c0039c0 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.481+0000 7f8051ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 10 monclient: dump: 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.511 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80440a59c0 con 0x7f803c0039c0 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 1 -- 172.21.15.119:0/3102619357 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804400f040 con 0x7f803c0039c0 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 4 set_mon_vals no callback set 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8051ffb640 10 monclient: dump: 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:43.512 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 10 monclient: shutdown 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 -- 172.21.15.119:0/3102619357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0039c0 msgr2=0x7f803c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 --2- 172.21.15.119:0/3102619357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0039c0 0x7f803c003de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8044010950 tx=0x7f804401ccd0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f80537fe640 1 -- 172.21.15.119:0/3102619357 reap_dead start 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 -- 172.21.15.119:0/3102619357 shutdown_connections 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 -- 172.21.15.119:0/3102619357 >> 172.21.15.119:0/3102619357 conn(0x7f80540a66e0 msgr2=0x7f80540a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 -- 172.21.15.119:0/3102619357 shutdown_connections 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 -- 172.21.15.119:0/3102619357 wait complete. 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 10 monclient: build_initial_monmap 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 10 monclient: monmap: 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:43.514 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 Processor -- start 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 1 -- start start 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.485+0000 7f8059965640 10 monclient: init 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f803c0039c0 0x7f8054130c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): picked mon.noname-c con 0x7f803c0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80541cd9d0 0x7f8054131150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): picked mon.noname-a con 0x7f80541cd9d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 0x7f8054070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): picked mon.noname-b con 0x7f8054070080 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f80540a1820 con 0x7f80541cd9d0 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f80540a16a0 con 0x7f8054070080 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f805413e2a0 con 0x7f803c0039c0 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient(hunting): authenticate will time out at 1042.288681s 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8052ffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f803c0039c0 msgr2=0x7f8054130c10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8052ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f803c0039c0 0x7f8054130c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 0x7f8054070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80541cd9d0 0x7f8054131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 0x7f8054070500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39966/0 (socket says 172.21.15.119:39966) 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 -- 172.21.15.119:0/1348009540 learned_addr learned my addr 172.21.15.119:0/1348009540 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): get_auth_request con 0x7f8054070080 auth_method 0 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): get_auth_request con 0x7f80541cd9d0 auth_method 0 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient(hunting): handle_auth_done global_id 14161 payload 824 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 -- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f803c0039c0 msgr2=0x7f8054130c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f803c0039c0 0x7f8054130c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 -- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80541cd9d0 msgr2=0x7f8054131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80541cd9d0 0x7f8054131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 monclient: found mon.noname-b 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 -- 172.21.15.119:0/1348009540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805409d240 con 0x7f8054070080 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80527fc640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80541cd9d0 0x7f8054131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80537fe640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 0x7f8054070500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f805409fe40 tx=0x7f804c007b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c00ab00 con 0x7f8054070080 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: dump: 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.519 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804c018070 con 0x7f8054070080 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:43.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c0133f0 con 0x7f8054070080 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 5 monclient: authenticate success, global_id 14161 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _renew_subs 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805411ca60 con 0x7f8054070080 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 10 monclient: dump: 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.521 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f80517fa640 4 set_mon_vals no callback set 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _renew_subs 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805411d5c0 con 0x7f8054070080 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f804c013590 con 0x7f8054070080 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f804003b970 0x7f804003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f804c04fcd0 con 0x7f8054070080 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8052ffd640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f804003b970 0x7f804003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805409fe70 con 0x7f8054070080 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.489+0000 7f8052ffd640 10 monclient: get_auth_request con 0x7f804003b970 auth_method 0 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.493+0000 7f8052ffd640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f804003b970 0x7f804003de30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80440b0000 tx=0x7f8044010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.497+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f804c01ad20 con 0x7f8054070080 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.497+0000 7f803bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.497+0000 7f803bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71affd640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff708004810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff704007f50 tx=0x7ff70401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b87733e4520a975 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b2070 con 0x7ff7080040d0 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: dump: 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.525 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7040a57e0 con 0x7ff7080040d0 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 1 -- 172.21.15.17:0/3701103950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704007ba0 con 0x7ff7080040d0 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:43.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 4 set_mon_vals no callback set 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff71a7fc640 10 monclient: dump: 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.527 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 10 monclient: shutdown 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 -- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 msgr2=0x7ff708004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff708004810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff704007f50 tx=0x7ff70401c980 comp rx=0 tx=0).stop 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 -- 172.21.15.17:0/3701103950 shutdown_connections 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff70800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff708004810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 --2- 172.21.15.17:0/3701103950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff7080039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 -- 172.21.15.17:0/3701103950 >> 172.21.15.17:0/3701103950 conn(0x7ff71c0aac40 msgr2=0x7ff71c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 -- 172.21.15.17:0/3701103950 shutdown_connections 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.490+0000 7ff722020640 1 -- 172.21.15.17:0/3701103950 wait complete. 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: build_initial_monmap 2024-09-15T21:18:43.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: monmap: 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 Processor -- start 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 -- start start 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: init 2024-09-15T21:18:43.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff71c121dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): picked mon.noname-c con 0x7ff6fc0036d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff71c122300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): picked mon.noname-a con 0x7ff7080040d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff71c122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): picked mon.noname-b con 0x7ff708004e70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c06f340 con 0x7ff7080040d0 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c06f1c0 con 0x7ff708004e70 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff71c13ad70 con 0x7ff6fc0036d0 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient(hunting): authenticate will time out at 1046.894254s 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71b7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff71c121dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff71c121dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff71c122300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff71c122300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42886/0 (socket says 172.21.15.17:42886) 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 -- 172.21.15.17:0/607695604 learned_addr learned my addr 172.21.15.17:0/607695604 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): get_auth_request con 0x7ff7080040d0 auth_method 0 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff71c122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): get_auth_request con 0x7ff708004e70 auth_method 0 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient(hunting): handle_auth_done global_id 4521 payload 824 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 -- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 msgr2=0x7ff71c121dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff71c121dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 -- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 msgr2=0x7ff71c122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff71c122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 monclient: found mon.noname-a 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 -- 172.21.15.17:0/607695604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c125be0 con 0x7ff7080040d0 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff71affd640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff71c122300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff704007f50 tx=0x7ff7040197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b2040 con 0x7ff7080040d0 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: dump: 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7040bd070 con 0x7ff7080040d0 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 5 monclient: authenticate success, global_id 4521 2024-09-15T21:18:43.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: _renew_subs 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c0a9370 con 0x7ff7080040d0 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b8450 con 0x7ff7080040d0 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff713fff640 10 monclient: dump: 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff719ffb640 4 set_mon_vals no callback set 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: _renew_subs 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.494+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c124d70 con 0x7ff7080040d0 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff722020640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff722020640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff722020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c13aef0 con 0x7ff7080040d0 2024-09-15T21:18:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff7040b85f0 con 0x7ff7080040d0 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff713fff640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6fc03c2c0 0x7ff6fc03e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7ff7040e9a70 con 0x7ff7080040d0 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6fc03c2c0 0x7ff6fc03e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff71b7fe640 10 monclient: get_auth_request con 0x7ff6fc03c2c0 auth_method 0 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.498+0000 7ff71b7fe640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6fc03c2c0 0x7ff6fc03e780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff714009880 tx=0x7ff71400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.502+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff704016730 con 0x7ff7080040d0 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.502+0000 7ff713fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.502+0000 7ff713fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:43.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.689+0000 7f8059965640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T21:18:43.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.689+0000 7f8059965640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T21:18:43.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.689+0000 7f8059965640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:43.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:43.689+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) -- 0x7f805406a830 con 0x7f8054070080 2024-09-15T21:18:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff722020640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:18:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff722020640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff722020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff71c13aef0 con 0x7ff7080040d0 2024-09-15T21:18:43.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff713fff640 1 -- 172.21.15.17:0/607695604 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7ff7040e9e20 con 0x7ff7080040d0 2024-09-15T21:18:43.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff713fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:18:43.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.742+0000 7ff713fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:43.751 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:18:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6fc03c2c0 msgr2=0x7ff6fc03e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6fc03c2c0 0x7ff6fc03e780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff714009880 tx=0x7ff71400b040 comp rx=0 tx=0).stop 2024-09-15T21:18:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 10 monclient: shutdown 2024-09-15T21:18:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 msgr2=0x7ff71c122300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff71c122300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff704007f50 tx=0x7ff7040197c0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 shutdown_connections 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6fc03c2c0 0x7ff6fc03e780 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff708004e70 0x7ff71c122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080040d0 0x7ff71c122300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 --2- 172.21.15.17:0/607695604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff6fc0036d0 0x7ff71c121dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 >> 172.21.15.17:0/607695604 conn(0x7ff71c0aac40 msgr2=0x7ff71c0a98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 shutdown_connections 2024-09-15T21:18:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.746+0000 7ff722020640 1 -- 172.21.15.17:0/607695604 wait complete. 2024-09-15T21:18:43.772 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:18:43.772 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:18:43.772 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:18:43.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient: get_monmap_and_config 2024-09-15T21:18:43.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient: build_initial_monmap 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient: monmap: 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:43.887212+0000 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:43.887212+0000 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:43.890 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:43.891 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 1 Processor -- start 2024-09-15T21:18:43.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 1 -- start start 2024-09-15T21:18:43.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient: init 2024-09-15T21:18:43.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1fe81340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1fe809d6c0 0x7f1fe8138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient(hunting): picked mon.noname-c con 0x7f1fe809d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1fe8139390 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe809e1b0 con 0x7f1fe8139390 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe809e490 con 0x7f1fe81340c0 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe809d520 con 0x7f1fe809d6c0 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1feffa3640 10 monclient(hunting): authenticate will time out at 1047.283166s 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fee540640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1fe8139390 0x7f1fe813b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39896/0 (socket says 172.21.15.17:39896) 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fee540640 1 -- 172.21.15.17:0/940206414 learned_addr learned my addr 172.21.15.17:0/940206414 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:43.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 1 --2- 172.21.15.17:0/940206414 >> v2:172.21.15.119:3300/0 conn(0x7f1fe809d6c0 0x7f1fe8138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): get_auth_request con 0x7f1fe809d6c0 auth_method 0 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793391770 0 0) 0x7f1fe809e1b0 con 0x7f1fe8139390 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fecd3d640 10 monclient(hunting): _negotiate 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fecd3d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fecd3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fecd3d640 10 monclient(hunting): my global_id is 4524 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd4003530 con 0x7f1fe8139390 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.882+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103823699 0 0) 0x7f1fe809e490 con 0x7f1fe81340c0 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): _negotiate 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): my global_id is 14170 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe809e1b0 con 0x7f1fe81340c0 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1431472281 0 0) 0x7f1fd4003530 con 0x7f1fe8139390 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 >> v2:172.21.15.119:3300/0 conn(0x7f1fe809d6c0 msgr2=0x7f1fe8138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 --2- 172.21.15.17:0/940206414 >> v2:172.21.15.119:3300/0 conn(0x7f1fe809d6c0 0x7f1fe8138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 >> v1:172.21.15.119:6789/0 conn(0x7f1fe81340c0 legacy=0x7f1fe81344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:18:43.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe809e770 con 0x7f1fe8139390 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f1fe0001a80 con 0x7f1fe8139390 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 5 monclient: authenticate success, global_id 4524 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: dump: 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 >> v1:172.21.15.17:6789/0 conn(0x7f1fe8139390 legacy=0x7f1fe813b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fd4003c90 0x7f1fd4004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): picked mon.b con 0x7f1fd4003c90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:43.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fd4004c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): picked mon.a con 0x7f1fd4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fd4005380 0x7f1fd400dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): picked mon.c con 0x7f1fd4005380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe809d520 con 0x7f1fd4004810 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd400e4e0 con 0x7f1fd4003c90 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd400e6e0 con 0x7f1fd4005380 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fee540640 1 -- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fd4005380 msgr2=0x7f1fd400dc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fee540640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fd4005380 0x7f1fd400dc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fd4004c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): get_auth_request con 0x7f1fd4004810 auth_method 0 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fedd3f640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fd4003c90 0x7f1fd4004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fedd3f640 10 monclient(hunting): get_auth_request con 0x7f1fd4003c90 auth_method 0 2024-09-15T21:18:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fedd3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fedd3f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fedd3f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient(hunting): handle_auth_done global_id 4524 payload 293 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 -- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fd4005380 msgr2=0x7f1fd400dc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fd4005380 0x7f1fd400dc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 -- 172.21.15.17:0/940206414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fd4003c90 msgr2=0x7f1fd4004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fd4003c90 0x7f1fd4004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 monclient: found mon.a 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fee540640 1 -- 172.21.15.17:0/940206414 reap_dead start 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 -- 172.21.15.17:0/940206414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd4003530 con 0x7f1fd4004810 2024-09-15T21:18:43.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fedd3f640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fd4003c90 0x7f1fd4004090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fed53e640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fd4004c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa60 tx=0x7f1fd801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218edafacd5a21a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80b2070 con 0x7f1fd4004810 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: dump: 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.911 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd80a59b0 con 0x7f1fd4004810 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 4 set_mon_vals no callback set 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 1 -- 172.21.15.17:0/940206414 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800b040 con 0x7f1fd4004810 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1fecd3d640 10 monclient: dump: 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 10 monclient: shutdown 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 -- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 msgr2=0x7f1fd4004c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fd4004c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa60 tx=0x7f1fd801cc10 comp rx=0 tx=0).stop 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 -- 172.21.15.17:0/940206414 shutdown_connections 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 --2- 172.21.15.17:0/940206414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fd4004c70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 -- 172.21.15.17:0/940206414 >> 172.21.15.17:0/940206414 conn(0x7f1fe80a66a0 msgr2=0x7f1fe80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 -- 172.21.15.17:0/940206414 shutdown_connections 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.886+0000 7f1feffa3640 1 -- 172.21.15.17:0/940206414 wait complete. 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: build_initial_monmap 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: monmap: 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 Processor -- start 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- start start 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: init 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fe81cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): picked mon.noname-a con 0x7f1fd4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe8073720 0x7f1fe812f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): picked mon.noname-c con 0x7f1fe8073720 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe812fce0 0x7f1fe8071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): picked mon.noname-b con 0x7f1fe812fce0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe80a0890 con 0x7f1fd4004810 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe80a2770 con 0x7f1fe812fce0 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe80a0a10 con 0x7f1fe8073720 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): authenticate will time out at 1047.289478s 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe8073720 msgr2=0x7f1fe812f7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fe81cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fe81cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42912/0 (socket says 172.21.15.17:42912) 2024-09-15T21:18:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 -- 172.21.15.17:0/1818220443 learned_addr learned my addr 172.21.15.17:0/1818220443 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): get_auth_request con 0x7f1fd4004810 auth_method 0 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe8073720 0x7f1fe812f7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fee540640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe812fce0 0x7f1fe8071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fee540640 10 monclient(hunting): get_auth_request con 0x7f1fe812fce0 auth_method 0 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fee540640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fee540640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fee540640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient(hunting): handle_auth_done global_id 4527 payload 824 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe8073720 msgr2=0x7f1fe812f7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe8073720 0x7f1fe812f7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe812fce0 msgr2=0x7f1fe8071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe812fce0 0x7f1fe8071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient: _finish_hunting 0 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 monclient: found mon.noname-a 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe809d0f0 con 0x7f1fd4004810 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fee540640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe812fce0 0x7f1fe8071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fedd3f640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fe81cb940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e7b0 tx=0x7f1fe400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4001d40 con 0x7f1fd4004810 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: dump: 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe4018070 con 0x7f1fd4004810 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40052f0 con 0x7f1fd4004810 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 5 monclient: authenticate success, global_id 4527 2024-09-15T21:18:43.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: _renew_subs 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe8072bc0 con 0x7f1fd4004810 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 10 monclient: dump: 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.921 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fdffff640 4 set_mon_vals no callback set 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: _renew_subs 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe8071680 con 0x7f1fd4004810 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 0x7f1fe8136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81cc8a0 con 0x7f1fe80718b0 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 0x7f1fe8136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient: get_auth_request con 0x7f1fe80718b0 auth_method 0 2024-09-15T21:18:43.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 10 monclient(hunting): handle_auth_done global_id 4530 payload 824 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fed53e640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fe80a4d30 con 0x7f1fe80718b0 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.890+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1fe400c6c0 con 0x7f1fd4004810 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fed53e640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 0x7f1fe8136e40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa30 tx=0x7f1fd4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03bb40 0x7f1fcc03e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f1fe404f5f0 con 0x7f1fd4004810 2024-09-15T21:18:43.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80b2040 con 0x7f1fe80718b0 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 10 monclient: dump: 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.924 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fee540640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03bb40 0x7f1fcc03e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fee540640 10 monclient: get_auth_request con 0x7f1fcc03bb40 auth_method 0 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1fd8004020 con 0x7f1fe80718b0 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 msgr2=0x7f1fe8136e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 0x7f1fe8136e40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa30 tx=0x7f1fd4005840 comp rx=0 tx=0).stop 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 msgr2=0x7f1fe8136e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fde7fc640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe80718b0 0x7f1fe8136e40 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa30 tx=0x7f1fd4005840 comp rx=0 tx=0).stop 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.894+0000 7f1fee540640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03bb40 0x7f1fcc03e000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1fe809ebc0 tx=0x7f1fe0007c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1feffa3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1feffa3640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 0x7f1fe8137380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1feffa3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe80a0d40 con 0x7f1fe8071c90 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 0x7f1fe8137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient: get_auth_request con 0x7f1fe8071c90 auth_method 0 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 10 monclient(hunting): handle_auth_done global_id 4533 payload 824 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 1 -- 172.21.15.17:0/1818220443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1fe806d020 con 0x7f1fe8071c90 2024-09-15T21:18:43.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fed53e640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 0x7f1fe8137380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa60 tx=0x7f1fd800a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80b2040 con 0x7f1fe8071c90 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 10 monclient: dump: 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7f1fd8003bd0 con 0x7f1fe8071c90 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 msgr2=0x7f1fe8137380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 0x7f1fe8137380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa60 tx=0x7f1fd800a2a0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 msgr2=0x7f1fe8137380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.898+0000 7f1fde7fc640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe8071c90 0x7f1fe8137380 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800aa60 tx=0x7f1fd800a2a0 comp rx=0 tx=0).stop 2024-09-15T21:18:43.928 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 18, 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 89519, 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:43.929 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:43.930 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:43.931 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:43.932 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:43.933 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:43.934 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:43.937 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:43.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:43.940 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:43.941 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:43.942 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:43.942 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:43.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:43.942 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:43.942 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:43.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03bb40 msgr2=0x7f1fcc03e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03bb40 0x7f1fcc03e000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1fe809ebc0 tx=0x7f1fe0007c80 comp rx=0 tx=0).stop 2024-09-15T21:18:43.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 10 monclient: shutdown 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 msgr2=0x7f1fe81cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 --2- 172.21.15.17:0/1818220443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd4004810 0x7f1fe81cb940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e7b0 tx=0x7f1fe400ec70 comp rx=0 tx=0).stop 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1fee540640 1 -- 172.21.15.17:0/1818220443 reap_dead start 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1fee540640 1 -- 172.21.15.17:0/1818220443 reap_dead start 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 shutdown_connections 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 >> 172.21.15.17:0/1818220443 conn(0x7f1fe80a66a0 msgr2=0x7f1fe809c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 shutdown_connections 2024-09-15T21:18:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:43.902+0000 7f1feffa3640 1 -- 172.21.15.17:0/1818220443 wait complete. 2024-09-15T21:18:43.944 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:18:43.945 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:18:43.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:18:44.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient: get_monmap_and_config 2024-09-15T21:18:44.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient: build_initial_monmap 2024-09-15T21:18:44.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient: monmap: 2024-09-15T21:18:44.062 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:44.062 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:18:44.060162+0000 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:18:44.060162+0000 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:44.063 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 Processor -- start 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 -- start start 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient: init 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 0x7fb49c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): picked mon.noname-b con 0x7fb49c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb49c09e8c0 0x7fb49c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): picked mon.noname-c con 0x7fb49c09e8c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb49c13be60 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb49c0a31c0 con 0x7fb49c13be60 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb49c0a1ac0 con 0x7fb49c09e8c0 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb49c0a16f0 con 0x7fb49c0a0890 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a3003640 10 monclient(hunting): authenticate will time out at 1047.455596s 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a0d9f640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 msgr2=0x7fb49c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a0d9f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 0x7fb49c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a15a0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb49c13be60 0x7fb49c13c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39898/0 (socket says 172.21.15.17:39898) 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.054+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3726662858 learned_addr learned my addr 172.21.15.17:0/3726662858 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:18:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3726662858 >> v2:172.21.15.119:3300/0 conn(0x7fb49c09e8c0 0x7fb49c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): get_auth_request con 0x7fb49c09e8c0 auth_method 0 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490318137 0 0) 0x7fb49c0a31c0 con 0x7fb49c13be60 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): my global_id is 4536 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb484003530 con 0x7fb49c13be60 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2370241281 0 0) 0x7fb484003530 con 0x7fb49c13be60 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 msgr2=0x7fb49c0a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 0x7fb49c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 >> v2:172.21.15.119:3300/0 conn(0x7fb49c09e8c0 msgr2=0x7fb49c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 --2- 172.21.15.17:0/3726662858 >> v2:172.21.15.119:3300/0 conn(0x7fb49c09e8c0 0x7fb49c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c09d640 con 0x7fb49c13be60 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb494001700 con 0x7fb49c13be60 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: dump: 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.071 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 >> v1:172.21.15.17:6789/0 conn(0x7fb49c13be60 legacy=0x7fb49c13c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4840039c0 0x7fb484003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): picked mon.b con 0x7fb4840039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb484004810 0x7fb484004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): picked mon.a con 0x7fb484004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 0x7fb48400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): picked mon.c con 0x7fb49c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:44.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c0a1ac0 con 0x7fb484004810 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c0a16f0 con 0x7fb4840039c0 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb484005580 con 0x7fb49c0a0890 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 msgr2=0x7fb48400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a15a0640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 0x7fb48400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb484004810 0x7fb484004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): get_auth_request con 0x7fb484004810 auth_method 0 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4840039c0 0x7fb484003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): get_auth_request con 0x7fb4840039c0 auth_method 0 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient(hunting): handle_auth_done global_id 4536 payload 293 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 -- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 msgr2=0x7fb48400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb49c0a0890 0x7fb48400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 -- 172.21.15.17:0/3726662858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4840039c0 msgr2=0x7fb484003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4840039c0 0x7fb484003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 monclient: found mon.a 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 -- 172.21.15.17:0/3726662858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb484003530 con 0x7fb484004810 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient: _finish_auth 0 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 5 monclient: authenticate success, global_id 4536 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:44.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb484004810 0x7fb484004c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb490009fc0 tx=0x7fb49001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d637dfd603b4a215 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900b2070 con 0x7fb484004810 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: dump: 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4900a5c60 con 0x7fb484004810 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 1 -- 172.21.15.17:0/3726662858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490009c10 con 0x7fb484004810 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 4 set_mon_vals no callback set 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb49b7fe640 10 monclient: dump: 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.083 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 10 monclient: shutdown 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 1 -- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb484004810 msgr2=0x7fb484004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a3003640 1 --2- 172.21.15.17:0/3726662858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb484004810 0x7fb484004c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb490009fc0 tx=0x7fb49001cc70 comp rx=0 tx=0).stop 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.058+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3726662858 reap_dead start 2024-09-15T21:18:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3726662858 shutdown_connections 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3726662858 >> 172.21.15.17:0/3726662858 conn(0x7fb49c0a66a0 msgr2=0x7fb49c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3726662858 shutdown_connections 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3726662858 wait complete. 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: build_initial_monmap 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: monmap: 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:44.085 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 Processor -- start 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- start start 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: init 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb484004810 0x7fb49c072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): picked mon.noname-c con 0x7fb484004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c070670 0x7fb49c06f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): picked mon.noname-b con 0x7fb49c070670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:44.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 0x7fb49c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): picked mon.noname-a con 0x7fb49c06f770 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c09ce50 con 0x7fb49c06f770 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c09ccd0 con 0x7fb49c070670 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb49c1448d0 con 0x7fb484004810 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb484004810 msgr2=0x7fb49c072e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): authenticate will time out at 1047.461545s 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb484004810 0x7fb49c072e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 0x7fb49c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 0x7fb49c06fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42942/0 (socket says 172.21.15.17:42942) 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3204325039 learned_addr learned my addr 172.21.15.17:0/3204325039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): get_auth_request con 0x7fb49c06f770 auth_method 0 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c070670 0x7fb49c06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb49bfff640 10 monclient(hunting): get_auth_request con 0x7fb49c070670 auth_method 0 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb49bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb49bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb49bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient(hunting): handle_auth_done global_id 4539 payload 824 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb484004810 msgr2=0x7fb49c072e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb484004810 0x7fb49c072e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c070670 msgr2=0x7fb49c06f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c070670 0x7fb49c06f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient: _finish_hunting 0 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 monclient: found mon.noname-a 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c071bc0 con 0x7fb49c06f770 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a15a0640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 0x7fb49c06fc20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb494001230 tx=0x7fb49400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4940074e0 con 0x7fb49c06f770 2024-09-15T21:18:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: dump: 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 5 monclient: authenticate success, global_id 4539 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4940047f0 con 0x7fb49c06f770 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: _renew_subs 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c072190 con 0x7fb49c06f770 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494007680 con 0x7fb49c06f770 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 10 monclient: dump: 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: _renew_subs 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb49affd640 4 set_mon_vals no callback set 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c071490 con 0x7fb49c06f770 2024-09-15T21:18:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 0x7fb49c1d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c09d0f0 con 0x7fb49c1d48a0 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 0x7fb49c1d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient: get_auth_request con 0x7fb49c1d48a0 auth_method 0 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.062+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb494004bc0 con 0x7fb49c06f770 2024-09-15T21:18:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_done global_id 14182 payload 824 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4a0d9f640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb49c0a1160 con 0x7fb49c1d48a0 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb48803b970 0x7fb48803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fb49404eee0 con 0x7fb49c06f770 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4a0d9f640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 0x7fb49c1d4c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb48c003990 tx=0x7fb48c009a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb48803b970 0x7fb48803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb49bfff640 10 monclient: get_auth_request con 0x7fb48803b970 auth_method 0 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c016070 con 0x7fb49c1d48a0 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 10 monclient: dump: 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb49bfff640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb48803b970 0x7fb48803de30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4900af000 tx=0x7fb4900a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb48c00e660 con 0x7fb49c1d48a0 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 msgr2=0x7fb49c1d4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 0x7fb49c1d4c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb48c003990 tx=0x7fb48c009a00 comp rx=0 tx=0).stop 2024-09-15T21:18:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 msgr2=0x7fb49c1d4c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.066+0000 7fb4997fa640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d48a0 0x7fb49c1d4c80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a3003640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a3003640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 0x7fb49c13f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a3003640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c06d020 con 0x7fb49c1d51c0 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a0d9f640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 0x7fb49c13f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a0d9f640 10 monclient: get_auth_request con 0x7fb49c1d51c0 auth_method 0 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a0d9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a0d9f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.070+0000 7fb4a0d9f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a0d9f640 10 monclient(hunting): handle_auth_done global_id 14185 payload 824 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a0d9f640 1 -- 172.21.15.17:0/3204325039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb49c1d55a0 con 0x7fb49c1d51c0 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a0d9f640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 0x7fb49c13f500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb48c018000 tx=0x7fb48c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c016040 con 0x7fb49c1d51c0 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 10 monclient: dump: 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4700 (secure 0 0 0) 0x7fb48c005b00 con 0x7fb49c1d51c0 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 msgr2=0x7fb49c13f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 0x7fb49c13f500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb48c018000 tx=0x7fb48c005a90 comp rx=0 tx=0).stop 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 msgr2=0x7fb49c13f500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:18:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4997fa640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb49c1d51c0 0x7fb49c13f500 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb48c018000 tx=0x7fb48c005a90 comp rx=0 tx=0).stop 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 18, 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 18795, 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:18:44.099 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:44.100 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:44.101 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:18:44.102 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:18:44.103 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:44.104 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:18:44.106 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:44.107 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:44.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:18:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:18:44.111 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb48803b970 msgr2=0x7fb48803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb48803b970 0x7fb48803de30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4900af000 tx=0x7fb4900a7040 comp rx=0 tx=0).stop 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 10 monclient: shutdown 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 msgr2=0x7fb49c06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 0x7fb49c06fc20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb494001230 tx=0x7fb49400c890 comp rx=0 tx=0).stop 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a15a0640 1 -- 172.21.15.17:0/3204325039 reap_dead start 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 shutdown_connections 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 --2- 172.21.15.17:0/3204325039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49c06f770 0x7fb49c06fc20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 >> 172.21.15.17:0/3204325039 conn(0x7fb49c0a66a0 msgr2=0x7fb49c1390f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 shutdown_connections 2024-09-15T21:18:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:18:44.074+0000 7fb4a3003640 1 -- 172.21.15.17:0/3204325039 wait complete. 2024-09-15T21:18:44.114 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:18:44.114 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:18:44.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.493+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' created v20) ==== 152+0+0 (secure 0 0 0) 0x7f804c01d460 con 0x7f8054070080 2024-09-15T21:18:44.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.493+0000 7f803bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T21:18:44.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.493+0000 7f803bfff640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' created 2024-09-15T21:18:44.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f8059965640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T21:18:44.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f8059965640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T21:18:44.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f8059965640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) -- 0x7f80540726a0 con 0x7f8054070080 2024-09-15T21:18:44.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f803bfff640 1 -- 172.21.15.119:0/1348009540 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' already exists v20) ==== 159+0+0 (secure 0 0 0) 0x7f804c013c50 con 0x7f8054070080 2024-09-15T21:18:44.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f803bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T21:18:44.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.597+0000 7f803bfff640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' already exists 2024-09-15T21:18:44.606 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'testpool1234' already exists 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f804003b970 msgr2=0x7f804003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f804003b970 0x7f804003de30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80440b0000 tx=0x7f8044010040 comp rx=0 tx=0).stop 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 10 monclient: shutdown 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 msgr2=0x7f8054070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 0x7f8054070500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f805409fe40 tx=0x7f804c007b00 comp rx=0 tx=0).stop 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 shutdown_connections 2024-09-15T21:18:44.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f804003b970 0x7f804003de30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8054070080 0x7f8054070500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80541cd9d0 0x7f8054131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 --2- 172.21.15.119:0/1348009540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f803c0039c0 0x7f8054130c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 >> 172.21.15.119:0/1348009540 conn(0x7f80540a66e0 msgr2=0x7f8054136e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:44.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 shutdown_connections 2024-09-15T21:18:44.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.601+0000 7f8059965640 1 -- 172.21.15.119:0/1348009540 wait complete. 2024-09-15T21:18:44.623 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num_min 2 2024-09-15T21:18:44.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient: get_monmap_and_config 2024-09-15T21:18:44.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient: build_initial_monmap 2024-09-15T21:18:44.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient: monmap: 2024-09-15T21:18:44.746 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:44.746 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:44.745105+0000 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:44.745105+0000 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:44.747 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:44.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 Processor -- start 2024-09-15T21:18:44.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 -- start start 2024-09-15T21:18:44.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient: init 2024-09-15T21:18:44.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe4e00a29b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e0073a30 0x7fe4e013ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): picked mon.noname-b con 0x7fe4e0073a30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4e013b0a0 0x7fe4e013d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): picked mon.noname-c con 0x7fe4e013b0a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e00727d0 con 0x7fe4e00a29b0 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e006f900 con 0x7fe4e013b0a0 2024-09-15T21:18:44.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e0073070 con 0x7fe4e0073a30 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4e5944640 10 monclient(hunting): authenticate will time out at 1043.541823s 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4deffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe4e00a29b0 0x7fe4e00a2d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41990/0 (socket says 172.21.15.119:41990) 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4deffd640 1 -- 172.21.15.119:0/1269732965 learned_addr learned my addr 172.21.15.119:0/1269732965 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4de7fc640 1 -- 172.21.15.119:0/1269732965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e0073a30 msgr2=0x7fe4e013ab60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4de7fc640 1 --2- 172.21.15.119:0/1269732965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e0073a30 0x7fe4e013ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 1 --2- 172.21.15.119:0/1269732965 >> v2:172.21.15.119:3300/0 conn(0x7fe4e013b0a0 0x7fe4e013d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): get_auth_request con 0x7fe4e013b0a0 auth_method 0 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4ddffb640 1 -- 172.21.15.119:0/1269732965 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445679316 0 0) 0x7fe4e00727d0 con 0x7fe4e00a29b0 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4ddffb640 10 monclient(hunting): _negotiate 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4ddffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4ddffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4ddffb640 10 monclient(hunting): my global_id is 14191 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4ddffb640 1 -- 172.21.15.119:0/1269732965 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4cc003530 con 0x7fe4e00a29b0 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient(hunting): handle_auth_done global_id 14188 payload 274 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 1 -- 172.21.15.119:0/1269732965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e0073a30 msgr2=0x7fe4e013ab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 1 --2- 172.21.15.119:0/1269732965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e0073a30 0x7fe4e013ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 1 -- 172.21.15.119:0/1269732965 >> v1:172.21.15.119:6789/0 conn(0x7fe4e00a29b0 legacy=0x7fe4e00a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 1 monclient: found mon.noname-c 2024-09-15T21:18:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.741+0000 7fe4df7fe640 1 -- 172.21.15.119:0/1269732965 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e0072ab0 con 0x7fe4e013b0a0 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4df7fe640 1 --2- 172.21.15.119:0/1269732965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e013b0a0 0x7fe4e013d490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80097c0 tx=0x7fe4c801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43986b6a893ae05 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 1 -- 172.21.15.119:0/1269732965 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b0070 con 0x7fe4e013b0a0 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: dump: 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 1 -- 172.21.15.119:0/1269732965 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80ab600 con 0x7fe4e013b0a0 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 5 monclient: authenticate success, global_id 14188 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 1 -- 172.21.15.119:0/1269732965 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80ab9d0 con 0x7fe4e013b0a0 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 4 set_mon_vals no callback set 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4ddffb640 10 monclient: dump: 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:44.759 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: shutdown 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/1269732965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e013b0a0 msgr2=0x7fe4e013d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 --2- 172.21.15.119:0/1269732965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e013b0a0 0x7fe4e013d490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80097c0 tx=0x7fe4c801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/1269732965 shutdown_connections 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 --2- 172.21.15.119:0/1269732965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e013b0a0 0x7fe4e013d490 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 --2- 172.21.15.119:0/1269732965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e0073a30 0x7fe4e013ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/1269732965 >> 172.21.15.119:0/1269732965 conn(0x7fe4e00a3d00 msgr2=0x7fe4e006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:44.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/1269732965 shutdown_connections 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/1269732965 wait complete. 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: build_initial_monmap 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: monmap: 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 Processor -- start 2024-09-15T21:18:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- start start 2024-09-15T21:18:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: init 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4e0073a30 0x7fe4e01348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): picked mon.noname-a con 0x7fe4e0073a30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 0x7fe4e0134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): picked mon.noname-b con 0x7fe4e00a29b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e013b0a0 0x7fe4e0135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): picked mon.noname-c con 0x7fe4e013b0a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0069a90 con 0x7fe4e0073a30 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0069910 con 0x7fe4e00a29b0 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e013e1e0 con 0x7fe4e013b0a0 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient(hunting): authenticate will time out at 1043.546894s 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 0x7fe4e0134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4df7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e013b0a0 msgr2=0x7fe4e0135330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 0x7fe4e0134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39984/0 (socket says 172.21.15.119:39984) 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4df7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e013b0a0 0x7fe4e0135330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 -- 172.21.15.119:0/2037270667 learned_addr learned my addr 172.21.15.119:0/2037270667 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): get_auth_request con 0x7fe4e00a29b0 auth_method 0 2024-09-15T21:18:44.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4deffd640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4e0073a30 0x7fe4e01348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4deffd640 10 monclient(hunting): get_auth_request con 0x7fe4e0073a30 auth_method 0 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4deffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient(hunting): handle_auth_done global_id 14194 payload 824 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 -- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e013b0a0 msgr2=0x7fe4e0135330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:44.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e013b0a0 0x7fe4e0135330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 -- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4e0073a30 msgr2=0x7fe4e01348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4e0073a30 0x7fe4e01348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 monclient: found mon.noname-b 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 -- 172.21.15.119:0/2037270667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e01386d0 con 0x7fe4e00a29b0 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4de7fc640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 0x7fe4e0134df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b550 tx=0x7fe4d800ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8013020 con 0x7fe4e00a29b0 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: dump: 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:44.768 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8003de0 con 0x7fe4e00a29b0 2024-09-15T21:18:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 5 monclient: authenticate success, global_id 14194 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: _renew_subs 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e00a5b50 con 0x7fe4e00a29b0 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80052b0 con 0x7fe4e00a29b0 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4d7fff640 10 monclient: dump: 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4dd7fa640 4 set_mon_vals no callback set 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: _renew_subs 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.745+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e0074600 con 0x7fe4e00a29b0 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4e5944640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4e5944640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4e5944640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e013e640 con 0x7fe4e00a29b0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe4d8016020 con 0x7fe4e00a29b0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4d7fff640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe4c003b970 0x7fe4c003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4697+0+0 (secure 0 0 0) 0x7fe4d804f660 con 0x7fe4e00a29b0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4deffd640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe4c003b970 0x7fe4c003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4deffd640 10 monclient: get_auth_request con 0x7fe4c003b970 auth_method 0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.749+0000 7fe4deffd640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe4c003b970 0x7fe4c003de30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0001810 tx=0x7fe4d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.753+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe4d801d020 con 0x7fe4e00a29b0 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.753+0000 7fe4d7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.753+0000 7fe4d7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:44.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.929+0000 7fe4e5944640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T21:18:44.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.929+0000 7fe4e5944640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T21:18:44.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.929+0000 7fe4e5944640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:44.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:44.929+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) -- 0x7fe4e01384b0 con 0x7fe4e00a29b0 2024-09-15T21:18:45.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.493+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v21) ==== 143+0+0 (secure 0 0 0) 0x7fe4d800fe20 con 0x7fe4e00a29b0 2024-09-15T21:18:45.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.493+0000 7fe4d7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T21:18:45.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.493+0000 7fe4d7fff640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 2 2024-09-15T21:18:45.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.601+0000 7fe4e5944640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T21:18:45.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.601+0000 7fe4e5944640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T21:18:45.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.601+0000 7fe4e5944640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:45.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:45.601+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) -- 0x7fe4e0074bd0 con 0x7fe4e00a29b0 2024-09-15T21:18:46.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.505+0000 7fe4d7fff640 1 -- 172.21.15.119:0/2037270667 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v22) ==== 143+0+0 (secure 0 0 0) 0x7fe4d800d380 con 0x7fe4e00a29b0 2024-09-15T21:18:46.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.505+0000 7fe4d7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T21:18:46.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.505+0000 7fe4d7fff640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 2 2024-09-15T21:18:46.513 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 3 pg_num_min to 2 2024-09-15T21:18:46.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe4c003b970 msgr2=0x7fe4c003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe4c003b970 0x7fe4c003de30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0001810 tx=0x7fe4d0009040 comp rx=0 tx=0).stop 2024-09-15T21:18:46.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 10 monclient: shutdown 2024-09-15T21:18:46.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:46.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 msgr2=0x7fe4e0134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 0x7fe4e0134df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b550 tx=0x7fe4d800ba10 comp rx=0 tx=0).stop 2024-09-15T21:18:46.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 shutdown_connections 2024-09-15T21:18:46.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe4c003b970 0x7fe4c003de30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4e013b0a0 0x7fe4e0135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e00a29b0 0x7fe4e0134df0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 --2- 172.21.15.119:0/2037270667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4e0073a30 0x7fe4e01348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 >> 172.21.15.119:0/2037270667 conn(0x7fe4e00a3d00 msgr2=0x7fe4e013cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:46.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 shutdown_connections 2024-09-15T21:18:46.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.509+0000 7fe4e5944640 1 -- 172.21.15.119:0/2037270667 wait complete. 2024-09-15T21:18:46.534 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-09-15T21:18:46.534 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 2 2024-09-15T21:18:46.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.649+0000 7faba2644640 10 monclient: get_monmap_and_config 2024-09-15T21:18:46.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.649+0000 7faba2644640 10 monclient: build_initial_monmap 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.649+0000 7faba2644640 10 monclient: monmap: 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:46.655434+0000 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:46.655434+0000 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:46.657 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:46.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.649+0000 7faba2644640 1 Processor -- start 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.649+0000 7faba2644640 1 -- start start 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.649+0000 7faba2644640 10 monclient: init 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fab9c0a2450 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c139440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): picked mon.noname-b con 0x7fab9c0a2f40 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab9c139980 0x7fab9c13bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): picked mon.noname-c con 0x7fab9c139980 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab9c06fda0 con 0x7fab9c0a2450 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fab9c071120 con 0x7fab9c139980 2024-09-15T21:18:46.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fab9c0716e0 con 0x7fab9c0a2f40 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient(hunting): authenticate will time out at 1045.452485s 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fab9c0a2450 0x7fab9c0a2830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42004/0 (socket says 172.21.15.119:42004) 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9bfff640 1 -- 172.21.15.119:0/3688957134 learned_addr learned my addr 172.21.15.119:0/3688957134 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 --2- 172.21.15.119:0/3688957134 >> v2:172.21.15.119:3300/0 conn(0x7fab9c139980 0x7fab9c13bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9b7fe640 1 -- 172.21.15.119:0/3688957134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 msgr2=0x7fab9c139440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9b7fe640 1 --2- 172.21.15.119:0/3688957134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c139440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): get_auth_request con 0x7fab9c139980 auth_method 0 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:46.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient(hunting): handle_auth_done global_id 14197 payload 274 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 -- 172.21.15.119:0/3688957134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 msgr2=0x7fab9c139440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 --2- 172.21.15.119:0/3688957134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c139440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 -- 172.21.15.119:0/3688957134 >> v1:172.21.15.119:6789/0 conn(0x7fab9c0a2450 legacy=0x7fab9c0a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient: _finish_hunting 0 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 monclient: found mon.noname-c 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 -- 172.21.15.119:0/3688957134 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c073e90 con 0x7fab9c139980 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba0be1640 1 --2- 172.21.15.119:0/3688957134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c13bd70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab9400aea0 tx=0x7fab9401d8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab292e4d8c9bc7a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 1 -- 172.21.15.119:0/3688957134 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940a6660 con 0x7fab9c139980 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: dump: 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:46.668 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: _finish_auth 0 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 5 monclient: authenticate success, global_id 14197 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 1 -- 172.21.15.119:0/3688957134 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab940a6800 con 0x7fab9c139980 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:46.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 4 set_mon_vals no callback set 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 1 -- 172.21.15.119:0/3688957134 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940a6bd0 con 0x7fab9c139980 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7fab9affd640 10 monclient: dump: 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 10 monclient: shutdown 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- 172.21.15.119:0/3688957134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 msgr2=0x7fab9c13bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 --2- 172.21.15.119:0/3688957134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c13bd70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab9400aea0 tx=0x7fab9401d8f0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- 172.21.15.119:0/3688957134 shutdown_connections 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 --2- 172.21.15.119:0/3688957134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c13bd70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 --2- 172.21.15.119:0/3688957134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- 172.21.15.119:0/3688957134 >> 172.21.15.119:0/3688957134 conn(0x7fab9c0a3e20 msgr2=0x7fab9c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.653+0000 7faba2644640 1 -- 172.21.15.119:0/3688957134 shutdown_connections 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- 172.21.15.119:0/3688957134 wait complete. 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: build_initial_monmap 2024-09-15T21:18:46.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: monmap: 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 Processor -- start 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- start start 2024-09-15T21:18:46.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: init 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9c0a2450 0x7fab9c131520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): picked mon.noname-a con 0x7fab9c0a2450 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c131a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): picked mon.noname-c con 0x7fab9c0a2f40 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c1cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): picked mon.noname-b con 0x7fab9c139980 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c06fa00 con 0x7fab9c0a2450 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c06ffb0 con 0x7fab9c139980 2024-09-15T21:18:46.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fab9c06fb80 con 0x7fab9c0a2f40 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient(hunting): authenticate will time out at 1045.457451s 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c1cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9b7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 msgr2=0x7fab9c131a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c131a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c1cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40006/0 (socket says 172.21.15.119:40006) 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 -- 172.21.15.119:0/1048139141 learned_addr learned my addr 172.21.15.119:0/1048139141 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): get_auth_request con 0x7fab9c139980 auth_method 0 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:46.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9bfff640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9c0a2450 0x7fab9c131520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient(hunting): handle_auth_done global_id 14203 payload 824 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 -- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 msgr2=0x7fab9c131a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c131a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 -- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9c0a2450 msgr2=0x7fab9c131520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9c0a2450 0x7fab9c131520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient: _finish_hunting 0 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 monclient: found mon.noname-b 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 -- 172.21.15.119:0/1048139141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c09fc00 con 0x7fab9c139980 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9bfff640 10 monclient: get_auth_request con 0x7fab9c0a2450 auth_method 0 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9bfff640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9c0a2450 0x7fab9c131520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba0be1640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c1cb540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab940b0040 tx=0x7fab9401a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940b7680 con 0x7fab9c139980 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: dump: 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:46.676 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: _finish_auth 0 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab940bd070 con 0x7fab9c139980 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940b7a60 con 0x7fab9c139980 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 10 monclient: dump: 2024-09-15T21:18:46.677 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 5 monclient: authenticate success, global_id 14203 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _renew_subs 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9a7fc640 4 set_mon_vals no callback set 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c0a5950 con 0x7fab9c139980 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _renew_subs 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c09d8a0 con 0x7fab9c139980 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c0a2830 con 0x7fab9c139980 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fab940b7c00 con 0x7fab9c139980 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab8803b970 0x7fab8803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4713+0+0 (secure 0 0 0) 0x7fab940ea630 con 0x7fab9c139980 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9bfff640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab8803b970 0x7fab8803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.657+0000 7fab9bfff640 10 monclient: get_auth_request con 0x7fab8803b970 auth_method 0 2024-09-15T21:18:46.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.661+0000 7fab9bfff640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab8803b970 0x7fab8803de30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000f70 tx=0x7fab8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:46.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.665+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fab94017660 con 0x7fab9c139980 2024-09-15T21:18:46.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.665+0000 7fab98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:46.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.665+0000 7fab98ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:46.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7faba2644640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:46.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7faba2644640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:46.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7faba2644640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) -- 0x7fab9c074bd0 con 0x7fab9c139980 2024-09-15T21:18:46.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v22) ==== 105+0+14 (secure 0 0 0) 0x7fab940c1b90 con 0x7fab9c139980 2024-09-15T21:18:46.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7fab98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:46.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.833+0000 7fab98ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:46.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7faba2644640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:46.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7faba2644640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:46.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7faba2644640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:46.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) -- 0x7fab9c131fa0 con 0x7fab9c139980 2024-09-15T21:18:46.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7fab98ff9640 1 -- 172.21.15.119:0/1048139141 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v22) ==== 105+0+14 (secure 0 0 0) 0x7fab940c0550 con 0x7fab9c139980 2024-09-15T21:18:46.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7fab98ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:46.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.925+0000 7fab98ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:18:46.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab8803b970 msgr2=0x7fab8803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab8803b970 0x7fab8803de30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000f70 tx=0x7fab8c009040 comp rx=0 tx=0).stop 2024-09-15T21:18:46.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 10 monclient: shutdown 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 msgr2=0x7fab9c1cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c1cb540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab940b0040 tx=0x7fab9401a6f0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 shutdown_connections 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab8803b970 0x7fab8803de30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c139980 0x7fab9c1cb540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9c0a2f40 0x7fab9c131a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 --2- 172.21.15.119:0/1048139141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9c0a2450 0x7fab9c131520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 >> 172.21.15.119:0/1048139141 conn(0x7fab9c0a3e20 msgr2=0x7fab9c13b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 shutdown_connections 2024-09-15T21:18:46.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:46.929+0000 7faba2644640 1 -- 172.21.15.119:0/1048139141 wait complete. 2024-09-15T21:18:46.948 INFO:tasks.workunit.client.0.smithi119.stdout:pg_num_min: 2 2024-09-15T21:18:46.948 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num_max 33 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient: get_monmap_and_config 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient: build_initial_monmap 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient: monmap: 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:47.062740+0000 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:47.062740+0000 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:47.064 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:47.065 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:47.065 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:47.065 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:47.065 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:47.065 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:47.065 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:47.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 Processor -- start 2024-09-15T21:18:47.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 -- start start 2024-09-15T21:18:47.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient: init 2024-09-15T21:18:47.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc144135bf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14409fe70 0x7fc14413aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): picked mon.noname-b con 0x7fc14409fe70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc14413afb0 0x7fc14413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): picked mon.noname-c con 0x7fc14413afb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440a0300 con 0x7fc144135bf0 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc1440a3600 con 0x7fc14413afb0 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc1440a1d80 con 0x7fc14409fe70 2024-09-15T21:18:47.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc149009640 10 monclient(hunting): authenticate will time out at 1045.859375s 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14259c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14409fe70 msgr2=0x7fc14413aa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14409fe70 0x7fc14413aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc142d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc144135bf0 0x7fc144135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42014/0 (socket says 172.21.15.119:42014) 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc14413afb0 0x7fc14413d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc142d9d640 1 -- 172.21.15.119:0/2135166375 learned_addr learned my addr 172.21.15.119:0/2135166375 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): get_auth_request con 0x7fc14413afb0 auth_method 0 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:47.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.057+0000 7fc14359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 10 monclient(hunting): handle_auth_done global_id 14206 payload 274 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 1 -- 172.21.15.119:0/2135166375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14409fe70 msgr2=0x7fc14413aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 1 --2- 172.21.15.119:0/2135166375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14409fe70 0x7fc14413aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc141d9b640 1 -- 172.21.15.119:0/2135166375 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086368726 0 0) 0x7fc138001120 con 0x7fc144135bf0 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 1 -- 172.21.15.119:0/2135166375 >> v1:172.21.15.119:6789/0 conn(0x7fc144135bf0 legacy=0x7fc144135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 10 monclient: _finish_hunting 0 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 1 monclient: found mon.noname-c 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc14359e640 1 -- 172.21.15.119:0/2135166375 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440a2060 con 0x7fc14413afb0 2024-09-15T21:18:47.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.061+0000 7fc141d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:47.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14359e640 1 --2- 172.21.15.119:0/2135166375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14413afb0 0x7fc14413d3a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009d90 tx=0x7fc13c01c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78d574042a2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:47.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 1 -- 172.21.15.119:0/2135166375 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b2070 con 0x7fc14413afb0 2024-09-15T21:18:47.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: dump: 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:47.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: _finish_auth 0 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 5 monclient: authenticate success, global_id 14206 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 1 -- 172.21.15.119:0/2135166375 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c0a5820 con 0x7fc14413afb0 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 4 set_mon_vals no callback set 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 1 -- 172.21.15.119:0/2135166375 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0a5bf0 con 0x7fc14413afb0 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:47.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc141d9b640 10 monclient: dump: 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:47.074 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:47.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: shutdown 2024-09-15T21:18:47.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:47.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- 172.21.15.119:0/2135166375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14413afb0 msgr2=0x7fc14413d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 --2- 172.21.15.119:0/2135166375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14413afb0 0x7fc14413d3a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009d90 tx=0x7fc13c01c850 comp rx=0 tx=0).stop 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- 172.21.15.119:0/2135166375 shutdown_connections 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 --2- 172.21.15.119:0/2135166375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14413afb0 0x7fc14413d3a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 --2- 172.21.15.119:0/2135166375 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14409fe70 0x7fc14413aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- 172.21.15.119:0/2135166375 >> 172.21.15.119:0/2135166375 conn(0x7fc1440a66e0 msgr2=0x7fc1440a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- 172.21.15.119:0/2135166375 shutdown_connections 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- 172.21.15.119:0/2135166375 wait complete. 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: build_initial_monmap 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: monmap: 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 Processor -- start 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- start start 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: init 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc14409fe70 0x7fc1441cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): picked mon.noname-a con 0x7fc14409fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 0x7fc144134b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): picked mon.noname-b con 0x7fc144135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14413afb0 0x7fc144135060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): picked mon.noname-c con 0x7fc14413afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc144140100 con 0x7fc14409fe70 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc14406ece0 con 0x7fc144135bf0 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc144140280 con 0x7fc14413afb0 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc149009640 10 monclient(hunting): authenticate will time out at 1045.867753s 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 0x7fc144134b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 0x7fc144134b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40020/0 (socket says 172.21.15.119:40020) 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 1 -- 172.21.15.119:0/356102011 learned_addr learned my addr 172.21.15.119:0/356102011 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 10 monclient(hunting): get_auth_request con 0x7fc144135bf0 auth_method 0 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14359e640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14413afb0 msgr2=0x7fc144135060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.065+0000 7fc14359e640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14413afb0 0x7fc144135060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:47.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc142d9d640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc14409fe70 0x7fc1441cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 10 monclient(hunting): handle_auth_done global_id 14212 payload 824 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 -- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14413afb0 msgr2=0x7fc144135060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14413afb0 0x7fc144135060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 -- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc14409fe70 msgr2=0x7fc1441cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc14409fe70 0x7fc1441cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 10 monclient: _finish_hunting 0 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 monclient: found mon.noname-b 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 -- 172.21.15.119:0/356102011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440a5400 con 0x7fc144135bf0 2024-09-15T21:18:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc142d9d640 10 monclient: get_auth_request con 0x7fc14409fe70 auth_method 0 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc142d9d640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc14409fe70 0x7fc1441cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14259c640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 0x7fc144134b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b530 tx=0x7fc13400b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134013020 con 0x7fc144135bf0 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: dump: 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:47.084 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134003df0 con 0x7fc144135bf0 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 5 monclient: authenticate success, global_id 14212 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340052b0 con 0x7fc144135bf0 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 monclient: _renew_subs 2024-09-15T21:18:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14413ac30 con 0x7fc144135bf0 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc14159a640 4 set_mon_vals no callback set 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 10 monclient: dump: 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:47.086 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 monclient: _renew_subs 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc144121010 con 0x7fc144135bf0 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc134016020 con 0x7fc144135bf0 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc144135fd0 con 0x7fc144135bf0 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc13003b970 0x7fc13003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4713+0+0 (secure 0 0 0) 0x7fc13404f750 con 0x7fc144135bf0 2024-09-15T21:18:47.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc142d9d640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc13003b970 0x7fc13003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:47.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc142d9d640 10 monclient: get_auth_request con 0x7fc13003b970 auth_method 0 2024-09-15T21:18:47.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.069+0000 7fc142d9d640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc13003b970 0x7fc13003de30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc138001400 tx=0x7fc138009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:47.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.073+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc13401d020 con 0x7fc144135bf0 2024-09-15T21:18:47.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.073+0000 7fc12b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:47.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.073+0000 7fc12b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.245+0000 7fc149009640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T21:18:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.245+0000 7fc149009640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T21:18:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.245+0000 7fc149009640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.245+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) -- 0x7fc14406a830 con 0x7fc144135bf0 2024-09-15T21:18:47.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.521+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v23) ==== 145+0+0 (secure 0 0 0) 0x7fc13400fdc0 con 0x7fc144135bf0 2024-09-15T21:18:47.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.521+0000 7fc12b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T21:18:47.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.521+0000 7fc12b7fe640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 33 2024-09-15T21:18:47.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.625+0000 7fc149009640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T21:18:47.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.625+0000 7fc149009640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T21:18:47.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.625+0000 7fc149009640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:47.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:47.625+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) -- 0x7fc144071f60 con 0x7fc144135bf0 2024-09-15T21:18:48.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.521+0000 7fc12b7fe640 1 -- 172.21.15.119:0/356102011 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v24) ==== 145+0+0 (secure 0 0 0) 0x7fc13400d380 con 0x7fc144135bf0 2024-09-15T21:18:48.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.521+0000 7fc12b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T21:18:48.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.521+0000 7fc12b7fe640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 33 2024-09-15T21:18:48.529 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 3 pg_num_max to 33 2024-09-15T21:18:48.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc13003b970 msgr2=0x7fc13003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc13003b970 0x7fc13003de30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc138001400 tx=0x7fc138009040 comp rx=0 tx=0).stop 2024-09-15T21:18:48.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 10 monclient: shutdown 2024-09-15T21:18:48.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 msgr2=0x7fc144134b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 0x7fc144134b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b530 tx=0x7fc13400b9f0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 shutdown_connections 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc13003b970 0x7fc13003de30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc14413afb0 0x7fc144135060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc144135bf0 0x7fc144134b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 --2- 172.21.15.119:0/356102011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc14409fe70 0x7fc1441cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 >> 172.21.15.119:0/356102011 conn(0x7fc1440a66e0 msgr2=0x7fc14413bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 shutdown_connections 2024-09-15T21:18:48.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.525+0000 7fc149009640 1 -- 172.21.15.119:0/356102011 wait complete. 2024-09-15T21:18:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-09-15T21:18:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 33 2024-09-15T21:18:48.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.665+0000 7faecb867640 10 monclient: get_monmap_and_config 2024-09-15T21:18:48.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.665+0000 7faecb867640 10 monclient: build_initial_monmap 2024-09-15T21:18:48.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.665+0000 7faecb867640 10 monclient: monmap: 2024-09-15T21:18:48.673 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:48.673 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:48.672263+0000 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:48.672263+0000 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:48.674 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:48.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 Processor -- start 2024-09-15T21:18:48.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 -- start start 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient: init 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faec4137fe0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec40a2320 0x7faec413ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): picked mon.noname-b con 0x7faec40a2320 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faec413d3a0 0x7faec413f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): picked mon.noname-a con 0x7faec413d3a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faec40a3e10 con 0x7faec4137fe0 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faec40a1400 con 0x7faec413d3a0 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:48.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faec40a1960 con 0x7faec40a2320 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faecb867640 10 monclient(hunting): authenticate will time out at 1047.469230s 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec8e02640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec40a2320 msgr2=0x7faec413ce60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec8e02640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec40a2320 0x7faec413ce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faec413d3a0 0x7faec413f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9603640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faec4137fe0 0x7faec41383c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:50360/0 (socket says 172.21.15.119:50360) 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9603640 1 -- 172.21.15.119:0/2315469015 learned_addr learned my addr 172.21.15.119:0/2315469015 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): get_auth_request con 0x7faec413d3a0 auth_method 0 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:48.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient(hunting): handle_auth_done global_id 4545 payload 274 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 1 -- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec40a2320 msgr2=0x7faec413ce60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 1 --2- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec40a2320 0x7faec413ce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 1 -- 172.21.15.119:0/2315469015 >> v1:172.21.15.17:6789/0 conn(0x7faec4137fe0 legacy=0x7faec41383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient: _finish_hunting 0 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 1 monclient: found mon.noname-a 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:48.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.669+0000 7faec9e04640 1 -- 172.21.15.119:0/2315469015 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec40a6bf0 con 0x7faec413d3a0 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9e04640 1 --2- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec413d3a0 0x7faec413f790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faebc009d90 tx=0x7faebc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860cd1e7cf207876 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 1 -- 172.21.15.119:0/2315469015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc0b2070 con 0x7faec413d3a0 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: dump: 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.684 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: _finish_auth 0 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 1 -- 172.21.15.119:0/2315469015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faebc0a5710 con 0x7faec413d3a0 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 5 monclient: authenticate success, global_id 4545 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:48.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 4 set_mon_vals no callback set 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 1 -- 172.21.15.119:0/2315469015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc0a5ae0 con 0x7faec413d3a0 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebbfff640 10 monclient: dump: 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:48.686 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: shutdown 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec413d3a0 msgr2=0x7faec413f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 --2- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec413d3a0 0x7faec413f790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faebc009d90 tx=0x7faebc01c850 comp rx=0 tx=0).stop 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- 172.21.15.119:0/2315469015 shutdown_connections 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 --2- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec413d3a0 0x7faec413f790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 --2- 172.21.15.119:0/2315469015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec40a2320 0x7faec413ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- 172.21.15.119:0/2315469015 >> 172.21.15.119:0/2315469015 conn(0x7faec40a6f60 msgr2=0x7faec40a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- 172.21.15.119:0/2315469015 shutdown_connections 2024-09-15T21:18:48.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- 172.21.15.119:0/2315469015 wait complete. 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: build_initial_monmap 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: monmap: 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 Processor -- start 2024-09-15T21:18:48.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- start start 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: init 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 0x7faec413ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): picked mon.noname-b con 0x7faec40a2320 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec4137fe0 0x7faec413b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): picked mon.noname-a con 0x7faec4137fe0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec413d3a0 0x7faec413b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): picked mon.noname-c con 0x7faec413d3a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faec4142670 con 0x7faec4137fe0 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:48.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faec41424f0 con 0x7faec40a2320 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faec41427f0 con 0x7faec413d3a0 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient(hunting): authenticate will time out at 1047.475025s 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 0x7faec413ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9e04640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec413d3a0 msgr2=0x7faec413b820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9e04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec413d3a0 0x7faec413b820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 0x7faec413ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40032/0 (socket says 172.21.15.119:40032) 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 -- 172.21.15.119:0/1718057105 learned_addr learned my addr 172.21.15.119:0/1718057105 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): get_auth_request con 0x7faec40a2320 auth_method 0 2024-09-15T21:18:48.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec8e02640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec4137fe0 0x7faec413b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient(hunting): handle_auth_done global_id 14215 payload 824 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 -- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec413d3a0 msgr2=0x7faec413b820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec413d3a0 0x7faec413b820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 -- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec4137fe0 msgr2=0x7faec413b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec4137fe0 0x7faec413b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient: _finish_hunting 0 2024-09-15T21:18:48.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 monclient: found mon.noname-b 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 -- 172.21.15.119:0/1718057105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec40a1650 con 0x7faec40a2320 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec8e02640 10 monclient: get_auth_request con 0x7faec4137fe0 auth_method 0 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec8e02640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec4137fe0 0x7faec413b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faec9603640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 0x7faec413ada0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faec000c400 tx=0x7faec000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0017070 con 0x7faec40a2320 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 10 monclient: dump: 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:48.692 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec0005d30 con 0x7faec40a2320 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:48.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0007310 con 0x7faec40a2320 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 5 monclient: authenticate success, global_id 14215 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: _renew_subs 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec4071000 con 0x7faec40a2320 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faeb9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faeb9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.673+0000 7faebb7fe640 4 set_mon_vals no callback set 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faeb9ffb640 10 monclient: dump: 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:48.694 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 10 monclient: _renew_subs 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec413c860 con 0x7faec40a2320 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7faec001a070 con 0x7faec40a2320 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec4070bb0 con 0x7faec40a2320 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faeb9ffb640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faea803b970 0x7faea803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:48.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7faec004eeb0 con 0x7faec40a2320 2024-09-15T21:18:48.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faec8e02640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faea803b970 0x7faea803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:48.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faec8e02640 10 monclient: get_auth_request con 0x7faea803b970 auth_method 0 2024-09-15T21:18:48.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.677+0000 7faec8e02640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faea803b970 0x7faea803de30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faeb4006fd0 tx=0x7faeb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:48.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.681+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7faec40a1650 con 0x7faec40a2320 2024-09-15T21:18:48.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.681+0000 7faeb9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:48.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.681+0000 7faeb9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:48.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faecb867640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:48.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faecb867640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:48.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faecb867640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:48.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) -- 0x7faec41383c0 con 0x7faec40a2320 2024-09-15T21:18:48.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v24) ==== 105+0+15 (secure 0 0 0) 0x7faec0012400 con 0x7faec40a2320 2024-09-15T21:18:48.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faeb9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:48.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.845+0000 7faeb9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) -- 0x7faec40a10a0 con 0x7faec40a2320 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faeb9ffb640 1 -- 172.21.15.119:0/1718057105 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v24) ==== 105+0+15 (secure 0 0 0) 0x7faec004d820 con 0x7faec40a2320 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faeb9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:48.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faeb9ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:18:48.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faea803b970 msgr2=0x7faea803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faea803b970 0x7faea803de30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faeb4006fd0 tx=0x7faeb4008040 comp rx=0 tx=0).stop 2024-09-15T21:18:48.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 10 monclient: shutdown 2024-09-15T21:18:48.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:48.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 msgr2=0x7faec413ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 0x7faec413ada0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faec000c400 tx=0x7faec000c8c0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 shutdown_connections 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faea803b970 0x7faea803de30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faec413d3a0 0x7faec413b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faec4137fe0 0x7faec413b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 --2- 172.21.15.119:0/1718057105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec40a2320 0x7faec413ada0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 >> 172.21.15.119:0/1718057105 conn(0x7faec40a6f60 msgr2=0x7faec4136fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 shutdown_connections 2024-09-15T21:18:48.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:48.937+0000 7faecb867640 1 -- 172.21.15.119:0/1718057105 wait complete. 2024-09-15T21:18:48.959 INFO:tasks.workunit.client.0.smithi119.stdout:pg_num_max: 33 2024-09-15T21:18:48.960 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_min 9 2024-09-15T21:18:48.960 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:48.960 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num_min 9 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient: get_monmap_and_config 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient: monmap: 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:49.073884+0000 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:49.073884+0000 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.075 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.076 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:49.076 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:49.076 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:49.076 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:49.076 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:49.076 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 Processor -- start 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 -- start start 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient: init 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0698136b90 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06980a10d0 0x7f069813cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): picked mon.noname-a con 0x7f06980a10d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 0x7f069813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): picked mon.noname-b con 0x7f06980a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06980a18f0 con 0x7f0698136b90 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f069809fc40 con 0x7f06980a10d0 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f06980a1bd0 con 0x7f06980a25b0 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f069d8ba640 10 monclient(hunting): authenticate will time out at 1047.870843s 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0696ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0698136b90 0x7f0698136f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42028/0 (socket says 172.21.15.119:42028) 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0696ffd640 1 -- 172.21.15.119:0/230875090 learned_addr learned my addr 172.21.15.119:0/230875090 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 1 -- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 msgr2=0x7f069813f1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 0x7f069813f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 1 --2- 172.21.15.119:0/230875090 >> v2:172.21.15.17:3300/0 conn(0x7f06980a10d0 0x7f069813cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 338811076 0 0) 0x7f06880014a0 con 0x7f0698136b90 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): my global_id is 14218 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0680003530 con 0x7f0698136b90 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): get_auth_request con 0x7f06980a10d0 auth_method 0 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1574267485 0 0) 0x7f0680003530 con 0x7f0698136b90 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 msgr2=0x7f069813f1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 0x7f069813f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 >> v2:172.21.15.17:3300/0 conn(0x7f06980a10d0 msgr2=0x7f069813cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 --2- 172.21.15.119:0/230875090 >> v2:172.21.15.17:3300/0 conn(0x7f06980a10d0 0x7f069813cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980a3660 con 0x7f0698136b90 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0688001c50 con 0x7f0698136b90 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: dump: 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.085 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 >> v1:172.21.15.119:6789/0 conn(0x7f0698136b90 legacy=0x7f0698136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0680003a10 0x7f0680003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): picked mon.a con 0x7f0680003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 0x7f0680004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): picked mon.c con 0x7f06980a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0680004da0 0x7f0680005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): picked mon.b con 0x7f0680004da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f068000bc00 con 0x7f0680003a10 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f068000bdb0 con 0x7f0680004da0 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f068000bfb0 con 0x7f06980a25b0 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f0695ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 1 -- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 msgr2=0x7f0680004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0680004da0 0x7f0680005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06967fc640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 0x7f0680004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 10 monclient(hunting): get_auth_request con 0x7f0680004da0 auth_method 0 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.069+0000 7f06977fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0696ffd640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0680003a10 0x7f0680003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0696ffd640 10 monclient(hunting): get_auth_request con 0x7f0680003a10 auth_method 0 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0696ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0696ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0696ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient(hunting): handle_auth_done global_id 14218 payload 293 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 -- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 msgr2=0x7f0680004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06980a25b0 0x7f0680004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 -- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0680003a10 msgr2=0x7f0680003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0680003a10 0x7f0680003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 monclient: found mon.b 2024-09-15T21:18:49.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 -- 172.21.15.119:0/230875090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0680003530 con 0x7f0680004da0 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 5 monclient: authenticate success, global_id 14218 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0680004da0 0x7f0680005220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0690006fd0 tx=0x7f069001c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e71c23e034a4dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690009540 con 0x7f0680004da0 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: dump: 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.089 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690003bc0 con 0x7f0680004da0 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:49.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 1 -- 172.21.15.119:0/230875090 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900050d0 con 0x7f0680004da0 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 4 set_mon_vals no callback set 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f0695ffb640 10 monclient: dump: 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.091 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 10 monclient: shutdown 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 1 -- 172.21.15.119:0/230875090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0680004da0 msgr2=0x7f0680005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 1 --2- 172.21.15.119:0/230875090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0680004da0 0x7f0680005220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0690006fd0 tx=0x7f069001c3b0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f06977fe640 1 -- 172.21.15.119:0/230875090 reap_dead start 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 1 -- 172.21.15.119:0/230875090 shutdown_connections 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 1 -- 172.21.15.119:0/230875090 >> 172.21.15.119:0/230875090 conn(0x7f06980a66e0 msgr2=0x7f06980a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 1 -- 172.21.15.119:0/230875090 shutdown_connections 2024-09-15T21:18:49.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 1 -- 172.21.15.119:0/230875090 wait complete. 2024-09-15T21:18:49.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.073+0000 7f069d8ba640 10 monclient: monmap: 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 Processor -- start 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- start start 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: init 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06981cf380 0x7f0698134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): picked mon.noname-c con 0x7f06981cf380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f069811fe60 0x7f06981351f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): picked mon.noname-a con 0x7f069811fe60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f069811d840 0x7f069811dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): picked mon.noname-b con 0x7f069811d840 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f069813fc80 con 0x7f069811fe60 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f069813fb00 con 0x7f069811d840 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f069813fe00 con 0x7f06981cf380 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient(hunting): authenticate will time out at 1047.876679s 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f069811d840 0x7f069811dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f069811d840 0x7f069811dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40064/0 (socket says 172.21.15.119:40064) 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 -- 172.21.15.119:0/22874948 learned_addr learned my addr 172.21.15.119:0/22874948 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f0696ffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06981cf380 msgr2=0x7f0698134cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): get_auth_request con 0x7f069811d840 auth_method 0 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f0696ffd640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06981cf380 0x7f0698134cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06967fc640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f069811fe60 0x7f06981351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient(hunting): handle_auth_done global_id 14221 payload 824 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 -- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06981cf380 msgr2=0x7f0698134cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06981cf380 0x7f0698134cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 -- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f069811fe60 msgr2=0x7f06981351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f069811fe60 0x7f06981351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 monclient: found mon.noname-b 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 -- 172.21.15.119:0/22874948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0698121380 con 0x7f069811d840 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06967fc640 10 monclient: get_auth_request con 0x7f069811fe60 auth_method 0 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06967fc640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f069811fe60 0x7f06981351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06977fe640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f069811d840 0x7f069811dcb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f069001cc50 tx=0x7f0680005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900ba070 con 0x7f069811d840 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: dump: 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690009520 con 0x7f069811d840 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 5 monclient: authenticate success, global_id 14221 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _renew_subs 2024-09-15T21:18:49.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- 172.21.15.119:0/22874948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069809d240 con 0x7f069811d840 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900050d0 con 0x7f069811d840 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 10 monclient: dump: 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _renew_subs 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- 172.21.15.119:0/22874948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06981cf760 con 0x7f069811d840 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f06957fa640 4 set_mon_vals no callback set 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f06900bd070 con 0x7f069811d840 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f069d8ba640 1 -- 172.21.15.119:0/22874948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0698136f70 con 0x7f069811d840 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f068403b9a0 0x7f068403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f06900e8e30 con 0x7f069811d840 2024-09-15T21:18:49.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f0696ffd640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f068403b9a0 0x7f068403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f0696ffd640 10 monclient: get_auth_request con 0x7f068403b9a0 auth_method 0 2024-09-15T21:18:49.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.077+0000 7f0696ffd640 1 --2- 172.21.15.119:0/22874948 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f068403b9a0 0x7f068403de60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06880013a0 tx=0x7f0688002140 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.081+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f06900160f0 con 0x7f069811d840 2024-09-15T21:18:49.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.081+0000 7f067ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.081+0000 7f067ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:49.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.237+0000 7f069d8ba640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-09-15T21:18:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.237+0000 7f069d8ba640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-09-15T21:18:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.237+0000 7f069d8ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.237+0000 7f069d8ba640 1 -- 172.21.15.119:0/22874948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"} v 0) -- 0x7f069806a830 con 0x7f069811d840 2024-09-15T21:18:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.241+0000 7f067ffff640 1 -- 172.21.15.119:0/22874948 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}]=-22 specified pg_num_min 9 > pg_num 8 v24) ==== 150+0+0 (secure 0 0 0) 0x7f0690003b70 con 0x7f069811d840 2024-09-15T21:18:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.241+0000 7f067ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-09-15T21:18:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.241+0000 7f067ffff640 10 monclient: _finish_command 2 = mon:22 specified pg_num_min 9 > pg_num 8 2024-09-15T21:18:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: specified pg_num_min 9 > pg_num 8 2024-09-15T21:18:49.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:49.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_max 7 2024-09-15T21:18:49.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:49.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num_max 7 2024-09-15T21:18:49.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient: get_monmap_and_config 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient: monmap: 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:49.369506+0000 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:49.369506+0000 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:49.371 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:49.372 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:49.372 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 Processor -- start 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 -- start start 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient: init 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f091809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): picked mon.noname-b con 0x7f0918135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0918135fd0 0x7f09180a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): picked mon.noname-c con 0x7f0918135fd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0918136430 0x7f09180a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): picked mon.noname-a con 0x7f0918136430 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:49.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f09180a3600 con 0x7f0918136430 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f09180a1d80 con 0x7f0918135fd0 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f09180a2060 con 0x7f0918135bf0 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f09206b4640 10 monclient(hunting): authenticate will time out at 1048.166193s 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0918135fd0 0x7f09180a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091e450640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 msgr2=0x7f091809fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091e450640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f091809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0918135fd0 0x7f09180a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40074/0 (socket says 172.21.15.119:40074) 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 -- 172.21.15.119:0/2416769205 learned_addr learned my addr 172.21.15.119:0/2416769205 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): get_auth_request con 0x7f0918135fd0 auth_method 0 2024-09-15T21:18:49.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091ec51640 1 --2- 172.21.15.119:0/2416769205 >> v2:172.21.15.17:3300/0 conn(0x7f0918136430 0x7f09180a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient(hunting): handle_auth_done global_id 14224 payload 274 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 -- 172.21.15.119:0/2416769205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 msgr2=0x7f091809fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 --2- 172.21.15.119:0/2416769205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f091809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 -- 172.21.15.119:0/2416769205 >> v2:172.21.15.17:3300/0 conn(0x7f0918136430 msgr2=0x7f09180a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 --2- 172.21.15.119:0/2416769205 >> v2:172.21.15.17:3300/0 conn(0x7f0918136430 0x7f09180a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 monclient: found mon.noname-c 2024-09-15T21:18:49.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091dc4f640 1 -- 172.21.15.119:0/2416769205 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09180a2340 con 0x7f0918135fd0 2024-09-15T21:18:49.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091ec51640 10 monclient: get_auth_request con 0x7f0918136430 auth_method 0 2024-09-15T21:18:49.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.365+0000 7f091ec51640 1 --2- 172.21.15.119:0/2416769205 >> v2:172.21.15.17:3300/0 conn(0x7f0918136430 0x7f09180a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091dc4f640 1 --2- 172.21.15.119:0/2416769205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918135fd0 0x7f09180a0a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f091809d110 tx=0x7f090801c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13b8604f8c94db5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 1 -- 172.21.15.119:0/2416769205 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080b0070 con 0x7f0918135fd0 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: dump: 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.383 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 1 -- 172.21.15.119:0/2416769205 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09080a4e50 con 0x7f0918135fd0 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:49.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 1 -- 172.21.15.119:0/2416769205 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080ab820 con 0x7f0918135fd0 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 5 monclient: authenticate success, global_id 14224 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 4 set_mon_vals no callback set 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091d44e640 10 monclient: dump: 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.385 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: shutdown 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- 172.21.15.119:0/2416769205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918135fd0 msgr2=0x7f09180a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- 172.21.15.119:0/2416769205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918135fd0 0x7f09180a0a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f091809d110 tx=0x7f090801c400 comp rx=0 tx=0).stop 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- 172.21.15.119:0/2416769205 shutdown_connections 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- 172.21.15.119:0/2416769205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918135fd0 0x7f09180a0a50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- 172.21.15.119:0/2416769205 >> v2:172.21.15.17:3300/0 conn(0x7f0918136430 0x7f09180a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- 172.21.15.119:0/2416769205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f091809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- 172.21.15.119:0/2416769205 >> 172.21.15.119:0/2416769205 conn(0x7f09180a66e0 msgr2=0x7f09180a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:49.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- 172.21.15.119:0/2416769205 shutdown_connections 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- 172.21.15.119:0/2416769205 wait complete. 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: monmap: 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:49.387 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 Processor -- start 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- start start 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: init 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f0918132e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): picked mon.noname-c con 0x7f0918135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0918135fd0 0x7f09181cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): picked mon.noname-a con 0x7f0918135fd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918136430 0x7f09181ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): picked mon.noname-b con 0x7f0918136430 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f091813f2a0 con 0x7f0918135fd0 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09180a18e0 con 0x7f0918136430 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f091813f420 con 0x7f0918135bf0 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f09206b4640 10 monclient(hunting): authenticate will time out at 1048.171191s 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091e450640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 msgr2=0x7f0918132e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091e450640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f0918132e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918136430 0x7f09181ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918136430 0x7f09181ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40088/0 (socket says 172.21.15.119:40088) 2024-09-15T21:18:49.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 -- 172.21.15.119:0/2751844333 learned_addr learned my addr 172.21.15.119:0/2751844333 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): get_auth_request con 0x7f0918136430 auth_method 0 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091dc4f640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0918135fd0 0x7f09181cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091dc4f640 10 monclient(hunting): get_auth_request con 0x7f0918135fd0 auth_method 0 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091dc4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091dc4f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091dc4f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient(hunting): handle_auth_done global_id 14227 payload 824 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 -- 172.21.15.119:0/2751844333 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 msgr2=0x7f0918132e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0918135bf0 0x7f0918132e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 -- 172.21.15.119:0/2751844333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0918135fd0 msgr2=0x7f09181cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0918135fd0 0x7f09181cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 monclient: found mon.noname-b 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 -- 172.21.15.119:0/2751844333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091809f450 con 0x7f0918136430 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.369+0000 7f091ec51640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0918136430 0x7f09181ccba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f091000cef0 tx=0x7f091000a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091001a070 con 0x7f0918136430 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: dump: 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091000d860 con 0x7f0918136430 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091000dc50 con 0x7f0918136430 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 10 monclient: dump: 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.393 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 5 monclient: authenticate success, global_id 14227 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 monclient: _renew_subs 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f091cc4d640 4 set_mon_vals no callback set 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 1 -- 172.21.15.119:0/2751844333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091813c7d0 con 0x7f0918136430 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 monclient: _renew_subs 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 1 -- 172.21.15.119:0/2751844333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091809ee90 con 0x7f0918136430 2024-09-15T21:18:49.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f09206b4640 1 -- 172.21.15.119:0/2751844333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091809fe70 con 0x7f0918136430 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f091001d070 con 0x7f0918136430 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f08fc03b970 0x7f08fc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f09100503c0 con 0x7f0918136430 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f091e450640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f08fc03b970 0x7f08fc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f091e450640 10 monclient: get_auth_request con 0x7f08fc03b970 auth_method 0 2024-09-15T21:18:49.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.373+0000 7f091e450640 1 --2- 172.21.15.119:0/2751844333 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f08fc03b970 0x7f08fc03de30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f091809f730 tx=0x7f0914006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.377+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f091001ebb0 con 0x7f0918136430 2024-09-15T21:18:49.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.377+0000 7f090effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.377+0000 7f090effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:49.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f09206b4640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-09-15T21:18:49.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f09206b4640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-09-15T21:18:49.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f09206b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f09206b4640 1 -- 172.21.15.119:0/2751844333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"} v 0) -- 0x7f09180a5e30 con 0x7f0918136430 2024-09-15T21:18:49.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f090effd640 1 -- 172.21.15.119:0/2751844333 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}]=-22 specified pg_num_max 7 < pg_num 8 v24) ==== 150+0+0 (secure 0 0 0) 0x7f0910050080 con 0x7f0918136430 2024-09-15T21:18:49.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f090effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-09-15T21:18:49.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.541+0000 7f090effd640 10 monclient: _finish_command 2 = mon:22 specified pg_num_max 7 < pg_num 8 2024-09-15T21:18:49.550 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: specified pg_num_max 7 < pg_num 8 2024-09-15T21:18:49.556 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:49.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 1 2024-09-15T21:18:49.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:49.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num 1 2024-09-15T21:18:49.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient: get_monmap_and_config 2024-09-15T21:18:49.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient: monmap: 2024-09-15T21:18:49.674 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.674 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:49.673101+0000 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:49.673101+0000 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:49.675 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 1 Processor -- start 2024-09-15T21:18:49.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 1 -- start start 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient: init 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa8680a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa8680a41b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8680a45c0 0x7fa8681402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient(hunting): picked mon.noname-a con 0x7fa8680a45c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8681383e0 con 0x7fa8680a41b0 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8680a19d0 con 0x7fa8680a7ac0 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa8680a7ea0 con 0x7fa8680a45c0 2024-09-15T21:18:49.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 10 monclient(hunting): authenticate will time out at 1048.469809s 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa8680a7ac0 0x7fa868137cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42034/0 (socket says 172.21.15.119:42034) 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 1 -- 172.21.15.119:0/1175250535 learned_addr learned my addr 172.21.15.119:0/1175250535 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1334097125 0 0) 0x7fa8680a19d0 con 0x7fa8680a7ac0 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): _negotiate 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): my global_id is 14230 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa850003530 con 0x7fa8680a7ac0 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 --2- 172.21.15.119:0/1175250535 >> v2:172.21.15.17:3300/0 conn(0x7fa8680a45c0 0x7fa8681402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): get_auth_request con 0x7fa8680a45c0 auth_method 0 2024-09-15T21:18:49.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 121469149 0 0) 0x7fa850003530 con 0x7fa8680a7ac0 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 >> v2:172.21.15.17:3300/0 conn(0x7fa8680a45c0 msgr2=0x7fa8681402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 --2- 172.21.15.119:0/1175250535 >> v2:172.21.15.17:3300/0 conn(0x7fa8680a45c0 0x7fa8681402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 >> v1:172.21.15.17:6789/0 conn(0x7fa8680a41b0 legacy=0x7fa86813cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8680a6960 con 0x7fa8680a7ac0 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa858001700 con 0x7fa8680a7ac0 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 5 monclient: authenticate success, global_id 14230 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: dump: 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.685 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 >> v1:172.21.15.119:6789/0 conn(0x7fa8680a7ac0 legacy=0x7fa868137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa850003a10 0x7fa850003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): picked mon.a con 0x7fa850003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa850004810 0x7fa850004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): picked mon.c con 0x7fa850004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa85000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): picked mon.b con 0x7fa8500053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680a7ea0 con 0x7fa850003a10 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa85000c290 con 0x7fa8500053d0 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa85000c490 con 0x7fa850004810 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8667fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa86dd01640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa866ffd640 1 -- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa850004810 msgr2=0x7fa850004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa866ffd640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa850004810 0x7fa850004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa85000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): get_auth_request con 0x7fa8500053d0 auth_method 0 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa850003a10 0x7fa850003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 10 monclient(hunting): get_auth_request con 0x7fa850003a10 auth_method 0 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa8677fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:49.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient(hunting): handle_auth_done global_id 14230 payload 293 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 -- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa850004810 msgr2=0x7fa850004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa850004810 0x7fa850004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 -- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa850003a10 msgr2=0x7fa850003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa850003a10 0x7fa850003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 monclient: found mon.b 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 1 -- 172.21.15.119:0/1175250535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa850003530 con 0x7fa8500053d0 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.669+0000 7fa867fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa867fff640 1 -- 172.21.15.119:0/1175250535 reap_dead start 2024-09-15T21:18:49.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa867fff640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa85000b9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa860007360 tx=0x7fa86001ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30044c0e1429da99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860009be0 con 0x7fa8500053d0 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: dump: 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.690 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8600a5c10 con 0x7fa8500053d0 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 4 set_mon_vals no callback set 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 1 -- 172.21.15.119:0/1175250535 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600039a0 con 0x7fa8500053d0 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa8667fc640 10 monclient: dump: 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.691 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 10 monclient: shutdown 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 -- 172.21.15.119:0/1175250535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 msgr2=0x7fa85000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa85000b9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa860007360 tx=0x7fa86001ce60 comp rx=0 tx=0).stop 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 -- 172.21.15.119:0/1175250535 shutdown_connections 2024-09-15T21:18:49.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 --2- 172.21.15.119:0/1175250535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa85000b9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 -- 172.21.15.119:0/1175250535 >> 172.21.15.119:0/1175250535 conn(0x7fa8680a6fd0 msgr2=0x7fa8680a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 -- 172.21.15.119:0/1175250535 shutdown_connections 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 -- 172.21.15.119:0/1175250535 wait complete. 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 10 monclient: monmap: 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:49.693 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:49.694 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.673+0000 7fa86dd01640 1 Processor -- start 2024-09-15T21:18:49.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- start start 2024-09-15T21:18:49.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: init 2024-09-15T21:18:49.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:49.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa8681cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): picked mon.noname-b con 0x7fa8500053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86813edf0 0x7fa86813f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): picked mon.noname-a con 0x7fa86813edf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8680715a0 0x7fa868071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): picked mon.noname-c con 0x7fa8680715a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa868137e50 con 0x7fa86813edf0 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa868137cd0 con 0x7fa8500053d0 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa868137fd0 con 0x7fa8680715a0 2024-09-15T21:18:49.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient(hunting): authenticate will time out at 1048.476513s 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa867fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8680715a0 msgr2=0x7fa868071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa8681cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa867fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8680715a0 0x7fa868071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa8681cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40096/0 (socket says 172.21.15.119:40096) 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 -- 172.21.15.119:0/3489506838 learned_addr learned my addr 172.21.15.119:0/3489506838 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): get_auth_request con 0x7fa8500053d0 auth_method 0 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:49.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86813edf0 0x7fa86813f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 10 monclient(hunting): get_auth_request con 0x7fa86813edf0 auth_method 0 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient(hunting): handle_auth_done global_id 14233 payload 824 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 -- 172.21.15.119:0/3489506838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8680715a0 msgr2=0x7fa868071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8680715a0 0x7fa868071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 -- 172.21.15.119:0/3489506838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86813edf0 msgr2=0x7fa86813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86813edf0 0x7fa86813f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 monclient: found mon.noname-b 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 -- 172.21.15.119:0/3489506838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa868073060 con 0x7fa8500053d0 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa8677fe640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8500053d0 0x7fa8681cf6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa85800e600 tx=0x7fa85800eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858012590 con 0x7fa8500053d0 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: dump: 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.701 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858018070 con 0x7fa8500053d0 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 5 monclient: authenticate success, global_id 14233 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _renew_subs 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- 172.21.15.119:0/3489506838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86813f7b0 con 0x7fa8500053d0 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:49.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858012970 con 0x7fa8500053d0 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 10 monclient: dump: 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _renew_subs 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- 172.21.15.119:0/3489506838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa868139880 con 0x7fa8500053d0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa865ffb640 4 set_mon_vals no callback set 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa86dd01640 1 -- 172.21.15.119:0/3489506838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868138360 con 0x7fa8500053d0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa858012b10 con 0x7fa8500053d0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa85403b970 0x7fa85403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7fa85804f6c0 con 0x7fa8500053d0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa85403b970 0x7fa85403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 10 monclient: get_auth_request con 0x7fa85403b970 auth_method 0 2024-09-15T21:18:49.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.677+0000 7fa866ffd640 1 --2- 172.21.15.119:0/3489506838 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa85403b970 0x7fa85403de30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa868073b50 tx=0x7fa85c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:49.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.681+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa85801dbc0 con 0x7fa8500053d0 2024-09-15T21:18:49.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.681+0000 7fa84ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:49.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.681+0000 7fa84ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:49.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa86dd01640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-09-15T21:18:49.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa86dd01640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-09-15T21:18:49.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa86dd01640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:49.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa86dd01640 1 -- 172.21.15.119:0/3489506838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"} v 0) -- 0x7fa86806a760 con 0x7fa8500053d0 2024-09-15T21:18:49.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa84ffff640 1 -- 172.21.15.119:0/3489506838 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}]=-22 specified pg_num 1 < pg_num_min 2 v24) ==== 146+0+0 (secure 0 0 0) 0x7fa85804f030 con 0x7fa8500053d0 2024-09-15T21:18:49.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa84ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-09-15T21:18:49.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.853+0000 7fa84ffff640 10 monclient: _finish_command 2 = mon:22 specified pg_num 1 < pg_num_min 2 2024-09-15T21:18:49.860 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: specified pg_num 1 < pg_num_min 2 2024-09-15T21:18:49.865 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:49.865 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 44 2024-09-15T21:18:49.866 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:49.866 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num 44 2024-09-15T21:18:49.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient: get_monmap_and_config 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient: build_initial_monmap 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient: monmap: 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:49.985315+0000 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:49.985315+0000 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:49.987 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:49.988 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 1 Processor -- start 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 1 -- start start 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient: init 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdc28135bf0 0x7fdc2809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient(hunting): picked mon.noname-a con 0x7fdc28135bf0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdc28135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdc28136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc280a1870 con 0x7fdc28135fd0 2024-09-15T21:18:49.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc280a1d80 con 0x7fdc28136460 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fdc280a3600 con 0x7fdc28135bf0 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 10 monclient(hunting): authenticate will time out at 1048.781996s 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc28136460 0x7fdc280a1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42036/0 (socket says 172.21.15.119:42036) 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 1 -- 172.21.15.119:0/495681393 learned_addr learned my addr 172.21.15.119:0/495681393 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 1 --2- 172.21.15.119:0/495681393 >> v2:172.21.15.17:3300/0 conn(0x7fdc28135bf0 0x7fdc2809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79822283 0 0) 0x7fdc280a1d80 con 0x7fdc28136460 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): my global_id is 14236 2024-09-15T21:18:49.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc10003530 con 0x7fdc28136460 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 10 monclient(hunting): get_auth_request con 0x7fdc28135bf0 auth_method 0 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1300940437 0 0) 0x7fdc10003530 con 0x7fdc28136460 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 >> v2:172.21.15.17:3300/0 conn(0x7fdc28135bf0 msgr2=0x7fdc2809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 --2- 172.21.15.119:0/495681393 >> v2:172.21.15.17:3300/0 conn(0x7fdc28135bf0 0x7fdc2809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 >> v1:172.21.15.17:6789/0 conn(0x7fdc28135fd0 legacy=0x7fdc280a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc280a2060 con 0x7fdc28136460 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fdc20001a80 con 0x7fdc28136460 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: dump: 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:49.995 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:49.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 >> v1:172.21.15.119:6789/0 conn(0x7fdc28136460 legacy=0x7fdc280a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc10003a60 0x7fdc10003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): picked mon.c con 0x7fdc10003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): picked mon.b con 0x7fdc10004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc100053b0 0x7fdc1000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): picked mon.a con 0x7fdc100053b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280a3600 con 0x7fdc100053b0 2024-09-15T21:18:49.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1000c2f0 con 0x7fdc10004810 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1000c4f0 con 0x7fdc10003a60 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc26ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 1 -- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc10003a60 msgr2=0x7fdc10003e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc27fff640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc10003a60 0x7fdc10003e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): get_auth_request con 0x7fdc10004810 auth_method 0 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:49.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc100053b0 0x7fdc1000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 10 monclient(hunting): get_auth_request con 0x7fdc100053b0 auth_method 0 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient(hunting): handle_auth_done global_id 14236 payload 293 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 -- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc10003a60 msgr2=0x7fdc10003e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc10003a60 0x7fdc10003e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 -- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc100053b0 msgr2=0x7fdc1000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc100053b0 0x7fdc1000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:49.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 monclient: found mon.b 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 1 -- 172.21.15.119:0/495681393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc10003530 con 0x7fdc10004810 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 5 monclient: authenticate success, global_id 14236 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2e368640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc100053b0 0x7fdc1000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:18:50.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.981+0000 7fdc2c905640 1 -- 172.21.15.119:0/495681393 reap_dead start 2024-09-15T21:18:50.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc277fe640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc10004c90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c001790 tx=0x7fdc1c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c415bd23ce2b7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:50.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0013e0 con 0x7fdc10004810 2024-09-15T21:18:50.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:50.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: dump: 2024-09-15T21:18:50.009 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.009 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.009 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.009 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.009 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.010 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.010 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.010 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.010 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.010 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:50.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0052d0 con 0x7fdc10004810 2024-09-15T21:18:50.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:50.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:50.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 1 -- 172.21.15.119:0/495681393 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c003840 con 0x7fdc10004810 2024-09-15T21:18:50.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 4 set_mon_vals no callback set 2024-09-15T21:18:50.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:50.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc26ffd640 10 monclient: dump: 2024-09-15T21:18:50.012 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.012 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.013 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.013 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.013 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.013 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.013 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.013 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 10 monclient: shutdown 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 -- 172.21.15.119:0/495681393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 msgr2=0x7fdc10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc10004c90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c001790 tx=0x7fdc1c01c380 comp rx=0 tx=0).stop 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 -- 172.21.15.119:0/495681393 shutdown_connections 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 --2- 172.21.15.119:0/495681393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 -- 172.21.15.119:0/495681393 >> 172.21.15.119:0/495681393 conn(0x7fdc280a66e0 msgr2=0x7fdc280a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 -- 172.21.15.119:0/495681393 shutdown_connections 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.985+0000 7fdc2e368640 1 -- 172.21.15.119:0/495681393 wait complete. 2024-09-15T21:18:50.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: build_initial_monmap 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: monmap: 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 Processor -- start 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- start start 2024-09-15T21:18:50.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: init 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc281cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): picked mon.noname-b con 0x7fdc10004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc28125a10 0x7fdc281ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): picked mon.noname-c con 0x7fdc28125a10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc28123440 0x7fdc28123870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): picked mon.noname-a con 0x7fdc28123440 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2809ce50 con 0x7fdc28123440 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2809ccd0 con 0x7fdc10004810 2024-09-15T21:18:50.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280a1260 con 0x7fdc28125a10 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient(hunting): authenticate will time out at 1048.788918s 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc281cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc281cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40112/0 (socket says 172.21.15.119:40112) 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 -- 172.21.15.119:0/514799603 learned_addr learned my addr 172.21.15.119:0/514799603 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): get_auth_request con 0x7fdc10004810 auth_method 0 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc277fe640 1 -- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc28125a10 msgr2=0x7fdc281ce0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc277fe640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc28125a10 0x7fdc281ce0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:50.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2c905640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc28123440 0x7fdc28123870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient(hunting): handle_auth_done global_id 14239 payload 824 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 -- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc28125a10 msgr2=0x7fdc281ce0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc28125a10 0x7fdc281ce0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 -- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc28123440 msgr2=0x7fdc28123870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc28123440 0x7fdc28123870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 monclient: found mon.noname-b 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 -- 172.21.15.119:0/514799603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc28124eb0 con 0x7fdc10004810 2024-09-15T21:18:50.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc27fff640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc10004810 0x7fdc281cdb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e870 tx=0x7fdc1800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017020 con 0x7fdc10004810 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: dump: 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.019 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.020 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.020 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.020 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.020 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.020 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: _finish_auth 0 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1800c8f0 con 0x7fdc10004810 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 5 monclient: authenticate success, global_id 14239 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _renew_subs 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- 172.21.15.119:0/514799603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2809d240 con 0x7fdc10004810 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800f040 con 0x7fdc10004810 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 10 monclient: dump: 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.022 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc267fc640 4 set_mon_vals no callback set 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _renew_subs 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- 172.21.15.119:0/514799603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281ce5e0 con 0x7fdc10004810 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc2e368640 1 -- 172.21.15.119:0/514799603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc280a1620 con 0x7fdc10004810 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fdc1801a020 con 0x7fdc10004810 2024-09-15T21:18:50.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc1403b9a0 0x7fdc1403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7fdc1804f9c0 con 0x7fdc10004810 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc277fe640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc1403b9a0 0x7fdc1403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc277fe640 10 monclient: get_auth_request con 0x7fdc1403b9a0 auth_method 0 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.989+0000 7fdc277fe640 1 --2- 172.21.15.119:0/514799603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc1403b9a0 0x7fdc1403de60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0ac000 tx=0x7fdc1c0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.993+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fdc18021020 con 0x7fdc10004810 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.993+0000 7fdc24ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:50.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:49.993+0000 7fdc24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc2e368640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-09-15T21:18:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc2e368640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-09-15T21:18:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc2e368640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc2e368640 1 -- 172.21.15.119:0/514799603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"} v 0) -- 0x7fdc2806a830 con 0x7fdc10004810 2024-09-15T21:18:50.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc24ff9640 1 -- 172.21.15.119:0/514799603 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}]=-22 specified pg_num 44 < pg_num_max 33 v24) ==== 149+0+0 (secure 0 0 0) 0x7fdc1800ee60 con 0x7fdc10004810 2024-09-15T21:18:50.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc24ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-09-15T21:18:50.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.165+0000 7fdc24ff9640 10 monclient: _finish_command 2 = mon:22 specified pg_num 44 < pg_num_max 33 2024-09-15T21:18:50.173 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: specified pg_num 44 < pg_num_max 33 2024-09-15T21:18:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num_min 0 2024-09-15T21:18:50.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient: get_monmap_and_config 2024-09-15T21:18:50.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient: build_initial_monmap 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient: monmap: 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:50.293371+0000 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:50.293371+0000 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:50.295 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 Processor -- start 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 -- start start 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient: init 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e2809d550 0x7f7e2809d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): picked mon.noname-b con 0x7f7e2809d550 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e2809fae0 0x7f7e2809de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): picked mon.noname-c con 0x7f7e2809fae0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7e28134820 0x7f7e28134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): picked mon.noname-a con 0x7f7e28134820 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2809f1b0 con 0x7f7e28134820 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7e280a2920 con 0x7f7e2809fae0 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2809e3b0 con 0x7f7e2809d550 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2f0d4640 10 monclient(hunting): authenticate will time out at 1049.090535s 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2ce70640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e2809d550 msgr2=0x7f7e2809d930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2ce70640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e2809d550 0x7f7e2809d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:50.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e2809fae0 0x7f7e2809de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e2809fae0 0x7f7e2809de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40120/0 (socket says 172.21.15.119:40120) 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 -- 172.21.15.119:0/2366978225 learned_addr learned my addr 172.21.15.119:0/2366978225 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): get_auth_request con 0x7f7e2809fae0 auth_method 0 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2d671640 1 --2- 172.21.15.119:0/2366978225 >> v2:172.21.15.17:3300/0 conn(0x7f7e28134820 0x7f7e28134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2d671640 10 monclient(hunting): get_auth_request con 0x7f7e28134820 auth_method 0 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2d671640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:50.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2d671640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e2d671640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient(hunting): handle_auth_done global_id 14242 payload 274 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 -- 172.21.15.119:0/2366978225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e2809d550 msgr2=0x7f7e2809d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 --2- 172.21.15.119:0/2366978225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e2809d550 0x7f7e2809d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 -- 172.21.15.119:0/2366978225 >> v2:172.21.15.17:3300/0 conn(0x7f7e28134820 msgr2=0x7f7e28134cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 --2- 172.21.15.119:0/2366978225 >> v2:172.21.15.17:3300/0 conn(0x7f7e28134820 0x7f7e28134cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 monclient: found mon.noname-c 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.289+0000 7f7e27fff640 1 -- 172.21.15.119:0/2366978225 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e280a3600 con 0x7f7e2809fae0 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e27fff640 1 --2- 172.21.15.119:0/2366978225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809fae0 0x7f7e2809de70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e280a37d0 tx=0x7f7e1401c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbcc1737b9060c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 1 -- 172.21.15.119:0/2366978225 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140b0070 con 0x7f7e2809fae0 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: dump: 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.306 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 1 -- 172.21.15.119:0/2366978225 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e140ab600 con 0x7f7e2809fae0 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 5 monclient: authenticate success, global_id 14242 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:50.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 4 set_mon_vals no callback set 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 1 -- 172.21.15.119:0/2366978225 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140a4e60 con 0x7f7e2809fae0 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e277fe640 10 monclient: dump: 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.308 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 10 monclient: shutdown 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/2366978225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809fae0 msgr2=0x7f7e2809de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/2366978225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809fae0 0x7f7e2809de70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e280a37d0 tx=0x7f7e1401c400 comp rx=0 tx=0).stop 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/2366978225 shutdown_connections 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/2366978225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809fae0 0x7f7e2809de70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/2366978225 >> v2:172.21.15.17:3300/0 conn(0x7f7e28134820 0x7f7e28134cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/2366978225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e2809d550 0x7f7e2809d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/2366978225 >> 172.21.15.119:0/2366978225 conn(0x7f7e280a66e0 msgr2=0x7f7e280a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/2366978225 shutdown_connections 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/2366978225 wait complete. 2024-09-15T21:18:50.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 10 monclient: build_initial_monmap 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 10 monclient: monmap: 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 Processor -- start 2024-09-15T21:18:50.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 1 -- start start 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.293+0000 7f7e2f0d4640 10 monclient: init 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 0x7f7e28132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): picked mon.noname-b con 0x7f7e2809d550 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e2809fae0 0x7f7e28133520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): picked mon.noname-a con 0x7f7e2809fae0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e28134820 0x7f7e2813a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): picked mon.noname-c con 0x7f7e28134820 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2809ce50 con 0x7f7e2809fae0 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2809ccd0 con 0x7f7e2809d550 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7e280a1260 con 0x7f7e28134820 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient(hunting): authenticate will time out at 1049.096301s 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 0x7f7e28132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 0x7f7e28132fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40130/0 (socket says 172.21.15.119:40130) 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 -- 172.21.15.119:0/1613357962 learned_addr learned my addr 172.21.15.119:0/1613357962 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2d671640 1 -- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e28134820 msgr2=0x7f7e2813a5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): get_auth_request con 0x7f7e2809d550 auth_method 0 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:50.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2d671640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e28134820 0x7f7e2813a5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient(hunting): handle_auth_done global_id 14245 payload 824 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e27fff640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e2809fae0 0x7f7e28133520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 -- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e28134820 msgr2=0x7f7e2813a5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e28134820 0x7f7e2813a5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 -- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e2809fae0 msgr2=0x7f7e28133520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e2809fae0 0x7f7e28133520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient: _finish_hunting 0 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 monclient: found mon.noname-b 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:50.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 -- 172.21.15.119:0/1613357962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e280a5780 con 0x7f7e2809d550 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2ce70640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 0x7f7e28132fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b530 tx=0x7f7e1c00b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c013020 con 0x7f7e2809d550 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: dump: 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1c003df0 con 0x7f7e2809d550 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 5 monclient: authenticate success, global_id 14245 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _renew_subs 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2813dad0 con 0x7f7e2809d550 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0052b0 con 0x7f7e2809d550 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:50.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 10 monclient: dump: 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e26ffd640 4 set_mon_vals no callback set 2024-09-15T21:18:50.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _renew_subs 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2813d830 con 0x7f7e2809d550 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7e1c016020 con 0x7f7e2809d550 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e280a16e0 con 0x7f7e2809d550 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e0c03b970 0x7f7e0c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f7e1c04f710 con 0x7f7e2809d550 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e27fff640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e0c03b970 0x7f7e0c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e27fff640 10 monclient: get_auth_request con 0x7f7e0c03b970 auth_method 0 2024-09-15T21:18:50.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.297+0000 7f7e27fff640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e0c03b970 0x7f7e0c03de30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e140ad000 tx=0x7f7e14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:50.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.301+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7e1c01d020 con 0x7f7e2809d550 2024-09-15T21:18:50.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.301+0000 7f7e257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:50.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.301+0000 7f7e257fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:50.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.457+0000 7f7e2f0d4640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T21:18:50.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.457+0000 7f7e2f0d4640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T21:18:50.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.457+0000 7f7e2f0d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.457+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) -- 0x7f7e2813c010 con 0x7f7e2809d550 2024-09-15T21:18:50.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.537+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v25) ==== 143+0+0 (secure 0 0 0) 0x7f7e280a5780 con 0x7f7e2809d550 2024-09-15T21:18:50.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.537+0000 7f7e257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T21:18:50.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.537+0000 7f7e257fa640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 0 2024-09-15T21:18:50.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.641+0000 7f7e2f0d4640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T21:18:50.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.641+0000 7f7e2f0d4640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T21:18:50.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.641+0000 7f7e2f0d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:50.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:50.641+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) -- 0x7f7e28120d60 con 0x7f7e2809d550 2024-09-15T21:18:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.553+0000 7f7e257fa640 1 -- 172.21.15.119:0/1613357962 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v26) ==== 143+0+0 (secure 0 0 0) 0x7f7e1c00d380 con 0x7f7e2809d550 2024-09-15T21:18:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.553+0000 7f7e257fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T21:18:51.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.553+0000 7f7e257fa640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 0 2024-09-15T21:18:51.561 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 3 pg_num_min to 0 2024-09-15T21:18:51.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e0c03b970 msgr2=0x7f7e0c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e0c03b970 0x7f7e0c03de30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e140ad000 tx=0x7f7e14008040 comp rx=0 tx=0).stop 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 10 monclient: shutdown 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 msgr2=0x7f7e28132fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 0x7f7e28132fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b530 tx=0x7f7e1c00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 shutdown_connections 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e0c03b970 0x7f7e0c03de30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e28134820 0x7f7e2813a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e2809fae0 0x7f7e28133520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 --2- 172.21.15.119:0/1613357962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2809d550 0x7f7e28132fe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 >> 172.21.15.119:0/1613357962 conn(0x7f7e280a66e0 msgr2=0x7f7e28071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 shutdown_connections 2024-09-15T21:18:51.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.557+0000 7f7e2f0d4640 1 -- 172.21.15.119:0/1613357962 wait complete. 2024-09-15T21:18:51.582 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_min 2024-09-15T21:18:51.582 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:51.582 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-09-15T21:18:51.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient: get_monmap_and_config 2024-09-15T21:18:51.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient: build_initial_monmap 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient: monmap: 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:51.701483+0000 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:51.701483+0000 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:51.703 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:51.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 Processor -- start 2024-09-15T21:18:51.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 -- start start 2024-09-15T21:18:51.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient: init 2024-09-15T21:18:51.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:51.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:18:51.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f64ac0a79e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:51.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64ac0a1100 0x7f64ac13cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:51.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): picked mon.noname-a con 0x7f64ac0a1100 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64ac13d0a0 0x7f64ac13f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): picked mon.noname-c con 0x7f64ac13d0a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ac135180 con 0x7f64ac0a79e0 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0a7dc0 con 0x7f64ac0a1100 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0a6920 con 0x7f64ac13d0a0 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64b0c72640 10 monclient(hunting): authenticate will time out at 1050.498909s 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64ac13d0a0 0x7f64ac13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64ac13d0a0 0x7f64ac13f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40134/0 (socket says 172.21.15.119:40134) 2024-09-15T21:18:51.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 -- 172.21.15.119:0/2864014734 learned_addr learned my addr 172.21.15.119:0/2864014734 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): get_auth_request con 0x7f64ac13d0a0 auth_method 0 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aa59c640 1 --1- 172.21.15.119:0/2864014734 >> v1:172.21.15.119:6789/0 conn(0x7f64ac0a79e0 0x7f64ac0a4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42050/0 (socket says 172.21.15.119:42050) 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a1d9b640 1 --2- 172.21.15.119:0/2864014734 >> v2:172.21.15.17:3300/0 conn(0x7f64ac0a1100 0x7f64ac13cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:51.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a1d9b640 10 monclient(hunting): get_auth_request con 0x7f64ac0a1100 auth_method 0 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient(hunting): handle_auth_done global_id 14248 payload 274 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 -- 172.21.15.119:0/2864014734 >> v2:172.21.15.17:3300/0 conn(0x7f64ac0a1100 msgr2=0x7f64ac13cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 --2- 172.21.15.119:0/2864014734 >> v2:172.21.15.17:3300/0 conn(0x7f64ac0a1100 0x7f64ac13cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 -- 172.21.15.119:0/2864014734 >> v1:172.21.15.119:6789/0 conn(0x7f64ac0a79e0 legacy=0x7f64ac0a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 monclient: found mon.noname-c 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a9d9b640 1 -- 172.21.15.119:0/2864014734 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956378404 0 0) 0x7f64ac135180 con 0x7f64ac0a79e0 2024-09-15T21:18:51.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64aad9d640 1 -- 172.21.15.119:0/2864014734 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0a1c30 con 0x7f64ac13d0a0 2024-09-15T21:18:51.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.697+0000 7f64a9d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:51.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64aad9d640 1 --2- 172.21.15.119:0/2864014734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac13d0a0 0x7f64ac13f490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0a20e0 tx=0x7f64a401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f53576bb402556 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:51.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 1 -- 172.21.15.119:0/2864014734 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40a5820 con 0x7f64ac13d0a0 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: dump: 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:51.715 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: _finish_auth 0 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 1 -- 172.21.15.119:0/2864014734 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a40a59c0 con 0x7f64ac13d0a0 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 5 monclient: authenticate success, global_id 14248 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 1 -- 172.21.15.119:0/2864014734 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40a5d90 con 0x7f64ac13d0a0 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 4 set_mon_vals no callback set 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:51.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a9d9b640 10 monclient: dump: 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: shutdown 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- 172.21.15.119:0/2864014734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac13d0a0 msgr2=0x7f64ac13f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:51.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 --2- 172.21.15.119:0/2864014734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac13d0a0 0x7f64ac13f490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0a20e0 tx=0x7f64a401c940 comp rx=0 tx=0).stop 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- 172.21.15.119:0/2864014734 shutdown_connections 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 --2- 172.21.15.119:0/2864014734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac13d0a0 0x7f64ac13f490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 --2- 172.21.15.119:0/2864014734 >> v2:172.21.15.17:3300/0 conn(0x7f64ac0a1100 0x7f64ac13cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- 172.21.15.119:0/2864014734 >> 172.21.15.119:0/2864014734 conn(0x7f64ac0a6ef0 msgr2=0x7f64ac0a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- 172.21.15.119:0/2864014734 shutdown_connections 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- 172.21.15.119:0/2864014734 wait complete. 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: build_initial_monmap 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: monmap: 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:51.718 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 Processor -- start 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- start start 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: init 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64ac0a1100 0x7f64ac130da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): picked mon.noname-c con 0x7f64ac0a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac0a79e0 0x7f64ac1312e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): picked mon.noname-b con 0x7f64ac0a79e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64ac13d0a0 0x7f64ac0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): picked mon.noname-a con 0x7f64ac13d0a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1348d0 con 0x7f64ac13d0a0 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac134750 con 0x7f64ac0a79e0 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac1421f0 con 0x7f64ac0a1100 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64b0c72640 10 monclient(hunting): authenticate will time out at 1050.503902s 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.701+0000 7f64a1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac0a79e0 0x7f64ac1312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:51.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac0a79e0 0x7f64ac1312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40136/0 (socket says 172.21.15.119:40136) 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aa59c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64ac0a1100 msgr2=0x7f64ac130da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 -- 172.21.15.119:0/642825966 learned_addr learned my addr 172.21.15.119:0/642825966 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): get_auth_request con 0x7f64ac0a79e0 auth_method 0 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aa59c640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64ac0a1100 0x7f64ac130da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aad9d640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64ac13d0a0 0x7f64ac0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient(hunting): handle_auth_done global_id 14254 payload 824 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 -- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64ac0a1100 msgr2=0x7f64ac130da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64ac0a1100 0x7f64ac130da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 -- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64ac13d0a0 msgr2=0x7f64ac0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64ac13d0a0 0x7f64ac0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 monclient: found mon.noname-b 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 -- 172.21.15.119:0/642825966 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac073c10 con 0x7f64ac0a79e0 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aad9d640 10 monclient: get_auth_request con 0x7f64ac13d0a0 auth_method 0 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aad9d640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64ac13d0a0 0x7f64ac0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a1d9b640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64ac0a79e0 0x7f64ac1312e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f649400d800 tx=0x7f649400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649400b6c0 con 0x7f64ac0a79e0 2024-09-15T21:18:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: dump: 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:51.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494019070 con 0x7f64ac0a79e0 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649400ba60 con 0x7f64ac0a79e0 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 5 monclient: authenticate success, global_id 14254 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 monclient: _renew_subs 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 1 -- 172.21.15.119:0/642825966 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac136710 con 0x7f64ac0a79e0 2024-09-15T21:18:51.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a959a640 4 set_mon_vals no callback set 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 10 monclient: dump: 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:51.725 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 monclient: _renew_subs 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 1 -- 172.21.15.119:0/642825966 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac1353b0 con 0x7f64ac0a79e0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64b0c72640 1 -- 172.21.15.119:0/642825966 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac0a4170 con 0x7f64ac0a79e0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f649400bc00 con 0x7f64ac0a79e0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f648c03b9a0 0x7f648c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4713+0+0 (secure 0 0 0) 0x7f649404ff50 con 0x7f64ac0a79e0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aa59c640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f648c03b9a0 0x7f648c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aa59c640 10 monclient: get_auth_request con 0x7f648c03b9a0 auth_method 0 2024-09-15T21:18:51.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.705+0000 7f64aa59c640 1 --2- 172.21.15.119:0/642825966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f648c03b9a0 0x7f648c03de60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f649c001090 tx=0x7f649c0079f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:51.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.709+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f649401cd70 con 0x7f64ac0a79e0 2024-09-15T21:18:51.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.709+0000 7f64a37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:51.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.709+0000 7f64a37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:51.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64b0c72640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:51.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64b0c72640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:51.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64b0c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:51.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64b0c72640 1 -- 172.21.15.119:0/642825966 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) -- 0x7f64ac06a760 con 0x7f64ac0a79e0 2024-09-15T21:18:51.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64a37fe640 1 -- 172.21.15.119:0/642825966 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=-2 option 'pg_num_min' is not set on pool 'testpool1234' v26) ==== 158+0+0 (secure 0 0 0) 0x7f6494013cb0 con 0x7f64ac0a79e0 2024-09-15T21:18:51.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64a37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T21:18:51.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:51.877+0000 7f64a37fe640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_min' is not set on pool 'testpool1234' 2024-09-15T21:18:51.883 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOENT: option 'pg_num_min' is not set on pool 'testpool1234' 2024-09-15T21:18:51.888 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:51.888 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set testpool1234 pg_num_max 0 2024-09-15T21:18:52.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient: get_monmap_and_config 2024-09-15T21:18:52.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient: build_initial_monmap 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient: monmap: 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:52.005363+0000 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:52.005363+0000 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:52.007 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:52.008 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:52.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 Processor -- start 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 -- start start 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient: init 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39a809ca70 0x7f39a80a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): picked mon.noname-c con 0x7f39a809ca70 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f39a809fd80 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39a80a01f0 0x7f39a81343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): picked mon.noname-a con 0x7f39a80a01f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a809dfd0 con 0x7f39a809fd80 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:52.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f39a809ce50 con 0x7f39a80a01f0 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f39a809e2b0 con 0x7f39a809ca70 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39ae0b9640 10 monclient(hunting): authenticate will time out at 1050.802136s 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39a809ca70 0x7f39a80a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f399ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f39a809fd80 0x7f39a8133ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42056/0 (socket says 172.21.15.119:42056) 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f399ffff640 1 -- 172.21.15.119:0/662214586 learned_addr learned my addr 172.21.15.119:0/662214586 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): get_auth_request con 0x7f39a809ca70 auth_method 0 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:52.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a7fff640 1 --2- 172.21.15.119:0/662214586 >> v2:172.21.15.17:3300/0 conn(0x7f39a80a01f0 0x7f39a81343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient(hunting): handle_auth_done global_id 14257 payload 274 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 1 -- 172.21.15.119:0/662214586 >> v2:172.21.15.17:3300/0 conn(0x7f39a80a01f0 msgr2=0x7f39a81343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 1 --2- 172.21.15.119:0/662214586 >> v2:172.21.15.17:3300/0 conn(0x7f39a80a01f0 0x7f39a81343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a6ffd640 1 -- 172.21.15.119:0/662214586 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3163751807 0 0) 0x7f39a809dfd0 con 0x7f39a809fd80 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 1 -- 172.21.15.119:0/662214586 >> v1:172.21.15.119:6789/0 conn(0x7f39a809fd80 legacy=0x7f39a8133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 1 monclient: found mon.noname-c 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a77fe640 1 -- 172.21.15.119:0/662214586 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80a1a50 con 0x7f39a809ca70 2024-09-15T21:18:52.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a7fff640 10 monclient: get_auth_request con 0x7f39a80a01f0 auth_method 0 2024-09-15T21:18:52.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a7fff640 1 --2- 172.21.15.119:0/662214586 >> v2:172.21.15.17:3300/0 conn(0x7f39a80a01f0 0x7f39a81343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:52.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.001+0000 7f39a6ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 --2- 172.21.15.119:0/662214586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a80a0f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39a809d010 tx=0x7f399801ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ff720ef335f3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 1 -- 172.21.15.119:0/662214586 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b4070 con 0x7f39a809ca70 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: dump: 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 1 -- 172.21.15.119:0/662214586 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980a7970 con 0x7f39a809ca70 2024-09-15T21:18:52.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 1 -- 172.21.15.119:0/662214586 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980a7d40 con 0x7f39a809ca70 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 5 monclient: authenticate success, global_id 14257 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 4 set_mon_vals no callback set 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a6ffd640 10 monclient: dump: 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:52.019 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: shutdown 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/662214586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 msgr2=0x7f39a80a0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/662214586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a80a0f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39a809d010 tx=0x7f399801ea10 comp rx=0 tx=0).stop 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/662214586 shutdown_connections 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/662214586 >> v2:172.21.15.17:3300/0 conn(0x7f39a80a01f0 0x7f39a81343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/662214586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a80a0f60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/662214586 >> 172.21.15.119:0/662214586 conn(0x7f39a80a66e0 msgr2=0x7f39a80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/662214586 shutdown_connections 2024-09-15T21:18:52.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/662214586 wait complete. 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: build_initial_monmap 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: monmap: 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:52.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 Processor -- start 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- start start 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: init 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a813ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): picked mon.noname-b con 0x7f39a809ca70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39a809fd80 0x7f39a8130840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): picked mon.noname-a con 0x7f39a809fd80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f39a80a01f0 0x7f39a8130d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): picked mon.noname-c con 0x7f39a80a01f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f39a813e090 con 0x7f39a809fd80 2024-09-15T21:18:52.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8134b40 con 0x7f39a809ca70 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f39a813e210 con 0x7f39a80a01f0 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient(hunting): authenticate will time out at 1050.807085s 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a813ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a813ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40158/0 (socket says 172.21.15.119:40158) 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 -- 172.21.15.119:0/1753060998 learned_addr learned my addr 172.21.15.119:0/1753060998 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): get_auth_request con 0x7f39a809ca70 auth_method 0 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:52.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a7fff640 1 -- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f39a80a01f0 msgr2=0x7f39a8130d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a7fff640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f39a80a01f0 0x7f39a8130d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient(hunting): handle_auth_done global_id 14263 payload 824 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 -- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f39a80a01f0 msgr2=0x7f39a8130d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f39a80a01f0 0x7f39a8130d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 -- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39a809fd80 msgr2=0x7f39a8130840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39a809fd80 0x7f39a8130840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 monclient: found mon.noname-b 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:52.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 -- 172.21.15.119:0/1753060998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a811f310 con 0x7f39a809ca70 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a77fe640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a813ad40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39980b2040 tx=0x7f399801b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980bf070 con 0x7f39a809ca70 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: dump: 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:52.025 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980af7c0 con 0x7f39a809ca70 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 5 monclient: authenticate success, global_id 14263 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: _renew_subs 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a80a4ab0 con 0x7f39a809ca70 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b4040 con 0x7f39a809ca70 2024-09-15T21:18:52.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a4ff9640 10 monclient: dump: 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:52.028 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: _renew_subs 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39a67fc640 4 set_mon_vals no callback set 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.005+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a813b710 con 0x7f39a809ca70 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39ae0b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39ae0b9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39ae0b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a80a0f60 con 0x7f39a809ca70 2024-09-15T21:18:52.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f39980c2070 con 0x7f39a809ca70 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39a4ff9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f398803b970 0x7f398803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4713+0+0 (secure 0 0 0) 0x7f39980e9ca0 con 0x7f39a809ca70 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f399ffff640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f398803b970 0x7f398803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f399ffff640 10 monclient: get_auth_request con 0x7f398803b970 auth_method 0 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.009+0000 7f399ffff640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f398803b970 0x7f398803de30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3990007fd0 tx=0x7f39900078e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.013+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3998018720 con 0x7f39a809ca70 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.013+0000 7f39a4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:52.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.013+0000 7f39a4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:52.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.181+0000 7f39ae0b9640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T21:18:52.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.181+0000 7f39ae0b9640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T21:18:52.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.181+0000 7f39ae0b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.181+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) -- 0x7f39a811db10 con 0x7f39a809ca70 2024-09-15T21:18:52.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.557+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v27) ==== 143+0+0 (secure 0 0 0) 0x7f39980b7c60 con 0x7f39a809ca70 2024-09-15T21:18:52.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.557+0000 7f39a4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T21:18:52.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.557+0000 7f39a4ff9640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 0 2024-09-15T21:18:52.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.657+0000 7f39ae0b9640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T21:18:52.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.657+0000 7f39ae0b9640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T21:18:52.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.657+0000 7f39ae0b9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:52.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:52.657+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) -- 0x7f39a806a830 con 0x7f39a809ca70 2024-09-15T21:18:53.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.569+0000 7f39a4ff9640 1 -- 172.21.15.119:0/1753060998 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v28) ==== 143+0+0 (secure 0 0 0) 0x7f39980c9030 con 0x7f39a809ca70 2024-09-15T21:18:53.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.569+0000 7f39a4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T21:18:53.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.569+0000 7f39a4ff9640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 0 2024-09-15T21:18:53.576 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 3 pg_num_max to 0 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f398803b970 msgr2=0x7f398803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f398803b970 0x7f398803de30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3990007fd0 tx=0x7f39900078e0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 10 monclient: shutdown 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 msgr2=0x7f39a813ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a813ad40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39980b2040 tx=0x7f399801b7b0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 shutdown_connections 2024-09-15T21:18:53.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f398803b970 0x7f398803de30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f39a80a01f0 0x7f39a8130d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39a809fd80 0x7f39a8130840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 --2- 172.21.15.119:0/1753060998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a809ca70 0x7f39a813ad40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 >> 172.21.15.119:0/1753060998 conn(0x7f39a80a66e0 msgr2=0x7f39a80a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:53.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 shutdown_connections 2024-09-15T21:18:53.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.573+0000 7f39ae0b9640 1 -- 172.21.15.119:0/1753060998 wait complete. 2024-09-15T21:18:53.597 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_max 2024-09-15T21:18:53.597 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:53.597 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-09-15T21:18:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.709+0000 7f338268d640 10 monclient: get_monmap_and_config 2024-09-15T21:18:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.709+0000 7f338268d640 10 monclient: build_initial_monmap 2024-09-15T21:18:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.709+0000 7f338268d640 10 monclient: monmap: 2024-09-15T21:18:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:53.716222+0000 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:53.716222+0000 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:53.718 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 Processor -- start 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 -- start start 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient: init 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f337c0a79e0 0x7f337c0a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): picked mon.noname-a con 0x7f337c0a79e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f337c0a46b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): picked mon.noname-b con 0x7f337c0a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f337c135180 con 0x7f337c0a46b0 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f337c0a7dc0 con 0x7f337c0a79e0 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f337c0a6920 con 0x7f337c0a1100 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f338268d640 10 monclient(hunting): authenticate will time out at 1052.513273s 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f3380c2a640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 msgr2=0x7f337c137ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f3380c2a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f337c0a79e0 0x7f337c0a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f337c0a79e0 0x7f337c0a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54750/0 (socket says 172.21.15.119:54750) 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 -- 172.21.15.119:0/421918641 learned_addr learned my addr 172.21.15.119:0/421918641 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): get_auth_request con 0x7f337c0a79e0 auth_method 0 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337b7fe640 1 --1- 172.21.15.119:0/421918641 >> v1:172.21.15.17:6789/0 conn(0x7f337c0a46b0 0x7f337c1377e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60022/0 (socket says 172.21.15.119:60022) 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient(hunting): handle_auth_done global_id 4575 payload 274 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 -- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 msgr2=0x7f337c137ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 --2- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 -- 172.21.15.119:0/421918641 >> v1:172.21.15.17:6789/0 conn(0x7f337c0a46b0 legacy=0x7f337c1377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 monclient: found mon.noname-a 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.713+0000 7f337bfff640 1 -- 172.21.15.119:0/421918641 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c0a1c30 con 0x7f337c0a79e0 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337bfff640 1 --2- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 0x7f337c0a4170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f336c009870 tx=0x7f336c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456acac4b6a2781a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 1 -- 172.21.15.119:0/421918641 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c134930 con 0x7f337c0a79e0 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: dump: 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.729 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: _finish_auth 0 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 5 monclient: authenticate success, global_id 4575 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 1 -- 172.21.15.119:0/421918641 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336c0af070 con 0x7f337c0a79e0 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:53.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 1 -- 172.21.15.119:0/421918641 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c0aa870 con 0x7f337c0a79e0 2024-09-15T21:18:53.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:53.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 4 set_mon_vals no callback set 2024-09-15T21:18:53.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:53.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337affd640 10 monclient: dump: 2024-09-15T21:18:53.731 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:53.731 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: shutdown 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 msgr2=0x7f337c0a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:53.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 --2- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 0x7f337c0a4170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f336c009870 tx=0x7f336c01c4d0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- 172.21.15.119:0/421918641 shutdown_connections 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 --2- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 --2- 172.21.15.119:0/421918641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 0x7f337c0a4170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- 172.21.15.119:0/421918641 >> 172.21.15.119:0/421918641 conn(0x7f337c0a6ef0 msgr2=0x7f337c0a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- 172.21.15.119:0/421918641 shutdown_connections 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- 172.21.15.119:0/421918641 wait complete. 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: build_initial_monmap 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: monmap: 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:53.733 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 Processor -- start 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- start start 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: init 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): picked mon.noname-c con 0x7f337c0a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:53.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f337c0a46b0 0x7f337c072a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): picked mon.noname-b con 0x7f337c0a46b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 0x7f337c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): picked mon.noname-a con 0x7f337c0a79e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f337c138670 con 0x7f337c0a79e0 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1384f0 con 0x7f337c0a46b0 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f337c1423a0 con 0x7f337c0a1100 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f338268d640 10 monclient(hunting): authenticate will time out at 1052.519218s 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f337c0a46b0 0x7f337c072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:53.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337bfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 msgr2=0x7f337c072540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c072540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f337c0a46b0 0x7f337c072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37834/0 (socket says 172.21.15.119:37834) 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 -- 172.21.15.119:0/250917004 learned_addr learned my addr 172.21.15.119:0/250917004 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): get_auth_request con 0x7f337c0a46b0 auth_method 0 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f3380c2a640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 0x7f337c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f3380c2a640 10 monclient(hunting): get_auth_request con 0x7f337c0a79e0 auth_method 0 2024-09-15T21:18:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f3380c2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f3380c2a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f3380c2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient(hunting): handle_auth_done global_id 14266 payload 824 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 -- 172.21.15.119:0/250917004 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 msgr2=0x7f337c072540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f337c0a1100 0x7f337c072540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 -- 172.21.15.119:0/250917004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 msgr2=0x7f337c072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f337c0a79e0 0x7f337c072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 -- 172.21.15.119:0/250917004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c133cd0 con 0x7f337c0a46b0 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.717+0000 7f337b7fe640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f337c0a46b0 0x7f337c072a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f337000c4b0 tx=0x7f337000c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:53.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370017070 con 0x7f337c0a46b0 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: dump: 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:53.738 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: _finish_auth 0 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3370005c30 con 0x7f337c0a46b0 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 5 monclient: authenticate success, global_id 14266 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 monclient: _renew_subs 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 1 -- 172.21.15.119:0/250917004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c074000 con 0x7f337c0a46b0 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370007270 con 0x7f337c0a46b0 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 10 monclient: dump: 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:53.739 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 monclient: _renew_subs 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 1 -- 172.21.15.119:0/250917004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c138fd0 con 0x7f337c0a46b0 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f337a7fc640 4 set_mon_vals no callback set 2024-09-15T21:18:53.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f338268d640 1 -- 172.21.15.119:0/250917004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337c0008e0 con 0x7f337c0a46b0 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f337001a070 con 0x7f337c0a46b0 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f336803b970 0x7f336803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4697+0+0 (secure 0 0 0) 0x7f337004ec90 con 0x7f337c0a46b0 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f337bfff640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f336803b970 0x7f336803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f337bfff640 10 monclient: get_auth_request con 0x7f336803b970 auth_method 0 2024-09-15T21:18:53.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.721+0000 7f337bfff640 1 --2- 172.21.15.119:0/250917004 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f336803b970 0x7f336803de30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f336c0ae000 tx=0x7f336c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:53.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.725+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3370005e20 con 0x7f337c0a46b0 2024-09-15T21:18:53.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.725+0000 7f3378ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:53.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.725+0000 7f3378ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:53.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f338268d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:53.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f338268d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:53.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f338268d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:53.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f338268d640 1 -- 172.21.15.119:0/250917004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) -- 0x7f337c06a760 con 0x7f337c0a46b0 2024-09-15T21:18:53.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f3378ff9640 1 -- 172.21.15.119:0/250917004 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=-2 option 'pg_num_max' is not set on pool 'testpool1234' v28) ==== 158+0+0 (secure 0 0 0) 0x7f337001b3b0 con 0x7f337c0a46b0 2024-09-15T21:18:53.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f3378ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T21:18:53.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:53.881+0000 7f3378ff9640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_max' is not set on pool 'testpool1234' 2024-09-15T21:18:53.889 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOENT: option 'pg_num_max' is not set on pool 'testpool1234' 2024-09-15T21:18:53.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:53.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete testpool1234 testpool1234 --yes-i-really-really-mean-it 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient: get_monmap_and_config 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient: build_initial_monmap 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient: monmap: 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:54.006603+0000 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:54.006603+0000 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.008 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:54.009 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:54.009 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:54.009 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:54.009 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:54.009 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 Processor -- start 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 -- start start 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient: init 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d2809ca70 0x7f2d280a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): picked mon.noname-c con 0x7f2d2809ca70 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d2809fd80 0x7f2d280a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): picked mon.noname-b con 0x7f2d2809fd80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d28133ce0 0x7f2d28134190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): picked mon.noname-a con 0x7f2d28133ce0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2809ce50 con 0x7f2d28133ce0 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2809dfd0 con 0x7f2d2809ca70 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2809e2b0 con 0x7f2d2809fd80 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d2e4d9640 10 monclient(hunting): authenticate will time out at 1052.803789s 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d2809ca70 0x7f2d280a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d2809ca70 0x7f2d280a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37838/0 (socket says 172.21.15.119:37838) 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 1 -- 172.21.15.119:0/456753248 learned_addr learned my addr 172.21.15.119:0/456753248 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:54.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 10 monclient(hunting): get_auth_request con 0x7f2d2809ca70 auth_method 0 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.001+0000 7f2d1f7fe640 1 -- 172.21.15.119:0/456753248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d2809fd80 msgr2=0x7f2d280a01c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d1f7fe640 1 --2- 172.21.15.119:0/456753248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d2809fd80 0x7f2d280a01c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2ca76640 1 --2- 172.21.15.119:0/456753248 >> v2:172.21.15.17:3300/0 conn(0x7f2d28133ce0 0x7f2d28134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 10 monclient(hunting): handle_auth_done global_id 14269 payload 274 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 -- 172.21.15.119:0/456753248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d2809fd80 msgr2=0x7f2d280a01c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 --2- 172.21.15.119:0/456753248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d2809fd80 0x7f2d280a01c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 -- 172.21.15.119:0/456753248 >> v2:172.21.15.17:3300/0 conn(0x7f2d28133ce0 msgr2=0x7f2d28134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 --2- 172.21.15.119:0/456753248 >> v2:172.21.15.17:3300/0 conn(0x7f2d28133ce0 0x7f2d28134190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 monclient: found mon.noname-c 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 -- 172.21.15.119:0/456753248 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280a1a50 con 0x7f2d2809ca70 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2ca76640 10 monclient: get_auth_request con 0x7f2d28133ce0 auth_method 0 2024-09-15T21:18:54.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2ca76640 1 --2- 172.21.15.119:0/456753248 >> v2:172.21.15.17:3300/0 conn(0x7f2d28133ce0 0x7f2d28134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:54.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d27fff640 1 --2- 172.21.15.119:0/456753248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809ca70 0x7f2d280a0f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d2809d010 tx=0x7f2d1801d7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2484eb0c92eba5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:54.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 1 -- 172.21.15.119:0/456753248 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800b040 con 0x7f2d2809ca70 2024-09-15T21:18:54.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: dump: 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:54.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 5 monclient: authenticate success, global_id 14269 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 1 -- 172.21.15.119:0/456753248 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d180a7700 con 0x7f2d2809ca70 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 1 -- 172.21.15.119:0/456753248 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d180a7ad0 con 0x7f2d2809ca70 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 4 set_mon_vals no callback set 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d277fe640 10 monclient: dump: 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.020 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 10 monclient: shutdown 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/456753248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809ca70 msgr2=0x7f2d280a0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/456753248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809ca70 0x7f2d280a0f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d2809d010 tx=0x7f2d1801d7d0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/456753248 shutdown_connections 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/456753248 >> v2:172.21.15.17:3300/0 conn(0x7f2d28133ce0 0x7f2d28134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/456753248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d2809fd80 0x7f2d280a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/456753248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809ca70 0x7f2d280a0f60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/456753248 >> 172.21.15.119:0/456753248 conn(0x7f2d280a66e0 msgr2=0x7f2d280a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/456753248 shutdown_connections 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/456753248 wait complete. 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 10 monclient: build_initial_monmap 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.005+0000 7f2d2e4d9640 10 monclient: monmap: 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:54.022 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 Processor -- start 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- start start 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: init 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2809ca70 0x7f2d281cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): picked mon.noname-a con 0x7f2d2809ca70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 0x7f2d28130c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): picked mon.noname-b con 0x7f2d2809fd80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d28133ce0 0x7f2d28131180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): picked mon.noname-c con 0x7f2d28133ce0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2813d200 con 0x7f2d2809ca70 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2813d080 con 0x7f2d2809fd80 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2813d380 con 0x7f2d28133ce0 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient(hunting): authenticate will time out at 1052.808823s 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 0x7f2d28130c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2ca76640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d28133ce0 msgr2=0x7f2d28131180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:54.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 0x7f2d28130c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37846/0 (socket says 172.21.15.119:37846) 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 -- 172.21.15.119:0/1208205866 learned_addr learned my addr 172.21.15.119:0/1208205866 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): get_auth_request con 0x7f2d2809fd80 auth_method 0 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2ca76640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d28133ce0 0x7f2d28131180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient(hunting): handle_auth_done global_id 14272 payload 824 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d27fff640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2809ca70 0x7f2d281cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 -- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d28133ce0 msgr2=0x7f2d28131180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:54.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d28133ce0 0x7f2d28131180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 -- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2809ca70 msgr2=0x7f2d281cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2809ca70 0x7f2d281cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 -- 172.21.15.119:0/1208205866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2813a810 con 0x7f2d2809fd80 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d27fff640 10 monclient: get_auth_request con 0x7f2d2809ca70 auth_method 0 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d27fff640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2809ca70 0x7f2d281cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d1f7fe640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 0x7f2d28130c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b550 tx=0x7f2d1000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10013020 con 0x7f2d2809fd80 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: dump: 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:54.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 5 monclient: authenticate success, global_id 14272 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _renew_subs 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d10003de0 con 0x7f2d2809fd80 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2809f2f0 con 0x7f2d2809fd80 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d100052b0 con 0x7f2d2809fd80 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 10 monclient: dump: 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.028 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _renew_subs 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d26ffd640 4 set_mon_vals no callback set 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281cfed0 con 0x7f2d2809fd80 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:54.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2d10016020 con 0x7f2d2809fd80 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d280a0f60 con 0x7f2d2809fd80 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2d0803b970 0x7f2d0803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4697+0+0 (secure 0 0 0) 0x7f2d1004f810 con 0x7f2d2809fd80 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d27fff640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2d0803b970 0x7f2d0803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d27fff640 10 monclient: get_auth_request con 0x7f2d0803b970 auth_method 0 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.009+0000 7f2d27fff640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2d0803b970 0x7f2d0803de30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b0000 tx=0x7f2d1800c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:54.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.013+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2d1001d020 con 0x7f2d2809fd80 2024-09-15T21:18:54.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.013+0000 7f2d257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:54.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.013+0000 7f2d257fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:54.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.169+0000 7f2d2e4d9640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T21:18:54.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.169+0000 7f2d2e4d9640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T21:18:54.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.169+0000 7f2d2e4d9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.169+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d28070450 con 0x7f2d2809fd80 2024-09-15T21:18:54.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.569+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' removed v29) ==== 176+0+0 (secure 0 0 0) 0x7f2d1000fe20 con 0x7f2d2809fd80 2024-09-15T21:18:54.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.569+0000 7f2d257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T21:18:54.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.569+0000 7f2d257fa640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' removed 2024-09-15T21:18:54.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.669+0000 7f2d2e4d9640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T21:18:54.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.669+0000 7f2d2e4d9640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T21:18:54.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.669+0000 7f2d2e4d9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.669+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d2806f110 con 0x7f2d2809fd80 2024-09-15T21:18:54.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d257fa640 1 -- 172.21.15.119:0/1208205866 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' does not exist v29) ==== 183+0+0 (secure 0 0 0) 0x7f2d1000d380 con 0x7f2d2809fd80 2024-09-15T21:18:54.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d257fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T21:18:54.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d257fa640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' does not exist 2024-09-15T21:18:54.679 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'testpool1234' does not exist 2024-09-15T21:18:54.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2d0803b970 msgr2=0x7f2d0803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2d0803b970 0x7f2d0803de30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b0000 tx=0x7f2d1800c040 comp rx=0 tx=0).stop 2024-09-15T21:18:54.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 10 monclient: shutdown 2024-09-15T21:18:54.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:54.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 msgr2=0x7f2d28130c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 0x7f2d28130c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b550 tx=0x7f2d1000ba10 comp rx=0 tx=0).stop 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 shutdown_connections 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2d0803b970 0x7f2d0803de30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2d28133ce0 0x7f2d28131180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2809fd80 0x7f2d28130c40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 --2- 172.21.15.119:0/1208205866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2809ca70 0x7f2d281cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 >> 172.21.15.119:0/1208205866 conn(0x7f2d280a66e0 msgr2=0x7f2d28132b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 shutdown_connections 2024-09-15T21:18:54.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.673+0000 7f2d2e4d9640 1 -- 172.21.15.119:0/1208205866 wait complete. 2024-09-15T21:18:54.697 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-09-15T21:18:54.697 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:18:54.697 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-09-15T21:18:54.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient: get_monmap_and_config 2024-09-15T21:18:54.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient: build_initial_monmap 2024-09-15T21:18:54.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient: monmap: 2024-09-15T21:18:54.819 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:54.819 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.819 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:54.818009+0000 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:54.818009+0000 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:54.820 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 Processor -- start 2024-09-15T21:18:54.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 -- start start 2024-09-15T21:18:54.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient: init 2024-09-15T21:18:54.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:54.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc73409d550 0x7fc73409d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): picked mon.noname-b con 0x7fc73409d550 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc73409fb70 0x7fc73409de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): picked mon.noname-c con 0x7fc73409fb70 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc734134820 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc73409e8f0 con 0x7fc734134820 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc7340a3ab0 con 0x7fc73409fb70 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc73409e3b0 con 0x7fc73409d550 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73b5df640 10 monclient(hunting): authenticate will time out at 1053.615428s 2024-09-15T21:18:54.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73937b640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc73409d550 msgr2=0x7fc73409d930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc739b7c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc734134820 0x7fc734134cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34208/0 (socket says 172.21.15.119:34208) 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc73409fb70 0x7fc73409de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc739b7c640 1 -- 172.21.15.119:0/4290395862 learned_addr learned my addr 172.21.15.119:0/4290395862 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc73937b640 1 --2- 172.21.15.119:0/4290395862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc73409d550 0x7fc73409d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): get_auth_request con 0x7fc73409fb70 auth_method 0 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc738b7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc72bfff640 1 -- 172.21.15.119:0/4290395862 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919381674 0 0) 0x7fc72c0014a0 con 0x7fc734134820 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc72bfff640 10 monclient(hunting): _negotiate 2024-09-15T21:18:54.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc72bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc72bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc72bfff640 10 monclient(hunting): my global_id is 14278 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.813+0000 7fc72bfff640 1 -- 172.21.15.119:0/4290395862 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc720003530 con 0x7fc734134820 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 10 monclient(hunting): handle_auth_done global_id 14275 payload 274 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 1 -- 172.21.15.119:0/4290395862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc73409d550 msgr2=0x7fc73409d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 1 --2- 172.21.15.119:0/4290395862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc73409d550 0x7fc73409d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 1 -- 172.21.15.119:0/4290395862 >> v1:172.21.15.119:6789/0 conn(0x7fc734134820 legacy=0x7fc734134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 10 monclient: _finish_hunting 0 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 1 monclient: found mon.noname-c 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 1 -- 172.21.15.119:0/4290395862 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc734069730 con 0x7fc73409fb70 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc738b7a640 1 --2- 172.21.15.119:0/4290395862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc73409fb70 0x7fc73409de70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7240097c0 tx=0x7fc72401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cceb5e77ad65a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 1 -- 172.21.15.119:0/4290395862 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240b0070 con 0x7fc73409fb70 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: dump: 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.831 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: _finish_auth 0 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 1 -- 172.21.15.119:0/4290395862 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7240ab600 con 0x7fc73409fb70 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 1 -- 172.21.15.119:0/4290395862 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240ab9d0 con 0x7fc73409fb70 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 5 monclient: authenticate success, global_id 14275 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 4 set_mon_vals no callback set 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc72bfff640 10 monclient: dump: 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 10 monclient: shutdown 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:54.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 -- 172.21.15.119:0/4290395862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc73409fb70 msgr2=0x7fc73409de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 --2- 172.21.15.119:0/4290395862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc73409fb70 0x7fc73409de70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7240097c0 tx=0x7fc72401c410 comp rx=0 tx=0).stop 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 -- 172.21.15.119:0/4290395862 shutdown_connections 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 --2- 172.21.15.119:0/4290395862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc73409fb70 0x7fc73409de70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 --2- 172.21.15.119:0/4290395862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc73409d550 0x7fc73409d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 -- 172.21.15.119:0/4290395862 >> 172.21.15.119:0/4290395862 conn(0x7fc7340a2b50 msgr2=0x7fc7340a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 -- 172.21.15.119:0/4290395862 shutdown_connections 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 1 -- 172.21.15.119:0/4290395862 wait complete. 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 10 monclient: build_initial_monmap 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.817+0000 7fc73b5df640 10 monclient: monmap: 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:54.834 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 Processor -- start 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- start start 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: init 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc73409d550 0x7fc73413ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): picked mon.noname-a con 0x7fc73409d550 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:54.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc734134820 0x7fc734133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): picked mon.noname-b con 0x7fc734134820 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc734134240 0x7fc7341cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): picked mon.noname-c con 0x7fc734134240 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc7340a27d0 con 0x7fc73409d550 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7340a2650 con 0x7fc734134820 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc73413f520 con 0x7fc734134240 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient(hunting): authenticate will time out at 1053.620625s 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc734134820 0x7fc734133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc734134820 0x7fc734133d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37856/0 (socket says 172.21.15.119:37856) 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 -- 172.21.15.119:0/2923820600 learned_addr learned my addr 172.21.15.119:0/2923820600 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): get_auth_request con 0x7fc734134820 auth_method 0 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc739b7c640 1 -- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc734134240 msgr2=0x7fc7341cfb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc739b7c640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc734134240 0x7fc7341cfb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient(hunting): handle_auth_done global_id 14281 payload 824 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73937b640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc73409d550 0x7fc73413ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 -- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc734134240 msgr2=0x7fc7341cfb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc734134240 0x7fc7341cfb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 -- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc73409d550 msgr2=0x7fc73413ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc73409d550 0x7fc73413ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient: _finish_hunting 0 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 monclient: found mon.noname-b 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 -- 172.21.15.119:0/2923820600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc734073a60 con 0x7fc734134820 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc738b7a640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc734134820 0x7fc734133d00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc72401a040 tx=0x7fc72401ac50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 1 -- 172.21.15.119:0/2923820600 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240b0050 con 0x7fc734134820 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: dump: 2024-09-15T21:18:54.838 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: _finish_auth 0 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 1 -- 172.21.15.119:0/2923820600 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7240bc070 con 0x7fc734134820 2024-09-15T21:18:54.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 5 monclient: authenticate success, global_id 14281 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _renew_subs 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- 172.21.15.119:0/2923820600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341216c0 con 0x7fc734134820 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 1 -- 172.21.15.119:0/2923820600 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240b5a30 con 0x7fc734134820 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 10 monclient: dump: 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc72b7fe640 4 set_mon_vals no callback set 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _renew_subs 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- 172.21.15.119:0/2923820600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73409d240 con 0x7fc734134820 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73b5df640 1 -- 172.21.15.119:0/2923820600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734134cb0 con 0x7fc734134820 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 1 -- 172.21.15.119:0/2923820600 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc7240b5bd0 con 0x7fc734134820 2024-09-15T21:18:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71803b970 0x7fc71803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc729ffb640 1 -- 172.21.15.119:0/2923820600 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4346+0+0 (secure 0 0 0) 0x7fc7240ea470 con 0x7fc734134820 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73937b640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71803b970 0x7fc71803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73937b640 10 monclient: get_auth_request con 0x7fc71803b970 auth_method 0 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.821+0000 7fc73937b640 1 --2- 172.21.15.119:0/2923820600 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71803b970 0x7fc71803de30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc730003940 tx=0x7fc730008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.825+0000 7fc729ffb640 1 -- 172.21.15.119:0/2923820600 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc734073a60 con 0x7fc734134820 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.825+0000 7fc729ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:54.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:54.825+0000 7fc729ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:54.991 INFO:tasks.workunit.client.0.smithi119.stderr:foo-rule not valid: foo-rule not one of 'true', 'false' 2024-09-15T21:18:54.991 INFO:tasks.workunit.client.0.smithi119.stderr:Invalid command: unused arguments: ['foo-rule'] 2024-09-15T21:18:54.991 INFO:tasks.workunit.client.0.smithi119.stderr:osd pool create [] [] [] [] [] [] [] [] [] [] [--bulk] [] [] [--yes-i-really-mean-it] : create pool 2024-09-15T21:18:54.991 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: invalid command 2024-09-15T21:18:54.994 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:18:54.994 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create foo 123 123 replicated 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient: get_monmap_and_config 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient: build_initial_monmap 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient: monmap: 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:55.114924+0000 2024-09-15T21:18:55.116 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:55.114924+0000 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:55.117 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 1 Processor -- start 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 1 -- start start 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient: init 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2c0135bf0 0x7ff2c009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient(hunting): picked mon.noname-c con 0x7ff2c0135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2c0135fd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2c0136460 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00a1870 con 0x7ff2c0136460 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00a1d80 con 0x7ff2c0135fd0 2024-09-15T21:18:55.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c00a3600 con 0x7ff2c0135bf0 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c84ba640 10 monclient(hunting): authenticate will time out at 1053.911611s 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c5a55640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff2c0135fd0 0x7ff2c00a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34210/0 (socket says 172.21.15.119:34210) 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2c0135bf0 0x7ff2c009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c5a55640 1 -- 172.21.15.119:0/1794340360 learned_addr learned my addr 172.21.15.119:0/1794340360 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): get_auth_request con 0x7ff2c0135bf0 auth_method 0 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:55.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.109+0000 7ff2c6256640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 10 monclient(hunting): handle_auth_done global_id 14284 payload 274 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 1 -- 172.21.15.119:0/1794340360 >> v1:172.21.15.119:6789/0 conn(0x7ff2c0135fd0 legacy=0x7ff2c00a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 1 -- 172.21.15.119:0/1794340360 >> v1:172.21.15.17:6789/0 conn(0x7ff2c0136460 legacy=0x7ff2c00a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 10 monclient: _finish_hunting 0 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 1 monclient: found mon.noname-c 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 1 -- 172.21.15.119:0/1794340360 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00a2060 con 0x7ff2c0135bf0 2024-09-15T21:18:55.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c6256640 1 --2- 172.21.15.119:0/1794340360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135bf0 0x7ff2c009fe70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0097f0 tx=0x7ff2bc01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef7d7ec6722a00ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:55.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 1 -- 172.21.15.119:0/1794340360 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0b0070 con 0x7ff2c0135bf0 2024-09-15T21:18:55.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: dump: 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: _finish_auth 0 2024-09-15T21:18:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: _check_auth_tickets 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 5 monclient: authenticate success, global_id 14284 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 1 -- 172.21.15.119:0/1794340360 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc004230 con 0x7ff2c0135bf0 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 4 set_mon_vals no callback set 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 1 -- 172.21.15.119:0/1794340360 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc003620 con 0x7ff2c0135bf0 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c5254640 10 monclient: dump: 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 10 monclient: shutdown 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1794340360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135bf0 msgr2=0x7ff2c009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1794340360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135bf0 0x7ff2c009fe70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0097f0 tx=0x7ff2bc01c330 comp rx=0 tx=0).stop 2024-09-15T21:18:55.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1794340360 shutdown_connections 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1794340360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135bf0 0x7ff2c009fe70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1794340360 >> 172.21.15.119:0/1794340360 conn(0x7ff2c00a66e0 msgr2=0x7ff2c00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1794340360 shutdown_connections 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1794340360 wait complete. 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 10 monclient: build_initial_monmap 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 10 monclient: monmap: 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.130 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 Processor -- start 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 1 -- start start 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.113+0000 7ff2c84ba640 10 monclient: init 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2c0135bf0 0x7ff2c01cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): picked mon.noname-c con 0x7ff2c0135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 0x7ff2c0134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): picked mon.noname-b con 0x7ff2c0135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2c0136460 0x7ff2c009d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): picked mon.noname-a con 0x7ff2c0136460 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01416d0 con 0x7ff2c0136460 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c006ecf0 con 0x7ff2c0135fd0 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c0141850 con 0x7ff2c0135bf0 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient(hunting): authenticate will time out at 1053.916467s 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 0x7ff2c0134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6256640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2c0135bf0 msgr2=0x7ff2c01cfcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 0x7ff2c0134bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37876/0 (socket says 172.21.15.119:37876) 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 -- 172.21.15.119:0/1194247138 learned_addr learned my addr 172.21.15.119:0/1194247138 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): get_auth_request con 0x7ff2c0135fd0 auth_method 0 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6256640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2c0135bf0 0x7ff2c01cfcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6a57640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2c0136460 0x7ff2c009d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient(hunting): handle_auth_done global_id 14290 payload 824 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 -- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2c0135bf0 msgr2=0x7ff2c01cfcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2c0135bf0 0x7ff2c01cfcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 -- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2c0136460 msgr2=0x7ff2c009d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2c0136460 0x7ff2c009d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient: _finish_hunting 0 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 monclient: found mon.noname-b 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 -- 172.21.15.119:0/1194247138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c01d0bc0 con 0x7ff2c0135fd0 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6a57640 10 monclient: get_auth_request con 0x7ff2c0136460 auth_method 0 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6a57640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2c0136460 0x7ff2c009d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c5a55640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 0x7ff2c0134bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000c480 tx=0x7ff2b000c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:55.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0017070 con 0x7ff2c0135fd0 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: dump: 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.135 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b0005d10 con 0x7ff2c0135fd0 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 5 monclient: authenticate success, global_id 14290 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _renew_subs 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c009d240 con 0x7ff2c0135fd0 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00072b0 con 0x7ff2c0135fd0 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:55.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 10 monclient: dump: 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _renew_subs 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c4a53640 4 set_mon_vals no callback set 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c009f040 con 0x7ff2c0135fd0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c009fe70 con 0x7ff2c0135fd0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff2b001a070 con 0x7ff2c0135fd0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2a403b970 0x7ff2a403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4346+0+0 (secure 0 0 0) 0x7ff2b004ebf0 con 0x7ff2c0135fd0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6256640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2a403b970 0x7ff2a403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6256640 10 monclient: get_auth_request con 0x7ff2a403b970 auth_method 0 2024-09-15T21:18:55.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.117+0000 7ff2c6256640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2a403b970 0x7ff2a403de30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0ad000 tx=0x7ff2bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:55.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.121+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff2c009d240 con 0x7ff2c0135fd0 2024-09-15T21:18:55.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.121+0000 7ff2b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:55.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.121+0000 7ff2b6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:55.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.277+0000 7ff2c84ba640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T21:18:55.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.281+0000 7ff2c84ba640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T21:18:55.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.281+0000 7ff2c84ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.281+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) -- 0x7ff2c013e2a0 con 0x7ff2c0135fd0 2024-09-15T21:18:55.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.601+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' created v30) ==== 154+0+0 (secure 0 0 0) 0x7ff2c01d0bc0 con 0x7ff2c0135fd0 2024-09-15T21:18:55.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.601+0000 7ff2b6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T21:18:55.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.601+0000 7ff2b6ffd640 10 monclient: _finish_command 2 = system:0 pool 'foo' created 2024-09-15T21:18:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2c84ba640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T21:18:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2c84ba640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T21:18:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2c84ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) -- 0x7ff2c009e280 con 0x7ff2c0135fd0 2024-09-15T21:18:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2b6ffd640 1 -- 172.21.15.119:0/1194247138 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' already exists v30) ==== 161+0+0 (secure 0 0 0) 0x7ff2b0010300 con 0x7ff2c0135fd0 2024-09-15T21:18:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2b6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T21:18:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.713+0000 7ff2b6ffd640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-09-15T21:18:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'foo' already exists 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2a403b970 msgr2=0x7ff2a403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2a403b970 0x7ff2a403de30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0ad000 tx=0x7ff2bc008040 comp rx=0 tx=0).stop 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 10 monclient: shutdown 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 msgr2=0x7ff2c0134bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 0x7ff2c0134bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000c480 tx=0x7ff2b000c940 comp rx=0 tx=0).stop 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 shutdown_connections 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2a403b970 0x7ff2a403de30 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0ad000 tx=0x7ff2bc008040 comp rx=0 tx=0).stop 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2c0136460 0x7ff2c009d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2c0135fd0 0x7ff2c0134bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 --2- 172.21.15.119:0/1194247138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2c0135bf0 0x7ff2c01cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 >> 172.21.15.119:0/1194247138 conn(0x7ff2c00a66e0 msgr2=0x7ff2c00a4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:55.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 shutdown_connections 2024-09-15T21:18:55.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.717+0000 7ff2c84ba640 1 -- 172.21.15.119:0/1194247138 wait complete. 2024-09-15T21:18:55.738 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create fooo 123 123 erasure default 2024-09-15T21:18:55.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.853+0000 7f5964d6e640 10 monclient: get_monmap_and_config 2024-09-15T21:18:55.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.853+0000 7f5964d6e640 10 monclient: build_initial_monmap 2024-09-15T21:18:55.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.853+0000 7f5964d6e640 10 monclient: monmap: 2024-09-15T21:18:55.861 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:55.859852+0000 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:55.859852+0000 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:55.862 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 Processor -- start 2024-09-15T21:18:55.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 -- start start 2024-09-15T21:18:55.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient: init 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 0x7f5960137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): picked mon.noname-b con 0x7f59600a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f59600a41b0 0x7f59600a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): picked mon.noname-a con 0x7f59600a41b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5960134300 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59600a19d0 con 0x7f5960134300 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f59600a7ea0 con 0x7f59600a41b0 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f59600a6960 con 0x7f59600a7ac0 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 10 monclient(hunting): authenticate will time out at 1054.657203s 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 msgr2=0x7f5960137cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 0x7f5960137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5960134300 0x7f59601347b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34214/0 (socket says 172.21.15.119:34214) 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 -- 172.21.15.119:0/3557574827 learned_addr learned my addr 172.21.15.119:0/3557574827 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 1 --2- 172.21.15.119:0/3557574827 >> v2:172.21.15.17:3300/0 conn(0x7f59600a41b0 0x7f59600a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4108646841 0 0) 0x7f59600a19d0 con 0x7f5960134300 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): _negotiate 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): my global_id is 14293 2024-09-15T21:18:55.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5948003530 con 0x7f5960134300 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 10 monclient(hunting): get_auth_request con 0x7f59600a41b0 auth_method 0 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2746273355 0 0) 0x7f5948003530 con 0x7f5960134300 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 msgr2=0x7f5960137cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 0x7f5960137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 >> v2:172.21.15.17:3300/0 conn(0x7f59600a41b0 msgr2=0x7f59600a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 --2- 172.21.15.119:0/3557574827 >> v2:172.21.15.17:3300/0 conn(0x7f59600a41b0 0x7f59600a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: _finish_hunting 0 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:55.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59600a6c40 con 0x7f5960134300 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: _finish_auth 0 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5958001a80 con 0x7f5960134300 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: dump: 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.876 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 >> v1:172.21.15.119:6789/0 conn(0x7f5960134300 legacy=0x7f59601347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59480039c0 0x7f5948003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): picked mon.a con 0x7f59480039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 0x7f5948004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): picked mon.c con 0x7f59600a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5948004e70 0x7f594800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): picked mon.b con 0x7f5948004e70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f59600a7ea0 con 0x7f59480039c0 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59600a6960 con 0x7f5948004e70 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5948005610 con 0x7f59600a7ac0 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595d59a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 1 -- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 msgr2=0x7f5948004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:55.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595dd9b640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 0x7f5948004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5948004e70 0x7f594800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): get_auth_request con 0x7f5948004e70 auth_method 0 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59480039c0 0x7f5948003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 10 monclient(hunting): get_auth_request con 0x7f59480039c0 auth_method 0 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595e59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:18:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient(hunting): handle_auth_done global_id 14293 payload 293 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 -- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 msgr2=0x7f5948004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59600a7ac0 0x7f5948004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 -- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59480039c0 msgr2=0x7f5948003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59480039c0 0x7f5948003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 monclient: found mon.b 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 1 -- 172.21.15.119:0/3557574827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5948003530 con 0x7f5948004e70 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient: _finish_auth 0 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 10 monclient: _check_auth_tickets 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f595ed9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:55.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 5 monclient: authenticate success, global_id 14293 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.857+0000 7f5964d6e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595ed9d640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5948004e70 0x7f594800b9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f595800a950 tx=0x7f595801d320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1933d8a4d10bef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595800a4c0 con 0x7f5948004e70 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: dump: 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.881 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59580a5e50 con 0x7f5948004e70 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 1 -- 172.21.15.119:0/3557574827 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59580ad890 con 0x7f5948004e70 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 4 set_mon_vals no callback set 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595d59a640 10 monclient: dump: 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.882 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: shutdown 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/3557574827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5948004e70 msgr2=0x7f594800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 --2- 172.21.15.119:0/3557574827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5948004e70 0x7f594800b9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f595800a950 tx=0x7f595801d320 comp rx=0 tx=0).stop 2024-09-15T21:18:55.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595ed9d640 1 -- 172.21.15.119:0/3557574827 reap_dead start 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/3557574827 shutdown_connections 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/3557574827 >> 172.21.15.119:0/3557574827 conn(0x7f59600a6fd0 msgr2=0x7f59600a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/3557574827 shutdown_connections 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/3557574827 wait complete. 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: build_initial_monmap 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: monmap: 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.884 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 Processor -- start 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- start start 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: init 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5948004e70 0x7f59601cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): picked mon.noname-c con 0x7f5948004e70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 0x7f59601cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): picked mon.noname-b con 0x7f5960073b40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:55.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5960073f80 0x7f59601cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): picked mon.noname-a con 0x7f5960073f80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f596013fd20 con 0x7f5960073f80 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f596013fba0 con 0x7f5960073b40 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f596013fea0 con 0x7f5948004e70 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient(hunting): authenticate will time out at 1054.663076s 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595e59c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5948004e70 msgr2=0x7f59601cf600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5948004e70 0x7f59601cf600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 0x7f59601cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 0x7f59601cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37884/0 (socket says 172.21.15.119:37884) 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 -- 172.21.15.119:0/2933147644 learned_addr learned my addr 172.21.15.119:0/2933147644 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): get_auth_request con 0x7f5960073b40 auth_method 0 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595ed9d640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5960073f80 0x7f59601cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient(hunting): handle_auth_done global_id 14296 payload 824 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 -- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5948004e70 msgr2=0x7f59601cf600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5948004e70 0x7f59601cf600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 -- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5960073f80 msgr2=0x7f59601cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:55.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5960073f80 0x7f59601cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 monclient: found mon.noname-b 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 -- 172.21.15.119:0/2933147644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59600a59d0 con 0x7f5960073b40 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595ed9d640 10 monclient: get_auth_request con 0x7f5960073f80 auth_method 0 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595ed9d640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5960073f80 0x7f59601cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595dd9b640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 0x7f59601cc0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f595400e840 tx=0x7f595400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954017020 con 0x7f5960073b40 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: dump: 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: _finish_auth 0 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:55.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 5 monclient: authenticate success, global_id 14296 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595400c8d0 con 0x7f5960073b40 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: _renew_subs 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59600a33c0 con 0x7f5960073b40 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595400f040 con 0x7f5960073b40 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f594effd640 10 monclient: dump: 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:55.890 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f595cd99640 4 set_mon_vals no callback set 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: _renew_subs 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.861+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5960071620 con 0x7f5960073b40 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f5964d6e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:55.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f5964d6e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f5964d6e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59600a4590 con 0x7f5960073b40 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f595401a020 con 0x7f5960073b40 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f594effd640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f594003b9a0 0x7f594003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4688+0+0 (secure 0 0 0) 0x7f595404f8c0 con 0x7f5960073b40 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f595e59c640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f594003b9a0 0x7f594003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f595e59c640 10 monclient: get_auth_request con 0x7f594003b9a0 auth_method 0 2024-09-15T21:18:55.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.865+0000 7f595e59c640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f594003b9a0 0x7f594003de60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5950003910 tx=0x7f5950008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:55.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.869+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5954021020 con 0x7f5960073b40 2024-09-15T21:18:55.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.869+0000 7f594effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:55.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:55.869+0000 7f594effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:56.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:56.041+0000 7f5964d6e640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T21:18:56.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:56.041+0000 7f5964d6e640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T21:18:56.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:56.041+0000 7f5964d6e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:56.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:56.041+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) -- 0x7f596006a760 con 0x7f5960073b40 2024-09-15T21:18:57.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.617+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' created v32) ==== 188+0+0 (secure 0 0 0) 0x7f5954009e60 con 0x7f5960073b40 2024-09-15T21:18:57.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.617+0000 7f594effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T21:18:57.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.617+0000 7f594effd640 10 monclient: _finish_command 2 = system:0 pool 'fooo' created 2024-09-15T21:18:57.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f5964d6e640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T21:18:57.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f5964d6e640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T21:18:57.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f5964d6e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:57.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) -- 0x7f5960070d80 con 0x7f5960073b40 2024-09-15T21:18:57.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f594effd640 1 -- 172.21.15.119:0/2933147644 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' already exists v32) ==== 195+0+0 (secure 0 0 0) 0x7f5954012390 con 0x7f5960073b40 2024-09-15T21:18:57.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f594effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T21:18:57.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.741+0000 7f594effd640 10 monclient: _finish_command 3 = system:0 pool 'fooo' already exists 2024-09-15T21:18:57.749 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'fooo' already exists 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f594003b9a0 msgr2=0x7f594003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f594003b9a0 0x7f594003de60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5950003910 tx=0x7f5950008040 comp rx=0 tx=0).stop 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 10 monclient: shutdown 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 msgr2=0x7f59601cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 0x7f59601cc0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f595400e840 tx=0x7f595400ed00 comp rx=0 tx=0).stop 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 shutdown_connections 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f594003b9a0 0x7f594003de60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5960073f80 0x7f59601cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5960073b40 0x7f59601cc0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 --2- 172.21.15.119:0/2933147644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5948004e70 0x7f59601cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 >> 172.21.15.119:0/2933147644 conn(0x7f59600a6fd0 msgr2=0x7f596013f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:57.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 shutdown_connections 2024-09-15T21:18:57.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.745+0000 7f5964d6e640 1 -- 172.21.15.119:0/2933147644 wait complete. 2024-09-15T21:18:57.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create foooo 123 2024-09-15T21:18:57.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient: get_monmap_and_config 2024-09-15T21:18:57.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient: build_initial_monmap 2024-09-15T21:18:57.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient: monmap: 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:57.885047+0000 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:57.885047+0000 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:57.887 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:57.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 Processor -- start 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 -- start start 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient: init 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1540a79e0 0x7fa1540a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): picked mon.noname-c con 0x7fa1540a79e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1540a46b0 0x7fa1541377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): picked mon.noname-a con 0x7fa1540a46b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa1540a1100 addr v1:172.21.15.119:6789/0 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa154135180 con 0x7fa1540a1100 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa1540a7dc0 con 0x7fa1540a46b0 2024-09-15T21:18:57.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa1540a6920 con 0x7fa1540a79e0 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa15b050640 10 monclient(hunting): authenticate will time out at 1056.681743s 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1540a79e0 0x7fa1540a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa1595ed640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1540a1100 0x7fa154137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34224/0 (socket says 172.21.15.119:34224) 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa1595ed640 1 -- 172.21.15.119:0/2007575021 learned_addr learned my addr 172.21.15.119:0/2007575021 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): get_auth_request con 0x7fa1540a79e0 auth_method 0 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:57.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa153fff640 1 --2- 172.21.15.119:0/2007575021 >> v2:172.21.15.17:3300/0 conn(0x7fa1540a46b0 0x7fa1541377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient(hunting): handle_auth_done global_id 14299 payload 274 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 1 -- 172.21.15.119:0/2007575021 >> v2:172.21.15.17:3300/0 conn(0x7fa1540a46b0 msgr2=0x7fa1541377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 1 --2- 172.21.15.119:0/2007575021 >> v2:172.21.15.17:3300/0 conn(0x7fa1540a46b0 0x7fa1541377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 1 -- 172.21.15.119:0/2007575021 >> v1:172.21.15.119:6789/0 conn(0x7fa1540a1100 legacy=0x7fa154137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient: _finish_hunting 0 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 1 monclient: found mon.noname-c 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa1537fe640 1 -- 172.21.15.119:0/2007575021 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179509369 0 0) 0x7fa154135180 con 0x7fa1540a1100 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa158dec640 1 -- 172.21.15.119:0/2007575021 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1540a1c30 con 0x7fa1540a79e0 2024-09-15T21:18:57.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.881+0000 7fa1537fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa158dec640 1 --2- 172.21.15.119:0/2007575021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a79e0 0x7fa1540a4170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa15406f060 tx=0x7fa14401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2da2e79d90a05b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 1 -- 172.21.15.119:0/2007575021 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154134930 con 0x7fa1540a79e0 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: dump: 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.898 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 1 -- 172.21.15.119:0/2007575021 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1440b0070 con 0x7fa1540a79e0 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 5 monclient: authenticate success, global_id 14299 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:57.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 1 -- 172.21.15.119:0/2007575021 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1440ab8a0 con 0x7fa1540a79e0 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 4 set_mon_vals no callback set 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1537fe640 10 monclient: dump: 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:57.900 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: shutdown 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/2007575021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a79e0 msgr2=0x7fa1540a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 --2- 172.21.15.119:0/2007575021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a79e0 0x7fa1540a4170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa15406f060 tx=0x7fa14401c460 comp rx=0 tx=0).stop 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/2007575021 shutdown_connections 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 --2- 172.21.15.119:0/2007575021 >> v2:172.21.15.17:3300/0 conn(0x7fa1540a46b0 0x7fa1541377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 --2- 172.21.15.119:0/2007575021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a79e0 0x7fa1540a4170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/2007575021 >> 172.21.15.119:0/2007575021 conn(0x7fa1540a6ef0 msgr2=0x7fa1540a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/2007575021 shutdown_connections 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/2007575021 wait complete. 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: build_initial_monmap 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: monmap: 2024-09-15T21:18:57.901 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 Processor -- start 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- start start 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: init 2024-09-15T21:18:57.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1540a1100 0x7fa154130da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): picked mon.noname-c con 0x7fa1540a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 0x7fa1541312e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): picked mon.noname-b con 0x7fa1540a46b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1540a79e0 0x7fa154071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): picked mon.noname-a con 0x7fa1540a79e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541385b0 con 0x7fa1540a79e0 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa154138430 con 0x7fa1540a46b0 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa1541423a0 con 0x7fa1540a1100 2024-09-15T21:18:57.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient(hunting): authenticate will time out at 1056.686977s 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa158dec640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1540a1100 msgr2=0x7fa154130da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 0x7fa1541312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa158dec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1540a1100 0x7fa154130da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 0x7fa1541312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37904/0 (socket says 172.21.15.119:37904) 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 -- 172.21.15.119:0/119017014 learned_addr learned my addr 172.21.15.119:0/119017014 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): get_auth_request con 0x7fa1540a46b0 auth_method 0 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:57.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1595ed640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1540a79e0 0x7fa154071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient(hunting): handle_auth_done global_id 14305 payload 824 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 -- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1540a1100 msgr2=0x7fa154130da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1540a1100 0x7fa154130da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 -- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1540a79e0 msgr2=0x7fa154071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1540a79e0 0x7fa154071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient: _finish_hunting 0 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 monclient: found mon.noname-b 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 -- 172.21.15.119:0/119017014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1540a32d0 con 0x7fa1540a46b0 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1595ed640 10 monclient: get_auth_request con 0x7fa1540a79e0 auth_method 0 2024-09-15T21:18:57.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1595ed640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1540a79e0 0x7fa154071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa153fff640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 0x7fa1541312e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c7f0 tx=0x7fa14800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148004230 con 0x7fa1540a46b0 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: dump: 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:57.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1480043d0 con 0x7fa1540a46b0 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 5 monclient: authenticate success, global_id 14305 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: _renew_subs 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa154139e70 con 0x7fa1540a46b0 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148003860 con 0x7fa1540a46b0 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa1517fa640 10 monclient: dump: 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:57.907 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: _renew_subs 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1540a15e0 con 0x7fa1540a46b0 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.885+0000 7fa152ffd640 4 set_mon_vals no callback set 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa15b050640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:57.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa15b050640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa15b050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540a4170 con 0x7fa1540a46b0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa148003c10 con 0x7fa1540a46b0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa1517fa640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa14003b970 0x7fa14003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5130+0+0 (secure 0 0 0) 0x7fa14804fef0 con 0x7fa1540a46b0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa158dec640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa14003b970 0x7fa14003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa158dec640 10 monclient: get_auth_request con 0x7fa14003b970 auth_method 0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.889+0000 7fa158dec640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa14003b970 0x7fa14003de30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa1440af000 tx=0x7fa144008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.893+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa1540a32d0 con 0x7fa1540a46b0 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.893+0000 7fa1517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:57.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:57.893+0000 7fa1517fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:58.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.061+0000 7fa15b050640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T21:18:58.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.061+0000 7fa15b050640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T21:18:58.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.061+0000 7fa15b050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.061+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) -- 0x7fa15413efb0 con 0x7fa1540a46b0 2024-09-15T21:18:58.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.625+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' created v33) ==== 115+0+0 (secure 0 0 0) 0x7fa148015020 con 0x7fa1540a46b0 2024-09-15T21:18:58.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.625+0000 7fa1517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T21:18:58.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.625+0000 7fa1517fa640 10 monclient: _finish_command 2 = system:0 pool 'foooo' created 2024-09-15T21:18:58.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa15b050640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T21:18:58.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa15b050640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T21:18:58.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa15b050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) -- 0x7fa154131820 con 0x7fa1540a46b0 2024-09-15T21:18:58.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa1517fa640 1 -- 172.21.15.119:0/119017014 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' already exists v33) ==== 122+0+0 (secure 0 0 0) 0x7fa148007e20 con 0x7fa1540a46b0 2024-09-15T21:18:58.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa1517fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T21:18:58.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.785+0000 7fa1517fa640 10 monclient: _finish_command 3 = system:0 pool 'foooo' already exists 2024-09-15T21:18:58.792 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'foooo' already exists 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa14003b970 msgr2=0x7fa14003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa14003b970 0x7fa14003de30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa1440af000 tx=0x7fa144008040 comp rx=0 tx=0).stop 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 10 monclient: shutdown 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 msgr2=0x7fa1541312e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 0x7fa1541312e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c7f0 tx=0x7fa14800ccb0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 shutdown_connections 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa14003b970 0x7fa14003de30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1540a79e0 0x7fa154071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540a46b0 0x7fa1541312e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 --2- 172.21.15.119:0/119017014 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1540a1100 0x7fa154130da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 >> 172.21.15.119:0/119017014 conn(0x7fa1540a6ef0 msgr2=0x7fa1541373c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 shutdown_connections 2024-09-15T21:18:58.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.789+0000 7fa15b050640 1 -- 172.21.15.119:0/119017014 wait complete. 2024-09-15T21:18:58.810 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create foo 123 2024-09-15T21:18:58.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 10 monclient: get_monmap_and_config 2024-09-15T21:18:58.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 10 monclient: build_initial_monmap 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 10 monclient: monmap: 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:58.927420+0000 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:58.927420+0000 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:58.929 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:58.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 1 Processor -- start 2024-09-15T21:18:58.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 1 -- start start 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 10 monclient: init 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.921+0000 7fc44c60a640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc444137870 0x7fc4440a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): picked mon.noname-b con 0x7fc444137870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc444137c50 0x7fc444138030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): picked mon.noname-c con 0x7fc444137c50 addr v2:172.21.15.119:3300/0 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4441346d0 0x7fc444134b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): picked mon.noname-a con 0x7fc4441346d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440a1b60 con 0x7fc4441346d0 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:58.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440a27f0 con 0x7fc444137c50 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440a1e40 con 0x7fc444137870 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient(hunting): authenticate will time out at 1057.724109s 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc444137c50 0x7fc444138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44a3a6640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc444137870 msgr2=0x7fc4440a0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc444137c50 0x7fc444138030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37914/0 (socket says 172.21.15.119:37914) 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44a3a6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc444137870 0x7fc4440a0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 -- 172.21.15.119:0/779814500 learned_addr learned my addr 172.21.15.119:0/779814500 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): get_auth_request con 0x7fc444137c50 auth_method 0 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:58.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44aba7640 1 --2- 172.21.15.119:0/779814500 >> v2:172.21.15.17:3300/0 conn(0x7fc4441346d0 0x7fc444134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient(hunting): handle_auth_done global_id 14308 payload 274 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 -- 172.21.15.119:0/779814500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc444137870 msgr2=0x7fc4440a0e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 --2- 172.21.15.119:0/779814500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc444137870 0x7fc4440a0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 -- 172.21.15.119:0/779814500 >> v2:172.21.15.17:3300/0 conn(0x7fc4441346d0 msgr2=0x7fc444134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 --2- 172.21.15.119:0/779814500 >> v2:172.21.15.17:3300/0 conn(0x7fc4441346d0 0x7fc444134b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient: _finish_hunting 0 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 monclient: found mon.noname-c 2024-09-15T21:18:58.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:58.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 -- 172.21.15.119:0/779814500 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440a2120 con 0x7fc444137c50 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc449ba5640 1 --2- 172.21.15.119:0/779814500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc444138030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc44406a0c0 tx=0x7fc43401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8b701fbab24c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 1 -- 172.21.15.119:0/779814500 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340b1070 con 0x7fc444137c50 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: dump: 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: _finish_auth 0 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: _check_auth_tickets 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 5 monclient: authenticate success, global_id 14308 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 1 -- 172.21.15.119:0/779814500 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4340ac640 con 0x7fc444137c50 2024-09-15T21:18:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 4 set_mon_vals no callback set 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 1 -- 172.21.15.119:0/779814500 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340aca10 con 0x7fc444137c50 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc4493a4640 10 monclient: dump: 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:58.942 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 10 monclient: shutdown 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- 172.21.15.119:0/779814500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 msgr2=0x7fc444138030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- 172.21.15.119:0/779814500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc444138030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc44406a0c0 tx=0x7fc43401c460 comp rx=0 tx=0).stop 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- 172.21.15.119:0/779814500 shutdown_connections 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- 172.21.15.119:0/779814500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc444138030 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- 172.21.15.119:0/779814500 >> v2:172.21.15.17:3300/0 conn(0x7fc4441346d0 0x7fc444134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 --2- 172.21.15.119:0/779814500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc444137870 0x7fc4440a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- 172.21.15.119:0/779814500 >> 172.21.15.119:0/779814500 conn(0x7fc44409be60 msgr2=0x7fc44409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- 172.21.15.119:0/779814500 shutdown_connections 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.925+0000 7fc44c60a640 1 -- 172.21.15.119:0/779814500 wait complete. 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: build_initial_monmap 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: monmap: 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:58.944 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 Processor -- start 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- start start 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: init 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4441346d0 0x7fc4441cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): picked mon.noname-c con 0x7fc4441346d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc444137870 0x7fc4441cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): picked mon.noname-a con 0x7fc444137870 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc4441d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): picked mon.noname-b con 0x7fc444137c50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:58.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc44406de60 con 0x7fc444137870 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc444138680 con 0x7fc444137c50 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc44406dfe0 con 0x7fc4441346d0 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient(hunting): authenticate will time out at 1057.729193s 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44a3a6640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4441346d0 msgr2=0x7fc4441cf7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc4441d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44a3a6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4441346d0 0x7fc4441cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc4441d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37924/0 (socket says 172.21.15.119:37924) 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 -- 172.21.15.119:0/4179828311 learned_addr learned my addr 172.21.15.119:0/4179828311 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:58.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): get_auth_request con 0x7fc444137c50 auth_method 0 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc449ba5640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc444137870 0x7fc4441cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc449ba5640 10 monclient(hunting): get_auth_request con 0x7fc444137870 auth_method 0 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc449ba5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc449ba5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc449ba5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient(hunting): handle_auth_done global_id 14311 payload 824 2024-09-15T21:18:58.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 -- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4441346d0 msgr2=0x7fc4441cf7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4441346d0 0x7fc4441cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 -- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc444137870 msgr2=0x7fc4441cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc444137870 0x7fc4441cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient: _finish_hunting 0 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 monclient: found mon.noname-b 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 -- 172.21.15.119:0/4179828311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44411f1a0 con 0x7fc444137c50 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44aba7640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc4441d0230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00cd70 tx=0x7fc43c00a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c003ba0 con 0x7fc444137c50 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:58.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: dump: 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: _finish_auth 0 2024-09-15T21:18:58.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: _check_auth_tickets 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c003d40 con 0x7fc444137c50 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 5 monclient: authenticate success, global_id 14311 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _renew_subs 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44409eac0 con 0x7fc444137c50 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c004240 con 0x7fc444137c50 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 10 monclient: dump: 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:58.950 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc448ba3640 4 set_mon_vals no callback set 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _renew_subs 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44409d0f0 con 0x7fc444137c50 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:58.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4440a0e10 con 0x7fc444137c50 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc43c0043e0 con 0x7fc444137c50 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc42803b970 0x7fc42803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5474+0+0 (secure 0 0 0) 0x7fc43c050a90 con 0x7fc444137c50 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44a3a6640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc42803b970 0x7fc42803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44a3a6640 10 monclient: get_auth_request con 0x7fc42803b970 auth_method 0 2024-09-15T21:18:58.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.929+0000 7fc44a3a6640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc42803b970 0x7fc42803de30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc440003940 tx=0x7fc440006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:58.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.933+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc43c01fc40 con 0x7fc444137c50 2024-09-15T21:18:58.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.933+0000 7fc43affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:58.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:58.933+0000 7fc43affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:59.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc44c60a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T21:18:59.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc44c60a640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T21:18:59.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc44c60a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) -- 0x7fc44411ef80 con 0x7fc444137c50 2024-09-15T21:18:59.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v33) ==== 118+0+0 (secure 0 0 0) 0x7fc43c00d380 con 0x7fc444137c50 2024-09-15T21:18:59.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc43affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T21:18:59.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.105+0000 7fc43affd640 10 monclient: _finish_command 2 = system:0 pool 'foo' already exists 2024-09-15T21:18:59.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.205+0000 7fc44c60a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T21:18:59.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.205+0000 7fc44c60a640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T21:18:59.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.205+0000 7fc44c60a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.205+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) -- 0x7fc44411d9a0 con 0x7fc444137c50 2024-09-15T21:18:59.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc43affd640 1 -- 172.21.15.119:0/4179828311 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v33) ==== 118+0+0 (secure 0 0 0) 0x7fc43c00d560 con 0x7fc444137c50 2024-09-15T21:18:59.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc43affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T21:18:59.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc43affd640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-09-15T21:18:59.215 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'foo' already exists 2024-09-15T21:18:59.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc42803b970 msgr2=0x7fc42803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc42803b970 0x7fc42803de30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc440003940 tx=0x7fc440006d90 comp rx=0 tx=0).stop 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 10 monclient: shutdown 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 msgr2=0x7fc4441d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc4441d0230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00cd70 tx=0x7fc43c00a660 comp rx=0 tx=0).stop 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 shutdown_connections 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc42803b970 0x7fc42803de30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444137c50 0x7fc4441d0230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc444137870 0x7fc4441cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 --2- 172.21.15.119:0/4179828311 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4441346d0 0x7fc4441cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 >> 172.21.15.119:0/4179828311 conn(0x7fc44409be60 msgr2=0x7fc44409db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:59.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 shutdown_connections 2024-09-15T21:18:59.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.209+0000 7fc44c60a640 1 -- 172.21.15.119:0/4179828311 wait complete. 2024-09-15T21:18:59.233 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set foo size 1 --yes-i-really-mean-it 2024-09-15T21:18:59.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient: get_monmap_and_config 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient: build_initial_monmap 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient: monmap: 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:18:59.354448+0000 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:18:59.354448+0000 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:18:59.356 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:18:59.357 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:18:59.357 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:59.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 Processor -- start 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 -- start start 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient: init 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7facfc136b90 addr v1:172.21.15.17:6789/0 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a10d0 0x7facfc13cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): picked mon.noname-b con 0x7facfc0a10d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facfc0a25b0 0x7facfc13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): picked mon.noname-a con 0x7facfc0a25b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facfc0a18f0 con 0x7facfc136b90 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:59.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7facfc09fc40 con 0x7facfc0a25b0 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7facfc0a1bd0 con 0x7facfc0a10d0 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad030ff640 10 monclient(hunting): authenticate will time out at 1058.151030s 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7facfbfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a10d0 msgr2=0x7facfc13cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7facfbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a10d0 0x7facfc13cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facfc0a25b0 0x7facfc13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facfc0a25b0 0x7facfc13f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54872/0 (socket says 172.21.15.119:54872) 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 -- 172.21.15.119:0/1520460796 learned_addr learned my addr 172.21.15.119:0/1520460796 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): get_auth_request con 0x7facfc0a25b0 auth_method 0 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:59.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient(hunting): handle_auth_done global_id 4590 payload 274 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 -- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a10d0 msgr2=0x7facfc13cc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 --2- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a10d0 0x7facfc13cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 -- 172.21.15.119:0/1520460796 >> v1:172.21.15.17:6789/0 conn(0x7facfc136b90 legacy=0x7facfc136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient: _finish_hunting 0 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 monclient: found mon.noname-a 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:18:59.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.349+0000 7fad0169c640 1 -- 172.21.15.119:0/1520460796 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc0a3660 con 0x7facfc0a25b0 2024-09-15T21:18:59.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad0169c640 1 --2- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a25b0 0x7facfc13f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7facf4009e50 tx=0x7facf401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a3add091455645 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:59.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 1 -- 172.21.15.119:0/1520460796 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40a57c0 con 0x7facfc0a25b0 2024-09-15T21:18:59.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:59.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:18:59.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: dump: 2024-09-15T21:18:59.366 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:18:59.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 1 -- 172.21.15.119:0/1520460796 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf40a5960 con 0x7facfc0a25b0 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 5 monclient: authenticate success, global_id 4590 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 10 monclient: get_monmap_and_config success 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 1 -- 172.21.15.119:0/1520460796 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40a5d30 con 0x7facfc0a25b0 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 4 set_mon_vals no callback set 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7facfb7fe640 10 monclient: dump: 2024-09-15T21:18:59.368 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:59.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 10 monclient: shutdown 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 -- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a25b0 msgr2=0x7facfc13f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 --2- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a25b0 0x7facfc13f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7facf4009e50 tx=0x7facf401c8e0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 -- 172.21.15.119:0/1520460796 shutdown_connections 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 --2- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a25b0 0x7facfc13f1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 --2- 172.21.15.119:0/1520460796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a10d0 0x7facfc13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 -- 172.21.15.119:0/1520460796 >> 172.21.15.119:0/1520460796 conn(0x7facfc0a66e0 msgr2=0x7facfc0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 -- 172.21.15.119:0/1520460796 shutdown_connections 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 1 -- 172.21.15.119:0/1520460796 wait complete. 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 10 monclient: build_initial_monmap 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.353+0000 7fad030ff640 10 monclient: monmap: 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:18:59.370 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 Processor -- start 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- start start 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: init 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _reopen_session rank -1 2024-09-15T21:18:59.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a10d0 0x7facfc11cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): picked mon.noname-a con 0x7facfc0a10d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a25b0 0x7facfc11eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): picked mon.noname-c con 0x7facfc0a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 0x7facfc11d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): picked mon.noname-b con 0x7facfc136b90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7facfc1421c0 con 0x7facfc0a10d0 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7facfc142040 con 0x7facfc136b90 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:18:59.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7facfc142340 con 0x7facfc0a25b0 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): _renew_subs 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient(hunting): authenticate will time out at 1058.156806s 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facfbfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a25b0 msgr2=0x7facfc11eb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facfbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a25b0 0x7facfc11eb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 0x7facfc11d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 0x7facfc11d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37938/0 (socket says 172.21.15.119:37938) 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 -- 172.21.15.119:0/670910088 learned_addr learned my addr 172.21.15.119:0/670910088 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): get_auth_request con 0x7facfc136b90 auth_method 0 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:59.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a10d0 0x7facfc11cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): get_auth_request con 0x7facfc0a10d0 auth_method 0 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient(hunting): handle_auth_done global_id 14314 payload 824 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 -- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a25b0 msgr2=0x7facfc11eb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a25b0 0x7facfc11eb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 -- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a10d0 msgr2=0x7facfc11cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a10d0 0x7facfc11cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient: _finish_hunting 0 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 monclient: found mon.noname-b 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 -- 172.21.15.119:0/670910088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc11dff0 con 0x7facfc136b90 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad0169c640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 0x7facfc11d2d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facf40af040 tx=0x7facf4019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40bc020 con 0x7facfc136b90 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: dump: 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:59.375 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:59.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: _finish_auth 0 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf40b8670 con 0x7facfc136b90 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40b8a60 con 0x7facfc136b90 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 10 monclient: dump: 2024-09-15T21:18:59.377 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 5 monclient: authenticate success, global_id 14314 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _renew_subs 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facfaffd640 4 set_mon_vals no callback set 2024-09-15T21:18:59.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facfc11fab0 con 0x7facfc136b90 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _renew_subs 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facfc135fe0 con 0x7facfc136b90 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facfc136f70 con 0x7facfc136b90 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7facf40bf020 con 0x7facfc136b90 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b970 0x7face803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5474+0+0 (secure 0 0 0) 0x7facf40eaeb0 con 0x7facfc136b90 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b970 0x7face803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:18:59.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 10 monclient: get_auth_request con 0x7face803b970 auth_method 0 2024-09-15T21:18:59.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.357+0000 7fad00e9b640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b970 0x7face803de30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facf0007fd0 tx=0x7facf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:18:59.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.361+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7facf40c6030 con 0x7facfc136b90 2024-09-15T21:18:59.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.361+0000 7facf97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:18:59.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.361+0000 7facf97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:18:59.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.537+0000 7fad030ff640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T21:18:59.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.537+0000 7fad030ff640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T21:18:59.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.537+0000 7fad030ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.537+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) -- 0x7facfc06a830 con 0x7facfc136b90 2024-09-15T21:18:59.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.641+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v34) ==== 152+0+0 (secure 0 0 0) 0x7facf4016620 con 0x7facfc136b90 2024-09-15T21:18:59.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.641+0000 7facf97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T21:18:59.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.641+0000 7facf97fa640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 1 2024-09-15T21:18:59.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.789+0000 7fad030ff640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T21:18:59.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.789+0000 7fad030ff640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T21:18:59.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.789+0000 7fad030ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:18:59.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:18:59.789+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) -- 0x7facfc11d810 con 0x7facfc136b90 2024-09-15T21:19:00.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.629+0000 7facf97fa640 1 -- 172.21.15.119:0/670910088 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v35) ==== 152+0+0 (secure 0 0 0) 0x7facf40addb0 con 0x7facfc136b90 2024-09-15T21:19:00.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.629+0000 7facf97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T21:19:00.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.629+0000 7facf97fa640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 1 2024-09-15T21:19:00.637 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 4 size to 1 2024-09-15T21:19:00.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.637+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b970 msgr2=0x7face803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.637+0000 7fad030ff640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b970 0x7face803de30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facf0007fd0 tx=0x7facf0009040 comp rx=0 tx=0).stop 2024-09-15T21:19:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.637+0000 7fad030ff640 10 monclient: shutdown 2024-09-15T21:19:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.637+0000 7fad030ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.637+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 msgr2=0x7facfc11d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.637+0000 7fad030ff640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 0x7facfc11d2d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facf40af040 tx=0x7facf4019680 comp rx=0 tx=0).stop 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 shutdown_connections 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b970 0x7face803de30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facfc136b90 0x7facfc11d2d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facfc0a25b0 0x7facfc11eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 --2- 172.21.15.119:0/670910088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facfc0a10d0 0x7facfc11cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 >> 172.21.15.119:0/670910088 conn(0x7facfc0a66e0 msgr2=0x7facfc0a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:00.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.641+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 shutdown_connections 2024-09-15T21:19:00.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:00.645+0000 7fad030ff640 1 -- 172.21.15.119:0/670910088 wait complete. 2024-09-15T21:19:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_config_value foo min_size 1 2024-09-15T21:19:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:+ local pool_name config_opt expected_val val 2024-09-15T21:19:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:+ pool_name=foo 2024-09-15T21:19:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:+ config_opt=min_size 2024-09-15T21:19:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_val=1 2024-09-15T21:19:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:++ get_config_value_or_die foo min_size 2024-09-15T21:19:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:++ local pool_name config_opt raw val 2024-09-15T21:19:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:++ pool_name=foo 2024-09-15T21:19:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:++ config_opt=min_size 2024-09-15T21:19:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:+++ ceph osd pool get foo min_size 2024-09-15T21:19:01.102 INFO:tasks.workunit.client.0.smithi119.stderr:++ raw='min_size: 1' 2024-09-15T21:19:01.102 INFO:tasks.workunit.client.0.smithi119.stderr:++ [[ 0 -ne 0 ]] 2024-09-15T21:19:01.103 INFO:tasks.workunit.client.0.smithi119.stderr:+++ echo min_size: 1 2024-09-15T21:19:01.103 INFO:tasks.workunit.client.0.smithi119.stderr:+++ sed -e 's/[{} "]//g' 2024-09-15T21:19:01.105 INFO:tasks.workunit.client.0.smithi119.stderr:++ raw=min_size:1 2024-09-15T21:19:01.105 INFO:tasks.workunit.client.0.smithi119.stderr:+++ echo min_size:1 2024-09-15T21:19:01.105 INFO:tasks.workunit.client.0.smithi119.stderr:+++ cut -f2 -d: 2024-09-15T21:19:01.106 INFO:tasks.workunit.client.0.smithi119.stderr:++ val=1 2024-09-15T21:19:01.107 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo 1 2024-09-15T21:19:01.107 INFO:tasks.workunit.client.0.smithi119.stderr:++ return 0 2024-09-15T21:19:01.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ val=1 2024-09-15T21:19:01.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 1 != \1 ]] 2024-09-15T21:19:01.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set foo size 4 2024-09-15T21:19:01.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient: get_monmap_and_config 2024-09-15T21:19:01.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient: build_initial_monmap 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient: monmap: 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:01.225412+0000 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:01.225412+0000 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:01.227 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:01.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 Processor -- start 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 -- start start 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient: init 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efc74136b90 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc740a10d0 0x7efc7413cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): picked mon.noname-b con 0x7efc740a10d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc740a25b0 0x7efc7413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): picked mon.noname-c con 0x7efc740a25b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc740a18f0 con 0x7efc74136b90 2024-09-15T21:19:01.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7efc7409fc40 con 0x7efc740a25b0 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efc740a1bd0 con 0x7efc740a10d0 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7c3ca640 10 monclient(hunting): authenticate will time out at 1060.022323s 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc79965640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc740a10d0 msgr2=0x7efc7413cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc740a25b0 0x7efc7413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc79965640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc740a10d0 0x7efc7413cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc740a25b0 0x7efc7413f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37974/0 (socket says 172.21.15.119:37974) 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 -- 172.21.15.119:0/429700363 learned_addr learned my addr 172.21.15.119:0/429700363 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): get_auth_request con 0x7efc740a25b0 auth_method 0 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:01.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient(hunting): handle_auth_done global_id 14323 payload 274 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 -- 172.21.15.119:0/429700363 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc740a10d0 msgr2=0x7efc7413cc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 --2- 172.21.15.119:0/429700363 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc740a10d0 0x7efc7413cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 -- 172.21.15.119:0/429700363 >> v1:172.21.15.17:6789/0 conn(0x7efc74136b90 legacy=0x7efc74136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient: _finish_hunting 0 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 monclient: found mon.noname-c 2024-09-15T21:19:01.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:01.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.221+0000 7efc7a967640 1 -- 172.21.15.119:0/429700363 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc740a3660 con 0x7efc740a25b0 2024-09-15T21:19:01.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7a967640 1 --2- 172.21.15.119:0/429700363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a25b0 0x7efc7413f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009e50 tx=0x7efc6c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f3559669232788 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 1 -- 172.21.15.119:0/429700363 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0a57c0 con 0x7efc740a25b0 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: dump: 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:01.238 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: _finish_auth 0 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: _check_auth_tickets 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 1 -- 172.21.15.119:0/429700363 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c0a5960 con 0x7efc740a25b0 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 5 monclient: authenticate success, global_id 14323 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 4 set_mon_vals no callback set 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 1 -- 172.21.15.119:0/429700363 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0a5d30 con 0x7efc740a25b0 2024-09-15T21:19:01.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc79164640 10 monclient: dump: 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:01.240 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 10 monclient: shutdown 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 -- 172.21.15.119:0/429700363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a25b0 msgr2=0x7efc7413f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/429700363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a25b0 0x7efc7413f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009e50 tx=0x7efc6c01c8e0 comp rx=0 tx=0).stop 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 -- 172.21.15.119:0/429700363 shutdown_connections 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/429700363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a25b0 0x7efc7413f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/429700363 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc740a10d0 0x7efc7413cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 -- 172.21.15.119:0/429700363 >> 172.21.15.119:0/429700363 conn(0x7efc740a66e0 msgr2=0x7efc740a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 -- 172.21.15.119:0/429700363 shutdown_connections 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 1 -- 172.21.15.119:0/429700363 wait complete. 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 10 monclient: build_initial_monmap 2024-09-15T21:19:01.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.225+0000 7efc7c3ca640 10 monclient: monmap: 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:01.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 Processor -- start 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- start start 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: init 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 0x7efc7411cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): picked mon.noname-b con 0x7efc740a10d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc740a25b0 0x7efc7411eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): picked mon.noname-a con 0x7efc740a25b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc74136b90 0x7efc7411d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): picked mon.noname-c con 0x7efc74136b90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efc74142170 con 0x7efc740a25b0 2024-09-15T21:19:01.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efc74141ff0 con 0x7efc740a10d0 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efc741422f0 con 0x7efc74136b90 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient(hunting): authenticate will time out at 1060.028757s 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 0x7efc7411cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 0x7efc7411cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37978/0 (socket says 172.21.15.119:37978) 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 -- 172.21.15.119:0/4171917517 learned_addr learned my addr 172.21.15.119:0/4171917517 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): get_auth_request con 0x7efc740a10d0 auth_method 0 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:01.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a967640 1 -- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc74136b90 msgr2=0x7efc7411d2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a967640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc74136b90 0x7efc7411d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc79965640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc740a25b0 0x7efc7411eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient(hunting): handle_auth_done global_id 14326 payload 824 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 -- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc74136b90 msgr2=0x7efc7411d2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc74136b90 0x7efc7411d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 -- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc740a25b0 msgr2=0x7efc7411eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc740a25b0 0x7efc7411eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:01.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient: _finish_hunting 0 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 monclient: found mon.noname-b 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 -- 172.21.15.119:0/4171917517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7411dff0 con 0x7efc740a10d0 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7a166640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 0x7efc7411cb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c500 tx=0x7efc7000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70017070 con 0x7efc740a10d0 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: dump: 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.246 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: _finish_auth 0 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc700052d0 con 0x7efc740a10d0 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 5 monclient: authenticate success, global_id 14326 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _renew_subs 2024-09-15T21:19:01.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7411fab0 con 0x7efc740a10d0 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70012870 con 0x7efc740a10d0 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 10 monclient: dump: 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:01.248 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc78963640 4 set_mon_vals no callback set 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _renew_subs 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc74135fe0 con 0x7efc740a10d0 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7efc7001a070 con 0x7efc740a10d0 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc74136f70 con 0x7efc740a10d0 2024-09-15T21:19:01.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc5803b970 0x7efc5803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5474+0+0 (secure 0 0 0) 0x7efc7004efe0 con 0x7efc740a10d0 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc79965640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc5803b970 0x7efc5803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc79965640 10 monclient: get_auth_request con 0x7efc5803b970 auth_method 0 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.229+0000 7efc79965640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc5803b970 0x7efc5803de30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efc7413f240 tx=0x7efc64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.233+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7efc740a18f0 con 0x7efc740a10d0 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.233+0000 7efc6affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:01.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.233+0000 7efc6affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:01.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.405+0000 7efc7c3ca640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T21:19:01.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.405+0000 7efc7c3ca640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T21:19:01.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.405+0000 7efc7c3ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.405+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) -- 0x7efc7406a830 con 0x7efc740a10d0 2024-09-15T21:19:01.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.653+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v36) ==== 122+0+0 (secure 0 0 0) 0x7efc70003d60 con 0x7efc740a10d0 2024-09-15T21:19:01.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.653+0000 7efc6affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T21:19:01.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.653+0000 7efc6affd640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 4 2024-09-15T21:19:01.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.785+0000 7efc7c3ca640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T21:19:01.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.785+0000 7efc7c3ca640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T21:19:01.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.785+0000 7efc7c3ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:01.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:01.785+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) -- 0x7efc7411d810 con 0x7efc740a10d0 2024-09-15T21:19:02.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.653+0000 7efc6affd640 1 -- 172.21.15.119:0/4171917517 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v37) ==== 122+0+0 (secure 0 0 0) 0x7efc70019cd0 con 0x7efc740a10d0 2024-09-15T21:19:02.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.653+0000 7efc6affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T21:19:02.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.653+0000 7efc6affd640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 4 2024-09-15T21:19:02.664 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 4 size to 4 2024-09-15T21:19:02.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc5803b970 msgr2=0x7efc5803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:02.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc5803b970 0x7efc5803de30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efc7413f240 tx=0x7efc64008040 comp rx=0 tx=0).stop 2024-09-15T21:19:02.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 10 monclient: shutdown 2024-09-15T21:19:02.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:02.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 msgr2=0x7efc7411cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:02.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 0x7efc7411cb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c500 tx=0x7efc7000c9c0 comp rx=0 tx=0).stop 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 shutdown_connections 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc5803b970 0x7efc5803de30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc74136b90 0x7efc7411d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc740a25b0 0x7efc7411eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 --2- 172.21.15.119:0/4171917517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc740a10d0 0x7efc7411cb50 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c500 tx=0x7efc7000c9c0 comp rx=0 tx=0).stop 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 >> 172.21.15.119:0/4171917517 conn(0x7efc740a66e0 msgr2=0x7efc740a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 shutdown_connections 2024-09-15T21:19:02.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:02.665+0000 7efc7c3ca640 1 -- 172.21.15.119:0/4171917517 wait complete. 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_config_value foo min_size 2 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ local pool_name config_opt expected_val val 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ pool_name=foo 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ config_opt=min_size 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_val=2 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:++ get_config_value_or_die foo min_size 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:++ local pool_name config_opt raw val 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:++ pool_name=foo 2024-09-15T21:19:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:++ config_opt=min_size 2024-09-15T21:19:02.701 INFO:tasks.workunit.client.0.smithi119.stderr:+++ ceph osd pool get foo min_size 2024-09-15T21:19:03.141 INFO:tasks.workunit.client.0.smithi119.stderr:++ raw='min_size: 2' 2024-09-15T21:19:03.141 INFO:tasks.workunit.client.0.smithi119.stderr:++ [[ 0 -ne 0 ]] 2024-09-15T21:19:03.142 INFO:tasks.workunit.client.0.smithi119.stderr:+++ echo min_size: 2 2024-09-15T21:19:03.142 INFO:tasks.workunit.client.0.smithi119.stderr:+++ sed -e 's/[{} "]//g' 2024-09-15T21:19:03.144 INFO:tasks.workunit.client.0.smithi119.stderr:++ raw=min_size:2 2024-09-15T21:19:03.145 INFO:tasks.workunit.client.0.smithi119.stderr:+++ echo min_size:2 2024-09-15T21:19:03.145 INFO:tasks.workunit.client.0.smithi119.stderr:+++ cut -f2 -d: 2024-09-15T21:19:03.147 INFO:tasks.workunit.client.0.smithi119.stderr:++ val=2 2024-09-15T21:19:03.147 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo 2 2024-09-15T21:19:03.147 INFO:tasks.workunit.client.0.smithi119.stderr:++ return 0 2024-09-15T21:19:03.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ val=2 2024-09-15T21:19:03.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 2 != \2 ]] 2024-09-15T21:19:03.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set foo size 10 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient: get_monmap_and_config 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient: build_initial_monmap 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient: monmap: 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:03.260914+0000 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:03.260914+0000 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.262 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:03.263 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:03.263 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:03.263 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:03.263 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:03.263 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 1 Processor -- start 2024-09-15T21:19:03.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 1 -- start start 2024-09-15T21:19:03.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient: init 2024-09-15T21:19:03.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:03.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 0x7f4534137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient(hunting): picked mon.noname-b con 0x7f45340a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f45340a41b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f45340a4590 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4534135120 con 0x7f45340a41b0 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45340a19d0 con 0x7f45340a4590 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45340a4a40 con 0x7f45340a7ac0 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 10 monclient(hunting): authenticate will time out at 1062.057557s 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4532ffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 msgr2=0x7f4534137cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:03.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4532ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 0x7f4534137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f45340a4590 0x7f4534134a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47614/0 (socket says 172.21.15.119:47614) 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 -- 172.21.15.119:0/679763964 learned_addr learned my addr 172.21.15.119:0/679763964 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870282707 0 0) 0x7f45340a19d0 con 0x7f45340a4590 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): my global_id is 14335 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4520003530 con 0x7f45340a4590 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2341395635 0 0) 0x7f4520003530 con 0x7f45340a4590 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 msgr2=0x7f4534137cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 0x7f4534137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 >> v1:172.21.15.17:6789/0 conn(0x7f45340a41b0 legacy=0x7f4534134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:03.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45340a6960 con 0x7f45340a4590 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f4524001700 con 0x7f45340a4590 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: dump: 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.271 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 >> v1:172.21.15.119:6789/0 conn(0x7f45340a4590 legacy=0x7f4534134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:03.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 0x7f45200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): picked mon.c con 0x7f45340a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4520004140 0x7f4520004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): picked mon.a con 0x7f4520004140 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4520004f50 0x7f452000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): picked mon.b con 0x7f4520004f50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f45340a4a40 con 0x7f4520004140 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f452000bf10 con 0x7f4520004f50 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f452000c090 con 0x7f45340a7ac0 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4531ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4532ffd640 1 -- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 msgr2=0x7f45200039e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f4532ffd640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 0x7f45200039e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4520004f50 0x7f452000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): get_auth_request con 0x7f4520004f50 auth_method 0 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:03.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45327fc640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4520004140 0x7f4520004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient(hunting): handle_auth_done global_id 14335 payload 293 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 -- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 msgr2=0x7f45200039e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45340a7ac0 0x7f45200039e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 -- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4520004140 msgr2=0x7f4520004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4520004140 0x7f4520004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 monclient: found mon.b 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 1 -- 172.21.15.119:0/679763964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4520003530 con 0x7f4520004f50 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:03.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45327fc640 10 monclient: get_auth_request con 0x7f4520004140 auth_method 0 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45327fc640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4520004140 0x7f4520004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 5 monclient: authenticate success, global_id 14335 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.257+0000 7f45392c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45337fe640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4520004f50 0x7f452000b9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f452400c1c0 tx=0x7f452401cb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3878b5c41816395 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524007cf0 con 0x7f4520004f50 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: dump: 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:03.276 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45240a9020 con 0x7f4520004f50 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 4 set_mon_vals no callback set 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 1 -- 172.21.15.119:0/679763964 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524005ae0 con 0x7f4520004f50 2024-09-15T21:19:03.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4531ffb640 10 monclient: dump: 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:03.278 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: shutdown 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- 172.21.15.119:0/679763964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4520004f50 msgr2=0x7f452000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 --2- 172.21.15.119:0/679763964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4520004f50 0x7f452000b9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f452400c1c0 tx=0x7f452401cb60 comp rx=0 tx=0).stop 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45337fe640 1 -- 172.21.15.119:0/679763964 reap_dead start 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- 172.21.15.119:0/679763964 shutdown_connections 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- 172.21.15.119:0/679763964 >> 172.21.15.119:0/679763964 conn(0x7f45340a6fd0 msgr2=0x7f45340a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- 172.21.15.119:0/679763964 shutdown_connections 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- 172.21.15.119:0/679763964 wait complete. 2024-09-15T21:19:03.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: build_initial_monmap 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: monmap: 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 Processor -- start 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- start start 2024-09-15T21:19:03.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: init 2024-09-15T21:19:03.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 0x7f45341cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): picked mon.noname-b con 0x7f45341cf590 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4534073e90 0x7f45341cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): picked mon.noname-c con 0x7f4534073e90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340718c0 0x7f4534071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): picked mon.noname-a con 0x7f45340718c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4534140df0 con 0x7f45340718c0 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4534140c70 con 0x7f45341cf590 2024-09-15T21:19:03.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4534140f70 con 0x7f4534073e90 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45392c5640 10 monclient(hunting): authenticate will time out at 1062.063206s 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45327fc640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4534073e90 msgr2=0x7f45341cfeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45327fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4534073e90 0x7f45341cfeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 0x7f45341cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 0x7f45341cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52822/0 (socket says 172.21.15.119:52822) 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 -- 172.21.15.119:0/1124384479 learned_addr learned my addr 172.21.15.119:0/1124384479 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): get_auth_request con 0x7f45341cf590 auth_method 0 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:03.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:03.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:03.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45337fe640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340718c0 0x7f4534071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:03.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient(hunting): handle_auth_done global_id 14338 payload 824 2024-09-15T21:19:03.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 -- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4534073e90 msgr2=0x7f45341cfeb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4534073e90 0x7f45341cfeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 -- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340718c0 msgr2=0x7f4534071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340718c0 0x7f4534071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 monclient: found mon.noname-b 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 -- 172.21.15.119:0/1124384479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4534073330 con 0x7f45341cf590 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45337fe640 10 monclient: get_auth_request con 0x7f45340718c0 auth_method 0 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f45337fe640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340718c0 0x7f4534071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.261+0000 7f4532ffd640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 0x7f45341cf970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b530 tx=0x7f452c00b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c013020 con 0x7f45341cf590 2024-09-15T21:19:03.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: dump: 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:03.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c003df0 con 0x7f45341cf590 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0052b0 con 0x7f45341cf590 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 5 monclient: authenticate success, global_id 14338 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 monclient: _renew_subs 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45340a16c0 con 0x7f45341cf590 2024-09-15T21:19:03.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 10 monclient: dump: 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:03.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f4530ff9640 4 set_mon_vals no callback set 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 monclient: _renew_subs 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453413b540 con 0x7f45341cf590 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4534134300 con 0x7f45341cf590 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f452c016020 con 0x7f45341cf590 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451403b970 0x7f451403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5474+0+0 (secure 0 0 0) 0x7f452c04ffd0 con 0x7f45341cf590 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45327fc640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451403b970 0x7f451403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45327fc640 10 monclient: get_auth_request con 0x7f451403b970 auth_method 0 2024-09-15T21:19:03.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.265+0000 7f45327fc640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451403b970 0x7f451403de30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f451c007fd0 tx=0x7f451c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:03.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.269+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f452c01d020 con 0x7f45341cf590 2024-09-15T21:19:03.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.269+0000 7f452b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:03.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.269+0000 7f452b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:03.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.437+0000 7f45392c5640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T21:19:03.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.437+0000 7f45392c5640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T21:19:03.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.437+0000 7f45392c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.437+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) -- 0x7f45341d0450 con 0x7f45341cf590 2024-09-15T21:19:03.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.677+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v38) ==== 124+0+0 (secure 0 0 0) 0x7f452c0173b0 con 0x7f45341cf590 2024-09-15T21:19:03.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.677+0000 7f452b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T21:19:03.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.677+0000 7f452b7fe640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 10 2024-09-15T21:19:03.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.845+0000 7f45392c5640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T21:19:03.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.845+0000 7f45392c5640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T21:19:03.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.845+0000 7f45392c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:03.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:03.845+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) -- 0x7f453406a760 con 0x7f45341cf590 2024-09-15T21:19:04.115 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T21:19:04.115 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:19:04.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:19:04.119 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:19:04.119 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:19:04.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient: get_monmap_and_config 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient: build_initial_monmap 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient: monmap: 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:04.236014+0000 2024-09-15T21:19:04.238 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:04.236014+0000 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:04.239 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 Processor -- start 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 -- start start 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient: init 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f50481378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50481383b0 0x7f504813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): picked mon.noname-a con 0x7f50481383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f504813d2c0 0x7f504813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): picked mon.noname-c con 0x7f504813d2c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50480a1a20 con 0x7f50481378c0 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f50480a1d00 con 0x7f50481383b0 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f50480a1fe0 con 0x7f504813d2c0 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f504d291640 10 monclient(hunting): authenticate will time out at 1067.631120s 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f5046ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f50481378c0 0x7f5048137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53984/0 (socket says 172.21.15.17:53984) 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50481383b0 0x7f504813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f5046ffd640 1 -- 172.21.15.17:0/3944462835 learned_addr learned my addr 172.21.15.17:0/3944462835 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): get_auth_request con 0x7f50481383b0 auth_method 0 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient(hunting): handle_auth_done global_id 4608 payload 274 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50477fe640 1 --2- 172.21.15.17:0/3944462835 >> v2:172.21.15.119:3300/0 conn(0x7f504813d2c0 0x7f504813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 1 -- 172.21.15.17:0/3944462835 >> v2:172.21.15.119:3300/0 conn(0x7f504813d2c0 msgr2=0x7f504813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 1 --2- 172.21.15.17:0/3944462835 >> v2:172.21.15.119:3300/0 conn(0x7f504813d2c0 0x7f504813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 1 -- 172.21.15.17:0/3944462835 >> v1:172.21.15.17:6789/0 conn(0x7f50481378c0 legacy=0x7f5048137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient: _finish_hunting 0 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 1 monclient: found mon.noname-a 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f5045ffb640 1 -- 172.21.15.17:0/3944462835 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551603927 0 0) 0x7f5038001120 con 0x7f50481378c0 2024-09-15T21:19:04.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50467fc640 1 -- 172.21.15.17:0/3944462835 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50480a22c0 con 0x7f50481383b0 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50477fe640 10 monclient: get_auth_request con 0x7f504813d2c0 auth_method 0 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f5045ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.230+0000 7f50477fe640 1 --2- 172.21.15.17:0/3944462835 >> v2:172.21.15.119:3300/0 conn(0x7f504813d2c0 0x7f504813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f50467fc640 1 --2- 172.21.15.17:0/3944462835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481383b0 0x7f504813cd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f503c0097c0 tx=0x7f503c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438b2415b9b0ba98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 1 -- 172.21.15.17:0/3944462835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0b1070 con 0x7f50481383b0 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: dump: 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 1 -- 172.21.15.17:0/3944462835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f503c0a47b0 con 0x7f50481383b0 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 1 -- 172.21.15.17:0/3944462835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0a4b80 con 0x7f50481383b0 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 5 monclient: authenticate success, global_id 4608 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 4 set_mon_vals no callback set 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5045ffb640 10 monclient: dump: 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: shutdown 2024-09-15T21:19:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- 172.21.15.17:0/3944462835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481383b0 msgr2=0x7f504813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 --2- 172.21.15.17:0/3944462835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481383b0 0x7f504813cd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f503c0097c0 tx=0x7f503c01c000 comp rx=0 tx=0).stop 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- 172.21.15.17:0/3944462835 shutdown_connections 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 --2- 172.21.15.17:0/3944462835 >> v2:172.21.15.119:3300/0 conn(0x7f504813d2c0 0x7f504813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 --2- 172.21.15.17:0/3944462835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481383b0 0x7f504813cd80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- 172.21.15.17:0/3944462835 >> 172.21.15.17:0/3944462835 conn(0x7f50480aac40 msgr2=0x7f50480a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- 172.21.15.17:0/3944462835 shutdown_connections 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- 172.21.15.17:0/3944462835 wait complete. 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: build_initial_monmap 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: monmap: 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:04.249 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:04.250 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 Processor -- start 2024-09-15T21:19:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- start start 2024-09-15T21:19:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: init 2024-09-15T21:19:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 0x7f50481d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): picked mon.noname-a con 0x7f50481378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 0x7f5048136a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): picked mon.noname-c con 0x7f50481383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f504813d2c0 0x7f504809db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): picked mon.noname-b con 0x7f504813d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5048143020 con 0x7f50481378c0 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5048142ea0 con 0x7f504813d2c0 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f50481431a0 con 0x7f50481383b0 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f504d291640 10 monclient(hunting): authenticate will time out at 1067.635879s 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.234+0000 7f5046ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 0x7f50481d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 0x7f5048136a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 0x7f50481d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60616/0 (socket says 172.21.15.17:60616) 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 -- 172.21.15.17:0/1174360278 learned_addr learned my addr 172.21.15.17:0/1174360278 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): get_auth_request con 0x7f50481378c0 auth_method 0 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 10 monclient(hunting): get_auth_request con 0x7f50481383b0 auth_method 0 2024-09-15T21:19:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 -- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 msgr2=0x7f5048136a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 -- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 msgr2=0x7f5048136a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 0x7f5048136a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 0x7f5048136a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50477fe640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f504813d2c0 0x7f504809db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50477fe640 10 monclient(hunting): get_auth_request con 0x7f504813d2c0 auth_method 0 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient(hunting): handle_auth_done global_id 4614 payload 824 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 -- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 msgr2=0x7f5048136a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 0x7f5048136a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 -- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f504813d2c0 msgr2=0x7f504809db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f504813d2c0 0x7f504809db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 monclient: found mon.noname-a 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 -- 172.21.15.17:0/1174360278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504809f570 con 0x7f50481378c0 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f5046ffd640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 0x7f50481d0650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f503800c730 tx=0x7f503800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038018070 con 0x7f50481378c0 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: dump: 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50380070d0 con 0x7f50481378c0 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 5 monclient: authenticate success, global_id 4614 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 monclient: _renew_subs 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038007500 con 0x7f50481378c0 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50480a12f0 con 0x7f50481378c0 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 10 monclient: dump: 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50457fa640 4 set_mon_vals no callback set 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 monclient: _renew_subs 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50480a00d0 con 0x7f50481378c0 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5048137ca0 con 0x7f50481378c0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f503801b070 con 0x7f50481378c0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f503403b9f0 0x7f503403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5474+0+0 (secure 0 0 0) 0x7f503804f0e0 con 0x7f50481378c0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f503403b9f0 0x7f503403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 10 monclient: get_auth_request con 0x7f503403b9f0 auth_method 0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.238+0000 7f50467fc640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f503403b9f0 0x7f503403deb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f503c0ae000 tx=0x7f503c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.242+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f50480a12f0 con 0x7f50481378c0 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.242+0000 7f502f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:04.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.242+0000 7f502f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:04.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.490+0000 7f504d291640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:04.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.490+0000 7f504d291640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:04.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.490+0000 7f504d291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.490+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5048120eb0 con 0x7f50481378c0 2024-09-15T21:19:04.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f502f7fe640 1 -- 172.21.15.17:0/1174360278 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f5048120eb0 con 0x7f50481378c0 2024-09-15T21:19:04.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f502f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:04.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f502f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:04.501 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f503403b9f0 msgr2=0x7f503403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f503403b9f0 0x7f503403deb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f503c0ae000 tx=0x7f503c008040 comp rx=0 tx=0).stop 2024-09-15T21:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 10 monclient: shutdown 2024-09-15T21:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 msgr2=0x7f50481d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 0x7f50481d0650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f503800c730 tx=0x7f503800cbf0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 shutdown_connections 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f503403b9f0 0x7f503403deb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f504813d2c0 0x7f504809db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f50481383b0 0x7f5048136a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 --2- 172.21.15.17:0/1174360278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50481378c0 0x7f50481d0650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 >> 172.21.15.17:0/1174360278 conn(0x7f50480aac40 msgr2=0x7f504806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 shutdown_connections 2024-09-15T21:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.494+0000 7f504d291640 1 -- 172.21.15.17:0/1174360278 wait complete. 2024-09-15T21:19:04.519 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:19:04.520 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:19:04.520 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:19:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient: get_monmap_and_config 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient: build_initial_monmap 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient: monmap: 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:04.642607+0000 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:04.642607+0000 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:04.646 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 Processor -- start 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 -- start start 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient: init 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): picked mon.noname-b con 0x7fa48c1340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa48c09d6c0 0x7fa48c0a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): picked mon.noname-a con 0x7fa48c09d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa48c0a0dd0 0x7fa48c0a1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): picked mon.noname-c con 0x7fa48c0a0dd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c09d520 con 0x7fa48c09d6c0 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c1349e0 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c09e1b0 con 0x7fa48c1340c0 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa492e9f640 10 monclient(hunting): authenticate will time out at 1068.038169s 2024-09-15T21:19:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa48c09d6c0 0x7fa48c0a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38878/0 (socket says 172.21.15.17:38878) 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 -- 172.21.15.17:0/3481632158 learned_addr learned my addr 172.21.15.17:0/3481632158 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa48c09d6c0 0x7fa48c0a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60634/0 (socket says 172.21.15.17:60634) 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 10 monclient(hunting): get_auth_request con 0x7fa48c1340c0 auth_method 0 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): get_auth_request con 0x7fa48c09d6c0 auth_method 0 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 -- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 msgr2=0x7fa48c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 -- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 msgr2=0x7fa48c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa490c3b640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa49143c640 1 --2- 172.21.15.17:0/3481632158 >> v2:172.21.15.119:3300/0 conn(0x7fa48c0a0dd0 0x7fa48c0a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa49143c640 10 monclient(hunting): get_auth_request con 0x7fa48c0a0dd0 auth_method 0 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa49143c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa49143c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa49143c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient(hunting): handle_auth_done global_id 4617 payload 274 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 -- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 msgr2=0x7fa48c1344a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 -- 172.21.15.17:0/3481632158 >> v2:172.21.15.119:3300/0 conn(0x7fa48c0a0dd0 msgr2=0x7fa48c0a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 --2- 172.21.15.17:0/3481632158 >> v2:172.21.15.119:3300/0 conn(0x7fa48c0a0dd0 0x7fa48c0a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 monclient: found mon.noname-a 2024-09-15T21:19:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.638+0000 7fa48bfff640 1 -- 172.21.15.17:0/3481632158 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c09e490 con 0x7fa48c09d6c0 2024-09-15T21:19:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c09d6c0 0x7fa48c0a0890 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa48c06f1c0 tx=0x7fa47801c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bd5ccadd3b241a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 1 -- 172.21.15.17:0/3481632158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780b0070 con 0x7fa48c09d6c0 2024-09-15T21:19:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: dump: 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 1 -- 172.21.15.17:0/3481632158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4780ab400 con 0x7fa48c09d6c0 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 5 monclient: authenticate success, global_id 4617 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 1 -- 172.21.15.17:0/3481632158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780ab7d0 con 0x7fa48c09d6c0 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 4 set_mon_vals no callback set 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48b7fe640 10 monclient: dump: 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: shutdown 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c09d6c0 msgr2=0x7fa48c0a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c09d6c0 0x7fa48c0a0890 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa48c06f1c0 tx=0x7fa47801c320 comp rx=0 tx=0).stop 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- 172.21.15.17:0/3481632158 shutdown_connections 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- 172.21.15.17:0/3481632158 >> v2:172.21.15.119:3300/0 conn(0x7fa48c0a0dd0 0x7fa48c0a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c09d6c0 0x7fa48c0a0890 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- 172.21.15.17:0/3481632158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c1344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- 172.21.15.17:0/3481632158 >> 172.21.15.17:0/3481632158 conn(0x7fa48c0a66a0 msgr2=0x7fa48c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- 172.21.15.17:0/3481632158 shutdown_connections 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- 172.21.15.17:0/3481632158 wait complete. 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: build_initial_monmap 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: monmap: 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 Processor -- start 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- start start 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: init 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c09d6c0 0x7fa48c1386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): picked mon.noname-b con 0x7fa48c09d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c0a0dd0 0x7fa48c138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): picked mon.noname-a con 0x7fa48c0a0dd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): picked mon.noname-c con 0x7fa48c1340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c13ae60 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0a27a0 con 0x7fa48c09d6c0 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c13afe0 con 0x7fa48c1340c0 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa492e9f640 10 monclient(hunting): authenticate will time out at 1068.043526s 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38880/0 (socket says 172.21.15.17:38880) 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 -- 172.21.15.17:0/1976563840 learned_addr learned my addr 172.21.15.17:0/1976563840 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 10 monclient(hunting): get_auth_request con 0x7fa48c1340c0 auth_method 0 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c0a0dd0 0x7fa48c138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): get_auth_request con 0x7fa48c0a0dd0 auth_method 0 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 msgr2=0x7fa48c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 msgr2=0x7fa48c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa48bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa490c3b640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c09d6c0 0x7fa48c1386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa49143c640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c139160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa490c3b640 10 monclient(hunting): get_auth_request con 0x7fa48c09d6c0 auth_method 0 2024-09-15T21:19:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa490c3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa490c3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.642+0000 7fa490c3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 10 monclient(hunting): handle_auth_done global_id 4620 payload 824 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 msgr2=0x7fa48c139160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa48c1340c0 0x7fa48c139160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c09d6c0 msgr2=0x7fa48c1386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c09d6c0 0x7fa48c1386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 monclient: found mon.noname-a 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c073500 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48bfff640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c0a0dd0 0x7fa48c138c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4780ab000 tx=0x7fa4780190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780b8020 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: dump: 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.666 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: _finish_auth 0 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4780b4410 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 5 monclient: authenticate success, global_id 4620 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 10 monclient: _renew_subs 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c13a070 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780b0040 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: dump: 2024-09-15T21:19:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 10 monclient: _renew_subs 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa48affd640 4 set_mon_vals no callback set 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c0a3fb0 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 0x7fa48c071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c09d0f0 con 0x7fa48c1396a0 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 0x7fa48c071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient: get_auth_request con 0x7fa48c1396a0 auth_method 0 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa4780bb020 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 10 monclient(hunting): handle_auth_done global_id 4623 payload 824 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa48c0a0890 con 0x7fa48c1396a0 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa47003b9f0 0x7fa47003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5474+0+0 (secure 0 0 0) 0x7fa4780e9960 con 0x7fa48c0a0dd0 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa490c3b640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 0x7fa48c071dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa480010790 tx=0x7fa480010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48000f040 con 0x7fa48c1396a0 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: dump: 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa49143c640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa47003b9f0 0x7fa47003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa49143c640 10 monclient: get_auth_request con 0x7fa47003b9f0 auth_method 0 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa480015470 con 0x7fa48c1396a0 2024-09-15T21:19:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 msgr2=0x7fa48c071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 0x7fa48c071dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa480010790 tx=0x7fa480010c50 comp rx=0 tx=0).stop 2024-09-15T21:19:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 msgr2=0x7fa48c071dd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa4897fa640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1396a0 0x7fa48c071dd0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa480010790 tx=0x7fa480010c50 comp rx=0 tx=0).stop 2024-09-15T21:19:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.646+0000 7fa49143c640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa47003b9f0 0x7fa47003deb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa484009770 tx=0x7fa48400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 0x7fa48c0726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c12e680 con 0x7fa48c072310 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 0x7fa48c0726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient: get_auth_request con 0x7fa48c072310 auth_method 0 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 10 monclient(hunting): handle_auth_done global_id 4626 payload 824 2024-09-15T21:19:04.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 1 -- 172.21.15.17:0/1976563840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa48c139a80 con 0x7fa48c072310 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa490c3b640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 0x7fa48c0726f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa480010790 tx=0x7fa480004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480004020 con 0x7fa48c072310 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 10 monclient: dump: 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.675 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4881 (secure 0 0 0) 0x7fa480020800 con 0x7fa48c072310 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 msgr2=0x7fa48c0726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 0x7fa48c0726f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa480010790 tx=0x7fa480004740 comp rx=0 tx=0).stop 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 msgr2=0x7fa48c0726f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa4897fa640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c072310 0x7fa48c0726f0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa480010790 tx=0x7fa480004740 comp rx=0 tx=0).stop 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:04.676 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 110275, 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:04.677 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:04.678 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:04.681 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:04.682 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:04.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:04.684 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:04.689 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa47003b9f0 msgr2=0x7fa47003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa47003b9f0 0x7fa47003deb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa484009770 tx=0x7fa48400b040 comp rx=0 tx=0).stop 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 10 monclient: shutdown 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c0a0dd0 msgr2=0x7fa48c138c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 --2- 172.21.15.17:0/1976563840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c0a0dd0 0x7fa48c138c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4780ab000 tx=0x7fa4780190c0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa49143c640 1 -- 172.21.15.17:0/1976563840 reap_dead start 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa49143c640 1 -- 172.21.15.17:0/1976563840 reap_dead start 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 shutdown_connections 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 >> 172.21.15.17:0/1976563840 conn(0x7fa48c0a66a0 msgr2=0x7fa48c06f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 shutdown_connections 2024-09-15T21:19:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.654+0000 7fa492e9f640 1 -- 172.21.15.17:0/1976563840 wait complete. 2024-09-15T21:19:04.692 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:04.692 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:19:04.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:19:04.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.705+0000 7f452b7fe640 1 -- 172.21.15.119:0/1124384479 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v39) ==== 124+0+0 (secure 0 0 0) 0x7f452c00e560 con 0x7f45341cf590 2024-09-15T21:19:04.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.705+0000 7f452b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T21:19:04.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.705+0000 7f452b7fe640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 10 2024-09-15T21:19:04.719 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 4 size to 10 2024-09-15T21:19:04.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451403b970 msgr2=0x7f451403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451403b970 0x7f451403de30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f451c007fd0 tx=0x7f451c009040 comp rx=0 tx=0).stop 2024-09-15T21:19:04.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 10 monclient: shutdown 2024-09-15T21:19:04.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 msgr2=0x7f45341cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 0x7f45341cf970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b530 tx=0x7f452c00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 shutdown_connections 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451403b970 0x7f451403de30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340718c0 0x7f4534071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4534073e90 0x7f45341cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 --2- 172.21.15.119:0/1124384479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45341cf590 0x7f45341cf970 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 >> 172.21.15.119:0/1124384479 conn(0x7f45340a6fd0 msgr2=0x7f453413fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 shutdown_connections 2024-09-15T21:19:04.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:04.717+0000 7f45392c5640 1 -- 172.21.15.119:0/1124384479 wait complete. 2024-09-15T21:19:04.788 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_config_value foo min_size 5 2024-09-15T21:19:04.788 INFO:tasks.workunit.client.0.smithi119.stderr:+ local pool_name config_opt expected_val val 2024-09-15T21:19:04.788 INFO:tasks.workunit.client.0.smithi119.stderr:+ pool_name=foo 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:+ config_opt=min_size 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_val=5 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:++ get_config_value_or_die foo min_size 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:++ local pool_name config_opt raw val 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:++ pool_name=foo 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:++ config_opt=min_size 2024-09-15T21:19:04.789 INFO:tasks.workunit.client.0.smithi119.stderr:+++ ceph osd pool get foo min_size 2024-09-15T21:19:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient: get_monmap_and_config 2024-09-15T21:19:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient: build_initial_monmap 2024-09-15T21:19:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient: monmap: 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:04.886499+0000 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:04.886499+0000 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:04.889 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:04.890 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 Processor -- start 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 -- start start 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient: init 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7a40a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a40a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7a40a59a0 0x7fc7a413f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): picked mon.noname-c con 0x7fc7a40a59a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a40a2c30 con 0x7fc7a40a4a90 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a40a0f50 con 0x7fc7a40a59a0 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40a1a00 con 0x7fc7a40a5580 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7ac283640 10 monclient(hunting): authenticate will time out at 1068.281599s 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa01f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc7a40a4a90 0x7fc7a40a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53992/0 (socket says 172.21.15.17:53992) 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa01f640 1 -- 172.21.15.17:0/4040583635 learned_addr learned my addr 172.21.15.17:0/4040583635 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38894/0 (socket says 172.21.15.17:38894) 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 10 monclient(hunting): get_auth_request con 0x7fc7a40a5580 auth_method 0 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 1 --2- 172.21.15.17:0/4040583635 >> v2:172.21.15.119:3300/0 conn(0x7fc7a40a59a0 0x7fc7a413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): get_auth_request con 0x7fc7a40a59a0 auth_method 0 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 1 -- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 msgr2=0x7fc7a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 1 -- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 msgr2=0x7fc7a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a981e640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a901d640 1 -- 172.21.15.17:0/4040583635 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751329937 0 0) 0x7fc7a0001120 con 0x7fc7a40a4a90 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a901d640 10 monclient(hunting): _negotiate 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a901d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a901d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a901d640 10 monclient(hunting): my global_id is 4629 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7a901d640 1 -- 172.21.15.17:0/4040583635 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc790003530 con 0x7fc7a40a4a90 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient(hunting): handle_auth_done global_id 14350 payload 274 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 1 -- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 msgr2=0x7fc7a413cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 1 -- 172.21.15.17:0/4040583635 >> v1:172.21.15.17:6789/0 conn(0x7fc7a40a4a90 legacy=0x7fc7a40a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient: _finish_hunting 0 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 1 monclient: found mon.noname-c 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.882+0000 7fc7aa820640 1 -- 172.21.15.17:0/4040583635 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a40a1c90 con 0x7fc7a40a59a0 2024-09-15T21:19:04.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 0x7fc7a413f260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009d90 tx=0x7fc79c01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d36a9468a984bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 1 -- 172.21.15.17:0/4040583635 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0b2070 con 0x7fc7a40a59a0 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: dump: 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: _finish_auth 0 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: _check_auth_tickets 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 1 -- 172.21.15.17:0/4040583635 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0a57e0 con 0x7fc7a40a59a0 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 5 monclient: authenticate success, global_id 14350 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 1 -- 172.21.15.17:0/4040583635 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0a5bb0 con 0x7fc7a40a59a0 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 4 set_mon_vals no callback set 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a901d640 10 monclient: dump: 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.905 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: shutdown 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- 172.21.15.17:0/4040583635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 msgr2=0x7fc7a413f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 0x7fc7a413f260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009d90 tx=0x7fc79c01c800 comp rx=0 tx=0).stop 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- 172.21.15.17:0/4040583635 shutdown_connections 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 0x7fc7a413f260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 --2- 172.21.15.17:0/4040583635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a5580 0x7fc7a413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- 172.21.15.17:0/4040583635 >> 172.21.15.17:0/4040583635 conn(0x7fc7a40aabb0 msgr2=0x7fc7a40a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- 172.21.15.17:0/4040583635 shutdown_connections 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- 172.21.15.17:0/4040583635 wait complete. 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: build_initial_monmap 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: monmap: 2024-09-15T21:19:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 Processor -- start 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- start start 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: init 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 0x7fc7a41cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): picked mon.noname-c con 0x7fc7a40a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a40a5580 0x7fc7a41cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): picked mon.noname-a con 0x7fc7a40a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 0x7fc7a41d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a40a59a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41421d0 con 0x7fc7a40a5580 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a406f2c0 con 0x7fc7a40a59a0 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a4142350 con 0x7fc7a40a4a90 2024-09-15T21:19:04.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7ac283640 10 monclient(hunting): authenticate will time out at 1068.287487s 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 0x7fc7a41cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 0x7fc7a41cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38908/0 (socket says 172.21.15.17:38908) 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a40a5580 0x7fc7a41cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 learned_addr learned my addr 172.21.15.17:0/2280189507 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request con 0x7fc7a40a4a90 auth_method 0 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): get_auth_request con 0x7fc7a40a5580 auth_method 0 2024-09-15T21:19:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 msgr2=0x7fc7a41cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 msgr2=0x7fc7a41cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 0x7fc7a41cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 0x7fc7a41d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7a981e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): get_auth_request con 0x7fc7a40a59a0 auth_method 0 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 0x7fc7a41cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:04.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:04.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:04.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.886+0000 7fc7aa820640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:04.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 10 monclient(hunting): handle_auth_done global_id 4632 payload 824 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 msgr2=0x7fc7a41cf5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a40a4a90 0x7fc7a41cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 msgr2=0x7fc7a41d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a40a59a0 0x7fc7a41d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 monclient: found mon.noname-a 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a413f440 con 0x7fc7a40a5580 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7a981e640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a40a5580 0x7fc7a41cfb30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc79400e7e0 tx=0x7fc79400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794017030 con 0x7fc7a40a5580 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: dump: 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: _finish_auth 0 2024-09-15T21:19:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79400edd0 con 0x7fc7a40a5580 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79400f040 con 0x7fc7a40a5580 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 10 monclient: dump: 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:04.914 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79bfff640 4 set_mon_vals no callback set 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 5 monclient: authenticate success, global_id 4632 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 10 monclient: _renew_subs 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a40a93c0 con 0x7fc7a40a5580 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 10 monclient: _renew_subs 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a413da00 con 0x7fc7a40a5580 2024-09-15T21:19:04.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40a4e70 con 0x7fc7a4134fe0 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a4134fe0 auth_method 0 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:04.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc79401a070 con 0x7fc7a40a5580 2024-09-15T21:19:04.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78803b9c0 0x7fc78803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:04.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7fc79404f790 con 0x7fc7a40a5580 2024-09-15T21:19:04.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa820640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78803b9c0 0x7fc78803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:04.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa820640 10 monclient: get_auth_request con 0x7fc78803b9c0 auth_method 0 2024-09-15T21:19:04.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:04.890+0000 7fc7aa820640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78803b9c0 0x7fc78803de80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a4134fe0 auth_method 0 2024-09-15T21:19:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.094+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:05.224 INFO:tasks.workunit.client.0.smithi119.stderr:++ raw='min_size: 5' 2024-09-15T21:19:05.224 INFO:tasks.workunit.client.0.smithi119.stderr:++ [[ 0 -ne 0 ]] 2024-09-15T21:19:05.225 INFO:tasks.workunit.client.0.smithi119.stderr:+++ echo min_size: 5 2024-09-15T21:19:05.225 INFO:tasks.workunit.client.0.smithi119.stderr:+++ sed -e 's/[{} "]//g' 2024-09-15T21:19:05.226 INFO:tasks.workunit.client.0.smithi119.stderr:++ raw=min_size:5 2024-09-15T21:19:05.227 INFO:tasks.workunit.client.0.smithi119.stderr:+++ echo min_size:5 2024-09-15T21:19:05.227 INFO:tasks.workunit.client.0.smithi119.stderr:+++ cut -f2 -d: 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:++ val=5 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo 5 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:++ return 0 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ val=5 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 5 != \5 ]] 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool set foo size 0 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:05.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set foo size 0 2024-09-15T21:19:05.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient: get_monmap_and_config 2024-09-15T21:19:05.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient: build_initial_monmap 2024-09-15T21:19:05.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient: monmap: 2024-09-15T21:19:05.351 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:05.351 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.351 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:05.348911+0000 2024-09-15T21:19:05.351 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:05.348911+0000 2024-09-15T21:19:05.351 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:05.352 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.352 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:05.352 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:05.352 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:05.352 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:05.352 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:05.353 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 1 Processor -- start 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 1 -- start start 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient: init 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd5180a2b70 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:05.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd5180a3660 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 0x7fd51813e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient(hunting): picked mon.noname-b con 0x7fd5180a3ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5180a48e0 con 0x7fd5180a3660 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd518074970 con 0x7fd5180a2b70 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd51809c300 con 0x7fd5180a3ab0 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 10 monclient(hunting): authenticate will time out at 1064.145686s 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd5180a2b70 0x7fd5180a2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47630/0 (socket says 172.21.15.119:47630) 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 1 -- 172.21.15.119:0/3343150444 learned_addr learned my addr 172.21.15.119:0/3343150444 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 73915797 0 0) 0x7fd518074970 con 0x7fd5180a2b70 2024-09-15T21:19:05.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): _negotiate 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): my global_id is 14362 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd500003530 con 0x7fd5180a2b70 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 0x7fd51813e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): get_auth_request con 0x7fd5180a3ab0 auth_method 0 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 171584500 0 0) 0x7fd500003530 con 0x7fd5180a2b70 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 msgr2=0x7fd51813e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 0x7fd51813e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 >> v1:172.21.15.17:6789/0 conn(0x7fd5180a3660 legacy=0x7fd51813ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5180a4630 con 0x7fd5180a2b70 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fd510001700 con 0x7fd5180a2b70 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 5 monclient: authenticate success, global_id 14362 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: dump: 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:19:05.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 >> v1:172.21.15.119:6789/0 conn(0x7fd5180a2b70 legacy=0x7fd5180a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 0x7fd500003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): picked mon.c con 0x7fd5180a3ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd500004090 0x7fd500004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): picked mon.a con 0x7fd500004090 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd500005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): picked mon.b con 0x7fd500004da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd50000bc50 con 0x7fd500004090 2024-09-15T21:19:05.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd50000be00 con 0x7fd500004da0 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd50000c000 con 0x7fd5180a3ab0 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51d487640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd5206ed640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd500005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): get_auth_request con 0x7fd500004da0 auth_method 0 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:05.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51dc88640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd500004090 0x7fd500004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 0x7fd500003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51dc88640 10 monclient(hunting): get_auth_request con 0x7fd500004090 auth_method 0 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51dc88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51dc88640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51dc88640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 10 monclient(hunting): get_auth_request con 0x7fd5180a3ab0 auth_method 0 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51e489640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient(hunting): handle_auth_done global_id 14362 payload 293 2024-09-15T21:19:05.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 -- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 msgr2=0x7fd500003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd5180a3ab0 0x7fd500003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 -- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd500004090 msgr2=0x7fd500004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd500004090 0x7fd500004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 monclient: found mon.b 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 1 -- 172.21.15.119:0/3343150444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd500003530 con 0x7fd500004da0 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.345+0000 7fd51ec8a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51ec8a640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd500005220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd50800ba30 tx=0x7fd50801c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78786c2e722cd4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50800b600 con 0x7fd500004da0 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: dump: 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.372 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5080b1070 con 0x7fd500004da0 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 1 -- 172.21.15.119:0/3343150444 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080ac860 con 0x7fd500004da0 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 4 set_mon_vals no callback set 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51d487640 10 monclient: dump: 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.373 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 10 monclient: shutdown 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 -- 172.21.15.119:0/3343150444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 msgr2=0x7fd500005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 --2- 172.21.15.119:0/3343150444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd500005220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd50800ba30 tx=0x7fd50801c680 comp rx=0 tx=0).stop 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd51ec8a640 1 -- 172.21.15.119:0/3343150444 reap_dead start 2024-09-15T21:19:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 -- 172.21.15.119:0/3343150444 shutdown_connections 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 -- 172.21.15.119:0/3343150444 >> 172.21.15.119:0/3343150444 conn(0x7fd5180a8170 msgr2=0x7fd5180a8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 -- 172.21.15.119:0/3343150444 shutdown_connections 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 -- 172.21.15.119:0/3343150444 wait complete. 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 10 monclient: build_initial_monmap 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 10 monclient: monmap: 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.349+0000 7fd5206ed640 1 Processor -- start 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- start start 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: init 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd51813afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): picked mon.noname-b con 0x7fd500004da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd51806ec60 0x7fd51806f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): picked mon.noname-c con 0x7fd51806ec60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd51806f600 0x7fd5181d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): picked mon.noname-a con 0x7fd51806f600 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd51813ec30 con 0x7fd51806f600 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd51813eab0 con 0x7fd500004da0 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd51813edb0 con 0x7fd51806ec60 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient(hunting): authenticate will time out at 1064.152634s 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd51813afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd51813afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52878/0 (socket says 172.21.15.119:52878) 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 -- 172.21.15.119:0/669238334 learned_addr learned my addr 172.21.15.119:0/669238334 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): get_auth_request con 0x7fd500004da0 auth_method 0 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51ec8a640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd51806f600 0x7fd5181d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51ec8a640 10 monclient(hunting): get_auth_request con 0x7fd51806f600 auth_method 0 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51ec8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51ec8a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51ec8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51dc88640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd51806ec60 0x7fd51806f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient(hunting): handle_auth_done global_id 14365 payload 824 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 -- 172.21.15.119:0/669238334 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd51806ec60 msgr2=0x7fd51806f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd51806ec60 0x7fd51806f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 -- 172.21.15.119:0/669238334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd51806f600 msgr2=0x7fd5181d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd51806f600 0x7fd5181d5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 monclient: found mon.noname-b 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 -- 172.21.15.119:0/669238334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51813a470 con 0x7fd500004da0 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51e489640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd500004da0 0x7fd51813afc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd51000e8c0 tx=0x7fd51000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510019070 con 0x7fd500004da0 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: dump: 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 5 monclient: authenticate success, global_id 14365 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51000c8d0 con 0x7fd500004da0 2024-09-15T21:19:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _renew_subs 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- 172.21.15.119:0/669238334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd518070a40 con 0x7fd500004da0 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000ccc0 con 0x7fd500004da0 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 10 monclient: dump: 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51cc86640 4 set_mon_vals no callback set 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _renew_subs 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- 172.21.15.119:0/669238334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51813a750 con 0x7fd500004da0 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd5206ed640 1 -- 172.21.15.119:0/669238334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd518071710 con 0x7fd500004da0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd51001c070 con 0x7fd500004da0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd50403b9c0 0x7fd50403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7fd51004f660 con 0x7fd500004da0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51dc88640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd50403b9c0 0x7fd50403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51dc88640 10 monclient: get_auth_request con 0x7fd50403b9c0 auth_method 0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.353+0000 7fd51dc88640 1 --2- 172.21.15.119:0/669238334 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd50403b9c0 0x7fd50403de80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000f90 tx=0x7fd50c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.357+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd51001db10 con 0x7fd500004da0 2024-09-15T21:19:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.357+0000 7fd516ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.357+0000 7fd516ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:05.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a4134fe0 auth_method 0 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:05.498+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:05.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.529+0000 7fd5206ed640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-09-15T21:19:05.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.529+0000 7fd5206ed640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-09-15T21:19:05.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.529+0000 7fd5206ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.529+0000 7fd5206ed640 1 -- 172.21.15.119:0/669238334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"} v 0) -- 0x7fd5180a5710 con 0x7fd500004da0 2024-09-15T21:19:05.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.533+0000 7fd516ffd640 1 -- 172.21.15.119:0/669238334 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}]=-22 pool size must be between 1 and 10 v39) ==== 136+0+0 (secure 0 0 0) 0x7fd51001c360 con 0x7fd500004da0 2024-09-15T21:19:05.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.533+0000 7fd516ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-09-15T21:19:05.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.533+0000 7fd516ffd640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-09-15T21:19:05.539 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: pool size must be between 1 and 10 2024-09-15T21:19:05.543 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:05.543 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool set foo size 20 2024-09-15T21:19:05.543 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:05.543 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set foo size 20 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient: get_monmap_and_config 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient: build_initial_monmap 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient: monmap: 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:05.661046+0000 2024-09-15T21:19:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:05.661046+0000 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:05.663 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 Processor -- start 2024-09-15T21:19:05.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 -- start start 2024-09-15T21:19:05.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient: init 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f628409f780 0x7f628409fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): picked mon.noname-c con 0x7f628409f780 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f628409f070 0x7f62840a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): picked mon.noname-a con 0x7f628409f070 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6284137e30 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62840a1410 con 0x7f6284137e30 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f62840a48f0 con 0x7f628409f070 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f62840a4720 con 0x7f628409f780 2024-09-15T21:19:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f628b907640 10 monclient(hunting): authenticate will time out at 1064.457659s 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f628409f780 0x7f628409fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f628409f780 0x7f628409fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52880/0 (socket says 172.21.15.119:52880) 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 -- 172.21.15.119:0/3096863623 learned_addr learned my addr 172.21.15.119:0/3096863623 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): get_auth_request con 0x7f628409f780 auth_method 0 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f6288ea2640 1 --2- 172.21.15.119:0/3096863623 >> v2:172.21.15.17:3300/0 conn(0x7f628409f070 0x7f62840a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f6288ea2640 10 monclient(hunting): get_auth_request con 0x7f628409f070 auth_method 0 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f6288ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f6288ea2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f6288ea2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient(hunting): handle_auth_done global_id 14368 payload 274 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 -- 172.21.15.119:0/3096863623 >> v2:172.21.15.17:3300/0 conn(0x7f628409f070 msgr2=0x7f62840a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 --2- 172.21.15.119:0/3096863623 >> v2:172.21.15.17:3300/0 conn(0x7f628409f070 0x7f62840a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 -- 172.21.15.119:0/3096863623 >> v1:172.21.15.17:6789/0 conn(0x7f6284137e30 legacy=0x7f62841382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 monclient: found mon.noname-c 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.657+0000 7f62896a3640 1 -- 172.21.15.119:0/3096863623 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62840a16f0 con 0x7f628409f780 2024-09-15T21:19:05.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 --2- 172.21.15.119:0/3096863623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f780 0x7f628409fb60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6284076a40 tx=0x7f628001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110c0d5a98517ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 1 -- 172.21.15.119:0/3096863623 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b0070 con 0x7f628409f780 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: dump: 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.671 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 1 -- 172.21.15.119:0/3096863623 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62800ab600 con 0x7f628409f780 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 5 monclient: authenticate success, global_id 14368 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 1 -- 172.21.15.119:0/3096863623 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800ab9d0 con 0x7f628409f780 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 4 set_mon_vals no callback set 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f627bfff640 10 monclient: dump: 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: shutdown 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- 172.21.15.119:0/3096863623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f780 msgr2=0x7f628409fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 --2- 172.21.15.119:0/3096863623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f780 0x7f628409fb60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6284076a40 tx=0x7f628001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- 172.21.15.119:0/3096863623 shutdown_connections 2024-09-15T21:19:05.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 --2- 172.21.15.119:0/3096863623 >> v2:172.21.15.17:3300/0 conn(0x7f628409f070 0x7f62840a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 --2- 172.21.15.119:0/3096863623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f780 0x7f628409fb60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- 172.21.15.119:0/3096863623 >> 172.21.15.119:0/3096863623 conn(0x7f628406cbf0 msgr2=0x7f628406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:05.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- 172.21.15.119:0/3096863623 shutdown_connections 2024-09-15T21:19:05.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- 172.21.15.119:0/3096863623 wait complete. 2024-09-15T21:19:05.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: build_initial_monmap 2024-09-15T21:19:05.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: monmap: 2024-09-15T21:19:05.678 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 Processor -- start 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- start start 2024-09-15T21:19:05.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: init 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f070 0x7f6284139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): picked mon.noname-b con 0x7f628409f070 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628409f780 0x7f628413a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): picked mon.noname-a con 0x7f628409f780 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6284137e30 0x7f628413a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): picked mon.noname-c con 0x7f6284137e30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f62840a4070 con 0x7f628409f780 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62840a3ef0 con 0x7f628409f070 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f62841428b0 con 0x7f6284137e30 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f628b907640 10 monclient(hunting): authenticate will time out at 1064.463200s 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f070 0x7f6284139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f070 0x7f6284139dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52896/0 (socket says 172.21.15.119:52896) 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 -- 172.21.15.119:0/680565663 learned_addr learned my addr 172.21.15.119:0/680565663 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): get_auth_request con 0x7f628409f070 auth_method 0 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f6288ea2640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628409f780 0x7f628413a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f6288ea2640 10 monclient(hunting): get_auth_request con 0x7f628409f780 auth_method 0 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f6288ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f6288ea2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f6288ea2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient(hunting): handle_auth_done global_id 14371 payload 824 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f6289ea4640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6284137e30 0x7f628413a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 -- 172.21.15.119:0/680565663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6284137e30 msgr2=0x7f628413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6284137e30 0x7f628413a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 -- 172.21.15.119:0/680565663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628409f780 msgr2=0x7f628413a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628409f780 0x7f628413a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 monclient: found mon.noname-b 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 -- 172.21.15.119:0/680565663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628413b7f0 con 0x7f628409f070 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.661+0000 7f62896a3640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628409f070 0x7f6284139dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f628001aa90 tx=0x7f6280019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b9070 con 0x7f628409f070 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: dump: 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.683 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62800b4400 con 0x7f628409f070 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 5 monclient: authenticate success, global_id 14371 2024-09-15T21:19:05.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 monclient: _renew_subs 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 1 -- 172.21.15.119:0/680565663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62840a7b60 con 0x7f628409f070 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b0040 con 0x7f628409f070 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 10 monclient: dump: 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.685 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 monclient: _renew_subs 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f627b7fe640 4 set_mon_vals no callback set 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 1 -- 172.21.15.119:0/680565663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628413d2b0 con 0x7f628409f070 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f62800bc070 con 0x7f628409f070 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f628b907640 1 -- 172.21.15.119:0/680565663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62841382a0 con 0x7f628409f070 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f626803b9c0 0x7f626803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7f62800e9140 con 0x7f628409f070 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6288ea2640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f626803b9c0 0x7f626803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6288ea2640 10 monclient: get_auth_request con 0x7f626803b9c0 auth_method 0 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.665+0000 7f6288ea2640 1 --2- 172.21.15.119:0/680565663 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f626803b9c0 0x7f626803de80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f628413c2e0 tx=0x7f6274008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.669+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f62800160f0 con 0x7f628409f070 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.669+0000 7f6279ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.669+0000 7f6279ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.837+0000 7f628b907640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-09-15T21:19:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.837+0000 7f628b907640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-09-15T21:19:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.837+0000 7f628b907640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.837+0000 7f628b907640 1 -- 172.21.15.119:0/680565663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"} v 0) -- 0x7f628406a7f0 con 0x7f628409f070 2024-09-15T21:19:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.841+0000 7f6279ffb640 1 -- 172.21.15.119:0/680565663 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}]=-22 pool size must be between 1 and 10 v39) ==== 137+0+0 (secure 0 0 0) 0x7f62800bdb10 con 0x7f628409f070 2024-09-15T21:19:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.841+0000 7f6279ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-09-15T21:19:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.841+0000 7f6279ffb640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-09-15T21:19:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: pool size must be between 1 and 10 2024-09-15T21:19:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool delete foo 2024-09-15T21:19:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo 2024-09-15T21:19:05.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient: get_monmap_and_config 2024-09-15T21:19:05.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient: build_initial_monmap 2024-09-15T21:19:05.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient: monmap: 2024-09-15T21:19:05.971 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:05.971 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.971 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:05.970011+0000 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:05.970011+0000 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:05.972 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 Processor -- start 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 -- start start 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient: init 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc860135ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8600a0160 0x7fc86013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): picked mon.noname-b con 0x7fc8600a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc86006ec30 0x7fc86013cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): picked mon.noname-c con 0x7fc86006ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc86009cf30 con 0x7fc860135ca0 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc8600a1ca0 con 0x7fc86006ec30 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600a1f80 con 0x7fc8600a0160 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc865394640 10 monclient(hunting): authenticate will time out at 1064.766811s 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc86006ec30 0x7fc86013cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85effd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc860135ca0 0x7fc860136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47644/0 (socket says 172.21.15.119:47644) 2024-09-15T21:19:05.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85effd640 1 -- 172.21.15.119:0/366423868 learned_addr learned my addr 172.21.15.119:0/366423868 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): get_auth_request con 0x7fc86006ec30 auth_method 0 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85e7fc640 1 --2- 172.21.15.119:0/366423868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8600a0160 0x7fc86013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85e7fc640 10 monclient(hunting): get_auth_request con 0x7fc8600a0160 auth_method 0 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient(hunting): handle_auth_done global_id 14374 payload 274 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 1 -- 172.21.15.119:0/366423868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8600a0160 msgr2=0x7fc86013aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 1 --2- 172.21.15.119:0/366423868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8600a0160 0x7fc86013aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 1 -- 172.21.15.119:0/366423868 >> v1:172.21.15.119:6789/0 conn(0x7fc860135ca0 legacy=0x7fc860136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85f7fe640 1 -- 172.21.15.119:0/366423868 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600a3860 con 0x7fc86006ec30 2024-09-15T21:19:05.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85dffb640 1 -- 172.21.15.119:0/366423868 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673364515 0 0) 0x7fc86009cf30 con 0x7fc860135ca0 2024-09-15T21:19:05.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.965+0000 7fc85dffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:05.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85f7fe640 1 --2- 172.21.15.119:0/366423868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86006ec30 0x7fc86013cf50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8500097c0 tx=0x7fc85001c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77dade05a4e302ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 1 -- 172.21.15.119:0/366423868 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500b0070 con 0x7fc86006ec30 2024-09-15T21:19:05.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: dump: 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 5 monclient: authenticate success, global_id 14374 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 1 -- 172.21.15.119:0/366423868 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8500ab600 con 0x7fc86006ec30 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 4 set_mon_vals no callback set 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 1 -- 172.21.15.119:0/366423868 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500ab9d0 con 0x7fc86006ec30 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc85dffb640 10 monclient: dump: 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.987 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 10 monclient: shutdown 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:05.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 -- 172.21.15.119:0/366423868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86006ec30 msgr2=0x7fc86013cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 --2- 172.21.15.119:0/366423868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86006ec30 0x7fc86013cf50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8500097c0 tx=0x7fc85001c360 comp rx=0 tx=0).stop 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 -- 172.21.15.119:0/366423868 shutdown_connections 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 --2- 172.21.15.119:0/366423868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86006ec30 0x7fc86013cf50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 --2- 172.21.15.119:0/366423868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8600a0160 0x7fc86013aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 -- 172.21.15.119:0/366423868 >> 172.21.15.119:0/366423868 conn(0x7fc8600a2ae0 msgr2=0x7fc8600a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 -- 172.21.15.119:0/366423868 shutdown_connections 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 -- 172.21.15.119:0/366423868 wait complete. 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 10 monclient: build_initial_monmap 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 10 monclient: monmap: 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:05.989 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 Processor -- start 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 1 -- start start 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.969+0000 7fc865394640 10 monclient: init 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86006ec30 0x7fc8601cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): picked mon.noname-a con 0x7fc86006ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:05.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8600a0160 0x7fc8601cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): picked mon.noname-b con 0x7fc8600a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc860135ca0 0x7fc8601cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): picked mon.noname-c con 0x7fc860135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc86013fe30 con 0x7fc86006ec30 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc86013fcb0 con 0x7fc8600a0160 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc86013ffb0 con 0x7fc860135ca0 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient(hunting): authenticate will time out at 1064.772229s 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8600a0160 0x7fc8601cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8600a0160 0x7fc8601cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52926/0 (socket says 172.21.15.119:52926) 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 -- 172.21.15.119:0/2805598048 learned_addr learned my addr 172.21.15.119:0/2805598048 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): get_auth_request con 0x7fc8600a0160 auth_method 0 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85effd640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86006ec30 0x7fc8601cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85f7fe640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc860135ca0 0x7fc8601cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient(hunting): handle_auth_done global_id 14380 payload 824 2024-09-15T21:19:05.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 -- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc860135ca0 msgr2=0x7fc8601cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc860135ca0 0x7fc8601cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 -- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86006ec30 msgr2=0x7fc8601cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86006ec30 0x7fc8601cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 monclient: found mon.noname-b 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 -- 172.21.15.119:0/2805598048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600a5810 con 0x7fc8600a0160 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85f7fe640 10 monclient: get_auth_request con 0x7fc860135ca0 auth_method 0 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85effd640 10 monclient: get_auth_request con 0x7fc86006ec30 auth_method 0 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85f7fe640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc860135ca0 0x7fc8601cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85effd640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86006ec30 0x7fc8601cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:05.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85e7fc640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8600a0160 0x7fc8601cd6e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc84800d860 tx=0x7fc84800dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84800de50 con 0x7fc8600a0160 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: dump: 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.994 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848019070 con 0x7fc8600a0160 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 5 monclient: authenticate success, global_id 14380 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _renew_subs 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 -- 172.21.15.119:0/2805598048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86013a540 con 0x7fc8600a0160 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84800bad0 con 0x7fc8600a0160 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:05.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 10 monclient: dump: 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85d7fa640 4 set_mon_vals no callback set 2024-09-15T21:19:05.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _renew_subs 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 -- 172.21.15.119:0/2805598048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc860073f80 con 0x7fc8600a0160 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc865394640 1 -- 172.21.15.119:0/2805598048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc860136080 con 0x7fc8600a0160 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc84800bc70 con 0x7fc8600a0160 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc84003b9c0 0x7fc84003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7fc848050260 con 0x7fc8600a0160 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85effd640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc84003b9c0 0x7fc84003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85effd640 10 monclient: get_auth_request con 0x7fc84003b9c0 auth_method 0 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.973+0000 7fc85effd640 1 --2- 172.21.15.119:0/2805598048 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc84003b9c0 0x7fc84003de80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc858007fd0 tx=0x7fc858009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.977+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc84801cd70 con 0x7fc8600a0160 2024-09-15T21:19:05.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.977+0000 7fc8577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:05.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:05.977+0000 7fc8577fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:06.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.149+0000 7fc865394640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo"}] 2024-09-15T21:19:06.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.149+0000 7fc865394640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-09-15T21:19:06.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.149+0000 7fc865394640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.149+0000 7fc865394640 1 -- 172.21.15.119:0/2805598048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo"} v 0) -- 0x7fc86009f8f0 con 0x7fc8600a0160 2024-09-15T21:19:06.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.153+0000 7fc8577fe640 1 -- 172.21.15.119:0/2805598048 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v39) ==== 274+0+0 (secure 0 0 0) 0x7fc848013cb0 con 0x7fc8600a0160 2024-09-15T21:19:06.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.153+0000 7fc8577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-09-15T21:19:06.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.153+0000 7fc8577fe640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T21:19:06.159 INFO:tasks.workunit.client.0.smithi119.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T21:19:06.163 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:06.163 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool delete foo foo 2024-09-15T21:19:06.164 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:06.164 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo foo 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.269+0000 7fdc0fed6640 10 monclient: get_monmap_and_config 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.269+0000 7fdc0fed6640 10 monclient: build_initial_monmap 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.269+0000 7fdc0fed6640 10 monclient: monmap: 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:06.275873+0000 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:06.275873+0000 2024-09-15T21:19:06.277 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:06.278 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 Processor -- start 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 -- start start 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient: init 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): picked mon.noname-b con 0x7fdc080a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdc080a46b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdc080a1100 0x7fdc08137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): picked mon.noname-a con 0x7fdc080a1100 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc08135180 con 0x7fdc080a46b0 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fdc080a7dc0 con 0x7fdc080a1100 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fdc080a6920 con 0x7fdc080a79e0 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0fed6640 10 monclient(hunting): authenticate will time out at 1065.073026s 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdc080a1100 0x7fdc08137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:58946/0 (socket says 172.21.15.119:58946) 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0d471640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdc080a46b0 0x7fdc081377e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:47980/0 (socket says 172.21.15.119:47980) 2024-09-15T21:19:06.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 -- 172.21.15.119:0/1945979586 learned_addr learned my addr 172.21.15.119:0/1945979586 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 10 monclient(hunting): get_auth_request con 0x7fdc080a79e0 auth_method 0 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): get_auth_request con 0x7fdc080a1100 auth_method 0 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:06.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 -- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 msgr2=0x7fdc080a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 -- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 msgr2=0x7fdc080a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0dc72640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient(hunting): handle_auth_done global_id 4647 payload 274 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 1 -- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 msgr2=0x7fdc080a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 1 -- 172.21.15.119:0/1945979586 >> v1:172.21.15.17:6789/0 conn(0x7fdc080a46b0 legacy=0x7fdc081377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient: _finish_hunting 0 2024-09-15T21:19:06.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 1 monclient: found mon.noname-a 2024-09-15T21:19:06.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:06.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:06.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.273+0000 7fdc0e473640 1 -- 172.21.15.119:0/1945979586 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc080a1c30 con 0x7fdc080a1100 2024-09-15T21:19:06.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0e473640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a1100 0x7fdc08137ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009d90 tx=0x7fdc0001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78839fb6b0c893c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 1 -- 172.21.15.119:0/1945979586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000b2070 con 0x7fdc080a1100 2024-09-15T21:19:06.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:06.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: dump: 2024-09-15T21:19:06.290 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: _finish_auth 0 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: _check_auth_tickets 2024-09-15T21:19:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 1 -- 172.21.15.119:0/1945979586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc000a59a0 con 0x7fdc080a1100 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 1 -- 172.21.15.119:0/1945979586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08134930 con 0x7fdc080a1100 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0cc70640 10 monclient: dump: 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 5 monclient: authenticate success, global_id 4647 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 4 set_mon_vals no callback set 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 10 monclient: shutdown 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 -- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a1100 msgr2=0x7fdc08137ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a1100 0x7fdc08137ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009d90 tx=0x7fdc0001c8c0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 -- 172.21.15.119:0/1945979586 shutdown_connections 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a1100 0x7fdc08137ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 --2- 172.21.15.119:0/1945979586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc080a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 -- 172.21.15.119:0/1945979586 >> 172.21.15.119:0/1945979586 conn(0x7fdc080a6ef0 msgr2=0x7fdc080a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 -- 172.21.15.119:0/1945979586 shutdown_connections 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 -- 172.21.15.119:0/1945979586 wait complete. 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 10 monclient: build_initial_monmap 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 10 monclient: monmap: 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.294 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 Processor -- start 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 1 -- start start 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.277+0000 7fdc0fed6640 10 monclient: init 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc080a1100 0x7fdc081310f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): picked mon.noname-b con 0x7fdc080a1100 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a46b0 0x7fdc08131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): picked mon.noname-a con 0x7fdc080a46b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:06.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc08072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): picked mon.noname-c con 0x7fdc080a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08138670 con 0x7fdc080a46b0 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081384f0 con 0x7fdc080a1100 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fdc081423a0 con 0x7fdc080a79e0 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient(hunting): authenticate will time out at 1065.080193s 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc080a1100 0x7fdc081310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc080a1100 0x7fdc081310f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52942/0 (socket says 172.21.15.119:52942) 2024-09-15T21:19:06.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 -- 172.21.15.119:0/3764171256 learned_addr learned my addr 172.21.15.119:0/3764171256 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): get_auth_request con 0x7fdc080a1100 auth_method 0 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a46b0 0x7fdc08131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0e473640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc08072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 10 monclient(hunting): get_auth_request con 0x7fdc080a46b0 auth_method 0 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0e473640 10 monclient(hunting): get_auth_request con 0x7fdc080a79e0 auth_method 0 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0e473640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0e473640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0e473640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient(hunting): handle_auth_done global_id 14383 payload 824 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 -- 172.21.15.119:0/3764171256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 msgr2=0x7fdc08072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc080a79e0 0x7fdc08072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 -- 172.21.15.119:0/3764171256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a46b0 msgr2=0x7fdc08131630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc080a46b0 0x7fdc08131630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient: _finish_hunting 0 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 monclient: found mon.noname-b 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:06.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 -- 172.21.15.119:0/3764171256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0813a250 con 0x7fdc080a1100 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0dc72640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc080a1100 0x7fdc081310f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400e840 tx=0x7fdc0400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04017070 con 0x7fdc080a1100 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: dump: 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.299 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc04004020 con 0x7fdc080a1100 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 5 monclient: authenticate success, global_id 14383 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _renew_subs 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 -- 172.21.15.119:0/3764171256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc080742e0 con 0x7fdc080a1100 2024-09-15T21:19:06.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400f040 con 0x7fdc080a1100 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 10 monclient: dump: 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.301 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfffff640 4 set_mon_vals no callback set 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _renew_subs 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 -- 172.21.15.119:0/3764171256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc08138a50 con 0x7fdc080a1100 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0fed6640 1 -- 172.21.15.119:0/3764171256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc080a4170 con 0x7fdc080a1100 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fdc0400c6d0 con 0x7fdc080a1100 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdbec03b9c0 0x7fdbec03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7fdc0404fa20 con 0x7fdc080a1100 2024-09-15T21:19:06.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdbec03b9c0 0x7fdbec03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 10 monclient: get_auth_request con 0x7fdbec03b9c0 auth_method 0 2024-09-15T21:19:06.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.281+0000 7fdc0d471640 1 --2- 172.21.15.119:0/3764171256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdbec03b9c0 0x7fdbec03de80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc0813f930 tx=0x7fdbf8007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.285+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fdc04004410 con 0x7fdc080a1100 2024-09-15T21:19:06.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.285+0000 7fdbfe7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.285+0000 7fdbfe7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:06.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a4134fe0 auth_method 0 2024-09-15T21:19:06.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:06.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:06.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:06.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:06.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:06.302+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:19:06.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdc0fed6640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-09-15T21:19:06.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdc0fed6640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-09-15T21:19:06.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdc0fed6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdc0fed6640 1 -- 172.21.15.119:0/3764171256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"} v 0) -- 0x7fdc08139e40 con 0x7fdc080a1100 2024-09-15T21:19:06.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdbfe7fc640 1 -- 172.21.15.119:0/3764171256 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v39) ==== 290+0+0 (secure 0 0 0) 0x7fdc04003620 con 0x7fdc080a1100 2024-09-15T21:19:06.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdbfe7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-09-15T21:19:06.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.449+0000 7fdbfe7fc640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T21:19:06.457 INFO:tasks.workunit.client.0.smithi119.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T21:19:06.460 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:06.460 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool delete foo foo --force 2024-09-15T21:19:06.461 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:06.461 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo foo --force 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient: get_monmap_and_config 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient: build_initial_monmap 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient: monmap: 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:06.577697+0000 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:06.577697+0000 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.579 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:06.580 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:06.580 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:06.580 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:06.580 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:06.580 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 Processor -- start 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 -- start start 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient: init 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d3c135bf0 0x7f6d3c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): picked mon.noname-c con 0x7f6d3c135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:06.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6d3c135fd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 0x7f6d3c0a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): picked mon.noname-b con 0x7f6d3c136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3c0a1d80 con 0x7f6d3c135fd0 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c0a3600 con 0x7f6d3c135bf0 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c0a2060 con 0x7f6d3c136460 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d410e1640 10 monclient(hunting): authenticate will time out at 1065.374740s 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d3c135bf0 0x7f6d3c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d3c135bf0 0x7f6d3c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52948/0 (socket says 172.21.15.119:52948) 2024-09-15T21:19:06.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/1306797437 learned_addr learned my addr 172.21.15.119:0/1306797437 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): get_auth_request con 0x7f6d3c135bf0 auth_method 0 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_done global_id 14386 payload 274 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/1306797437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 msgr2=0x7f6d3c0a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 --2- 172.21.15.119:0/1306797437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 0x7f6d3c0a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/1306797437 >> v1:172.21.15.119:6789/0 conn(0x7f6d3c135fd0 legacy=0x7f6d3c0a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:06.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 monclient: found mon.noname-c 2024-09-15T21:19:06.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:06.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.573+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/1306797437 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c0a2340 con 0x7f6d3c135bf0 2024-09-15T21:19:06.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d3ad9d640 1 --2- 172.21.15.119:0/1306797437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c09fe70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0098a0 tx=0x7f6d2c01c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fcab357ea604ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 1 -- 172.21.15.119:0/1306797437 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0b0070 con 0x7f6d3c135bf0 2024-09-15T21:19:06.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:06.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: dump: 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: _finish_auth 0 2024-09-15T21:19:06.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 1 -- 172.21.15.119:0/1306797437 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2c0ab600 con 0x7f6d3c135bf0 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 1 -- 172.21.15.119:0/1306797437 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0ab9d0 con 0x7f6d3c135bf0 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d39d9b640 10 monclient: dump: 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.591 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 5 monclient: authenticate success, global_id 14386 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:06.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 4 set_mon_vals no callback set 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 10 monclient: shutdown 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 -- 172.21.15.119:0/1306797437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 msgr2=0x7f6d3c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 --2- 172.21.15.119:0/1306797437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c09fe70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0098a0 tx=0x7f6d2c01c490 comp rx=0 tx=0).stop 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 -- 172.21.15.119:0/1306797437 shutdown_connections 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 --2- 172.21.15.119:0/1306797437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 0x7f6d3c0a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 --2- 172.21.15.119:0/1306797437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c09fe70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 -- 172.21.15.119:0/1306797437 >> 172.21.15.119:0/1306797437 conn(0x7f6d3c0a66e0 msgr2=0x7f6d3c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 -- 172.21.15.119:0/1306797437 shutdown_connections 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 -- 172.21.15.119:0/1306797437 wait complete. 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 10 monclient: build_initial_monmap 2024-09-15T21:19:06.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 10 monclient: monmap: 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 Processor -- start 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 1 -- start start 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.577+0000 7f6d410e1640 10 monclient: init 2024-09-15T21:19:06.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c11da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): picked mon.noname-b con 0x7f6d3c135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d3c135fd0 0x7f6d3c11ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): picked mon.noname-a con 0x7f6d3c135fd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 0x7f6d3c1cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): picked mon.noname-c con 0x7f6d3c136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c0a1820 con 0x7f6d3c135fd0 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c0a16a0 con 0x7f6d3c135bf0 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c1402b0 con 0x7f6d3c136460 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient(hunting): authenticate will time out at 1065.380240s 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c11da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c11da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52954/0 (socket says 172.21.15.119:52954) 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/298052229 learned_addr learned my addr 172.21.15.119:0/298052229 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): get_auth_request con 0x7f6d3c135bf0 auth_method 0 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:06.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3b59e640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 0x7f6d3c1cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient(hunting): handle_auth_done global_id 14392 payload 824 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3a59c640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d3c135fd0 0x7f6d3c11ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/298052229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 msgr2=0x7f6d3c1cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d3c136460 0x7f6d3c1cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/298052229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d3c135fd0 msgr2=0x7f6d3c11ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d3c135fd0 0x7f6d3c11ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 -- 172.21.15.119:0/298052229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c13d720 con 0x7f6d3c135bf0 2024-09-15T21:19:06.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3ad9d640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3c135bf0 0x7f6d3c11da80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0ae040 tx=0x7f6d2c01acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 1 -- 172.21.15.119:0/298052229 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0ba020 con 0x7f6d3c135bf0 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: dump: 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.598 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 1 -- 172.21.15.119:0/298052229 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2c0b6640 con 0x7f6d3c135bf0 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 5 monclient: authenticate success, global_id 14392 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _renew_subs 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 -- 172.21.15.119:0/298052229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c072b30 con 0x7f6d3c135bf0 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 1 -- 172.21.15.119:0/298052229 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0b0040 con 0x7f6d3c135bf0 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 10 monclient: dump: 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3959a640 4 set_mon_vals no callback set 2024-09-15T21:19:06.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _renew_subs 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 -- 172.21.15.119:0/298052229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c11ecc0 con 0x7f6d3c135bf0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 1 -- 172.21.15.119:0/298052229 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6d2c0bd020 con 0x7f6d3c135bf0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d410e1640 1 -- 172.21.15.119:0/298052229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c09fe70 con 0x7f6d3c135bf0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d2803b9c0 0x7f6d2803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d237fe640 1 -- 172.21.15.119:0/298052229 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7f6d2c0ea1a0 con 0x7f6d3c135bf0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3a59c640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d2803b9c0 0x7f6d2803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3a59c640 10 monclient: get_auth_request con 0x7f6d2803b9c0 auth_method 0 2024-09-15T21:19:06.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.581+0000 7f6d3a59c640 1 --2- 172.21.15.119:0/298052229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d2803b9c0 0x7f6d2803de80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c13d9a0 tx=0x7f6d30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.585+0000 7f6d237fe640 1 -- 172.21.15.119:0/298052229 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6d2c0c4030 con 0x7f6d3c135bf0 2024-09-15T21:19:06.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.585+0000 7f6d237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.585+0000 7f6d237fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:06.766 INFO:tasks.workunit.client.0.smithi119.stderr:--force not valid: --force not one of 'true', 'false' 2024-09-15T21:19:06.766 INFO:tasks.workunit.client.0.smithi119.stderr:Invalid command: unused arguments: ['--force'] 2024-09-15T21:19:06.766 INFO:tasks.workunit.client.0.smithi119.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-09-15T21:19:06.767 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: invalid command 2024-09-15T21:19:06.771 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:06.771 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-09-15T21:19:06.771 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:06.771 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-09-15T21:19:06.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient: get_monmap_and_config 2024-09-15T21:19:06.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient: build_initial_monmap 2024-09-15T21:19:06.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient: monmap: 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:06.874170+0000 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:06.874170+0000 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:06.876 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:06.877 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:06.877 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 Processor -- start 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 -- start start 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient: init 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56d4135bf0 0x7f56d409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): picked mon.noname-c con 0x7f56d4135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135fd0 0x7f56d40a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): picked mon.noname-b con 0x7f56d4135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f56d4136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:06.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d40a1d80 con 0x7f56d4136460 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f56d40a3600 con 0x7f56d4135bf0 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f56d40a2060 con 0x7f56d4135fd0 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56da163640 10 monclient(hunting): authenticate will time out at 1065.670885s 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56d4135bf0 0x7f56d409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f56d4136460 0x7f56d40a0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47658/0 (socket says 172.21.15.119:47658) 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d3fff640 1 -- 172.21.15.119:0/603179261 learned_addr learned my addr 172.21.15.119:0/603179261 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:06.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): get_auth_request con 0x7f56d4135bf0 auth_method 0 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient(hunting): handle_auth_done global_id 14395 payload 274 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d2ffd640 1 --2- 172.21.15.119:0/603179261 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135fd0 0x7f56d40a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 1 -- 172.21.15.119:0/603179261 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135fd0 msgr2=0x7f56d40a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 1 --2- 172.21.15.119:0/603179261 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135fd0 0x7f56d40a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 1 -- 172.21.15.119:0/603179261 >> v1:172.21.15.119:6789/0 conn(0x7f56d4136460 legacy=0x7f56d40a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d27fc640 1 -- 172.21.15.119:0/603179261 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352131847 0 0) 0x7f56d40a1d80 con 0x7f56d4136460 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d37fe640 1 -- 172.21.15.119:0/603179261 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d40a2340 con 0x7f56d4135bf0 2024-09-15T21:19:06.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.869+0000 7f56d27fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:06.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d37fe640 1 --2- 172.21.15.119:0/603179261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135bf0 0x7f56d409fe70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009d90 tx=0x7f56cc01c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37fa4a4dc2602144 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 1 -- 172.21.15.119:0/603179261 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0a6700 con 0x7f56d4135bf0 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: dump: 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.887 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 5 monclient: authenticate success, global_id 14395 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 1 -- 172.21.15.119:0/603179261 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc0a68a0 con 0x7f56d4135bf0 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 4 set_mon_vals no callback set 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 1 -- 172.21.15.119:0/603179261 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0a6c70 con 0x7f56d4135bf0 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d27fc640 10 monclient: dump: 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.889 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: shutdown 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- 172.21.15.119:0/603179261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135bf0 msgr2=0x7f56d409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 --2- 172.21.15.119:0/603179261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135bf0 0x7f56d409fe70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009d90 tx=0x7f56cc01c830 comp rx=0 tx=0).stop 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- 172.21.15.119:0/603179261 shutdown_connections 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 --2- 172.21.15.119:0/603179261 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135fd0 0x7f56d40a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 --2- 172.21.15.119:0/603179261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135bf0 0x7f56d409fe70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- 172.21.15.119:0/603179261 >> 172.21.15.119:0/603179261 conn(0x7f56d40a66e0 msgr2=0x7f56d40a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- 172.21.15.119:0/603179261 shutdown_connections 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- 172.21.15.119:0/603179261 wait complete. 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: build_initial_monmap 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: monmap: 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.891 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 Processor -- start 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- start start 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: init 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135bf0 0x7f56d411c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): picked mon.noname-c con 0x7f56d4135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:06.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135fd0 0x7f56d411cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): picked mon.noname-b con 0x7f56d4135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56d4136460 0x7f56d411d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): picked mon.noname-a con 0x7f56d4136460 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40a1820 con 0x7f56d4136460 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40a16a0 con 0x7f56d4135fd0 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f56d41402b0 con 0x7f56d4135bf0 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56da163640 10 monclient(hunting): authenticate will time out at 1065.675676s 2024-09-15T21:19:06.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135fd0 0x7f56d411cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135fd0 0x7f56d411cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52968/0 (socket says 172.21.15.119:52968) 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 1 -- 172.21.15.119:0/190143497 learned_addr learned my addr 172.21.15.119:0/190143497 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): get_auth_request con 0x7f56d4135fd0 auth_method 0 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.873+0000 7f56d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 10 monclient(hunting): handle_auth_done global_id 14401 payload 824 2024-09-15T21:19:06.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d37fe640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135bf0 0x7f56d411c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 -- 172.21.15.119:0/190143497 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135bf0 msgr2=0x7f56d411c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56d4135bf0 0x7f56d411c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d3fff640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56d4136460 0x7f56d411d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 -- 172.21.15.119:0/190143497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56d4136460 msgr2=0x7f56d411d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56d4136460 0x7f56d411d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 monclient: found mon.noname-b 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 -- 172.21.15.119:0/190143497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d411ebe0 con 0x7f56d4135fd0 2024-09-15T21:19:06.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d2ffd640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d4135fd0 0x7f56d411cbe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c820 tx=0x7f56bc00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 1 -- 172.21.15.119:0/190143497 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc015070 con 0x7f56d4135fd0 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: dump: 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.896 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 1 -- 172.21.15.119:0/190143497 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc004240 con 0x7f56d4135fd0 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 5 monclient: authenticate success, global_id 14401 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 monclient: _renew_subs 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 1 -- 172.21.15.119:0/190143497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d409d240 con 0x7f56d4135fd0 2024-09-15T21:19:06.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 1 -- 172.21.15.119:0/190143497 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00ce10 con 0x7f56d4135fd0 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 10 monclient: dump: 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:06.898 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d1ffb640 4 set_mon_vals no callback set 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 monclient: _renew_subs 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 1 -- 172.21.15.119:0/190143497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d411e900 con 0x7f56d4135fd0 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56da163640 1 -- 172.21.15.119:0/190143497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d409fe70 con 0x7f56d4135fd0 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 1 -- 172.21.15.119:0/190143497 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f56bc007830 con 0x7f56d4135fd0 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56b403b9f0 0x7f56b403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:06.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56cbfff640 1 -- 172.21.15.119:0/190143497 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7f56bc04fd30 con 0x7f56d4135fd0 2024-09-15T21:19:06.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d37fe640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56b403b9f0 0x7f56b403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:06.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d37fe640 10 monclient: get_auth_request con 0x7f56b403b9f0 auth_method 0 2024-09-15T21:19:06.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.877+0000 7f56d37fe640 1 --2- 172.21.15.119:0/190143497 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56b403b9f0 0x7f56b403deb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0af000 tx=0x7f56cc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:06.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.881+0000 7f56cbfff640 1 -- 172.21.15.119:0/190143497 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f56bc003620 con 0x7f56d4135fd0 2024-09-15T21:19:06.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.881+0000 7f56cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:06.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:06.881+0000 7f56cbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:07.057 INFO:tasks.workunit.client.0.smithi119.stderr:--yes-i-really-mean-it not valid: --yes-i-really-mean-it not one of 'true', 'false' 2024-09-15T21:19:07.057 INFO:tasks.workunit.client.0.smithi119.stderr:Invalid command: unused arguments: ['--yes-i-really-mean-it'] 2024-09-15T21:19:07.057 INFO:tasks.workunit.client.0.smithi119.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-09-15T21:19:07.057 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: invalid command 2024-09-15T21:19:07.060 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:07.061 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd pool delete foo --yes-i-really-mean-it foo 2024-09-15T21:19:07.061 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:07.061 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo --yes-i-really-mean-it foo 2024-09-15T21:19:07.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient: get_monmap_and_config 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient: build_initial_monmap 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient: monmap: 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:07.174444+0000 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:07.174444+0000 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:07.176 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:07.177 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:07.177 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:07.177 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 1 Processor -- start 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 1 -- start start 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient: init 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2840a0a50 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:07.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff28409ea80 0x7ff284141420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient(hunting): picked mon.noname-c con 0x7ff28409ea80 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2840a1540 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2840741f0 con 0x7ff2840a1540 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2840a35a0 con 0x7ff2840a0a50 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff28409d800 con 0x7ff28409ea80 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff28a7fb640 10 monclient(hunting): authenticate will time out at 1065.971036s 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff28409ea80 0x7ff284141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff283fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff2840a0a50 0x7ff2840a0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47670/0 (socket says 172.21.15.119:47670) 2024-09-15T21:19:07.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff283fff640 1 -- 172.21.15.119:0/161040592 learned_addr learned my addr 172.21.15.119:0/161040592 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): get_auth_request con 0x7ff28409ea80 auth_method 0 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient(hunting): handle_auth_done global_id 14404 payload 274 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 1 -- 172.21.15.119:0/161040592 >> v1:172.21.15.119:6789/0 conn(0x7ff2840a0a50 legacy=0x7ff2840a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 1 -- 172.21.15.119:0/161040592 >> v1:172.21.15.17:6789/0 conn(0x7ff2840a1540 legacy=0x7ff284143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:07.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:07.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.169+0000 7ff2837fe640 1 -- 172.21.15.119:0/161040592 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28409dae0 con 0x7ff28409ea80 2024-09-15T21:19:07.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff2837fe640 1 --2- 172.21.15.119:0/161040592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff28409ea80 0x7ff284141420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff2740097f0 tx=0x7ff27401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7363e3b17865646a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 1 -- 172.21.15.119:0/161040592 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2740b0070 con 0x7ff28409ea80 2024-09-15T21:19:07.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: dump: 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.188 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 5 monclient: authenticate success, global_id 14404 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 1 -- 172.21.15.119:0/161040592 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff274004230 con 0x7ff28409ea80 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 4 set_mon_vals no callback set 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 1 -- 172.21.15.119:0/161040592 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274003620 con 0x7ff28409ea80 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff282ffd640 10 monclient: dump: 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.190 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 10 monclient: shutdown 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 -- 172.21.15.119:0/161040592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff28409ea80 msgr2=0x7ff284141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 --2- 172.21.15.119:0/161040592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff28409ea80 0x7ff284141420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff2740097f0 tx=0x7ff27401c330 comp rx=0 tx=0).stop 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 -- 172.21.15.119:0/161040592 shutdown_connections 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 --2- 172.21.15.119:0/161040592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff28409ea80 0x7ff284141420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 -- 172.21.15.119:0/161040592 >> 172.21.15.119:0/161040592 conn(0x7ff2840a66e0 msgr2=0x7ff2840a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 -- 172.21.15.119:0/161040592 shutdown_connections 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 1 -- 172.21.15.119:0/161040592 wait complete. 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 10 monclient: build_initial_monmap 2024-09-15T21:19:07.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.173+0000 7ff28a7fb640 10 monclient: monmap: 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 Processor -- start 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- start start 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: init 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28409ea80 0x7ff28413b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): picked mon.noname-a con 0x7ff28409ea80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840a0a50 0x7ff28413bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): picked mon.noname-b con 0x7ff2840a0a50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840a1540 0x7ff284140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): picked mon.noname-c con 0x7ff2840a1540 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:07.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841478b0 con 0x7ff28409ea80 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff28409cd60 con 0x7ff2840a0a50 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff284147a30 con 0x7ff2840a1540 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient(hunting): authenticate will time out at 1065.976835s 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840a0a50 0x7ff28413bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840a0a50 0x7ff28413bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52998/0 (socket says 172.21.15.119:52998) 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 -- 172.21.15.119:0/3444973922 learned_addr learned my addr 172.21.15.119:0/3444973922 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): get_auth_request con 0x7ff2840a0a50 auth_method 0 2024-09-15T21:19:07.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28409ea80 0x7ff28413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 10 monclient(hunting): get_auth_request con 0x7ff28409ea80 auth_method 0 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff288d98640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840a1540 0x7ff284140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient(hunting): handle_auth_done global_id 14410 payload 824 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 -- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840a1540 msgr2=0x7ff284140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840a1540 0x7ff284140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 -- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28409ea80 msgr2=0x7ff28413b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28409ea80 0x7ff28413b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 -- 172.21.15.119:0/3444973922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2841256c0 con 0x7ff2840a0a50 2024-09-15T21:19:07.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff288d98640 10 monclient: get_auth_request con 0x7ff2840a1540 auth_method 0 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff288d98640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840a1540 0x7ff284140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2837fe640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840a0a50 0x7ff28413bc50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2740ae040 tx=0x7ff2740190d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 1 -- 172.21.15.119:0/3444973922 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2740b9020 con 0x7ff2840a0a50 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: dump: 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.197 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 1 -- 172.21.15.119:0/3444973922 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2740043e0 con 0x7ff2840a0a50 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 1 -- 172.21.15.119:0/3444973922 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2740b0040 con 0x7ff2840a0a50 2024-09-15T21:19:07.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 10 monclient: dump: 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.199 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 5 monclient: authenticate success, global_id 14410 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _renew_subs 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff2827fc640 4 set_mon_vals no callback set 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- 172.21.15.119:0/3444973922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28409d240 con 0x7ff2840a0a50 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _renew_subs 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- 172.21.15.119:0/3444973922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff284143760 con 0x7ff2840a0a50 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff28a7fb640 1 -- 172.21.15.119:0/3444973922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2840a0e30 con 0x7ff2840a0a50 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 1 -- 172.21.15.119:0/3444973922 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff2740bc020 con 0x7ff2840a0a50 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff27003ba20 0x7ff27003dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff280ff9640 1 -- 172.21.15.119:0/3444973922 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7ff2740e9ff0 con 0x7ff2840a0a50 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff27003ba20 0x7ff27003dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 10 monclient: get_auth_request con 0x7ff27003ba20 auth_method 0 2024-09-15T21:19:07.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.177+0000 7ff283fff640 1 --2- 172.21.15.119:0/3444973922 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff27003ba20 0x7ff27003dee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff278007fd0 tx=0x7ff278009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.181+0000 7ff280ff9640 1 -- 172.21.15.119:0/3444973922 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff2740c3030 con 0x7ff2840a0a50 2024-09-15T21:19:07.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.181+0000 7ff280ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.181+0000 7ff280ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:07.361 INFO:tasks.workunit.client.0.smithi119.stderr:foo not valid: foo not one of 'true', 'false' 2024-09-15T21:19:07.361 INFO:tasks.workunit.client.0.smithi119.stderr:Invalid command: unused arguments: ['foo'] 2024-09-15T21:19:07.362 INFO:tasks.workunit.client.0.smithi119.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-09-15T21:19:07.362 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: invalid command 2024-09-15T21:19:07.365 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:07.365 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foooo foooo --yes-i-really-really-mean-it 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.473+0000 7ff697425640 10 monclient: get_monmap_and_config 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.473+0000 7ff697425640 10 monclient: build_initial_monmap 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.473+0000 7ff697425640 10 monclient: monmap: 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:07.480638+0000 2024-09-15T21:19:07.482 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:07.480638+0000 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:07.483 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 Processor -- start 2024-09-15T21:19:07.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 -- start start 2024-09-15T21:19:07.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient: init 2024-09-15T21:19:07.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:07.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2024-09-15T21:19:07.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff69009d550 0x7ff69009d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): picked mon.noname-c con 0x7ff69009d550 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff69009fae0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff690134f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): picked mon.noname-b con 0x7ff69009de70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6900a3600 con 0x7ff69009fae0 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff69009f1b0 con 0x7ff69009d550 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff6900a2920 con 0x7ff69009de70 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient(hunting): authenticate will time out at 1066.277244s 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff69009d550 0x7ff69009d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff69009d550 0x7ff69009d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53012/0 (socket says 172.21.15.119:53012) 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 -- 172.21.15.119:0/944179981 learned_addr learned my addr 172.21.15.119:0/944179981 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): get_auth_request con 0x7ff69009d550 auth_method 0 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient(hunting): handle_auth_done global_id 14413 payload 274 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 -- 172.21.15.119:0/944179981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 msgr2=0x7ff690134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 --2- 172.21.15.119:0/944179981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff690134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 -- 172.21.15.119:0/944179981 >> v1:172.21.15.17:6789/0 conn(0x7ff69009fae0 legacy=0x7ff690134820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient: _finish_hunting 0 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 monclient: found mon.noname-c 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 -- 172.21.15.119:0/944179981 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6900a1d60 con 0x7ff69009d550 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff6951c1640 1 --2- 172.21.15.119:0/944179981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009d550 0x7ff69009d930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff69009d010 tx=0x7ff68c01cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e720fe46b27d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 1 -- 172.21.15.119:0/944179981 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0b2070 con 0x7ff69009d550 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: dump: 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 1 -- 172.21.15.119:0/944179981 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c0a9070 con 0x7ff69009d550 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 1 -- 172.21.15.119:0/944179981 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0ad880 con 0x7ff69009d550 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 5 monclient: authenticate success, global_id 14413 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff697425640 4 set_mon_vals no callback set 2024-09-15T21:19:07.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.477+0000 7ff687fff640 10 monclient: dump: 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.498 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: shutdown 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/944179981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009d550 msgr2=0x7ff69009d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 --2- 172.21.15.119:0/944179981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009d550 0x7ff69009d930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff69009d010 tx=0x7ff68c01cde0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/944179981 shutdown_connections 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 --2- 172.21.15.119:0/944179981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff690134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 --2- 172.21.15.119:0/944179981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009d550 0x7ff69009d930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/944179981 >> 172.21.15.119:0/944179981 conn(0x7ff6900a66e0 msgr2=0x7ff6900a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/944179981 shutdown_connections 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/944179981 wait complete. 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: build_initial_monmap 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: monmap: 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 Processor -- start 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- start start 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: init 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d550 0x7ff6901cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): picked mon.noname-a con 0x7ff69009d550 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff6901d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): picked mon.noname-c con 0x7ff69009de70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:07.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 0x7ff6901376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): picked mon.noname-b con 0x7ff69009fae0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff690135470 con 0x7ff69009d550 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff69009cd80 con 0x7ff69009fae0 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff6901355f0 con 0x7ff69009de70 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient(hunting): authenticate will time out at 1066.281942s 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 0x7ff6901376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 0x7ff6901376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53028/0 (socket says 172.21.15.119:53028) 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 -- 172.21.15.119:0/2143188743 learned_addr learned my addr 172.21.15.119:0/2143188743 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:07.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): get_auth_request con 0x7ff69009fae0 auth_method 0 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6951c1640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d550 0x7ff6901cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6949c0640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff6901d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6949c0640 10 monclient(hunting): get_auth_request con 0x7ff69009de70 auth_method 0 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6949c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6949c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6949c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6951c1640 10 monclient(hunting): get_auth_request con 0x7ff69009d550 auth_method 0 2024-09-15T21:19:07.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6951c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6951c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6951c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient(hunting): handle_auth_done global_id 14416 payload 824 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 -- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 msgr2=0x7ff6901d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff6901d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 -- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d550 msgr2=0x7ff6901cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d550 0x7ff6901cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient: _finish_hunting 0 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 monclient: found mon.noname-b 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 -- 172.21.15.119:0/2143188743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69013af40 con 0x7ff69009fae0 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6959c2640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 0x7ff6901376e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b970 tx=0x7ff68800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688019070 con 0x7ff69009fae0 2024-09-15T21:19:07.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:07.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: dump: 2024-09-15T21:19:07.506 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.506 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.506 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.507 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.507 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.507 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.507 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68800cb60 con 0x7ff69009fae0 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 5 monclient: authenticate success, global_id 14416 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _renew_subs 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69013b220 con 0x7ff69009fae0 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:07.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688014440 con 0x7ff69009fae0 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 10 monclient: dump: 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.509 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _renew_subs 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6877fe640 4 set_mon_vals no callback set 2024-09-15T21:19:07.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6901316e0 con 0x7ff69009fae0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69009d930 con 0x7ff69009fae0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff68801c070 con 0x7ff69009fae0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67403b9c0 0x7ff67403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5474+0+0 (secure 0 0 0) 0x7ff68804fac0 con 0x7ff69009fae0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.481+0000 7ff6951c1640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67403b9c0 0x7ff67403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.485+0000 7ff6951c1640 10 monclient: get_auth_request con 0x7ff67403b9c0 auth_method 0 2024-09-15T21:19:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.485+0000 7ff6951c1640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67403b9c0 0x7ff67403de80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0af000 tx=0x7ff68c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.489+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff68801dd50 con 0x7ff69009fae0 2024-09-15T21:19:07.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.489+0000 7ff685ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.489+0000 7ff685ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.653+0000 7ff697425640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.653+0000 7ff697425640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.653+0000 7ff697425640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.653+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff69013b460 con 0x7ff69009fae0 2024-09-15T21:19:07.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.701+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' removed v40) ==== 155+0+0 (secure 0 0 0) 0x7ff68801bcd0 con 0x7ff69009fae0 2024-09-15T21:19:07.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.701+0000 7ff685ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:07.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.701+0000 7ff685ffb640 10 monclient: _finish_command 2 = system:0 pool 'foooo' removed 2024-09-15T21:19:07.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff697425640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:07.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff697425640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:07.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff697425640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff69006a830 con 0x7ff69009fae0 2024-09-15T21:19:07.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff685ffb640 1 -- 172.21.15.119:0/2143188743 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' does not exist v40) ==== 162+0+0 (secure 0 0 0) 0x7ff6880133a0 con 0x7ff69009fae0 2024-09-15T21:19:07.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff685ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:07.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.821+0000 7ff685ffb640 10 monclient: _finish_command 3 = system:0 pool 'foooo' does not exist 2024-09-15T21:19:07.828 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'foooo' does not exist 2024-09-15T21:19:07.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67403b9c0 msgr2=0x7ff67403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67403b9c0 0x7ff67403de80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0af000 tx=0x7ff68c004740 comp rx=0 tx=0).stop 2024-09-15T21:19:07.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 10 monclient: shutdown 2024-09-15T21:19:07.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:07.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 msgr2=0x7ff6901376e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 0x7ff6901376e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b970 tx=0x7ff68800be30 comp rx=0 tx=0).stop 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 shutdown_connections 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67403b9c0 0x7ff67403de80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009fae0 0x7ff6901376e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009de70 0x7ff6901d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 --2- 172.21.15.119:0/2143188743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d550 0x7ff6901cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 >> 172.21.15.119:0/2143188743 conn(0x7ff6900a66e0 msgr2=0x7ff6900a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 shutdown_connections 2024-09-15T21:19:07.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.825+0000 7ff697425640 1 -- 172.21.15.119:0/2143188743 wait complete. 2024-09-15T21:19:07.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-09-15T21:19:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.886+0000 7fc799ffb640 10 monclient: tick 2024-09-15T21:19:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.886+0000 7fc799ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.886+0000 7fc799ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.886+0000 7fc799ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a4134fe0 auth_method 0 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:07.906+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-15T21:19:07.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.949+0000 7fb45cec0640 10 monclient: get_monmap_and_config 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.949+0000 7fb45cec0640 10 monclient: build_initial_monmap 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.949+0000 7fb45cec0640 10 monclient: monmap: 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:07.956676+0000 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:07.956676+0000 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.958 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:07.959 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:07.959 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:07.959 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:07.959 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:07.959 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 Processor -- start 2024-09-15T21:19:07.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 -- start start 2024-09-15T21:19:07.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient: init 2024-09-15T21:19:07.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb458135bf0 0x7fb45809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): picked mon.noname-a con 0x7fb458135bf0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb458135fd0 0x7fb4580a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): picked mon.noname-c con 0x7fb458135fd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb458136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4580a1d80 con 0x7fb458136460 2024-09-15T21:19:07.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb4580a3600 con 0x7fb458135bf0 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb4580a2060 con 0x7fb458135fd0 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45cec0640 10 monclient(hunting): authenticate will time out at 1066.753118s 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb458135fd0 0x7fb4580a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb456d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb458136460 0x7fb4580a0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47684/0 (socket says 172.21.15.119:47684) 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb458135fd0 0x7fb4580a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53040/0 (socket says 172.21.15.119:53040) 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb456d9d640 1 -- 172.21.15.119:0/2784976403 learned_addr learned my addr 172.21.15.119:0/2784976403 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): get_auth_request con 0x7fb458135fd0 auth_method 0 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45659c640 1 --2- 172.21.15.119:0/2784976403 >> v2:172.21.15.17:3300/0 conn(0x7fb458135bf0 0x7fb45809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45659c640 10 monclient(hunting): get_auth_request con 0x7fb458135bf0 auth_method 0 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb45659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_done global_id 14419 payload 274 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2784976403 >> v2:172.21.15.17:3300/0 conn(0x7fb458135bf0 msgr2=0x7fb45809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 --2- 172.21.15.119:0/2784976403 >> v2:172.21.15.17:3300/0 conn(0x7fb458135bf0 0x7fb45809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2784976403 >> v1:172.21.15.119:6789/0 conn(0x7fb458136460 legacy=0x7fb4580a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 monclient: found mon.noname-c 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.953+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2784976403 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580a2340 con 0x7fb458135fd0 2024-09-15T21:19:07.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 --2- 172.21.15.119:0/2784976403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4580a0a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4400097c0 tx=0x7fb44001c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af05fc0ca7ddb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 1 -- 172.21.15.119:0/2784976403 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b0070 con 0x7fb458135fd0 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: dump: 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: _finish_auth 0 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 1 -- 172.21.15.119:0/2784976403 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4400ab600 con 0x7fb458135fd0 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 5 monclient: authenticate success, global_id 14419 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 1 -- 172.21.15.119:0/2784976403 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400ab9d0 con 0x7fb458135fd0 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 4 set_mon_vals no callback set 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb455d9b640 10 monclient: dump: 2024-09-15T21:19:07.972 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: shutdown 2024-09-15T21:19:07.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2784976403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 msgr2=0x7fb4580a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2784976403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4580a0a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4400097c0 tx=0x7fb44001c320 comp rx=0 tx=0).stop 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2784976403 shutdown_connections 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2784976403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4580a0a50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2784976403 >> v2:172.21.15.17:3300/0 conn(0x7fb458135bf0 0x7fb45809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2784976403 >> 172.21.15.119:0/2784976403 conn(0x7fb4580a66e0 msgr2=0x7fb4580a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2784976403 shutdown_connections 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2784976403 wait complete. 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: build_initial_monmap 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: monmap: 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:07.974 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 Processor -- start 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- start start 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: init 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb458135bf0 0x7fb45811da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): picked mon.noname-a con 0x7fb458135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4581348f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): picked mon.noname-b con 0x7fb458135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb458136460 0x7fb4581cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): picked mon.noname-c con 0x7fb458136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580a1850 con 0x7fb458135bf0 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580a16d0 con 0x7fb458135fd0 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb4581402b0 con 0x7fb458136460 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:07.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient(hunting): authenticate will time out at 1066.758696s 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4581348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4581348f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53052/0 (socket says 172.21.15.119:53052) 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2110377630 learned_addr learned my addr 172.21.15.119:0/2110377630 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): get_auth_request con 0x7fb458135fd0 auth_method 0 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:07.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45659c640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb458135bf0 0x7fb45811da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb456d9d640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb458136460 0x7fb4581cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb456d9d640 10 monclient(hunting): get_auth_request con 0x7fb458136460 auth_method 0 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb456d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb456d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb456d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45659c640 10 monclient(hunting): get_auth_request con 0x7fb458135bf0 auth_method 0 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient(hunting): handle_auth_done global_id 14425 payload 824 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb458136460 msgr2=0x7fb4581cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb458136460 0x7fb4581cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb458135bf0 msgr2=0x7fb45811da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb458135bf0 0x7fb45811da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 monclient: found mon.noname-b 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 -- 172.21.15.119:0/2110377630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb458072a20 con 0x7fb458135fd0 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44dd9b640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4581348f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb440019f50 tx=0x7fb44001ab60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b9070 con 0x7fb458135fd0 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: dump: 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.979 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 5 monclient: authenticate success, global_id 14425 2024-09-15T21:19:07.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4400b4420 con 0x7fb458135fd0 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: _renew_subs 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45811d620 con 0x7fb458135fd0 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b0040 con 0x7fb458135fd0 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:07.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb44f7fe640 10 monclient: dump: 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45559a640 4 set_mon_vals no callback set 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: _renew_subs 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.957+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45811e9e0 con 0x7fb458135fd0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45cec0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45cec0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45cec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45809fe70 con 0x7fb458135fd0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb4400bc070 con 0x7fb458135fd0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb44f7fe640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb43803b9c0 0x7fb43803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5130+0+0 (secure 0 0 0) 0x7fb4400e8fd0 con 0x7fb458135fd0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45659c640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb43803b9c0 0x7fb43803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45659c640 10 monclient: get_auth_request con 0x7fb43803b9c0 auth_method 0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.961+0000 7fb45659c640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb43803b9c0 0x7fb43803de80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b9d0 tx=0x7fb448008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.965+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb440016030 con 0x7fb458135fd0 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.965+0000 7fb44f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:07.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:07.965+0000 7fb44f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:08.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.133+0000 7fb45cec0640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:08.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.133+0000 7fb45cec0640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:08.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.133+0000 7fb45cec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:08.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.133+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb45806a830 con 0x7fb458135fd0 2024-09-15T21:19:08.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.701+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' removed v41) ==== 152+0+0 (secure 0 0 0) 0x7fb4400b3b70 con 0x7fb458135fd0 2024-09-15T21:19:08.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.701+0000 7fb44f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:08.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.701+0000 7fb44f7fe640 10 monclient: _finish_command 2 = system:0 pool 'fooo' removed 2024-09-15T21:19:08.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.821+0000 7fb45cec0640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:08.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.821+0000 7fb45cec0640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:08.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.821+0000 7fb45cec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:08.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.821+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb458135590 con 0x7fb458135fd0 2024-09-15T21:19:08.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb44f7fe640 1 -- 172.21.15.119:0/2110377630 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v41) ==== 159+0+0 (secure 0 0 0) 0x7fb4400bc360 con 0x7fb458135fd0 2024-09-15T21:19:08.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb44f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:08.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb44f7fe640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-09-15T21:19:08.831 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'fooo' does not exist 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb43803b9c0 msgr2=0x7fb43803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb43803b9c0 0x7fb43803de80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b9d0 tx=0x7fb448008040 comp rx=0 tx=0).stop 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 10 monclient: shutdown 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 msgr2=0x7fb4581348f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4581348f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb440019f50 tx=0x7fb44001ab60 comp rx=0 tx=0).stop 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 shutdown_connections 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb43803b9c0 0x7fb43803de80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:08.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb458136460 0x7fb4581cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:08.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb458135fd0 0x7fb4581348f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:08.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 --2- 172.21.15.119:0/2110377630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb458135bf0 0x7fb45811da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:08.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.825+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 >> 172.21.15.119:0/2110377630 conn(0x7fb4580a66e0 msgr2=0x7fb4580a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:08.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.829+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 shutdown_connections 2024-09-15T21:19:08.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.829+0000 7fb45cec0640 1 -- 172.21.15.119:0/2110377630 wait complete. 2024-09-15T21:19:08.849 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-09-15T21:19:08.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 10 monclient: tick 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 5 monclient: _check_tell_commands timeout tell command 1 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 msgr2=0x7fc7a402d450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a4134fe0 0x7fc7a402d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:08.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.886+0000 7fc799ffb640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40a4e70 con 0x7fc780003550 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc780003550 auth_method 0 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 msgr2=0x7fc7800059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 msgr2=0x7fc7800059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:08.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:08.890+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:08.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.977+0000 7fe28e52d640 10 monclient: get_monmap_and_config 2024-09-15T21:19:08.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.977+0000 7fe28e52d640 10 monclient: build_initial_monmap 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.977+0000 7fe28e52d640 10 monclient: monmap: 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:08.984484+0000 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:08.984484+0000 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:08.986 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 1 Processor -- start 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 1 -- start start 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient: init 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2880a7ac0 0x7fe288137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient(hunting): picked mon.noname-c con 0x7fe2880a7ac0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe2880a41b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:08.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe2880a45c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe288135120 con 0x7fe2880a45c0 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2880a19d0 con 0x7fe2880a41b0 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe2880a7ea0 con 0x7fe2880a7ac0 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe28e52d640 10 monclient(hunting): authenticate will time out at 1067.781174s 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2880a7ac0 0x7fe288137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe2877fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2880a41b0 0x7fe288134300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47696/0 (socket says 172.21.15.119:47696) 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe2877fe640 1 -- 172.21.15.119:0/1281627016 learned_addr learned my addr 172.21.15.119:0/1281627016 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): get_auth_request con 0x7fe2880a7ac0 auth_method 0 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:08.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe286ffd640 1 -- 172.21.15.119:0/1281627016 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311278073 0 0) 0x7fe2880a19d0 con 0x7fe2880a41b0 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe286ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe286ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe286ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe286ffd640 10 monclient(hunting): my global_id is 14431 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe286ffd640 1 -- 172.21.15.119:0/1281627016 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe270003530 con 0x7fe2880a41b0 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient(hunting): handle_auth_done global_id 14428 payload 274 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 1 -- 172.21.15.119:0/1281627016 >> v1:172.21.15.119:6789/0 conn(0x7fe2880a41b0 legacy=0x7fe288134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 1 -- 172.21.15.119:0/1281627016 >> v1:172.21.15.17:6789/0 conn(0x7fe2880a45c0 legacy=0x7fe288134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:08.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:08.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 1 monclient: found mon.noname-c 2024-09-15T21:19:08.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:08.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:08.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 1 -- 172.21.15.119:0/1281627016 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2880a6960 con 0x7fe2880a7ac0 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.981+0000 7fe287fff640 1 --2- 172.21.15.119:0/1281627016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a7ac0 0x7fe288137cd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2780097f0 tx=0x7fe27801c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb63091c784d98da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 1 -- 172.21.15.119:0/1281627016 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780b0070 con 0x7fe2880a7ac0 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: dump: 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:08.997 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 1 -- 172.21.15.119:0/1281627016 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2780ab600 con 0x7fe2880a7ac0 2024-09-15T21:19:08.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 5 monclient: authenticate success, global_id 14428 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 1 -- 172.21.15.119:0/1281627016 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780ab9d0 con 0x7fe2880a7ac0 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 4 set_mon_vals no callback set 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe286ffd640 10 monclient: dump: 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:08.999 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: shutdown 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/1281627016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a7ac0 msgr2=0x7fe288137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 --2- 172.21.15.119:0/1281627016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a7ac0 0x7fe288137cd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2780097f0 tx=0x7fe27801c360 comp rx=0 tx=0).stop 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/1281627016 shutdown_connections 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 --2- 172.21.15.119:0/1281627016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a7ac0 0x7fe288137cd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/1281627016 >> 172.21.15.119:0/1281627016 conn(0x7fe2880a6fd0 msgr2=0x7fe2880a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/1281627016 shutdown_connections 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/1281627016 wait complete. 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: build_initial_monmap 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: monmap: 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:09.001 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 Processor -- start 2024-09-15T21:19:09.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- start start 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: init 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 0x7fe2881cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): picked mon.noname-b con 0x7fe2880a41b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2880a45c0 0x7fe2881cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): picked mon.noname-a con 0x7fe2880a45c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2880a7ac0 0x7fe2881d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): picked mon.noname-c con 0x7fe2880a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881437e0 con 0x7fe2880a45c0 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881382f0 con 0x7fe2880a41b0 2024-09-15T21:19:09.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe288143960 con 0x7fe2880a7ac0 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient(hunting): authenticate will time out at 1067.786182s 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 0x7fe2881cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 0x7fe2881cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53072/0 (socket says 172.21.15.119:53072) 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 -- 172.21.15.119:0/2248158681 learned_addr learned my addr 172.21.15.119:0/2248158681 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): get_auth_request con 0x7fe2880a41b0 auth_method 0 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe2877fe640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2880a45c0 0x7fe2881cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient(hunting): handle_auth_done global_id 14434 payload 824 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 -- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2880a7ac0 msgr2=0x7fe2881d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28caca640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2880a7ac0 0x7fe2881d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2880a7ac0 0x7fe2881d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 -- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2880a45c0 msgr2=0x7fe2881cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2880a45c0 0x7fe2881cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 monclient: found mon.noname-b 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 -- 172.21.15.119:0/2248158681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28813a110 con 0x7fe2880a41b0 2024-09-15T21:19:09.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe2877fe640 10 monclient: get_auth_request con 0x7fe2880a45c0 auth_method 0 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe2877fe640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2880a45c0 0x7fe2881cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe287fff640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 0x7fe2881cfa20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe278018f70 tx=0x7fe278019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780b8020 con 0x7fe2880a41b0 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: dump: 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.008 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 5 monclient: authenticate success, global_id 14434 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _renew_subs 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2780b4410 con 0x7fe2880a41b0 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2880a16c0 con 0x7fe2880a41b0 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:09.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780b0040 con 0x7fe2880a41b0 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe284ff9640 10 monclient: dump: 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _renew_subs 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe2867fc640 4 set_mon_vals no callback set 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe288139e30 con 0x7fe2880a41b0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.985+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe288137cd0 con 0x7fe2880a41b0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.989+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe2780bb020 con 0x7fe2880a41b0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.989+0000 7fe284ff9640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe27403b9f0 0x7fe27403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.989+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4780+0+0 (secure 0 0 0) 0x7fe2780e9840 con 0x7fe2880a41b0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.989+0000 7fe2877fe640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe27403b9f0 0x7fe27403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.989+0000 7fe2877fe640 10 monclient: get_auth_request con 0x7fe27403b9f0 auth_method 0 2024-09-15T21:19:09.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.989+0000 7fe2877fe640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe27403b9f0 0x7fe27403deb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe27c001230 tx=0x7fe27c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.993+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe2780c2030 con 0x7fe2880a41b0 2024-09-15T21:19:09.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.993+0000 7fe284ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:08.993+0000 7fe284ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc780003550 auth_method 0 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 msgr2=0x7fc7800059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 msgr2=0x7fc7800059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:09.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.094+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:09.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.161+0000 7fe28e52d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.161+0000 7fe28e52d640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.161+0000 7fe28e52d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.161+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe2880a9a70 con 0x7fe2880a41b0 2024-09-15T21:19:09.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.409+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' removed v42) ==== 149+0+0 (secure 0 0 0) 0x7fe278016070 con 0x7fe2880a41b0 2024-09-15T21:19:09.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.409+0000 7fe284ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.409+0000 7fe284ff9640 10 monclient: _finish_command 2 = system:0 pool 'foo' removed 2024-09-15T21:19:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc780003550 auth_method 0 2024-09-15T21:19:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_done global_id 24101 payload 824 2024-09-15T21:19:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7800060c0 con 0x7fc780003550 2024-09-15T21:19:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0005890 tx=0x7fc7a0001bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0007730 con 0x7fc780003550 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 10 monclient: dump: 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc7a0007730 con 0x7fc780003550 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 msgr2=0x7fc7800059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0005890 tx=0x7fc7a0001bb0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 msgr2=0x7fc7800059e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.498+0000 7fc79affd640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc780003550 0x7fc7800059e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7ac283640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7ac283640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7ac283640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a413c200 con 0x7fc7a41353c0 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a41353c0 auth_method 0 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 10 monclient(hunting): handle_auth_done global_id 24104 payload 824 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc7a4069200 con 0x7fc7a41353c0 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 msgr2=0x7fc7a402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 msgr2=0x7fc7a402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0020000 tx=0x7fc7a0007a20 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.506+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:09.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe28e52d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe28e52d640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe28e52d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe288071e70 con 0x7fe2880a41b0 2024-09-15T21:19:09.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe284ff9640 1 -- 172.21.15.119:0/2248158681 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v42) ==== 156+0+0 (secure 0 0 0) 0x7fe2780aa3a0 con 0x7fe2880a41b0 2024-09-15T21:19:09.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe284ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.545+0000 7fe284ff9640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-09-15T21:19:09.554 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'foo' does not exist 2024-09-15T21:19:09.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe27403b9f0 msgr2=0x7fe27403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe27403b9f0 0x7fe27403deb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe27c001230 tx=0x7fe27c009040 comp rx=0 tx=0).stop 2024-09-15T21:19:09.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 10 monclient: shutdown 2024-09-15T21:19:09.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 msgr2=0x7fe2881cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 0x7fe2881cfa20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe278018f70 tx=0x7fe278019100 comp rx=0 tx=0).stop 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 shutdown_connections 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe27403b9f0 0x7fe27403deb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2880a7ac0 0x7fe2881d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2880a45c0 0x7fe2881cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 --2- 172.21.15.119:0/2248158681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2880a41b0 0x7fe2881cfa20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 >> 172.21.15.119:0/2248158681 conn(0x7fe2880a6fd0 msgr2=0x7fe288134210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 shutdown_connections 2024-09-15T21:19:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.549+0000 7fe28e52d640 1 -- 172.21.15.119:0/2248158681 wait complete. 2024-09-15T21:19:09.571 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient: get_monmap_and_config 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient: build_initial_monmap 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient: monmap: 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:09.688846+0000 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:09.688846+0000 2024-09-15T21:19:09.690 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:09.691 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 Processor -- start 2024-09-15T21:19:09.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 -- start start 2024-09-15T21:19:09.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient: init 2024-09-15T21:19:09.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f5868073570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): picked mon.noname-b con 0x7f586806dbb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58680a0750 0x7f5868073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): picked mon.noname-a con 0x7f58680a0750 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f58680a2450 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58680a0410 con 0x7f58680a2450 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5868134b00 con 0x7f58680a0750 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f58680a0240 con 0x7f586806dbb0 2024-09-15T21:19:09.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586e93b640 10 monclient(hunting): authenticate will time out at 1068.485447s 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f5868073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58680a0750 0x7f5868073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f5868073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:59088/0 (socket says 172.21.15.119:59088) 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f586ced8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f58680a2450 0x7f58680a2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48006/0 (socket says 172.21.15.119:48006) 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 -- 172.21.15.119:0/336920930 learned_addr learned my addr 172.21.15.119:0/336920930 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): get_auth_request con 0x7f58680a0750 auth_method 0 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): get_auth_request con 0x7f586806dbb0 auth_method 0 2024-09-15T21:19:09.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f58677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient(hunting): handle_auth_done global_id 24107 payload 274 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5866ffd640 1 -- 172.21.15.119:0/336920930 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1842159312 0 0) 0x7f58680a0410 con 0x7f58680a2450 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 -- 172.21.15.119:0/336920930 >> v2:172.21.15.17:3300/0 conn(0x7f58680a0750 msgr2=0x7f5868073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:09.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 --2- 172.21.15.119:0/336920930 >> v2:172.21.15.17:3300/0 conn(0x7f58680a0750 0x7f5868073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 -- 172.21.15.119:0/336920930 >> v1:172.21.15.17:6789/0 conn(0x7f58680a2450 legacy=0x7f58680a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 monclient: found mon.noname-b 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5867fff640 1 -- 172.21.15.119:0/336920930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5868134d30 con 0x7f586806dbb0 2024-09-15T21:19:09.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.685+0000 7f5866ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5867fff640 1 --2- 172.21.15.119:0/336920930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f5868073570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5858009870 tx=0x7f585801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323f671e78192128 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 1 -- 172.21.15.119:0/336920930 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580ab680 con 0x7f586806dbb0 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: dump: 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.703 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 1 -- 172.21.15.119:0/336920930 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58580b0070 con 0x7f586806dbb0 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 1 -- 172.21.15.119:0/336920930 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580aba40 con 0x7f586806dbb0 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f5866ffd640 10 monclient: dump: 2024-09-15T21:19:09.704 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 5 monclient: authenticate success, global_id 24107 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 4 set_mon_vals no callback set 2024-09-15T21:19:09.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: shutdown 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- 172.21.15.119:0/336920930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 msgr2=0x7f5868073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 --2- 172.21.15.119:0/336920930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f5868073570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5858009870 tx=0x7f585801c480 comp rx=0 tx=0).stop 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- 172.21.15.119:0/336920930 shutdown_connections 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 --2- 172.21.15.119:0/336920930 >> v2:172.21.15.17:3300/0 conn(0x7f58680a0750 0x7f5868073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 --2- 172.21.15.119:0/336920930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f5868073570 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- 172.21.15.119:0/336920930 >> 172.21.15.119:0/336920930 conn(0x7f586806cbf0 msgr2=0x7f586806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- 172.21.15.119:0/336920930 shutdown_connections 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- 172.21.15.119:0/336920930 wait complete. 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: build_initial_monmap 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: monmap: 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:09.706 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 Processor -- start 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- start start 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: init 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:09.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f586813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): picked mon.noname-c con 0x7f586806dbb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 0x7f586813aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): picked mon.noname-b con 0x7f58680a2450 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586813af50 0x7f58681311d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): picked mon.noname-a con 0x7f586813af50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f58681426f0 con 0x7f586813af50 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5868073ff0 con 0x7f58680a2450 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5868142870 con 0x7f586806dbb0 2024-09-15T21:19:09.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f586e93b640 10 monclient(hunting): authenticate will time out at 1068.491703s 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 0x7f586813aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 0x7f586813aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53076/0 (socket says 172.21.15.119:53076) 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 1 -- 172.21.15.119:0/2815022538 learned_addr learned my addr 172.21.15.119:0/2815022538 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): get_auth_request con 0x7f58680a2450 auth_method 0 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.689+0000 7f58677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586ced8640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586813af50 0x7f58681311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f586813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586ced8640 10 monclient(hunting): get_auth_request con 0x7f586813af50 auth_method 0 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586ced8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586ced8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586ced8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 10 monclient(hunting): get_auth_request con 0x7f586806dbb0 auth_method 0 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 10 monclient(hunting): handle_auth_done global_id 14437 payload 824 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 -- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 msgr2=0x7f586813a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f586813a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 -- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586813af50 msgr2=0x7f58681311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586813af50 0x7f58681311d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 -- 172.21.15.119:0/2815022538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680739c0 con 0x7f58680a2450 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58677fe640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 0x7f586813aa10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e900 tx=0x7f585c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c017070 con 0x7f58680a2450 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: dump: 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.711 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c00ca00 con 0x7f58680a2450 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 5 monclient: authenticate success, global_id 14437 2024-09-15T21:19:09.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 monclient: _renew_subs 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5868074610 con 0x7f58680a2450 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c00cdf0 con 0x7f58680a2450 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 10 monclient: dump: 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.713 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 monclient: _renew_subs 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5868132270 con 0x7f58680a2450 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f58667fc640 4 set_mon_vals no callback set 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680a2900 con 0x7f58680a2450 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f585c01a070 con 0x7f58680a2450 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f585403ba20 0x7f585403dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f585c04f9b0 con 0x7f58680a2450 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f585403ba20 0x7f585403dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 10 monclient: get_auth_request con 0x7f585403ba20 auth_method 0 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.693+0000 7f5867fff640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f585403ba20 0x7f585403dee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58580af000 tx=0x7f58580a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.697+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f585c013c20 con 0x7f58680a2450 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.697+0000 7f5864ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.697+0000 7f5864ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: get_auth_request con 0x7fc7a41353c0 auth_method 0 2024-09-15T21:19:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: _init_auth method 2 2024-09-15T21:19:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: _init_auth already have auth, reseting 2024-09-15T21:19:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: handle_auth_reply_more payload 9 2024-09-15T21:19:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 10 monclient: handle_auth_done global_id 24104 payload 293 2024-09-15T21:19:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc7aa01f640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0020000 tx=0x7fc7a0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0034070 con 0x7fc7a41353c0 2024-09-15T21:19:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc79affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc79affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.710+0000 7fc79affd640 10 monclient: dump: 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.720 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7fc7a0030470 con 0x7fc7a41353c0 2024-09-15T21:19:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 msgr2=0x7fc7a402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0020000 tx=0x7fc7a0019000 comp rx=0 tx=0).stop 2024-09-15T21:19:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 msgr2=0x7fc7a402d990 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc79affd640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a41353c0 0x7fc7a402d990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7aa820640 1 -- 172.21.15.17:0/2280189507 reap_dead start 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 5515, 2024-09-15T21:19:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:09.735 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78803b9c0 msgr2=0x7fc78803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78803b9c0 0x7fc78803de80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c0a7070 comp rx=0 tx=0).stop 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 10 monclient: shutdown 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a40a5580 msgr2=0x7fc7a41cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a40a5580 0x7fc7a41cfb30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc79400e7e0 tx=0x7fc79400eca0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 shutdown_connections 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78803b9c0 0x7fc78803de80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 --2- 172.21.15.17:0/2280189507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a40a5580 0x7fc7a41cfb30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 >> 172.21.15.17:0/2280189507 conn(0x7fc7a40aabb0 msgr2=0x7fc7a40a8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 shutdown_connections 2024-09-15T21:19:09.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.714+0000 7fc7ac283640 1 -- 172.21.15.17:0/2280189507 wait complete. 2024-09-15T21:19:09.751 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:09.751 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:19:09.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:19:09.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f586e93b640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f586e93b640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f586e93b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f586813da80 con 0x7f58680a2450 2024-09-15T21:19:09.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v42) ==== 156+0+0 (secure 0 0 0) 0x7f585c011350 con 0x7f58680a2450 2024-09-15T21:19:09.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f5864ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.869+0000 7f5864ff9640 10 monclient: _finish_command 2 = system:0 pool 'foo' does not exist 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient: get_monmap_and_config 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient: build_initial_monmap 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient: monmap: 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:09.882494+0000 2024-09-15T21:19:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:09.882494+0000 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:09.885 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 1 Processor -- start 2024-09-15T21:19:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 1 -- start start 2024-09-15T21:19:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient: init 2024-09-15T21:19:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc1200a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc1200a5580 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1200a59d0 0x7fc120140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient(hunting): picked mon.noname-a con 0x7fc1200a59d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1200a2c30 con 0x7fc1200a4a90 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1200a19b0 con 0x7fc1200a5580 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc1200a0f40 con 0x7fc1200a59d0 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc1263a7640 10 monclient(hunting): authenticate will time out at 1073.277955s 2024-09-15T21:19:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc1200a4a90 0x7fc1200a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43706/0 (socket says 172.21.15.17:43706) 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11ffff640 1 -- 172.21.15.17:0/2159492638 learned_addr learned my addr 172.21.15.17:0/2159492638 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 1 --2- 172.21.15.17:0/2159492638 >> v2:172.21.15.17:3300/0 conn(0x7fc1200a59d0 0x7fc120140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): get_auth_request con 0x7fc1200a59d0 auth_method 0 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11effd640 1 -- 172.21.15.17:0/2159492638 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576076652 0 0) 0x7fc118001120 con 0x7fc1200a4a90 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11effd640 10 monclient(hunting): _negotiate 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11effd640 10 monclient(hunting): my global_id is 4686 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc11effd640 1 -- 172.21.15.17:0/2159492638 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc108003530 con 0x7fc1200a4a90 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient(hunting): handle_auth_done global_id 4683 payload 274 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 1 -- 172.21.15.17:0/2159492638 >> v1:172.21.15.119:6789/0 conn(0x7fc1200a5580 legacy=0x7fc12013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 1 -- 172.21.15.17:0/2159492638 >> v1:172.21.15.17:6789/0 conn(0x7fc1200a4a90 legacy=0x7fc1200a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient: _finish_hunting 0 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 1 monclient: found mon.noname-a 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:09.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.878+0000 7fc124944640 1 -- 172.21.15.17:0/2159492638 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1200a1ce0 con 0x7fc1200a59d0 2024-09-15T21:19:09.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 --2- 172.21.15.17:0/2159492638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a59d0 0x7fc120140440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc110009dc0 tx=0x7fc11001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6afb09d162c1c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 1 -- 172.21.15.17:0/2159492638 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100b2070 con 0x7fc1200a59d0 2024-09-15T21:19:09.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:09.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: dump: 2024-09-15T21:19:09.896 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: _finish_auth 0 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 1 -- 172.21.15.17:0/2159492638 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1100a5850 con 0x7fc1200a59d0 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 1 -- 172.21.15.17:0/2159492638 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100a5c20 con 0x7fc1200a59d0 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 5 monclient: authenticate success, global_id 4683 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 4 set_mon_vals no callback set 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11effd640 10 monclient: dump: 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: shutdown 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- 172.21.15.17:0/2159492638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a59d0 msgr2=0x7fc120140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 --2- 172.21.15.17:0/2159492638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a59d0 0x7fc120140440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc110009dc0 tx=0x7fc11001c870 comp rx=0 tx=0).stop 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- 172.21.15.17:0/2159492638 shutdown_connections 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 --2- 172.21.15.17:0/2159492638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a59d0 0x7fc120140440 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- 172.21.15.17:0/2159492638 >> 172.21.15.17:0/2159492638 conn(0x7fc1200aabb0 msgr2=0x7fc1200a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- 172.21.15.17:0/2159492638 shutdown_connections 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- 172.21.15.17:0/2159492638 wait complete. 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: build_initial_monmap 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: monmap: 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 Processor -- start 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- start start 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: init 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1200a4a90 0x7fc120133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): picked mon.noname-b con 0x7fc1200a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a5580 0x7fc120133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): picked mon.noname-a con 0x7fc1200a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1200a59d0 0x7fc120138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): picked mon.noname-c con 0x7fc1200a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1201437c0 con 0x7fc1200a5580 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc12006f2f0 con 0x7fc1200a4a90 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc120143940 con 0x7fc1200a59d0 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc1263a7640 10 monclient(hunting): authenticate will time out at 1073.283117s 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a5580 0x7fc120133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1200a59d0 0x7fc120138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a5580 0x7fc120133fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45944/0 (socket says 172.21.15.17:45944) 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 1 -- 172.21.15.17:0/482228202 learned_addr learned my addr 172.21.15.17:0/482228202 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): get_auth_request con 0x7fc1200a59d0 auth_method 0 2024-09-15T21:19:09.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): get_auth_request con 0x7fc1200a5580 auth_method 0 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1200a4a90 0x7fc120133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): get_auth_request con 0x7fc1200a4a90 auth_method 0 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient(hunting): handle_auth_done global_id 24113 payload 824 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1200a4a90 msgr2=0x7fc120133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1200a4a90 0x7fc120133a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a5580 msgr2=0x7fc120133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1200a5580 0x7fc120133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient: _finish_hunting 0 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 monclient: found mon.noname-c 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1200a1280 con 0x7fc1200a59d0 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.882+0000 7fc124944640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1200a59d0 0x7fc120138cf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1100ac000 tx=0x7fc110019610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100b2050 con 0x7fc1200a59d0 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: dump: 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1100be070 con 0x7fc1200a59d0 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100ad870 con 0x7fc1200a59d0 2024-09-15T21:19:09.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: dump: 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 5 monclient: authenticate success, global_id 24113 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 10 monclient: _renew_subs 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11e7fc640 4 set_mon_vals no callback set 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12013f720 con 0x7fc1200a59d0 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 10 monclient: _renew_subs 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12013af70 con 0x7fc1200a59d0 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 0x7fc120139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1200a4e70 con 0x7fc120139230 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc1100ada10 con 0x7fc1200a59d0 2024-09-15T21:19:09.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 0x7fc120139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient: get_auth_request con 0x7fc120139230 auth_method 0 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10c03b9f0 0x7fc10c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fc1100ea9a0 con 0x7fc1200a59d0 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 10 monclient(hunting): handle_auth_done global_id 14446 payload 824 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11f7fe640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10c03b9f0 0x7fc10c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc12013a410 con 0x7fc120139230 2024-09-15T21:19:09.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11f7fe640 10 monclient: get_auth_request con 0x7fc10c03b9f0 auth_method 0 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11ffff640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 0x7fc120139610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc118005890 tx=0x7fc11800fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180058c0 con 0x7fc120139230 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: dump: 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.910 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11f7fe640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10c03b9f0 0x7fc10c03deb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc120138ab0 tx=0x7fc114009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc118018070 con 0x7fc120139230 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 msgr2=0x7fc120139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 0x7fc120139610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc118005890 tx=0x7fc11800fc00 comp rx=0 tx=0).stop 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 msgr2=0x7fc120139610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:09.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.886+0000 7fc11cff9640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139230 0x7fc120139610 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc118005890 tx=0x7fc11800fc00 comp rx=0 tx=0).stop 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 0x7fc120134a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc120069200 con 0x7fc120139b50 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 0x7fc120134a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient: get_auth_request con 0x7fc120139b50 auth_method 0 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:09.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 10 monclient(hunting): handle_auth_done global_id 14449 payload 824 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 1 -- 172.21.15.17:0/482228202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc12013fc90 con 0x7fc120139b50 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11ffff640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 0x7fc120134a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc118017000 tx=0x7fc118005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118007720 con 0x7fc120139b50 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 10 monclient: dump: 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:09.914 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7fc118007720 con 0x7fc120139b50 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 msgr2=0x7fc120134a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 0x7fc120134a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc118017000 tx=0x7fc118005800 comp rx=0 tx=0).stop 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 msgr2=0x7fc120134a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:09.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc11cff9640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc120139b50 0x7fc120134a30 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc118017000 tx=0x7fc118005800 comp rx=0 tx=0).stop 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 44618, 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:19:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:09.933 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10c03b9f0 msgr2=0x7fc10c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10c03b9f0 0x7fc10c03deb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc120138ab0 tx=0x7fc114009040 comp rx=0 tx=0).stop 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 10 monclient: shutdown 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1200a59d0 msgr2=0x7fc120138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 --2- 172.21.15.17:0/482228202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc1200a59d0 0x7fc120138cf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1100ac000 tx=0x7fc110019610 comp rx=0 tx=0).stop 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc124944640 1 -- 172.21.15.17:0/482228202 reap_dead start 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc124944640 1 -- 172.21.15.17:0/482228202 reap_dead start 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 shutdown_connections 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 >> 172.21.15.17:0/482228202 conn(0x7fc1200aabb0 msgr2=0x7fc1200a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 shutdown_connections 2024-09-15T21:19:09.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:09.894+0000 7fc1263a7640 1 -- 172.21.15.17:0/482228202 wait complete. 2024-09-15T21:19:09.935 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:09.935 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:19:09.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:19:09.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.973+0000 7f586e93b640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.973+0000 7f586e93b640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.973+0000 7f586e93b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:09.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.973+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5868074bd0 con 0x7f58680a2450 2024-09-15T21:19:09.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f5864ff9640 1 -- 172.21.15.119:0/2815022538 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v42) ==== 156+0+0 (secure 0 0 0) 0x7f585c023b20 con 0x7f58680a2450 2024-09-15T21:19:09.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f5864ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:09.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f5864ff9640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-09-15T21:19:09.983 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'foo' does not exist 2024-09-15T21:19:09.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f585403ba20 msgr2=0x7f585403dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f585403ba20 0x7f585403dee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58580af000 tx=0x7f58580a7000 comp rx=0 tx=0).stop 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 10 monclient: shutdown 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 msgr2=0x7f586813aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 0x7f586813aa10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e900 tx=0x7f585c00edc0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 shutdown_connections 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f585403ba20 0x7f585403dee0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586813af50 0x7f58681311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58680a2450 0x7f586813aa10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 --2- 172.21.15.119:0/2815022538 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f586806dbb0 0x7f586813a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 >> 172.21.15.119:0/2815022538 conn(0x7f586806cbf0 msgr2=0x7f5868072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 shutdown_connections 2024-09-15T21:19:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:09.977+0000 7f586e93b640 1 -- 172.21.15.119:0/2815022538 wait complete. 2024-09-15T21:19:10.000 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-09-15T21:19:10.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.038+0000 7f0d2fcd3640 10 monclient: get_monmap_and_config 2024-09-15T21:19:10.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.038+0000 7f0d2fcd3640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.038+0000 7f0d2fcd3640 10 monclient: monmap: 2024-09-15T21:19:10.047 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:10.047 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.047 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:10.045356+0000 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:10.045356+0000 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:10.048 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.038+0000 7f0d2fcd3640 1 Processor -- start 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 1 -- start start 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient: init 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0d280a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0d2809e8c0 0x7f0d280a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient(hunting): picked mon.noname-a con 0x7f0d2809e8c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0d281410a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d280a31c0 con 0x7f0d280a0890 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d2809d640 con 0x7f0d281410a0 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2809d920 con 0x7f0d2809e8c0 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient(hunting): authenticate will time out at 1073.440695s 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2da6f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0d280a0890 0x7f0d280a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43710/0 (socket says 172.21.15.17:43710) 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2da6f640 1 -- 172.21.15.17:0/413004070 learned_addr learned my addr 172.21.15.17:0/413004070 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 1 --2- 172.21.15.17:0/413004070 >> v2:172.21.15.17:3300/0 conn(0x7f0d2809e8c0 0x7f0d280a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): get_auth_request con 0x7f0d2809e8c0 auth_method 0 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_done global_id 4692 payload 274 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 1 -- 172.21.15.17:0/413004070 >> v1:172.21.15.119:6789/0 conn(0x7f0d281410a0 legacy=0x7f0d28143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 1 -- 172.21.15.17:0/413004070 >> v1:172.21.15.17:6789/0 conn(0x7f0d280a0890 legacy=0x7f0d280a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 1 monclient: found mon.noname-a 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 1 -- 172.21.15.17:0/413004070 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2809dc00 con 0x7f0d2809e8c0 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 1 -- 172.21.15.17:0/413004070 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1223497650 0 0) 0x7f0d280a31c0 con 0x7f0d280a0890 2024-09-15T21:19:10.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:10.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2d26e640 1 --2- 172.21.15.17:0/413004070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2809e8c0 0x7f0d280a1380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d180097f0 tx=0x7f0d1801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86016b39f64a01c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 1 -- 172.21.15.17:0/413004070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180b0070 con 0x7f0d2809e8c0 2024-09-15T21:19:10.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:10.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: dump: 2024-09-15T21:19:10.057 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 1 -- 172.21.15.17:0/413004070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180ab600 con 0x7f0d2809e8c0 2024-09-15T21:19:10.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 5 monclient: authenticate success, global_id 4692 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2fcd3640 4 set_mon_vals no callback set 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 1 -- 172.21.15.17:0/413004070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180ab9d0 con 0x7f0d2809e8c0 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.042+0000 7f0d2ca6d640 10 monclient: dump: 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: shutdown 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/413004070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2809e8c0 msgr2=0x7f0d280a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/413004070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2809e8c0 0x7f0d280a1380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d180097f0 tx=0x7f0d1801c360 comp rx=0 tx=0).stop 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/413004070 shutdown_connections 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/413004070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2809e8c0 0x7f0d280a1380 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/413004070 >> 172.21.15.17:0/413004070 conn(0x7f0d280a66a0 msgr2=0x7f0d280a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/413004070 shutdown_connections 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/413004070 wait complete. 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: monmap: 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 Processor -- start 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- start start 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: init 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d280a0890 0x7f0d2813bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): picked mon.noname-a con 0x7f0d280a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281410a0 0x7f0d2813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): picked mon.noname-c con 0x7f0d281410a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d28070fc0 0x7f0d28071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): picked mon.noname-b con 0x7f0d28070fc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0d28147b70 con 0x7f0d280a0890 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281479f0 con 0x7f0d28070fc0 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0d28147cf0 con 0x7f0d281410a0 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient(hunting): authenticate will time out at 1073.446172s 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d280a0890 0x7f0d2813bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281410a0 0x7f0d2813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d280a0890 0x7f0d2813bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45954/0 (socket says 172.21.15.17:45954) 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 1 -- 172.21.15.17:0/2051149318 learned_addr learned my addr 172.21.15.17:0/2051149318 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): get_auth_request con 0x7f0d280a0890 auth_method 0 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): get_auth_request con 0x7f0d281410a0 auth_method 0 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d28070fc0 0x7f0d28071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2da6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): get_auth_request con 0x7f0d28070fc0 auth_method 0 2024-09-15T21:19:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2e270640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient(hunting): handle_auth_done global_id 24116 payload 824 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 -- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d28070fc0 msgr2=0x7f0d28071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d28070fc0 0x7f0d28071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 -- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d280a0890 msgr2=0x7f0d2813bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d280a0890 0x7f0d2813bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 monclient: found mon.noname-c 2024-09-15T21:19:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 -- 172.21.15.17:0/2051149318 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d281251b0 con 0x7f0d281410a0 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2d26e640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281410a0 0x7f0d2813c130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d180ab000 tx=0x7f0d1801aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180b0050 con 0x7f0d281410a0 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: dump: 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 5 monclient: authenticate success, global_id 24116 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180bc070 con 0x7f0d281410a0 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: _renew_subs 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d280a4c90 con 0x7f0d281410a0 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180b5a50 con 0x7f0d281410a0 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1e7fc640 10 monclient: dump: 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d1ffff640 4 set_mon_vals no callback set 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: _renew_subs 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d28125770 con 0x7f0d281410a0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.046+0000 7f0d2fcd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d2fcd3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d2fcd3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d280a0c70 con 0x7f0d281410a0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0d180b5bf0 con 0x7f0d281410a0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d1e7fc640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0c03b9c0 0x7f0d0c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f0d180ea340 con 0x7f0d281410a0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d2da6f640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0c03b9c0 0x7f0d0c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d2da6f640 10 monclient: get_auth_request con 0x7f0d0c03b9c0 auth_method 0 2024-09-15T21:19:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.050+0000 7f0d2da6f640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0c03b9c0 0x7f0d0c03de80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007fd0 tx=0x7f0d24007f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.054+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0d18016070 con 0x7f0d281410a0 2024-09-15T21:19:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.054+0000 7f0d1e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.054+0000 7f0d1e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient: get_monmap_and_config 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient: monmap: 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:10.120954+0000 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:10.120954+0000 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.122 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:10.123 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:10.123 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:10.123 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:10.123 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:10.123 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 1 Processor -- start 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 1 -- start start 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient: init 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e2c0a7ac0 0x7f5e2c137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient(hunting): picked mon.noname-c con 0x7f5e2c0a7ac0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e2c0a41b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e2c0a45c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2c135120 con 0x7f5e2c0a45c0 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2c0a19d0 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2c0a7ea0 con 0x7f5e2c0a7ac0 2024-09-15T21:19:10.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e30f25640 10 monclient(hunting): authenticate will time out at 1068.917691s 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e2c0a7ac0 0x7f5e2c137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e29d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5e2c0a41b0 0x7f5e2c134300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47698/0 (socket says 172.21.15.119:47698) 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e29d9b640 1 -- 172.21.15.119:0/1728170215 learned_addr learned my addr 172.21.15.119:0/1728170215 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a7ac0 auth_method 0 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_done global_id 14455 payload 274 2024-09-15T21:19:10.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1728170215 >> v1:172.21.15.119:6789/0 conn(0x7f5e2c0a41b0 legacy=0x7f5e2c134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1728170215 >> v1:172.21.15.17:6789/0 conn(0x7f5e2c0a45c0 legacy=0x7f5e2c134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 1 monclient: found mon.noname-c 2024-09-15T21:19:10.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.117+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1728170215 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c0a6960 con 0x7f5e2c0a7ac0 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 --2- 172.21.15.119:0/1728170215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c137cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009dc0 tx=0x7f5e1c01c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8533264fad68a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 1 -- 172.21.15.119:0/1728170215 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0b2070 con 0x7f5e2c0a7ac0 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: dump: 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.132 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 1 -- 172.21.15.119:0/1728170215 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0a5760 con 0x7f5e2c0a7ac0 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 1 -- 172.21.15.119:0/1728170215 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0a5b30 con 0x7f5e2c0a7ac0 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2959a640 10 monclient: dump: 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 5 monclient: authenticate success, global_id 14455 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:10.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 4 set_mon_vals no callback set 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: shutdown 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1728170215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a7ac0 msgr2=0x7f5e2c137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1728170215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c137cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009dc0 tx=0x7f5e1c01c810 comp rx=0 tx=0).stop 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1728170215 shutdown_connections 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1728170215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c137cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1728170215 >> 172.21.15.119:0/1728170215 conn(0x7f5e2c0a6fd0 msgr2=0x7f5e2c0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1728170215 shutdown_connections 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1728170215 wait complete. 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: monmap: 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 Processor -- start 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- start start 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: init 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 0x7f5e2c130e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): picked mon.noname-b con 0x7f5e2c0a41b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2c0a45c0 0x7f5e2c1313c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): picked mon.noname-c con 0x7f5e2c0a45c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c072150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): picked mon.noname-a con 0x7f5e2c0a7ac0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1437e0 con 0x7f5e2c0a7ac0 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1382f0 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c143960 con 0x7f5e2c0a45c0 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient(hunting): authenticate will time out at 1068.922667s 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 0x7f5e2c130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 0x7f5e2c130e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53094/0 (socket says 172.21.15.119:53094) 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1147744831 learned_addr learned my addr 172.21.15.119:0/1147744831 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a41b0 auth_method 0 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e29d9b640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2c0a45c0 0x7f5e2c1313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e29d9b640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a45c0 auth_method 0 2024-09-15T21:19:10.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e29d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2ad9d640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_done global_id 14461 payload 824 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2c0a45c0 msgr2=0x7f5e2c1313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2c0a45c0 0x7f5e2c1313c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c0a7ac0 msgr2=0x7f5e2c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 monclient: found mon.noname-b 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 -- 172.21.15.119:0/1147744831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c139ab0 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2ad9d640 10 monclient: get_auth_request con 0x7f5e2c0a7ac0 auth_method 0 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2ad9d640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e2a59c640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 0x7f5e2c130e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0b0040 tx=0x7f5e1c0195b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0b2050 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: dump: 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.140 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0be070 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 5 monclient: authenticate success, global_id 14461 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: _renew_subs 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c139d90 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0ada90 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e22ffd640 10 monclient: dump: 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e23fff640 4 set_mon_vals no callback set 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: _renew_subs 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.121+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c13a070 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e30f25640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e30f25640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e30f25640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2c137cd0 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e1c0adc30 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e22ffd640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9c0 0x7f5e0c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f5e1c0eabd0 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e29d9b640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9c0 0x7f5e0c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e29d9b640 10 monclient: get_auth_request con 0x7f5e0c03b9c0 auth_method 0 2024-09-15T21:19:10.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.125+0000 7f5e29d9b640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9c0 0x7f5e0c03de80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007fd0 tx=0x7f5e24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.129+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5e1c016520 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.129+0000 7f5e22ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.129+0000 7f5e22ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:10.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.293+0000 7f5e30f25640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.293+0000 7f5e30f25640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.293+0000 7f5e30f25640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.293+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e2c140250 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.297+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v42) ==== 159+0+0 (secure 0 0 0) 0x7f5e1c0c3b20 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.297+0000 7f5e22ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.297+0000 7f5e22ffd640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-09-15T21:19:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d2fcd3640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:19:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d2fcd3640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:19:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d2fcd3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0d28073360 con 0x7f0d281410a0 2024-09-15T21:19:10.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d1e7fc640 1 -- 172.21.15.17:0/2051149318 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0d180ea0d0 con 0x7f0d281410a0 2024-09-15T21:19:10.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d1e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:19:10.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.310+0000 7f0d1e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:10.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.310+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.310+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=989950822,config=3541143443,health=3155990764,kv=3079713637,logm=2834637117}) 2024-09-15T21:19:10.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.310+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=8,config=2,health=12,kv=11,logm=67} crc {auth=779082819,config=3541143443,health=3155990764,kv=2762157375,logm=3276876762}) 2024-09-15T21:19:10.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.310+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.310+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=989950822,config=3541143443,health=3155990764,kv=3079713637,logm=2834637117}) 2024-09-15T21:19:10.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.310+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=9,config=2,health=12,kv=11,logm=66} crc {auth=989950822,config=3541143443,health=3155990764,kv=1736545667,logm=784452287}) 2024-09-15T21:19:10.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935312040}) 2024-09-15T21:19:10.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=188642533}) 2024-09-15T21:19:10.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935312040}) 2024-09-15T21:19:10.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2378452630}) 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0c03b9c0 msgr2=0x7f0d0c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0c03b9c0 0x7f0d0c03de80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007fd0 tx=0x7f0d24007f60 comp rx=0 tx=0).stop 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 10 monclient: shutdown 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281410a0 msgr2=0x7f0d2813c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281410a0 0x7f0d2813c130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d180ab000 tx=0x7f0d1801aba0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 shutdown_connections 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0c03b9c0 0x7f0d0c03de80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d28070fc0 0x7f0d28071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281410a0 0x7f0d2813c130 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 --2- 172.21.15.17:0/2051149318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d280a0890 0x7f0d2813bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 >> 172.21.15.17:0/2051149318 conn(0x7f0d280a66a0 msgr2=0x7f0d280a5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 shutdown_connections 2024-09-15T21:19:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:10.314+0000 7f0d2fcd3640 1 -- 172.21.15.17:0/2051149318 wait complete. 2024-09-15T21:19:10.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4044409064}) 2024-09-15T21:19:10.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4014681265}) 2024-09-15T21:19:10.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4044409064}) 2024-09-15T21:19:10.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.314+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=434572532}) 2024-09-15T21:19:10.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2924882597}) 2024-09-15T21:19:10.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1096294724}) 2024-09-15T21:19:10.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2924882597}) 2024-09-15T21:19:10.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3767678383}) 2024-09-15T21:19:10.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3587717771}) 2024-09-15T21:19:10.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=153092029}) 2024-09-15T21:19:10.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3587717771}) 2024-09-15T21:19:10.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.318+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=646386505}) 2024-09-15T21:19:10.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161549200}) 2024-09-15T21:19:10.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=254524982}) 2024-09-15T21:19:10.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161549200}) 2024-09-15T21:19:10.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2852717729}) 2024-09-15T21:19:10.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=46,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=4} crc {logm=1736618753,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2798085437,mon_config_key=2985153859,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=50005377,osd_snap=841190635,osdmap=1491512105}) 2024-09-15T21:19:10.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=50,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1} crc {logm=3410601240,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2798085437,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=50005377,osd_snap=3823785925}) 2024-09-15T21:19:10.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=46,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=4} crc {logm=1736618753,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2798085437,mon_config_key=2985153859,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=50005377,osd_snap=841190635,osdmap=1491512105}) 2024-09-15T21:19:10.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.322+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=48,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=4116948861,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2798085437,mon_config_key=521969046,monmap=186996059,nvmeofgw=2560202831,osd_metadata=2194678291,osd_pg_creating=50005377,osd_snap=841190635,osdmap=2694903497}) 2024-09-15T21:19:10.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.326+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.326+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=79} crc {osdmap=3587766777}) 2024-09-15T21:19:10.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.326+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=243319388}) 2024-09-15T21:19:10.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.326+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:10.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.326+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=79} crc {osdmap=3587766777}) 2024-09-15T21:19:10.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:10.326+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=3453233741}) 2024-09-15T21:19:10.351 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:19:10.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.389+0000 7f5e30f25640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.389+0000 7f5e30f25640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.389+0000 7f5e30f25640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.389+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e2c131900 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e22ffd640 1 -- 172.21.15.119:0/1147744831 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v42) ==== 159+0+0 (secure 0 0 0) 0x7f5e1c0ab340 con 0x7f5e2c0a41b0 2024-09-15T21:19:10.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e22ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e22ffd640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-09-15T21:19:10.399 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'fooo' does not exist 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9c0 msgr2=0x7f5e0c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9c0 0x7f5e0c03de80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007fd0 tx=0x7f5e24009040 comp rx=0 tx=0).stop 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 10 monclient: shutdown 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 msgr2=0x7f5e2c130e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 0x7f5e2c130e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0b0040 tx=0x7f5e1c0195b0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 shutdown_connections 2024-09-15T21:19:10.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9c0 0x7f5e0c03de80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c0a7ac0 0x7f5e2c072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2c0a45c0 0x7f5e2c1313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 --2- 172.21.15.119:0/1147744831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e2c0a41b0 0x7f5e2c130e80 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0b0040 tx=0x7f5e1c0195b0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 >> 172.21.15.119:0/1147744831 conn(0x7f5e2c0a6fd0 msgr2=0x7f5e2c134210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 shutdown_connections 2024-09-15T21:19:10.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.393+0000 7f5e30f25640 1 -- 172.21.15.119:0/1147744831 wait complete. 2024-09-15T21:19:10.414 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-09-15T21:19:10.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient: get_monmap_and_config 2024-09-15T21:19:10.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient: monmap: 2024-09-15T21:19:10.523 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:10.523 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.523 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:10.520864+0000 2024-09-15T21:19:10.524 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:10.520864+0000 2024-09-15T21:19:10.524 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.524 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.524 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:10.524 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 Processor -- start 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 -- start start 2024-09-15T21:19:10.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient: init 2024-09-15T21:19:10.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:19:10.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05940a2b70 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:10.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05940a3660 0x7f059413d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): picked mon.noname-b con 0x7f05940a3660 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f059406ec30 0x7f059413f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): picked mon.noname-c con 0x7f059406ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05940702c0 con 0x7f05940a2b70 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0594070070 con 0x7f059406ec30 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f05940a4900 con 0x7f05940a3660 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05997fa640 10 monclient(hunting): authenticate will time out at 1069.317513s 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f059406ec30 0x7f059413f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f059406ec30 0x7f059413f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53104/0 (socket says 172.21.15.119:53104) 2024-09-15T21:19:10.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 -- 172.21.15.119:0/2881496922 learned_addr learned my addr 172.21.15.119:0/2881496922 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): get_auth_request con 0x7f059406ec30 auth_method 0 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05927fc640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05940a3660 0x7f059413d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient(hunting): handle_auth_done global_id 14464 payload 274 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 -- 172.21.15.119:0/2881496922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05940a3660 msgr2=0x7f059413d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05940a3660 0x7f059413d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 -- 172.21.15.119:0/2881496922 >> v1:172.21.15.17:6789/0 conn(0x7f05940a2b70 legacy=0x7f05940a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05937fe640 1 -- 172.21.15.119:0/2881496922 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05940a4240 con 0x7f059406ec30 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05927fc640 10 monclient: get_auth_request con 0x7f05940a3660 auth_method 0 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.517+0000 7f05927fc640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05940a3660 0x7f059413d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f059406ec30 0x7f059413f6c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f058c009e50 tx=0x7f058c01c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb7911c5053f616 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 1 -- 172.21.15.119:0/2881496922 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0a55b0 con 0x7f059406ec30 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: dump: 2024-09-15T21:19:10.535 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 1 -- 172.21.15.119:0/2881496922 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f058c0a5750 con 0x7f059406ec30 2024-09-15T21:19:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 5 monclient: authenticate success, global_id 14464 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 4 set_mon_vals no callback set 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 1 -- 172.21.15.119:0/2881496922 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0a5b20 con 0x7f059406ec30 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0591ffb640 10 monclient: dump: 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.537 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: shutdown 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/2881496922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f059406ec30 msgr2=0x7f059413f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f059406ec30 0x7f059413f6c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f058c009e50 tx=0x7f058c01c8f0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/2881496922 shutdown_connections 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f059406ec30 0x7f059413f6c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 --2- 172.21.15.119:0/2881496922 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05940a3660 0x7f059413d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/2881496922 >> 172.21.15.119:0/2881496922 conn(0x7f059406d750 msgr2=0x7f05940a8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/2881496922 shutdown_connections 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/2881496922 wait complete. 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: monmap: 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 Processor -- start 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- start start 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: init 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f059406ec30 0x7f05941cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): picked mon.noname-c con 0x7f059406ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05940a2b70 0x7f0594137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): picked mon.noname-a con 0x7f05940a2b70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 0x7f059413b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): picked mon.noname-b con 0x7f05940a3660 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0594142420 con 0x7f05940a2b70 2024-09-15T21:19:10.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0594070b20 con 0x7f05940a3660 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f05941425a0 con 0x7f059406ec30 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient(hunting): authenticate will time out at 1069.322370s 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 0x7f059413b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 0x7f059413b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53114/0 (socket says 172.21.15.119:53114) 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 -- 172.21.15.119:0/1804204063 learned_addr learned my addr 172.21.15.119:0/1804204063 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): get_auth_request con 0x7f05940a3660 auth_method 0 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0592ffd640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f059406ec30 0x7f05941cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0592ffd640 10 monclient(hunting): get_auth_request con 0x7f059406ec30 auth_method 0 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0592ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0592ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0592ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient(hunting): handle_auth_done global_id 14467 payload 824 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05927fc640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05940a2b70 0x7f0594137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 -- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f059406ec30 msgr2=0x7f05941cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f059406ec30 0x7f05941cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 -- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05940a2b70 msgr2=0x7f0594137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05940a2b70 0x7f0594137f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 -- 172.21.15.119:0/1804204063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059413edb0 con 0x7f05940a3660 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05937fe640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 0x7f059413b270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f058c01a520 tx=0x7f058c019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0bc020 con 0x7f05940a3660 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: dump: 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.543 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f058c0b8670 con 0x7f05940a3660 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 5 monclient: authenticate success, global_id 14467 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c009510 con 0x7f05940a3660 2024-09-15T21:19:10.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f057b7fe640 10 monclient: dump: 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f0590ff9640 4 set_mon_vals no callback set 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: _renew_subs 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05940a6b50 con 0x7f05940a3660 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: _renew_subs 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.521+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059409f360 con 0x7f05940a3660 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f05997fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f05997fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f05997fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05940a2f50 con 0x7f05940a3660 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f058c0bf020 con 0x7f05940a3660 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f057b7fe640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f058003ba20 0x7f058003dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f058c0ea9d0 con 0x7f05940a3660 2024-09-15T21:19:10.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f0592ffd640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f058003ba20 0x7f058003dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f0592ffd640 10 monclient: get_auth_request con 0x7f058003ba20 auth_method 0 2024-09-15T21:19:10.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.525+0000 7f0592ffd640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f058003ba20 0x7f058003dee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0588000f70 tx=0x7f0588009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.529+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f058c0c6030 con 0x7f05940a3660 2024-09-15T21:19:10.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.529+0000 7f057b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.529+0000 7f057b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:10.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f05997fa640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f05997fa640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f05997fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f059413eff0 con 0x7f05940a3660 2024-09-15T21:19:10.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v42) ==== 159+0+0 (secure 0 0 0) 0x7f058c016600 con 0x7f05940a3660 2024-09-15T21:19:10.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f057b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.685+0000 7f057b7fe640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-09-15T21:19:10.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f05997fa640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f05997fa640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f05997fa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f059413d890 con 0x7f05940a3660 2024-09-15T21:19:10.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f057b7fe640 1 -- 172.21.15.119:0/1804204063 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v42) ==== 159+0+0 (secure 0 0 0) 0x7f058c0c0e10 con 0x7f05940a3660 2024-09-15T21:19:10.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f057b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:10.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.789+0000 7f057b7fe640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-09-15T21:19:10.797 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'fooo' does not exist 2024-09-15T21:19:10.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f058003ba20 msgr2=0x7f058003dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f058003ba20 0x7f058003dee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0588000f70 tx=0x7f0588009040 comp rx=0 tx=0).stop 2024-09-15T21:19:10.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 10 monclient: shutdown 2024-09-15T21:19:10.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 msgr2=0x7f059413b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 0x7f059413b270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f058c01a520 tx=0x7f058c019690 comp rx=0 tx=0).stop 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 shutdown_connections 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f058003ba20 0x7f058003dee0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05940a3660 0x7f059413b270 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05940a2b70 0x7f0594137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 --2- 172.21.15.119:0/1804204063 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f059406ec30 0x7f05941cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 >> 172.21.15.119:0/1804204063 conn(0x7f059406d750 msgr2=0x7f059409e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 shutdown_connections 2024-09-15T21:19:10.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.793+0000 7f05997fa640 1 -- 172.21.15.119:0/1804204063 wait complete. 2024-09-15T21:19:10.814 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool delete fuggg fuggg --yes-i-really-really-mean-it 2024-09-15T21:19:10.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient: get_monmap_and_config 2024-09-15T21:19:10.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient: monmap: 2024-09-15T21:19:10.956 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:10.956 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:10.955184+0000 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:10.955184+0000 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:10.957 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 Processor -- start 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 -- start start 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient: init 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6730137bb0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67301386a0 0x7f673013d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): picked mon.noname-a con 0x7f67301386a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:10.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f673013d5b0 0x7f673013f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): picked mon.noname-c con 0x7f673013d5b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67300a3e60 con 0x7f6730137bb0 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f67300a6c40 con 0x7f67301386a0 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f67300a1c70 con 0x7f673013d5b0 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f673779c640 10 monclient(hunting): authenticate will time out at 1069.751675s 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f673013d5b0 0x7f673013f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f673013d5b0 0x7f673013f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53122/0 (socket says 172.21.15.119:53122) 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 1 -- 172.21.15.119:0/2245508008 learned_addr learned my addr 172.21.15.119:0/2245508008 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): get_auth_request con 0x7f673013d5b0 auth_method 0 2024-09-15T21:19:10.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.949+0000 7f6735d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 10 monclient(hunting): handle_auth_done global_id 14470 payload 274 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 1 -- 172.21.15.119:0/2245508008 >> v2:172.21.15.17:3300/0 conn(0x7f67301386a0 msgr2=0x7f673013d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 1 --2- 172.21.15.119:0/2245508008 >> v2:172.21.15.17:3300/0 conn(0x7f67301386a0 0x7f673013d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 1 -- 172.21.15.119:0/2245508008 >> v1:172.21.15.17:6789/0 conn(0x7f6730137bb0 legacy=0x7f6730137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 1 monclient: found mon.noname-c 2024-09-15T21:19:10.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 1 -- 172.21.15.119:0/2245508008 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67300a1990 con 0x7f673013d5b0 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6735d39640 1 --2- 172.21.15.119:0/2245508008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f673013d5b0 0x7f673013f9a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6730069770 tx=0x7f672801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8e8b6b3c8e189c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 1 -- 172.21.15.119:0/2245508008 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67280a5640 con 0x7f673013d5b0 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: dump: 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.963 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 1 -- 172.21.15.119:0/2245508008 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67280a57e0 con 0x7f673013d5b0 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 5 monclient: authenticate success, global_id 14470 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:10.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 4 set_mon_vals no callback set 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 1 -- 172.21.15.119:0/2245508008 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67280a5bb0 con 0x7f673013d5b0 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f6727fff640 10 monclient: dump: 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.965 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 10 monclient: shutdown 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 -- 172.21.15.119:0/2245508008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f673013d5b0 msgr2=0x7f673013f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 --2- 172.21.15.119:0/2245508008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f673013d5b0 0x7f673013f9a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6730069770 tx=0x7f672801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 -- 172.21.15.119:0/2245508008 shutdown_connections 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 --2- 172.21.15.119:0/2245508008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f673013d5b0 0x7f673013f9a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 --2- 172.21.15.119:0/2245508008 >> v2:172.21.15.17:3300/0 conn(0x7f67301386a0 0x7f673013d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 -- 172.21.15.119:0/2245508008 >> 172.21.15.119:0/2245508008 conn(0x7f67300a6ef0 msgr2=0x7f67300a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 -- 172.21.15.119:0/2245508008 shutdown_connections 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 -- 172.21.15.119:0/2245508008 wait complete. 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 10 monclient: build_initial_monmap 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 10 monclient: monmap: 2024-09-15T21:19:10.970 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 Processor -- start 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 1 -- start start 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.953+0000 7f673779c640 10 monclient: init 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:10.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 0x7f67301cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): picked mon.noname-b con 0x7f6730137bb0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67301386a0 0x7f6730136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): picked mon.noname-c con 0x7f67301386a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f673013d5b0 0x7f67300713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): picked mon.noname-a con 0x7f673013d5b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f67300a62c0 con 0x7f673013d5b0 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f67300a1140 con 0x7f6730137bb0 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:10.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f67300a6440 con 0x7f67301386a0 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient(hunting): authenticate will time out at 1069.756459s 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 0x7f67301cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 0x7f67301cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53132/0 (socket says 172.21.15.119:53132) 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 -- 172.21.15.119:0/1309001426 learned_addr learned my addr 172.21.15.119:0/1309001426 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): get_auth_request con 0x7f6730137bb0 auth_method 0 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:10.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient(hunting): handle_auth_done global_id 14473 payload 824 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735d39640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f673013d5b0 0x7f67300713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 -- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67301386a0 msgr2=0x7f6730136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67301386a0 0x7f6730136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 -- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f673013d5b0 msgr2=0x7f67300713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f673013d5b0 0x7f67300713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient: _finish_hunting 0 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 monclient: found mon.noname-b 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 -- 172.21.15.119:0/1309001426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6730072eb0 con 0x7f6730137bb0 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6735538640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 0x7f67301cf790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c4c0 tx=0x7f672c001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c017070 con 0x7f6730137bb0 2024-09-15T21:19:10.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: dump: 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:10.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c003600 con 0x7f6730137bb0 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c0039f0 con 0x7f6730137bb0 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 5 monclient: authenticate success, global_id 14473 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _renew_subs 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6730073a10 con 0x7f6730137bb0 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f67277fe640 4 set_mon_vals no callback set 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 10 monclient: dump: 2024-09-15T21:19:10.976 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _renew_subs 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6730120e20 con 0x7f6730137bb0 2024-09-15T21:19:10.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f672c01a070 con 0x7f6730137bb0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6730137f90 con 0x7f6730137bb0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f671403b9c0 0x7f671403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f672c04ecc0 con 0x7f6730137bb0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6734d37640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f671403b9c0 0x7f671403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6734d37640 10 monclient: get_auth_request con 0x7f671403b9c0 auth_method 0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.957+0000 7f6734d37640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f671403b9c0 0x7f671403de80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67300aa2f0 tx=0x7f6720006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.961+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f672c010300 con 0x7f6730137bb0 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.961+0000 7f6725ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:10.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:10.961+0000 7f6725ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:11.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f673779c640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:11.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f673779c640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:11.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f673779c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6730074380 con 0x7f6730137bb0 2024-09-15T21:19:11.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v42) ==== 162+0+0 (secure 0 0 0) 0x7f672c010e20 con 0x7f6730137bb0 2024-09-15T21:19:11.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f6725ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:11.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.121+0000 7f6725ffb640 10 monclient: _finish_command 2 = system:0 pool 'fuggg' does not exist 2024-09-15T21:19:11.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f673779c640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:11.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f673779c640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:11.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f673779c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) -- 0x7f67301211c0 con 0x7f6730137bb0 2024-09-15T21:19:11.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f6725ffb640 1 -- 172.21.15.119:0/1309001426 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v42) ==== 162+0+0 (secure 0 0 0) 0x7f672c011d90 con 0x7f6730137bb0 2024-09-15T21:19:11.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f6725ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:11.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.221+0000 7f6725ffb640 10 monclient: _finish_command 3 = system:0 pool 'fuggg' does not exist 2024-09-15T21:19:11.230 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'fuggg' does not exist 2024-09-15T21:19:11.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f671403b9c0 msgr2=0x7f671403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f671403b9c0 0x7f671403de80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67300aa2f0 tx=0x7f6720006ca0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 10 monclient: shutdown 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 msgr2=0x7f67301cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 0x7f67301cf790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c4c0 tx=0x7f672c001e80 comp rx=0 tx=0).stop 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 shutdown_connections 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f671403b9c0 0x7f671403de80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f673013d5b0 0x7f67300713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67301386a0 0x7f6730136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 --2- 172.21.15.119:0/1309001426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6730137bb0 0x7f67301cf790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 >> 172.21.15.119:0/1309001426 conn(0x7f67300a6ef0 msgr2=0x7f67300a2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 shutdown_connections 2024-09-15T21:19:11.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.225+0000 7f673779c640 1 -- 172.21.15.119:0/1309001426 wait complete. 2024-09-15T21:19:11.248 INFO:tasks.workunit.client.0.smithi119.stdout:OK 2024-09-15T21:19:11.249 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo OK 2024-09-15T21:19:11.250 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:19:11.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:19:11.262 INFO:tasks.workunit:Running workunits matching mon/crush_ops.sh on client.0... 2024-09-15T21:19:11.263 INFO:tasks.workunit:Running workunit mon/crush_ops.sh... 2024-09-15T21:19:11.263 DEBUG:teuthology.orchestra.run.smithi119:workunit test mon/crush_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7498493279982dfe87f00616198a5967475f6169 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh 2024-09-15T21:19:11.316 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush dump 2024-09-15T21:19:11.352 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:19:11.352 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:11.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient: get_monmap_and_config 2024-09-15T21:19:11.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient: monmap: 2024-09-15T21:19:11.427 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:11.427 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.427 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:11.426041+0000 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:11.426041+0000 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:11.428 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 Processor -- start 2024-09-15T21:19:11.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 -- start start 2024-09-15T21:19:11.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient: init 2024-09-15T21:19:11.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90b409d4e0 0x7f90b409d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): picked mon.noname-c con 0x7f90b409d4e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f90b409f960 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409de00 0x7f90b4134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): picked mon.noname-b con 0x7f90b409de00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b409ef80 con 0x7f90b409f960 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f90b40746f0 con 0x7f90b409d4e0 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90b409e6e0 con 0x7f90b409de00 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b8a70640 10 monclient(hunting): authenticate will time out at 1070.222618s 2024-09-15T21:19:11.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90b409d4e0 0x7f90b409d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90b409d4e0 0x7f90b409d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53140/0 (socket says 172.21.15.119:53140) 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 -- 172.21.15.119:0/3718925731 learned_addr learned my addr 172.21.15.119:0/3718925731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): get_auth_request con 0x7f90b409d4e0 auth_method 0 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b2d9d640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409de00 0x7f90b4134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient(hunting): handle_auth_done global_id 14476 payload 274 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 -- 172.21.15.119:0/3718925731 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409de00 msgr2=0x7f90b4134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:11.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409de00 0x7f90b4134ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 -- 172.21.15.119:0/3718925731 >> v1:172.21.15.17:6789/0 conn(0x7f90b409f960 legacy=0x7f90b41347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 monclient: found mon.noname-c 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b259c640 1 -- 172.21.15.119:0/3718925731 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b409e9c0 con 0x7f90b409d4e0 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b2d9d640 10 monclient: get_auth_request con 0x7f90b409de00 auth_method 0 2024-09-15T21:19:11.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.421+0000 7f90b2d9d640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409de00 0x7f90b4134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b259c640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409d4e0 0x7f90b409d8c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f909c00bb10 tx=0x7f909c01eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24393aeac1b1afbd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 1 -- 172.21.15.119:0/3718925731 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0b4070 con 0x7f90b409d4e0 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: dump: 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.440 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 1 -- 172.21.15.119:0/3718925731 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c0a7950 con 0x7f90b409d4e0 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 1 -- 172.21.15.119:0/3718925731 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0a7d20 con 0x7f90b409d4e0 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 5 monclient: authenticate success, global_id 14476 2024-09-15T21:19:11.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 4 set_mon_vals no callback set 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b159a640 10 monclient: dump: 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.442 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: shutdown 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- 172.21.15.119:0/3718925731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409d4e0 msgr2=0x7f90b409d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409d4e0 0x7f90b409d8c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f909c00bb10 tx=0x7f909c01eaf0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- 172.21.15.119:0/3718925731 shutdown_connections 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409de00 0x7f90b4134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 --2- 172.21.15.119:0/3718925731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409d4e0 0x7f90b409d8c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- 172.21.15.119:0/3718925731 >> 172.21.15.119:0/3718925731 conn(0x7f90b406cdc0 msgr2=0x7f90b406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- 172.21.15.119:0/3718925731 shutdown_connections 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- 172.21.15.119:0/3718925731 wait complete. 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: monmap: 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 Processor -- start 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- start start 2024-09-15T21:19:11.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: init 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b409d4e0 0x7f90b413af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): picked mon.noname-a con 0x7f90b409d4e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 0x7f90b41306b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): picked mon.noname-b con 0x7f90b409de00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409f960 0x7f90b4130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): picked mon.noname-c con 0x7f90b409f960 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4135400 con 0x7f90b409d4e0 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f90b40a26f0 con 0x7f90b409de00 2024-09-15T21:19:11.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90b4135580 con 0x7f90b409f960 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b8a70640 10 monclient(hunting): authenticate will time out at 1070.227515s 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 0x7f90b41306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 0x7f90b41306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53142/0 (socket says 172.21.15.119:53142) 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 1 -- 172.21.15.119:0/2116264295 learned_addr learned my addr 172.21.15.119:0/2116264295 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): get_auth_request con 0x7f90b409de00 auth_method 0 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b259c640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b409d4e0 0x7f90b413af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b259c640 10 monclient(hunting): get_auth_request con 0x7f90b409d4e0 auth_method 0 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.425+0000 7f90b2d9d640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409f960 0x7f90b4130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b2d9d640 10 monclient(hunting): get_auth_request con 0x7f90b409f960 auth_method 0 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 10 monclient(hunting): handle_auth_done global_id 14479 payload 824 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 -- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409f960 msgr2=0x7f90b4130bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409f960 0x7f90b4130bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 -- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b409d4e0 msgr2=0x7f90b413af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b409d4e0 0x7f90b413af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 monclient: found mon.noname-b 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 -- 172.21.15.119:0/2116264295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b4131130 con 0x7f90b409de00 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b1d9b640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 0x7f90b41306b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c480 tx=0x7f90a400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4012830 con 0x7f90b409de00 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: dump: 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a4017070 con 0x7f90b409de00 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4012c30 con 0x7f90b409de00 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 10 monclient: dump: 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.450 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.452 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.452 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.452 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.452 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.452 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.452 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b0d99640 4 set_mon_vals no callback set 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 5 monclient: authenticate success, global_id 14479 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 monclient: _renew_subs 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b413b920 con 0x7f90b409de00 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 monclient: _renew_subs 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41c3e20 con 0x7f90b409de00 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b411d9a0 con 0x7f90b409de00 2024-09-15T21:19:11.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f90a401c460 con 0x7f90b409de00 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f909403ba10 0x7f909403ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f90a404f750 con 0x7f90b409de00 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b259c640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f909403ba10 0x7f909403ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b259c640 10 monclient: get_auth_request con 0x7f909403ba10 auth_method 0 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.429+0000 7f90b259c640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f909403ba10 0x7f909403ded0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f909c0b1000 tx=0x7f909c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.433+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f90a4012dd0 con 0x7f90b409de00 2024-09-15T21:19:11.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.433+0000 7f90aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.433+0000 7f90aaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:11.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient: get_monmap_and_config 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient: monmap: 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:11.483683+0000 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:11.483683+0000 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:11.487 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:11.487 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 1 Processor -- start 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 1 -- start start 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient: init 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9581378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa9581383b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa95813e4a0 0x7fa958140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient(hunting): picked mon.noname-a con 0x7fa95813e4a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9580a1a20 con 0x7fa9581378c0 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9580a1d00 con 0x7fa9581383b0 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa9580a1fe0 con 0x7fa95813e4a0 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ff1c640 10 monclient(hunting): authenticate will time out at 1074.879412s 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95dcb8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa9581378c0 0x7fa958137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43714/0 (socket says 172.21.15.17:43714) 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95dcb8640 1 -- 172.21.15.17:0/2734425274 learned_addr learned my addr 172.21.15.17:0/2734425274 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 1 --2- 172.21.15.17:0/2734425274 >> v2:172.21.15.17:3300/0 conn(0x7fa95813e4a0 0x7fa958140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): get_auth_request con 0x7fa95813e4a0 auth_method 0 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ccb6640 1 -- 172.21.15.17:0/2734425274 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274673612 0 0) 0x7fa9500014a0 con 0x7fa9581378c0 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ccb6640 10 monclient(hunting): _negotiate 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ccb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ccb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ccb6640 10 monclient(hunting): my global_id is 4710 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95ccb6640 1 -- 172.21.15.17:0/2734425274 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa940003530 con 0x7fa9581378c0 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_done global_id 4707 payload 274 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 1 -- 172.21.15.17:0/2734425274 >> v1:172.21.15.119:6789/0 conn(0x7fa9581383b0 legacy=0x7fa95813cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.478+0000 7fa95e4b9640 1 -- 172.21.15.17:0/2734425274 >> v1:172.21.15.17:6789/0 conn(0x7fa9581378c0 legacy=0x7fa958137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95e4b9640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95e4b9640 1 monclient: found mon.noname-a 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95e4b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95e4b9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95e4b9640 1 -- 172.21.15.17:0/2734425274 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9580a22c0 con 0x7fa95813e4a0 2024-09-15T21:19:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95e4b9640 1 --2- 172.21.15.17:0/2734425274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa95813e4a0 0x7fa958140890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9480097f0 tx=0x7fa94801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c38b6e110186c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 1 -- 172.21.15.17:0/2734425274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480b1070 con 0x7fa95813e4a0 2024-09-15T21:19:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: dump: 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 1 -- 172.21.15.17:0/2734425274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9480a47f0 con 0x7fa95813e4a0 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 1 -- 172.21.15.17:0/2734425274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480a4bc0 con 0x7fa95813e4a0 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 5 monclient: authenticate success, global_id 4707 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 4 set_mon_vals no callback set 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ccb6640 10 monclient: dump: 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 10 monclient: shutdown 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 -- 172.21.15.17:0/2734425274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa95813e4a0 msgr2=0x7fa958140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/2734425274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa95813e4a0 0x7fa958140890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9480097f0 tx=0x7fa94801c040 comp rx=0 tx=0).stop 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 -- 172.21.15.17:0/2734425274 shutdown_connections 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/2734425274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa95813e4a0 0x7fa958140890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 -- 172.21.15.17:0/2734425274 >> 172.21.15.17:0/2734425274 conn(0x7fa9580aac40 msgr2=0x7fa9580a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 -- 172.21.15.17:0/2734425274 shutdown_connections 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 1 -- 172.21.15.17:0/2734425274 wait complete. 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.482+0000 7fa95ff1c640 10 monclient: monmap: 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 Processor -- start 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- start start 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: init 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9581378c0 0x7fa95813a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): picked mon.noname-c con 0x7fa9581378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 0x7fa95813c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): picked mon.noname-a con 0x7fa9581383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95813e4a0 0x7fa958136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): picked mon.noname-b con 0x7fa95813e4a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa958144280 con 0x7fa9581383b0 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa958144100 con 0x7fa95813e4a0 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa958144400 con 0x7fa9581378c0 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient(hunting): authenticate will time out at 1074.884566s 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 0x7fa95813c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9581378c0 0x7fa95813a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 0x7fa95813c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45970/0 (socket says 172.21.15.17:45970) 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 -- 172.21.15.17:0/1814478044 learned_addr learned my addr 172.21.15.17:0/1814478044 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): get_auth_request con 0x7fa9581383b0 auth_method 0 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): get_auth_request con 0x7fa9581378c0 auth_method 0 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95813e4a0 0x7fa958136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): get_auth_request con 0x7fa95813e4a0 auth_method 0 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95e4b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient(hunting): handle_auth_done global_id 4713 payload 824 2024-09-15T21:19:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 -- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9581378c0 msgr2=0x7fa95813a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9581378c0 0x7fa95813a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 -- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95813e4a0 msgr2=0x7fa958136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95813e4a0 0x7fa958136d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 monclient: found mon.noname-a 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 -- 172.21.15.17:0/1814478044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9580a12f0 con 0x7fa9581383b0 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9581378c0 0x7fa95813a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95d4b7640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 0x7fa95813c680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c4b0 tx=0x7fa94c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0052d0 con 0x7fa9581383b0 2024-09-15T21:19:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: dump: 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c005470 con 0x7fa9581383b0 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 5 monclient: authenticate success, global_id 4713 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _renew_subs 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95813ab70 con 0x7fa9581383b0 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c012850 con 0x7fa9581383b0 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 10 monclient: dump: 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.508 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa957fff640 4 set_mon_vals no callback set 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _renew_subs 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa958137280 con 0x7fa9581383b0 2024-09-15T21:19:11.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958137ca0 con 0x7fa9581383b0 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa94c0129f0 con 0x7fa9581383b0 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa94403b9f0 0x7fa94403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fa94c04f510 con 0x7fa9581383b0 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa94403b9f0 0x7fa94403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 10 monclient: get_auth_request con 0x7fa94403b9f0 auth_method 0 2024-09-15T21:19:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.486+0000 7fa95dcb8640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa94403b9f0 0x7fa94403deb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa950005c10 tx=0x7fa950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.490+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa94c003db0 con 0x7fa9581383b0 2024-09-15T21:19:11.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.490+0000 7fa9567fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.490+0000 7fa9567fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:11.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.605+0000 7f90b8a70640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-09-15T21:19:11.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.605+0000 7f90b8a70640 10 monclient: _send_command 2 [{"prefix": "osd crush dump"}] 2024-09-15T21:19:11.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.605+0000 7f90b8a70640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.605+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush dump"} v 0) -- 0x7f90b409d8c0 con 0x7f90b409de00 2024-09-15T21:19:11.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.609+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v42) ==== 62+0+7331 (secure 0 0 0) 0x7f90a404f400 con 0x7f90b409de00 2024-09-15T21:19:11.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.609+0000 7f90aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush dump"}] 2024-09-15T21:19:11.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.609+0000 7f90aaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:11.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90b8a70640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-09-15T21:19:11.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90b8a70640 10 monclient: _send_command 3 [{"prefix": "osd crush dump"}] 2024-09-15T21:19:11.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90b8a70640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush dump"} v 0) -- 0x7f90b4131670 con 0x7f90b409de00 2024-09-15T21:19:11.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90aaffd640 1 -- 172.21.15.119:0/2116264295 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v42) ==== 62+0+7331 (secure 0 0 0) 0x7f90a401b3c0 con 0x7f90b409de00 2024-09-15T21:19:11.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90aaffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush dump"}] 2024-09-15T21:19:11.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.709+0000 7f90aaffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout:{ 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "devices": [ 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 0, 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd.0", 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "class": "ssd" 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 1, 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd.1", 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: "class": "ssd" 2024-09-15T21:19:11.718 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 2, 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd.2", 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "class": "ssd" 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 3, 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd.3", 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "class": "ssd" 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 4, 2024-09-15T21:19:11.719 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd.4", 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "class": "ssd" 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 5, 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd.5", 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "class": "ssd" 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "types": [ 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 0, 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "osd" 2024-09-15T21:19:11.720 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 1, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "host" 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 2, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "chassis" 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 3, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "rack" 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.721 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 4, 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "row" 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 5, 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "pdu" 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 6, 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "pod" 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.722 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 7, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "room" 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 8, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "datacenter" 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 9, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "zone" 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 10, 2024-09-15T21:19:11.723 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "region" 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 11, 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "root" 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "buckets": [ 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -1, 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "default", 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 11, 2024-09-15T21:19:11.724 INFO:tasks.workunit.client.0.smithi119.stdout: "type_name": "root", 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 34560, 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "alg": "straw2", 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "hash": "rjenkins1", 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "items": [ 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -3, 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 0 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -5, 2024-09-15T21:19:11.725 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 1 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -2, 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "default~ssd", 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 11, 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "type_name": "root", 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 34560, 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "alg": "straw2", 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "hash": "rjenkins1", 2024-09-15T21:19:11.726 INFO:tasks.workunit.client.0.smithi119.stdout: "items": [ 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -4, 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 0 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -6, 2024-09-15T21:19:11.727 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 1 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -3, 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "smithi017", 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 1, 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "type_name": "host", 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "alg": "straw2", 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "hash": "rjenkins1", 2024-09-15T21:19:11.728 INFO:tasks.workunit.client.0.smithi119.stdout: "items": [ 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 2, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 0 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 0, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 1 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 1, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.729 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 2 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -4, 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "smithi017~ssd", 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 1, 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "type_name": "host", 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "alg": "straw2", 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "hash": "rjenkins1", 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: "items": [ 2024-09-15T21:19:11.730 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 2, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 0 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 0, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 1 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 1, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 2 2024-09-15T21:19:11.731 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -5, 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "smithi119", 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 1, 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "type_name": "host", 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "alg": "straw2", 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "hash": "rjenkins1", 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "items": [ 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.732 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 3, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 0 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 5, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 1 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 4, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.733 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 2 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: "id": -6, 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: "name": "smithi119~ssd", 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: "type_id": 1, 2024-09-15T21:19:11.734 INFO:tasks.workunit.client.0.smithi119.stdout: "type_name": "host", 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 17280, 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "alg": "straw2", 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "hash": "rjenkins1", 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "items": [ 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 3, 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 0 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 5, 2024-09-15T21:19:11.735 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 1 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: "id": 4, 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: "weight": 5760, 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: "pos": 2 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: "rules": [ 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.736 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_id": 0, 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_name": "replicated_rule", 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "type": 1, 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "steps": [ 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "take", 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "item": -1, 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "item_name": "default" 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.737 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "choose_firstn", 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 0, 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "type": "osd" 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "emit" 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_id": 1, 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_name": "erasure-code", 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "type": 3, 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: "steps": [ 2024-09-15T21:19:11.738 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "set_chooseleaf_tries", 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 5 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "set_choose_tries", 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 100 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "take", 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "item": -1, 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "item_name": "default" 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.739 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "choose_indep", 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 0, 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: "type": "osd" 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "emit" 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:11.740 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "tunables": { 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "choose_local_tries": 0, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "choose_local_fallback_tries": 0, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "choose_total_tries": 50, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "chooseleaf_descend_once": 1, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "chooseleaf_vary_r": 1, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "chooseleaf_stable": 1, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "msr_descents": 100, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "msr_collision_tries": 100, 2024-09-15T21:19:11.741 INFO:tasks.workunit.client.0.smithi119.stdout: "straw_calc_version": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "allowed_bucket_algs": 54, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "profile": "jewel", 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "optimal_tunables": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "legacy_tunables": 0, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "minimum_required_version": "jewel", 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "require_feature_tunables": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "require_feature_tunables2": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "has_v2_rules": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "require_feature_tunables3": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "has_v3_rules": 0, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "has_v4_buckets": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "require_feature_tunables5": 1, 2024-09-15T21:19:11.742 INFO:tasks.workunit.client.0.smithi119.stdout: "has_v5_rules": 0, 2024-09-15T21:19:11.743 INFO:tasks.workunit.client.0.smithi119.stdout: "has_msr_rules": 0 2024-09-15T21:19:11.743 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:11.743 INFO:tasks.workunit.client.0.smithi119.stdout: "choose_args": {} 2024-09-15T21:19:11.743 INFO:tasks.workunit.client.0.smithi119.stdout:} 2024-09-15T21:19:11.743 INFO:tasks.workunit.client.0.smithi119.stdout: 2024-09-15T21:19:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa95ff1c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa95ff1c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa95ff1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa958069270 con 0x7fa9581383b0 2024-09-15T21:19:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa9567fc640 1 -- 172.21.15.17:0/1814478044 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fa94c010cf0 con 0x7fa9581383b0 2024-09-15T21:19:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa9567fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.730+0000 7fa9567fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:11.745 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:11.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f909403ba10 msgr2=0x7f909403ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f909403ba10 0x7f909403ded0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f909c0b1000 tx=0x7f909c008040 comp rx=0 tx=0).stop 2024-09-15T21:19:11.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 10 monclient: shutdown 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 msgr2=0x7f90b41306b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 0x7f90b41306b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c480 tx=0x7f90a400c940 comp rx=0 tx=0).stop 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 shutdown_connections 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f909403ba10 0x7f909403ded0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90b409f960 0x7f90b4130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b409de00 0x7f90b41306b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 --2- 172.21.15.119:0/2116264295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b409d4e0 0x7f90b413af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 >> 172.21.15.119:0/2116264295 conn(0x7f90b406cdc0 msgr2=0x7f90b40a5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 shutdown_connections 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.713+0000 7f90b8a70640 1 -- 172.21.15.119:0/2116264295 wait complete. 2024-09-15T21:19:11.746 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule dump 2024-09-15T21:19:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa94403b9f0 msgr2=0x7fa94403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa94403b9f0 0x7fa94403deb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa950005c10 tx=0x7fa950009040 comp rx=0 tx=0).stop 2024-09-15T21:19:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 10 monclient: shutdown 2024-09-15T21:19:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 msgr2=0x7fa95813c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 0x7fa95813c680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c4b0 tx=0x7fa94c00c970 comp rx=0 tx=0).stop 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 shutdown_connections 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa94403b9f0 0x7fa94403deb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95813e4a0 0x7fa958136d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9581383b0 0x7fa95813c680 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 --2- 172.21.15.17:0/1814478044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9581378c0 0x7fa95813a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 >> 172.21.15.17:0/1814478044 conn(0x7fa9580aac40 msgr2=0x7fa95806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 shutdown_connections 2024-09-15T21:19:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.734+0000 7fa95ff1c640 1 -- 172.21.15.17:0/1814478044 wait complete. 2024-09-15T21:19:11.758 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:19:11.758 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:19:11.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:19:11.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient: get_monmap_and_config 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient: monmap: 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:11.853863+0000 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:11.853863+0000 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.855 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.856 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:11.856 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:11.856 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:11.856 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:11.856 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:11.856 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 Processor -- start 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 -- start start 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient: init 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe974135ca0 0x7fe974136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): picked mon.noname-a con 0x7fe974135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9740a0160 0x7fe9740a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): picked mon.noname-b con 0x7fe9740a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe97406ec30 0x7fe9740a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): picked mon.noname-c con 0x7fe97406ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe9741365c0 con 0x7fe974135ca0 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe97409cf30 con 0x7fe97406ec30 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740a1ca0 con 0x7fe9740a0160 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe97b5f8640 10 monclient(hunting): authenticate will time out at 1070.650974s 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe97406ec30 0x7fe9740a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe97406ec30 0x7fe9740a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53146/0 (socket says 172.21.15.119:53146) 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 -- 172.21.15.119:0/1035040390 learned_addr learned my addr 172.21.15.119:0/1035040390 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): get_auth_request con 0x7fe97406ec30 auth_method 0 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979394640 1 --2- 172.21.15.119:0/1035040390 >> v2:172.21.15.17:3300/0 conn(0x7fe974135ca0 0x7fe974136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe978b93640 1 --2- 172.21.15.119:0/1035040390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9740a0160 0x7fe9740a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe978b93640 10 monclient(hunting): get_auth_request con 0x7fe9740a0160 auth_method 0 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe978b93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe978b93640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe978b93640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979394640 10 monclient(hunting): get_auth_request con 0x7fe974135ca0 auth_method 0 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979394640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979394640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979394640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient(hunting): handle_auth_done global_id 14488 payload 274 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 -- 172.21.15.119:0/1035040390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9740a0160 msgr2=0x7fe9740a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 --2- 172.21.15.119:0/1035040390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9740a0160 0x7fe9740a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 -- 172.21.15.119:0/1035040390 >> v2:172.21.15.17:3300/0 conn(0x7fe974135ca0 msgr2=0x7fe974136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 --2- 172.21.15.119:0/1035040390 >> v2:172.21.15.17:3300/0 conn(0x7fe974135ca0 0x7fe974136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 monclient: found mon.noname-c 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.849+0000 7fe979b95640 1 -- 172.21.15.119:0/1035040390 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9740a1f80 con 0x7fe97406ec30 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe979b95640 1 --2- 172.21.15.119:0/1035040390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe97406ec30 0x7fe9740a0f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9740a29c0 tx=0x7fe96c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc3a86e79bf72974 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 1 -- 172.21.15.119:0/1035040390 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0a5530 con 0x7fe97406ec30 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: dump: 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 1 -- 172.21.15.119:0/1035040390 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c0a56d0 con 0x7fe97406ec30 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 5 monclient: authenticate success, global_id 14488 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 1 -- 172.21.15.119:0/1035040390 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0a5aa0 con 0x7fe97406ec30 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 4 set_mon_vals no callback set 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe96bfff640 10 monclient: dump: 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 10 monclient: shutdown 2024-09-15T21:19:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 -- 172.21.15.119:0/1035040390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe97406ec30 msgr2=0x7fe9740a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/1035040390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe97406ec30 0x7fe9740a0f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9740a29c0 tx=0x7fe96c01c8b0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 -- 172.21.15.119:0/1035040390 shutdown_connections 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/1035040390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe97406ec30 0x7fe9740a0f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/1035040390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9740a0160 0x7fe9740a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/1035040390 >> v2:172.21.15.17:3300/0 conn(0x7fe974135ca0 0x7fe974136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 -- 172.21.15.119:0/1035040390 >> 172.21.15.119:0/1035040390 conn(0x7fe9740a2ae0 msgr2=0x7fe9740a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 -- 172.21.15.119:0/1035040390 shutdown_connections 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 -- 172.21.15.119:0/1035040390 wait complete. 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 10 monclient: monmap: 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.853+0000 7fe97b5f8640 1 Processor -- start 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- start start 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: init 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:11.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe97406ec30 0x7fe97413bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): picked mon.noname-c con 0x7fe97406ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 0x7fe97409d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): picked mon.noname-b con 0x7fe9740a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe974135ca0 0x7fe97409db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): picked mon.noname-a con 0x7fe974135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe97413f230 con 0x7fe974135ca0 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740a1870 con 0x7fe9740a0160 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe97413f3b0 con 0x7fe97406ec30 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient(hunting): authenticate will time out at 1070.656485s 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 0x7fe97409d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 0x7fe97409d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53160/0 (socket says 172.21.15.119:53160) 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 -- 172.21.15.119:0/3372354536 learned_addr learned my addr 172.21.15.119:0/3372354536 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): get_auth_request con 0x7fe9740a0160 auth_method 0 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979b95640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe974135ca0 0x7fe97409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979394640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe97406ec30 0x7fe97413bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979b95640 10 monclient(hunting): get_auth_request con 0x7fe974135ca0 auth_method 0 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979b95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979b95640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979b95640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient(hunting): handle_auth_done global_id 14491 payload 824 2024-09-15T21:19:11.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 -- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe97406ec30 msgr2=0x7fe97413bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe97406ec30 0x7fe97413bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 -- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe974135ca0 msgr2=0x7fe97409db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe974135ca0 0x7fe97409db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 monclient: found mon.noname-b 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 -- 172.21.15.119:0/3372354536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97409ef90 con 0x7fe9740a0160 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979394640 10 monclient: get_auth_request con 0x7fe97406ec30 auth_method 0 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979394640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe97406ec30 0x7fe97413bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe978b93640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 0x7fe97409d5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe96400d800 tx=0x7fe96400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400b6c0 con 0x7fe9740a0160 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: dump: 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964019070 con 0x7fe9740a0160 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400ba60 con 0x7fe9740a0160 2024-09-15T21:19:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 5 monclient: authenticate success, global_id 14491 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _renew_subs 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97413ab70 con 0x7fe9740a0160 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 10 monclient: dump: 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe96b7fe640 4 set_mon_vals no callback set 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _renew_subs 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe974132050 con 0x7fe9740a0160 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97413ad40 con 0x7fe9740a0160 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe96400bc00 con 0x7fe9740a0160 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe95803b9c0 0x7fe95803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fe96404fea0 con 0x7fe9740a0160 2024-09-15T21:19:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979394640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe95803b9c0 0x7fe95803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979394640 10 monclient: get_auth_request con 0x7fe95803b9c0 auth_method 0 2024-09-15T21:19:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.857+0000 7fe979394640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe95803b9c0 0x7fe95803de80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9700097c0 tx=0x7fe970008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.861+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe96401cd70 con 0x7fe9740a0160 2024-09-15T21:19:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.861+0000 7fe969ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:11.861+0000 7fe969ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient: get_monmap_and_config 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient: monmap: 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:11.874979+0000 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:11.874979+0000 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 Processor -- start 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 -- start start 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient: init 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2024-09-15T21:19:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa04c0a0890 0x7fa04c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): picked mon.noname-c con 0x7fa04c0a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa04c09ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): picked mon.noname-b con 0x7fa04c0a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c0a3130 con 0x7fa04c09ffc0 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c073df0 con 0x7fa04c0a0890 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0a1bf0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa0529b0640 10 monclient(hunting): authenticate will time out at 1075.270487s 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa04c09ffc0 0x7fa04c135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43728/0 (socket says 172.21.15.17:43728) 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): get_auth_request con 0x7fa04c0a11b0 auth_method 0 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04b7fe640 1 -- 172.21.15.17:0/4239659623 learned_addr learned my addr 172.21.15.17:0/4239659623 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 1 --2- 172.21.15.17:0/4239659623 >> v2:172.21.15.119:3300/0 conn(0x7fa04c0a0890 0x7fa04c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): get_auth_request con 0x7fa04c0a0890 auth_method 0 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient(hunting): handle_auth_done global_id 24134 payload 274 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04affd640 1 -- 172.21.15.17:0/4239659623 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067086705 0 0) 0x7fa04c0a3130 con 0x7fa04c09ffc0 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 1 -- 172.21.15.17:0/4239659623 >> v2:172.21.15.119:3300/0 conn(0x7fa04c0a0890 msgr2=0x7fa04c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 1 --2- 172.21.15.17:0/4239659623 >> v2:172.21.15.119:3300/0 conn(0x7fa04c0a0890 0x7fa04c0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 1 -- 172.21.15.17:0/4239659623 >> v1:172.21.15.17:6789/0 conn(0x7fa04c09ffc0 legacy=0x7fa04c135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 1 monclient: found mon.noname-b 2024-09-15T21:19:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa050f4d640 1 -- 172.21.15.17:0/4239659623 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0a15e0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04affd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.870+0000 7fa04bfff640 1 --2- 172.21.15.17:0/4239659623 >> v2:172.21.15.119:3300/0 conn(0x7fa04c0a0890 0x7fa04c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa050f4d640 1 --2- 172.21.15.17:0/4239659623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c135d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa044009dc0 tx=0x7fa04401c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b0eb00cd68dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 1 -- 172.21.15.17:0/4239659623 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440b2070 con 0x7fa04c0a11b0 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: dump: 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 5 monclient: authenticate success, global_id 24134 2024-09-15T21:19:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 1 -- 172.21.15.17:0/4239659623 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0440a57e0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 1 -- 172.21.15.17:0/4239659623 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440a5bb0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 4 set_mon_vals no callback set 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa04affd640 10 monclient: dump: 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: shutdown 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- 172.21.15.17:0/4239659623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 msgr2=0x7fa04c135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 --2- 172.21.15.17:0/4239659623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c135d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa044009dc0 tx=0x7fa04401c810 comp rx=0 tx=0).stop 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- 172.21.15.17:0/4239659623 shutdown_connections 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 --2- 172.21.15.17:0/4239659623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c135d80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 --2- 172.21.15.17:0/4239659623 >> v2:172.21.15.119:3300/0 conn(0x7fa04c0a0890 0x7fa04c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- 172.21.15.17:0/4239659623 >> 172.21.15.17:0/4239659623 conn(0x7fa04c0a66a0 msgr2=0x7fa04c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- 172.21.15.17:0/4239659623 shutdown_connections 2024-09-15T21:19:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- 172.21.15.17:0/4239659623 wait complete. 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: build_initial_monmap 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: monmap: 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:11.900 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 Processor -- start 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- start start 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: init 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c09ffc0 0x7fa04c12e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): picked mon.noname-a con 0x7fa04c09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c0a0890 0x7fa04c12ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): picked mon.noname-b con 0x7fa04c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c13aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): picked mon.noname-c con 0x7fa04c0a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c140510 con 0x7fa04c09ffc0 2024-09-15T21:19:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c1364d0 con 0x7fa04c0a0890 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c140690 con 0x7fa04c0a11b0 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.874+0000 7fa0529b0640 10 monclient(hunting): authenticate will time out at 1075.275932s 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c09ffc0 0x7fa04c12e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c13aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c09ffc0 0x7fa04c12e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45978/0 (socket says 172.21.15.17:45978) 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c13aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38266/0 (socket says 172.21.15.17:38266) 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 1 -- 172.21.15.17:0/2362464530 learned_addr learned my addr 172.21.15.17:0/2362464530 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): get_auth_request con 0x7fa04c09ffc0 auth_method 0 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): get_auth_request con 0x7fa04c0a11b0 auth_method 0 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c0a0890 0x7fa04c12ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 10 monclient(hunting): get_auth_request con 0x7fa04c0a0890 auth_method 0 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient(hunting): handle_auth_done global_id 24137 payload 824 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c0a0890 msgr2=0x7fa04c12ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c0a0890 0x7fa04c12ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c09ffc0 msgr2=0x7fa04c12e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c09ffc0 0x7fa04c12e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 monclient: found mon.noname-c 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c09e010 con 0x7fa04c0a11b0 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c0a0890 0x7fa04c12ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa050f4d640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c13aeb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa0440ac000 tx=0x7fa0440195b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440b2050 con 0x7fa04c0a11b0 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: dump: 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0440b3040 con 0x7fa04c0a11b0 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044003da0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: dump: 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 5 monclient: authenticate success, global_id 24137 2024-09-15T21:19:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 10 monclient: _renew_subs 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04a7fc640 4 set_mon_vals no callback set 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c06f5e0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 10 monclient: _renew_subs 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c13cad0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 0x7fa04c13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c09d0f0 con 0x7fa04c13b3f0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 0x7fa04c13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient: get_auth_request con 0x7fa04c13b3f0 auth_method 0 2024-09-15T21:19:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa0440a56a0 con 0x7fa04c0a11b0 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 10 monclient(hunting): handle_auth_done global_id 4728 payload 824 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa04c0a0c70 con 0x7fa04c13b3f0 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa03003b9c0 0x7fa03003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04bfff640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 0x7fa04c13b7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03c011c00 tx=0x7fa03c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fa0440ea030 con 0x7fa04c0a11b0 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c011cf0 con 0x7fa04c13b3f0 2024-09-15T21:19:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: dump: 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa03003b9c0 0x7fa03003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 10 monclient: get_auth_request con 0x7fa03003b9c0 auth_method 0 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa03c011cf0 con 0x7fa04c13b3f0 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 msgr2=0x7fa04c13b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 0x7fa04c13b7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03c011c00 tx=0x7fa03c00f610 comp rx=0 tx=0).stop 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 msgr2=0x7fa04c13b7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa048ff9640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c13b3f0 0x7fa04c13b7d0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03c011c00 tx=0x7fa03c00f610 comp rx=0 tx=0).stop 2024-09-15T21:19:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.878+0000 7fa04b7fe640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa03003b9c0 0x7fa03003de80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa04c09e220 tx=0x7fa034007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 0x7fa04c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c13bd10 con 0x7fa04c070ad0 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 0x7fa04c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient: get_auth_request con 0x7fa04c070ad0 auth_method 0 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:11.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 10 monclient(hunting): handle_auth_done global_id 4731 payload 824 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 1 -- 172.21.15.17:0/2362464530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa04c12f3d0 con 0x7fa04c070ad0 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa04bfff640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 0x7fa04c070ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa03c011c00 tx=0x7fa03c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c004020 con 0x7fa04c070ad0 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 10 monclient: dump: 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7fa03c02b480 con 0x7fa04c070ad0 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 msgr2=0x7fa04c070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 0x7fa04c070ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa03c011c00 tx=0x7fa03c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 msgr2=0x7fa04c070ee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa048ff9640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa04c070ad0 0x7fa04c070ee0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa03c011c00 tx=0x7fa03c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 117507, 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 7, 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:11.929 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa03003b9c0 msgr2=0x7fa03003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa03003b9c0 0x7fa03003de80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa04c09e220 tx=0x7fa034007be0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 10 monclient: shutdown 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 msgr2=0x7fa04c13aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c13aeb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa0440ac000 tx=0x7fa0440195b0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa050f4d640 1 -- 172.21.15.17:0/2362464530 reap_dead start 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 shutdown_connections 2024-09-15T21:19:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 --2- 172.21.15.17:0/2362464530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa04c0a11b0 0x7fa04c13aeb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:11.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 >> 172.21.15.17:0/2362464530 conn(0x7fa04c0a66a0 msgr2=0x7fa04c06fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:11.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 shutdown_connections 2024-09-15T21:19:11.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:11.886+0000 7fa0529b0640 1 -- 172.21.15.17:0/2362464530 wait complete. 2024-09-15T21:19:11.931 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:11.931 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:19:11.932 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:19:12.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.033+0000 7fe97b5f8640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-09-15T21:19:12.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.033+0000 7fe97b5f8640 10 monclient: _send_command 2 [{"prefix": "osd crush rule dump"}] 2024-09-15T21:19:12.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.033+0000 7fe97b5f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:12.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.033+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule dump"} v 0) -- 0x7fe974136080 con 0x7fe9740a0160 2024-09-15T21:19:12.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.037+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v42) ==== 67+0+1058 (secure 0 0 0) 0x7fe964013cb0 con 0x7fe9740a0160 2024-09-15T21:19:12.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.037+0000 7fe969ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule dump"}] 2024-09-15T21:19:12.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.037+0000 7fe969ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient: get_monmap_and_config 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient: monmap: 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:12.048086+0000 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:12.048086+0000 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:12.051 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 1 Processor -- start 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 1 -- start start 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient: init 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc5380a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc53809ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5380a1380 0x7fc53813e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient(hunting): picked mon.noname-a con 0x7fc5380a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380a3130 con 0x7fc53809ffc0 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc538073df0 con 0x7fc5380a0890 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380a1bf0 con 0x7fc5380a1380 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.042+0000 7fc53ef89640 10 monclient(hunting): authenticate will time out at 1075.443896s 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5380a1380 0x7fc53813e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc537fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc53809ffc0 0x7fc53813aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43736/0 (socket says 172.21.15.17:43736) 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc537fff640 1 -- 172.21.15.17:0/4159314452 learned_addr learned my addr 172.21.15.17:0/4159314452 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): get_auth_request con 0x7fc5380a1380 auth_method 0 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient(hunting): handle_auth_done global_id 4734 payload 274 2024-09-15T21:19:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 1 -- 172.21.15.17:0/4159314452 >> v1:172.21.15.119:6789/0 conn(0x7fc5380a0890 legacy=0x7fc5380a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 1 -- 172.21.15.17:0/4159314452 >> v1:172.21.15.17:6789/0 conn(0x7fc53809ffc0 legacy=0x7fc53813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 1 monclient: found mon.noname-a 2024-09-15T21:19:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 1 -- 172.21.15.17:0/4159314452 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53809ccd0 con 0x7fc5380a1380 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53d526640 1 --2- 172.21.15.17:0/4159314452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a1380 0x7fc53813e1f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc530009dc0 tx=0x7fc53001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37232026e99bfc21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 1 -- 172.21.15.17:0/4159314452 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5300b2070 con 0x7fc5380a1380 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: dump: 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 1 -- 172.21.15.17:0/4159314452 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5300a57e0 con 0x7fc5380a1380 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 1 -- 172.21.15.17:0/4159314452 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5300a5bb0 con 0x7fc5380a1380 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 5 monclient: authenticate success, global_id 4734 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 4 set_mon_vals no callback set 2024-09-15T21:19:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc5377fe640 10 monclient: dump: 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 10 monclient: shutdown 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 -- 172.21.15.17:0/4159314452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a1380 msgr2=0x7fc53813e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 --2- 172.21.15.17:0/4159314452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a1380 0x7fc53813e1f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc530009dc0 tx=0x7fc53001c810 comp rx=0 tx=0).stop 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 -- 172.21.15.17:0/4159314452 shutdown_connections 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 --2- 172.21.15.17:0/4159314452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a1380 0x7fc53813e1f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 -- 172.21.15.17:0/4159314452 >> 172.21.15.17:0/4159314452 conn(0x7fc5380a66a0 msgr2=0x7fc5380a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 -- 172.21.15.17:0/4159314452 shutdown_connections 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.046+0000 7fc53ef89640 1 -- 172.21.15.17:0/4159314452 wait complete. 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: monmap: 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 Processor -- start 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- start start 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: init 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc53809ffc0 0x7fc538134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): picked mon.noname-c con 0x7fc53809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a0890 0x7fc538135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): picked mon.noname-a con 0x7fc5380a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5380a1380 0x7fc538135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): picked mon.noname-b con 0x7fc5380a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381412e0 con 0x7fc5380a0890 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc538141160 con 0x7fc5380a1380 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc538141460 con 0x7fc53809ffc0 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): authenticate will time out at 1075.449149s 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc53809ffc0 0x7fc538134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc53809ffc0 0x7fc538134dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38272/0 (socket says 172.21.15.17:38272) 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 -- 172.21.15.17:0/1173930961 learned_addr learned my addr 172.21.15.17:0/1173930961 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): get_auth_request con 0x7fc53809ffc0 auth_method 0 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a0890 0x7fc538135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): get_auth_request con 0x7fc5380a0890 auth_method 0 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5380a1380 0x7fc538135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): get_auth_request con 0x7fc5380a1380 auth_method 0 2024-09-15T21:19:12.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient(hunting): handle_auth_done global_id 24140 payload 824 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5380a1380 msgr2=0x7fc538135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5380a1380 0x7fc538135850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a0890 msgr2=0x7fc538135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a0890 0x7fc538135310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 monclient: found mon.noname-c 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53809d0f0 con 0x7fc53809ffc0 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380a0890 0x7fc538135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53cd25640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc53809ffc0 0x7fc538134dd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c4b0 tx=0x7fc52800c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280052d0 con 0x7fc53809ffc0 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: dump: 2024-09-15T21:19:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc528005470 con 0x7fc53809ffc0 2024-09-15T21:19:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 5 monclient: authenticate success, global_id 24140 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: _renew_subs 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53811e9e0 con 0x7fc53809ffc0 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528012850 con 0x7fc53809ffc0 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: dump: 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc536ffd640 4 set_mon_vals no callback set 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: _renew_subs 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53813dbb0 con 0x7fc53809ffc0 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 0x7fc538138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc53811f090 con 0x7fc538138ab0 2024-09-15T21:19:12.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 0x7fc538138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient: get_auth_request con 0x7fc538138ab0 auth_method 0 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc5280129f0 con 0x7fc53809ffc0 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 10 monclient(hunting): handle_auth_done global_id 24143 payload 824 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5380a0c70 con 0x7fc538138ab0 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc52403b9c0 0x7fc52403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fc52804f540 con 0x7fc53809ffc0 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc537fff640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 0x7fc538138e90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5381361b0 tx=0x7fc52c00ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c005ad0 con 0x7fc538138ab0 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc5357fa640 10 monclient: dump: 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc52403b9c0 0x7fc52403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.050+0000 7fc53d526640 10 monclient: get_auth_request con 0x7fc52403b9c0 auth_method 0 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc52c018070 con 0x7fc538138ab0 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 msgr2=0x7fc538138e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 0x7fc538138e90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5381361b0 tx=0x7fc52c00ff70 comp rx=0 tx=0).stop 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 msgr2=0x7fc538138e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc5357fa640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc538138ab0 0x7fc538138e90 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5381361b0 tx=0x7fc52c00ff70 comp rx=0 tx=0).stop 2024-09-15T21:19:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.054+0000 7fc53d526640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc52403b9c0 0x7fc52403de80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5300af000 tx=0x7fc5300a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 0x7fc53811c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc53811ec20 con 0x7fc5381393d0 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 0x7fc53811c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient: get_auth_request con 0x7fc5381393d0 auth_method 0 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 10 monclient(hunting): handle_auth_done global_id 24146 payload 824 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 1 -- 172.21.15.17:0/1173930961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc53806d020 con 0x7fc5381393d0 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc537fff640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 0x7fc53811c4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc52c017000 tx=0x7fc52c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0072e0 con 0x7fc5381393d0 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 10 monclient: dump: 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7fc52c0072e0 con 0x7fc5381393d0 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 msgr2=0x7fc53811c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 0x7fc53811c4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc52c017000 tx=0x7fc52c005800 comp rx=0 tx=0).stop 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 msgr2=0x7fc53811c4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc5357fa640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc5381393d0 0x7fc53811c4e0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc52c017000 tx=0x7fc52c005800 comp rx=0 tx=0).stop 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7863, 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 7, 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:12.091 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:12.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.092 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:12.092 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc52403b9c0 msgr2=0x7fc52403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc52403b9c0 0x7fc52403de80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5300af000 tx=0x7fc5300a7070 comp rx=0 tx=0).stop 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 10 monclient: shutdown 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc53809ffc0 msgr2=0x7fc538134dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 --2- 172.21.15.17:0/1173930961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc53809ffc0 0x7fc538134dd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c4b0 tx=0x7fc52800c970 comp rx=0 tx=0).stop 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53d526640 1 -- 172.21.15.17:0/1173930961 reap_dead start 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53d526640 1 -- 172.21.15.17:0/1173930961 reap_dead start 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 shutdown_connections 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 >> 172.21.15.17:0/1173930961 conn(0x7fc5380a66a0 msgr2=0x7fc5380a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 shutdown_connections 2024-09-15T21:19:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.058+0000 7fc53ef89640 1 -- 172.21.15.17:0/1173930961 wait complete. 2024-09-15T21:19:12.095 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:12.095 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:19:12.095 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:19:12.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe97b5f8640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-09-15T21:19:12.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe97b5f8640 10 monclient: _send_command 3 [{"prefix": "osd crush rule dump"}] 2024-09-15T21:19:12.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe97b5f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:12.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule dump"} v 0) -- 0x7fe97409f8f0 con 0x7fe9740a0160 2024-09-15T21:19:12.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe969ffb640 1 -- 172.21.15.119:0/3372354536 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v42) ==== 67+0+1058 (secure 0 0 0) 0x7fe964013390 con 0x7fe9740a0160 2024-09-15T21:19:12.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe969ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule dump"}] 2024-09-15T21:19:12.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.141+0000 7fe969ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout:[ 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_id": 0, 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_name": "replicated_rule", 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "type": 1, 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "steps": [ 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "take", 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "item": -1, 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "item_name": "default" 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "choose_firstn", 2024-09-15T21:19:12.150 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 0, 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "type": "osd" 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "emit" 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_id": 1, 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "rule_name": "erasure-code", 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "type": 3, 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "steps": [ 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.151 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "set_chooseleaf_tries", 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 5 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "set_choose_tries", 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 100 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "take", 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "item": -1, 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "item_name": "default" 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "choose_indep", 2024-09-15T21:19:12.152 INFO:tasks.workunit.client.0.smithi119.stdout: "num": 0, 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: "type": "osd" 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: "op": "emit" 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout:] 2024-09-15T21:19:12.153 INFO:tasks.workunit.client.0.smithi119.stdout: 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe95803b9c0 msgr2=0x7fe95803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe95803b9c0 0x7fe95803de80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9700097c0 tx=0x7fe970008040 comp rx=0 tx=0).stop 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 10 monclient: shutdown 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 msgr2=0x7fe97409d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 0x7fe97409d5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe96400d800 tx=0x7fe96400dcc0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 shutdown_connections 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe95803b9c0 0x7fe95803de80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe974135ca0 0x7fe97409db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9740a0160 0x7fe97409d5c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 --2- 172.21.15.119:0/3372354536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe97406ec30 0x7fe97413bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 >> 172.21.15.119:0/3372354536 conn(0x7fe9740a2ae0 msgr2=0x7fe974135010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 shutdown_connections 2024-09-15T21:19:12.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.145+0000 7fe97b5f8640 1 -- 172.21.15.119:0/3372354536 wait complete. 2024-09-15T21:19:12.165 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.202+0000 7fb67dbff640 10 monclient: get_monmap_and_config 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.202+0000 7fb67dbff640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.202+0000 7fb67dbff640 10 monclient: monmap: 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:12.209928+0000 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:12.209928+0000 2024-09-15T21:19:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:12.213 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 1 Processor -- start 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 1 -- start start 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient: init 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6781340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb67809d6c0 0x7fb678138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient(hunting): picked mon.noname-c con 0x7fb67809d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb678139390 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb67809e1b0 con 0x7fb678139390 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb67809e490 con 0x7fb6781340c0 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb67809d520 con 0x7fb67809d6c0 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 10 monclient(hunting): authenticate will time out at 1075.605437s 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb678139390 0x7fb67813b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43748/0 (socket says 172.21.15.17:43748) 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 -- 172.21.15.17:0/1155175743 learned_addr learned my addr 172.21.15.17:0/1155175743 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 1 --2- 172.21.15.17:0/1155175743 >> v2:172.21.15.119:3300/0 conn(0x7fb67809d6c0 0x7fb678138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): get_auth_request con 0x7fb67809d6c0 auth_method 0 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414500515 0 0) 0x7fb67809e1b0 con 0x7fb678139390 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): _negotiate 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): my global_id is 4743 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb660003530 con 0x7fb678139390 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2502782644 0 0) 0x7fb660003530 con 0x7fb678139390 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 >> v2:172.21.15.119:3300/0 conn(0x7fb67809d6c0 msgr2=0x7fb678138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 --2- 172.21.15.17:0/1155175743 >> v2:172.21.15.119:3300/0 conn(0x7fb67809d6c0 0x7fb678138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 >> v1:172.21.15.119:6789/0 conn(0x7fb6781340c0 legacy=0x7fb6781344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67809e770 con 0x7fb678139390 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb670001a80 con 0x7fb678139390 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 5 monclient: authenticate success, global_id 4743 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: dump: 2024-09-15T21:19:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 >> v1:172.21.15.17:6789/0 conn(0x7fb678139390 legacy=0x7fb67813b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb660003a60 0x7fb660003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): picked mon.c con 0x7fb660003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb660004810 0x7fb660004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): picked mon.b con 0x7fb660004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6600053d0 0x7fb66000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): picked mon.a con 0x7fb6600053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb67809d520 con 0x7fb6600053d0 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb66000c240 con 0x7fb660004810 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb66000c440 con 0x7fb660003a60 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6767fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb67dbff640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb660003a60 0x7fb660003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): get_auth_request con 0x7fb660003a60 auth_method 0 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6600053d0 0x7fb66000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): get_auth_request con 0x7fb6600053d0 auth_method 0 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb660004810 0x7fb660004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): get_auth_request con 0x7fb660004810 auth_method 0 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient(hunting): handle_auth_done global_id 4743 payload 293 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 -- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb660003a60 msgr2=0x7fb660003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb660003a60 0x7fb660003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 -- 172.21.15.17:0/1155175743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb660004810 msgr2=0x7fb660004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb660004810 0x7fb660004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 monclient: found mon.a 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 1 -- 172.21.15.17:0/1155175743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660003530 con 0x7fb6600053d0 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb677fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.206+0000 7fb676ffd640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb660004810 0x7fb660004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 1 -- 172.21.15.17:0/1155175743 reap_dead start 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6600053d0 0x7fb66000b9d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb67000a8b0 tx=0x7fb67001d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59396ad02c6dc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67000a4d0 con 0x7fb6600053d0 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: dump: 2024-09-15T21:19:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6700ac620 con 0x7fb6600053d0 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 1 -- 172.21.15.17:0/1155175743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700aca10 con 0x7fb6600053d0 2024-09-15T21:19:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 4 set_mon_vals no callback set 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6767fc640 10 monclient: dump: 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: shutdown 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6600053d0 msgr2=0x7fb66000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6600053d0 0x7fb66000b9d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb67000a8b0 tx=0x7fb67001d2a0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- 172.21.15.17:0/1155175743 shutdown_connections 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 --2- 172.21.15.17:0/1155175743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6600053d0 0x7fb66000b9d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- 172.21.15.17:0/1155175743 >> 172.21.15.17:0/1155175743 conn(0x7fb6780a66a0 msgr2=0x7fb6780a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- 172.21.15.17:0/1155175743 shutdown_connections 2024-09-15T21:19:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- 172.21.15.17:0/1155175743 wait complete. 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: monmap: 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 Processor -- start 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- start start 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: init 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6600053d0 0x7fb6780738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): picked mon.noname-b con 0x7fb6600053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb67811c4e0 0x7fb67811c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): picked mon.noname-c con 0x7fb67811c4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb67811ce00 0x7fb6781d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): picked mon.noname-a con 0x7fb67811ce00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb678134830 con 0x7fb67811ce00 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781346b0 con 0x7fb6600053d0 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6781349b0 con 0x7fb67811c4e0 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb67dbff640 10 monclient(hunting): authenticate will time out at 1075.611469s 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb67811c4e0 0x7fb67811c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb67811ce00 0x7fb6781d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb67811ce00 0x7fb6781d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46020/0 (socket says 172.21.15.17:46020) 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb67811c4e0 0x7fb67811c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38304/0 (socket says 172.21.15.17:38304) 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 1 -- 172.21.15.17:0/4004882321 learned_addr learned my addr 172.21.15.17:0/4004882321 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): get_auth_request con 0x7fb67811ce00 auth_method 0 2024-09-15T21:19:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): get_auth_request con 0x7fb67811c4e0 auth_method 0 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6777fe640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6600053d0 0x7fb6780738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6777fe640 10 monclient(hunting): get_auth_request con 0x7fb6600053d0 auth_method 0 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb6777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient(hunting): handle_auth_done global_id 24149 payload 824 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6600053d0 msgr2=0x7fb6780738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6600053d0 0x7fb6780738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb67811ce00 msgr2=0x7fb6781d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb67811ce00 0x7fb6781d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 monclient: found mon.noname-c 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb676ffd640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67811f4a0 con 0x7fb67811c4e0 2024-09-15T21:19:12.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.210+0000 7fb677fff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb67811ce00 0x7fb6781d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb676ffd640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb67811c4e0 0x7fb67811c8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0105d0 tx=0x7fb66c010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c019070 con 0x7fb67811c4e0 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: dump: 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.240 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66c004220 con 0x7fb67811c4e0 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 5 monclient: authenticate success, global_id 24149 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 10 monclient: _renew_subs 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67811f780 con 0x7fb67811c4e0 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00b040 con 0x7fb67811c4e0 2024-09-15T21:19:12.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: dump: 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.242 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 10 monclient: _renew_subs 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67811f1c0 con 0x7fb67811c4e0 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb675ffb640 4 set_mon_vals no callback set 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 0x7fb67806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6780a0920 con 0x7fb67806f230 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb66c00e840 con 0x7fb67811c4e0 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 0x7fb67806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient: get_auth_request con 0x7fb67806f230 auth_method 0 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb66403b9f0 0x7fb66403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fb66c04faf0 con 0x7fb67811c4e0 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb677fff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb66403b9f0 0x7fb66403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb677fff640 10 monclient: get_auth_request con 0x7fb66403b9f0 auth_method 0 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 10 monclient(hunting): handle_auth_done global_id 14515 payload 824 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb67811d640 con 0x7fb67806f230 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb677fff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb66403b9f0 0x7fb66403deb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb670000f90 tx=0x7fb67809d490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb6777fe640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 0x7fb67806f610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6680014d0 tx=0x7fb668001990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6680067e0 con 0x7fb67806f230 2024-09-15T21:19:12.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: dump: 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb6680067e0 con 0x7fb67806f230 2024-09-15T21:19:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 msgr2=0x7fb67806f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 0x7fb67806f610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6680014d0 tx=0x7fb668001990 comp rx=0 tx=0).stop 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 msgr2=0x7fb67806f610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.214+0000 7fb65ffff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806f230 0x7fb67806f610 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6680014d0 tx=0x7fb668001990 comp rx=0 tx=0).stop 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb67dbff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb67dbff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 0x7fb6781da210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb67dbff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb67806d020 con 0x7fb67806fb50 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 0x7fb6781da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient: get_auth_request con 0x7fb67806fb50 auth_method 0 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 10 monclient(hunting): handle_auth_done global_id 14518 payload 824 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 1 -- 172.21.15.17:0/4004882321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb67811d210 con 0x7fb67806fb50 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb6777fe640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 0x7fb6781da210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6680014d0 tx=0x7fb668001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66801e760 con 0x7fb67806fb50 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 10 monclient: dump: 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7fb66801e760 con 0x7fb67806fb50 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 msgr2=0x7fb6781da210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 0x7fb6781da210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6680014d0 tx=0x7fb668001130 comp rx=0 tx=0).stop 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.222+0000 7fb65ffff640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 msgr2=0x7fb6781da210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb65ffff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb67806fb50 0x7fb6781da210 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6680014d0 tx=0x7fb668001130 comp rx=0 tx=0).stop 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 46946, 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:12.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 7, 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:12.250 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:12.251 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:12.252 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:12.253 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:12.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:12.255 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:12.256 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:12.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:12.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:12.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:12.261 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:12.262 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb66403b9f0 msgr2=0x7fb66403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb66403b9f0 0x7fb66403deb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb670000f90 tx=0x7fb67809d490 comp rx=0 tx=0).stop 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 10 monclient: shutdown 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb67811c4e0 msgr2=0x7fb67811c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 --2- 172.21.15.17:0/4004882321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb67811c4e0 0x7fb67811c8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0105d0 tx=0x7fb66c010a90 comp rx=0 tx=0).stop 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb677fff640 1 -- 172.21.15.17:0/4004882321 reap_dead start 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb677fff640 1 -- 172.21.15.17:0/4004882321 reap_dead start 2024-09-15T21:19:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 shutdown_connections 2024-09-15T21:19:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 >> 172.21.15.17:0/4004882321 conn(0x7fb6780a66a0 msgr2=0x7fb678132850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 shutdown_connections 2024-09-15T21:19:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.226+0000 7fb67dbff640 1 -- 172.21.15.17:0/4004882321 wait complete. 2024-09-15T21:19:12.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:12.264 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:19:12.264 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:19:12.264 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:19:12.264 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:19:12.265 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:19:12.265 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:19:12.276 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:19:12.276 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:19:12.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:12.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.273+0000 7f70f64bc640 10 monclient: get_monmap_and_config 2024-09-15T21:19:12.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.273+0000 7f70f64bc640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.273+0000 7f70f64bc640 10 monclient: monmap: 2024-09-15T21:19:12.281 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:12.281 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:12.280461+0000 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:12.280461+0000 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:12.282 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 Processor -- start 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 -- start start 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient: init 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70f00a3a80 0x7f70f00a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): picked mon.noname-c con 0x7f70f00a3a80 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70f009fb90 0x7f70f00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): picked mon.noname-a con 0x7f70f009fb90 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 0x7f70f01380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): picked mon.noname-b con 0x7f70f0137c90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f70f00a0010 con 0x7f70f009fb90 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f70f00a48e0 con 0x7f70f00a3a80 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f70f00a1950 con 0x7f70f0137c90 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 10 monclient(hunting): authenticate will time out at 1071.076965s 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70effff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f70f00a3a80 msgr2=0x7f70f00a3e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70effff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70f00a3a80 0x7f70f00a3e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 0x7f70f01380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 0x7f70f01380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:34952/0 (socket says 172.21.15.119:34952) 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 -- 172.21.15.119:0/2798878075 learned_addr learned my addr 172.21.15.119:0/2798878075 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): get_auth_request con 0x7f70f0137c90 auth_method 0 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 1 --2- 172.21.15.119:0/2798878075 >> v2:172.21.15.17:3300/0 conn(0x7f70f009fb90 0x7f70f00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): get_auth_request con 0x7f70f009fb90 auth_method 0 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient(hunting): handle_auth_done global_id 24152 payload 274 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 -- 172.21.15.119:0/2798878075 >> v2:172.21.15.119:3300/0 conn(0x7f70f00a3a80 msgr2=0x7f70f00a3e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 --2- 172.21.15.119:0/2798878075 >> v2:172.21.15.119:3300/0 conn(0x7f70f00a3a80 0x7f70f00a3e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 -- 172.21.15.119:0/2798878075 >> v2:172.21.15.17:3300/0 conn(0x7f70f009fb90 msgr2=0x7f70f00a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 --2- 172.21.15.119:0/2798878075 >> v2:172.21.15.17:3300/0 conn(0x7f70f009fb90 0x7f70f00a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 monclient: found mon.noname-b 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 -- 172.21.15.119:0/2798878075 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f006d980 con 0x7f70f0137c90 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f4a59640 1 --2- 172.21.15.119:0/2798878075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 0x7f70f01380e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009e50 tx=0x7f70e801c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7538e974044eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 1 -- 172.21.15.119:0/2798878075 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80a5550 con 0x7f70f0137c90 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 10 monclient: dump: 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.294 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 1 -- 172.21.15.119:0/2798878075 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e80a56f0 con 0x7f70f0137c90 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 5 monclient: authenticate success, global_id 24152 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70f64bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.277+0000 7f70eeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70eeffd640 1 -- 172.21.15.119:0/2798878075 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80a5ac0 con 0x7f70f0137c90 2024-09-15T21:19:12.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70eeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70eeffd640 10 monclient: dump: 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:12.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 4 set_mon_vals no callback set 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: shutdown 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/2798878075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 msgr2=0x7f70f01380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- 172.21.15.119:0/2798878075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 0x7f70f01380e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009e50 tx=0x7f70e801c8d0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/2798878075 shutdown_connections 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- 172.21.15.119:0/2798878075 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f0137c90 0x7f70f01380e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- 172.21.15.119:0/2798878075 >> v2:172.21.15.17:3300/0 conn(0x7f70f009fb90 0x7f70f00a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- 172.21.15.119:0/2798878075 >> v2:172.21.15.119:3300/0 conn(0x7f70f00a3a80 0x7f70f00a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/2798878075 >> 172.21.15.119:0/2798878075 conn(0x7f70f00a55c0 msgr2=0x7f70f006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/2798878075 shutdown_connections 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/2798878075 wait complete. 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: monmap: 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.298 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 Processor -- start 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- start start 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: init 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 0x7f70f01cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): picked mon.noname-c con 0x7f70f009fb90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00a3a80 0x7f70f0136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): picked mon.noname-b con 0x7f70f00a3a80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f0137c90 0x7f70f006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): picked mon.noname-a con 0x7f70f0137c90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70f0141770 con 0x7f70f0137c90 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70f00a2330 con 0x7f70f00a3a80 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f70f01418f0 con 0x7f70f009fb90 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient(hunting): authenticate will time out at 1071.081933s 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ef7fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00a3a80 msgr2=0x7f70f0136d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00a3a80 0x7f70f0136d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 0x7f70f01cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 0x7f70f01cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:34962/0 (socket says 172.21.15.119:34962) 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 -- 172.21.15.119:0/287287646 learned_addr learned my addr 172.21.15.119:0/287287646 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:12.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): get_auth_request con 0x7f70f009fb90 auth_method 0 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f0137c90 0x7f70f006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): get_auth_request con 0x7f70f0137c90 auth_method 0 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f4a59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient(hunting): handle_auth_done global_id 24155 payload 824 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 -- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00a3a80 msgr2=0x7f70f0136d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00a3a80 0x7f70f0136d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 -- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f0137c90 msgr2=0x7f70f006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f0137c90 0x7f70f006fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 monclient: found mon.noname-c 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 -- 172.21.15.119:0/287287646 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f0073960 con 0x7f70f009fb90 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70effff640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 0x7f70f01cf880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b580 tx=0x7f70e000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0003dd0 con 0x7f70f009fb90 2024-09-15T21:19:12.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: dump: 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e0015070 con 0x7f70f009fb90 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 5 monclient: authenticate success, global_id 24155 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _renew_subs 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f009f260 con 0x7f70f009fb90 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0005330 con 0x7f70f009fb90 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 10 monclient: dump: 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.304 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _renew_subs 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f00730c0 con 0x7f70f009fb90 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ee7fc640 4 set_mon_vals no callback set 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f01380e0 con 0x7f70f009fb90 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.281+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f70e00054d0 con 0x7f70f009fb90 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.285+0000 7f70ecff9640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f70dc03b9a0 0x7f70dc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.285+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f70e004f830 con 0x7f70f009fb90 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.285+0000 7f70ef7fe640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f70dc03b9a0 0x7f70dc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.285+0000 7f70ef7fe640 10 monclient: get_auth_request con 0x7f70dc03b9a0 auth_method 0 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.285+0000 7f70ef7fe640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f70dc03b9a0 0x7f70dc03de60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70f009df20 tx=0x7f70e4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.289+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f70e0051cf0 con 0x7f70f009fb90 2024-09-15T21:19:12.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.289+0000 7f70ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.289+0000 7f70ecff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient: get_monmap_and_config 2024-09-15T21:19:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient: monmap: 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:12.403532+0000 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:12.403532+0000 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:12.406 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 1 Processor -- start 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 1 -- start start 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient: init 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa54409d400 0x7fa54409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient(hunting): picked mon.noname-c con 0x7fa54409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa54409dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa544134d70 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5440a3130 con 0x7fa544134d70 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa54409e7a0 con 0x7fa54409dd20 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa5440a1bc0 con 0x7fa54409d400 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa54b938640 10 monclient(hunting): authenticate will time out at 1075.799100s 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa5496d4640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fa54409d400 msgr2=0x7fa54409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa5496d4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa54409d400 0x7fa54409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa548ed3640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fa54409dd20 legacy=0x7fa544134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa549ed5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa544134d70 0x7fa54413cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43764/0 (socket says 172.21.15.17:43764) 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa549ed5640 1 -- 172.21.15.17:0/3737481090 learned_addr learned my addr 172.21.15.17:0/3737481090 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 510833417 0 0) 0x7fa5440a3130 con 0x7fa544134d70 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 10 monclient(hunting): _negotiate 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 10 monclient(hunting): my global_id is 4755 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa530003530 con 0x7fa544134d70 2024-09-15T21:19:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.398+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 417424862 0 0) 0x7fa530003530 con 0x7fa544134d70 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 >> v2:172.21.15.119:3300/0 conn(0x7fa54409d400 msgr2=0x7fa54409d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 --2- 172.21.15.17:0/3737481090 >> v2:172.21.15.119:3300/0 conn(0x7fa54409d400 0x7fa54409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 >> v1:172.21.15.119:6789/0 conn(0x7fa54409dd20 legacy=0x7fa544134660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5440a0320 con 0x7fa544134d70 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa53c001a80 con 0x7fa544134d70 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: dump: 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 >> v1:172.21.15.17:6789/0 conn(0x7fa544134d70 legacy=0x7fa54413cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5300039c0 0x7fa530003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): picked mon.a con 0x7fa5300039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa530004810 0x7fa530004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): picked mon.c con 0x7fa530004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530005380 0x7fa53000dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): picked mon.b con 0x7fa530005380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa53000e4b0 con 0x7fa5300039c0 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa53000e660 con 0x7fa530005380 2024-09-15T21:19:12.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa53000e860 con 0x7fa530004810 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5300039c0 0x7fa530003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): get_auth_request con 0x7fa5300039c0 auth_method 0 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa530004810 0x7fa530004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa549ed5640 1 -- 172.21.15.17:0/3737481090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530005380 msgr2=0x7fa53000dc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): get_auth_request con 0x7fa530004810 auth_method 0 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa549ed5640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530005380 0x7fa53000dc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa548ed3640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient(hunting): handle_auth_done global_id 4755 payload 293 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 -- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa530004810 msgr2=0x7fa530004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa530004810 0x7fa530004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 -- 172.21.15.17:0/3737481090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530005380 msgr2=0x7fa53000dc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530005380 0x7fa53000dc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 monclient: found mon.a 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa549ed5640 1 -- 172.21.15.17:0/3737481090 reap_dead start 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 -- 172.21.15.17:0/3737481090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa530003530 con 0x7fa5300039c0 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 5 monclient: authenticate success, global_id 4755 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa5496d4640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5300039c0 0x7fa530003dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa540003940 tx=0x7fa54001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99e53162c4c5a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540009520 con 0x7fa5300039c0 2024-09-15T21:19:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: dump: 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa540003bc0 con 0x7fa5300039c0 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 4 set_mon_vals no callback set 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 1 -- 172.21.15.17:0/3737481090 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540005070 con 0x7fa5300039c0 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa53bfff640 10 monclient: dump: 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 10 monclient: shutdown 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 -- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5300039c0 msgr2=0x7fa530003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5300039c0 0x7fa530003dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa540003940 tx=0x7fa54001c420 comp rx=0 tx=0).stop 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 -- 172.21.15.17:0/3737481090 shutdown_connections 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 --2- 172.21.15.17:0/3737481090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5300039c0 0x7fa530003dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 -- 172.21.15.17:0/3737481090 >> 172.21.15.17:0/3737481090 conn(0x7fa5440a66a0 msgr2=0x7fa5440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 -- 172.21.15.17:0/3737481090 shutdown_connections 2024-09-15T21:19:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.402+0000 7fa54b938640 1 -- 172.21.15.17:0/3737481090 wait complete. 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: monmap: 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:12.423 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 Processor -- start 2024-09-15T21:19:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- start start 2024-09-15T21:19:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: init 2024-09-15T21:19:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 0x7fa5441330c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): picked mon.noname-c con 0x7fa5300039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa544133600 0x7fa54411d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): picked mon.noname-b con 0x7fa544133600 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544133a80 0x7fa54411d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): picked mon.noname-a con 0x7fa544133a80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa544134660 con 0x7fa544133a80 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5440a3490 con 0x7fa544133600 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa5441347e0 con 0x7fa5300039c0 2024-09-15T21:19:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient(hunting): authenticate will time out at 1075.804945s 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa548ed3640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa544133600 msgr2=0x7fa54411d060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa548ed3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa544133600 0x7fa54411d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 0x7fa5441330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544133a80 0x7fa54411d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 0x7fa5441330c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38322/0 (socket says 172.21.15.17:38322) 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): get_auth_request con 0x7fa544133a80 auth_method 0 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 1 -- 172.21.15.17:0/1923362459 learned_addr learned my addr 172.21.15.17:0/1923362459 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): get_auth_request con 0x7fa5300039c0 auth_method 0 2024-09-15T21:19:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient(hunting): handle_auth_done global_id 4758 payload 824 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 -- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 msgr2=0x7fa5441330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 0x7fa5441330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 -- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa544133600 msgr2=0x7fa54411d060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa544133600 0x7fa54411d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 monclient: found mon.noname-a 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 -- 172.21.15.17:0/1923362459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa544134290 con 0x7fa544133a80 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 0x7fa5441330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa549ed5640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544133a80 0x7fa54411d5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5440a19a0 tx=0x7fa53c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c007110 con 0x7fa544133a80 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: dump: 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 5 monclient: authenticate success, global_id 4758 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _renew_subs 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa53c0072f0 con 0x7fa544133a80 2024-09-15T21:19:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa544120d90 con 0x7fa544133a80 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0076e0 con 0x7fa544133a80 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 10 monclient: dump: 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _renew_subs 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54411ea30 con 0x7fa544133a80 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa53b7fe640 4 set_mon_vals no callback set 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54409e7a0 con 0x7fa544133a80 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa53c011070 con 0x7fa544133a80 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa52803b970 0x7fa52803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fa53c04f3d0 con 0x7fa544133a80 2024-09-15T21:19:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa52803b970 0x7fa52803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.406+0000 7fa5496d4640 10 monclient: get_auth_request con 0x7fa52803b970 auth_method 0 2024-09-15T21:19:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.410+0000 7fa5496d4640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa52803b970 0x7fa52803de30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa540003940 tx=0x7fa5400b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.410+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa53c01bd50 con 0x7fa544133a80 2024-09-15T21:19:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.410+0000 7fa539ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.410+0000 7fa539ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:12.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.461+0000 7f70f64bc640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:12.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.461+0000 7f70f64bc640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:12.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.461+0000 7f70f64bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.461+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f70f00732c0 con 0x7f70f009fb90 2024-09-15T21:19:12.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.465+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v42) ==== 65+0+29 (secure 0 0 0) 0x7f70e0010c30 con 0x7f70f009fb90 2024-09-15T21:19:12.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.465+0000 7f70ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:12.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.465+0000 7f70ecff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:12.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:12.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:12.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f70f0072160 con 0x7f70f009fb90 2024-09-15T21:19:12.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70ecff9640 1 -- 172.21.15.119:0/287287646 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v42) ==== 65+0+29 (secure 0 0 0) 0x7f70e000d360 con 0x7f70f009fb90 2024-09-15T21:19:12.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70ecff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:12.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70ecff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:12.579 INFO:tasks.workunit.client.0.smithi119.stdout:replicated_rule 2024-09-15T21:19:12.579 INFO:tasks.workunit.client.0.smithi119.stdout:erasure-code 2024-09-15T21:19:12.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f70dc03b9a0 msgr2=0x7f70dc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f70dc03b9a0 0x7f70dc03de60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70f009df20 tx=0x7f70e4009190 comp rx=0 tx=0).stop 2024-09-15T21:19:12.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 10 monclient: shutdown 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 msgr2=0x7f70f01cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 0x7f70f01cf880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b580 tx=0x7f70e000ba40 comp rx=0 tx=0).stop 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 shutdown_connections 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f70dc03b9a0 0x7f70dc03de60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f0137c90 0x7f70f006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00a3a80 0x7f70f0136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 --2- 172.21.15.119:0/287287646 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70f009fb90 0x7f70f01cf880 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 >> 172.21.15.119:0/287287646 conn(0x7f70f00a55c0 msgr2=0x7f70f009c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 shutdown_connections 2024-09-15T21:19:12.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.573+0000 7f70f64bc640 1 -- 172.21.15.119:0/287287646 wait complete. 2024-09-15T21:19:12.597 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule list 2024-09-15T21:19:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa54b938640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa54b938640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa54b938640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa544120b10 con 0x7fa544133a80 2024-09-15T21:19:12.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa539ffb640 1 -- 172.21.15.17:0/1923362459 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fa53c00ad60 con 0x7fa544133a80 2024-09-15T21:19:12.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa539ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:12.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.642+0000 7fa539ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:12.651 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:12.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa52803b970 msgr2=0x7fa52803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa52803b970 0x7fa52803de30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa540003940 tx=0x7fa5400b0040 comp rx=0 tx=0).stop 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 10 monclient: shutdown 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544133a80 msgr2=0x7fa54411d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544133a80 0x7fa54411d5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5440a19a0 tx=0x7fa53c00ccd0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 shutdown_connections 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa52803b970 0x7fa52803de30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544133a80 0x7fa54411d5a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa544133600 0x7fa54411d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 --2- 172.21.15.17:0/1923362459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300039c0 0x7fa5441330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 >> 172.21.15.17:0/1923362459 conn(0x7fa5440a66a0 msgr2=0x7fa5440a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 shutdown_connections 2024-09-15T21:19:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:12.646+0000 7fa54b938640 1 -- 172.21.15.17:0/1923362459 wait complete. 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.705+0000 7fed7be25640 10 monclient: get_monmap_and_config 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.705+0000 7fed7be25640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.705+0000 7fed7be25640 10 monclient: monmap: 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:12.711552+0000 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:12.711552+0000 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:12.713 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:12.714 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:12.714 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:12.714 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.705+0000 7fed7be25640 1 Processor -- start 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.705+0000 7fed7be25640 1 -- start start 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.705+0000 7fed7be25640 10 monclient: init 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): picked mon.noname-b con 0x7fed74135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fed740a0160 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:12.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed7406ec30 0x7fed740a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): picked mon.noname-c con 0x7fed7406ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed741365c0 con 0x7fed740a0160 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fed7409cf30 con 0x7fed7406ec30 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fed740a1ca0 con 0x7fed74135ca0 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient(hunting): authenticate will time out at 1071.508600s 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7a3c2640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fed7406ec30 msgr2=0x7fed740a10f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7a3c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed7406ec30 0x7fed740a10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed793c0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fed740a0160 0x7fed740a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:33410/0 (socket says 172.21.15.119:33410) 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed793c0640 1 -- 172.21.15.119:0/615805892 learned_addr learned my addr 172.21.15.119:0/615805892 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:12.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 --2- 172.21.15.119:0/615805892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): get_auth_request con 0x7fed74135ca0 auth_method 0 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient(hunting): handle_auth_done global_id 24161 payload 274 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 -- 172.21.15.119:0/615805892 >> v2:172.21.15.119:3300/0 conn(0x7fed7406ec30 msgr2=0x7fed740a10f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 --2- 172.21.15.119:0/615805892 >> v2:172.21.15.119:3300/0 conn(0x7fed7406ec30 0x7fed740a10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 -- 172.21.15.119:0/615805892 >> v1:172.21.15.17:6789/0 conn(0x7fed740a0160 legacy=0x7fed740a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 monclient: found mon.noname-b 2024-09-15T21:19:12.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 1 -- 172.21.15.119:0/615805892 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1520519796 0 0) 0x7fed741365c0 con 0x7fed740a0160 2024-09-15T21:19:12.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 -- 172.21.15.119:0/615805892 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740a1f80 con 0x7fed74135ca0 2024-09-15T21:19:12.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:12.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed79bc1640 1 --2- 172.21.15.119:0/615805892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74136080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed700097c0 tx=0x7fed7001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec7635c4d7d6c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 1 -- 172.21.15.119:0/615805892 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700b0070 con 0x7fed74135ca0 2024-09-15T21:19:12.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: dump: 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 1 -- 172.21.15.119:0/615805892 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed700ab600 con 0x7fed74135ca0 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 1 -- 172.21.15.119:0/615805892 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700ab9d0 con 0x7fed74135ca0 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed78bbf640 10 monclient: dump: 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.725 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 5 monclient: authenticate success, global_id 24161 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.709+0000 7fed7be25640 4 set_mon_vals no callback set 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: shutdown 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/615805892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 msgr2=0x7fed74136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 --2- 172.21.15.119:0/615805892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74136080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed700097c0 tx=0x7fed7001c380 comp rx=0 tx=0).stop 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/615805892 shutdown_connections 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 --2- 172.21.15.119:0/615805892 >> v2:172.21.15.119:3300/0 conn(0x7fed7406ec30 0x7fed740a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 --2- 172.21.15.119:0/615805892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74136080 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/615805892 >> 172.21.15.119:0/615805892 conn(0x7fed740a2ae0 msgr2=0x7fed740a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/615805892 shutdown_connections 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/615805892 wait complete. 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: build_initial_monmap 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: monmap: 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:12.727 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 Processor -- start 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- start start 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: init 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7406ec30 0x7fed7411eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): picked mon.noname-b con 0x7fed7406ec30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 0x7fed74133940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): picked mon.noname-a con 0x7fed740a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:12.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): picked mon.noname-c con 0x7fed74135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fed74140240 con 0x7fed740a0160 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fed740a1840 con 0x7fed7406ec30 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fed741403c0 con 0x7fed74135ca0 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient(hunting): authenticate will time out at 1071.513722s 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed79bc1640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7406ec30 msgr2=0x7fed7411eaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed79bc1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7406ec30 0x7fed7411eaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 0x7fed74133940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 0x7fed74133940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:50328/0 (socket says 172.21.15.119:50328) 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 1 -- 172.21.15.119:0/3604883126 learned_addr learned my addr 172.21.15.119:0/3604883126 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): get_auth_request con 0x7fed740a0160 auth_method 0 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): get_auth_request con 0x7fed74135ca0 auth_method 0 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient(hunting): handle_auth_done global_id 24164 payload 824 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 -- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7406ec30 msgr2=0x7fed7411eaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7406ec30 0x7fed7411eaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 -- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 msgr2=0x7fed74133940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 0x7fed74133940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient: _finish_hunting 0 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 monclient: found mon.noname-c 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 -- 172.21.15.119:0/3604883126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed741202f0 con 0x7fed74135ca0 2024-09-15T21:19:12.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed793c0640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 0x7fed74133940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7a3c2640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74133e80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fed7406f0c0 tx=0x7fed6c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00cd90 con 0x7fed74135ca0 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: dump: 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.732 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c010030 con 0x7fed74135ca0 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 5 monclient: authenticate success, global_id 24164 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _renew_subs 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c016670 con 0x7fed74135ca0 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed74135060 con 0x7fed74135ca0 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6bfff640 4 set_mon_vals no callback set 2024-09-15T21:19:12.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 10 monclient: dump: 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _renew_subs 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7411fa50 con 0x7fed74135ca0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed74136080 con 0x7fed74135ca0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.713+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fed6c0153e0 con 0x7fed74135ca0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.717+0000 7fed6a7fc640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed5803b970 0x7fed5803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.717+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fed6c04fce0 con 0x7fed74135ca0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.717+0000 7fed79bc1640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed5803b970 0x7fed5803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.717+0000 7fed79bc1640 10 monclient: get_auth_request con 0x7fed5803b970 auth_method 0 2024-09-15T21:19:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.717+0000 7fed79bc1640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed5803b970 0x7fed5803de30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed700ad000 tx=0x7fed70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:12.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.721+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fed6c014d50 con 0x7fed74135ca0 2024-09-15T21:19:12.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.721+0000 7fed6a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:12.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.721+0000 7fed6a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:12.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.881+0000 7fed7be25640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-09-15T21:19:12.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.881+0000 7fed7be25640 10 monclient: _send_command 2 [{"prefix": "osd crush rule list"}] 2024-09-15T21:19:12.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.881+0000 7fed7be25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:12.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:12.885+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule list"} v 0) -- 0x7fed7411fc50 con 0x7fed74135ca0 2024-09-15T21:19:15.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:15.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:15.713+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:15.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:15.713+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:15.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:15.713+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:15.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:15.713+0000 7fed697fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient: get_monmap_and_config 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient: build_initial_monmap 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient: monmap: 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:15.803779+0000 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:15.803779+0000 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:15.807 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 Processor -- start 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 -- start start 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient: init 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20100a4a90 0x7f20100a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): picked mon.noname-a con 0x7f20100a4a90 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f20100a53b0 0x7f20100a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): picked mon.noname-b con 0x7f20100a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2010137850 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20100a2c30 con 0x7f2010137850 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f20100a0f00 con 0x7f20100a4a90 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f20100a19b0 con 0x7f20100a53b0 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f20172b9640 10 monclient(hunting): authenticate will time out at 1079.199162s 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20100a4a90 0x7f20100a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f20100a53b0 0x7f20100a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20100a4a90 0x7f20100a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46050/0 (socket says 172.21.15.17:46050) 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 1 -- 172.21.15.17:0/380980926 learned_addr learned my addr 172.21.15.17:0/380980926 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): get_auth_request con 0x7f20100a53b0 auth_method 0 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): get_auth_request con 0x7f20100a4a90 auth_method 0 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2015055640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient(hunting): handle_auth_done global_id 24167 payload 274 2024-09-15T21:19:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 1 -- 172.21.15.17:0/380980926 >> v2:172.21.15.17:3300/0 conn(0x7f20100a4a90 msgr2=0x7f20100a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 1 --2- 172.21.15.17:0/380980926 >> v2:172.21.15.17:3300/0 conn(0x7f20100a4a90 0x7f20100a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 1 -- 172.21.15.17:0/380980926 >> v1:172.21.15.17:6789/0 conn(0x7f2010137850 legacy=0x7f2010137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient: _finish_hunting 0 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 1 monclient: found mon.noname-b 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.798+0000 7f2014854640 1 -- 172.21.15.17:0/380980926 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20100a1c90 con 0x7f20100a53b0 2024-09-15T21:19:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2014854640 1 --2- 172.21.15.17:0/380980926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f20100a53b0 0x7f20100a57d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20000097c0 tx=0x7f200001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61b97664275ff1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 1 -- 172.21.15.17:0/380980926 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000b0070 con 0x7f20100a53b0 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: dump: 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 1 -- 172.21.15.17:0/380980926 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20000ab600 con 0x7f20100a53b0 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 1 -- 172.21.15.17:0/380980926 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000ab9d0 con 0x7f20100a53b0 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 5 monclient: authenticate success, global_id 24167 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 4 set_mon_vals no callback set 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f2007fff640 10 monclient: dump: 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 10 monclient: shutdown 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 -- 172.21.15.17:0/380980926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f20100a53b0 msgr2=0x7f20100a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 --2- 172.21.15.17:0/380980926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f20100a53b0 0x7f20100a57d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20000097c0 tx=0x7f200001c330 comp rx=0 tx=0).stop 2024-09-15T21:19:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 -- 172.21.15.17:0/380980926 shutdown_connections 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 --2- 172.21.15.17:0/380980926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f20100a53b0 0x7f20100a57d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 --2- 172.21.15.17:0/380980926 >> v2:172.21.15.17:3300/0 conn(0x7f20100a4a90 0x7f20100a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 -- 172.21.15.17:0/380980926 >> 172.21.15.17:0/380980926 conn(0x7f20100aabb0 msgr2=0x7f20100a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 -- 172.21.15.17:0/380980926 shutdown_connections 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.802+0000 7f20172b9640 1 -- 172.21.15.17:0/380980926 wait complete. 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: build_initial_monmap 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: monmap: 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 Processor -- start 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- start start 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: init 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 0x7f20101cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): picked mon.noname-a con 0x7f20100a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20100a53b0 0x7f2010136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): picked mon.noname-b con 0x7f20100a53b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2010137850 0x7f201013bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): picked mon.noname-c con 0x7f2010137850 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f20101426d0 con 0x7f20100a4a90 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2010142550 con 0x7f20100a53b0 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2010142850 con 0x7f2010137850 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient(hunting): authenticate will time out at 1079.205142s 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 0x7f20101cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2010137850 0x7f201013bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 0x7f20101cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46066/0 (socket says 172.21.15.17:46066) 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 -- 172.21.15.17:0/3928137300 learned_addr learned my addr 172.21.15.17:0/3928137300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): get_auth_request con 0x7f20100a4a90 auth_method 0 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2014854640 1 -- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20100a53b0 msgr2=0x7f2010136960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): get_auth_request con 0x7f2010137850 auth_method 0 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2014854640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20100a53b0 0x7f2010136960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient(hunting): handle_auth_done global_id 4773 payload 824 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 -- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2010137850 msgr2=0x7f201013bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2010137850 0x7f201013bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 -- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20100a53b0 msgr2=0x7f2010136960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20100a53b0 0x7f2010136960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient: _finish_hunting 0 2024-09-15T21:19:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 monclient: found mon.noname-a 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 -- 172.21.15.17:0/3928137300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101212f0 con 0x7f20100a4a90 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015856640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2010137850 0x7f201013bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2015055640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 0x7f20101cfa90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f200c009d90 tx=0x7f200c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00cd40 con 0x7f20100a4a90 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: dump: 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c019070 con 0x7f20100a4a90 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c014680 con 0x7f20100a4a90 2024-09-15T21:19:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 5 monclient: authenticate success, global_id 4773 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _renew_subs 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20100a0710 con 0x7f20100a4a90 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20077fe640 4 set_mon_vals no callback set 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 10 monclient: dump: 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _renew_subs 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201013d5a0 con 0x7f20100a4a90 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2010120bc0 con 0x7f20100a4a90 2024-09-15T21:19:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f200c012050 con 0x7f20100a4a90 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ff403b970 0x7f1ff403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f200c04ff10 con 0x7f20100a4a90 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2014854640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ff403b970 0x7f1ff403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.806+0000 7f2014854640 10 monclient: get_auth_request con 0x7f1ff403b970 auth_method 0 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.810+0000 7f2014854640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ff403b970 0x7f1ff403de30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20000ad000 tx=0x7f2000008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.814+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2010120bc0 con 0x7f20100a4a90 2024-09-15T21:19:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.814+0000 7f2005ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:15.814+0000 7f2005ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f20172b9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f20172b9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f20172b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2010069200 con 0x7f20100a4a90 2024-09-15T21:19:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f2005ffb640 1 -- 172.21.15.17:0/3928137300 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f200c010040 con 0x7f20100a4a90 2024-09-15T21:19:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f2005ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.050+0000 7f2005ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:16.059 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ff403b970 msgr2=0x7f1ff403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ff403b970 0x7f1ff403de30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20000ad000 tx=0x7f2000008040 comp rx=0 tx=0).stop 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 10 monclient: shutdown 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 msgr2=0x7f20101cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 0x7f20101cfa90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f200c009d90 tx=0x7f200c00c490 comp rx=0 tx=0).stop 2024-09-15T21:19:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 shutdown_connections 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ff403b970 0x7f1ff403de30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2010137850 0x7f201013bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20100a53b0 0x7f2010136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 --2- 172.21.15.17:0/3928137300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20100a4a90 0x7f20101cfa90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 >> 172.21.15.17:0/3928137300 conn(0x7f20100aabb0 msgr2=0x7f201013ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 shutdown_connections 2024-09-15T21:19:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:16.054+0000 7f20172b9640 1 -- 172.21.15.17:0/3928137300 wait complete. 2024-09-15T21:19:16.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:16.717+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:16.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:16.717+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:16.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:16.717+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:17.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:17.713+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:17.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:17.713+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:17.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:17.713+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:18.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:18.713+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:18.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:18.713+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:18.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:18.713+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:19.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:19.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient: get_monmap_and_config 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient: build_initial_monmap 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient: monmap: 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:19.208032+0000 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:19.208032+0000 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:19.212 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:19.213 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:19.213 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 Processor -- start 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 -- start start 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient: init 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcaec1378c0 0x7fcaec137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): picked mon.noname-a con 0x7fcaec1378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): picked mon.noname-b con 0x7fcaec1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcaec0a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaec0a1d00 con 0x7fcaec0a5040 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0a1fe0 con 0x7fcaec1378c0 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec0a22c0 con 0x7fcaec1381e0 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf45dc640 10 monclient(hunting): authenticate will time out at 1082.603814s 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf1b77640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.202+0000 7fcaf2b79640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fcaec0a5040 legacy=0x7fcaec0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec0a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48632/0 (socket says 172.21.15.17:48632) 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 -- 172.21.15.17:0/3181353740 learned_addr learned my addr 172.21.15.17:0/3181353740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): get_auth_request con 0x7fcaec1381e0 auth_method 0 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 1 --2- 172.21.15.17:0/3181353740 >> v2:172.21.15.17:3300/0 conn(0x7fcaec1378c0 0x7fcaec137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): get_auth_request con 0x7fcaec1378c0 auth_method 0 2024-09-15T21:19:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf2378640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient(hunting): handle_auth_done global_id 24173 payload 274 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 -- 172.21.15.17:0/3181353740 >> v2:172.21.15.17:3300/0 conn(0x7fcaec1378c0 msgr2=0x7fcaec137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 --2- 172.21.15.17:0/3181353740 >> v2:172.21.15.17:3300/0 conn(0x7fcaec1378c0 0x7fcaec137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 -- 172.21.15.17:0/3181353740 >> v1:172.21.15.119:6789/0 conn(0x7fcaec0a5040 legacy=0x7fcaec0a54f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient: _finish_hunting 0 2024-09-15T21:19:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 monclient: found mon.noname-b 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 -- 172.21.15.17:0/3181353740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0a5c00 con 0x7fcaec1381e0 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1b77640 1 --2- 172.21.15.17:0/3181353740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec0a4b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009e50 tx=0x7fcae401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d464c9c8f9e00e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 1 -- 172.21.15.17:0/3181353740 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40a56c0 con 0x7fcaec1381e0 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: dump: 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: _finish_auth 0 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: _check_auth_tickets 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 5 monclient: authenticate success, global_id 24173 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 1 -- 172.21.15.17:0/3181353740 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae40a5860 con 0x7fcaec1381e0 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 4 set_mon_vals no callback set 2024-09-15T21:19:19.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 1 -- 172.21.15.17:0/3181353740 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40a5c30 con 0x7fcaec1381e0 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf1376640 10 monclient: dump: 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:19.226 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 10 monclient: shutdown 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 -- 172.21.15.17:0/3181353740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 msgr2=0x7fcaec0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/3181353740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec0a4b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009e50 tx=0x7fcae401c8f0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 -- 172.21.15.17:0/3181353740 shutdown_connections 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/3181353740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec0a4b00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/3181353740 >> v2:172.21.15.17:3300/0 conn(0x7fcaec1378c0 0x7fcaec137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 -- 172.21.15.17:0/3181353740 >> 172.21.15.17:0/3181353740 conn(0x7fcaec0aac40 msgr2=0x7fcaec0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 -- 172.21.15.17:0/3181353740 shutdown_connections 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 1 -- 172.21.15.17:0/3181353740 wait complete. 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 10 monclient: build_initial_monmap 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.206+0000 7fcaf45dc640 10 monclient: monmap: 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 Processor -- start 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- start start 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: init 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:19.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 0x7fcaec1cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): picked mon.noname-a con 0x7fcaec0a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaec1378c0 0x7fcaec136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): picked mon.noname-b con 0x7fcaec1378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec13cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): picked mon.noname-c con 0x7fcaec1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec06f340 con 0x7fcaec0a5040 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec06f1c0 con 0x7fcaec1378c0 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec1425c0 con 0x7fcaec1381e0 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:19.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient(hunting): authenticate will time out at 1082.608919s 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 0x7fcaec1cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 0x7fcaec1cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54438/0 (socket says 172.21.15.17:54438) 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 -- 172.21.15.17:0/2519042878 learned_addr learned my addr 172.21.15.17:0/2519042878 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): get_auth_request con 0x7fcaec0a5040 auth_method 0 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec13cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): get_auth_request con 0x7fcaec1381e0 auth_method 0 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf1b77640 1 -- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaec1378c0 msgr2=0x7fcaec136ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf1b77640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaec1378c0 0x7fcaec136ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2b79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient(hunting): handle_auth_done global_id 4779 payload 824 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 -- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 msgr2=0x7fcaec13cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec13cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 -- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaec1378c0 msgr2=0x7fcaec136ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaec1378c0 0x7fcaec136ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient: _finish_hunting 0 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 monclient: found mon.noname-a 2024-09-15T21:19:19.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 -- 172.21.15.17:0/2519042878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec13f290 con 0x7fcaec0a5040 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf2378640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 0x7fcaec1cf950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e930 tx=0x7fcae800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017070 con 0x7fcaec0a5040 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: dump: 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:19.232 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae800ca00 con 0x7fcaec0a5040 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 5 monclient: authenticate success, global_id 4779 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _renew_subs 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec0a89e0 con 0x7fcaec0a5040 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae800cdb0 con 0x7fcaec0a5040 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 10 monclient: dump: 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _renew_subs 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf0b75640 4 set_mon_vals no callback set 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec13e730 con 0x7fcaec0a5040 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec137ca0 con 0x7fcaec0a5040 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcae801a070 con 0x7fcaec0a5040 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcad803b9a0 0x7fcad803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.210+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7fcae8050270 con 0x7fcaec0a5040 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.214+0000 7fcaf1b77640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcad803b9a0 0x7fcad803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.214+0000 7fcaf1b77640 10 monclient: get_auth_request con 0x7fcad803b9a0 auth_method 0 2024-09-15T21:19:19.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.214+0000 7fcaf1b77640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcad803b9a0 0x7fcad803de60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcae40ae000 tx=0x7fcae40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:19.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.214+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcaec13f290 con 0x7fcaec0a5040 2024-09-15T21:19:19.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.214+0000 7fcae2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:19.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.214+0000 7fcae2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcaf45dc640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcaf45dc640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcaf45dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcaec06cba0 con 0x7fcaec0a5040 2024-09-15T21:19:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcae2ffd640 1 -- 172.21.15.17:0/2519042878 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fcae804ded0 con 0x7fcaec0a5040 2024-09-15T21:19:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcae2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.454+0000 7fcae2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:19.463 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcad803b9a0 msgr2=0x7fcad803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcad803b9a0 0x7fcad803de60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcae40ae000 tx=0x7fcae40a7070 comp rx=0 tx=0).stop 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 10 monclient: shutdown 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 msgr2=0x7fcaec1cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 0x7fcaec1cf950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e930 tx=0x7fcae800edf0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 shutdown_connections 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcad803b9a0 0x7fcad803de60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcaec1381e0 0x7fcaec13cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaec1378c0 0x7fcaec136ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 --2- 172.21.15.17:0/2519042878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaec0a5040 0x7fcaec1cf950 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 >> 172.21.15.17:0/2519042878 conn(0x7fcaec0aac40 msgr2=0x7fcaec06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 shutdown_connections 2024-09-15T21:19:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:19.458+0000 7fcaf45dc640 1 -- 172.21.15.17:0/2519042878 wait complete. 2024-09-15T21:19:19.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:19.717+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:19.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:19.717+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:19.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:19.717+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:20.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:20.713+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:20.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:20.713+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:20.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:20.713+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:21.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:21.713+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:21.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:21.713+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:21.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:21.713+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:22.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient: get_monmap_and_config 2024-09-15T21:19:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient: build_initial_monmap 2024-09-15T21:19:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient: monmap: 2024-09-15T21:19:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:22.615438+0000 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:22.615438+0000 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:22.618 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:22.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 Processor -- start 2024-09-15T21:19:22.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 -- start start 2024-09-15T21:19:22.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient: init 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 0x7ff2640a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): picked mon.noname-c con 0x7ff2640a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): picked mon.noname-b con 0x7ff2640a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff264137850 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2640a2c30 con 0x7ff264137850 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:22.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2640a0f00 con 0x7ff2640a4a90 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff2640a19b0 con 0x7ff2640a53b0 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26bad1640 10 monclient(hunting): authenticate will time out at 1086.011547s 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26986d640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 msgr2=0x7ff2640a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26986d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 0x7ff2640a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26a06e640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7ff264137850 legacy=0x7ff264137cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48642/0 (socket says 172.21.15.17:48642) 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 learned_addr learned my addr 172.21.15.17:0/2228682686 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 10 monclient(hunting): get_auth_request con 0x7ff2640a53b0 auth_method 0 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:22.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:22.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.610+0000 7ff26906c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:22.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.614+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:19:22.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.614+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:22.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.614+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:22.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.614+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:22.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:22.716+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:22.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:22.716+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:22.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:22.716+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:22.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.814+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 msgr2=0x7ff2640a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:22.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.814+0000 7ff26a06e640 1 -- 172.21.15.17:0/2228682686 >> v1:172.21.15.119:6789/0 conn(0x7ff264137850 legacy=0x7ff264137cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:22.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.814+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 0x7ff2640a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 10 monclient(hunting): get_auth_request con 0x7ff2640a53b0 auth_method 0 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:22.818+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:23.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.218+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 msgr2=0x7ff2640a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:23.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.218+0000 7ff26a06e640 1 -- 172.21.15.17:0/2228682686 >> v1:172.21.15.119:6789/0 conn(0x7ff264137850 legacy=0x7ff264137cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:23.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.218+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 0x7ff2640a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 10 monclient(hunting): get_auth_request con 0x7ff2640a53b0 auth_method 0 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:23.222+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:23.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:23.712+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:23.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:23.712+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:23.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:23.712+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:24.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.022+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 msgr2=0x7ff2640a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:24.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.022+0000 7ff26a06e640 1 -- 172.21.15.17:0/2228682686 >> v1:172.21.15.119:6789/0 conn(0x7ff264137850 legacy=0x7ff264137cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:24.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.022+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 0x7ff2640a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 10 monclient(hunting): get_auth_request con 0x7ff2640a53b0 auth_method 0 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:24.026+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:19:24.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:24.716+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:24.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:24.716+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:24.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:24.716+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:25.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): tick 2024-09-15T21:19:25.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:19:25.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 msgr2=0x7ff2640a57d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2640a53b0 0x7ff2640a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 msgr2=0x7ff2640a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.119:3300/0 conn(0x7ff2640a4a90 0x7ff2640a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 -- 172.21.15.17:0/2228682686 >> v1:172.21.15.119:6789/0 conn(0x7ff264137850 legacy=0x7ff264137cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient: _add_conns ranks=[3,0] 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7ff2480036b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff248006220 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 -- 172.21.15.17:0/2228682686 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2640a1c90 con 0x7ff248006220 2024-09-15T21:19:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 1 -- 172.21.15.17:0/2228682686 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff2640a0f00 con 0x7ff2480036b0 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff25b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 10 monclient(hunting): get_auth_request con 0x7ff2480036b0 auth_method 0 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:25.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.614+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:25.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:25.716+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:25.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:25.716+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:25.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:25.716+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:25.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:25.716+0000 7fed697fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 10 monclient(hunting): get_auth_request con 0x7ff2480036b0 auth_method 0 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:25.818+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 10 monclient(hunting): get_auth_request con 0x7ff2480036b0 auth_method 0 2024-09-15T21:19:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.222+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:26.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.716+0000 7fed697fa640 10 monclient: tick 2024-09-15T21:19:26.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.716+0000 7fed697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:26.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.716+0000 7fed697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.748+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v42) ==== 67+0+29 (secure 0 0 0) 0x7fed6c0060d0 con 0x7fed74135ca0 2024-09-15T21:19:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.748+0000 7fed6a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule list"}] 2024-09-15T21:19:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.748+0000 7fed6a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.762+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651164435 0 0) 0x7ff2640a1c90 con 0x7ff248006220 2024-09-15T21:19:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.762+0000 7ff26886b640 10 monclient(hunting): _negotiate 2024-09-15T21:19:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.762+0000 7ff26886b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.762+0000 7ff26886b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.762+0000 7ff26886b640 10 monclient(hunting): my global_id is 4782 2024-09-15T21:19:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.762+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff250003530 con 0x7ff248006220 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 88454003 0 0) 0x7ff250003530 con 0x7ff248006220 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 msgr2=0x7ff248005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 --2- 172.21.15.17:0/2228682686 >> v2:172.21.15.17:3300/0 conn(0x7ff2480036b0 0x7ff248005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24800af90 con 0x7ff248006220 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: _finish_auth 0 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff25400f040 con 0x7ff248006220 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: dump: 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 >> v1:172.21.15.17:6789/0 conn(0x7ff248006220 legacy=0x7ff2480086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26a06e640 1 -- 172.21.15.17:0/2228682686 reap_dead start 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2500039c0 0x7ff250003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): picked mon.a con 0x7ff2500039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff250004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): picked mon.c con 0x7ff250004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff250005220 0x7ff25000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): picked mon.b con 0x7ff250005220 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2640a0f00 con 0x7ff2500039c0 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff25000c150 con 0x7ff250005220 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff25000c350 con 0x7ff250004810 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26886b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26a06e640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff250005220 msgr2=0x7ff25000b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26a06e640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff250005220 0x7ff25000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2500039c0 0x7ff250003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): get_auth_request con 0x7ff2500039c0 auth_method 0 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff250004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): get_auth_request con 0x7ff250004810 auth_method 0 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26906c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient(hunting): handle_auth_done global_id 4782 payload 293 2024-09-15T21:19:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 msgr2=0x7ff250004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff250004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff250005220 msgr2=0x7ff25000b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff250005220 0x7ff25000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 monclient: found mon.a 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:26.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:26.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 1 -- 172.21.15.17:0/2228682686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24800af90 con 0x7ff2500039c0 2024-09-15T21:19:26.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient: _finish_auth 0 2024-09-15T21:19:26.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 10 monclient: _check_auth_tickets 2024-09-15T21:19:26.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26986d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:26.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26bad1640 5 monclient: authenticate success, global_id 4782 2024-09-15T21:19:26.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.770+0000 7ff26bad1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26986d640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2500039c0 0x7ff250003dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff260003cc0 tx=0x7ff26001dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e9a84abaaac9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260007d30 con 0x7ff2500039c0 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: dump: 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26bad1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2600a7730 con 0x7ff2500039c0 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26bad1640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 1 -- 172.21.15.17:0/2228682686 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600a7b20 con 0x7ff2500039c0 2024-09-15T21:19:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26bad1640 4 set_mon_vals no callback set 2024-09-15T21:19:26.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:26.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:26.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.778+0000 7ff26886b640 10 monclient: dump: 2024-09-15T21:19:26.792 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:26.793 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 10 monclient: shutdown 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 -- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2500039c0 msgr2=0x7ff250003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2500039c0 0x7ff250003dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff260003cc0 tx=0x7ff26001dbe0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 -- 172.21.15.17:0/2228682686 shutdown_connections 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff250005220 0x7ff25000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff250004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 --2- 172.21.15.17:0/2228682686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2500039c0 0x7ff250003dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 -- 172.21.15.17:0/2228682686 >> 172.21.15.17:0/2228682686 conn(0x7ff2640aabb0 msgr2=0x7ff2640a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 -- 172.21.15.17:0/2228682686 shutdown_connections 2024-09-15T21:19:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 -- 172.21.15.17:0/2228682686 wait complete. 2024-09-15T21:19:26.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 10 monclient: build_initial_monmap 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 10 monclient: monmap: 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 Processor -- start 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 1 -- start start 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.782+0000 7ff26bad1640 10 monclient: init 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2500039c0 0x7ff264136d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): picked mon.noname-b con 0x7ff2500039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff2641372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): picked mon.noname-c con 0x7ff250004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff250005220 0x7ff2641cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): picked mon.noname-a con 0x7ff250005220 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff264138650 con 0x7ff250005220 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641384d0 con 0x7ff2500039c0 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff26413ad00 con 0x7ff250004810 2024-09-15T21:19:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient(hunting): authenticate will time out at 1090.185179s 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26986d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2500039c0 msgr2=0x7ff264136d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26986d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2500039c0 0x7ff264136d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff2641372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff250005220 0x7ff2641cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff2641372b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48692/0 (socket says 172.21.15.17:48692) 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 1 -- 172.21.15.17:0/1629308953 learned_addr learned my addr 172.21.15.17:0/1629308953 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): get_auth_request con 0x7ff250004810 auth_method 0 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): get_auth_request con 0x7ff250005220 auth_method 0 2024-09-15T21:19:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient(hunting): handle_auth_done global_id 4785 payload 824 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 -- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 msgr2=0x7ff2641372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff2641372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 -- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2500039c0 msgr2=0x7ff264136d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2500039c0 0x7ff264136d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 monclient: found mon.noname-a 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 -- 172.21.15.17:0/1629308953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264125840 con 0x7ff250005220 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26906c640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff2641372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26a06e640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff250005220 0x7ff2641cc310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff2640a2670 tx=0x7ff25c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c016070 con 0x7ff250005220 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: dump: 2024-09-15T21:19:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:26.809 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c003d80 con 0x7ff250005220 2024-09-15T21:19:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0063b0 con 0x7ff250005220 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 10 monclient: dump: 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 5 monclient: authenticate success, global_id 4785 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25bfff640 4 set_mon_vals no callback set 2024-09-15T21:19:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _renew_subs 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff264124fa0 con 0x7ff250005220 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _renew_subs 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641cc850 con 0x7ff250005220 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff25c019070 con 0x7ff250005220 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26413ae80 con 0x7ff250005220 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.786+0000 7ff25a7fc640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff24803c5f0 0x7ff24803eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.790+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7ff25c04ec00 con 0x7ff250005220 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.790+0000 7ff26986d640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff24803c5f0 0x7ff24803eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.790+0000 7ff26986d640 10 monclient: get_auth_request con 0x7ff24803c5f0 auth_method 0 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.790+0000 7ff26986d640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff24803c5f0 0x7ff24803eab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b1000 tx=0x7ff2600b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.794+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff25c00f300 con 0x7ff250005220 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.794+0000 7ff25a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:26.794+0000 7ff25a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:26.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed7be25640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-09-15T21:19:26.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed7be25640 10 monclient: _send_command 3 [{"prefix": "osd crush rule list"}] 2024-09-15T21:19:26.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed7be25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:26.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule list"} v 0) -- 0x7fed7411caf0 con 0x7fed74135ca0 2024-09-15T21:19:26.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed6a7fc640 1 -- 172.21.15.119:0/3604883126 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v42) ==== 67+0+29 (secure 0 0 0) 0x7fed6c01dc20 con 0x7fed74135ca0 2024-09-15T21:19:26.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed6a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule list"}] 2024-09-15T21:19:26.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.844+0000 7fed6a7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:26.853 INFO:tasks.workunit.client.0.smithi119.stdout:replicated_rule 2024-09-15T21:19:26.853 INFO:tasks.workunit.client.0.smithi119.stdout:erasure-code 2024-09-15T21:19:26.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed5803b970 msgr2=0x7fed5803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:26.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed5803b970 0x7fed5803de30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed700ad000 tx=0x7fed70008040 comp rx=0 tx=0).stop 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 10 monclient: shutdown 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 msgr2=0x7fed74133e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74133e80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fed7406f0c0 tx=0x7fed6c009ed0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 shutdown_connections 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed5803b970 0x7fed5803de30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed74135ca0 0x7fed74133e80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed740a0160 0x7fed74133940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 --2- 172.21.15.119:0/3604883126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7406ec30 0x7fed7411eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 >> 172.21.15.119:0/3604883126 conn(0x7fed740a2ae0 msgr2=0x7fed7409ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 shutdown_connections 2024-09-15T21:19:26.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.852+0000 7fed7be25640 1 -- 172.21.15.119:0/3604883126 wait complete. 2024-09-15T21:19:26.870 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule create-simple foo default host 2024-09-15T21:19:26.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient: get_monmap_and_config 2024-09-15T21:19:26.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient: build_initial_monmap 2024-09-15T21:19:26.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient: monmap: 2024-09-15T21:19:26.993 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:26.993 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:26.994 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:26.990976+0000 2024-09-15T21:19:26.994 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:26.990976+0000 2024-09-15T21:19:26.994 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:26.994 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:26.994 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:26.994 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:26.995 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:26.995 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:26.995 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:26.995 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:26.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 Processor -- start 2024-09-15T21:19:26.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 -- start start 2024-09-15T21:19:26.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient: init 2024-09-15T21:19:27.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:27.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:19:27.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): picked mon.noname-b con 0x7f5ba8135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:27.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ba8135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:27.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ba8136430 0x7f5ba80a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): picked mon.noname-c con 0x7f5ba8136430 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:27.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba80a1d80 con 0x7f5ba8135fd0 2024-09-15T21:19:27.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba80a3600 con 0x7f5ba8136430 2024-09-15T21:19:27.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba80a2060 con 0x7f5ba8135bf0 2024-09-15T21:19:27.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:27.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5bacc1c640 10 monclient(hunting): authenticate will time out at 1085.787569s 2024-09-15T21:19:27.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba6d9d640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f5ba8136430 msgr2=0x7f5ba80a1160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:27.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ba8136430 0x7f5ba80a1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:27.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba5d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5ba8135fd0 0x7f5ba80a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60978/0 (socket says 172.21.15.119:60978) 2024-09-15T21:19:27.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba659c640 10 monclient(hunting): get_auth_request con 0x7f5ba8135bf0 auth_method 0 2024-09-15T21:19:27.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba5d9b640 1 -- 172.21.15.119:0/1019458292 learned_addr learned my addr 172.21.15.119:0/1019458292 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:27.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.988+0000 7f5ba659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 10 monclient(hunting): handle_auth_done global_id 24182 payload 274 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 1 -- 172.21.15.119:0/1019458292 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444688370 0 0) 0x7f5ba80a1d80 con 0x7f5ba8135fd0 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 1 -- 172.21.15.119:0/1019458292 >> v2:172.21.15.119:3300/0 conn(0x7f5ba8136430 msgr2=0x7f5ba80a1160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 1 --2- 172.21.15.119:0/1019458292 >> v2:172.21.15.119:3300/0 conn(0x7f5ba8136430 0x7f5ba80a1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 1 -- 172.21.15.119:0/1019458292 >> v1:172.21.15.17:6789/0 conn(0x7f5ba8135fd0 legacy=0x7f5ba80a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 1 monclient: found mon.noname-b 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 1 -- 172.21.15.119:0/1019458292 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba80a2340 con 0x7f5ba8135bf0 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba659c640 1 --2- 172.21.15.119:0/1019458292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba809fe70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b980097c0 tx=0x7f5b9801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d052c481927e781c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 1 -- 172.21.15.119:0/1019458292 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980b0070 con 0x7f5ba8135bf0 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: dump: 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:27.014 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: _finish_auth 0 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:27.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:27.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 5 monclient: authenticate success, global_id 24182 2024-09-15T21:19:27.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:27.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 1 -- 172.21.15.119:0/1019458292 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b980ab600 con 0x7f5ba8135bf0 2024-09-15T21:19:27.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 1 -- 172.21.15.119:0/1019458292 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980ab9d0 con 0x7f5ba8135bf0 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 4 set_mon_vals no callback set 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5ba559a640 10 monclient: dump: 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.017 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 10 monclient: shutdown 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1019458292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 msgr2=0x7f5ba809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1019458292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba809fe70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b980097c0 tx=0x7f5b9801c380 comp rx=0 tx=0).stop 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1019458292 shutdown_connections 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1019458292 >> v2:172.21.15.119:3300/0 conn(0x7f5ba8136430 0x7f5ba80a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1019458292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba809fe70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1019458292 >> 172.21.15.119:0/1019458292 conn(0x7f5ba80a66e0 msgr2=0x7f5ba80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1019458292 shutdown_connections 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.992+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1019458292 wait complete. 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: build_initial_monmap 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: monmap: 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:27.019 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 Processor -- start 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- start start 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: init 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:27.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba81343d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): picked mon.noname-c con 0x7f5ba8135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba8135fd0 0x7f5ba8134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): picked mon.noname-b con 0x7f5ba8135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 0x7f5ba811c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): picked mon.noname-a con 0x7f5ba8136430 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba80a1820 con 0x7f5ba8136430 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba80a16a0 con 0x7f5ba8135fd0 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba81402b0 con 0x7f5ba8135bf0 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient(hunting): authenticate will time out at 1085.793203s 2024-09-15T21:19:27.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba5d9b640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba8135fd0 msgr2=0x7f5ba8134910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba8135fd0 0x7f5ba8134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 0x7f5ba811c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba81343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 0x7f5ba811c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34266/0 (socket says 172.21.15.119:34266) 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 -- 172.21.15.119:0/1363662955 learned_addr learned my addr 172.21.15.119:0/1363662955 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): get_auth_request con 0x7f5ba8136430 auth_method 0 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): get_auth_request con 0x7f5ba8135bf0 auth_method 0 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient(hunting): handle_auth_done global_id 4791 payload 824 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 -- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 msgr2=0x7f5ba81343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba81343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 -- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba8135fd0 msgr2=0x7f5ba8134910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba8135fd0 0x7f5ba8134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 monclient: found mon.noname-a 2024-09-15T21:19:27.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 -- 172.21.15.119:0/1363662955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba811eec0 con 0x7f5ba8136430 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba6d9d640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 0x7f5ba811c6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000be90 tx=0x7f5ba000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0016430 con 0x7f5ba8136430 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: dump: 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:27.025 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 5 monclient: authenticate success, global_id 4791 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba0010030 con 0x7f5ba8136430 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0016810 con 0x7f5ba8136430 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5ba4d99640 4 set_mon_vals no callback set 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5b96ffd640 10 monclient: dump: 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _renew_subs 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba811f1a0 con 0x7f5ba8136430 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _renew_subs 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba809d240 con 0x7f5ba8136430 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:26.996+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba809fe70 con 0x7f5ba8136430 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.000+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5ba002e430 con 0x7f5ba8136430 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.000+0000 7f5b96ffd640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8803b970 0x7f5b8803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.000+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f5ba0050a40 con 0x7f5ba8136430 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.000+0000 7f5ba659c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8803b970 0x7f5b8803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.000+0000 7f5ba659c640 10 monclient: get_auth_request con 0x7f5b8803b970 auth_method 0 2024-09-15T21:19:27.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.000+0000 7f5ba659c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8803b970 0x7f5b8803de30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5b980ad000 tx=0x7f5b980a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.012+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5ba00069d0 con 0x7f5ba8136430 2024-09-15T21:19:27.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.012+0000 7f5b96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.012+0000 7f5b96ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:27.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.034+0000 7ff26bad1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:27.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.034+0000 7ff26bad1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:27.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.034+0000 7ff26bad1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.034+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff2641248d0 con 0x7ff250005220 2024-09-15T21:19:27.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff25a7fc640 1 -- 172.21.15.17:0/1629308953 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7ff25c00fe20 con 0x7ff250005220 2024-09-15T21:19:27.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff25a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:27.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff25a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:27.044 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":24,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff24803c5f0 msgr2=0x7ff24803eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff24803c5f0 0x7ff24803eab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b1000 tx=0x7ff2600b3040 comp rx=0 tx=0).stop 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 10 monclient: shutdown 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff250005220 msgr2=0x7ff2641cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff250005220 0x7ff2641cc310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff2640a2670 tx=0x7ff25c009850 comp rx=0 tx=0).stop 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 shutdown_connections 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff24803c5f0 0x7ff24803eab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff250005220 0x7ff2641cc310 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff250004810 0x7ff2641372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 --2- 172.21.15.17:0/1629308953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2500039c0 0x7ff264136d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 >> 172.21.15.17:0/1629308953 conn(0x7ff2640aabb0 msgr2=0x7ff2641bffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 shutdown_connections 2024-09-15T21:19:27.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.038+0000 7ff26bad1640 1 -- 172.21.15.17:0/1629308953 wait complete. 2024-09-15T21:19:27.063 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:19:27.063 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:19:27.064 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:19:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.174+0000 7f8295b1c640 10 monclient: get_monmap_and_config 2024-09-15T21:19:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.174+0000 7f8295b1c640 10 monclient: build_initial_monmap 2024-09-15T21:19:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.174+0000 7f8295b1c640 10 monclient: monmap: 2024-09-15T21:19:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:27.181284+0000 2024-09-15T21:19:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:27.181284+0000 2024-09-15T21:19:27.185 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:27.185 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.185 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:27.185 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:27.185 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:27.185 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:27.186 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:27.186 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.184+0000 7f5bacc1c640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:27.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.184+0000 7f5bacc1c640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:27.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.184+0000 7f5bacc1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.184+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f5ba813d630 con 0x7f5ba8136430 2024-09-15T21:19:27.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 Processor -- start 2024-09-15T21:19:27.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 -- start start 2024-09-15T21:19:27.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient: init 2024-09-15T21:19:27.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:27.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:19:27.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 0x7f82900a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): picked mon.noname-b con 0x7f82900a2920 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f82900a3240 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8290135df0 0x7f829013db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): picked mon.noname-c con 0x7f8290135df0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f829009ecf0 con 0x7f82900a3240 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f829009ea10 con 0x7f8290135df0 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f82900a1230 con 0x7f82900a2920 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f8295b1c640 10 monclient(hunting): authenticate will time out at 1090.577270s 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828ffff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f8290135df0 msgr2=0x7f829013db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828effd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f82900a3240 legacy=0x7f82901356e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8290135df0 0x7f829013db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 0x7f82900a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 0x7f82900a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48702/0 (socket says 172.21.15.17:48702) 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 -- 172.21.15.17:0/3411045885 learned_addr learned my addr 172.21.15.17:0/3411045885 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): get_auth_request con 0x7f82900a2920 auth_method 0 2024-09-15T21:19:27.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient(hunting): handle_auth_done global_id 24188 payload 274 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 -- 172.21.15.17:0/3411045885 >> v2:172.21.15.119:3300/0 conn(0x7f8290135df0 msgr2=0x7f829013db50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 --2- 172.21.15.17:0/3411045885 >> v2:172.21.15.119:3300/0 conn(0x7f8290135df0 0x7f829013db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 -- 172.21.15.17:0/3411045885 >> v1:172.21.15.119:6789/0 conn(0x7f82900a3240 legacy=0x7f82901356e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.178+0000 7f828f7fe640 1 -- 172.21.15.17:0/3411045885 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829006d1b0 con 0x7f82900a2920 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828f7fe640 1 --2- 172.21.15.17:0/3411045885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 0x7f82900a2d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8280009870 tx=0x7f828001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4253d1619060b551 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 1 -- 172.21.15.17:0/3411045885 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800b1070 con 0x7f82900a2920 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: dump: 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 5 monclient: authenticate success, global_id 24188 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 1 -- 172.21.15.17:0/3411045885 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82800a8050 con 0x7f82900a2920 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:27.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 1 -- 172.21.15.17:0/3411045885 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800ac860 con 0x7f82900a2920 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 4 set_mon_vals no callback set 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828e7fc640 10 monclient: dump: 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: shutdown 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3411045885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 msgr2=0x7f82900a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3411045885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 0x7f82900a2d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8280009870 tx=0x7f828001c440 comp rx=0 tx=0).stop 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3411045885 shutdown_connections 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3411045885 >> v2:172.21.15.119:3300/0 conn(0x7f8290135df0 0x7f829013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3411045885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a2920 0x7f82900a2d00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3411045885 >> 172.21.15.17:0/3411045885 conn(0x7f82900a7510 msgr2=0x7f82900a3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3411045885 shutdown_connections 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3411045885 wait complete. 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: build_initial_monmap 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: monmap: 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:27.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 Processor -- start 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- start start 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: init 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82900a2920 0x7f82901cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): picked mon.noname-b con 0x7f82900a2920 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a3240 0x7f8290135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): picked mon.noname-c con 0x7f82900a3240 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135df0 0x7f82901cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): picked mon.noname-a con 0x7f8290135df0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f829006cf20 con 0x7f8290135df0 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f829006cda0 con 0x7f82900a2920 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f82901408b0 con 0x7f82900a3240 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient(hunting): authenticate will time out at 1090.582549s 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828f7fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82900a2920 msgr2=0x7f82901cc890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82900a2920 0x7f82901cc890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a3240 0x7f8290135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a3240 0x7f8290135440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48704/0 (socket says 172.21.15.17:48704) 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 -- 172.21.15.17:0/3242540319 learned_addr learned my addr 172.21.15.17:0/3242540319 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): get_auth_request con 0x7f82900a3240 auth_method 0 2024-09-15T21:19:27.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135df0 0x7f82901cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): get_auth_request con 0x7f8290135df0 auth_method 0 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient(hunting): handle_auth_done global_id 24191 payload 824 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82900a2920 msgr2=0x7f82901cc890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82900a2920 0x7f82901cc890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135df0 msgr2=0x7f82901cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135df0 0x7f82901cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 monclient: found mon.noname-c 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82900a0200 con 0x7f82900a3240 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828ffff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135df0 0x7f82901cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828effd640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a3240 0x7f8290135440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f828400b4e0 tx=0x7f828400b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284003d80 con 0x7f82900a3240 2024-09-15T21:19:27.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: dump: 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8284015070 con 0x7f82900a3240 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 5 monclient: authenticate success, global_id 24191 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: _renew_subs 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829009e100 con 0x7f82900a3240 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840052b0 con 0x7f82900a3240 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8277fff640 10 monclient: dump: 2024-09-15T21:19:27.226 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f828dffb640 4 set_mon_vals no callback set 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: _renew_subs 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.182+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8290121df0 con 0x7f82900a3240 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8295b1c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 0x7f82901d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82900a2d00 con 0x7f8290135980 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 0x7f82901d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient: get_auth_request con 0x7f8290135980 auth_method 0 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8284005450 con 0x7f82900a3240 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 10 monclient(hunting): handle_auth_done global_id 4797 payload 824 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82901cd2f0 con 0x7f8290135980 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f827c03b970 0x7f827c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f828404f730 con 0x7f82900a3240 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828f7fe640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 0x7f82901d0310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82900a7f20 tx=0x7f828001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800bb030 con 0x7f8290135980 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:27.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 10 monclient: dump: 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828ffff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f827c03b970 0x7f827c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f82800a8040 con 0x7f8290135980 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828ffff640 10 monclient: get_auth_request con 0x7f827c03b970 auth_method 0 2024-09-15T21:19:27.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 msgr2=0x7f82901d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 0x7f82901d0310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82900a7f20 tx=0x7f828001ac80 comp rx=0 tx=0).stop 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 msgr2=0x7f82901d0310 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f8277fff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8290135980 0x7f82901d0310 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82900a7f20 tx=0x7f828001ac80 comp rx=0 tx=0).stop 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.186+0000 7f828ffff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f827c03b970 0x7f827c03de30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f829009d8b0 tx=0x7f828800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f8295b1c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 0x7f8290138b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f8295b1c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82901d0c60 con 0x7f82901d0850 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 0x7f8290138b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient: get_auth_request con 0x7f82901d0850 auth_method 0 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 10 monclient(hunting): handle_auth_done global_id 4800 payload 824 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.190+0000 7f828f7fe640 1 -- 172.21.15.17:0/3242540319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f829006e050 con 0x7f82901d0850 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f828f7fe640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 0x7f8290138b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f828001c270 tx=0x7f82800191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800bb040 con 0x7f82901d0850 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 10 monclient: dump: 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.232 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4703 (secure 0 0 0) 0x7f82800c1ce0 con 0x7f82901d0850 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 msgr2=0x7f8290138b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 0x7f8290138b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f828001c270 tx=0x7f82800191e0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 msgr2=0x7f8290138b20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8277fff640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82901d0850 0x7f8290138b20 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f828001c270 tx=0x7f82800191e0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 24, 2024-09-15T21:19:27.234 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 132815, 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:27.235 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:27.236 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:27.237 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:27.238 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.239 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:27.241 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:27.242 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:27.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:27.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:27.246 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.247 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.249 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:27.249 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:27.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f827c03b970 msgr2=0x7f827c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f827c03b970 0x7f827c03de30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f829009d8b0 tx=0x7f828800d040 comp rx=0 tx=0).stop 2024-09-15T21:19:27.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 10 monclient: shutdown 2024-09-15T21:19:27.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a3240 msgr2=0x7f8290135440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 --2- 172.21.15.17:0/3242540319 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82900a3240 0x7f8290135440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f828400b4e0 tx=0x7f828400b9a0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f828ffff640 1 -- 172.21.15.17:0/3242540319 reap_dead start 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f828ffff640 1 -- 172.21.15.17:0/3242540319 reap_dead start 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 shutdown_connections 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 >> 172.21.15.17:0/3242540319 conn(0x7f82900a7510 msgr2=0x7f8290072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 shutdown_connections 2024-09-15T21:19:27.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.194+0000 7f8295b1c640 1 -- 172.21.15.17:0/3242540319 wait complete. 2024-09-15T21:19:27.251 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:19:27.251 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:19:27.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:19:27.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient: get_monmap_and_config 2024-09-15T21:19:27.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient: build_initial_monmap 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient: monmap: 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:27.366595+0000 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:27.366595+0000 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:27.369 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 1 Processor -- start 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 1 -- start start 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient: init 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f51b00a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f51b00a5580 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b0140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient(hunting): picked mon.noname-b con 0x7f51b00a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b00a2c30 con 0x7f51b00a5580 2024-09-15T21:19:27.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b00a19b0 con 0x7f51b00a4a90 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f51b00a1c90 con 0x7f51b00a59d0 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b871c640 10 monclient(hunting): authenticate will time out at 1090.761940s 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b64b8640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f51b00a4a90 legacy=0x7f51b00a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b5cb7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f51b00a5580 0x7f51b013cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59744/0 (socket says 172.21.15.17:59744) 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b5cb7640 1 -- 172.21.15.17:0/1362974314 learned_addr learned my addr 172.21.15.17:0/1362974314 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b0140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): get_auth_request con 0x7f51b00a59d0 auth_method 0 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b54b6640 1 -- 172.21.15.17:0/1362974314 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 983852164 0 0) 0x7f51ac001120 con 0x7f51b00a5580 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b54b6640 10 monclient(hunting): _negotiate 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b54b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b54b6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b54b6640 10 monclient(hunting): my global_id is 4803 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b54b6640 1 -- 172.21.15.17:0/1362974314 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f519c003530 con 0x7f51b00a5580 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_done global_id 24194 payload 274 2024-09-15T21:19:27.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 1 -- 172.21.15.17:0/1362974314 >> v1:172.21.15.119:6789/0 conn(0x7f51b00a4a90 legacy=0x7f51b00a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:27.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 1 -- 172.21.15.17:0/1362974314 >> v1:172.21.15.17:6789/0 conn(0x7f51b00a5580 legacy=0x7f51b013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient: _finish_hunting 0 2024-09-15T21:19:27.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 1 monclient: found mon.noname-b 2024-09-15T21:19:27.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:27.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.362+0000 7f51b6cb9640 1 -- 172.21.15.17:0/1362974314 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b00a1f70 con 0x7f51b00a59d0 2024-09-15T21:19:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 --2- 172.21.15.17:0/1362974314 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b0140440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009df0 tx=0x7f51a801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=761024c4e37d314b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 1 -- 172.21.15.17:0/1362974314 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80b2070 con 0x7f51b00a59d0 2024-09-15T21:19:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: dump: 2024-09-15T21:19:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: _finish_auth 0 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: _check_auth_tickets 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 1 -- 172.21.15.17:0/1362974314 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a80a57e0 con 0x7f51b00a59d0 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 5 monclient: authenticate success, global_id 24194 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 1 -- 172.21.15.17:0/1362974314 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80a5bb0 con 0x7f51b00a59d0 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 4 set_mon_vals no callback set 2024-09-15T21:19:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b54b6640 10 monclient: dump: 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: shutdown 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- 172.21.15.17:0/1362974314 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 msgr2=0x7f51b0140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 --2- 172.21.15.17:0/1362974314 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b0140440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009df0 tx=0x7f51a801c850 comp rx=0 tx=0).stop 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- 172.21.15.17:0/1362974314 shutdown_connections 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 --2- 172.21.15.17:0/1362974314 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b0140440 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- 172.21.15.17:0/1362974314 >> 172.21.15.17:0/1362974314 conn(0x7f51b00aabb0 msgr2=0x7f51b00a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- 172.21.15.17:0/1362974314 shutdown_connections 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- 172.21.15.17:0/1362974314 wait complete. 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: build_initial_monmap 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: monmap: 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 Processor -- start 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- start start 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: init 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51b00a4a90 0x7f51b01372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): picked mon.noname-b con 0x7f51b00a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51b00a5580 0x7f51b01377f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): picked mon.noname-a con 0x7f51b00a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b013cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): picked mon.noname-c con 0x7f51b00a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f51b01439d0 con 0x7f51b00a5580 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51b0143850 con 0x7f51b00a4a90 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f51b0143b50 con 0x7f51b00a59d0 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b871c640 10 monclient(hunting): authenticate will time out at 1090.767437s 2024-09-15T21:19:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51b00a5580 0x7f51b01377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b013cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b64b8640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51b00a4a90 msgr2=0x7f51b01372b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51b00a5580 0x7f51b01377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54528/0 (socket says 172.21.15.17:54528) 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b013cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48730/0 (socket says 172.21.15.17:48730) 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 1 -- 172.21.15.17:0/2496049770 learned_addr learned my addr 172.21.15.17:0/2496049770 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): get_auth_request con 0x7f51b00a5580 auth_method 0 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b64b8640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51b00a4a90 0x7f51b01372b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): get_auth_request con 0x7f51b00a59d0 auth_method 0 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b5cb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient(hunting): handle_auth_done global_id 24197 payload 824 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51b00a4a90 msgr2=0x7f51b01372b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51b00a4a90 0x7f51b01372b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51b00a5580 msgr2=0x7f51b01377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51b00a5580 0x7f51b01377f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient: _finish_hunting 0 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 monclient: found mon.noname-c 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.366+0000 7f51b6cb9640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b01206e0 con 0x7f51b00a59d0 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b6cb9640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b013cb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51a80ac000 tx=0x7f51a80195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80bd020 con 0x7f51b00a59d0 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: dump: 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a80b8470 con 0x7f51b00a59d0 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 5 monclient: authenticate success, global_id 24197 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 10 monclient: _renew_subs 2024-09-15T21:19:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b0120f20 con 0x7f51b00a59d0 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80b2040 con 0x7f51b00a59d0 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: dump: 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b4cb5640 4 set_mon_vals no callback set 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 10 monclient: _renew_subs 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b0121200 con 0x7f51b00a59d0 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 0x7f51b013d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f51b00a9800 con 0x7f51b013d090 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 0x7f51b013d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient: get_auth_request con 0x7f51b013d090 auth_method 0 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f51a80c0020 con 0x7f51b00a59d0 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f519403b970 0x7f519403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 10 monclient(hunting): handle_auth_done global_id 24200 payload 824 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51b00a4e70 con 0x7f51b013d090 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4438+0+0 (secure 0 0 0) 0x7f51a80ea6d0 con 0x7f51b00a59d0 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b64b8640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 0x7f51b013d470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b450 tx=0x7f51a000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0003dd0 con 0x7f51b013d090 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: dump: 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b5cb7640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f519403b970 0x7f519403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b5cb7640 10 monclient: get_auth_request con 0x7f519403b970 auth_method 0 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f51a0015070 con 0x7f51b013d090 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 msgr2=0x7f51b013d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 0x7f51b013d470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b450 tx=0x7f51a000b910 comp rx=0 tx=0).stop 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 msgr2=0x7f51b013d470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51a6ffd640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d090 0x7f51b013d470 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b450 tx=0x7f51a000b910 comp rx=0 tx=0).stop 2024-09-15T21:19:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.370+0000 7f51b5cb7640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f519403b970 0x7f519403de30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 0x7f51b0138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f51b0069200 con 0x7f51b013d9b0 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 0x7f51b0138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient: get_auth_request con 0x7f51b013d9b0 auth_method 0 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 10 monclient(hunting): handle_auth_done global_id 24203 payload 824 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 1 -- 172.21.15.17:0/2496049770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f51b0137d30 con 0x7f51b013d9b0 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b64b8640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 0x7f51b0138cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b450 tx=0x7f51a00059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00055d0 con 0x7f51b013d9b0 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 10 monclient: dump: 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f51a00055d0 con 0x7f51b013d9b0 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 msgr2=0x7f51b0138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 0x7f51b0138cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b450 tx=0x7f51a00059e0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 msgr2=0x7f51b0138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51a6ffd640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b013d9b0 0x7f51b0138cf0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b450 tx=0x7f51a00059e0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 24, 2024-09-15T21:19:27.401 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 23183, 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:27.402 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:27.403 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:27.404 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:27.405 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:27.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:27.407 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:27.408 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.411 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:27.411 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.412 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.413 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.414 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:27.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.414 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.414 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:27.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.414 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:27.415 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:27.416 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:27.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f519403b970 msgr2=0x7f519403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f519403b970 0x7f519403de30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac007df0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 10 monclient: shutdown 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 msgr2=0x7f51b013cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 --2- 172.21.15.17:0/2496049770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51b00a59d0 0x7f51b013cb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51a80ac000 tx=0x7f51a80195f0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b6cb9640 1 -- 172.21.15.17:0/2496049770 reap_dead start 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b6cb9640 1 -- 172.21.15.17:0/2496049770 reap_dead start 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 shutdown_connections 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 >> 172.21.15.17:0/2496049770 conn(0x7f51b00aabb0 msgr2=0x7f51b00a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 shutdown_connections 2024-09-15T21:19:27.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:27.378+0000 7f51b871c640 1 -- 172.21.15.17:0/2496049770 wait complete. 2024-09-15T21:19:27.419 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:19:27.419 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:19:27.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.772+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 v43) ==== 126+0+0 (secure 0 0 0) 0x7f5ba001dd10 con 0x7f5ba8136430 2024-09-15T21:19:27.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.772+0000 7f5b96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:27.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.772+0000 7f5b96ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:27.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.872+0000 7f5bacc1c640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:27.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.872+0000 7f5bacc1c640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:27.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.872+0000 7f5bacc1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:27.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.872+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f5ba8134e50 con 0x7f5ba8136430 2024-09-15T21:19:27.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5b96ffd640 1 -- 172.21.15.119:0/1363662955 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v43) ==== 149+0+0 (secure 0 0 0) 0x7f5ba0024020 con 0x7f5ba8136430 2024-09-15T21:19:27.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5b96ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:27.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5b96ffd640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-09-15T21:19:27.879 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo already exists 2024-09-15T21:19:27.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8803b970 msgr2=0x7f5b8803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8803b970 0x7f5b8803de30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5b980ad000 tx=0x7f5b980a6000 comp rx=0 tx=0).stop 2024-09-15T21:19:27.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 10 monclient: shutdown 2024-09-15T21:19:27.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:27.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 msgr2=0x7f5ba811c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:27.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 0x7f5ba811c6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000be90 tx=0x7f5ba000bf80 comp rx=0 tx=0).stop 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 shutdown_connections 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8803b970 0x7f5b8803de30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba8136430 0x7f5ba811c6a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba8135fd0 0x7f5ba8134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 --2- 172.21.15.119:0/1363662955 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba8135bf0 0x7f5ba81343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 >> 172.21.15.119:0/1363662955 conn(0x7f5ba80a66e0 msgr2=0x7f5ba80a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 shutdown_connections 2024-09-15T21:19:27.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:27.876+0000 7f5bacc1c640 1 -- 172.21.15.119:0/1363662955 wait complete. 2024-09-15T21:19:27.896 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule create-simple foo default host 2024-09-15T21:19:28.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient: get_monmap_and_config 2024-09-15T21:19:28.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient: build_initial_monmap 2024-09-15T21:19:28.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient: monmap: 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:28.015231+0000 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:28.015231+0000 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:28.020 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 Processor -- start 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 -- start start 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient: init 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 0x7f851c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f851c135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f851c135fd0 0x7f851c0a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f851c135fd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f851c136430 0x7f851c0a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f851c136430 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f851c0a3600 con 0x7f851c136430 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f851c0a1d80 con 0x7f851c135fd0 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f851c0a2060 con 0x7f851c135bf0 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8523ff9640 10 monclient(hunting): authenticate will time out at 1086.811849s 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8521594640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f851c135fd0 msgr2=0x7f851c0a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.012+0000 7f8521594640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f851c135fd0 0x7f851c0a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f851c136430 0x7f851c0a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 0x7f851c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 0x7f851c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:60876/0 (socket says 172.21.15.119:60876) 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f851c136430 0x7f851c0a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34278/0 (socket says 172.21.15.119:34278) 2024-09-15T21:19:28.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 -- 172.21.15.119:0/3138838777 learned_addr learned my addr 172.21.15.119:0/3138838777 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): get_auth_request con 0x7f851c135bf0 auth_method 0 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): get_auth_request con 0x7f851c136430 auth_method 0 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8522596640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient(hunting): handle_auth_done global_id 24206 payload 274 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 -- 172.21.15.119:0/3138838777 >> v2:172.21.15.119:3300/0 conn(0x7f851c135fd0 msgr2=0x7f851c0a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 --2- 172.21.15.119:0/3138838777 >> v2:172.21.15.119:3300/0 conn(0x7f851c135fd0 0x7f851c0a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 -- 172.21.15.119:0/3138838777 >> v2:172.21.15.17:3300/0 conn(0x7f851c136430 msgr2=0x7f851c0a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 --2- 172.21.15.119:0/3138838777 >> v2:172.21.15.17:3300/0 conn(0x7f851c136430 0x7f851c0a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient: _finish_hunting 0 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 monclient: found mon.noname-b 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:28.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 -- 172.21.15.119:0/3138838777 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0a2340 con 0x7f851c135bf0 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8521d95640 1 --2- 172.21.15.119:0/3138838777 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 0x7f851c09fe70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f851c09d110 tx=0x7f851801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d592ae47cc1546 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 1 -- 172.21.15.119:0/3138838777 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85180b0070 con 0x7f851c135bf0 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: dump: 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.030 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: _finish_auth 0 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: _check_auth_tickets 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 1 -- 172.21.15.119:0/3138838777 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85180a4e50 con 0x7f851c135bf0 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 1 -- 172.21.15.119:0/3138838777 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85180ab820 con 0x7f851c135bf0 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8523ff9640 5 monclient: authenticate success, global_id 24206 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8523ff9640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8523ff9640 4 set_mon_vals no callback set 2024-09-15T21:19:28.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.016+0000 7f8520d93640 10 monclient: dump: 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.032 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient: shutdown 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- 172.21.15.119:0/3138838777 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 msgr2=0x7f851c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- 172.21.15.119:0/3138838777 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 0x7f851c09fe70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f851c09d110 tx=0x7f851801c400 comp rx=0 tx=0).stop 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- 172.21.15.119:0/3138838777 shutdown_connections 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- 172.21.15.119:0/3138838777 >> v2:172.21.15.119:3300/0 conn(0x7f851c135fd0 0x7f851c0a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- 172.21.15.119:0/3138838777 >> v2:172.21.15.17:3300/0 conn(0x7f851c136430 0x7f851c0a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- 172.21.15.119:0/3138838777 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c135bf0 0x7f851c09fe70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- 172.21.15.119:0/3138838777 >> 172.21.15.119:0/3138838777 conn(0x7f851c0a66e0 msgr2=0x7f851c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- 172.21.15.119:0/3138838777 shutdown_connections 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- 172.21.15.119:0/3138838777 wait complete. 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient: build_initial_monmap 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient: monmap: 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:28.034 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 Processor -- start 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- start start 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient: init 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c135bf0 0x7f851c11c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f851c135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f851c135fd0 0x7f851c11cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f851c135fd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c136430 0x7f851c11d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f851c136430 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f851c13f2a0 con 0x7f851c135fd0 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0a18e0 con 0x7f851c135bf0 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f851c13f420 con 0x7f851c136430 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 10 monclient(hunting): authenticate will time out at 1086.817714s 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521d95640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c135bf0 msgr2=0x7f851c11c6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:28.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521d95640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c135bf0 0x7f851c11c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f851c135fd0 0x7f851c11cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c136430 0x7f851c11d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f851c135fd0 0x7f851c11cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34292/0 (socket says 172.21.15.119:34292) 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 1 -- 172.21.15.119:0/2842546579 learned_addr learned my addr 172.21.15.119:0/2842546579 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): get_auth_request con 0x7f851c135fd0 auth_method 0 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): get_auth_request con 0x7f851c136430 auth_method 0 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8521594640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient(hunting): handle_auth_done global_id 24209 payload 824 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 -- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c135bf0 msgr2=0x7f851c11c6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c135bf0 0x7f851c11c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 -- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f851c135fd0 msgr2=0x7f851c11cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f851c135fd0 0x7f851c11cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient: _finish_hunting 0 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 monclient: found mon.noname-c 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:28.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 -- 172.21.15.119:0/2842546579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0a4fd0 con 0x7f851c136430 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8522596640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c136430 0x7f851c11d120 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f851400efc0 tx=0x7f851400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514018020 con 0x7f851c136430 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: dump: 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.039 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8514014480 con 0x7f851c136430 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514014870 con 0x7f851c136430 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:28.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f85127fc640 10 monclient: dump: 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8513fff640 4 set_mon_vals no callback set 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.020+0000 7f8523ff9640 5 monclient: authenticate success, global_id 24209 2024-09-15T21:19:28.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 monclient: _renew_subs 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c0a5900 con 0x7f851c136430 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 monclient: _renew_subs 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c072a10 con 0x7f851c136430 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c11f0a0 con 0x7f851c136430 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f851401b020 con 0x7f851c136430 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f85127fc640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f850003b970 0x7f850003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(43..43 src has 1..43) ==== 4497+0+0 (secure 0 0 0) 0x7f8514050ec0 con 0x7f851c136430 2024-09-15T21:19:28.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8521d95640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f850003b970 0x7f850003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8521d95640 10 monclient: get_auth_request con 0x7f850003b970 auth_method 0 2024-09-15T21:19:28.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.024+0000 7f8521d95640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f850003b970 0x7f850003de30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f85180ad000 tx=0x7f8518008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:28.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.036+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8514022020 con 0x7f851c136430 2024-09-15T21:19:28.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.036+0000 7f85127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:28.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.036+0000 7f85127fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:28.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.208+0000 7f8523ff9640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:28.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.208+0000 7f8523ff9640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:28.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.208+0000 7f8523ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.208+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f851c06a830 con 0x7f851c136430 2024-09-15T21:19:28.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.212+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v43) ==== 149+0+0 (secure 0 0 0) 0x7f851c0a5900 con 0x7f851c136430 2024-09-15T21:19:28.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.212+0000 7f85127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:28.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.212+0000 7f85127fc640 10 monclient: _finish_command 2 = system:0 rule foo already exists 2024-09-15T21:19:28.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.308+0000 7f8523ff9640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:28.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.308+0000 7f8523ff9640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:28.313 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.308+0000 7f8523ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.308+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f851c13c090 con 0x7f851c136430 2024-09-15T21:19:28.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f85127fc640 1 -- 172.21.15.119:0/2842546579 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v43) ==== 149+0+0 (secure 0 0 0) 0x7f8514010040 con 0x7f851c136430 2024-09-15T21:19:28.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f85127fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T21:19:28.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f85127fc640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-09-15T21:19:28.316 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo already exists 2024-09-15T21:19:28.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f850003b970 msgr2=0x7f850003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f850003b970 0x7f850003de30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f85180ad000 tx=0x7f8518008040 comp rx=0 tx=0).stop 2024-09-15T21:19:28.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 10 monclient: shutdown 2024-09-15T21:19:28.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:28.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c136430 msgr2=0x7f851c11d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c136430 0x7f851c11d120 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f851400efc0 tx=0x7f851400c490 comp rx=0 tx=0).stop 2024-09-15T21:19:28.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 shutdown_connections 2024-09-15T21:19:28.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f850003b970 0x7f850003de30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f851c136430 0x7f851c11d120 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f851c135fd0 0x7f851c11cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 --2- 172.21.15.119:0/2842546579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c135bf0 0x7f851c11c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 >> 172.21.15.119:0/2842546579 conn(0x7f851c0a66e0 msgr2=0x7f851c134e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:28.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 shutdown_connections 2024-09-15T21:19:28.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.312+0000 7f8523ff9640 1 -- 172.21.15.119:0/2842546579 wait complete. 2024-09-15T21:19:28.331 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule create-simple bar default host 2024-09-15T21:19:28.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient: get_monmap_and_config 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient: build_initial_monmap 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient: monmap: 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:28.438610+0000 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:28.438610+0000 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:28.440 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:28.441 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:28.441 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:28.441 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:28.441 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 Processor -- start 2024-09-15T21:19:28.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 -- start start 2024-09-15T21:19:28.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient: init 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 0x7fe7ac137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): picked mon.noname-a con 0x7fe7ac0a7ac0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7ac0a41b0 0x7fe7ac0a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): picked mon.noname-c con 0x7fe7ac0a41b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac1347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): picked mon.noname-b con 0x7fe7ac134300 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:28.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ac0a7ea0 con 0x7fe7ac0a7ac0 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ac0a19d0 con 0x7fe7ac0a41b0 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac0a6960 con 0x7fe7ac134300 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.436+0000 7fe7b1080640 10 monclient(hunting): authenticate will time out at 1087.235949s 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aa59c640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fe7ac0a41b0 msgr2=0x7fe7ac0a45c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:28.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aa59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7ac0a41b0 0x7fe7ac0a45c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 0x7fe7ac137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac1347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 0x7fe7ac137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34302/0 (socket says 172.21.15.119:34302) 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 1 -- 172.21.15.119:0/2428451253 learned_addr learned my addr 172.21.15.119:0/2428451253 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): get_auth_request con 0x7fe7ac0a7ac0 auth_method 0 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): get_auth_request con 0x7fe7ac134300 auth_method 0 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_done global_id 24212 payload 274 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2428451253 >> v2:172.21.15.119:3300/0 conn(0x7fe7ac0a41b0 msgr2=0x7fe7ac0a45c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 --2- 172.21.15.119:0/2428451253 >> v2:172.21.15.119:3300/0 conn(0x7fe7ac0a41b0 0x7fe7ac0a45c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2428451253 >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 msgr2=0x7fe7ac137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 --2- 172.21.15.119:0/2428451253 >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 0x7fe7ac137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 monclient: found mon.noname-b 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2428451253 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac0a6c40 con 0x7fe7ac134300 2024-09-15T21:19:28.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7aad9d640 1 --2- 172.21.15.119:0/2428451253 >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 0x7fe7ac137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7ab59e640 1 --2- 172.21.15.119:0/2428451253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac1347b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac074490 tx=0x7fe7a401c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a421e1c8c92cad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 1 -- 172.21.15.119:0/2428451253 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40b2070 con 0x7fe7ac134300 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: dump: 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.454 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: _finish_auth 0 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:28.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7b1080640 5 monclient: authenticate success, global_id 24212 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7b1080640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 1 -- 172.21.15.119:0/2428451253 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a40a5830 con 0x7fe7ac134300 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7b1080640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 1 -- 172.21.15.119:0/2428451253 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40a5c00 con 0x7fe7ac134300 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7b1080640 4 set_mon_vals no callback set 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.440+0000 7fe7a9d9b640 10 monclient: dump: 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: shutdown 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2428451253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 msgr2=0x7fe7ac1347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2428451253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac1347b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac074490 tx=0x7fe7a401c810 comp rx=0 tx=0).stop 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2428451253 shutdown_connections 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2428451253 >> v2:172.21.15.119:3300/0 conn(0x7fe7ac0a41b0 0x7fe7ac0a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2428451253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac1347b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2428451253 >> v2:172.21.15.17:3300/0 conn(0x7fe7ac0a7ac0 0x7fe7ac137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2428451253 >> 172.21.15.119:0/2428451253 conn(0x7fe7ac0a6fd0 msgr2=0x7fe7ac0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2428451253 shutdown_connections 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2428451253 wait complete. 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: build_initial_monmap 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: monmap: 2024-09-15T21:19:28.458 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 Processor -- start 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- start start 2024-09-15T21:19:28.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: init 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7ac0a41b0 0x7fe7ac072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): picked mon.noname-b con 0x7fe7ac0a41b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0a7ac0 0x7fe7ac0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): picked mon.noname-a con 0x7fe7ac0a7ac0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): picked mon.noname-c con 0x7fe7ac134300 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac138390 con 0x7fe7ac0a7ac0 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac138210 con 0x7fe7ac0a41b0 2024-09-15T21:19:28.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac1413b0 con 0x7fe7ac134300 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient(hunting): authenticate will time out at 1087.241684s 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aad9d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7ac0a41b0 msgr2=0x7fe7ac072480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7ac0a41b0 0x7fe7ac072480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:60912/0 (socket says 172.21.15.119:60912) 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0a7ac0 0x7fe7ac0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2509892779 learned_addr learned my addr 172.21.15.119:0/2509892779 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:28.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): get_auth_request con 0x7fe7ac134300 auth_method 0 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): get_auth_request con 0x7fe7ac0a7ac0 auth_method 0 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:28.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient(hunting): handle_auth_done global_id 24215 payload 824 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7ac0a41b0 msgr2=0x7fe7ac072480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7ac0a41b0 0x7fe7ac072480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0a7ac0 msgr2=0x7fe7ac0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0a7ac0 0x7fe7ac0729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 monclient: found mon.noname-c 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 -- 172.21.15.119:0/2509892779 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac133e20 con 0x7fe7ac134300 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7ab59e640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac072f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7a401d0b0 tx=0x7fe7a40195b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:28.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40b2050 con 0x7fe7ac134300 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: dump: 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.464 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 5 monclient: authenticate success, global_id 24215 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: _renew_subs 2024-09-15T21:19:28.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac073e80 con 0x7fe7ac134300 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: _renew_subs 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac13a6d0 con 0x7fe7ac134300 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a40be070 con 0x7fe7ac134300 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40ad850 con 0x7fe7ac134300 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7937fe640 10 monclient: dump: 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.444+0000 7fe7a959a640 4 set_mon_vals no callback set 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7b1080640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7b1080640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7b1080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac077180 con 0x7fe7ac134300 2024-09-15T21:19:28.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe7a40ad9f0 con 0x7fe7ac134300 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7937fe640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe79803b970 0x7fe79803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(43..43 src has 1..43) ==== 4497+0+0 (secure 0 0 0) 0x7fe7a40eb370 con 0x7fe7ac134300 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7aad9d640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe79803b970 0x7fe79803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7aad9d640 10 monclient: get_auth_request con 0x7fe79803b970 auth_method 0 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.448+0000 7fe7aad9d640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe79803b970 0x7fe79803de30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006c30 tx=0x7fe79c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.460+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe7a40c8020 con 0x7fe7ac134300 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.460+0000 7fe7937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:28.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.460+0000 7fe7937fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:28.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.624+0000 7fe7b1080640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T21:19:28.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.624+0000 7fe7b1080640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T21:19:28.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.624+0000 7fe7b1080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.624+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) -- 0x7fe7ac137cd0 con 0x7fe7ac134300 2024-09-15T21:19:28.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.788+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 v44) ==== 126+0+0 (secure 0 0 0) 0x7fe7a4016520 con 0x7fe7ac134300 2024-09-15T21:19:28.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.788+0000 7fe7937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T21:19:28.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.788+0000 7fe7937fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:28.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.884+0000 7fe7b1080640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T21:19:28.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.884+0000 7fe7b1080640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T21:19:28.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.884+0000 7fe7b1080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:28.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.884+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) -- 0x7fe7ac138ed0 con 0x7fe7ac134300 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7937fe640 1 -- 172.21.15.119:0/2509892779 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 rule bar already exists v44) ==== 149+0+0 (secure 0 0 0) 0x7fe7a40eb080 con 0x7fe7ac134300 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7937fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7937fe640 10 monclient: _finish_command 3 = system:0 rule bar already exists 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:rule bar already exists 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe79803b970 msgr2=0x7fe79803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe79803b970 0x7fe79803de30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006c30 tx=0x7fe79c009240 comp rx=0 tx=0).stop 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 10 monclient: shutdown 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 msgr2=0x7fe7ac072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac072f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7a401d0b0 tx=0x7fe7a40195b0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 shutdown_connections 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe79803b970 0x7fe79803de30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7ac134300 0x7fe7ac072f00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0a7ac0 0x7fe7ac0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 --2- 172.21.15.119:0/2509892779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7ac0a41b0 0x7fe7ac072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 >> 172.21.15.119:0/2509892779 conn(0x7fe7ac0a6fd0 msgr2=0x7fe7ac132860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 shutdown_connections 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:28.888+0000 7fe7b1080640 1 -- 172.21.15.119:0/2509892779 wait complete. 2024-09-15T21:19:28.920 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm-device-class all 2024-09-15T21:19:29.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient: get_monmap_and_config 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient: build_initial_monmap 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient: monmap: 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:29.022410+0000 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:29.022410+0000 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:29.024 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:29.025 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 1 Processor -- start 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 1 -- start start 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient: init 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8934135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f89340a0160 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f893406ec30 0x7f893413e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient(hunting): picked mon.noname-a con 0x7f893406ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:29.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893409cf30 con 0x7f8934135ca0 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89340a1ca0 con 0x7f89340a0160 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f89340a1f80 con 0x7f893406ec30 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f893a222640 10 monclient(hunting): authenticate will time out at 1087.819024s 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f892affd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f89340a0160 legacy=0x7f893413aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f893406ec30 0x7f893413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f89337fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8934135ca0 0x7f8934136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60994/0 (socket says 172.21.15.119:60994) 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f89337fe640 1 -- 172.21.15.119:0/3158525759 learned_addr learned my addr 172.21.15.119:0/3158525759 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f893406ec30 0x7f893413e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34318/0 (socket says 172.21.15.119:34318) 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): get_auth_request con 0x7f893406ec30 auth_method 0 2024-09-15T21:19:29.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.020+0000 7f8933fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 10 monclient(hunting): handle_auth_done global_id 4821 payload 274 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 1 -- 172.21.15.119:0/3158525759 >> v1:172.21.15.119:6789/0 conn(0x7f89340a0160 legacy=0x7f893413aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 1 -- 172.21.15.119:0/3158525759 >> v1:172.21.15.17:6789/0 conn(0x7f8934135ca0 legacy=0x7f8934136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 1 monclient: found mon.noname-a 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.024+0000 7f8933fff640 1 -- 172.21.15.119:0/3158525759 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89340a3860 con 0x7f893406ec30 2024-09-15T21:19:29.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8933fff640 1 --2- 172.21.15.119:0/3158525759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406ec30 0x7f893413e130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f892c009dc0 tx=0x7f892c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd61ca3fc8c605f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:29.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 1 -- 172.21.15.119:0/3158525759 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0b2070 con 0x7f893406ec30 2024-09-15T21:19:29.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: dump: 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:29.032 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 5 monclient: authenticate success, global_id 4821 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 1 -- 172.21.15.119:0/3158525759 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c0a57e0 con 0x7f893406ec30 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 4 set_mon_vals no callback set 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 1 -- 172.21.15.119:0/3158525759 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0a5bb0 con 0x7f893406ec30 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8932ffd640 10 monclient: dump: 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:29.033 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: shutdown 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- 172.21.15.119:0/3158525759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406ec30 msgr2=0x7f893413e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 --2- 172.21.15.119:0/3158525759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406ec30 0x7f893413e130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f892c009dc0 tx=0x7f892c01c810 comp rx=0 tx=0).stop 2024-09-15T21:19:29.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- 172.21.15.119:0/3158525759 shutdown_connections 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 --2- 172.21.15.119:0/3158525759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406ec30 0x7f893413e130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- 172.21.15.119:0/3158525759 >> 172.21.15.119:0/3158525759 conn(0x7f89340a2ae0 msgr2=0x7f89340a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- 172.21.15.119:0/3158525759 shutdown_connections 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- 172.21.15.119:0/3158525759 wait complete. 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: build_initial_monmap 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: monmap: 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:29.035 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:29.036 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:29.036 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:29.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 Processor -- start 2024-09-15T21:19:29.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- start start 2024-09-15T21:19:29.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: init 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f893406ec30 0x7f893412f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): picked mon.noname-c con 0x7f893406ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 0x7f893412ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): picked mon.noname-a con 0x7f89340a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934135ca0 0x7f8934136d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:29.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): picked mon.noname-b con 0x7f8934135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8934141010 con 0x7f89340a0160 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8934140e90 con 0x7f8934135ca0 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8934141190 con 0x7f893406ec30 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f893a222640 10 monclient(hunting): authenticate will time out at 1087.826459s 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8933fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934135ca0 msgr2=0x7f8934136d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f8933fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934135ca0 0x7f8934136d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f893406ec30 0x7f893412f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 0x7f893412ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f893406ec30 0x7f893412f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:60924/0 (socket says 172.21.15.119:60924) 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 0x7f893412ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34332/0 (socket says 172.21.15.119:34332) 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 1 -- 172.21.15.119:0/455336038 learned_addr learned my addr 172.21.15.119:0/455336038 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): get_auth_request con 0x7f893406ec30 auth_method 0 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): get_auth_request con 0x7f89340a0160 auth_method 0 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f89337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient(hunting): handle_auth_done global_id 4827 payload 824 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 -- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f893406ec30 msgr2=0x7f893412f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f893406ec30 0x7f893412f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 -- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934135ca0 msgr2=0x7f8934136d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934135ca0 0x7f8934136d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 monclient: found mon.noname-a 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 -- 172.21.15.119:0/455336038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893413e0a0 con 0x7f89340a0160 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.028+0000 7f892affd640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 0x7f893412ff00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b470 tx=0x7f891c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c003dd0 con 0x7f89340a0160 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: dump: 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c005070 con 0x7f89340a0160 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c005460 con 0x7f89340a0160 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 5 monclient: authenticate success, global_id 4827 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 monclient: _renew_subs 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893409d1d0 con 0x7f89340a0160 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f89327fc640 4 set_mon_vals no callback set 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 10 monclient: dump: 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:29.046 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 monclient: _renew_subs 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893413aba0 con 0x7f89340a0160 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8934136080 con 0x7f89340a0160 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f891c005600 con 0x7f89340a0160 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f891403b970 0x7f891403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4556+0+0 (secure 0 0 0) 0x7f891c04fff0 con 0x7f89340a0160 2024-09-15T21:19:29.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f89337fe640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f891403b970 0x7f891403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:29.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f89337fe640 10 monclient: get_auth_request con 0x7f891403b970 auth_method 0 2024-09-15T21:19:29.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.032+0000 7f89337fe640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f891403b970 0x7f891403de30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8924000f70 tx=0x7f8924009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:29.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.044+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f891c019d70 con 0x7f89340a0160 2024-09-15T21:19:29.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.044+0000 7f8930ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:29.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.044+0000 7f8930ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.220+0000 7f893a222640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T21:19:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.220+0000 7f893a222640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T21:19:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.220+0000 7f893a222640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.220+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) -- 0x7f893409f8f0 con 0x7f89340a0160 2024-09-15T21:19:29.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.412+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 done removing class of osd(s): 0,1,2,3,4,5 v45) ==== 131+0+0 (secure 0 0 0) 0x7f891c04dbe0 con 0x7f89340a0160 2024-09-15T21:19:29.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.412+0000 7f8930ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T21:19:29.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.412+0000 7f8930ff9640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0,1,2,3,4,5 2024-09-15T21:19:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.516+0000 7f893a222640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T21:19:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.516+0000 7f893a222640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T21:19:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.516+0000 7f893a222640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:29.516+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) -- 0x7f89340008e0 con 0x7f89340a0160 2024-09-15T21:19:30.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f8930ff9640 1 -- 172.21.15.119:0/455336038 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): v46) ==== 282+0+0 (secure 0 0 0) 0x7f891c019ad0 con 0x7f89340a0160 2024-09-15T21:19:30.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f8930ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T21:19:30.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f8930ff9640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-09-15T21:19:30.419 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f891403b970 msgr2=0x7f891403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f891403b970 0x7f891403de30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8924000f70 tx=0x7f8924009040 comp rx=0 tx=0).stop 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 10 monclient: shutdown 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 msgr2=0x7f893412ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 0x7f893412ff00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b470 tx=0x7f891c00b930 comp rx=0 tx=0).stop 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 shutdown_connections 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f891403b970 0x7f891403de30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8934135ca0 0x7f8934136d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340a0160 0x7f893412ff00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 --2- 172.21.15.119:0/455336038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f893406ec30 0x7f893412f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 >> 172.21.15.119:0/455336038 conn(0x7f89340a2ae0 msgr2=0x7f8934072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:30.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 shutdown_connections 2024-09-15T21:19:30.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.416+0000 7f893a222640 1 -- 172.21.15.119:0/455336038 wait complete. 2024-09-15T21:19:30.439 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush set-device-class ssd osd.0 2024-09-15T21:19:30.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.552+0000 7fcefbf0f640 10 monclient: get_monmap_and_config 2024-09-15T21:19:30.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.552+0000 7fcefbf0f640 10 monclient: build_initial_monmap 2024-09-15T21:19:30.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.552+0000 7fcefbf0f640 10 monclient: monmap: 2024-09-15T21:19:30.557 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:30.557 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:30.557 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:30.555752+0000 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:30.555752+0000 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:30.558 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:30.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 Processor -- start 2024-09-15T21:19:30.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 -- start start 2024-09-15T21:19:30.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient: init 2024-09-15T21:19:30.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcef40a0900 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 0x7fcef413ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): picked mon.noname-b con 0x7fcef40a13f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcef413b150 0x7fcef413d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): picked mon.noname-c con 0x7fcef413b150 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef40a2240 con 0x7fcef40a0900 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcef409fdd0 con 0x7fcef413b150 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcef40a1c80 con 0x7fcef40a13f0 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:30.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefbf0f640 10 monclient(hunting): authenticate will time out at 1089.352912s 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef9cab640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fcef40a0900 legacy=0x7fcef40a0ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefa4ac640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fcef413b150 msgr2=0x7fcef413d540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcefa4ac640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcef413b150 0x7fcef413d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 0x7fcef413ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 0x7fcef413ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:60936/0 (socket says 172.21.15.119:60936) 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 -- 172.21.15.119:0/65859998 learned_addr learned my addr 172.21.15.119:0/65859998 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): get_auth_request con 0x7fcef40a13f0 auth_method 0 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:30.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient(hunting): handle_auth_done global_id 24221 payload 274 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 -- 172.21.15.119:0/65859998 >> v2:172.21.15.119:3300/0 conn(0x7fcef413b150 msgr2=0x7fcef413d540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 --2- 172.21.15.119:0/65859998 >> v2:172.21.15.119:3300/0 conn(0x7fcef413b150 0x7fcef413d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 -- 172.21.15.119:0/65859998 >> v1:172.21.15.119:6789/0 conn(0x7fcef40a0900 legacy=0x7fcef40a0ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient: _finish_hunting 0 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 monclient: found mon.noname-b 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:30.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.556+0000 7fcef94aa640 1 -- 172.21.15.119:0/65859998 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef40a1f60 con 0x7fcef40a13f0 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef94aa640 1 --2- 172.21.15.119:0/65859998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 0x7fcef413ac10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009870 tx=0x7fcee401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8a283f6889e7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 1 -- 172.21.15.119:0/65859998 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40b0070 con 0x7fcef40a13f0 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: dump: 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:30.565 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: _finish_auth 0 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 1 -- 172.21.15.119:0/65859998 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee40ab600 con 0x7fcef40a13f0 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 1 -- 172.21.15.119:0/65859998 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40ab9d0 con 0x7fcef40a13f0 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 5 monclient: authenticate success, global_id 24221 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:30.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 4 set_mon_vals no callback set 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcef8ca9640 10 monclient: dump: 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:30.567 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: shutdown 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- 172.21.15.119:0/65859998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 msgr2=0x7fcef413ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/65859998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 0x7fcef413ac10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009870 tx=0x7fcee401c440 comp rx=0 tx=0).stop 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- 172.21.15.119:0/65859998 shutdown_connections 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/65859998 >> v2:172.21.15.119:3300/0 conn(0x7fcef413b150 0x7fcef413d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/65859998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a13f0 0x7fcef413ac10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- 172.21.15.119:0/65859998 >> 172.21.15.119:0/65859998 conn(0x7fcef409be60 msgr2=0x7fcef409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- 172.21.15.119:0/65859998 shutdown_connections 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- 172.21.15.119:0/65859998 wait complete. 2024-09-15T21:19:30.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: build_initial_monmap 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: monmap: 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 Processor -- start 2024-09-15T21:19:30.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- start start 2024-09-15T21:19:30.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: init 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 0x7fcef41cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): picked mon.noname-c con 0x7fcef40a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40a13f0 0x7fcef4135d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): picked mon.noname-a con 0x7fcef40a13f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef413b150 0x7fcef41cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): picked mon.noname-b con 0x7fcef413b150 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcef40a01d0 con 0x7fcef40a13f0 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:30.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcef40a0050 con 0x7fcef413b150 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcef41402a0 con 0x7fcef40a0900 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefbf0f640 10 monclient(hunting): authenticate will time out at 1089.359768s 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefa4ac640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef413b150 msgr2=0x7fcef41cc3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.560+0000 7fcefa4ac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef413b150 0x7fcef41cc3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 0x7fcef41cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40a13f0 0x7fcef4135d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 0x7fcef41cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:60948/0 (socket says 172.21.15.119:60948) 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 -- 172.21.15.119:0/543203687 learned_addr learned my addr 172.21.15.119:0/543203687 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): get_auth_request con 0x7fcef40a0900 auth_method 0 2024-09-15T21:19:30.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): get_auth_request con 0x7fcef40a13f0 auth_method 0 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient(hunting): handle_auth_done global_id 24224 payload 824 2024-09-15T21:19:30.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 -- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef413b150 msgr2=0x7fcef41cc3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef413b150 0x7fcef41cc3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 -- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40a13f0 msgr2=0x7fcef4135d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40a13f0 0x7fcef4135d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient: _finish_hunting 0 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 monclient: found mon.noname-c 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 -- 172.21.15.119:0/543203687 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef409edd0 con 0x7fcef40a0900 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef9cab640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 0x7fcef41cf970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b530 tx=0x7fcef000b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0013020 con 0x7fcef40a0900 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: dump: 2024-09-15T21:19:30.578 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:30.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef0003df0 con 0x7fcef40a0900 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00052b0 con 0x7fcef40a0900 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 5 monclient: authenticate success, global_id 24224 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 monclient: _renew_subs 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef409d0f0 con 0x7fcef40a0900 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceebfff640 4 set_mon_vals no callback set 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 10 monclient: dump: 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.580 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 monclient: _renew_subs 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef4121e40 con 0x7fcef40a0900 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:30.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef40a0ce0 con 0x7fcef40a0900 2024-09-15T21:19:30.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcef0016020 con 0x7fcef40a0900 2024-09-15T21:19:30.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fced803b970 0x7fced803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:30.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 4252+0+0 (secure 0 0 0) 0x7fcef004fbb0 con 0x7fcef40a0900 2024-09-15T21:19:30.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fced803b970 0x7fced803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:30.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 10 monclient: get_auth_request con 0x7fced803b970 auth_method 0 2024-09-15T21:19:30.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.564+0000 7fcef94aa640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fced803b970 0x7fced803de30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcee40ad000 tx=0x7fcee40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:30.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.580+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcef001d020 con 0x7fcef40a0900 2024-09-15T21:19:30.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.580+0000 7fceea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:30.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.580+0000 7fceea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.748+0000 7fcefbf0f640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T21:19:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.748+0000 7fcefbf0f640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T21:19:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.748+0000 7fcefbf0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:30.748+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) -- 0x7fcef4120ee0 con 0x7fcef40a0900 2024-09-15T21:19:31.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.428+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'ssd' v47) ==== 135+0+0 (secure 0 0 0) 0x7fcef409d0f0 con 0x7fcef40a0900 2024-09-15T21:19:31.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.428+0000 7fceea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T21:19:31.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.428+0000 7fceea7fc640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'ssd' 2024-09-15T21:19:31.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.528+0000 7fcefbf0f640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T21:19:31.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.528+0000 7fcefbf0f640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T21:19:31.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.528+0000 7fcefbf0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:31.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:31.528+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) -- 0x7fcef40008e0 con 0x7fcef40a0900 2024-09-15T21:19:32.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.428+0000 7fceea7fc640 1 -- 172.21.15.119:0/543203687 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' v48) ==== 237+0+0 (secure 0 0 0) 0x7fcef000fe20 con 0x7fcef40a0900 2024-09-15T21:19:32.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.428+0000 7fceea7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T21:19:32.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.428+0000 7fceea7fc640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:19:32.434 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:19:32.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fced803b970 msgr2=0x7fced803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fced803b970 0x7fced803de30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcee40ad000 tx=0x7fcee40a6000 comp rx=0 tx=0).stop 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 10 monclient: shutdown 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 msgr2=0x7fcef41cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 0x7fcef41cf970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b530 tx=0x7fcef000b9f0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 shutdown_connections 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fced803b970 0x7fced803de30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef413b150 0x7fcef41cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40a13f0 0x7fcef4135d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 --2- 172.21.15.119:0/543203687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcef40a0900 0x7fcef41cf970 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 >> 172.21.15.119:0/543203687 conn(0x7fcef409be60 msgr2=0x7fcef40a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:32.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 shutdown_connections 2024-09-15T21:19:32.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.432+0000 7fcefbf0f640 1 -- 172.21.15.119:0/543203687 wait complete. 2024-09-15T21:19:32.457 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush set-device-class hdd osd.1 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.576+0000 7fc7a7447640 10 monclient: get_monmap_and_config 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.576+0000 7fc7a7447640 10 monclient: build_initial_monmap 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.576+0000 7fc7a7447640 10 monclient: monmap: 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:32.580659+0000 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:32.580659+0000 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:32.582 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:32.583 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:32.583 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:32.583 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:32.583 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:32.583 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:32.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 Processor -- start 2024-09-15T21:19:32.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 -- start start 2024-09-15T21:19:32.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient: init 2024-09-15T21:19:32.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:32.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a00a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a00a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7a00a46b0 0x7fc7a01377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): picked mon.noname-c con 0x7fc7a00a46b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7a00a1100 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a0135180 con 0x7fc7a00a1100 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a00a7dc0 con 0x7fc7a00a46b0 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a00a6920 con 0x7fc7a00a79e0 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:32.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a7447640 10 monclient(hunting): authenticate will time out at 1091.377791s 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a49e2640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fc7a00a46b0 msgr2=0x7fc7a01377e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a49e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7a00a46b0 0x7fc7a01377e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a00a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a59e4640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc7a00a1100 0x7fc7a0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60614/0 (socket says 172.21.15.119:60614) 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a59e4640 1 -- 172.21.15.119:0/3582562224 learned_addr learned my addr 172.21.15.119:0/3582562224 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): get_auth_request con 0x7fc7a00a79e0 auth_method 0 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc797fff640 1 -- 172.21.15.119:0/3582562224 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3994427986 0 0) 0x7fc7a0135180 con 0x7fc7a00a1100 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc797fff640 10 monclient(hunting): _negotiate 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc797fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc797fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc797fff640 10 monclient(hunting): my global_id is 4833 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc797fff640 1 -- 172.21.15.119:0/3582562224 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc78c003530 con 0x7fc7a00a1100 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient(hunting): handle_auth_done global_id 24227 payload 274 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 1 -- 172.21.15.119:0/3582562224 >> v2:172.21.15.119:3300/0 conn(0x7fc7a00a46b0 msgr2=0x7fc7a01377e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 1 --2- 172.21.15.119:0/3582562224 >> v2:172.21.15.119:3300/0 conn(0x7fc7a00a46b0 0x7fc7a01377e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 1 -- 172.21.15.119:0/3582562224 >> v1:172.21.15.17:6789/0 conn(0x7fc7a00a1100 legacy=0x7fc7a0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient: _finish_hunting 0 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 1 monclient: found mon.noname-b 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:32.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:32.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.580+0000 7fc7a51e3640 1 -- 172.21.15.119:0/3582562224 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a00a1c30 con 0x7fc7a00a79e0 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a51e3640 1 --2- 172.21.15.119:0/3582562224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a00a4170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097c0 tx=0x7fc79c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92db8337c79bf85f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 1 -- 172.21.15.119:0/3582562224 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0134930 con 0x7fc7a00a79e0 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: dump: 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:32.594 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 1 -- 172.21.15.119:0/3582562224 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0b0070 con 0x7fc7a00a79e0 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 5 monclient: authenticate success, global_id 24227 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 4 set_mon_vals no callback set 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 1 -- 172.21.15.119:0/3582562224 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0ab630 con 0x7fc7a00a79e0 2024-09-15T21:19:32.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc797fff640 10 monclient: dump: 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: shutdown 2024-09-15T21:19:32.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- 172.21.15.119:0/3582562224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 msgr2=0x7fc7a00a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 --2- 172.21.15.119:0/3582562224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a00a4170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097c0 tx=0x7fc79c01c410 comp rx=0 tx=0).stop 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- 172.21.15.119:0/3582562224 shutdown_connections 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 --2- 172.21.15.119:0/3582562224 >> v2:172.21.15.119:3300/0 conn(0x7fc7a00a46b0 0x7fc7a01377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 --2- 172.21.15.119:0/3582562224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a00a4170 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- 172.21.15.119:0/3582562224 >> 172.21.15.119:0/3582562224 conn(0x7fc7a00a6ef0 msgr2=0x7fc7a00a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- 172.21.15.119:0/3582562224 shutdown_connections 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- 172.21.15.119:0/3582562224 wait complete. 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: build_initial_monmap 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: monmap: 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:32.597 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 Processor -- start 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- start start 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: init 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a00a1100 0x7fc7a0130da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:32.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a00a1100 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 0x7fc7a01312e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): picked mon.noname-a con 0x7fc7a00a46b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a0071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): picked mon.noname-c con 0x7fc7a00a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01385b0 con 0x7fc7a00a46b0 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0138430 con 0x7fc7a00a1100 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a01423a0 con 0x7fc7a00a79e0 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a7447640 10 monclient(hunting): authenticate will time out at 1091.383593s 2024-09-15T21:19:32.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a51e3640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a00a1100 msgr2=0x7fc7a0130da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a51e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a00a1100 0x7fc7a0130da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 0x7fc7a01312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a59e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a0071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 0x7fc7a01312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57618/0 (socket says 172.21.15.119:57618) 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 1 -- 172.21.15.119:0/1131409309 learned_addr learned my addr 172.21.15.119:0/1131409309 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 10 monclient(hunting): get_auth_request con 0x7fc7a00a46b0 auth_method 0 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.584+0000 7fc7a49e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): get_auth_request con 0x7fc7a00a79e0 auth_method 0 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:32.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a49e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a49e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a49e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient(hunting): handle_auth_done global_id 24230 payload 824 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 -- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a00a1100 msgr2=0x7fc7a0130da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a00a1100 0x7fc7a0130da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 -- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 msgr2=0x7fc7a01312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 0x7fc7a01312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient: _finish_hunting 0 2024-09-15T21:19:32.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 monclient: found mon.noname-c 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 -- 172.21.15.119:0/1131409309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a013a1b0 con 0x7fc7a00a79e0 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a49e2640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 0x7fc7a01312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a59e4640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a0071ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc79800ca50 tx=0x7fc79800cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79800abb0 con 0x7fc7a00a79e0 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: dump: 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.602 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 5 monclient: authenticate success, global_id 24230 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79800ad50 con 0x7fc7a00a79e0 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 monclient: _renew_subs 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:32.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a013f7f0 con 0x7fc7a00a79e0 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798012600 con 0x7fc7a00a79e0 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 10 monclient: dump: 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:32.604 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 monclient: _renew_subs 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7977fe640 4 set_mon_vals no callback set 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a013f010 con 0x7fc7a00a79e0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a00a4170 con 0x7fc7a00a79e0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc7980127a0 con 0x7fc7a00a79e0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78403b970 0x7fc78403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 4476+0+0 (secure 0 0 0) 0x7fc798050ac0 con 0x7fc7a00a79e0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a51e3640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78403b970 0x7fc78403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:32.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a51e3640 10 monclient: get_auth_request con 0x7fc78403b970 auth_method 0 2024-09-15T21:19:32.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.588+0000 7fc7a51e3640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78403b970 0x7fc78403de30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:32.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.600+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc79801a360 con 0x7fc7a00a79e0 2024-09-15T21:19:32.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.600+0000 7fc795ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:32.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.600+0000 7fc795ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:32.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.780+0000 7fc7a7447640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T21:19:32.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.780+0000 7fc7a7447640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T21:19:32.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.780+0000 7fc7a7447640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:32.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:32.780+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) -- 0x7fc7a006a760 con 0x7fc7a00a79e0 2024-09-15T21:19:33.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.444+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 set osd(s) 1 to class 'hdd' v49) ==== 135+0+0 (secure 0 0 0) 0x7fc798007a90 con 0x7fc7a00a79e0 2024-09-15T21:19:33.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.444+0000 7fc795ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T21:19:33.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.444+0000 7fc795ffb640 10 monclient: _finish_command 2 = system:0 set osd(s) 1 to class 'hdd' 2024-09-15T21:19:33.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.552+0000 7fc7a7447640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T21:19:33.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.552+0000 7fc7a7447640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T21:19:33.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.552+0000 7fc7a7447640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:33.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:33.552+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) -- 0x7fc7a0139fb0 con 0x7fc7a00a79e0 2024-09-15T21:19:34.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc795ffb640 1 -- 172.21.15.119:0/1131409309 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' v50) ==== 237+0+0 (secure 0 0 0) 0x7fc79804e600 con 0x7fc7a00a79e0 2024-09-15T21:19:34.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc795ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T21:19:34.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc795ffb640 10 monclient: _finish_command 3 = system:0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-09-15T21:19:34.455 INFO:tasks.workunit.client.0.smithi119.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78403b970 msgr2=0x7fc78403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78403b970 0x7fc78403de30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 10 monclient: shutdown 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 msgr2=0x7fc7a0071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a0071ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc79800ca50 tx=0x7fc79800cf10 comp rx=0 tx=0).stop 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 shutdown_connections 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc78403b970 0x7fc78403de30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7a00a79e0 0x7fc7a0071ff0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7a00a46b0 0x7fc7a01312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 --2- 172.21.15.119:0/1131409309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a00a1100 0x7fc7a0130da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.452+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 >> 172.21.15.119:0/1131409309 conn(0x7fc7a00a6ef0 msgr2=0x7fc7a01373c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:34.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.456+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 shutdown_connections 2024-09-15T21:19:34.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.456+0000 7fc7a7447640 1 -- 172.21.15.119:0/1131409309 wait complete. 2024-09-15T21:19:34.474 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule create-replicated foo-ssd default host ssd 2024-09-15T21:19:34.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.596+0000 7f106c2af640 10 monclient: get_monmap_and_config 2024-09-15T21:19:34.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.596+0000 7f106c2af640 10 monclient: build_initial_monmap 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.596+0000 7f106c2af640 10 monclient: monmap: 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:34.599157+0000 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:34.599157+0000 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.601 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:34.602 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:34.602 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:34.602 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:34.602 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:34.602 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.596+0000 7f106c2af640 1 Processor -- start 2024-09-15T21:19:34.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.596+0000 7f106c2af640 1 -- start start 2024-09-15T21:19:34.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.596+0000 7f106c2af640 10 monclient: init 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f106409d550 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f106409fae0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f106409e040 0x7f106413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient(hunting): picked mon.noname-c con 0x7f106409e040 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10640a3600 con 0x7f106409d550 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10640a1d60 con 0x7f106409fae0 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f106409f1b0 con 0x7f106409e040 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 10 monclient(hunting): authenticate will time out at 1093.396608s 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a84c640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f106409e040 msgr2=0x7f106413d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f106409fae0 legacy=0x7f1064139c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a84c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f106409e040 0x7f106413d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f106409d550 0x7f106409d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60626/0 (socket says 172.21.15.119:60626) 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 -- 172.21.15.119:0/4129059509 learned_addr learned my addr 172.21.15.119:0/4129059509 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435260976 0 0) 0x7f10640a3600 con 0x7f106409d550 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): _negotiate 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:34.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): my global_id is 4839 2024-09-15T21:19:34.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1050003530 con 0x7f106409d550 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145128006 0 0) 0x7f1050003530 con 0x7f106409d550 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 >> v2:172.21.15.119:3300/0 conn(0x7f106409e040 msgr2=0x7f106413d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 --2- 172.21.15.119:0/4129059509 >> v2:172.21.15.119:3300/0 conn(0x7f106409e040 0x7f106413d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 >> v1:172.21.15.119:6789/0 conn(0x7f106409fae0 legacy=0x7f1064139c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: _finish_hunting 0 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106409e880 con 0x7f106409d550 2024-09-15T21:19:34.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: _finish_auth 0 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: _check_auth_tickets 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 5 monclient: authenticate success, global_id 4839 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f1060001a80 con 0x7f106409d550 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: dump: 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.615 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 >> v1:172.21.15.17:6789/0 conn(0x7f106409d550 legacy=0x7f106409d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1050003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): picked mon.a con 0x7f1050003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1050004810 0x7f1050004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): picked mon.c con 0x7f1050004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1050005420 0x7f105000ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): picked mon.b con 0x7f1050005420 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f106409f1b0 con 0x7f1050003a10 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f105000c360 con 0x7f1050005420 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f105000c560 con 0x7f1050004810 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f1069049640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:34.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106c2af640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a84c640 1 -- 172.21.15.119:0/4129059509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1050005420 msgr2=0x7f105000ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a84c640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1050005420 0x7f105000ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1050003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): get_auth_request con 0x7f1050003a10 auth_method 0 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1050004810 0x7f1050004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): get_auth_request con 0x7f1050004810 auth_method 0 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:34.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106984a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient(hunting): handle_auth_done global_id 4839 payload 293 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 -- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1050004810 msgr2=0x7f1050004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1050004810 0x7f1050004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 -- 172.21.15.119:0/4129059509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1050005420 msgr2=0x7f105000ba80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1050005420 0x7f105000ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 monclient: found mon.a 2024-09-15T21:19:34.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a84c640 1 -- 172.21.15.119:0/4129059509 reap_dead start 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 1 -- 172.21.15.119:0/4129059509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1050003530 con 0x7f1050003a10 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient: _finish_auth 0 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.600+0000 7f106a04b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106a04b640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1050003e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f106000a7a0 tx=0x7f106001d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c15dd620e9cebda server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106000a3f0 con 0x7f1050003a10 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: dump: 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:34.620 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10600ad630 con 0x7f1050003a10 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:34.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 4 set_mon_vals no callback set 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 1 -- 172.21.15.119:0/4129059509 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10600ada40 con 0x7f1050003a10 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f1069049640 10 monclient: dump: 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:34.622 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: shutdown 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 msgr2=0x7f1050003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1050003e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f106000a7a0 tx=0x7f106001d200 comp rx=0 tx=0).stop 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- 172.21.15.119:0/4129059509 shutdown_connections 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 --2- 172.21.15.119:0/4129059509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1050003e10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- 172.21.15.119:0/4129059509 >> 172.21.15.119:0/4129059509 conn(0x7f10640a66e0 msgr2=0x7f10640a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- 172.21.15.119:0/4129059509 shutdown_connections 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- 172.21.15.119:0/4129059509 wait complete. 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: build_initial_monmap 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: monmap: 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:34.623 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 Processor -- start 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- start start 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: init 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1064120f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): picked mon.noname-a con 0x7f1050003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1064121480 0x7f1064121900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): picked mon.noname-b con 0x7f1064121480 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10641cba60 0x7f10641cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): picked mon.noname-c con 0x7f10641cba60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1064135030 con 0x7f1050003a10 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10640a2920 con 0x7f1064121480 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f10641351b0 con 0x7f10641cba60 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106c2af640 10 monclient(hunting): authenticate will time out at 1093.403837s 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106984a640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1064121480 msgr2=0x7f1064121900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.604+0000 7f106984a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1064121480 0x7f1064121900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1064120f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10641cba60 0x7f10641cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1064120f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57638/0 (socket says 172.21.15.119:57638) 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 -- 172.21.15.119:0/2896423213 learned_addr learned my addr 172.21.15.119:0/2896423213 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): get_auth_request con 0x7f1050003a10 auth_method 0 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): get_auth_request con 0x7f10641cba60 auth_method 0 2024-09-15T21:19:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a84c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient(hunting): handle_auth_done global_id 4842 payload 824 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 -- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10641cba60 msgr2=0x7f10641cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10641cba60 0x7f10641cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 -- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1064121480 msgr2=0x7f1064121900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1064121480 0x7f1064121900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 monclient: found mon.noname-a 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 -- 172.21.15.119:0/2896423213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10641249e0 con 0x7f1050003a10 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106a04b640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1064120f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f106001b900 tx=0x7f1050005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10600bc070 con 0x7f1050003a10 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: dump: 2024-09-15T21:19:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: _finish_auth 0 2024-09-15T21:19:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10600ad430 con 0x7f1050003a10 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 5 monclient: authenticate success, global_id 4842 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 monclient: _renew_subs 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1064125170 con 0x7f1050003a10 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060004d70 con 0x7f1050003a10 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 10 monclient: dump: 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f1068848640 4 set_mon_vals no callback set 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 monclient: _renew_subs 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10640a5730 con 0x7f1050003a10 2024-09-15T21:19:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1064135540 con 0x7f1050003a10 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1060001780 con 0x7f1050003a10 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f104803b970 0x7f104803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4708+0+0 (secure 0 0 0) 0x7f10600ea290 con 0x7f1050003a10 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106984a640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f104803b970 0x7f104803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106984a640 10 monclient: get_auth_request con 0x7f104803b970 auth_method 0 2024-09-15T21:19:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.608+0000 7f106984a640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f104803b970 0x7f104803de30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1064067030 tx=0x7f1054006f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.620+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f10600c5020 con 0x7f1050003a10 2024-09-15T21:19:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.620+0000 7f105affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.620+0000 7f105affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:34.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.800+0000 7f106c2af640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T21:19:34.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.800+0000 7f106c2af640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T21:19:34.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.800+0000 7f106c2af640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:34.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:34.800+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) -- 0x7f1064134600 con 0x7f1050003a10 2024-09-15T21:19:35.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.452+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 v51) ==== 150+0+0 (secure 0 0 0) 0x7f1060016fb0 con 0x7f1050003a10 2024-09-15T21:19:35.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.452+0000 7f105affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T21:19:35.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.452+0000 7f105affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:35.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f106c2af640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T21:19:35.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f106c2af640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T21:19:35.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f106c2af640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:35.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) -- 0x7f10640008e0 con 0x7f1050003a10 2024-09-15T21:19:35.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f105affd640 1 -- 172.21.15.119:0/2896423213 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 rule foo-ssd already exists v51) ==== 177+0+0 (secure 0 0 0) 0x7f10600ea080 con 0x7f1050003a10 2024-09-15T21:19:35.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f105affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T21:19:35.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.560+0000 7f105affd640 10 monclient: _finish_command 3 = system:0 rule foo-ssd already exists 2024-09-15T21:19:35.565 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo-ssd already exists 2024-09-15T21:19:35.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f104803b970 msgr2=0x7f104803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:35.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f104803b970 0x7f104803de30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1064067030 tx=0x7f1054006f60 comp rx=0 tx=0).stop 2024-09-15T21:19:35.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 10 monclient: shutdown 2024-09-15T21:19:35.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:35.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 msgr2=0x7f1064120f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:35.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1064120f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f106001b900 tx=0x7f1050005840 comp rx=0 tx=0).stop 2024-09-15T21:19:35.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 shutdown_connections 2024-09-15T21:19:35.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f104803b970 0x7f104803de30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10641cba60 0x7f10641cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1064121480 0x7f1064121900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 --2- 172.21.15.119:0/2896423213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1050003a10 0x7f1064120f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 >> 172.21.15.119:0/2896423213 conn(0x7f10640a66e0 msgr2=0x7f10640714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:35.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 shutdown_connections 2024-09-15T21:19:35.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.564+0000 7f106c2af640 1 -- 172.21.15.119:0/2896423213 wait complete. 2024-09-15T21:19:35.582 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule create-replicated foo-hdd default host hdd 2024-09-15T21:19:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.704+0000 7f3d5045c640 10 monclient: get_monmap_and_config 2024-09-15T21:19:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.704+0000 7f3d5045c640 10 monclient: build_initial_monmap 2024-09-15T21:19:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.704+0000 7f3d5045c640 10 monclient: monmap: 2024-09-15T21:19:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:35.708227+0000 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:35.708227+0000 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:35.710 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 1 Processor -- start 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 1 -- start start 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient: init 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d480a2cf0 0x7f3d480a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient(hunting): picked mon.noname-c con 0x7f3d480a2cf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3d4806cbf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3d4809cb90 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:35.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d4806d030 con 0x7f3d4806cbf0 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d4809c700 con 0x7f3d4809cb90 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3d4806d750 con 0x7f3d480a2cf0 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 10 monclient(hunting): authenticate will time out at 1094.505595s 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f3d480a2cf0 msgr2=0x7f3d480a30d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e9f9640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f3d4809cb90 legacy=0x7f3d481385b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d480a2cf0 0x7f3d480a30d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3d4806cbf0 0x7f3d48137ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60636/0 (socket says 172.21.15.119:60636) 2024-09-15T21:19:35.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 1 -- 172.21.15.119:0/4243390578 learned_addr learned my addr 172.21.15.119:0/4243390578 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874786145 0 0) 0x7f3d38001120 con 0x7f3d4806cbf0 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): _negotiate 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): my global_id is 4845 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d34003530 con 0x7f3d4806cbf0 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1656999644 0 0) 0x7f3d34003530 con 0x7f3d4806cbf0 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 >> v2:172.21.15.119:3300/0 conn(0x7f3d480a2cf0 msgr2=0x7f3d480a30d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 --2- 172.21.15.119:0/4243390578 >> v2:172.21.15.119:3300/0 conn(0x7f3d480a2cf0 0x7f3d480a30d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 >> v1:172.21.15.119:6789/0 conn(0x7f3d4809cb90 legacy=0x7f3d481385b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:35.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: _finish_hunting 0 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d48138cc0 con 0x7f3d4806cbf0 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: _finish_auth 0 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: _check_auth_tickets 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f3d380018d0 con 0x7f3d4806cbf0 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: dump: 2024-09-15T21:19:35.719 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:35.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 >> v1:172.21.15.17:6789/0 conn(0x7f3d4806cbf0 legacy=0x7f3d48137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): picked mon.a con 0x7f3d340039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d34004810 0x7f3d34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): picked mon.c con 0x7f3d34004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340053d0 0x7f3d3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): picked mon.b con 0x7f3d340053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4806d750 con 0x7f3d340039c0 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3400c270 con 0x7f3d340053d0 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3400c470 con 0x7f3d34004810 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d1f6640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e9f9640 1 -- 172.21.15.119:0/4243390578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340053d0 msgr2=0x7f3d3400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e9f9640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340053d0 0x7f3d3400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): get_auth_request con 0x7f3d340039c0 auth_method 0 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d34004810 0x7f3d34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:35.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): get_auth_request con 0x7f3d34004810 auth_method 0 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_done global_id 4845 payload 293 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 -- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d34004810 msgr2=0x7f3d34004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d34004810 0x7f3d34004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 -- 172.21.15.119:0/4243390578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340053d0 msgr2=0x7f3d3400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340053d0 0x7f3d3400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient: _finish_hunting 0 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 monclient: found mon.a 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e9f9640 1 -- 172.21.15.119:0/4243390578 reap_dead start 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 1 -- 172.21.15.119:0/4243390578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d34003530 con 0x7f3d340039c0 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient: _finish_auth 0 2024-09-15T21:19:35.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 10 monclient: _check_auth_tickets 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4e1f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d4d9f7640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d34004810 0x7f3d34004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 5 monclient: authenticate success, global_id 4845 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.708+0000 7f3d5045c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4e1f8640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d34003dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d44003990 tx=0x7f3d4401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54411e17ad3a1650 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44009520 con 0x7f3d340039c0 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: dump: 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:35.725 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d44003bc0 con 0x7f3d340039c0 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 1 -- 172.21.15.119:0/4243390578 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44005070 con 0x7f3d340039c0 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:35.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d4d1f6640 10 monclient: dump: 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 4 set_mon_vals no callback set 2024-09-15T21:19:35.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 10 monclient: shutdown 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 -- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 msgr2=0x7f3d34003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d34003dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d44003990 tx=0x7f3d4401c3f0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 -- 172.21.15.119:0/4243390578 shutdown_connections 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 --2- 172.21.15.119:0/4243390578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d34003dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 -- 172.21.15.119:0/4243390578 >> 172.21.15.119:0/4243390578 conn(0x7f3d480a3de0 msgr2=0x7f3d480a41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 -- 172.21.15.119:0/4243390578 shutdown_connections 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 1 -- 172.21.15.119:0/4243390578 wait complete. 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 10 monclient: build_initial_monmap 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.712+0000 7f3d5045c640 10 monclient: monmap: 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:35.728 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 Processor -- start 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- start start 2024-09-15T21:19:35.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: init 2024-09-15T21:19:35.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:35.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:35.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d48073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): picked mon.noname-a con 0x7f3d340039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:35.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 0x7f3d48121d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): picked mon.noname-c con 0x7f3d48123360 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d48122280 0x7f3d48122730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): picked mon.noname-b con 0x7f3d48122280 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3d480a3a60 con 0x7f3d340039c0 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d480a38e0 con 0x7f3d48122280 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3d48137ea0 con 0x7f3d48123360 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient(hunting): authenticate will time out at 1094.512809s 2024-09-15T21:19:35.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e9f9640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d48122280 msgr2=0x7f3d48122730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e9f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d48122280 0x7f3d48122730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 0x7f3d48121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d48073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 0x7f3d48121d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55514/0 (socket says 172.21.15.119:55514) 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 -- 172.21.15.119:0/3435589480 learned_addr learned my addr 172.21.15.119:0/3435589480 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): get_auth_request con 0x7f3d48123360 auth_method 0 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): get_auth_request con 0x7f3d340039c0 auth_method 0 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:35.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4e1f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient(hunting): handle_auth_done global_id 24236 payload 824 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 -- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d48122280 msgr2=0x7f3d48122730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d48122280 0x7f3d48122730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 -- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 msgr2=0x7f3d48073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d48073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:35.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient: _finish_hunting 0 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 monclient: found mon.noname-c 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 -- 172.21.15.119:0/3435589480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d48124d00 con 0x7f3d48123360 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d4d9f7640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 0x7f3d48121d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d38001a20 tx=0x7f3d380074f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800c0e0 con 0x7f3d48123360 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: dump: 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.738 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: _finish_auth 0 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:35.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 5 monclient: authenticate success, global_id 24236 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _renew_subs 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d48124fe0 con 0x7f3d48123360 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d38018070 con 0x7f3d48123360 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3ffff640 4 set_mon_vals no callback set 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _renew_subs 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d48123ab0 con 0x7f3d48123360 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800cc70 con 0x7f3d48123360 2024-09-15T21:19:35.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 10 monclient: dump: 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:35.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d48138230 con 0x7f3d48123360 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3d3801d460 con 0x7f3d48123360 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d2c03b970 0x7f3d2c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.716+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 4771+0+0 (secure 0 0 0) 0x7f3d38050650 con 0x7f3d48123360 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.720+0000 7f3d4e1f8640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d2c03b970 0x7f3d2c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.720+0000 7f3d4e1f8640 10 monclient: get_auth_request con 0x7f3d2c03b970 auth_method 0 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.720+0000 7f3d4e1f8640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d2c03b970 0x7f3d2c03de30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d440aa000 tx=0x7f3d440b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.732+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3d4806d030 con 0x7f3d48123360 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.732+0000 7f3d3effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:35.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.732+0000 7f3d3effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:35.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.904+0000 7f3d5045c640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T21:19:35.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.904+0000 7f3d5045c640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T21:19:35.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.904+0000 7f3d5045c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:35.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:35.904+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) -- 0x7f3d481247f0 con 0x7f3d48123360 2024-09-15T21:19:36.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.460+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 v52) ==== 150+0+0 (secure 0 0 0) 0x7f3d3800ce10 con 0x7f3d48123360 2024-09-15T21:19:36.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.460+0000 7f3d3effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T21:19:36.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.460+0000 7f3d3effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:36.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d5045c640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T21:19:36.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d5045c640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T21:19:36.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d5045c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:36.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) -- 0x7f3d481237e0 con 0x7f3d48123360 2024-09-15T21:19:36.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d3effd640 1 -- 172.21.15.119:0/3435589480 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 rule foo-hdd already exists v52) ==== 177+0+0 (secure 0 0 0) 0x7f3d3804e160 con 0x7f3d48123360 2024-09-15T21:19:36.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d3effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T21:19:36.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.568+0000 7f3d3effd640 10 monclient: _finish_command 3 = system:0 rule foo-hdd already exists 2024-09-15T21:19:36.573 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo-hdd already exists 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d2c03b970 msgr2=0x7f3d2c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d2c03b970 0x7f3d2c03de30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d440aa000 tx=0x7f3d440b0040 comp rx=0 tx=0).stop 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 10 monclient: shutdown 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 msgr2=0x7f3d48121d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 0x7f3d48121d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d38001a20 tx=0x7f3d380074f0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 shutdown_connections 2024-09-15T21:19:36.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d2c03b970 0x7f3d2c03de30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d48122280 0x7f3d48122730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d48123360 0x7f3d48121d40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 --2- 172.21.15.119:0/3435589480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d340039c0 0x7f3d48073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 >> 172.21.15.119:0/3435589480 conn(0x7f3d480a3de0 msgr2=0x7f3d480a5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:36.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 shutdown_connections 2024-09-15T21:19:36.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.572+0000 7f3d5045c640 1 -- 172.21.15.119:0/3435589480 wait complete. 2024-09-15T21:19:36.591 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls-by-class ssd 2024-09-15T21:19:36.592 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo-ssd 2024-09-15T21:19:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient: get_monmap_and_config 2024-09-15T21:19:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient: build_initial_monmap 2024-09-15T21:19:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient: monmap: 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:36.709539+0000 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:36.709539+0000 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:36.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 1 Processor -- start 2024-09-15T21:19:36.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 1 -- start start 2024-09-15T21:19:36.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient: init 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2aa40a0900 0x7f2aa40a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient(hunting): picked mon.noname-a con 0x7f2aa40a0900 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2aa40a1220 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2aa4135df0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa40a2240 con 0x7f2aa4135df0 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa40a1c80 con 0x7f2aa40a1220 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:36.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa409fdd0 con 0x7f2aa40a0900 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa8ff1640 10 monclient(hunting): authenticate will time out at 1095.506252s 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2a99d9b640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f2aa40a1220 legacy=0x7f2aa41356e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa2d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2aa4135df0 0x7f2aa413dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60640/0 (socket says 172.21.15.119:60640) 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa2d9d640 1 -- 172.21.15.119:0/1539752226 learned_addr learned my addr 172.21.15.119:0/1539752226 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 1 --2- 172.21.15.119:0/1539752226 >> v2:172.21.15.17:3300/0 conn(0x7f2aa40a0900 0x7f2aa40a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): get_auth_request con 0x7f2aa40a0900 auth_method 0 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa1d9b640 1 -- 172.21.15.119:0/1539752226 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938914735 0 0) 0x7f2aa40a2240 con 0x7f2aa4135df0 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa1d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa1d9b640 10 monclient(hunting): my global_id is 4854 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa1d9b640 1 -- 172.21.15.119:0/1539752226 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a90003530 con 0x7f2aa4135df0 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient(hunting): handle_auth_done global_id 4851 payload 274 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 1 -- 172.21.15.119:0/1539752226 >> v1:172.21.15.119:6789/0 conn(0x7f2aa40a1220 legacy=0x7f2aa41356e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 1 -- 172.21.15.119:0/1539752226 >> v1:172.21.15.17:6789/0 conn(0x7f2aa4135df0 legacy=0x7f2aa413dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:36.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 1 monclient: found mon.noname-a 2024-09-15T21:19:36.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:36.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.708+0000 7f2aa259c640 1 -- 172.21.15.119:0/1539752226 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa40a1f60 con 0x7f2aa40a0900 2024-09-15T21:19:36.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa259c640 1 --2- 172.21.15.119:0/1539752226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa40a0900 0x7f2aa40a0ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2aa409ff50 tx=0x7f2a9401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87d20423bafd318 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:36.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 1 -- 172.21.15.119:0/1539752226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940b0070 con 0x7f2aa40a0900 2024-09-15T21:19:36.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:36.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:36.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: dump: 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: _finish_auth 0 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:36.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 1 -- 172.21.15.119:0/1539752226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a940ab600 con 0x7f2aa40a0900 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 1 -- 172.21.15.119:0/1539752226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940ab9d0 con 0x7f2aa40a0900 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 5 monclient: authenticate success, global_id 4851 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 4 set_mon_vals no callback set 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa1d9b640 10 monclient: dump: 2024-09-15T21:19:36.724 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 10 monclient: shutdown 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:36.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/1539752226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa40a0900 msgr2=0x7f2aa40a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/1539752226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa40a0900 0x7f2aa40a0ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2aa409ff50 tx=0x7f2a9401c360 comp rx=0 tx=0).stop 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/1539752226 shutdown_connections 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/1539752226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa40a0900 0x7f2aa40a0ce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/1539752226 >> 172.21.15.119:0/1539752226 conn(0x7f2aa409be60 msgr2=0x7f2aa409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/1539752226 shutdown_connections 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/1539752226 wait complete. 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 10 monclient: build_initial_monmap 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 10 monclient: monmap: 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:36.726 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 Processor -- start 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 1 -- start start 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.712+0000 7f2aa8ff1640 10 monclient: init 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aa40a0900 0x7f2aa41cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): picked mon.noname-b con 0x7f2aa40a0900 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 0x7f2aa41ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:36.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): picked mon.noname-c con 0x7f2aa40a1220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa4135df0 0x7f2aa413c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): picked mon.noname-a con 0x7f2aa4135df0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa406ee90 con 0x7f2aa4135df0 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa406ed10 con 0x7f2aa40a0900 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa40a0050 con 0x7f2aa40a1220 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient(hunting): authenticate will time out at 1095.512281s 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa259c640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aa40a0900 msgr2=0x7f2aa41cdb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aa40a0900 0x7f2aa41cdb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:36.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa4135df0 0x7f2aa413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 0x7f2aa41ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 0x7f2aa41ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55530/0 (socket says 172.21.15.119:55530) 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 1 -- 172.21.15.119:0/582389826 learned_addr learned my addr 172.21.15.119:0/582389826 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): get_auth_request con 0x7f2aa40a1220 auth_method 0 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): get_auth_request con 0x7f2aa4135df0 auth_method 0 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:36.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient(hunting): handle_auth_done global_id 4857 payload 824 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 -- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 msgr2=0x7f2aa41ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 0x7f2aa41ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 -- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aa40a0900 msgr2=0x7f2aa41cdb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aa40a0900 0x7f2aa41cdb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 monclient: found mon.noname-a 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:36.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 -- 172.21.15.119:0/582389826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa411d990 con 0x7f2aa4135df0 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a99d9b640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 0x7f2aa41ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa2d9d640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa4135df0 0x7f2aa413c8c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00ca80 tx=0x7f2a9c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00ac30 con 0x7f2aa4135df0 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: dump: 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:36.731 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 5 monclient: authenticate success, global_id 4857 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9c00add0 con 0x7f2aa4135df0 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c012600 con 0x7f2aa4135df0 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:36.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa159a640 4 set_mon_vals no callback set 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 10 monclient: dump: 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _renew_subs 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa411dc70 con 0x7f2aa4135df0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _renew_subs 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa411d6b0 con 0x7f2aa4135df0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa40a0ce0 con 0x7f2aa4135df0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2a9c0127a0 con 0x7f2aa4135df0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a8403b970 0x7f2a8403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.716+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4834+0+0 (secure 0 0 0) 0x7f2a9c050b30 con 0x7f2aa4135df0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.720+0000 7f2aa259c640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a8403b970 0x7f2a8403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.720+0000 7f2aa259c640 10 monclient: get_auth_request con 0x7f2a8403b970 auth_method 0 2024-09-15T21:19:36.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.720+0000 7f2aa259c640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a8403b970 0x7f2a8403de30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a940ad000 tx=0x7f2a94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:36.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.728+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2a9c0243e0 con 0x7f2aa4135df0 2024-09-15T21:19:36.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.732+0000 7f2a9b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:36.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.732+0000 7f2a9b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:36.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2aa8ff1640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:36.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2aa8ff1640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:36.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2aa8ff1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7f2aa409d0f0 con 0x7f2aa4135df0 2024-09-15T21:19:36.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f2a9c025d90 con 0x7f2aa4135df0 2024-09-15T21:19:36.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2a9b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:36.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.896+0000 7f2a9b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:36.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2aa8ff1640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:36.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2aa8ff1640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:36.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2aa8ff1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:36.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7f2aa41ce5d0 con 0x7f2aa4135df0 2024-09-15T21:19:36.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2a9b7fe640 1 -- 172.21.15.119:0/582389826 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f2a9c04e600 con 0x7f2aa4135df0 2024-09-15T21:19:36.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2a9b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:36.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.992+0000 7f2a9b7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:36.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a8403b970 msgr2=0x7f2a8403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a8403b970 0x7f2a8403de30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a940ad000 tx=0x7f2a94008040 comp rx=0 tx=0).stop 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 10 monclient: shutdown 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa4135df0 msgr2=0x7f2aa413c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa4135df0 0x7f2aa413c8c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00ca80 tx=0x7f2a9c00cf40 comp rx=0 tx=0).stop 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 shutdown_connections 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a8403b970 0x7f2a8403de30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa4135df0 0x7f2aa413c8c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aa40a1220 0x7f2aa41ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 --2- 172.21.15.119:0/582389826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aa40a0900 0x7f2aa41cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 >> 172.21.15.119:0/582389826 conn(0x7f2aa409be60 msgr2=0x7f2aa40a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:37.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 shutdown_connections 2024-09-15T21:19:37.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:36.996+0000 7f2aa8ff1640 1 -- 172.21.15.119:0/582389826 wait complete. 2024-09-15T21:19:37.013 INFO:tasks.workunit.client.0.smithi119.stdout:foo-ssd 2024-09-15T21:19:37.014 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls-by-class ssd 2024-09-15T21:19:37.014 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false grep foo-hdd 2024-09-15T21:19:37.014 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:37.014 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo-hdd 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 10 monclient: get_monmap_and_config 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 10 monclient: build_initial_monmap 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 10 monclient: monmap: 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:37.127460+0000 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:37.127460+0000 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:37.130 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 1 Processor -- start 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 1 -- start start 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 10 monclient: init 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.124+0000 7f5ea7561640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ea00a0900 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ea00a13f0 0x7f5ea013ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): picked mon.noname-c con 0x7f5ea00a13f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea013d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): picked mon.noname-b con 0x7f5ea013b150 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea00a2240 con 0x7f5ea00a0900 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea009fdd0 con 0x7f5ea00a13f0 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea00a1c80 con 0x7f5ea013b150 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient(hunting): authenticate will time out at 1095.924084s 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea4afc640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f5ea00a13f0 msgr2=0x7f5ea013ac10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea4afc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ea00a13f0 0x7f5ea013ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea52fd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5ea00a0900 0x7f5ea00a0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:60650/0 (socket says 172.21.15.119:60650) 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea52fd640 1 -- 172.21.15.119:0/1709726054 learned_addr learned my addr 172.21.15.119:0/1709726054 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): get_auth_request con 0x7f5ea013b150 auth_method 0 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_done global_id 24242 payload 274 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1709726054 >> v2:172.21.15.119:3300/0 conn(0x7f5ea00a13f0 msgr2=0x7f5ea013ac10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 --2- 172.21.15.119:0/1709726054 >> v2:172.21.15.119:3300/0 conn(0x7f5ea00a13f0 0x7f5ea013ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1709726054 >> v1:172.21.15.17:6789/0 conn(0x7f5ea00a0900 legacy=0x7f5ea00a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 1 -- 172.21.15.119:0/1709726054 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374060071 0 0) 0x7f5ea00a2240 con 0x7f5ea00a0900 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 monclient: found mon.noname-b 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1709726054 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea00a1f60 con 0x7f5ea013b150 2024-09-15T21:19:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea5afe640 1 --2- 172.21.15.119:0/1709726054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea013d540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009d90 tx=0x7f5e9801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9097af4cea8f731e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 1 -- 172.21.15.119:0/1709726054 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980b2070 con 0x7f5ea013b150 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: dump: 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 5 monclient: authenticate success, global_id 24242 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 1 -- 172.21.15.119:0/1709726054 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e980a5930 con 0x7f5ea013b150 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5ea7561640 4 set_mon_vals no callback set 2024-09-15T21:19:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 1 -- 172.21.15.119:0/1709726054 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980a5d00 con 0x7f5ea013b150 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.128+0000 7f5e97fff640 10 monclient: dump: 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: shutdown 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1709726054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 msgr2=0x7f5ea013d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1709726054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea013d540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009d90 tx=0x7f5e9801c850 comp rx=0 tx=0).stop 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1709726054 shutdown_connections 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1709726054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea013d540 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1709726054 >> v2:172.21.15.119:3300/0 conn(0x7f5ea00a13f0 0x7f5ea013ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1709726054 >> 172.21.15.119:0/1709726054 conn(0x7f5ea009be60 msgr2=0x7f5ea009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1709726054 shutdown_connections 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1709726054 wait complete. 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: build_initial_monmap 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: monmap: 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:37.142 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 Processor -- start 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- start start 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: init 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea00a0900 0x7f5ea011dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): picked mon.noname-b con 0x7f5ea00a0900 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea00a13f0 0x7f5ea011e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): picked mon.noname-a con 0x7f5ea00a13f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea011e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): picked mon.noname-c con 0x7f5ea013b150 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00a01d0 con 0x7f5ea00a13f0 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00a0050 con 0x7f5ea00a0900 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea01402a0 con 0x7f5ea013b150 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient(hunting): authenticate will time out at 1095.929854s 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea52fd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea00a0900 msgr2=0x7f5ea011dbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:37.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea52fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea00a0900 0x7f5ea011dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea00a13f0 0x7f5ea011e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea011e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea011e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55540/0 (socket says 172.21.15.119:55540) 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1258971387 learned_addr learned my addr 172.21.15.119:0/1258971387 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): get_auth_request con 0x7f5ea00a13f0 auth_method 0 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): get_auth_request con 0x7f5ea013b150 auth_method 0 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea4afc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient(hunting): handle_auth_done global_id 24245 payload 824 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea00a0900 msgr2=0x7f5ea011dbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea00a0900 0x7f5ea011dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea00a13f0 msgr2=0x7f5ea011e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea00a13f0 0x7f5ea011e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 monclient: found mon.noname-c 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:37.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 -- 172.21.15.119:0/1258971387 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea00702b0 con 0x7f5ea013b150 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea5afe640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea011e660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e980b0040 tx=0x7f5e980195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980bd030 con 0x7f5ea013b150 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: dump: 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.147 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e980ad580 con 0x7f5ea013b150 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 5 monclient: authenticate success, global_id 24245 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: _renew_subs 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea009d0f0 con 0x7f5ea013b150 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980b2040 con 0x7f5ea013b150 2024-09-15T21:19:37.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e95ffb640 10 monclient: dump: 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5e977fe640 4 set_mon_vals no callback set 2024-09-15T21:19:37.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: _renew_subs 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.132+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea006ffd0 con 0x7f5ea013b150 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea7561640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea7561640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea7561640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea00a0ce0 con 0x7f5ea013b150 2024-09-15T21:19:37.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e980c0070 con 0x7f5ea013b150 2024-09-15T21:19:37.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5e95ffb640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e8403b970 0x7f5e8403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 4834+0+0 (secure 0 0 0) 0x7f5e980eb0f0 con 0x7f5ea013b150 2024-09-15T21:19:37.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea52fd640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e8403b970 0x7f5e8403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea52fd640 10 monclient: get_auth_request con 0x7f5e8403b970 auth_method 0 2024-09-15T21:19:37.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.136+0000 7f5ea52fd640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e8403b970 0x7f5e8403de30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e900010f0 tx=0x7f5e90007a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.148+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5e980c7070 con 0x7f5ea013b150 2024-09-15T21:19:37.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.148+0000 7f5e95ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.148+0000 7f5e95ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:37.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.320+0000 7f5ea7561640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:37.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.320+0000 7f5ea7561640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:37.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.320+0000 7f5ea7561640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.320+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7f5ea013ae50 con 0x7f5ea013b150 2024-09-15T21:19:37.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.324+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f5e98016560 con 0x7f5ea013b150 2024-09-15T21:19:37.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.324+0000 7f5e95ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:37.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.324+0000 7f5e95ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:37.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5ea7561640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:37.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5ea7561640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:37.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5ea7561640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7f5ea0120100 con 0x7f5ea013b150 2024-09-15T21:19:37.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5e95ffb640 1 -- 172.21.15.119:0/1258971387 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f5e980c2d10 con 0x7f5ea013b150 2024-09-15T21:19:37.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5e95ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T21:19:37.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.424+0000 7f5e95ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:37.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e8403b970 msgr2=0x7f5e8403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e8403b970 0x7f5e8403de30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e900010f0 tx=0x7f5e90007a90 comp rx=0 tx=0).stop 2024-09-15T21:19:37.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 10 monclient: shutdown 2024-09-15T21:19:37.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:37.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 msgr2=0x7f5ea011e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea011e660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e980b0040 tx=0x7f5e980195f0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 shutdown_connections 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e8403b970 0x7f5e8403de30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ea013b150 0x7f5ea011e660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea00a13f0 0x7f5ea011e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 --2- 172.21.15.119:0/1258971387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea00a0900 0x7f5ea011dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 >> 172.21.15.119:0/1258971387 conn(0x7f5ea009be60 msgr2=0x7f5ea00a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 shutdown_connections 2024-09-15T21:19:37.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.428+0000 7f5ea7561640 1 -- 172.21.15.119:0/1258971387 wait complete. 2024-09-15T21:19:37.447 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:37.447 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo-hdd 2024-09-15T21:19:37.448 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls-by-class hdd 2024-09-15T21:19:37.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.556+0000 7f5dfb348640 10 monclient: get_monmap_and_config 2024-09-15T21:19:37.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.556+0000 7f5dfb348640 10 monclient: build_initial_monmap 2024-09-15T21:19:37.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.556+0000 7f5dfb348640 10 monclient: monmap: 2024-09-15T21:19:37.562 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:37.562 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.562 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:37.559961+0000 2024-09-15T21:19:37.562 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:37.559961+0000 2024-09-15T21:19:37.562 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:37.563 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.563 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:37.563 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:37.563 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:37.563 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:37.563 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:37.564 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.556+0000 7f5dfb348640 1 Processor -- start 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.556+0000 7f5dfb348640 1 -- start start 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.556+0000 7f5dfb348640 10 monclient: init 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df40a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): picked mon.noname-b con 0x7f5df40a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5df40a1220 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:37.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5df4135df0 0x7f5df413dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): picked mon.noname-a con 0x7f5df4135df0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df40a2240 con 0x7f5df40a1220 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5df409fdd0 con 0x7f5df4135df0 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40a1c80 con 0x7f5df40a0900 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df88e3640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f5df40a1220 legacy=0x7f5df41356e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5dfb348640 10 monclient(hunting): authenticate will time out at 1096.356701s 2024-09-15T21:19:37.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df40a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5df4135df0 0x7f5df413dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df40a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55544/0 (socket says 172.21.15.119:55544) 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5df4135df0 0x7f5df413dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57696/0 (socket says 172.21.15.119:57696) 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 1 -- 172.21.15.119:0/640745925 learned_addr learned my addr 172.21.15.119:0/640745925 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): get_auth_request con 0x7f5df40a0900 auth_method 0 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): get_auth_request con 0x7f5df4135df0 auth_method 0 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient(hunting): handle_auth_done global_id 4866 payload 274 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 -- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 msgr2=0x7f5df40a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:37.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 --2- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df40a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 -- 172.21.15.119:0/640745925 >> v1:172.21.15.119:6789/0 conn(0x7f5df40a1220 legacy=0x7f5df41356e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:37.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient: _finish_hunting 0 2024-09-15T21:19:37.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 monclient: found mon.noname-a 2024-09-15T21:19:37.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:37.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:37.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df98e5640 1 -- 172.21.15.119:0/640745925 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df40a1f60 con 0x7f5df4135df0 2024-09-15T21:19:37.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.560+0000 7f5df90e4640 1 --2- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df40a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:37.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 1 --2- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df413dc30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009d90 tx=0x7f5dec01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a7d025a0081587 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 1 -- 172.21.15.119:0/640745925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0b2070 con 0x7f5df4135df0 2024-09-15T21:19:37.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:37.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: dump: 2024-09-15T21:19:37.582 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.583 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.583 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.583 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.583 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.583 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.583 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.584 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.584 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.584 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:37.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:37.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:37.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 1 -- 172.21.15.119:0/640745925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec0a5890 con 0x7f5df4135df0 2024-09-15T21:19:37.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:37.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 1 -- 172.21.15.119:0/640745925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a5c60 con 0x7f5df4135df0 2024-09-15T21:19:37.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 5 monclient: authenticate success, global_id 4866 2024-09-15T21:19:37.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:37.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 4 set_mon_vals no callback set 2024-09-15T21:19:37.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:37.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5debfff640 10 monclient: dump: 2024-09-15T21:19:37.586 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.587 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.587 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.587 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.587 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.587 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.587 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.588 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.588 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.588 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: shutdown 2024-09-15T21:19:37.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:37.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 msgr2=0x7f5df413dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 --2- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df413dc30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009d90 tx=0x7f5dec01c830 comp rx=0 tx=0).stop 2024-09-15T21:19:37.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- 172.21.15.119:0/640745925 shutdown_connections 2024-09-15T21:19:37.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 --2- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df413dc30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 --2- 172.21.15.119:0/640745925 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df40a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- 172.21.15.119:0/640745925 >> 172.21.15.119:0/640745925 conn(0x7f5df409be60 msgr2=0x7f5df409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:37.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- 172.21.15.119:0/640745925 shutdown_connections 2024-09-15T21:19:37.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- 172.21.15.119:0/640745925 wait complete. 2024-09-15T21:19:37.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: build_initial_monmap 2024-09-15T21:19:37.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: monmap: 2024-09-15T21:19:37.590 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:37.590 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:37.591 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:37.591 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:37.591 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:37.591 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.591 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:37.592 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:37.592 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:37.592 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 Processor -- start 2024-09-15T21:19:37.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- start start 2024-09-15T21:19:37.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: init 2024-09-15T21:19:37.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:37.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:37.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df4072510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): picked mon.noname-c con 0x7f5df40a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:37.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df40a1220 0x7f5df4132be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): picked mon.noname-b con 0x7f5df40a1220 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:37.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df4133120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): picked mon.noname-a con 0x7f5df4135df0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:37.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5df406ee90 con 0x7f5df4135df0 2024-09-15T21:19:37.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5df406ed10 con 0x7f5df40a1220 2024-09-15T21:19:37.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40a0050 con 0x7f5df40a0900 2024-09-15T21:19:37.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:37.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5dfb348640 10 monclient(hunting): authenticate will time out at 1096.362919s 2024-09-15T21:19:37.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df88e3640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df40a1220 msgr2=0x7f5df4132be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:37.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df88e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df40a1220 0x7f5df4132be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:37.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df4072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df4072510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55550/0 (socket says 172.21.15.119:55550) 2024-09-15T21:19:37.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 -- 172.21.15.119:0/653547345 learned_addr learned my addr 172.21.15.119:0/653547345 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:37.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df4133120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): get_auth_request con 0x7f5df40a0900 auth_method 0 2024-09-15T21:19:37.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): get_auth_request con 0x7f5df4135df0 auth_method 0 2024-09-15T21:19:37.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df98e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient(hunting): handle_auth_done global_id 24251 payload 824 2024-09-15T21:19:37.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 -- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df40a1220 msgr2=0x7f5df4132be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:37.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df40a1220 0x7f5df4132be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 -- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 msgr2=0x7f5df4133120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df4133120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient: _finish_hunting 0 2024-09-15T21:19:37.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 monclient: found mon.noname-c 2024-09-15T21:19:37.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:37.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.564+0000 7f5df90e4640 1 -- 172.21.15.119:0/653547345 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df40a5790 con 0x7f5df40a0900 2024-09-15T21:19:37.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5df90e4640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df4072510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5de400e930 tx=0x7f5de400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5df40a0900 2024-09-15T21:19:37.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:37.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: dump: 2024-09-15T21:19:37.602 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.602 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.602 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.602 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.602 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.603 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.603 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.603 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.603 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.603 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:37.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:37.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:37.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 5 monclient: authenticate success, global_id 24251 2024-09-15T21:19:37.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de400ca00 con 0x7f5df40a0900 2024-09-15T21:19:37.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 monclient: _renew_subs 2024-09-15T21:19:37.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df411f2d0 con 0x7f5df40a0900 2024-09-15T21:19:37.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:37.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400cdd0 con 0x7f5df40a0900 2024-09-15T21:19:37.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:37.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5deb7fe640 4 set_mon_vals no callback set 2024-09-15T21:19:37.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 10 monclient: dump: 2024-09-15T21:19:37.606 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.606 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.606 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.606 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.607 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.607 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.607 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.607 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.607 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.608 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 monclient: _renew_subs 2024-09-15T21:19:37.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df409db70 con 0x7f5df40a0900 2024-09-15T21:19:37.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df409d620 con 0x7f5df40a0900 2024-09-15T21:19:37.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5de401a070 con 0x7f5df40a0900 2024-09-15T21:19:37.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5dd803b970 0x7f5dd803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 4834+0+0 (secure 0 0 0) 0x7f5de404fff0 con 0x7f5df40a0900 2024-09-15T21:19:37.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5df88e3640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5dd803b970 0x7f5dd803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5df88e3640 10 monclient: get_auth_request con 0x7f5dd803b970 auth_method 0 2024-09-15T21:19:37.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.568+0000 7f5df88e3640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5dd803b970 0x7f5dd803de30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5df41d0ca0 tx=0x7f5df0006d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.580+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5de404da60 con 0x7f5df40a0900 2024-09-15T21:19:37.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.580+0000 7f5de9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.580+0000 7f5de9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:37.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5dfb348640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:37.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5dfb348640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:37.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5dfb348640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f5df409d0f0 con 0x7f5df40a0900 2024-09-15T21:19:37.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f5de404dde0 con 0x7f5df40a0900 2024-09-15T21:19:37.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5de9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:37.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.748+0000 7f5de9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:37.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:37.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:37.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:37.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f5df40a0ce0 con 0x7f5df40a0900 2024-09-15T21:19:37.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5de9ffb640 1 -- 172.21.15.119:0/653547345 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f5de4013c20 con 0x7f5df40a0900 2024-09-15T21:19:37.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5de9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:37.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5de9ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:37.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5dd803b970 msgr2=0x7f5dd803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5dd803b970 0x7f5dd803de30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5df41d0ca0 tx=0x7f5df0006d10 comp rx=0 tx=0).stop 2024-09-15T21:19:37.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 10 monclient: shutdown 2024-09-15T21:19:37.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:37.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 msgr2=0x7f5df4072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df4072510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5de400e930 tx=0x7f5de400edf0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 shutdown_connections 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5dd803b970 0x7f5dd803de30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5df4135df0 0x7f5df4133120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df40a1220 0x7f5df4132be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 --2- 172.21.15.119:0/653547345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5df40a0900 0x7f5df4072510 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.844+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 >> 172.21.15.119:0/653547345 conn(0x7f5df409be60 msgr2=0x7f5df40a5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.848+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 shutdown_connections 2024-09-15T21:19:37.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.848+0000 7f5dfb348640 1 -- 172.21.15.119:0/653547345 wait complete. 2024-09-15T21:19:37.863 INFO:tasks.workunit.client.0.smithi119.stdout:foo-hdd 2024-09-15T21:19:37.864 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls-by-class hdd 2024-09-15T21:19:37.864 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false grep foo-ssd 2024-09-15T21:19:37.864 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:37.864 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo-ssd 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient: get_monmap_and_config 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient: build_initial_monmap 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient: monmap: 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:37.965718+0000 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:37.965718+0000 2024-09-15T21:19:37.967 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:37.968 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 Processor -- start 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 -- start start 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient: init 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f883006dc00 0x7f88300a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): picked mon.noname-a con 0x7f883006dc00 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f883009ec20 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f88300a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): picked mon.noname-b con 0x7f88300a18a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:37.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883009e240 con 0x7f883009ec20 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f883009e520 con 0x7f883006dc00 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f883009e800 con 0x7f88300a18a0 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f883611d640 10 monclient(hunting): authenticate will time out at 1096.762943s 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882effd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f883009ec20 legacy=0x7f88300a1190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f88300a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f883006dc00 0x7f88300a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f88300a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55554/0 (socket says 172.21.15.119:55554) 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f883006dc00 0x7f88300a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57722/0 (socket says 172.21.15.119:57722) 2024-09-15T21:19:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 1 -- 172.21.15.119:0/1568486490 learned_addr learned my addr 172.21.15.119:0/1568486490 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): get_auth_request con 0x7f88300a18a0 auth_method 0 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): get_auth_request con 0x7f883006dc00 auth_method 0 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient(hunting): handle_auth_done global_id 4872 payload 274 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 -- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 msgr2=0x7f88300a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 --2- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f88300a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 -- 172.21.15.119:0/1568486490 >> v1:172.21.15.119:6789/0 conn(0x7f883009ec20 legacy=0x7f88300a1190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:37.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882f7fe640 1 -- 172.21.15.119:0/1568486490 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8830074110 con 0x7f883006dc00 2024-09-15T21:19:37.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.964+0000 7f882ffff640 1 --2- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f88300a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882f7fe640 1 --2- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883006dc00 0x7f88300a0010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f883006a030 tx=0x7f882001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d784800b19d611bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 1 -- 172.21.15.119:0/1568486490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88200b0070 con 0x7f883006dc00 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: dump: 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.980 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 1 -- 172.21.15.119:0/1568486490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88200ab400 con 0x7f883006dc00 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:37.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 1 -- 172.21.15.119:0/1568486490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88200ab7d0 con 0x7f883006dc00 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 5 monclient: authenticate success, global_id 4872 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 4 set_mon_vals no callback set 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f882e7fc640 10 monclient: dump: 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.982 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 10 monclient: shutdown 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 -- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883006dc00 msgr2=0x7f88300a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 --2- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883006dc00 0x7f88300a0010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f883006a030 tx=0x7f882001c360 comp rx=0 tx=0).stop 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 -- 172.21.15.119:0/1568486490 shutdown_connections 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 --2- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f88300a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 --2- 172.21.15.119:0/1568486490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883006dc00 0x7f88300a0010 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 -- 172.21.15.119:0/1568486490 >> 172.21.15.119:0/1568486490 conn(0x7f883009be60 msgr2=0x7f883009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 -- 172.21.15.119:0/1568486490 shutdown_connections 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 -- 172.21.15.119:0/1568486490 wait complete. 2024-09-15T21:19:37.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 10 monclient: build_initial_monmap 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 10 monclient: monmap: 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 Processor -- start 2024-09-15T21:19:37.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 1 -- start start 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.968+0000 7f883611d640 10 monclient: init 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f883006dc00 0x7f88301cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): picked mon.noname-b con 0x7f883006dc00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 0x7f88301378f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): picked mon.noname-a con 0x7f883009ec20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f8830137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): picked mon.noname-c con 0x7f88300a18a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f883013f1c0 con 0x7f883009ec20 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f883013f040 con 0x7f883006dc00 2024-09-15T21:19:37.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f883013f340 con 0x7f88300a18a0 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient(hunting): authenticate will time out at 1096.768400s 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882f7fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f883006dc00 msgr2=0x7f88301cf960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f883006dc00 0x7f88301cf960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 0x7f88301378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 0x7f88301378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57726/0 (socket says 172.21.15.119:57726) 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 -- 172.21.15.119:0/265850148 learned_addr learned my addr 172.21.15.119:0/265850148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): get_auth_request con 0x7f883009ec20 auth_method 0 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f8830137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): get_auth_request con 0x7f88300a18a0 auth_method 0 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient(hunting): handle_auth_done global_id 4875 payload 824 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 -- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 msgr2=0x7f8830137e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:37.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f8830137e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 -- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f883006dc00 msgr2=0x7f88301cf960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f883006dc00 0x7f88301cf960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 monclient: found mon.noname-a 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 -- 172.21.15.119:0/265850148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8830123390 con 0x7f883009ec20 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882effd640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 0x7f88301378f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f881800b4a0 tx=0x7f881800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818003ba0 con 0x7f883009ec20 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: dump: 2024-09-15T21:19:37.988 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 5 monclient: authenticate success, global_id 4875 2024-09-15T21:19:37.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8818015070 con 0x7f883009ec20 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _renew_subs 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883009d0f0 con 0x7f883009ec20 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818005070 con 0x7f883009ec20 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 10 monclient: dump: 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882dffb640 4 set_mon_vals no callback set 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _renew_subs 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301411b0 con 0x7f883009ec20 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88300a0010 con 0x7f883009ec20 2024-09-15T21:19:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8818005210 con 0x7f883009ec20 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f881003b970 0x7f881003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4834+0+0 (secure 0 0 0) 0x7f881804ff80 con 0x7f883009ec20 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882f7fe640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f881003b970 0x7f881003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882f7fe640 10 monclient: get_auth_request con 0x7f881003b970 auth_method 0 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.972+0000 7f882f7fe640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f881003b970 0x7f881003de30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88200ad000 tx=0x7f8820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.984+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f88180193c0 con 0x7f883009ec20 2024-09-15T21:19:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.984+0000 7f8827fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:37.984+0000 7f8827fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:38.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f883611d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:38.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f883611d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:38.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f883611d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:38.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f883006a810 con 0x7f883009ec20 2024-09-15T21:19:38.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f881801adc0 con 0x7f883009ec20 2024-09-15T21:19:38.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f8827fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:38.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.144+0000 7f8827fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:38.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f883611d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:38.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f883611d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:38.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f883611d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:38.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f8830121b90 con 0x7f883009ec20 2024-09-15T21:19:38.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f8827fff640 1 -- 172.21.15.119:0/265850148 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v52) ==== 90+0+8 (secure 0 0 0) 0x7f881804da50 con 0x7f883009ec20 2024-09-15T21:19:38.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f8827fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T21:19:38.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.240+0000 7f8827fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f881003b970 msgr2=0x7f881003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f881003b970 0x7f881003de30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88200ad000 tx=0x7f8820008040 comp rx=0 tx=0).stop 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 10 monclient: shutdown 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 msgr2=0x7f88301378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 0x7f88301378f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f881800b4a0 tx=0x7f881800b960 comp rx=0 tx=0).stop 2024-09-15T21:19:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 shutdown_connections 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f881003b970 0x7f881003de30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88300a18a0 0x7f8830137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f883009ec20 0x7f88301378f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 --2- 172.21.15.119:0/265850148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f883006dc00 0x7f88301cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 >> 172.21.15.119:0/265850148 conn(0x7f883009be60 msgr2=0x7f883009f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 shutdown_connections 2024-09-15T21:19:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.244+0000 7f883611d640 1 -- 172.21.15.119:0/265850148 wait complete. 2024-09-15T21:19:38.261 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:38.261 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd erasure-code-profile set ec-foo-ssd crush-device-class=ssd m=2 k=2 2024-09-15T21:19:38.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient: get_monmap_and_config 2024-09-15T21:19:38.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient: build_initial_monmap 2024-09-15T21:19:38.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient: monmap: 2024-09-15T21:19:38.380 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:38.380 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:38.380 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:38.378823+0000 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:38.378823+0000 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:38.381 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:38.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 Processor -- start 2024-09-15T21:19:38.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 -- start start 2024-09-15T21:19:38.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient: init 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ef009d550 0x7f3ef009d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): picked mon.noname-a con 0x7f3ef009d550 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef009de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): picked mon.noname-b con 0x7f3ef009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3ef0134820 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef00a3600 con 0x7f3ef0134820 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef009f1b0 con 0x7f3ef009d550 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef00a2920 con 0x7f3ef009fae0 2024-09-15T21:19:38.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3ef66f9640 10 monclient(hunting): authenticate will time out at 1097.175417s 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef009de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eeffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ef009d550 0x7f3ef009d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef009de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55562/0 (socket says 172.21.15.119:55562) 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 1 -- 172.21.15.119:0/3444365863 learned_addr learned my addr 172.21.15.119:0/3444365863 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 10 monclient(hunting): get_auth_request con 0x7f3ef009fae0 auth_method 0 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eeffff640 10 monclient(hunting): get_auth_request con 0x7f3ef009d550 auth_method 0 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:38.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.376+0000 7f3eeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_done global_id 24260 payload 274 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 1 -- 172.21.15.119:0/3444365863 >> v2:172.21.15.17:3300/0 conn(0x7f3ef009d550 msgr2=0x7f3ef009d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 1 --2- 172.21.15.119:0/3444365863 >> v2:172.21.15.17:3300/0 conn(0x7f3ef009d550 0x7f3ef009d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 1 -- 172.21.15.119:0/3444365863 >> v1:172.21.15.17:6789/0 conn(0x7f3ef0134820 legacy=0x7f3ef0134cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:38.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:38.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:38.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:38.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:38.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 1 -- 172.21.15.119:0/3444365863 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef00a1d60 con 0x7f3ef009fae0 2024-09-15T21:19:38.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eef7fe640 1 --2- 172.21.15.119:0/3444365863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef009de70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009870 tx=0x7f3ee401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6d7a0b1b1c9f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:38.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 1 -- 172.21.15.119:0/3444365863 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b0070 con 0x7f3ef009fae0 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: dump: 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:38.393 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 1 -- 172.21.15.119:0/3444365863 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee40ab600 con 0x7f3ef009fae0 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 5 monclient: authenticate success, global_id 24260 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 4 set_mon_vals no callback set 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 1 -- 172.21.15.119:0/3444365863 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40ab9d0 con 0x7f3ef009fae0 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3eeeffd640 10 monclient: dump: 2024-09-15T21:19:38.394 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:38.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 10 monclient: shutdown 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 -- 172.21.15.119:0/3444365863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 msgr2=0x7f3ef009de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/3444365863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef009de70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009870 tx=0x7f3ee401c420 comp rx=0 tx=0).stop 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 -- 172.21.15.119:0/3444365863 shutdown_connections 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/3444365863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef009de70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/3444365863 >> v2:172.21.15.17:3300/0 conn(0x7f3ef009d550 0x7f3ef009d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 -- 172.21.15.119:0/3444365863 >> 172.21.15.119:0/3444365863 conn(0x7f3ef00a66e0 msgr2=0x7f3ef00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 -- 172.21.15.119:0/3444365863 shutdown_connections 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.380+0000 7f3ef66f9640 1 -- 172.21.15.119:0/3444365863 wait complete. 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: build_initial_monmap 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: monmap: 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:38.396 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 Processor -- start 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- start start 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: init 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:38.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 0x7f3ef0138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): picked mon.noname-a con 0x7f3ef009d550 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef0138aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): picked mon.noname-c con 0x7f3ef009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef0134820 0x7f3ef0138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): picked mon.noname-b con 0x7f3ef0134820 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef013d4a0 con 0x7f3ef009d550 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef009e3b0 con 0x7f3ef0134820 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef013d620 con 0x7f3ef009fae0 2024-09-15T21:19:38.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient(hunting): authenticate will time out at 1097.181308s 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef4c96640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef0134820 msgr2=0x7f3ef0138fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef4c96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef0134820 0x7f3ef0138fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 0x7f3ef0138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef0138aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 0x7f3ef0138560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57750/0 (socket says 172.21.15.119:57750) 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): get_auth_request con 0x7f3ef009fae0 auth_method 0 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 1 -- 172.21.15.119:0/195880930 learned_addr learned my addr 172.21.15.119:0/195880930 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): get_auth_request con 0x7f3ef009d550 auth_method 0 2024-09-15T21:19:38.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient(hunting): handle_auth_done global_id 24263 payload 824 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 -- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef0134820 msgr2=0x7f3ef0138fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef0134820 0x7f3ef0138fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 -- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 msgr2=0x7f3ef0138560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:38.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 0x7f3ef0138560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 -- 172.21.15.119:0/195880930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef009d240 con 0x7f3ef009fae0 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eeffff640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 0x7f3ef0138560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eef7fe640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef0138aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40ab000 tx=0x7f3ee40191c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b8020 con 0x7f3ef009fae0 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: dump: 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:38.401 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 5 monclient: authenticate success, global_id 24263 2024-09-15T21:19:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee40b4410 con 0x7f3ef009fae0 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b0040 con 0x7f3ef009fae0 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _renew_subs 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef011c740 con 0x7f3ef009fae0 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eee7fc640 4 set_mon_vals no callback set 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3eecff9640 10 monclient: dump: 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _renew_subs 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef00a47a0 con 0x7f3ef009fae0 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.384+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef013d9b0 con 0x7f3ef009fae0 2024-09-15T21:19:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.388+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3ee40bb020 con 0x7f3ef009fae0 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.388+0000 7f3eecff9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3edc03b970 0x7f3edc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.388+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 4834+0+0 (secure 0 0 0) 0x7f3ee40e9ae0 con 0x7f3ef009fae0 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.388+0000 7f3eeffff640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3edc03b970 0x7f3edc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.388+0000 7f3eeffff640 10 monclient: get_auth_request con 0x7f3edc03b970 auth_method 0 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.388+0000 7f3eeffff640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3edc03b970 0x7f3edc03de30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009a40 tx=0x7f3ee800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.400+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3ee40c2030 con 0x7f3ef009fae0 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.400+0000 7f3eecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.400+0000 7f3eecff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:38.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.576+0000 7f3ef66f9640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T21:19:38.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.576+0000 7f3ef66f9640 10 monclient: _send_command 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T21:19:38.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.576+0000 7f3ef66f9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:38.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:38.576+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) -- 0x7f3ef006a830 con 0x7f3ef009fae0 2024-09-15T21:19:39.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.420+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v53) ==== 151+0+0 (secure 0 0 0) 0x7f3ef009d240 con 0x7f3ef009fae0 2024-09-15T21:19:39.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.420+0000 7f3eecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T21:19:39.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.420+0000 7f3eecff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:39.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.528+0000 7f3ef66f9640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T21:19:39.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.528+0000 7f3ef66f9640 10 monclient: _send_command 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T21:19:39.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.528+0000 7f3ef66f9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:39.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.528+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) -- 0x7f3ef013ada0 con 0x7f3ef009fae0 2024-09-15T21:19:39.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3eecff9640 1 -- 172.21.15.119:0/195880930 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v53) ==== 151+0+0 (secure 0 0 0) 0x7f3ee4016130 con 0x7f3ef009fae0 2024-09-15T21:19:39.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3eecff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T21:19:39.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3eecff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3edc03b970 msgr2=0x7f3edc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3edc03b970 0x7f3edc03de30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009a40 tx=0x7f3ee800b040 comp rx=0 tx=0).stop 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 10 monclient: shutdown 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 msgr2=0x7f3ef0138aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef0138aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40ab000 tx=0x7f3ee40191c0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 shutdown_connections 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3edc03b970 0x7f3edc03de30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef0134820 0x7f3ef0138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef009fae0 0x7f3ef0138aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 --2- 172.21.15.119:0/195880930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef009d550 0x7f3ef0138560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 >> 172.21.15.119:0/195880930 conn(0x7f3ef00a66e0 msgr2=0x7f3ef00a4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:39.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 shutdown_connections 2024-09-15T21:19:39.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.532+0000 7f3ef66f9640 1 -- 172.21.15.119:0/195880930 wait complete. 2024-09-15T21:19:39.553 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create ec-foo 2 erasure ec-foo-ssd 2024-09-15T21:19:39.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient: get_monmap_and_config 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient: build_initial_monmap 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient: monmap: 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:39.670258+0000 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:39.670258+0000 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:39.672 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:39.673 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:39.673 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 Processor -- start 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 -- start start 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient: init 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f03d8135bf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03d809fe70 0x7f03d813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:39.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): picked mon.noname-c con 0x7f03d809fe70 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d813d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): picked mon.noname-b con 0x7f03d813afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d80a0300 con 0x7f03d8135bf0 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f03d80a3600 con 0x7f03d809fe70 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f03d80a1d80 con 0x7f03d813afb0 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:39.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03de4dc640 10 monclient(hunting): authenticate will time out at 1098.467168s 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03d7fff640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f03d8135bf0 legacy=0x7f03d8135fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03d77fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f03d809fe70 msgr2=0x7f03d813aa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03d77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03d809fe70 0x7f03d813aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d813d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d813d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55582/0 (socket says 172.21.15.119:55582) 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 1 -- 172.21.15.119:0/3540672010 learned_addr learned my addr 172.21.15.119:0/3540672010 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): get_auth_request con 0x7f03d813afb0 auth_method 0 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:39.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.668+0000 7f03dca79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 10 monclient(hunting): handle_auth_done global_id 24266 payload 274 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 1 -- 172.21.15.119:0/3540672010 >> v2:172.21.15.119:3300/0 conn(0x7f03d809fe70 msgr2=0x7f03d813aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 1 --2- 172.21.15.119:0/3540672010 >> v2:172.21.15.119:3300/0 conn(0x7f03d809fe70 0x7f03d813aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 1 -- 172.21.15.119:0/3540672010 >> v1:172.21.15.119:6789/0 conn(0x7f03d8135bf0 legacy=0x7f03d8135fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 10 monclient: _finish_hunting 0 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 1 monclient: found mon.noname-b 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:39.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 1 -- 172.21.15.119:0/3540672010 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d80a2060 con 0x7f03d813afb0 2024-09-15T21:19:39.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03dca79640 1 --2- 172.21.15.119:0/3540672010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d813d3a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009e80 tx=0x7f03d001c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27e2852271c61fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:39.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 1 -- 172.21.15.119:0/3540672010 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00ac020 con 0x7f03d813afb0 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: dump: 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:39.685 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 5 monclient: authenticate success, global_id 24266 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 1 -- 172.21.15.119:0/3540672010 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d00a6470 con 0x7f03d813afb0 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 4 set_mon_vals no callback set 2024-09-15T21:19:39.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 1 -- 172.21.15.119:0/3540672010 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00a6840 con 0x7f03d813afb0 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03d6ffd640 10 monclient: dump: 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:39.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 10 monclient: shutdown 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 -- 172.21.15.119:0/3540672010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 msgr2=0x7f03d813d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 --2- 172.21.15.119:0/3540672010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d813d3a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009e80 tx=0x7f03d001c710 comp rx=0 tx=0).stop 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 -- 172.21.15.119:0/3540672010 shutdown_connections 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 --2- 172.21.15.119:0/3540672010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d813d3a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 --2- 172.21.15.119:0/3540672010 >> v2:172.21.15.119:3300/0 conn(0x7f03d809fe70 0x7f03d813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 -- 172.21.15.119:0/3540672010 >> 172.21.15.119:0/3540672010 conn(0x7f03d80a66e0 msgr2=0x7f03d80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 -- 172.21.15.119:0/3540672010 shutdown_connections 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 1 -- 172.21.15.119:0/3540672010 wait complete. 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 10 monclient: build_initial_monmap 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.672+0000 7f03de4dc640 10 monclient: monmap: 2024-09-15T21:19:39.688 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 Processor -- start 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- start start 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: init 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:39.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 0x7f03d8072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): picked mon.noname-a con 0x7f03d809fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03d8135bf0 0x7f03d8134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): picked mon.noname-b con 0x7f03d8135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d81351e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): picked mon.noname-c con 0x7f03d813afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8140100 con 0x7f03d809fe70 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f03d806eda0 con 0x7f03d8135bf0 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f03d8140280 con 0x7f03d813afb0 2024-09-15T21:19:39.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient(hunting): authenticate will time out at 1098.472591s 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d77fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03d8135bf0 msgr2=0x7f03d8134ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03d8135bf0 0x7f03d8134ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d81351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 0x7f03d8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 0x7f03d8072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57758/0 (socket says 172.21.15.119:57758) 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d81351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:55584/0 (socket says 172.21.15.119:55584) 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 -- 172.21.15.119:0/1890998136 learned_addr learned my addr 172.21.15.119:0/1890998136 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): get_auth_request con 0x7f03d809fe70 auth_method 0 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): get_auth_request con 0x7f03d813afb0 auth_method 0 2024-09-15T21:19:39.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03dca79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:39.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient(hunting): handle_auth_done global_id 4887 payload 824 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 -- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 msgr2=0x7f03d81351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d81351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 -- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03d8135bf0 msgr2=0x7f03d8134ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03d8135bf0 0x7f03d8134ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 monclient: found mon.noname-a 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 -- 172.21.15.119:0/1890998136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d81cfcb0 con 0x7f03d809fe70 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d7fff640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 0x7f03d8072000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03c800b580 tx=0x7f03c800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8016070 con 0x7f03d809fe70 2024-09-15T21:19:39.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: dump: 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:39.694 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 5 monclient: authenticate success, global_id 4887 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _renew_subs 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d8121650 con 0x7f03d809fe70 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c8003df0 con 0x7f03d809fe70 2024-09-15T21:19:39.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8005310 con 0x7f03d809fe70 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 10 monclient: dump: 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:39.696 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _renew_subs 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d80a5340 con 0x7f03d809fe70 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d67fc640 4 set_mon_vals no callback set 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d809d240 con 0x7f03d809fe70 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f03c80054b0 con 0x7f03d809fe70 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f03c403b970 0x7f03c403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:39.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.676+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5131+0+0 (secure 0 0 0) 0x7f03c804f290 con 0x7f03d809fe70 2024-09-15T21:19:39.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.680+0000 7f03d77fe640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f03c403b970 0x7f03c403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:39.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.680+0000 7f03d77fe640 10 monclient: get_auth_request con 0x7f03c403b970 auth_method 0 2024-09-15T21:19:39.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.680+0000 7f03d77fe640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f03c403b970 0x7f03c403de30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03cc006fd0 tx=0x7f03cc006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:39.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.692+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f03d81cfcb0 con 0x7f03d809fe70 2024-09-15T21:19:39.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.692+0000 7f03d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:39.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.692+0000 7f03d4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:39.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.844+0000 7f03de4dc640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T21:19:39.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.844+0000 7f03de4dc640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T21:19:39.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.844+0000 7f03de4dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:39.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:39.844+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) -- 0x7f03d80a5de0 con 0x7f03d809fe70 2024-09-15T21:19:41.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.500+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' created v55) ==== 177+0+0 (secure 0 0 0) 0x7f03c804de20 con 0x7f03d809fe70 2024-09-15T21:19:41.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.500+0000 7f03d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T21:19:41.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.500+0000 7f03d4ff9640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' created 2024-09-15T21:19:41.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03de4dc640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T21:19:41.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03de4dc640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T21:19:41.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03de4dc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) -- 0x7f03d806a830 con 0x7f03d809fe70 2024-09-15T21:19:41.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03d4ff9640 1 -- 172.21.15.119:0/1890998136 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' already exists v55) ==== 184+0+0 (secure 0 0 0) 0x7f03c804f080 con 0x7f03d809fe70 2024-09-15T21:19:41.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03d4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T21:19:41.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.600+0000 7f03d4ff9640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' already exists 2024-09-15T21:19:41.605 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'ec-foo' already exists 2024-09-15T21:19:41.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f03c403b970 msgr2=0x7f03c403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:41.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f03c403b970 0x7f03c403de30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03cc006fd0 tx=0x7f03cc006d40 comp rx=0 tx=0).stop 2024-09-15T21:19:41.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 10 monclient: shutdown 2024-09-15T21:19:41.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:41.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 msgr2=0x7f03d8072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:41.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 0x7f03d8072000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03c800b580 tx=0x7f03c800ba40 comp rx=0 tx=0).stop 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 shutdown_connections 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f03c403b970 0x7f03c403de30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f03d813afb0 0x7f03d81351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03d8135bf0 0x7f03d8134ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 --2- 172.21.15.119:0/1890998136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03d809fe70 0x7f03d8072000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 >> 172.21.15.119:0/1890998136 conn(0x7f03d80a66e0 msgr2=0x7f03d80a3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 shutdown_connections 2024-09-15T21:19:41.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.604+0000 7f03de4dc640 1 -- 172.21.15.119:0/1890998136 wait complete. 2024-09-15T21:19:41.622 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool rm ec-foo ec-foo --yes-i-really-really-mean-it 2024-09-15T21:19:41.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient: get_monmap_and_config 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient: build_initial_monmap 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient: monmap: 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:41.741601+0000 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:41.741601+0000 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:41.747 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 Processor -- start 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 -- start start 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient: init 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1e6809d550 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809fae0 0x7f1e68139c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): picked mon.noname-b con 0x7f1e6809fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e6809e040 0x7f1e6813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): picked mon.noname-a con 0x7f1e6809e040 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e677fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f1e6809d550 legacy=0x7f1e6809d930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e680a3600 con 0x7f1e6809d550 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6809f1b0 con 0x7f1e6809e040 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1e680a2920 con 0x7f1e6809fae0 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e6db28640 10 monclient(hunting): authenticate will time out at 1100.538330s 2024-09-15T21:19:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e6809e040 0x7f1e6813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e6809e040 0x7f1e6813c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57766/0 (socket says 172.21.15.119:57766) 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 -- 172.21.15.119:0/3113629494 learned_addr learned my addr 172.21.15.119:0/3113629494 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): get_auth_request con 0x7f1e6809e040 auth_method 0 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 1 --2- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809fae0 0x7f1e68139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): get_auth_request con 0x7f1e6809fae0 auth_method 0 2024-09-15T21:19:41.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient(hunting): handle_auth_done global_id 4890 payload 274 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 -- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809fae0 msgr2=0x7f1e68139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 --2- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809fae0 0x7f1e68139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 -- 172.21.15.119:0/3113629494 >> v1:172.21.15.119:6789/0 conn(0x7f1e6809d550 legacy=0x7f1e6809d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 monclient: found mon.noname-a 2024-09-15T21:19:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.740+0000 7f1e67fff640 1 -- 172.21.15.119:0/3113629494 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680a1d60 con 0x7f1e6809e040 2024-09-15T21:19:41.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e67fff640 1 --2- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809e040 0x7f1e6813c1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e680a3550 tx=0x7f1e6001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903aaeb638600218 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:41.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 1 -- 172.21.15.119:0/3113629494 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600a5620 con 0x7f1e6809e040 2024-09-15T21:19:41.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: dump: 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:41.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 1 -- 172.21.15.119:0/3113629494 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e600a57c0 con 0x7f1e6809e040 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 1 -- 172.21.15.119:0/3113629494 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600a5b90 con 0x7f1e6809e040 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 5 monclient: authenticate success, global_id 4890 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 4 set_mon_vals no callback set 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e667fc640 10 monclient: dump: 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:41.760 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 10 monclient: shutdown 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 -- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809e040 msgr2=0x7f1e6813c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 --2- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809e040 0x7f1e6813c1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e680a3550 tx=0x7f1e6001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 -- 172.21.15.119:0/3113629494 shutdown_connections 2024-09-15T21:19:41.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 --2- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809e040 0x7f1e6813c1c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 --2- 172.21.15.119:0/3113629494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809fae0 0x7f1e68139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 -- 172.21.15.119:0/3113629494 >> 172.21.15.119:0/3113629494 conn(0x7f1e680a66e0 msgr2=0x7f1e680a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 -- 172.21.15.119:0/3113629494 shutdown_connections 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.744+0000 7f1e6db28640 1 -- 172.21.15.119:0/3113629494 wait complete. 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: build_initial_monmap 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: monmap: 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:41.762 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 Processor -- start 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 -- start start 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: init 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809d550 0x7f1e68133950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): picked mon.noname-c con 0x7f1e6809d550 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e6809e040 0x7f1e68133e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): picked mon.noname-b con 0x7f1e6809e040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 0x7f1e6811d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:41.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): picked mon.noname-a con 0x7f1e6809fae0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6813f0a0 con 0x7f1e6809fae0 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6813ef20 con 0x7f1e6809e040 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6813f220 con 0x7f1e6809d550 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient(hunting): authenticate will time out at 1100.545511s 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e66ffd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e6809e040 msgr2=0x7f1e68133e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e66ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e6809e040 0x7f1e68133e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:41.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 0x7f1e6811d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 0x7f1e6811d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:57770/0 (socket says 172.21.15.119:57770) 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 -- 172.21.15.119:0/1975992317 learned_addr learned my addr 172.21.15.119:0/1975992317 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809d550 0x7f1e68133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): get_auth_request con 0x7f1e6809fae0 auth_method 0 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): get_auth_request con 0x7f1e6809d550 auth_method 0 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:41.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient(hunting): handle_auth_done global_id 4893 payload 824 2024-09-15T21:19:41.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 -- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809d550 msgr2=0x7f1e68133950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809d550 0x7f1e68133950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 -- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e6809e040 msgr2=0x7f1e68133e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e6809e040 0x7f1e68133e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 monclient: found mon.noname-a 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 -- 172.21.15.119:0/1975992317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e68134810 con 0x7f1e6809fae0 2024-09-15T21:19:41.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e67fff640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 0x7f1e6811d4f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e600af040 tx=0x7f1e60019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600bc030 con 0x7f1e6809fae0 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: dump: 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.768 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:41.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: _finish_auth 0 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e600094d0 con 0x7f1e6809fae0 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600b8690 con 0x7f1e6809fae0 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 5 monclient: authenticate success, global_id 4893 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: _renew_subs 2024-09-15T21:19:41.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6809d240 con 0x7f1e6809fae0 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e65ffb640 4 set_mon_vals no callback set 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e4ffff640 10 monclient: dump: 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.771 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: _renew_subs 2024-09-15T21:19:41.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.748+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e681c4a10 con 0x7f1e6809fae0 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e6db28640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e6db28640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e6db28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6809d930 con 0x7f1e6809fae0 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1e600bf070 con 0x7f1e6809fae0 2024-09-15T21:19:41.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e4ffff640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1e5403b9d0 0x7f1e5403de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:41.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5572+0+0 (secure 0 0 0) 0x7f1e600ea6c0 con 0x7f1e6809fae0 2024-09-15T21:19:41.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e677fe640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1e5403b9d0 0x7f1e5403de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:41.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e677fe640 10 monclient: get_auth_request con 0x7f1e5403b9d0 auth_method 0 2024-09-15T21:19:41.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.752+0000 7f1e677fe640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1e5403b9d0 0x7f1e5403de90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e58003990 tx=0x7f1e58009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:41.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.764+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1e600165e0 con 0x7f1e6809fae0 2024-09-15T21:19:41.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.764+0000 7f1e4ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:41.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.764+0000 7f1e4ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:41.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.936+0000 7f1e6db28640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:41.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.936+0000 7f1e6db28640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:41.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.936+0000 7f1e6db28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:41.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:41.936+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e6806a830 con 0x7f1e6809fae0 2024-09-15T21:19:42.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.500+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' removed v56) ==== 154+0+0 (secure 0 0 0) 0x7f1e600adc70 con 0x7f1e6809fae0 2024-09-15T21:19:42.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.500+0000 7f1e4ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:42.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.500+0000 7f1e4ffff640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' removed 2024-09-15T21:19:42.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.604+0000 7f1e6db28640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:42.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.604+0000 7f1e6db28640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:42.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.604+0000 7f1e6db28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:42.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.604+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e68120d80 con 0x7f1e6809fae0 2024-09-15T21:19:42.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e4ffff640 1 -- 172.21.15.119:0/1975992317 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' does not exist v56) ==== 161+0+0 (secure 0 0 0) 0x7f1e600e7eb0 con 0x7f1e6809fae0 2024-09-15T21:19:42.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e4ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T21:19:42.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e4ffff640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' does not exist 2024-09-15T21:19:42.611 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'ec-foo' does not exist 2024-09-15T21:19:42.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1e5403b9d0 msgr2=0x7f1e5403de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:42.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1e5403b9d0 0x7f1e5403de90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e58003990 tx=0x7f1e58009110 comp rx=0 tx=0).stop 2024-09-15T21:19:42.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 10 monclient: shutdown 2024-09-15T21:19:42.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 msgr2=0x7f1e6811d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 0x7f1e6811d4f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e600af040 tx=0x7f1e60019670 comp rx=0 tx=0).stop 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 shutdown_connections 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1e5403b9d0 0x7f1e5403de90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e6809fae0 0x7f1e6811d4f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e6809e040 0x7f1e68133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 --2- 172.21.15.119:0/1975992317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1e6809d550 0x7f1e68133950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 >> 172.21.15.119:0/1975992317 conn(0x7f1e680a66e0 msgr2=0x7f1e680a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 shutdown_connections 2024-09-15T21:19:42.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.608+0000 7f1e6db28640 1 -- 172.21.15.119:0/1975992317 wait complete. 2024-09-15T21:19:42.629 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls 2024-09-15T21:19:42.629 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo 2024-09-15T21:19:42.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.740+0000 7f104b8e1640 10 monclient: get_monmap_and_config 2024-09-15T21:19:42.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.740+0000 7f104b8e1640 10 monclient: build_initial_monmap 2024-09-15T21:19:42.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.740+0000 7f104b8e1640 10 monclient: monmap: 2024-09-15T21:19:42.745 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:42.745 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:42.745 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:42.743705+0000 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:42.743705+0000 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:42.746 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:42.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 Processor -- start 2024-09-15T21:19:42.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 -- start start 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient: init 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f10440a4ed0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10440a59c0 0x7f104413ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): picked mon.noname-a con 0x7f10440a59c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): picked mon.noname-b con 0x7f10440a5dd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10440a6db0 con 0x7f10440a4ed0 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f10440a1210 con 0x7f10440a59c0 2024-09-15T21:19:42.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f10440a1b00 con 0x7f10440a5dd0 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104b8e1640 10 monclient(hunting): authenticate will time out at 1101.540867s 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f104967d640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f10440a4ed0 legacy=0x7f10440a52b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39472/0 (socket says 172.21.15.119:39472) 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 -- 172.21.15.119:0/366129738 learned_addr learned my addr 172.21.15.119:0/366129738 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): get_auth_request con 0x7f10440a5dd0 auth_method 0 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 1 --2- 172.21.15.119:0/366129738 >> v2:172.21.15.17:3300/0 conn(0x7f10440a59c0 0x7f104413ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:42.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): get_auth_request con 0x7f10440a59c0 auth_method 0 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient(hunting): handle_auth_done global_id 24278 payload 274 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 -- 172.21.15.119:0/366129738 >> v2:172.21.15.17:3300/0 conn(0x7f10440a59c0 msgr2=0x7f104413ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 --2- 172.21.15.119:0/366129738 >> v2:172.21.15.17:3300/0 conn(0x7f10440a59c0 0x7f104413ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 -- 172.21.15.119:0/366129738 >> v1:172.21.15.119:6789/0 conn(0x7f10440a4ed0 legacy=0x7f10440a52b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:42.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:42.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 monclient: found mon.noname-b 2024-09-15T21:19:42.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:42.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:42.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1049e7e640 1 -- 172.21.15.119:0/366129738 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10440a6ad0 con 0x7f10440a5dd0 2024-09-15T21:19:42.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.744+0000 7f1048e7c640 1 --2- 172.21.15.119:0/366129738 >> v2:172.21.15.17:3300/0 conn(0x7f10440a59c0 0x7f104413ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 --2- 172.21.15.119:0/366129738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413f420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f103c009e50 tx=0x7f103c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2176487b33488a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 1 -- 172.21.15.119:0/366129738 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0a55f0 con 0x7f10440a5dd0 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: dump: 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.757 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 1 -- 172.21.15.119:0/366129738 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103c0a5790 con 0x7f10440a5dd0 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 1 -- 172.21.15.119:0/366129738 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0a5b60 con 0x7f10440a5dd0 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 5 monclient: authenticate success, global_id 24278 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:42.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 4 set_mon_vals no callback set 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f103bfff640 10 monclient: dump: 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:42.759 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: shutdown 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- 172.21.15.119:0/366129738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 msgr2=0x7f104413f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 --2- 172.21.15.119:0/366129738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413f420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f103c009e50 tx=0x7f103c01c890 comp rx=0 tx=0).stop 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- 172.21.15.119:0/366129738 shutdown_connections 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 --2- 172.21.15.119:0/366129738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413f420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 --2- 172.21.15.119:0/366129738 >> v2:172.21.15.17:3300/0 conn(0x7f10440a59c0 0x7f104413ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- 172.21.15.119:0/366129738 >> 172.21.15.119:0/366129738 conn(0x7f10440a6fd0 msgr2=0x7f10440a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- 172.21.15.119:0/366129738 shutdown_connections 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- 172.21.15.119:0/366129738 wait complete. 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: build_initial_monmap 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: monmap: 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:42.760 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 Processor -- start 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- start start 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: init 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:42.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440a4ed0 0x7f1044127c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): picked mon.noname-b con 0x7f10440a4ed0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 0x7f1044125b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): picked mon.noname-a con 0x7f10440a59c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): picked mon.noname-c con 0x7f10440a5dd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1044142180 con 0x7f10440a59c0 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10440a7dd0 con 0x7f10440a4ed0 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1044142300 con 0x7f10440a5dd0 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:42.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104b8e1640 10 monclient(hunting): authenticate will time out at 1101.546554s 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104967d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440a4ed0 msgr2=0x7f1044127c50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f104967d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440a4ed0 0x7f1044127c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 0x7f1044125b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39480/0 (socket says 172.21.15.119:39480) 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 0x7f1044125b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34574/0 (socket says 172.21.15.119:34574) 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 -- 172.21.15.119:0/2111334717 learned_addr learned my addr 172.21.15.119:0/2111334717 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): get_auth_request con 0x7f10440a5dd0 auth_method 0 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): get_auth_request con 0x7f10440a59c0 auth_method 0 2024-09-15T21:19:42.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient(hunting): handle_auth_done global_id 24281 payload 824 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 -- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440a4ed0 msgr2=0x7f1044127c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440a4ed0 0x7f1044127c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 -- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 msgr2=0x7f1044125b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:42.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 0x7f1044125b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient: _finish_hunting 0 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 monclient: found mon.noname-c 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 -- 172.21.15.119:0/2111334717 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10441270f0 con 0x7f10440a5dd0 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1048e7c640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 0x7f1044125b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.748+0000 7f1049e7e640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413eac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f103c0af040 tx=0x7f103c019690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0bc020 con 0x7f10440a5dd0 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: dump: 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:42.765 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 5 monclient: authenticate success, global_id 24281 2024-09-15T21:19:42.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 monclient: _renew_subs 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10440a16c0 con 0x7f10440a5dd0 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103c0b8670 con 0x7f10440a5dd0 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c009510 con 0x7f10440a5dd0 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 10 monclient: dump: 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:42.767 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 monclient: _renew_subs 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1044128bb0 con 0x7f10440a5dd0 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f103affd640 4 set_mon_vals no callback set 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:42.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10440a52b0 con 0x7f10440a5dd0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f103c0bf020 con 0x7f10440a5dd0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f102803b9a0 0x7f102803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 5217+0+0 (secure 0 0 0) 0x7f103c0eb1f0 con 0x7f10440a5dd0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104967d640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f102803b9a0 0x7f102803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104967d640 10 monclient: get_auth_request con 0x7f102803b9a0 auth_method 0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.752+0000 7f104967d640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f102803b9a0 0x7f102803de60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.764+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f103c0c6070 con 0x7f10440a5dd0 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.764+0000 7f1039ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:42.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.764+0000 7f1039ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:42.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.936+0000 7f104b8e1640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:42.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.936+0000 7f104b8e1640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:42.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.936+0000 7f104b8e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:42.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.936+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f104406c480 con 0x7f10440a5dd0 2024-09-15T21:19:42.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.940+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v56) ==== 65+0+60 (secure 0 0 0) 0x7f103c016600 con 0x7f10440a5dd0 2024-09-15T21:19:42.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.940+0000 7f1039ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:42.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:42.940+0000 7f1039ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.036+0000 7f104b8e1640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.036+0000 7f104b8e1640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.036+0000 7f104b8e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.036+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f1044128f10 con 0x7f10440a5dd0 2024-09-15T21:19:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f1039ffb640 1 -- 172.21.15.119:0/2111334717 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v56) ==== 65+0+60 (secure 0 0 0) 0x7f103c0e8dc0 con 0x7f10440a5dd0 2024-09-15T21:19:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f1039ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f1039ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f102803b9a0 msgr2=0x7f102803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f102803b9a0 0x7f102803de60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040006d40 comp rx=0 tx=0).stop 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 10 monclient: shutdown 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 msgr2=0x7f104413eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413eac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f103c0af040 tx=0x7f103c019690 comp rx=0 tx=0).stop 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 shutdown_connections 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f102803b9a0 0x7f102803de60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f10440a5dd0 0x7f104413eac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10440a59c0 0x7f1044125b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 --2- 172.21.15.119:0/2111334717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440a4ed0 0x7f1044127c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 >> 172.21.15.119:0/2111334717 conn(0x7f10440a6fd0 msgr2=0x7f10440720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:43.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 shutdown_connections 2024-09-15T21:19:43.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.040+0000 7f104b8e1640 1 -- 172.21.15.119:0/2111334717 wait complete. 2024-09-15T21:19:43.058 INFO:tasks.workunit.client.0.smithi119.stdout:foo 2024-09-15T21:19:43.059 INFO:tasks.workunit.client.0.smithi119.stdout:foo-ssd 2024-09-15T21:19:43.059 INFO:tasks.workunit.client.0.smithi119.stdout:foo-hdd 2024-09-15T21:19:43.059 INFO:tasks.workunit.client.0.smithi119.stdout:ec-foo 2024-09-15T21:19:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient: get_monmap_and_config 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient: build_initial_monmap 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient: monmap: 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:43.177113+0000 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:43.177113+0000 2024-09-15T21:19:43.183 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 Processor -- start 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 -- start start 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient: init 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:43.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5e8136b90 0x7fe5e8136f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): picked mon.noname-c con 0x7fe5e8136b90 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e8137930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): picked mon.noname-b con 0x7fe5e81374b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe5e80a10d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e80a18f0 con 0x7fe5e80a10d0 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe5e809fc40 con 0x7fe5e8136b90 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e80a1bd0 con 0x7fe5e81374b0 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5eda51640 10 monclient(hunting): authenticate will time out at 1101.973767s 2024-09-15T21:19:43.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e77fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fe5e80a10d0 legacy=0x7fe5e8137e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e6ffd640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fe5e8136b90 msgr2=0x7fe5e8136f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5e8136b90 0x7fe5e8136f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e8137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e8137930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39486/0 (socket says 172.21.15.119:39486) 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 -- 172.21.15.119:0/1022110259 learned_addr learned my addr 172.21.15.119:0/1022110259 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): get_auth_request con 0x7fe5e81374b0 auth_method 0 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_done global_id 24284 payload 274 2024-09-15T21:19:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 -- 172.21.15.119:0/1022110259 >> v2:172.21.15.119:3300/0 conn(0x7fe5e8136b90 msgr2=0x7fe5e8136f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 --2- 172.21.15.119:0/1022110259 >> v2:172.21.15.119:3300/0 conn(0x7fe5e8136b90 0x7fe5e8136f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 -- 172.21.15.119:0/1022110259 >> v1:172.21.15.119:6789/0 conn(0x7fe5e80a10d0 legacy=0x7fe5e8137e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 monclient: found mon.noname-b 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.176+0000 7fe5e67fc640 1 -- 172.21.15.119:0/1022110259 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e80a3660 con 0x7fe5e81374b0 2024-09-15T21:19:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 1 --2- 172.21.15.119:0/1022110259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e8137930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5e809d010 tx=0x7fe5d801c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932dcbf17a9bead2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:43.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 1 -- 172.21.15.119:0/1022110259 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80a5570 con 0x7fe5e81374b0 2024-09-15T21:19:43.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:43.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: dump: 2024-09-15T21:19:43.192 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 1 -- 172.21.15.119:0/1022110259 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e8138790 con 0x7fe5e81374b0 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 5 monclient: authenticate success, global_id 24284 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 1 -- 172.21.15.119:0/1022110259 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80a5930 con 0x7fe5e81374b0 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e5ffb640 10 monclient: dump: 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.194 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 4 set_mon_vals no callback set 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: shutdown 2024-09-15T21:19:43.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- 172.21.15.119:0/1022110259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 msgr2=0x7fe5e8137930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 --2- 172.21.15.119:0/1022110259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e8137930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5e809d010 tx=0x7fe5d801c710 comp rx=0 tx=0).stop 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- 172.21.15.119:0/1022110259 shutdown_connections 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 --2- 172.21.15.119:0/1022110259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e8137930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 --2- 172.21.15.119:0/1022110259 >> v2:172.21.15.119:3300/0 conn(0x7fe5e8136b90 0x7fe5e8136f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- 172.21.15.119:0/1022110259 >> 172.21.15.119:0/1022110259 conn(0x7fe5e80a66e0 msgr2=0x7fe5e80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- 172.21.15.119:0/1022110259 shutdown_connections 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- 172.21.15.119:0/1022110259 wait complete. 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: build_initial_monmap 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: monmap: 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:43.196 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 Processor -- start 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- start start 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: init 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:43.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5e80a10d0 0x7fe5e81cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): picked mon.noname-b con 0x7fe5e80a10d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 0x7fe5e8132990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): picked mon.noname-a con 0x7fe5e8136b90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e813c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): picked mon.noname-c con 0x7fe5e81374b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8142740 con 0x7fe5e8136b90 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81425c0 con 0x7fe5e80a10d0 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e81428c0 con 0x7fe5e81374b0 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5eda51640 10 monclient(hunting): authenticate will time out at 1101.979533s 2024-09-15T21:19:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e6ffd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5e80a10d0 msgr2=0x7fe5e81cfcd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5e80a10d0 0x7fe5e81cfcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 0x7fe5e8132990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e813c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e813c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39488/0 (socket says 172.21.15.119:39488) 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 1 -- 172.21.15.119:0/31118414 learned_addr learned my addr 172.21.15.119:0/31118414 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 0x7fe5e8132990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34580/0 (socket says 172.21.15.119:34580) 2024-09-15T21:19:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 10 monclient(hunting): get_auth_request con 0x7fe5e81374b0 auth_method 0 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 10 monclient(hunting): get_auth_request con 0x7fe5e8136b90 auth_method 0 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.180+0000 7fe5e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 10 monclient(hunting): handle_auth_done global_id 24287 payload 824 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 -- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5e80a10d0 msgr2=0x7fe5e81cfcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5e80a10d0 0x7fe5e81cfcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 -- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 msgr2=0x7fe5e8132990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 0x7fe5e8132990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 -- 172.21.15.119:0/31118414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e80a4ea0 con 0x7fe5e81374b0 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e67fc640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 0x7fe5e8132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e77fe640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e813c880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b3c0 tx=0x7fe5e000b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0017070 con 0x7fe5e81374b0 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: dump: 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: _finish_auth 0 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e0003dc0 con 0x7fe5e81374b0 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 5 monclient: authenticate success, global_id 24287 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 monclient: _renew_subs 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e809d240 con 0x7fe5e81374b0 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0006310 con 0x7fe5e81374b0 2024-09-15T21:19:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 10 monclient: dump: 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 monclient: _renew_subs 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e57fa640 4 set_mon_vals no callback set 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e813ee00 con 0x7fe5e81374b0 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e8136f70 con 0x7fe5e81374b0 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe5e001a070 con 0x7fe5e81374b0 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe5d403b970 0x7fe5d403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 5217+0+0 (secure 0 0 0) 0x7fe5e004f190 con 0x7fe5e81374b0 2024-09-15T21:19:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e6ffd640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe5d403b970 0x7fe5d403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e6ffd640 10 monclient: get_auth_request con 0x7fe5d403b970 auth_method 0 2024-09-15T21:19:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.184+0000 7fe5e6ffd640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe5d403b970 0x7fe5d403de30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.196+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe5e001ada0 con 0x7fe5e81374b0 2024-09-15T21:19:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.196+0000 7fe5cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.196+0000 7fe5cffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:43.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.376+0000 7fe5eda51640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.376+0000 7fe5eda51640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.376+0000 7fe5eda51640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.376+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7fe5e81d0b50 con 0x7fe5e81374b0 2024-09-15T21:19:43.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.520+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 v57) ==== 110+0+0 (secure 0 0 0) 0x7fe5e004cae0 con 0x7fe5e81374b0 2024-09-15T21:19:43.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.520+0000 7fe5cffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.520+0000 7fe5cffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:43.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.628+0000 7fe5eda51640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.628+0000 7fe5eda51640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.628+0000 7fe5eda51640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:43.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.628+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7fe5e811c6a0 con 0x7fe5e81374b0 2024-09-15T21:19:43.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.632+0000 7fe5cffff640 1 -- 172.21.15.119:0/31118414 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v57) ==== 139+0+0 (secure 0 0 0) 0x7fe5e001b8d0 con 0x7fe5e81374b0 2024-09-15T21:19:43.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.632+0000 7fe5cffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.632+0000 7fe5cffff640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-09-15T21:19:43.636 INFO:tasks.workunit.client.0.smithi119.stderr:already renamed to 'foo-asdf' 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe5d403b970 msgr2=0x7fe5d403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe5d403b970 0x7fe5d403de30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006d40 comp rx=0 tx=0).stop 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 10 monclient: shutdown 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 msgr2=0x7fe5e813c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e813c880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b3c0 tx=0x7fe5e000b880 comp rx=0 tx=0).stop 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 shutdown_connections 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe5d403b970 0x7fe5d403de30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe5e81374b0 0x7fe5e813c880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e8136b90 0x7fe5e8132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 --2- 172.21.15.119:0/31118414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5e80a10d0 0x7fe5e81cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 >> 172.21.15.119:0/31118414 conn(0x7fe5e80a66e0 msgr2=0x7fe5e809e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:43.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 shutdown_connections 2024-09-15T21:19:43.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.636+0000 7fe5eda51640 1 -- 172.21.15.119:0/31118414 wait complete. 2024-09-15T21:19:43.654 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-09-15T21:19:43.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient: get_monmap_and_config 2024-09-15T21:19:43.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient: build_initial_monmap 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient: monmap: 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:43.774202+0000 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:43.774202+0000 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:43.776 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:43.777 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 1 Processor -- start 2024-09-15T21:19:43.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 1 -- start start 2024-09-15T21:19:43.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient: init 2024-09-15T21:19:43.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:43.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:19:43.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd12406da80 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd124073660 0x7fd124139b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient(hunting): picked mon.noname-a con 0x7fd124073660 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd124073ad0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1240a0f40 con 0x7fd12406da80 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd124071890 con 0x7fd124073ad0 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd1240721c0 con 0x7fd124073660 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12c766640 10 monclient(hunting): authenticate will time out at 1102.571684s 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.772+0000 7fd12ad03640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fd124073ad0 legacy=0x7fd12413c0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd124073660 0x7fd124139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd124073660 0x7fd124139b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34592/0 (socket says 172.21.15.119:34592) 2024-09-15T21:19:43.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 -- 172.21.15.119:0/2810230430 learned_addr learned my addr 172.21.15.119:0/2810230430 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): get_auth_request con 0x7fd124073660 auth_method 0 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd12a502640 1 --1- 172.21.15.119:0/2810230430 >> v1:172.21.15.17:6789/0 conn(0x7fd12406da80 0x7fd124131790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:35870/0 (socket says 172.21.15.119:35870) 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:43.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:43.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient(hunting): handle_auth_done global_id 4905 payload 274 2024-09-15T21:19:43.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 -- 172.21.15.119:0/2810230430 >> v1:172.21.15.119:6789/0 conn(0x7fd124073ad0 legacy=0x7fd12413c0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:43.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 -- 172.21.15.119:0/2810230430 >> v1:172.21.15.17:6789/0 conn(0x7fd12406da80 legacy=0x7fd124131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 1 -- 172.21.15.119:0/2810230430 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677607869 0 0) 0x7fd1240a0f40 con 0x7fd12406da80 2024-09-15T21:19:43.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient: _finish_hunting 0 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 monclient: found mon.noname-a 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 -- 172.21.15.119:0/2810230430 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd124071b70 con 0x7fd124073660 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129d01640 1 --2- 172.21.15.119:0/2810230430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073660 0x7fd124139b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd114009870 tx=0x7fd11401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de84108ee6a96b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 1 -- 172.21.15.119:0/2810230430 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b0070 con 0x7fd124073660 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: dump: 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.783 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: _finish_auth 0 2024-09-15T21:19:43.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: _check_auth_tickets 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd12c766640 5 monclient: authenticate success, global_id 4905 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd12c766640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 1 -- 172.21.15.119:0/2810230430 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1140ab600 con 0x7fd124073660 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd12c766640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd12c766640 4 set_mon_vals no callback set 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 1 -- 172.21.15.119:0/2810230430 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140ab9d0 con 0x7fd124073660 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.776+0000 7fd129500640 10 monclient: dump: 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.785 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: shutdown 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/2810230430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073660 msgr2=0x7fd124139b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 --2- 172.21.15.119:0/2810230430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073660 0x7fd124139b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd114009870 tx=0x7fd11401c420 comp rx=0 tx=0).stop 2024-09-15T21:19:43.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/2810230430 shutdown_connections 2024-09-15T21:19:43.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 --2- 172.21.15.119:0/2810230430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073660 0x7fd124139b70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/2810230430 >> 172.21.15.119:0/2810230430 conn(0x7fd12406cbf0 msgr2=0x7fd12406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:43.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/2810230430 shutdown_connections 2024-09-15T21:19:43.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/2810230430 wait complete. 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: build_initial_monmap 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: monmap: 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.791 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 Processor -- start 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- start start 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: init 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:43.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd12406da80 0x7fd124130230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): picked mon.noname-c con 0x7fd12406da80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd124073660 0x7fd124130770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): picked mon.noname-b con 0x7fd124073660 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073ad0 0x7fd12413b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): picked mon.noname-a con 0x7fd124073ad0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd1241405f0 con 0x7fd124073ad0 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd124140470 con 0x7fd124073660 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd124140770 con 0x7fd12406da80 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient(hunting): authenticate will time out at 1102.577807s 2024-09-15T21:19:43.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd129d01640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd124073660 msgr2=0x7fd124130770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:43.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd129d01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd124073660 0x7fd124130770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:43.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd12406da80 0x7fd124130230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd12406da80 0x7fd124130230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39494/0 (socket says 172.21.15.119:39494) 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073ad0 0x7fd12413b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 1 -- 172.21.15.119:0/457842472 learned_addr learned my addr 172.21.15.119:0/457842472 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): get_auth_request con 0x7fd12406da80 auth_method 0 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): get_auth_request con 0x7fd124073ad0 auth_method 0 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:43.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12a502640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient(hunting): handle_auth_done global_id 4911 payload 824 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 -- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd12406da80 msgr2=0x7fd124130230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd12406da80 0x7fd124130230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 -- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd124073660 msgr2=0x7fd124130770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd124073660 0x7fd124130770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:43.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient: _finish_hunting 0 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 monclient: found mon.noname-a 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 -- 172.21.15.119:0/457842472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1241c5cd0 con 0x7fd124073ad0 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12ad03640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073ad0 0x7fd12413b510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd11c003d10 tx=0x7fd11c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00cdf0 con 0x7fd124073ad0 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: dump: 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.798 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: _finish_auth 0 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 5 monclient: authenticate success, global_id 4911 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: _renew_subs 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c010030 con 0x7fd124073ad0 2024-09-15T21:19:43.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12409d4b0 con 0x7fd124073ad0 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c016670 con 0x7fd124073ad0 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd11affd640 10 monclient: dump: 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:43.800 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: _renew_subs 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd128cff640 4 set_mon_vals no callback set 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.780+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1240a5bb0 con 0x7fd124073ad0 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12c766640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12c766640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12c766640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12413cde0 con 0x7fd124073ad0 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd11c0153e0 con 0x7fd124073ad0 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd11affd640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd10803b9a0 0x7fd10803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5222+0+0 (secure 0 0 0) 0x7fd11c0506d0 con 0x7fd124073ad0 2024-09-15T21:19:43.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12a502640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd10803b9a0 0x7fd10803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:43.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12a502640 10 monclient: get_auth_request con 0x7fd10803b9a0 auth_method 0 2024-09-15T21:19:43.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.784+0000 7fd12a502640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd10803b9a0 0x7fd10803de60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd120000f70 tx=0x7fd120007db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:43.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.796+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd1241c5cd0 con 0x7fd124073ad0 2024-09-15T21:19:43.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.796+0000 7fd11affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:43.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.796+0000 7fd11affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:43.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.952+0000 7fd12c766640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.952+0000 7fd12c766640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.952+0000 7fd12c766640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:43.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.952+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7fd124131770 con 0x7fd124073ad0 2024-09-15T21:19:43.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.956+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v57) ==== 139+0+0 (secure 0 0 0) 0x7fd11c013cf0 con 0x7fd124073ad0 2024-09-15T21:19:43.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.956+0000 7fd11affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:43.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:43.956+0000 7fd11affd640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo-asdf' 2024-09-15T21:19:44.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:44.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:44.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7fd124074bd0 con 0x7fd124073ad0 2024-09-15T21:19:44.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd11affd640 1 -- 172.21.15.119:0/457842472 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v57) ==== 139+0+0 (secure 0 0 0) 0x7fd11c04df20 con 0x7fd124073ad0 2024-09-15T21:19:44.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd11affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T21:19:44.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd11affd640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-09-15T21:19:44.063 INFO:tasks.workunit.client.0.smithi119.stderr:already renamed to 'foo-asdf' 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd10803b9a0 msgr2=0x7fd10803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd10803b9a0 0x7fd10803de60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd120000f70 tx=0x7fd120007db0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 10 monclient: shutdown 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073ad0 msgr2=0x7fd12413b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073ad0 0x7fd12413b510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd11c003d10 tx=0x7fd11c009ed0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 shutdown_connections 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd10803b9a0 0x7fd10803de60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd124073ad0 0x7fd12413b510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd124073660 0x7fd124130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 --2- 172.21.15.119:0/457842472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd12406da80 0x7fd124130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 >> 172.21.15.119:0/457842472 conn(0x7fd12406cbf0 msgr2=0x7fd12406f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:44.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 shutdown_connections 2024-09-15T21:19:44.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.060+0000 7fd12c766640 1 -- 172.21.15.119:0/457842472 wait complete. 2024-09-15T21:19:44.081 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rename bar bar-asdf 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient: get_monmap_and_config 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient: build_initial_monmap 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient: monmap: 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:44.201585+0000 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:44.201585+0000 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:44.204 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 Processor -- start 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 -- start start 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient: init 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1af4135bf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 0x7f1af413aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): picked mon.noname-b con 0x7f1af409fe70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1af413afb0 0x7f1af413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): picked mon.noname-c con 0x7f1af413afb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af40a0300 con 0x7f1af4135bf0 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1af40a3600 con 0x7f1af413afb0 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1af40a1d80 con 0x7f1af409fe70 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1afb9f6640 10 monclient(hunting): authenticate will time out at 1102.998233s 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af9792640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f1af4135bf0 legacy=0x7f1af4135fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af9f93640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f1af413afb0 msgr2=0x7f1af413d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af9f93640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1af413afb0 0x7f1af413d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 0x7f1af413aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 0x7f1af413aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39496/0 (socket says 172.21.15.119:39496) 2024-09-15T21:19:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 -- 172.21.15.119:0/373241410 learned_addr learned my addr 172.21.15.119:0/373241410 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): get_auth_request con 0x7f1af409fe70 auth_method 0 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient(hunting): handle_auth_done global_id 24293 payload 274 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 -- 172.21.15.119:0/373241410 >> v2:172.21.15.119:3300/0 conn(0x7f1af413afb0 msgr2=0x7f1af413d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 --2- 172.21.15.119:0/373241410 >> v2:172.21.15.119:3300/0 conn(0x7f1af413afb0 0x7f1af413d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 -- 172.21.15.119:0/373241410 >> v1:172.21.15.119:6789/0 conn(0x7f1af4135bf0 legacy=0x7f1af4135fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient: _finish_hunting 0 2024-09-15T21:19:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 monclient: found mon.noname-b 2024-09-15T21:19:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.200+0000 7f1af8f91640 1 -- 172.21.15.119:0/373241410 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af40a2060 con 0x7f1af409fe70 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1af8f91640 1 --2- 172.21.15.119:0/373241410 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 0x7f1af413aa70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009870 tx=0x7f1af001c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a876052d1809e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 1 -- 172.21.15.119:0/373241410 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00b0070 con 0x7f1af409fe70 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: dump: 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: _finish_auth 0 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 1 -- 172.21.15.119:0/373241410 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af00ab600 con 0x7f1af409fe70 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 1 -- 172.21.15.119:0/373241410 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00ab9d0 con 0x7f1af409fe70 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1aebfff640 10 monclient: dump: 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 5 monclient: authenticate success, global_id 24293 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 4 set_mon_vals no callback set 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 10 monclient: shutdown 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 -- 172.21.15.119:0/373241410 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 msgr2=0x7f1af413aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/373241410 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 0x7f1af413aa70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009870 tx=0x7f1af001c260 comp rx=0 tx=0).stop 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 -- 172.21.15.119:0/373241410 shutdown_connections 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/373241410 >> v2:172.21.15.119:3300/0 conn(0x7f1af413afb0 0x7f1af413d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/373241410 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af409fe70 0x7f1af413aa70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 -- 172.21.15.119:0/373241410 >> 172.21.15.119:0/373241410 conn(0x7f1af40a66e0 msgr2=0x7f1af40a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:44.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 -- 172.21.15.119:0/373241410 shutdown_connections 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 -- 172.21.15.119:0/373241410 wait complete. 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 10 monclient: build_initial_monmap 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 10 monclient: monmap: 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:44.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.204+0000 7f1afb9f6640 1 Processor -- start 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- start start 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: init 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af409fe70 0x7f1af4139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): picked mon.noname-b con 0x7f1af409fe70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 0x7f1af4134c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): picked mon.noname-a con 0x7f1af4135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af413afb0 0x7f1af411c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): picked mon.noname-c con 0x7f1af413afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4140100 con 0x7f1af4135bf0 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1af406ed20 con 0x7f1af409fe70 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1af4140280 con 0x7f1af413afb0 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient(hunting): authenticate will time out at 1103.004615s 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9792640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af409fe70 msgr2=0x7f1af4139ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9792640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af409fe70 0x7f1af4139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af413afb0 0x7f1af411c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 0x7f1af4134c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af413afb0 0x7f1af411c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39506/0 (socket says 172.21.15.119:39506) 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 0x7f1af4134c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34612/0 (socket says 172.21.15.119:34612) 2024-09-15T21:19:44.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 1 -- 172.21.15.119:0/175131354 learned_addr learned my addr 172.21.15.119:0/175131354 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): get_auth_request con 0x7f1af413afb0 auth_method 0 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): get_auth_request con 0x7f1af4135bf0 auth_method 0 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af9f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient(hunting): handle_auth_done global_id 4914 payload 824 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 -- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af413afb0 msgr2=0x7f1af411c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af413afb0 0x7f1af411c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 -- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af409fe70 msgr2=0x7f1af4139ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af409fe70 0x7f1af4139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient: _finish_hunting 0 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 monclient: found mon.noname-a 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 -- 172.21.15.119:0/175131354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af41ce670 con 0x7f1af4135bf0 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1af8f91640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 0x7f1af4134c00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1af00ab000 tx=0x7f1af001aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00ba020 con 0x7f1af4135bf0 2024-09-15T21:19:44.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: dump: 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af00b6620 con 0x7f1af4135bf0 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00b0040 con 0x7f1af4135bf0 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 5 monclient: authenticate success, global_id 4914 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _renew_subs 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af409d240 con 0x7f1af4135bf0 2024-09-15T21:19:44.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 10 monclient: dump: 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1aeb7fe640 4 set_mon_vals no callback set 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _renew_subs 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af411e2a0 con 0x7f1af4135bf0 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af4135fd0 con 0x7f1af4135bf0 2024-09-15T21:19:44.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.208+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1af00bd020 con 0x7f1af4135bf0 2024-09-15T21:19:44.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.212+0000 7f1ae9ffb640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ae003b970 0x7f1ae003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.212+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5222+0+0 (secure 0 0 0) 0x7f1af00ea3f0 con 0x7f1af4135bf0 2024-09-15T21:19:44.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.212+0000 7f1af9792640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ae003b970 0x7f1ae003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.212+0000 7f1af9792640 10 monclient: get_auth_request con 0x7f1ae003b970 auth_method 0 2024-09-15T21:19:44.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.212+0000 7f1af9792640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ae003b970 0x7f1ae003de30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1aec003d10 tx=0x7f1aec009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:44.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.224+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1af00c4030 con 0x7f1af4135bf0 2024-09-15T21:19:44.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.224+0000 7f1ae9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:44.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.224+0000 7f1ae9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:44.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.384+0000 7f1afb9f6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T21:19:44.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.384+0000 7f1afb9f6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T21:19:44.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.384+0000 7f1afb9f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.384+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) -- 0x7f1af40a50b0 con 0x7f1af4135bf0 2024-09-15T21:19:44.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.520+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 v58) ==== 110+0+0 (secure 0 0 0) 0x7f1af40a0300 con 0x7f1af4135bf0 2024-09-15T21:19:44.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.520+0000 7f1ae9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T21:19:44.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.520+0000 7f1ae9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:44.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T21:19:44.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T21:19:44.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) -- 0x7f1af40008e0 con 0x7f1af4135bf0 2024-09-15T21:19:44.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1ae9ffb640 1 -- 172.21.15.119:0/175131354 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 already renamed to 'bar-asdf' v58) ==== 139+0+0 (secure 0 0 0) 0x7f1af00df070 con 0x7f1af4135bf0 2024-09-15T21:19:44.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1ae9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T21:19:44.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1ae9ffb640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar-asdf' 2024-09-15T21:19:44.624 INFO:tasks.workunit.client.0.smithi119.stderr:already renamed to 'bar-asdf' 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ae003b970 msgr2=0x7f1ae003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ae003b970 0x7f1ae003de30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1aec003d10 tx=0x7f1aec009270 comp rx=0 tx=0).stop 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 10 monclient: shutdown 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.620+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 msgr2=0x7f1af4134c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 0x7f1af4134c00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1af00ab000 tx=0x7f1af001aaa0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 shutdown_connections 2024-09-15T21:19:44.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1ae003b970 0x7f1ae003de30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1af413afb0 0x7f1af411c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1af4135bf0 0x7f1af4134c00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 --2- 172.21.15.119:0/175131354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af409fe70 0x7f1af4139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 >> 172.21.15.119:0/175131354 conn(0x7f1af40a66e0 msgr2=0x7f1af413bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:44.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 shutdown_connections 2024-09-15T21:19:44.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.624+0000 7f1afb9f6640 1 -- 172.21.15.119:0/175131354 wait complete. 2024-09-15T21:19:44.640 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls 2024-09-15T21:19:44.640 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo-asdf 2024-09-15T21:19:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.752+0000 7f4248948640 10 monclient: get_monmap_and_config 2024-09-15T21:19:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.752+0000 7f4248948640 10 monclient: build_initial_monmap 2024-09-15T21:19:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.752+0000 7f4248948640 10 monclient: monmap: 2024-09-15T21:19:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:44.755850+0000 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:44.755850+0000 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:44.758 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 Processor -- start 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 -- start start 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient: init 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 0x7f4244136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): picked mon.noname-a con 0x7f4244135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f42440a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): picked mon.noname-b con 0x7f42440a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f424406ec30 0x7f42440a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): picked mon.noname-c con 0x7f424406ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:44.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f42441365c0 con 0x7f4244135ca0 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f424409cf30 con 0x7f424406ec30 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f42440a1ca0 con 0x7f42440a0160 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4248948640 10 monclient(hunting): authenticate will time out at 1103.553029s 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4242d9d640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f424406ec30 msgr2=0x7f42440a0f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4242d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f424406ec30 0x7f42440a0f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f42440a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 0x7f4244136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f42440a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39516/0 (socket says 172.21.15.119:39516) 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 0x7f4244136080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34616/0 (socket says 172.21.15.119:34616) 2024-09-15T21:19:44.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 -- 172.21.15.119:0/148524968 learned_addr learned my addr 172.21.15.119:0/148524968 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): get_auth_request con 0x7f42440a0160 auth_method 0 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): get_auth_request con 0x7f4244135ca0 auth_method 0 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient(hunting): handle_auth_done global_id 24299 payload 274 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 -- 172.21.15.119:0/148524968 >> v2:172.21.15.119:3300/0 conn(0x7f424406ec30 msgr2=0x7f42440a0f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 --2- 172.21.15.119:0/148524968 >> v2:172.21.15.119:3300/0 conn(0x7f424406ec30 0x7f42440a0f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 -- 172.21.15.119:0/148524968 >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 msgr2=0x7f4244136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 --2- 172.21.15.119:0/148524968 >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 0x7f4244136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 monclient: found mon.noname-b 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f4241d9b640 1 -- 172.21.15.119:0/148524968 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440a1f80 con 0x7f42440a0160 2024-09-15T21:19:44.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.756+0000 7f424259c640 1 --2- 172.21.15.119:0/148524968 >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 0x7f4244136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 1 --2- 172.21.15.119:0/148524968 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f42440a09e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42440a29c0 tx=0x7f423801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fb7bed891d087d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 1 -- 172.21.15.119:0/148524968 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380b0070 con 0x7f42440a0160 2024-09-15T21:19:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:44.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: dump: 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: _finish_auth 0 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:44.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 1 -- 172.21.15.119:0/148524968 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42380a4e50 con 0x7f42440a0160 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 1 -- 172.21.15.119:0/148524968 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380ab820 con 0x7f42440a0160 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 5 monclient: authenticate success, global_id 24299 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 4 set_mon_vals no callback set 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424159a640 10 monclient: dump: 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.771 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: shutdown 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- 172.21.15.119:0/148524968 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 msgr2=0x7f42440a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- 172.21.15.119:0/148524968 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f42440a09e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42440a29c0 tx=0x7f423801c400 comp rx=0 tx=0).stop 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- 172.21.15.119:0/148524968 shutdown_connections 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- 172.21.15.119:0/148524968 >> v2:172.21.15.119:3300/0 conn(0x7f424406ec30 0x7f42440a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- 172.21.15.119:0/148524968 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f42440a09e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- 172.21.15.119:0/148524968 >> v2:172.21.15.17:3300/0 conn(0x7f4244135ca0 0x7f4244136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- 172.21.15.119:0/148524968 >> 172.21.15.119:0/148524968 conn(0x7f42440a2ae0 msgr2=0x7f42440a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- 172.21.15.119:0/148524968 shutdown_connections 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- 172.21.15.119:0/148524968 wait complete. 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: build_initial_monmap 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: monmap: 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.773 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 Processor -- start 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- start start 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: init 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 0x7f4244136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): picked mon.noname-a con 0x7f424406ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f4244136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): picked mon.noname-c con 0x7f42440a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244135ca0 0x7f42441374e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): picked mon.noname-b con 0x7f4244135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f424413f230 con 0x7f424406ec30 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f42440a1870 con 0x7f4244135ca0 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f424413f3b0 con 0x7f42440a0160 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4248948640 10 monclient(hunting): authenticate will time out at 1103.558702s 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4242d9d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244135ca0 msgr2=0x7f42441374e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4242d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244135ca0 0x7f42441374e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 0x7f4244136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 0x7f4244136a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34622/0 (socket says 172.21.15.119:34622) 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 -- 172.21.15.119:0/1766426349 learned_addr learned my addr 172.21.15.119:0/1766426349 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): get_auth_request con 0x7f424406ec30 auth_method 0 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f4244136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): get_auth_request con 0x7f42440a0160 auth_method 0 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f4241d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient(hunting): handle_auth_done global_id 4920 payload 824 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 -- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 msgr2=0x7f4244136fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f4244136fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 -- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244135ca0 msgr2=0x7f42441374e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244135ca0 0x7f42441374e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 monclient: found mon.noname-a 2024-09-15T21:19:44.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.760+0000 7f424259c640 1 -- 172.21.15.119:0/1766426349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424413a260 con 0x7f424406ec30 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f424259c640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 0x7f4244136a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f423400ebe0 tx=0x7f423400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234017070 con 0x7f424406ec30 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: dump: 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.778 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423400cc30 con 0x7f424406ec30 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 5 monclient: authenticate success, global_id 4920 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 monclient: _renew_subs 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424413a540 con 0x7f424406ec30 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234013470 con 0x7f424406ec30 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 10 monclient: dump: 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:44.780 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 monclient: _renew_subs 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424413a820 con 0x7f424406ec30 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4240d99640 4 set_mon_vals no callback set 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424413c6b0 con 0x7f424406ec30 2024-09-15T21:19:44.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f423401a070 con 0x7f424406ec30 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7f4234050550 con 0x7f424406ec30 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4241d9b640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4241d9b640 10 monclient: get_auth_request con 0x7f422403b970 auth_method 0 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.764+0000 7f4241d9b640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42380ad000 tx=0x7f4238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.776+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f424413c6b0 con 0x7f424406ec30 2024-09-15T21:19:44.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.776+0000 7f4232ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:44.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.776+0000 7f4232ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:44.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4248948640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:44.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4248948640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:44.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4248948640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:44.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f4244136080 con 0x7f424406ec30 2024-09-15T21:19:44.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v58) ==== 65+0+70 (secure 0 0 0) 0x7f423404dda0 con 0x7f424406ec30 2024-09-15T21:19:44.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4232ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:44.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:44.948+0000 7f4232ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:45.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4248948640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4248948640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4248948640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f424409f8f0 con 0x7f424406ec30 2024-09-15T21:19:45.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4232ffd640 1 -- 172.21.15.119:0/1766426349 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v58) ==== 65+0+70 (secure 0 0 0) 0x7f4234013ae0 con 0x7f424406ec30 2024-09-15T21:19:45.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4232ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.044+0000 7f4232ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 msgr2=0x7f422403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42380ad000 tx=0x7f4238008040 comp rx=0 tx=0).stop 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 10 monclient: shutdown 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 msgr2=0x7f4244136a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 0x7f4244136a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f423400ebe0 tx=0x7f423400c520 comp rx=0 tx=0).stop 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 shutdown_connections 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244135ca0 0x7f42441374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42440a0160 0x7f4244136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 --2- 172.21.15.119:0/1766426349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f424406ec30 0x7f4244136a60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 >> 172.21.15.119:0/1766426349 conn(0x7f42440a2ae0 msgr2=0x7f42440a4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:45.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 shutdown_connections 2024-09-15T21:19:45.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.048+0000 7f4248948640 1 -- 172.21.15.119:0/1766426349 wait complete. 2024-09-15T21:19:45.064 INFO:tasks.workunit.client.0.smithi119.stdout:foo-asdf 2024-09-15T21:19:45.065 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep bar-asdf 2024-09-15T21:19:45.065 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls 2024-09-15T21:19:45.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.176+0000 7fb3c2240640 10 monclient: get_monmap_and_config 2024-09-15T21:19:45.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.176+0000 7fb3c2240640 10 monclient: build_initial_monmap 2024-09-15T21:19:45.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.176+0000 7fb3c2240640 10 monclient: monmap: 2024-09-15T21:19:45.182 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:45.179863+0000 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:45.179863+0000 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.176+0000 7fb3c2240640 1 Processor -- start 2024-09-15T21:19:45.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.176+0000 7fb3c2240640 1 -- start start 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.176+0000 7fb3c2240640 10 monclient: init 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb3bc0a4e60 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3bc0a5950 0x7fb3bc13cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): picked mon.noname-a con 0x7fb3bc0a5950 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:45.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc13f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): picked mon.noname-b con 0x7fb3bc13d480 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3bc0a1150 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb3bc0a3c80 con 0x7fb3bc0a5950 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3bc0a0ef0 con 0x7fb3bc13d480 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:45.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3c2240640 10 monclient(hunting): authenticate will time out at 1103.976506s 2024-09-15T21:19:45.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:45.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3bc0a5950 0x7fb3bc13cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:45.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc13f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39536/0 (socket says 172.21.15.119:39536) 2024-09-15T21:19:45.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3bc0a5950 0x7fb3bc13cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34634/0 (socket says 172.21.15.119:34634) 2024-09-15T21:19:45.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb3bc0a4e60 0x7fb3bc0a5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:35872/0 (socket says 172.21.15.119:35872) 2024-09-15T21:19:45.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 1 -- 172.21.15.119:0/3364578892 learned_addr learned my addr 172.21.15.119:0/3364578892 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): get_auth_request con 0x7fb3bc13d480 auth_method 0 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): get_auth_request con 0x7fb3bc0a5950 auth_method 0 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:45.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient(hunting): handle_auth_done global_id 4923 payload 274 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 -- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 msgr2=0x7fb3bc13f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 --2- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc13f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 -- 172.21.15.119:0/3364578892 >> v1:172.21.15.17:6789/0 conn(0x7fb3bc0a4e60 legacy=0x7fb3bc0a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 monclient: found mon.noname-a 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 -- 172.21.15.119:0/3364578892 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc0a6cf0 con 0x7fb3bc0a5950 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.180+0000 7fb3baffd640 1 --2- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc13cf40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0097c0 tx=0x7fb3ac01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d123587f8a9d9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:45.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 1 -- 172.21.15.119:0/3364578892 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0b0070 con 0x7fb3bc0a5950 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: dump: 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:45.191 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 1 -- 172.21.15.119:0/3364578892 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac0ab600 con 0x7fb3bc0a5950 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 1 -- 172.21.15.119:0/3364578892 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0ab9d0 con 0x7fb3bc0a5950 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 5 monclient: authenticate success, global_id 4923 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 4 set_mon_vals no callback set 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:45.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3ba7fc640 10 monclient: dump: 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: shutdown 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a5950 msgr2=0x7fb3bc13cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 --2- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc13cf40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0097c0 tx=0x7fb3ac01c360 comp rx=0 tx=0).stop 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- 172.21.15.119:0/3364578892 shutdown_connections 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 --2- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc13f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 --2- 172.21.15.119:0/3364578892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc13cf40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- 172.21.15.119:0/3364578892 >> 172.21.15.119:0/3364578892 conn(0x7fb3bc0a6f60 msgr2=0x7fb3bc0a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- 172.21.15.119:0/3364578892 shutdown_connections 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- 172.21.15.119:0/3364578892 wait complete. 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: build_initial_monmap 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: monmap: 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:45.194 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 Processor -- start 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- start start 2024-09-15T21:19:45.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: init 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a4e60 0x7fb3bc1cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): picked mon.noname-a con 0x7fb3bc0a4e60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc1cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): picked mon.noname-b con 0x7fb3bc0a5950 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc1d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): picked mon.noname-c con 0x7fb3bc13d480 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc0a2310 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:45.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc0a2190 con 0x7fb3bc0a5950 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3bc1425d0 con 0x7fb3bc13d480 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3c2240640 10 monclient(hunting): authenticate will time out at 1103.982844s 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3baffd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3bc0a5950 msgr2=0x7fb3bc1cfc20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc1cfc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a4e60 0x7fb3bc1cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc1d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc1d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:39538/0 (socket says 172.21.15.119:39538) 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 1 -- 172.21.15.119:0/4193886132 learned_addr learned my addr 172.21.15.119:0/4193886132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): get_auth_request con 0x7fb3bc13d480 auth_method 0 2024-09-15T21:19:45.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): get_auth_request con 0x7fb3bc0a4e60 auth_method 0 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:45.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient(hunting): handle_auth_done global_id 4929 payload 824 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 -- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 msgr2=0x7fb3bc1d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc1d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 -- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3bc0a5950 msgr2=0x7fb3bc1cfc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc1cfc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 monclient: found mon.noname-a 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.184+0000 7fb3bb7fe640 1 -- 172.21.15.119:0/4193886132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc071360 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3bb7fe640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a4e60 0x7fb3bc1cf6e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0a1320 tx=0x7fb3b000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b00074e0 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:45.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: dump: 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:45.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b0007680 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004a30 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 10 monclient: dump: 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 5 monclient: authenticate success, global_id 4929 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 monclient: _renew_subs 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3b97fa640 4 set_mon_vals no callback set 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc070c10 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 monclient: _renew_subs 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc0a3200 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc0a5240 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb3b001a020 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3a803b9a0 0x7fb3a803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7fb3b004f7a0 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3baffd640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3a803b9a0 0x7fb3a803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3baffd640 10 monclient: get_auth_request con 0x7fb3a803b9a0 auth_method 0 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.188+0000 7fb3baffd640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3a803b9a0 0x7fb3a803de60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ad000 tx=0x7fb3ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.200+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb3b0012de0 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.200+0000 7fb3a3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:45.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.200+0000 7fb3a3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:45.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.372+0000 7fb3c2240640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.372+0000 7fb3c2240640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.372+0000 7fb3c2240640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.372+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7fb3bc077100 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.376+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v58) ==== 65+0+70 (secure 0 0 0) 0x7fb3b00113a0 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.376+0000 7fb3a3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.376+0000 7fb3a3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:45.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3c2240640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3c2240640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3c2240640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:45.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7fb3bc0008e0 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3a3fff640 1 -- 172.21.15.119:0/4193886132 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v58) ==== 65+0+70 (secure 0 0 0) 0x7fb3b004d0f0 con 0x7fb3bc0a4e60 2024-09-15T21:19:45.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3a3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:45.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.476+0000 7fb3a3fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:45.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3a803b9a0 msgr2=0x7fb3a803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3a803b9a0 0x7fb3a803de60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ad000 tx=0x7fb3ac008040 comp rx=0 tx=0).stop 2024-09-15T21:19:45.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 10 monclient: shutdown 2024-09-15T21:19:45.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a4e60 msgr2=0x7fb3bc1cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a4e60 0x7fb3bc1cf6e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0a1320 tx=0x7fb3b000c760 comp rx=0 tx=0).stop 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 shutdown_connections 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3a803b9a0 0x7fb3a803de60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3bc13d480 0x7fb3bc1d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3bc0a5950 0x7fb3bc1cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 --2- 172.21.15.119:0/4193886132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3bc0a4e60 0x7fb3bc1cf6e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 >> 172.21.15.119:0/4193886132 conn(0x7fb3bc0a6f60 msgr2=0x7fb3bc13f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 shutdown_connections 2024-09-15T21:19:45.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:45.480+0000 7fb3c2240640 1 -- 172.21.15.119:0/4193886132 wait complete. 2024-09-15T21:19:45.498 INFO:tasks.workunit.client.0.smithi119.stdout:bar-asdf 2024-09-15T21:19:45.499 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm foo 2024-09-15T21:19:45.499 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'does not exist' 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:45.800+0000 7fa841ffb640 1 -- 172.21.15.119:0/832279827 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v58) ==== 103+0+0 (secure 0 0 0) 0x7fa84404e510 con 0x7fa84c11d2c0 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:45.800+0000 7fa841ffb640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:45.896+0000 7fa841ffb640 1 -- 172.21.15.119:0/832279827 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v58) ==== 103+0+0 (secure 0 0 0) 0x7fa84404e890 con 0x7fa84c11d2c0 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:45.896+0000 7fa841ffb640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stdout:rule foo does not exist 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'does not exist' 2024-09-15T21:19:45.915 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm bar 2024-09-15T21:19:46.349 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:46.228+0000 7f0f897fa640 1 -- 172.21.15.119:0/2442143778 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v58) ==== 103+0+0 (secure 0 0 0) 0x7f0f800e7120 con 0x7f0f7c003a10 2024-09-15T21:19:46.349 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:46.232+0000 7f0f897fa640 10 monclient: _finish_command 2 = system:0 rule bar does not exist 2024-09-15T21:19:46.349 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:46.328+0000 7f0f897fa640 1 -- 172.21.15.119:0/2442143778 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v58) ==== 103+0+0 (secure 0 0 0) 0x7f0f800e74a0 con 0x7f0f7c003a10 2024-09-15T21:19:46.349 INFO:tasks.workunit.client.0.smithi119.stdout:2024-09-15T21:19:46.328+0000 7f0f897fa640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-09-15T21:19:46.349 INFO:tasks.workunit.client.0.smithi119.stdout:rule bar does not exist 2024-09-15T21:19:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-09-15T21:19:46.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient: get_monmap_and_config 2024-09-15T21:19:46.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient: build_initial_monmap 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient: monmap: 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:46.466959+0000 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:46.466959+0000 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:46.469 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 Processor -- start 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 -- start start 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient: init 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5700135bf0 0x7f570009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): picked mon.noname-a con 0x7f5700135bf0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5700135fd0 0x7f57000a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): picked mon.noname-c con 0x7f5700135fd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5700136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57000a1d80 con 0x7f5700136460 2024-09-15T21:19:46.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f57000a3600 con 0x7f5700135bf0 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f57000a2060 con 0x7f5700135fd0 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f57076a6640 10 monclient(hunting): authenticate will time out at 1105.263575s 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f5705c43640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f5700136460 legacy=0x7f57000a0f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f5704c41640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f5700135fd0 msgr2=0x7f57000a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f5704c41640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5700135fd0 0x7f57000a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.464+0000 7f5705442640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5700135bf0 0x7f570009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5700135bf0 0x7f570009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34694/0 (socket says 172.21.15.119:34694) 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 -- 172.21.15.119:0/4148384728 learned_addr learned my addr 172.21.15.119:0/4148384728 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): get_auth_request con 0x7f5700135bf0 auth_method 0 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:46.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient(hunting): handle_auth_done global_id 4944 payload 274 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 -- 172.21.15.119:0/4148384728 >> v2:172.21.15.119:3300/0 conn(0x7f5700135fd0 msgr2=0x7f57000a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 --2- 172.21.15.119:0/4148384728 >> v2:172.21.15.119:3300/0 conn(0x7f5700135fd0 0x7f57000a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 -- 172.21.15.119:0/4148384728 >> v1:172.21.15.119:6789/0 conn(0x7f5700136460 legacy=0x7f57000a0f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient: _finish_hunting 0 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 monclient: found mon.noname-a 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:46.474 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.468+0000 7f5705442640 1 -- 172.21.15.119:0/4148384728 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57000a2340 con 0x7f5700135bf0 2024-09-15T21:19:46.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f5705442640 1 --2- 172.21.15.119:0/4148384728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570009fe70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009870 tx=0x7f56fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56022501f5ca8f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:46.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 1 -- 172.21.15.119:0/4148384728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0b1070 con 0x7f5700135bf0 2024-09-15T21:19:46.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:46.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: dump: 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:46.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 1 -- 172.21.15.119:0/4148384728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc0a8050 con 0x7f5700135bf0 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 1 -- 172.21.15.119:0/4148384728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0ac860 con 0x7f5700135bf0 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f56f7fff640 10 monclient: dump: 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:46.478 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 5 monclient: authenticate success, global_id 4944 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 4 set_mon_vals no callback set 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: shutdown 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- 172.21.15.119:0/4148384728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 msgr2=0x7f570009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 --2- 172.21.15.119:0/4148384728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570009fe70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009870 tx=0x7f56fc01c400 comp rx=0 tx=0).stop 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- 172.21.15.119:0/4148384728 shutdown_connections 2024-09-15T21:19:46.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 --2- 172.21.15.119:0/4148384728 >> v2:172.21.15.119:3300/0 conn(0x7f5700135fd0 0x7f57000a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 --2- 172.21.15.119:0/4148384728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570009fe70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- 172.21.15.119:0/4148384728 >> 172.21.15.119:0/4148384728 conn(0x7f57000a66e0 msgr2=0x7f57000a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- 172.21.15.119:0/4148384728 shutdown_connections 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- 172.21.15.119:0/4148384728 wait complete. 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: build_initial_monmap 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: monmap: 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:46.480 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:46.481 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:46.481 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:46.481 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:46.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 Processor -- start 2024-09-15T21:19:46.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- start start 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: init 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570013bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): picked mon.noname-a con 0x7f5700135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5700135fd0 0x7f5700139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): picked mon.noname-c con 0x7f5700135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5700136460 0x7f57001319f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): picked mon.noname-b con 0x7f5700136460 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f57000a1880 con 0x7f5700135bf0 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:46.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57000a1700 con 0x7f5700136460 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57001402b0 con 0x7f5700135fd0 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f57076a6640 10 monclient(hunting): authenticate will time out at 1105.271779s 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f5705c43640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5700136460 msgr2=0x7f57001319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.472+0000 7f5705c43640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5700136460 0x7f57001319f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570013bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5700135fd0 0x7f5700139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570013bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:34702/0 (socket says 172.21.15.119:34702) 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 -- 172.21.15.119:0/1455428562 learned_addr learned my addr 172.21.15.119:0/1455428562 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:46.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): get_auth_request con 0x7f5700135bf0 auth_method 0 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): get_auth_request con 0x7f5700135fd0 auth_method 0 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:46.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:46.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:46.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:46.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient(hunting): handle_auth_done global_id 4947 payload 824 2024-09-15T21:19:46.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 -- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5700135fd0 msgr2=0x7f5700139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:46.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5700135fd0 0x7f5700139da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:46.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 -- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5700136460 msgr2=0x7f57001319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:46.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5700136460 0x7f57001319f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:46.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient: _finish_hunting 0 2024-09-15T21:19:46.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 monclient: found mon.noname-a 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 -- 172.21.15.119:0/1455428562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570013b3a0 con 0x7f5700135bf0 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5705442640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570013bf00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0af040 tx=0x7f56fc01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0bb020 con 0x7f5700135bf0 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: dump: 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:46.491 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:46.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:46.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc0b7420 con 0x7f5700135bf0 2024-09-15T21:19:46.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:46.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0b1040 con 0x7f5700135bf0 2024-09-15T21:19:46.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:46.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 10 monclient: dump: 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 5 monclient: authenticate success, global_id 4947 2024-09-15T21:19:46.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 monclient: _renew_subs 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f77fe640 4 set_mon_vals no callback set 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5700072b00 con 0x7f5700135bf0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 monclient: _renew_subs 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570013d700 con 0x7f5700135bf0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570009fe70 con 0x7f5700135bf0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f56fc0be020 con 0x7f5700135bf0 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56e403b970 0x7f56e403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:46.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7f56fc0ea9d0 con 0x7f5700135bf0 2024-09-15T21:19:46.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56e403b970 0x7f56e403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:46.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 10 monclient: get_auth_request con 0x7f56e403b970 auth_method 0 2024-09-15T21:19:46.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.476+0000 7f5704c41640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56e403b970 0x7f56e403de30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56f00038c0 tx=0x7f56f0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:46.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.488+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f56fc0c5030 con 0x7f5700135bf0 2024-09-15T21:19:46.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.488+0000 7f56f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:46.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.488+0000 7f56f5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.664+0000 7f57076a6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.664+0000 7f57076a6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.664+0000 7f57076a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:46.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:46.664+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7f570006a830 con 0x7f5700135bf0 2024-09-15T21:19:47.420 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:19:47.420 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:19:47.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:19:47.423 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:19:47.423 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:19:47.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient: get_monmap_and_config 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient: build_initial_monmap 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient: monmap: 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:47.555807+0000 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:47.555807+0000 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:47.559 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:47.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 Processor -- start 2024-09-15T21:19:47.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 -- start start 2024-09-15T21:19:47.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient: init 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): picked mon.noname-a con 0x7f550c137c50 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f550c138030 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): picked mon.noname-c con 0x7f550c0a3030 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f550c09ba60 con 0x7f550c138030 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f550c06d590 con 0x7f550c137c50 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f550c09bf80 con 0x7f550c0a3030 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f5510a89640 10 monclient(hunting): authenticate will time out at 1110.951227s 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56552/0 (socket says 172.21.15.17:56552) 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 learned_addr learned my addr 172.21.15.17:0/2864995709 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 10 monclient(hunting): get_auth_request con 0x7f550c137c50 auth_method 0 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 10 monclient(hunting): get_auth_request con 0x7f550c0a3030 auth_method 0 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:47.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.554+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:47.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 10 monclient(hunting): get_auth_request con 0x7f550c0a3030 auth_method 0 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 10 monclient(hunting): get_auth_request con 0x7f550c137c50 auth_method 0 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:47.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:47.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:47.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:47.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:47.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:47.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:47.758+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 10 monclient(hunting): get_auth_request con 0x7f550c137c50 auth_method 0 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 10 monclient(hunting): get_auth_request con 0x7f550c0a3030 auth_method 0 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.162+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:48.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:48.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 10 monclient(hunting): get_auth_request con 0x7f550c137c50 auth_method 0 2024-09-15T21:19:48.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:48.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 10 monclient(hunting): get_auth_request con 0x7f550c0a3030 auth_method 0 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:48.966+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:19:49.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:49.471+0000 7f56f4ff9640 10 monclient: tick 2024-09-15T21:19:49.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:49.471+0000 7f56f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:49.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:49.471+0000 7f56f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:49.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:49.471+0000 7f56f4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:50.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:50.475+0000 7f56f4ff9640 10 monclient: tick 2024-09-15T21:19:50.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:50.475+0000 7f56f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:50.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:50.475+0000 7f56f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): tick 2024-09-15T21:19:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:19:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 monclient(hunting): continuing hunt 2024-09-15T21:19:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:19:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 msgr2=0x7f550c13f9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.119:3300/0 conn(0x7f550c0a3030 0x7f550c13f9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 -- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 msgr2=0x7f550c09cd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 --2- 172.21.15.17:0/2864995709 >> v2:172.21.15.17:3300/0 conn(0x7f550c137c50 0x7f550c09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 -- 172.21.15.17:0/2864995709 >> v1:172.21.15.119:6789/0 conn(0x7f550c138030 legacy=0x7f550c0a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient: _add_conns ranks=[0,2] 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f54ec003680 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): picked mon.noname-b con 0x7f54ec007370 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 -- 172.21.15.17:0/2864995709 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f550c09c260 con 0x7f54ec003680 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 1 -- 172.21.15.17:0/2864995709 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f550c06d590 con 0x7f54ec007370 2024-09-15T21:19:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5503fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5509d9b640 10 monclient(hunting): get_auth_request con 0x7f54ec007370 auth_method 0 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5509d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5509d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.554+0000 7f5509d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.558+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.558+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.558+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.558+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:50.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 10 monclient(hunting): get_auth_request con 0x7f54ec007370 auth_method 0 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:50.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:50.762+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 10 monclient(hunting): get_auth_request con 0x7f54ec007370 auth_method 0 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:51.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:51.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.166+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:19:51.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:51.475+0000 7f56f4ff9640 10 monclient: tick 2024-09-15T21:19:51.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:51.475+0000 7f56f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:51.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:51.475+0000 7f56f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 10 monclient(hunting): get_auth_request con 0x7f54ec007370 auth_method 0 2024-09-15T21:19:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:19:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:19:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:51.970+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:19:52.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:52.475+0000 7f56f4ff9640 10 monclient: tick 2024-09-15T21:19:52.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:52.475+0000 7f56f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:52.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:52.475+0000 7f56f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.554+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343341130 0 0) 0x7f550c09c260 con 0x7f54ec003680 2024-09-15T21:19:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.554+0000 7f550959a640 10 monclient(hunting): _negotiate 2024-09-15T21:19:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.554+0000 7f550959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.554+0000 7f550959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.554+0000 7f550959a640 10 monclient(hunting): my global_id is 4950 2024-09-15T21:19:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.554+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54f8003530 con 0x7f54ec003680 2024-09-15T21:19:52.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4011666522 0 0) 0x7f54f8003530 con 0x7f54ec003680 2024-09-15T21:19:52.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 msgr2=0x7f54ec009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:52.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54ec007370 0x7f54ec009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: _finish_hunting 0 2024-09-15T21:19:52.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:52.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec00b010 con 0x7f54ec003680 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: _finish_auth 0 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5504009900 con 0x7f54ec003680 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: dump: 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.618+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 >> v1:172.21.15.17:6789/0 conn(0x7f54ec003680 legacy=0x7f54ec005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 1 -- 172.21.15.17:0/2864995709 reap_dead start 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f8003a60 0x7f54f8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): picked mon.c con 0x7f54f8003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8004810 0x7f54f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): picked mon.a con 0x7f54f8004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f80053d0 0x7f54f800b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): picked mon.b con 0x7f54f80053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f550c06d590 con 0x7f54f8004810 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54f800c2d0 con 0x7f54f80053d0 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f54f800c4d0 con 0x7f54f8003a60 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550959a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f8003a60 0x7f54f8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8004810 0x7f54f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): get_auth_request con 0x7f54f8003a60 auth_method 0 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): get_auth_request con 0x7f54f8004810 auth_method 0 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f80053d0 0x7f54f800b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): get_auth_request con 0x7f54f80053d0 auth_method 0 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5509d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient(hunting): handle_auth_done global_id 4950 payload 293 2024-09-15T21:19:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f80053d0 msgr2=0x7f54f800b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f80053d0 0x7f54f800b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8004810 msgr2=0x7f54f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8004810 0x7f54f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 monclient: found mon.c 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 1 -- 172.21.15.17:0/2864995709 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f8003530 con 0x7f54f8003a60 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient: _finish_auth 0 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 10 monclient: _check_auth_tickets 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f550a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5510a89640 5 monclient: authenticate success, global_id 4950 2024-09-15T21:19:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.622+0000 7f5510a89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550a59c640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f8003a60 0x7f54f8003e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55040077a0 tx=0x7f550401d980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4d11bf9e5cc02f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040ac070 con 0x7f54f8003a60 2024-09-15T21:19:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: dump: 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55040039a0 con 0x7f54f8003a60 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f5510a89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f5510a89640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f5510a89640 4 set_mon_vals no callback set 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 1 -- 172.21.15.17:0/2864995709 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040073f0 con 0x7f54f8003a60 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.626+0000 7f550959a640 10 monclient: dump: 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient: shutdown 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f8003a60 msgr2=0x7f54f8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f8003a60 0x7f54f8003e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55040077a0 tx=0x7f550401d980 comp rx=0 tx=0).stop 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- 172.21.15.17:0/2864995709 shutdown_connections 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f80053d0 0x7f54f800b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8004810 0x7f54f8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- 172.21.15.17:0/2864995709 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f8003a60 0x7f54f8003e60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- 172.21.15.17:0/2864995709 >> 172.21.15.17:0/2864995709 conn(0x7f550c0a7510 msgr2=0x7f550c0a3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- 172.21.15.17:0/2864995709 shutdown_connections 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- 172.21.15.17:0/2864995709 wait complete. 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient: build_initial_monmap 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient: monmap: 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:52.639 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:52.639 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:52.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 Processor -- start 2024-09-15T21:19:52.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- start start 2024-09-15T21:19:52.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient: init 2024-09-15T21:19:52.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 0x7f550c1cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): picked mon.noname-a con 0x7f54f8003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 0x7f550c135a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): picked mon.noname-b con 0x7f54f8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f80053d0 0x7f550c11f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): picked mon.noname-c con 0x7f54f80053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f550c0a4500 con 0x7f54f8003a60 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f550c0a4380 con 0x7f54f8004810 2024-09-15T21:19:52.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f550c0a2920 con 0x7f54f80053d0 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.630+0000 7f5510a89640 10 monclient(hunting): authenticate will time out at 1116.027903s 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 0x7f550c1cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 0x7f550c1cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47434/0 (socket says 172.21.15.17:47434) 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 -- 172.21.15.17:0/1067504256 learned_addr learned my addr 172.21.15.17:0/1067504256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f80053d0 0x7f550c11f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): get_auth_request con 0x7f54f8003a60 auth_method 0 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): get_auth_request con 0x7f54f80053d0 auth_method 0 2024-09-15T21:19:52.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 0x7f550c135a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 10 monclient(hunting): get_auth_request con 0x7f54f8004810 auth_method 0 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient(hunting): handle_auth_done global_id 4953 payload 824 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 1 -- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 msgr2=0x7f550c135a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 1 -- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 msgr2=0x7f550c135a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 0x7f550c135a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 -- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f80053d0 msgr2=0x7f550c11f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f80053d0 0x7f550c11f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5509d9b640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 0x7f550c135a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 -- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 msgr2=0x7f550c135a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 0x7f550c135a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 monclient: found mon.noname-a 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:52.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 -- 172.21.15.17:0/1067504256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c0a4f30 con 0x7f54f8003a60 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f550a59c640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 0x7f550c1cf9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55040093f0 tx=0x7f54f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040ac040 con 0x7f54f8003a60 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: dump: 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:52.648 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5504003d50 con 0x7f54f8003a60 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 5 monclient: authenticate success, global_id 4953 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 monclient: _renew_subs 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c09e100 con 0x7f54f8003a60 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040a6830 con 0x7f54f8003a60 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:52.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 10 monclient: dump: 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 monclient: _renew_subs 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:52.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c123170 con 0x7f54f8003a60 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5508d99640 4 set_mon_vals no callback set 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550c121c50 con 0x7f54f8003a60 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f55040a69d0 con 0x7f54f8003a60 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54ec03c370 0x7f54ec03e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.634+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7f55040bd070 con 0x7f54f8003a60 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.638+0000 7f5509d9b640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54ec03c370 0x7f54ec03e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.638+0000 7f5509d9b640 10 monclient: get_auth_request con 0x7f54ec03c370 auth_method 0 2024-09-15T21:19:52.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.638+0000 7f5509d9b640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54ec03c370 0x7f54ec03e830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f550c123380 tx=0x7f54f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:52.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.642+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f550c09e100 con 0x7f54f8003a60 2024-09-15T21:19:52.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.642+0000 7f5502ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:52.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.642+0000 7f5502ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5510a89640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5510a89640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5510a89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f550c09bf80 con 0x7f54f8003a60 2024-09-15T21:19:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5502ffd640 1 -- 172.21.15.17:0/1067504256 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f550c0a4f30 con 0x7f54f8003a60 2024-09-15T21:19:52.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5502ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:52.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.878+0000 7f5502ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:52.883 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54ec03c370 msgr2=0x7f54ec03e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54ec03c370 0x7f54ec03e830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f550c123380 tx=0x7f54f4009040 comp rx=0 tx=0).stop 2024-09-15T21:19:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 10 monclient: shutdown 2024-09-15T21:19:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 msgr2=0x7f550c1cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 0x7f550c1cf9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55040093f0 tx=0x7f54f8005840 comp rx=0 tx=0).stop 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 shutdown_connections 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54ec03c370 0x7f54ec03e830 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54f80053d0 0x7f550c11f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f8004810 0x7f550c135a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 --2- 172.21.15.17:0/1067504256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f8003a60 0x7f550c1cf9a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 >> 172.21.15.17:0/1067504256 conn(0x7f550c0a7510 msgr2=0x7f550c0a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 shutdown_connections 2024-09-15T21:19:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:52.882+0000 7f5510a89640 1 -- 172.21.15.17:0/1067504256 wait complete. 2024-09-15T21:19:52.902 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:19:52.902 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:19:52.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient: get_monmap_and_config 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient: monmap: 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:53.018247+0000 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:53.018247+0000 2024-09-15T21:19:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:53.021 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 Processor -- start 2024-09-15T21:19:53.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 -- start start 2024-09-15T21:19:53.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient: init 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f46b80a4c00 0x7f46b809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): picked mon.noname-c con 0x7f46b80a4c00 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f46b80a4fe0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46b813ce10 0x7f46b813f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): picked mon.noname-a con 0x7f46b813ce10 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b809c700 con 0x7f46b80a4fe0 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f46b806f820 con 0x7f46b813ce10 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f46b80a5fc0 con 0x7f46b80a4c00 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46c01e3640 10 monclient(hunting): authenticate will time out at 1116.413588s 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46b813ce10 0x7f46b813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46b813ce10 0x7f46b813f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47450/0 (socket says 172.21.15.17:47450) 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 -- 172.21.15.17:0/361475016 learned_addr learned my addr 172.21.15.17:0/361475016 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): get_auth_request con 0x7f46b813ce10 auth_method 0 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46bdf7f640 1 --2- 172.21.15.17:0/361475016 >> v2:172.21.15.119:3300/0 conn(0x7f46b80a4c00 0x7f46b809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46bdf7f640 10 monclient(hunting): get_auth_request con 0x7f46b80a4c00 auth_method 0 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46bdf7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46bdf7f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46bdf7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient(hunting): handle_auth_done global_id 4956 payload 274 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 -- 172.21.15.17:0/361475016 >> v2:172.21.15.119:3300/0 conn(0x7f46b80a4c00 msgr2=0x7f46b809cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 --2- 172.21.15.17:0/361475016 >> v2:172.21.15.119:3300/0 conn(0x7f46b80a4c00 0x7f46b809cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 -- 172.21.15.17:0/361475016 >> v1:172.21.15.119:6789/0 conn(0x7f46b80a4fe0 legacy=0x7f46b80a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 monclient: found mon.noname-a 2024-09-15T21:19:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.018+0000 7f46be780640 1 -- 172.21.15.17:0/361475016 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b80a5d10 con 0x7f46b813ce10 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 --2- 172.21.15.17:0/361475016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b813f200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46b80a7bc0 tx=0x7f46b001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d365f6664e274 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 1 -- 172.21.15.17:0/361475016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00b0070 con 0x7f46b813ce10 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: dump: 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 1 -- 172.21.15.17:0/361475016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b00ab600 con 0x7f46b813ce10 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 1 -- 172.21.15.17:0/361475016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00ab9d0 con 0x7f46b813ce10 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 5 monclient: authenticate success, global_id 4956 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 4 set_mon_vals no callback set 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bcf7d640 10 monclient: dump: 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.035 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: shutdown 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- 172.21.15.17:0/361475016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 msgr2=0x7f46b813f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 --2- 172.21.15.17:0/361475016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b813f200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46b80a7bc0 tx=0x7f46b001c470 comp rx=0 tx=0).stop 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- 172.21.15.17:0/361475016 shutdown_connections 2024-09-15T21:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 --2- 172.21.15.17:0/361475016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b813f200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 --2- 172.21.15.17:0/361475016 >> v2:172.21.15.119:3300/0 conn(0x7f46b80a4c00 0x7f46b809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- 172.21.15.17:0/361475016 >> 172.21.15.17:0/361475016 conn(0x7f46b80aaeb0 msgr2=0x7f46b80a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- 172.21.15.17:0/361475016 shutdown_connections 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- 172.21.15.17:0/361475016 wait complete. 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: monmap: 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 Processor -- start 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- start start 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: init 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46b80a4c00 0x7f46b8120d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): picked mon.noname-b con 0x7f46b80a4c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46b80a4fe0 0x7f46b8137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): picked mon.noname-c con 0x7f46b80a4fe0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b81cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): picked mon.noname-a con 0x7f46b813ce10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81422f0 con 0x7f46b813ce10 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8142170 con 0x7f46b80a4c00 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f46b8142470 con 0x7f46b80a4fe0 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46c01e3640 10 monclient(hunting): authenticate will time out at 1116.419139s 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46b80a4fe0 0x7f46b8137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46b80a4fe0 0x7f46b8137bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59342/0 (socket says 172.21.15.17:59342) 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 1 -- 172.21.15.17:0/3740627866 learned_addr learned my addr 172.21.15.17:0/3740627866 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): get_auth_request con 0x7f46b80a4fe0 auth_method 0 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b81cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): get_auth_request con 0x7f46b813ce10 auth_method 0 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bdf7f640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46b80a4c00 0x7f46b8120d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bdf7f640 10 monclient(hunting): get_auth_request con 0x7f46b80a4c00 auth_method 0 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bdf7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bdf7f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bdf7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient(hunting): handle_auth_done global_id 4959 payload 824 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46b80a4fe0 msgr2=0x7f46b8137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46b80a4fe0 0x7f46b8137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46b80a4c00 msgr2=0x7f46b8120d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46b80a4c00 0x7f46b8120d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 monclient: found mon.noname-a 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b8122560 con 0x7f46b813ce10 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46bd77e640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46b80a4fe0 0x7f46b8137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.022+0000 7f46be780640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b81cf510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46b00ab000 tx=0x7f46b0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00b0040 con 0x7f46b813ce10 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: dump: 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.042 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 5 monclient: authenticate success, global_id 4959 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b00aa6d0 con 0x7f46b813ce10 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 10 monclient: _renew_subs 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b80a13f0 con 0x7f46b813ce10 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00aaac0 con 0x7f46b813ce10 2024-09-15T21:19:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: dump: 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46af7fe640 4 set_mon_vals no callback set 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 10 monclient: _renew_subs 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b80a0450 con 0x7f46b813ce10 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 0x7f46b81cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f46b806a080 con 0x7f46b8121cc0 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 0x7f46b81cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient: get_auth_request con 0x7f46b8121cc0 auth_method 0 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f46b00aac60 con 0x7f46b813ce10 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_done global_id 4962 payload 824 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46b809cb20 con 0x7f46b8121cc0 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f469c03b9c0 0x7f469c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bdf7f640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 0x7f46b81cfa50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f46b8138bb0 tx=0x7f46b4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7f46b00e9cd0 con 0x7f46b813ce10 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4014610 con 0x7f46b8121cc0 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: dump: 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bd77e640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f469c03b9c0 0x7f469c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bd77e640 10 monclient: get_auth_request con 0x7f469c03b9c0 auth_method 0 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f46b4019070 con 0x7f46b8121cc0 2024-09-15T21:19:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 msgr2=0x7f46b81cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 0x7f46b81cfa50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f46b8138bb0 tx=0x7f46b4007700 comp rx=0 tx=0).stop 2024-09-15T21:19:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 msgr2=0x7f46b81cfa50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46ae7fc640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8121cc0 0x7f46b81cfa50 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f46b8138bb0 tx=0x7f46b4007700 comp rx=0 tx=0).stop 2024-09-15T21:19:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.026+0000 7f46bd77e640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f469c03b9c0 0x7f469c03de80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f46b8120cf0 tx=0x7f46a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 0x7f46b81cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f46b80692d0 con 0x7f46b8138110 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 0x7f46b81cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient: get_auth_request con 0x7f46b8138110 auth_method 0 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 10 monclient(hunting): handle_auth_done global_id 4965 payload 824 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 1 -- 172.21.15.17:0/3740627866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f46b81da860 con 0x7f46b8138110 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46bdf7f640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 0x7f46b81cff90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f46b8138bb0 tx=0x7f46b4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40148d0 con 0x7f46b8138110 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 10 monclient: dump: 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f46b40148d0 con 0x7f46b8138110 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 msgr2=0x7f46b81cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 0x7f46b81cff90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f46b8138bb0 tx=0x7f46b4004a50 comp rx=0 tx=0).stop 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 msgr2=0x7f46b81cff90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46ae7fc640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b8138110 0x7f46b81cff90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 32, 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 158655, 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.055 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:53.065 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f469c03b9c0 msgr2=0x7f469c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f469c03b9c0 0x7f469c03de80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f46b8120cf0 tx=0x7f46a8009040 comp rx=0 tx=0).stop 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 10 monclient: shutdown 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 msgr2=0x7f46b81cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 --2- 172.21.15.17:0/3740627866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b813ce10 0x7f46b81cf510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46b00ab000 tx=0x7f46b0019210 comp rx=0 tx=0).stop 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46be780640 1 -- 172.21.15.17:0/3740627866 reap_dead start 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46be780640 1 -- 172.21.15.17:0/3740627866 reap_dead start 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 shutdown_connections 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 >> 172.21.15.17:0/3740627866 conn(0x7f46b80aaeb0 msgr2=0x7f46b813d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 shutdown_connections 2024-09-15T21:19:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.034+0000 7f46c01e3640 1 -- 172.21.15.17:0/3740627866 wait complete. 2024-09-15T21:19:53.068 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:53.068 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:19:53.068 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient: get_monmap_and_config 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient: monmap: 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:53.180116+0000 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:53.180116+0000 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:53.183 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 Processor -- start 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 -- start start 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient: init 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7febfc0a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febfc09e8c0 0x7febfc0a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): picked mon.noname-a con 0x7febfc09e8c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7febfc1410a0 0x7febfc143490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): picked mon.noname-c con 0x7febfc1410a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febfc0a31c0 con 0x7febfc0a0890 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7febfc09d640 con 0x7febfc09e8c0 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7febfc09d920 con 0x7febfc1410a0 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec02b93640 10 monclient(hunting): authenticate will time out at 1116.575182s 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febfc09e8c0 0x7febfc0a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec0092f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7febfc0a0890 0x7febfc0a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39908/0 (socket says 172.21.15.17:39908) 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec0092f640 1 -- 172.21.15.17:0/1442205079 learned_addr learned my addr 172.21.15.17:0/1442205079 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): get_auth_request con 0x7febfc09e8c0 auth_method 0 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec01130640 1 --2- 172.21.15.17:0/1442205079 >> v2:172.21.15.119:3300/0 conn(0x7febfc1410a0 0x7febfc143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec01130640 10 monclient(hunting): get_auth_request con 0x7febfc1410a0 auth_method 0 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec01130640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec01130640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7fec01130640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient(hunting): handle_auth_done global_id 4968 payload 274 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 1 -- 172.21.15.17:0/1442205079 >> v2:172.21.15.119:3300/0 conn(0x7febfc1410a0 msgr2=0x7febfc143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 1 --2- 172.21.15.17:0/1442205079 >> v2:172.21.15.119:3300/0 conn(0x7febfc1410a0 0x7febfc143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 1 -- 172.21.15.17:0/1442205079 >> v1:172.21.15.17:6789/0 conn(0x7febfc0a0890 legacy=0x7febfc0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 1 monclient: found mon.noname-a 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.178+0000 7febfbfff640 1 -- 172.21.15.17:0/1442205079 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc09dc00 con 0x7febfc09e8c0 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfbfff640 1 --2- 172.21.15.17:0/1442205079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc09e8c0 0x7febfc0a1380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7febf00097c0 tx=0x7febf001c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2c2cf115c81407 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 1 -- 172.21.15.17:0/1442205079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00b0070 con 0x7febfc09e8c0 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: dump: 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 5 monclient: authenticate success, global_id 4968 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 1 -- 172.21.15.17:0/1442205079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf00ab600 con 0x7febfc09e8c0 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 4 set_mon_vals no callback set 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 1 -- 172.21.15.17:0/1442205079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00ab9d0 con 0x7febfc09e8c0 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7febfb7fe640 10 monclient: dump: 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 10 monclient: shutdown 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 -- 172.21.15.17:0/1442205079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc09e8c0 msgr2=0x7febfc0a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 --2- 172.21.15.17:0/1442205079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc09e8c0 0x7febfc0a1380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7febf00097c0 tx=0x7febf001c320 comp rx=0 tx=0).stop 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 -- 172.21.15.17:0/1442205079 shutdown_connections 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 --2- 172.21.15.17:0/1442205079 >> v2:172.21.15.119:3300/0 conn(0x7febfc1410a0 0x7febfc143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 --2- 172.21.15.17:0/1442205079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc09e8c0 0x7febfc0a1380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 -- 172.21.15.17:0/1442205079 >> 172.21.15.17:0/1442205079 conn(0x7febfc0a66a0 msgr2=0x7febfc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 -- 172.21.15.17:0/1442205079 shutdown_connections 2024-09-15T21:19:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 1 -- 172.21.15.17:0/1442205079 wait complete. 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.182+0000 7fec02b93640 10 monclient: monmap: 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 Processor -- start 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- start start 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: init 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc0a0890 0x7febfc1d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): picked mon.noname-c con 0x7febfc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1410a0 0x7febfc1d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): picked mon.noname-b con 0x7febfc1410a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc1d4700 0x7febfc0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): picked mon.noname-a con 0x7febfc1d4700 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7febfc146580 con 0x7febfc1d4700 2024-09-15T21:19:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febfc146400 con 0x7febfc1410a0 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febfc146700 con 0x7febfc0a0890 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): authenticate will time out at 1116.580845s 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc0a0890 0x7febfc1d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc1d4700 0x7febfc0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc0a0890 0x7febfc1d3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59356/0 (socket says 172.21.15.17:59356) 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 -- 172.21.15.17:0/2265656500 learned_addr learned my addr 172.21.15.17:0/2265656500 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): get_auth_request con 0x7febfc0a0890 auth_method 0 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): get_auth_request con 0x7febfc1d4700 auth_method 0 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1410a0 0x7febfc1d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): get_auth_request con 0x7febfc1410a0 auth_method 0 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient(hunting): handle_auth_done global_id 24326 payload 824 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1410a0 msgr2=0x7febfc1d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1410a0 0x7febfc1d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc1d4700 msgr2=0x7febfc0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febfc1d4700 0x7febfc0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 monclient: found mon.noname-c 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc09d0f0 con 0x7febfc0a0890 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec0092f640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc0a0890 0x7febfc1d3a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7febec00c480 tx=0x7febec00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec017070 con 0x7febfc0a0890 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: dump: 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 5 monclient: authenticate success, global_id 24326 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec005d10 con 0x7febfc0a0890 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: _renew_subs 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc126740 con 0x7febfc0a0890 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0072b0 con 0x7febfc0a0890 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: dump: 2024-09-15T21:19:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: _renew_subs 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfaffd640 4 set_mon_vals no callback set 2024-09-15T21:19:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc073d80 con 0x7febfc0a0890 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 0x7febfc072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febfc1263c0 con 0x7febfc09e8c0 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 0x7febfc072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient: get_auth_request con 0x7febfc09e8c0 auth_method 0 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7febec01a070 con 0x7febfc0a0890 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 10 monclient(hunting): handle_auth_done global_id 24329 payload 824 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febfc0a0c70 con 0x7febfc09e8c0 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febe803b9f0 0x7febe803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7febec04eff0 con 0x7febfc0a0890 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febfbfff640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 0x7febfc072800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febfc09d0a0 tx=0x7febf001ab60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00b0040 con 0x7febfc09e8c0 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: dump: 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febe803b9f0 0x7febe803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7fec01130640 10 monclient: get_auth_request con 0x7febe803b9f0 auth_method 0 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7febf00abde0 con 0x7febfc09e8c0 2024-09-15T21:19:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 msgr2=0x7febfc072800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 0x7febfc072800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febfc09d0a0 tx=0x7febf001ab60 comp rx=0 tx=0).stop 2024-09-15T21:19:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 msgr2=0x7febfc072800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.186+0000 7febf97fa640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc09e8c0 0x7febfc072800 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febfc09d0a0 tx=0x7febf001ab60 comp rx=0 tx=0).stop 2024-09-15T21:19:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.190+0000 7fec01130640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febe803b9f0 0x7febe803deb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febfc1d4ae0 tx=0x7febf4009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 0x7febfc06f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febfc073150 con 0x7febfc072d40 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 0x7febfc06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient: get_auth_request con 0x7febfc072d40 auth_method 0 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 10 monclient(hunting): handle_auth_done global_id 24332 payload 824 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 1 -- 172.21.15.17:0/2265656500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7febfc06d020 con 0x7febfc072d40 2024-09-15T21:19:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febfbfff640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 0x7febfc06f230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c10 tx=0x7febf00190c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00b0050 con 0x7febfc072d40 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 10 monclient: dump: 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7febf0003c60 con 0x7febfc072d40 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 msgr2=0x7febfc06f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 0x7febfc06f230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c10 tx=0x7febf00190c0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 msgr2=0x7febfc06f230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7febf97fa640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc072d40 0x7febfc06f230 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c10 tx=0x7febf00190c0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.213 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 32, 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 48999, 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:53.214 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.225 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:53.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.227 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.227 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:53.227 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:53.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febe803b9f0 msgr2=0x7febe803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febe803b9f0 0x7febe803deb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febfc1d4ae0 tx=0x7febf4009820 comp rx=0 tx=0).stop 2024-09-15T21:19:53.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 10 monclient: shutdown 2024-09-15T21:19:53.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc0a0890 msgr2=0x7febfc1d3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 --2- 172.21.15.17:0/2265656500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febfc0a0890 0x7febfc1d3a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7febec00c480 tx=0x7febec00c940 comp rx=0 tx=0).stop 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec01130640 1 -- 172.21.15.17:0/2265656500 reap_dead start 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec01130640 1 -- 172.21.15.17:0/2265656500 reap_dead start 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 shutdown_connections 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 >> 172.21.15.17:0/2265656500 conn(0x7febfc0a66a0 msgr2=0x7febfc1418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 shutdown_connections 2024-09-15T21:19:53.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.194+0000 7fec02b93640 1 -- 172.21.15.17:0/2265656500 wait complete. 2024-09-15T21:19:53.229 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:53.229 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:19:53.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient: get_monmap_and_config 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient: monmap: 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:53.343308+0000 2024-09-15T21:19:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:53.343308+0000 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 1 Processor -- start 2024-09-15T21:19:53.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 1 -- start start 2024-09-15T21:19:53.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient: init 2024-09-15T21:19:53.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ea01340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3ea009d6c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ea013a570 0x7f3ea013c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient(hunting): picked mon.noname-a con 0x7f3ea013a570 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea009e1b0 con 0x7f3ea009d6c0 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea009e490 con 0x7f3ea01340c0 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea009d520 con 0x7f3ea013a570 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea6c92640 10 monclient(hunting): authenticate will time out at 1116.739500s 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ea013a570 0x7f3ea013c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3e9ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3ea009d6c0 0x7f3ea0138e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39922/0 (socket says 172.21.15.17:39922) 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ea013a570 0x7f3ea013c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47512/0 (socket says 172.21.15.17:47512) 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3e9ffff640 1 -- 172.21.15.17:0/675844045 learned_addr learned my addr 172.21.15.17:0/675844045 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): get_auth_request con 0x7f3ea013a570 auth_method 0 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.342+0000 7f3ea522f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 1 -- 172.21.15.17:0/675844045 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499932956 0 0) 0x7f3ea009e1b0 con 0x7f3ea009d6c0 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient(hunting): my global_id is 4980 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 1 -- 172.21.15.17:0/675844045 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e88003530 con 0x7f3ea009d6c0 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 10 monclient(hunting): handle_auth_done global_id 4977 payload 274 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 1 -- 172.21.15.17:0/675844045 >> v1:172.21.15.119:6789/0 conn(0x7f3ea01340c0 legacy=0x7f3ea01344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 1 -- 172.21.15.17:0/675844045 >> v1:172.21.15.17:6789/0 conn(0x7f3ea009d6c0 legacy=0x7f3ea0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 1 monclient: found mon.noname-a 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 1 -- 172.21.15.17:0/675844045 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea009e770 con 0x7f3ea013a570 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3ea522f640 1 --2- 172.21.15.17:0/675844045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea013c960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009d90 tx=0x7f3e9801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de2f51487051834 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 1 -- 172.21.15.17:0/675844045 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980a6680 con 0x7f3ea013a570 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.346+0000 7f3e9f7fe640 10 monclient: dump: 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 1 -- 172.21.15.17:0/675844045 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e980a6820 con 0x7f3ea013a570 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 1 -- 172.21.15.17:0/675844045 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980a6bf0 con 0x7f3ea013a570 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 5 monclient: authenticate success, global_id 4977 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 4 set_mon_vals no callback set 2024-09-15T21:19:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9f7fe640 10 monclient: dump: 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: shutdown 2024-09-15T21:19:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/675844045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 msgr2=0x7f3ea013c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 --2- 172.21.15.17:0/675844045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea013c960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009d90 tx=0x7f3e9801c830 comp rx=0 tx=0).stop 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/675844045 shutdown_connections 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 --2- 172.21.15.17:0/675844045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea013c960 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/675844045 >> 172.21.15.17:0/675844045 conn(0x7f3ea00a66a0 msgr2=0x7f3ea00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/675844045 shutdown_connections 2024-09-15T21:19:53.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/675844045 wait complete. 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: monmap: 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:53.362 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 Processor -- start 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- start start 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: init 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea009d6c0 0x7f3ea006f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): picked mon.noname-b con 0x7f3ea009d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ea01340c0 0x7f3ea006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): picked mon.noname-c con 0x7f3ea01340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea0070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): picked mon.noname-a con 0x7f3ea013a570 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0143650 con 0x7f3ea013a570 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01434d0 con 0x7f3ea009d6c0 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea01437d0 con 0x7f3ea01340c0 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient(hunting): authenticate will time out at 1116.746406s 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea0070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ea01340c0 0x7f3ea006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea0070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47514/0 (socket says 172.21.15.17:47514) 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 1 -- 172.21.15.17:0/3956067766 learned_addr learned my addr 172.21.15.17:0/3956067766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ea01340c0 0x7f3ea006fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59384/0 (socket says 172.21.15.17:59384) 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): get_auth_request con 0x7f3ea013a570 auth_method 0 2024-09-15T21:19:53.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): get_auth_request con 0x7f3ea01340c0 auth_method 0 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea4a2e640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea009d6c0 0x7f3ea006f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea4a2e640 10 monclient(hunting): get_auth_request con 0x7f3ea009d6c0 auth_method 0 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea4a2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea4a2e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea4a2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient(hunting): handle_auth_done global_id 24335 payload 824 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea009d6c0 msgr2=0x7f3ea006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea009d6c0 0x7f3ea006f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 msgr2=0x7f3ea0070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea0070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 monclient: found mon.noname-c 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea009d0f0 con 0x7f3ea01340c0 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea522f640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ea013a570 0x7f3ea0070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9ffff640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ea01340c0 0x7f3ea006fee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c400 tx=0x7f3e9400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94006e00 con 0x7f3ea01340c0 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: dump: 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.367 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e94006110 con 0x7f3ea01340c0 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94006500 con 0x7f3ea01340c0 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9d7fa640 10 monclient: dump: 2024-09-15T21:19:53.368 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 5 monclient: authenticate success, global_id 24335 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: _renew_subs 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3e9effd640 4 set_mon_vals no callback set 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea00734e0 con 0x7f3ea01340c0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: _renew_subs 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.350+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea0073200 con 0x7f3ea01340c0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea6c92640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea6c92640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 0x7f3ea002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea6c92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0133300 con 0x7f3ea01d0290 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3e94011070 con 0x7f3ea01340c0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e8c03b9c0 0x7f3e8c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7f3e9404f790 con 0x7f3ea01340c0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 0x7f3ea002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient: get_auth_request con 0x7f3ea01d0290 auth_method 0 2024-09-15T21:19:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea522f640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e8c03b9c0 0x7f3e8c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea522f640 10 monclient: get_auth_request con 0x7f3e8c03b9c0 auth_method 0 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_done global_id 34102 payload 824 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ea01344a0 con 0x7f3ea01d0290 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea4a2e640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 0x7f3ea002d450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e4b0 tx=0x7f3e9000e970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3ea522f640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e8c03b9c0 0x7f3e8c03de80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e9801c870 tx=0x7f3e980a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900052e0 con 0x7f3ea01d0290 2024-09-15T21:19:53.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 10 monclient: dump: 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3e90018070 con 0x7f3ea01d0290 2024-09-15T21:19:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 msgr2=0x7f3ea002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 0x7f3ea002d450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e4b0 tx=0x7f3e9000e970 comp rx=0 tx=0).stop 2024-09-15T21:19:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 msgr2=0x7f3ea002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.354+0000 7f3e9d7fa640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0290 0x7f3ea002d450 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e4b0 tx=0x7f3e9000e970 comp rx=0 tx=0).stop 2024-09-15T21:19:53.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:53.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 0x7f3ea002d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea013c770 con 0x7f3ea01d0670 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 0x7f3ea002d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient: get_auth_request con 0x7f3ea01d0670 auth_method 0 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 10 monclient(hunting): handle_auth_done global_id 34105 payload 824 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 1 -- 172.21.15.17:0/3956067766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3ea0072400 con 0x7f3ea01d0670 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea4a2e640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 0x7f3ea002d990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e90017000 tx=0x7f3e90001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90003a10 con 0x7f3ea01d0670 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 10 monclient: dump: 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.376 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f3e90003a10 con 0x7f3ea01d0670 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 msgr2=0x7f3ea002d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 0x7f3ea002d990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e90017000 tx=0x7f3e90001130 comp rx=0 tx=0).stop 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 msgr2=0x7f3ea002d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3e9d7fa640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea01d0670 0x7f3ea002d990 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e90017000 tx=0x7f3e90001130 comp rx=0 tx=0).stop 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:53.377 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 32, 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 5863, 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.378 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.379 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.384 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:53.386 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:53.389 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e8c03b9c0 msgr2=0x7f3e8c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e8c03b9c0 0x7f3e8c03de80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e9801c870 tx=0x7f3e980a7000 comp rx=0 tx=0).stop 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 10 monclient: shutdown 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ea01340c0 msgr2=0x7f3ea006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 --2- 172.21.15.17:0/3956067766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ea01340c0 0x7f3ea006fee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c400 tx=0x7f3e9400c8c0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea522f640 1 -- 172.21.15.17:0/3956067766 reap_dead start 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea522f640 1 -- 172.21.15.17:0/3956067766 reap_dead start 2024-09-15T21:19:53.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 shutdown_connections 2024-09-15T21:19:53.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 >> 172.21.15.17:0/3956067766 conn(0x7f3ea00a66a0 msgr2=0x7f3ea0139ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 shutdown_connections 2024-09-15T21:19:53.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.362+0000 7f3ea6c92640 1 -- 172.21.15.17:0/3956067766 wait complete. 2024-09-15T21:19:53.392 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:53.392 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:19:53.392 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:19:53.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.475+0000 7f56f4ff9640 10 monclient: tick 2024-09-15T21:19:53.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.475+0000 7f56f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.479 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.475+0000 7f56f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.502+0000 7fb10566f640 10 monclient: get_monmap_and_config 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.502+0000 7fb10566f640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.502+0000 7fb10566f640 10 monclient: monmap: 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:53.505054+0000 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:53.505054+0000 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:53.508 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.502+0000 7fb10566f640 1 Processor -- start 2024-09-15T21:19:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.502+0000 7fb10566f640 1 -- start start 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.502+0000 7fb10566f640 10 monclient: init 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001378c0 0x7fb100137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient(hunting): picked mon.noname-b con 0x7fb1001378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb1001381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb1000a5210 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1000a1a20 con 0x7fb1001381e0 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1000a1d00 con 0x7fb1000a5210 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb1000a1fe0 con 0x7fb1001378c0 2024-09-15T21:19:53.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient(hunting): authenticate will time out at 1116.900562s 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001378c0 0x7fb100137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fe7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb1001381e0 0x7fb1000a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39924/0 (socket says 172.21.15.17:39924) 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fe7fc640 1 -- 172.21.15.17:0/3174250556 learned_addr learned my addr 172.21.15.17:0/3174250556 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): get_auth_request con 0x7fb1001378c0 auth_method 0 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient(hunting): handle_auth_done global_id 24338 payload 274 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 1 -- 172.21.15.17:0/3174250556 >> v1:172.21.15.119:6789/0 conn(0x7fb1000a5210 legacy=0x7fb1000a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 1 -- 172.21.15.17:0/3174250556 >> v1:172.21.15.17:6789/0 conn(0x7fb1001381e0 legacy=0x7fb1000a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 1 -- 172.21.15.17:0/3174250556 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445643585 0 0) 0x7fb0f4001120 con 0x7fb1001381e0 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 1 monclient: found mon.noname-b 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 1 -- 172.21.15.17:0/3174250556 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1000a22c0 con 0x7fb1001378c0 2024-09-15T21:19:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0feffd640 1 --2- 172.21.15.17:0/3174250556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001378c0 0x7fb100137ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00097f0 tx=0x7fb0f001c080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74210c4e7adb798e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 1 -- 172.21.15.17:0/3174250556 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00b1070 con 0x7fb1001378c0 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: dump: 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 5 monclient: authenticate success, global_id 24338 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 1 -- 172.21.15.17:0/3174250556 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f00a4730 con 0x7fb1001378c0 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 1 -- 172.21.15.17:0/3174250556 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00a4b00 con 0x7fb1001378c0 2024-09-15T21:19:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb10566f640 4 set_mon_vals no callback set 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.506+0000 7fb0fdffb640 10 monclient: dump: 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: shutdown 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/3174250556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001378c0 msgr2=0x7fb100137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 --2- 172.21.15.17:0/3174250556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001378c0 0x7fb100137ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00097f0 tx=0x7fb0f001c080 comp rx=0 tx=0).stop 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/3174250556 shutdown_connections 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 --2- 172.21.15.17:0/3174250556 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001378c0 0x7fb100137ca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/3174250556 >> 172.21.15.17:0/3174250556 conn(0x7fb1000aac40 msgr2=0x7fb1000a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/3174250556 shutdown_connections 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/3174250556 wait complete. 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: monmap: 2024-09-15T21:19:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 Processor -- start 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- start start 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: init 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1000a5210 0x7fb100136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): picked mon.noname-b con 0x7fb1000a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1001378c0 0x7fb10013ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): picked mon.noname-a con 0x7fb1001378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 0x7fb10013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): picked mon.noname-c con 0x7fb1001381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb100144260 con 0x7fb1001378c0 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001440e0 con 0x7fb1000a5210 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb1001443e0 con 0x7fb1001381e0 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient(hunting): authenticate will time out at 1116.905705s 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 0x7fb10013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1001378c0 0x7fb10013ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 0x7fb10013d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59402/0 (socket says 172.21.15.17:59402) 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 -- 172.21.15.17:0/154021984 learned_addr learned my addr 172.21.15.17:0/154021984 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): get_auth_request con 0x7fb1001381e0 auth_method 0 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): get_auth_request con 0x7fb1001378c0 auth_method 0 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0feffd640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1000a5210 0x7fb100136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0feffd640 10 monclient(hunting): get_auth_request con 0x7fb1000a5210 auth_method 0 2024-09-15T21:19:53.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient(hunting): handle_auth_done global_id 24341 payload 824 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 -- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1000a5210 msgr2=0x7fb100136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1000a5210 0x7fb100136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 -- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1001378c0 msgr2=0x7fb10013ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1001378c0 0x7fb10013ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 monclient: found mon.noname-c 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 -- 172.21.15.17:0/154021984 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1001d15c0 con 0x7fb1001381e0 2024-09-15T21:19:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0ff7fe640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 0x7fb10013d370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c7c0 tx=0x7fb0f800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003b80 con 0x7fb1001381e0 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: dump: 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 5 monclient: authenticate success, global_id 24341 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8003d20 con 0x7fb1001381e0 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _renew_subs 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10013f5e0 con 0x7fb1001381e0 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800aa90 con 0x7fb1001381e0 2024-09-15T21:19:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 10 monclient: dump: 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _renew_subs 2024-09-15T21:19:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0fd7fa640 4 set_mon_vals no callback set 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb100140830 con 0x7fb1001381e0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100137ca0 con 0x7fb1001381e0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb0f800ac30 con 0x7fb1001381e0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0ec03b9f0 0x7fb0ec03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.510+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7fb0f804fb10 con 0x7fb1001381e0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.514+0000 7fb0feffd640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0ec03b9f0 0x7fb0ec03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.514+0000 7fb0feffd640 10 monclient: get_auth_request con 0x7fb0ec03b9f0 auth_method 0 2024-09-15T21:19:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.514+0000 7fb0feffd640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0ec03b9f0 0x7fb0ec03deb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00ae000 tx=0x7fb0f00a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.518+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb0f80052b0 con 0x7fb1001381e0 2024-09-15T21:19:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.518+0000 7fb0e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.518+0000 7fb0e77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:53.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.603+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 v59) ==== 110+0+0 (secure 0 0 0) 0x7f56fc016110 con 0x7f5700135bf0 2024-09-15T21:19:53.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.603+0000 7f56f5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:53.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.603+0000 7f56f5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.711+0000 7f57076a6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:53.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.711+0000 7f57076a6640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.711+0000 7f57076a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.711+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7f5700131f30 con 0x7f5700135bf0 2024-09-15T21:19:53.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f56f5ffb640 1 -- 172.21.15.119:0/1455428562 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v59) ==== 134+0+0 (secure 0 0 0) 0x7f56fc0c93e0 con 0x7f5700135bf0 2024-09-15T21:19:53.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f56f5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:53.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f56f5ffb640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-09-15T21:19:53.719 INFO:tasks.workunit.client.0.smithi119.stderr:already renamed to 'foo' 2024-09-15T21:19:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56e403b970 msgr2=0x7f56e403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56e403b970 0x7f56e403de30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56f00038c0 tx=0x7f56f0009110 comp rx=0 tx=0).stop 2024-09-15T21:19:53.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 10 monclient: shutdown 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 msgr2=0x7f570013bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570013bf00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0af040 tx=0x7f56fc01ac40 comp rx=0 tx=0).stop 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 shutdown_connections 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f56e403b970 0x7f56e403de30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5700136460 0x7f57001319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5700135fd0 0x7f5700139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 --2- 172.21.15.119:0/1455428562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5700135bf0 0x7f570013bf00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 >> 172.21.15.119:0/1455428562 conn(0x7f57000a66e0 msgr2=0x7f5700132a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 shutdown_connections 2024-09-15T21:19:53.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.715+0000 7f57076a6640 1 -- 172.21.15.119:0/1455428562 wait complete. 2024-09-15T21:19:53.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-09-15T21:19:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.750+0000 7fb10566f640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:19:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.750+0000 7fb10566f640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:19:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.750+0000 7fb10566f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:19:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.750+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fb100069270 con 0x7fb1001381e0 2024-09-15T21:19:53.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=12,kv=12,logm=64} crc {auth=2475377951,config=3541143443,health=2157093694,kv=3079713637,logm=1713258149}) 2024-09-15T21:19:53.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fb0e77fe640 1 -- 172.21.15.17:0/154021984 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fb0f8005080 con 0x7fb1001381e0 2024-09-15T21:19:53.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fb0e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:19:53.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fb0e77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:53.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=2475377951,config=3541143443,health=1886805112,kv=2758333334,logm=327290449}) 2024-09-15T21:19:53.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=12,kv=12,logm=64} crc {auth=2475377951,config=3541143443,health=2157093694,kv=3079713637,logm=1713258149}) 2024-09-15T21:19:53.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.754+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=2475377951,config=3541143443,health=1886805112,kv=3079713637,logm=3911305214}) 2024-09-15T21:19:53.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0ec03b9f0 msgr2=0x7fb0ec03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0ec03b9f0 0x7fb0ec03deb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00ae000 tx=0x7fb0f00a6070 comp rx=0 tx=0).stop 2024-09-15T21:19:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 10 monclient: shutdown 2024-09-15T21:19:53.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 msgr2=0x7fb10013d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 0x7fb10013d370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c7c0 tx=0x7fb0f800cc80 comp rx=0 tx=0).stop 2024-09-15T21:19:53.761 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3333678560}) 2024-09-15T21:19:53.761 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2725351769}) 2024-09-15T21:19:53.761 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.761 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3333678560}) 2024-09-15T21:19:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 shutdown_connections 2024-09-15T21:19:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0ec03b9f0 0x7fb0ec03deb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb1001381e0 0x7fb10013d370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1001378c0 0x7fb10013ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3028439536}) 2024-09-15T21:19:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 --2- 172.21.15.17:0/154021984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1000a5210 0x7fb100136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 >> 172.21.15.17:0/154021984 conn(0x7fb1000aac40 msgr2=0x7fb1000a8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 shutdown_connections 2024-09-15T21:19:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fb10566f640 1 -- 172.21.15.17:0/154021984 wait complete. 2024-09-15T21:19:53.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169997803}) 2024-09-15T21:19:53.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=173243011}) 2024-09-15T21:19:53.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169997803}) 2024-09-15T21:19:53.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.758+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1774888996}) 2024-09-15T21:19:53.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=463499970}) 2024-09-15T21:19:53.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3787229871}) 2024-09-15T21:19:53.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=463499970}) 2024-09-15T21:19:53.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1208787711}) 2024-09-15T21:19:53.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=115218365}) 2024-09-15T21:19:53.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1075100183}) 2024-09-15T21:19:53.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=115218365}) 2024-09-15T21:19:53.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2099542451}) 2024-09-15T21:19:53.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=824730227}) 2024-09-15T21:19:53.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3663954599}) 2024-09-15T21:19:53.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=824730227}) 2024-09-15T21:19:53.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.762+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1257169129}) 2024-09-15T21:19:53.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268522292}) 2024-09-15T21:19:53.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3050091393}) 2024-09-15T21:19:53.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268522292}) 2024-09-15T21:19:53.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3515104994}) 2024-09-15T21:19:53.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.771 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4212274849}) 2024-09-15T21:19:53.771 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1974192354}) 2024-09-15T21:19:53.771 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.771 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4212274849}) 2024-09-15T21:19:53.771 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.766+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3051859175}) 2024-09-15T21:19:53.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=422307930}) 2024-09-15T21:19:53.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2714469447}) 2024-09-15T21:19:53.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=422307930}) 2024-09-15T21:19:53.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1226472395}) 2024-09-15T21:19:53.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=14,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=29} crc {logm=102234693,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3026308494,mon_config_key=3144974445,monmap=186996059,nvmeofgw=1596038039,osd_metadata=2194678291,osd_pg_creating=2420761471,osd_snap=3058936622,osdmap=2221503521}) 2024-09-15T21:19:53.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=27} crc {logm=706752595,mdsmap=173658045,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1474188300,mon_config_key=2889471931,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=2420761471,osd_snap=841190635,osdmap=2307578850}) 2024-09-15T21:19:53.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=14,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=29} crc {logm=102234693,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3026308494,mon_config_key=3144974445,monmap=186996059,nvmeofgw=1596038039,osd_metadata=2194678291,osd_pg_creating=2420761471,osd_snap=3058936622,osdmap=2221503521}) 2024-09-15T21:19:53.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.770+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=22,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=27} crc {logm=2850457447,mdsmap=173658045,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2928586318,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=2420761471,osd_snap=841190635,osdmap=366506391}) 2024-09-15T21:19:53.777 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:19:53.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.778+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.778+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=75} crc {osdmap=2821522996}) 2024-09-15T21:19:53.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.778+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=74} crc {osdmap=1295988542}) 2024-09-15T21:19:53.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.778+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:19:53.781 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.778+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=75} crc {osdmap=2821522996}) 2024-09-15T21:19:53.781 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:19:53.778+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=70} crc {osdmap=2461117309}) 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient: get_monmap_and_config 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient: monmap: 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:53.858283+0000 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:53.858283+0000 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:53.860 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:53.861 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:53.861 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:53.861 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:53.861 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:53.861 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:53.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 Processor -- start 2024-09-15T21:19:53.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 -- start start 2024-09-15T21:19:53.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient: init 2024-09-15T21:19:53.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:19:53.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef80a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): picked mon.noname-b con 0x7f3ef80a0a50 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ef809ea80 0x7f3ef80a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): picked mon.noname-c con 0x7f3ef809ea80 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ef813c170 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef80741f0 con 0x7f3ef813c170 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef80a18b0 con 0x7f3ef809ea80 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef80a35a0 con 0x7f3ef80a0a50 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3eff6d6640 10 monclient(hunting): authenticate will time out at 1112.654910s 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ef809ea80 0x7f3ef80a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efdc73640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3ef813c170 0x7f3ef813c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33548/0 (socket says 172.21.15.119:33548) 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efdc73640 1 -- 172.21.15.119:0/3439611433 learned_addr learned my addr 172.21.15.119:0/3439611433 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): get_auth_request con 0x7f3ef809ea80 auth_method 0 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efd472640 1 --2- 172.21.15.119:0/3439611433 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef80a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efd472640 10 monclient(hunting): get_auth_request con 0x7f3ef80a0a50 auth_method 0 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efd472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efd472640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efd472640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient(hunting): handle_auth_done global_id 34111 payload 274 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 1 -- 172.21.15.119:0/3439611433 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 msgr2=0x7f3ef80a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 1 --2- 172.21.15.119:0/3439611433 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef80a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 1 -- 172.21.15.119:0/3439611433 >> v1:172.21.15.119:6789/0 conn(0x7f3ef813c170 legacy=0x7f3ef813c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 1 monclient: found mon.noname-c 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:53.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.855+0000 7f3efcc71640 1 -- 172.21.15.119:0/3439611433 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef809d800 con 0x7f3ef809ea80 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3efcc71640 1 --2- 172.21.15.119:0/3439611433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef80a1370 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0097f0 tx=0x7f3eec01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6636b01e07eff3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 1 -- 172.21.15.119:0/3439611433 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0b0070 con 0x7f3ef809ea80 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: dump: 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:53.871 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 5 monclient: authenticate success, global_id 34111 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 1 -- 172.21.15.119:0/3439611433 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec0ab600 con 0x7f3ef809ea80 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 4 set_mon_vals no callback set 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 1 -- 172.21.15.119:0/3439611433 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0ab9d0 con 0x7f3ef809ea80 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3ef7fff640 10 monclient: dump: 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.873 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 10 monclient: shutdown 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3439611433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 msgr2=0x7f3ef80a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3439611433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef80a1370 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0097f0 tx=0x7f3eec01c390 comp rx=0 tx=0).stop 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3439611433 shutdown_connections 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3439611433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef80a1370 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3439611433 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef80a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3439611433 >> 172.21.15.119:0/3439611433 conn(0x7f3ef80a66e0 msgr2=0x7f3ef80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3439611433 shutdown_connections 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3439611433 wait complete. 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 10 monclient: build_initial_monmap 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 10 monclient: monmap: 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:53.875 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 Processor -- start 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 1 -- start start 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.859+0000 7f3eff6d6640 10 monclient: init 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef81d4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): picked mon.noname-b con 0x7f3ef809ea80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef813b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): picked mon.noname-c con 0x7f3ef80a0a50 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef813c170 0x7f3ef81412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): picked mon.noname-a con 0x7f3ef813c170 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:53.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef809ccd0 con 0x7f3ef813c170 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef80a1c90 con 0x7f3ef809ea80 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef809ce50 con 0x7f3ef80a0a50 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient(hunting): authenticate will time out at 1112.660387s 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef81d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef81d4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38702/0 (socket says 172.21.15.119:38702) 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 -- 172.21.15.119:0/3701413802 learned_addr learned my addr 172.21.15.119:0/3701413802 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): get_auth_request con 0x7f3ef809ea80 auth_method 0 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef813b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efdc73640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef813c170 0x7f3ef81412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 10 monclient(hunting): get_auth_request con 0x7f3ef80a0a50 auth_method 0 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient(hunting): handle_auth_done global_id 34117 payload 824 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 -- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 msgr2=0x7f3ef813b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef813b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 -- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef813c170 msgr2=0x7f3ef81412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef813c170 0x7f3ef81412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient: _finish_hunting 0 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 monclient: found mon.noname-b 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 -- 172.21.15.119:0/3701413802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef81438a0 con 0x7f3ef809ea80 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efdc73640 10 monclient: get_auth_request con 0x7f3ef813c170 auth_method 0 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efdc73640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef813c170 0x7f3ef81412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efd472640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef81d4890 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d800 tx=0x7f3ef000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000b6c0 con 0x7f3ef809ea80 2024-09-15T21:19:53.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: dump: 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.880 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0019070 con 0x7f3ef809ea80 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 5 monclient: authenticate success, global_id 34117 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _renew_subs 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef8125b10 con 0x7f3ef809ea80 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000ba60 con 0x7f3ef809ea80 2024-09-15T21:19:53.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 10 monclient: dump: 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:53.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _renew_subs 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef8142d40 con 0x7f3ef809ea80 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef6ffd640 4 set_mon_vals no callback set 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3ef000bc00 con 0x7f3ef809ea80 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef813c600 con 0x7f3ef809ea80 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3ee403b9c0 0x7f3ee403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5222+0+0 (secure 0 0 0) 0x7f3ef00501a0 con 0x7f3ef809ea80 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3ee403b9c0 0x7f3ee403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:53.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 10 monclient: get_auth_request con 0x7f3ee403b9c0 auth_method 0 2024-09-15T21:19:53.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.863+0000 7f3efcc71640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3ee403b9c0 0x7f3ee403de80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0ad000 tx=0x7f3eec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:53.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.871+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3ef81438a0 con 0x7f3ef809ea80 2024-09-15T21:19:53.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.871+0000 7f3ef5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:53.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:53.871+0000 7f3ef5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:54.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3eff6d6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:54.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3eff6d6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:54.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3eff6d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7f3ef8125390 con 0x7f3ef809ea80 2024-09-15T21:19:54.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v59) ==== 134+0+0 (secure 0 0 0) 0x7f3ef001cd70 con 0x7f3ef809ea80 2024-09-15T21:19:54.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3ef5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:54.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.039+0000 7f3ef5ffb640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo' 2024-09-15T21:19:54.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3eff6d6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:54.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3eff6d6640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:54.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3eff6d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7f3ef8125790 con 0x7f3ef809ea80 2024-09-15T21:19:54.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3ef5ffb640 1 -- 172.21.15.119:0/3701413802 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v59) ==== 134+0+0 (secure 0 0 0) 0x7f3ef0013390 con 0x7f3ef809ea80 2024-09-15T21:19:54.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3ef5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T21:19:54.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.135+0000 7f3ef5ffb640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-09-15T21:19:54.141 INFO:tasks.workunit.client.0.smithi119.stderr:already renamed to 'foo' 2024-09-15T21:19:54.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3ee403b9c0 msgr2=0x7f3ee403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3ee403b9c0 0x7f3ee403de80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0ad000 tx=0x7f3eec008040 comp rx=0 tx=0).stop 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 10 monclient: shutdown 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 msgr2=0x7f3ef81d4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef81d4890 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d800 tx=0x7f3ef000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 shutdown_connections 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3ee403b9c0 0x7f3ee403de80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef813c170 0x7f3ef81412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ef80a0a50 0x7f3ef813b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 --2- 172.21.15.119:0/3701413802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef809ea80 0x7f3ef81d4890 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 >> 172.21.15.119:0/3701413802 conn(0x7f3ef80a66e0 msgr2=0x7f3ef80a4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 shutdown_connections 2024-09-15T21:19:54.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.139+0000 7f3eff6d6640 1 -- 172.21.15.119:0/3701413802 wait complete. 2024-09-15T21:19:54.156 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rename bar-asdf bar 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient: get_monmap_and_config 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient: build_initial_monmap 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient: monmap: 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:54.273392+0000 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:54.273392+0000 2024-09-15T21:19:54.275 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:54.276 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 1 Processor -- start 2024-09-15T21:19:54.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 1 -- start start 2024-09-15T21:19:54.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient: init 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f453009d550 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f453009fae0 0x7f4530139c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient(hunting): picked mon.noname-a con 0x7f453009fae0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f453009e040 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45300a3600 con 0x7f453009d550 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45300a1d60 con 0x7f453009e040 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f453009f1b0 con 0x7f453009fae0 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4534f39640 10 monclient(hunting): authenticate will time out at 1113.070599s 2024-09-15T21:19:54.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f453009e040 0x7f453013c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33554/0 (socket says 172.21.15.119:33554) 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 1 -- 172.21.15.119:0/646939077 learned_addr learned my addr 172.21.15.119:0/646939077 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 1 --2- 172.21.15.119:0/646939077 >> v2:172.21.15.17:3300/0 conn(0x7f453009fae0 0x7f4530139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): get_auth_request con 0x7f453009fae0 auth_method 0 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050270836 0 0) 0x7f45300a1d60 con 0x7f453009e040 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): _negotiate 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): my global_id is 34120 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f451c005740 con 0x7f453009e040 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2036427619 0 0) 0x7f451c005740 con 0x7f453009e040 2024-09-15T21:19:54.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 >> v2:172.21.15.17:3300/0 conn(0x7f453009fae0 msgr2=0x7f4530139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 --2- 172.21.15.119:0/646939077 >> v2:172.21.15.17:3300/0 conn(0x7f453009fae0 0x7f4530139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 >> v1:172.21.15.17:6789/0 conn(0x7f453009d550 legacy=0x7f453009d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453009e880 con 0x7f453009e040 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f4528001a80 con 0x7f453009e040 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: dump: 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.285 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 >> v1:172.21.15.119:6789/0 conn(0x7f453009e040 legacy=0x7f453013c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f451c005c70 0x7f451c006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): picked mon.c con 0x7f451c005c70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f451c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): picked mon.b con 0x7f451c006a20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f451c0075e0 0x7f451c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): picked mon.a con 0x7f451c0075e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f453009f1b0 con 0x7f451c0075e0 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00e450 con 0x7f451c006a20 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f451c00e650 con 0x7f451c005c70 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452dd9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f451c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): get_auth_request con 0x7f451c006a20 auth_method 0 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f451c0075e0 0x7f451c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 10 monclient(hunting): get_auth_request con 0x7f451c0075e0 auth_method 0 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f452e59c640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f451c005c70 0x7f451c006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 10 monclient(hunting): handle_auth_done global_id 34120 payload 293 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 1 -- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f451c005c70 msgr2=0x7f451c006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.271+0000 7f4527fff640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f451c005c70 0x7f451c006090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 1 -- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f451c0075e0 msgr2=0x7f451c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f451c0075e0 0x7f451c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 1 monclient: found mon.b 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 1 -- 172.21.15.119:0/646939077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c005740 con 0x7f451c006a20 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452ed9d640 1 -- 172.21.15.119:0/646939077 reap_dead start 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 5 monclient: authenticate success, global_id 34120 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452e59c640 10 monclient: get_auth_request con 0x7f451c005c70 auth_method 0 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452e59c640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f451c005c70 0x7f451c006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4527fff640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f451c006ea0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ae0 tx=0x7f451801ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf195156722deec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45180b2070 con 0x7f451c006a20 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: dump: 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.290 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45180a5530 con 0x7f451c006a20 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:54.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 1 -- 172.21.15.119:0/646939077 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45180096b0 con 0x7f451c006a20 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 4 set_mon_vals no callback set 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f452dd9b640 10 monclient: dump: 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.292 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 10 monclient: shutdown 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 -- 172.21.15.119:0/646939077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 msgr2=0x7f451c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f451c006ea0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ae0 tx=0x7f451801ca50 comp rx=0 tx=0).stop 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 -- 172.21.15.119:0/646939077 shutdown_connections 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 --2- 172.21.15.119:0/646939077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f451c006ea0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 -- 172.21.15.119:0/646939077 >> 172.21.15.119:0/646939077 conn(0x7f45300a66e0 msgr2=0x7f45300a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 -- 172.21.15.119:0/646939077 shutdown_connections 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 1 -- 172.21.15.119:0/646939077 wait complete. 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 10 monclient: build_initial_monmap 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.275+0000 7f4534f39640 10 monclient: monmap: 2024-09-15T21:19:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:54.294 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 Processor -- start 2024-09-15T21:19:54.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- start start 2024-09-15T21:19:54.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: init 2024-09-15T21:19:54.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.298 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f4530121000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): picked mon.noname-b con 0x7f451c006a20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4530121540 0x7f45301219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): picked mon.noname-c con 0x7f4530121540 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45301cba60 0x7f45301cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): picked mon.noname-a con 0x7f45301cba60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4530135030 con 0x7f45301cba60 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45300a2920 con 0x7f451c006a20 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45301351b0 con 0x7f4530121540 2024-09-15T21:19:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient(hunting): authenticate will time out at 1113.076669s 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f4530121000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f4530121000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38716/0 (socket says 172.21.15.119:38716) 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 -- 172.21.15.119:0/4212368822 learned_addr learned my addr 172.21.15.119:0/4212368822 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): get_auth_request con 0x7f451c006a20 auth_method 0 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4530121540 0x7f45301219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452ed9d640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45301cba60 0x7f45301cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 10 monclient(hunting): get_auth_request con 0x7f4530121540 auth_method 0 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452ed9d640 10 monclient(hunting): get_auth_request con 0x7f45301cba60 auth_method 0 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient(hunting): handle_auth_done global_id 34123 payload 824 2024-09-15T21:19:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 -- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4530121540 msgr2=0x7f45301219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4530121540 0x7f45301219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 -- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45301cba60 msgr2=0x7f45301cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45301cba60 0x7f45301cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 monclient: found mon.noname-b 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 -- 172.21.15.119:0/4212368822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4530123e90 con 0x7f451c006a20 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452e59c640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f4530121000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4520001ec0 tx=0x7f452000d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520018070 con 0x7f451c006a20 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:54.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: dump: 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 5 monclient: authenticate success, global_id 34123 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4520005480 con 0x7f451c006a20 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _renew_subs 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4530124d00 con 0x7f451c006a20 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520003840 con 0x7f451c006a20 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 10 monclient: dump: 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.304 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f452d59a640 4 set_mon_vals no callback set 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _renew_subs 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453009d240 con 0x7f451c006a20 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45301354b0 con 0x7f451c006a20 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f452001b070 con 0x7f451c006a20 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451003b9c0 0x7f451003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5222+0+0 (secure 0 0 0) 0x7f452004f3a0 con 0x7f451c006a20 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451003b9c0 0x7f451003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 10 monclient: get_auth_request con 0x7f451003b9c0 auth_method 0 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.279+0000 7f4527fff640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451003b9c0 0x7f451003de80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45180af000 tx=0x7f4518008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.283+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f45200039e0 con 0x7f451c006a20 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.283+0000 7f4526ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.283+0000 7f4526ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.459+0000 7f4534f39640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T21:19:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.459+0000 7f4534f39640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T21:19:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.459+0000 7f4534f39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.459+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) -- 0x7f4530125120 con 0x7f451c006a20 2024-09-15T21:19:54.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.611+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 v60) ==== 110+0+0 (secure 0 0 0) 0x7f452004f080 con 0x7f451c006a20 2024-09-15T21:19:54.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.611+0000 7f4526ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T21:19:54.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.611+0000 7f4526ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:54.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.711+0000 7f4534f39640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T21:19:54.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.711+0000 7f4534f39640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T21:19:54.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.711+0000 7f4534f39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.711+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) -- 0x7f45301cc420 con 0x7f451c006a20 2024-09-15T21:19:54.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4526ffd640 1 -- 172.21.15.119:0/4212368822 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 already renamed to 'bar' v60) ==== 134+0+0 (secure 0 0 0) 0x7f4520013470 con 0x7f451c006a20 2024-09-15T21:19:54.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4526ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T21:19:54.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4526ffd640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar' 2024-09-15T21:19:54.719 INFO:tasks.workunit.client.0.smithi119.stderr:already renamed to 'bar' 2024-09-15T21:19:54.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451003b9c0 msgr2=0x7f451003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451003b9c0 0x7f451003de80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45180af000 tx=0x7f4518008040 comp rx=0 tx=0).stop 2024-09-15T21:19:54.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 10 monclient: shutdown 2024-09-15T21:19:54.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:54.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 msgr2=0x7f4530121000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f4530121000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4520001ec0 tx=0x7f452000d9d0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 shutdown_connections 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f451003b9c0 0x7f451003de80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45301cba60 0x7f45301cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4530121540 0x7f45301219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 --2- 172.21.15.119:0/4212368822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f451c006a20 0x7f4530121000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 >> 172.21.15.119:0/4212368822 conn(0x7f45300a66e0 msgr2=0x7f45300715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 shutdown_connections 2024-09-15T21:19:54.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.715+0000 7f4534f39640 1 -- 172.21.15.119:0/4212368822 wait complete. 2024-09-15T21:19:54.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls 2024-09-15T21:19:54.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false grep foo-asdf 2024-09-15T21:19:54.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:54.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo-asdf 2024-09-15T21:19:54.778 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:19:54.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient: get_monmap_and_config 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient: build_initial_monmap 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient: monmap: 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:54.850549+0000 2024-09-15T21:19:54.852 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:54.850549+0000 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:54.853 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 Processor -- start 2024-09-15T21:19:54.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 -- start start 2024-09-15T21:19:54.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient: init 2024-09-15T21:19:54.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 0x7febd8136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): picked mon.noname-b con 0x7febd8135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febd80a0160 0x7febd80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): picked mon.noname-a con 0x7febd80a0160 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7febd806ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd81365c0 con 0x7febd806ec30 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7febd809cf30 con 0x7febd80a0160 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febd80a1ca0 con 0x7febd8135ca0 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.847+0000 7febde561640 10 monclient(hunting): authenticate will time out at 1113.647809s 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7febd806ec30 0x7febd80a0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33558/0 (socket says 172.21.15.119:33558) 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 1 -- 172.21.15.119:0/3618069815 learned_addr learned my addr 172.21.15.119:0/3618069815 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 1 --2- 172.21.15.119:0/3618069815 >> v2:172.21.15.17:3300/0 conn(0x7febd80a0160 0x7febd80a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641309236 0 0) 0x7febd81365c0 con 0x7febd806ec30 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): my global_id is 34126 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 0x7febd8136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc0003530 con 0x7febd806ec30 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): get_auth_request con 0x7febd80a0160 auth_method 0 2024-09-15T21:19:54.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): get_auth_request con 0x7febd8135ca0 auth_method 0 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2349289450 0 0) 0x7febc0003530 con 0x7febd806ec30 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 msgr2=0x7febd8136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 0x7febd8136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 >> v2:172.21.15.17:3300/0 conn(0x7febd80a0160 msgr2=0x7febd80a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 --2- 172.21.15.119:0/3618069815 >> v2:172.21.15.17:3300/0 conn(0x7febd80a0160 0x7febd80a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd80a1f80 con 0x7febd806ec30 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7febd0001a80 con 0x7febd806ec30 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: dump: 2024-09-15T21:19:54.866 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 >> v1:172.21.15.119:6789/0 conn(0x7febd806ec30 legacy=0x7febd80a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc0003a10 0x7febc0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): picked mon.b con 0x7febc0003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 0x7febc0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): picked mon.c con 0x7febd8135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc0004d50 0x7febc00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): picked mon.a con 0x7febc0004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7febd809cf30 con 0x7febc0004d50 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febd80a1ca0 con 0x7febc0003a10 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febc000bbb0 con 0x7febd8135ca0 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc0003a10 0x7febc0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): get_auth_request con 0x7febc0003a10 auth_method 0 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc0004d50 0x7febc00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 10 monclient(hunting): get_auth_request con 0x7febc0004d50 auth_method 0 2024-09-15T21:19:54.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febdcafe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 0x7febc0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): get_auth_request con 0x7febd8135ca0 auth_method 0 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient(hunting): handle_auth_done global_id 34126 payload 293 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 -- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 msgr2=0x7febc0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febd8135ca0 0x7febc0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 -- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc0004d50 msgr2=0x7febc00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc0004d50 0x7febc00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 monclient: found mon.b 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 -- 172.21.15.119:0/3618069815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc0003530 con 0x7febc0003a10 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febde561640 5 monclient: authenticate success, global_id 34126 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febde561640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd7fff640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc0003a10 0x7febc0003e30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7febc800ba30 tx=0x7febc801c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75b9e51c28230ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80b3070 con 0x7febc0003a10 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: dump: 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febde561640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:54.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc80ae410 con 0x7febc0003a10 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febde561640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febde561640 4 set_mon_vals no callback set 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 1 -- 172.21.15.119:0/3618069815 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc800b600 con 0x7febc0003a10 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.851+0000 7febd6ffd640 10 monclient: dump: 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.873 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: shutdown 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/3618069815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc0003a10 msgr2=0x7febc0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 --2- 172.21.15.119:0/3618069815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febc0003a10 0x7febc0003e30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7febc800ba30 tx=0x7febc801c680 comp rx=0 tx=0).stop 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 1 -- 172.21.15.119:0/3618069815 reap_dead start 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/3618069815 shutdown_connections 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/3618069815 >> 172.21.15.119:0/3618069815 conn(0x7febd80a2ae0 msgr2=0x7febd80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/3618069815 shutdown_connections 2024-09-15T21:19:54.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/3618069815 wait complete. 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: build_initial_monmap 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: monmap: 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 Processor -- start 2024-09-15T21:19:54.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- start start 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: init 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febc0003a10 0x7febd81329e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): picked mon.noname-c con 0x7febc0003a10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 0x7febd811f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): picked mon.noname-b con 0x7febd8121990 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febd8132f20 0x7febd811f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): picked mon.noname-a con 0x7febd8132f20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7febd809dcf0 con 0x7febd8132f20 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febd80a1840 con 0x7febd8121990 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7febd809de70 con 0x7febc0003a10 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient(hunting): authenticate will time out at 1113.653814s 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 0x7febd811f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 0x7febd811f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38740/0 (socket says 172.21.15.119:38740) 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 -- 172.21.15.119:0/4242441402 learned_addr learned my addr 172.21.15.119:0/4242441402 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): get_auth_request con 0x7febd8121990 auth_method 0 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febd8132f20 0x7febd811f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febc0003a10 0x7febd81329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): get_auth_request con 0x7febd8132f20 auth_method 0 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): get_auth_request con 0x7febc0003a10 auth_method 0 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febdcafe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient(hunting): handle_auth_done global_id 34129 payload 824 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 -- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febc0003a10 msgr2=0x7febd81329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febc0003a10 0x7febd81329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 -- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febd8132f20 msgr2=0x7febd811f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febd8132f20 0x7febd811f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 -- 172.21.15.119:0/4242441402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8122eb0 con 0x7febd8121990 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd77fe640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 0x7febd811f370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7febcc011660 tx=0x7febcc011a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc015870 con 0x7febd8121990 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: dump: 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.880 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 5 monclient: authenticate success, global_id 34129 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc015a10 con 0x7febd8121990 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc015dc0 con 0x7febd8121990 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 10 monclient: dump: 2024-09-15T21:19:54.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _renew_subs 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd67fc640 4 set_mon_vals no callback set 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd8123190 con 0x7febd8121990 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _renew_subs 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd8120e30 con 0x7febd8121990 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7febcc00f480 con 0x7febd8121990 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8136490 con 0x7febd8121990 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febc403b9c0 0x7febc403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.855+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7febcc050000 con 0x7febd8121990 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.859+0000 7febd7fff640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febc403b9c0 0x7febc403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.859+0000 7febd7fff640 10 monclient: get_auth_request con 0x7febc403b9c0 auth_method 0 2024-09-15T21:19:54.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.859+0000 7febd7fff640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febc403b9c0 0x7febc403de80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febc800ba30 tx=0x7febc80046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.863+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7febcc019030 con 0x7febd8121990 2024-09-15T21:19:54.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.863+0000 7febd4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:54.863+0000 7febd4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient: get_monmap_and_config 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient: build_initial_monmap 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient: monmap: 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:54.907968+0000 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:54.907968+0000 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:54.911 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 Processor -- start 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 -- start start 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient: init 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f670c0a4a90 0x7f670c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): picked mon.noname-c con 0x7f670c0a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 0x7f670c0a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): picked mon.noname-b con 0x7f670c0a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f670c137850 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670c0a2c30 con 0x7f670c137850 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f670c0a0f00 con 0x7f670c0a4a90 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f670c0a19b0 con 0x7f670c0a53b0 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6710aee640 10 monclient(hunting): authenticate will time out at 1118.303739s 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 0x7f670c0a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 0x7f670c0a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59412/0 (socket says 172.21.15.17:59412) 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 1 -- 172.21.15.17:0/1942098659 learned_addr learned my addr 172.21.15.17:0/1942098659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 10 monclient(hunting): get_auth_request con 0x7f670c0a53b0 auth_method 0 2024-09-15T21:19:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.906+0000 7f6709d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670a59c640 1 --2- 172.21.15.17:0/1942098659 >> v2:172.21.15.119:3300/0 conn(0x7f670c0a4a90 0x7f670c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670a59c640 10 monclient(hunting): get_auth_request con 0x7f670c0a4a90 auth_method 0 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 10 monclient(hunting): handle_auth_done global_id 24359 payload 274 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 1 -- 172.21.15.17:0/1942098659 >> v2:172.21.15.119:3300/0 conn(0x7f670c0a4a90 msgr2=0x7f670c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 1 --2- 172.21.15.17:0/1942098659 >> v2:172.21.15.119:3300/0 conn(0x7f670c0a4a90 0x7f670c0a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 1 -- 172.21.15.17:0/1942098659 >> v1:172.21.15.119:6789/0 conn(0x7f670c137850 legacy=0x7f670c137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 1 monclient: found mon.noname-b 2024-09-15T21:19:54.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:54.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 1 -- 172.21.15.17:0/1942098659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c0a1c90 con 0x7f670c0a53b0 2024-09-15T21:19:54.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6709d9b640 1 --2- 172.21.15.17:0/1942098659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 0x7f670c0a57d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009e50 tx=0x7f66f401c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96771178ec5e5ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 1 -- 172.21.15.17:0/1942098659 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40a56a0 con 0x7f670c0a53b0 2024-09-15T21:19:54.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: dump: 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:54.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 1 -- 172.21.15.17:0/1942098659 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f40a5840 con 0x7f670c0a53b0 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 5 monclient: authenticate success, global_id 24359 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 4 set_mon_vals no callback set 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 1 -- 172.21.15.17:0/1942098659 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40a5c10 con 0x7f670c0a53b0 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:54.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f670959a640 10 monclient: dump: 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 10 monclient: shutdown 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 -- 172.21.15.17:0/1942098659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 msgr2=0x7f670c0a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 --2- 172.21.15.17:0/1942098659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 0x7f670c0a57d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009e50 tx=0x7f66f401c8c0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 -- 172.21.15.17:0/1942098659 shutdown_connections 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 --2- 172.21.15.17:0/1942098659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c0a53b0 0x7f670c0a57d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 --2- 172.21.15.17:0/1942098659 >> v2:172.21.15.119:3300/0 conn(0x7f670c0a4a90 0x7f670c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 -- 172.21.15.17:0/1942098659 >> 172.21.15.17:0/1942098659 conn(0x7f670c0aabb0 msgr2=0x7f670c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 -- 172.21.15.17:0/1942098659 shutdown_connections 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 1 -- 172.21.15.17:0/1942098659 wait complete. 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 10 monclient: build_initial_monmap 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.910+0000 7f6710aee640 10 monclient: monmap: 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 Processor -- start 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- start start 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: init 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f670c0a4a90 0x7f670c136690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): picked mon.noname-b con 0x7f670c0a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 0x7f670c136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): picked mon.noname-a con 0x7f670c0a53b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c137850 0x7f670c13cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): picked mon.noname-c con 0x7f670c137850 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1426d0 con 0x7f670c0a53b0 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f670c142550 con 0x7f670c0a4a90 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f670c142850 con 0x7f670c137850 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient(hunting): authenticate will time out at 1118.308813s 2024-09-15T21:19:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 0x7f670c136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 0x7f670c136bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47528/0 (socket says 172.21.15.17:47528) 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 -- 172.21.15.17:0/1458140778 learned_addr learned my addr 172.21.15.17:0/1458140778 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): get_auth_request con 0x7f670c0a53b0 auth_method 0 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f670c0a4a90 0x7f670c136690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c137850 0x7f670c13cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): get_auth_request con 0x7f670c0a4a90 auth_method 0 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): get_auth_request con 0x7f670c137850 auth_method 0 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:54.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient(hunting): handle_auth_done global_id 5007 payload 824 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 -- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c137850 msgr2=0x7f670c13cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c137850 0x7f670c13cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 -- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f670c0a4a90 msgr2=0x7f670c136690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f670c0a4a90 0x7f670c136690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 monclient: found mon.noname-a 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 -- 172.21.15.17:0/1458140778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c120f20 con 0x7f670c0a53b0 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6709d9b640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 0x7f670c136bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f66f40af040 tx=0x7f66f4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40bc020 con 0x7f670c0a53b0 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: dump: 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f40b8690 con 0x7f670c0a53b0 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 5 monclient: authenticate success, global_id 5007 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _renew_subs 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c121200 con 0x7f670c0a53b0 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40b8a80 con 0x7f670c0a53b0 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 10 monclient: dump: 2024-09-15T21:19:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _renew_subs 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6708d99640 4 set_mon_vals no callback set 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c0a8a60 con 0x7f670c0a53b0 2024-09-15T21:19:54.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c0a4e70 con 0x7f670c0a53b0 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f66f40bf020 con 0x7f670c0a53b0 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f66ec03b9f0 0x7f66ec03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7f66f40eabf0 con 0x7f670c0a53b0 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f66ec03b9f0 0x7f66ec03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.914+0000 7f670a59c640 10 monclient: get_auth_request con 0x7f66ec03b9f0 auth_method 0 2024-09-15T21:19:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.918+0000 7f670a59c640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f66ec03b9f0 0x7f66ec03deb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66fc007a60 tx=0x7f66fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.918+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f66f40c6030 con 0x7f670c0a53b0 2024-09-15T21:19:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.918+0000 7f6702ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:54.918+0000 7f6702ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.019+0000 7febde561640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.019+0000 7febde561640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.019+0000 7febde561640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.019+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7febd809f7f0 con 0x7febd8121990 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.023+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) ==== 65+0+60 (secure 0 0 0) 0x7febcc013c60 con 0x7febd8121990 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.023+0000 7febd4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.023+0000 7febd4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:55.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febde561640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febde561640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febde561640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7febd81228f0 con 0x7febd8121990 2024-09-15T21:19:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febd4ff9640 1 -- 172.21.15.119:0/4242441402 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) ==== 65+0+60 (secure 0 0 0) 0x7febcc0143e0 con 0x7febd8121990 2024-09-15T21:19:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febd4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.123+0000 7febd4ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febc403b9c0 msgr2=0x7febc403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febc403b9c0 0x7febc403de80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febc800ba30 tx=0x7febc80046d0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 10 monclient: shutdown 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 msgr2=0x7febd811f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 0x7febd811f370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7febcc011660 tx=0x7febcc011a60 comp rx=0 tx=0).stop 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 shutdown_connections 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7febc403b9c0 0x7febc403de80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febd8132f20 0x7febd811f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febd8121990 0x7febd811f370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 --2- 172.21.15.119:0/4242441402 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7febc0003a10 0x7febd81329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 >> 172.21.15.119:0/4242441402 conn(0x7febd80a2ae0 msgr2=0x7febd80a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 shutdown_connections 2024-09-15T21:19:55.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.127+0000 7febde561640 1 -- 172.21.15.119:0/4242441402 wait complete. 2024-09-15T21:19:55.146 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:55.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule ls 2024-09-15T21:19:55.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false grep bar-asdf 2024-09-15T21:19:55.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:19:55.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep bar-asdf 2024-09-15T21:19:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6710aee640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6710aee640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6710aee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f670c13edd0 con 0x7f670c0a53b0 2024-09-15T21:19:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6702ffd640 1 -- 172.21.15.17:0/1458140778 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f66f4016600 con 0x7f670c0a53b0 2024-09-15T21:19:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6702ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.154+0000 7f6702ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:55.159 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f66ec03b9f0 msgr2=0x7f66ec03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f66ec03b9f0 0x7f66ec03deb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66fc007a60 tx=0x7f66fc008040 comp rx=0 tx=0).stop 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 10 monclient: shutdown 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 msgr2=0x7f670c136bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 0x7f670c136bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f66f40af040 tx=0x7f66f4019660 comp rx=0 tx=0).stop 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 shutdown_connections 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f66ec03b9f0 0x7f66ec03deb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f670c137850 0x7f670c13cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f670c0a53b0 0x7f670c136bd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 --2- 172.21.15.17:0/1458140778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f670c0a4a90 0x7f670c136690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 >> 172.21.15.17:0/1458140778 conn(0x7f670c0aabb0 msgr2=0x7f670c09da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 shutdown_connections 2024-09-15T21:19:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.158+0000 7f6710aee640 1 -- 172.21.15.17:0/1458140778 wait complete. 2024-09-15T21:19:55.178 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:19:55.178 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:19:55.178 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:19:55.178 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:19:55.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient: get_monmap_and_config 2024-09-15T21:19:55.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient: monmap: 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:55.250123+0000 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:55.250123+0000 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:55.252 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:55.253 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:55.253 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 1 Processor -- start 2024-09-15T21:19:55.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 1 -- start start 2024-09-15T21:19:55.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient: init 2024-09-15T21:19:55.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9aa80735c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9aa80739a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9aa813e240 0x7f9aa8140630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient(hunting): picked mon.noname-c con 0x7f9aa813e240 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa8069890 con 0x7f9aa80739a0 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa809ff90 con 0x7f9aa80735c0 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa806f660 con 0x7f9aa813e240 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aaea2b640 10 monclient(hunting): authenticate will time out at 1114.046888s 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa7fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9aa80735c0 0x7f9aa8137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33560/0 (socket says 172.21.15.119:33560) 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa7fff640 1 -- 172.21.15.119:0/4009266524 learned_addr learned my addr 172.21.15.119:0/4009266524 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:55.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 1 --2- 172.21.15.119:0/4009266524 >> v2:172.21.15.119:3300/0 conn(0x7f9aa813e240 0x7f9aa8140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): get_auth_request con 0x7f9aa813e240 auth_method 0 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 1 -- 172.21.15.119:0/4009266524 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16084959 0 0) 0x7f9aa809ff90 con 0x7f9aa80735c0 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 10 monclient(hunting): my global_id is 34141 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 1 -- 172.21.15.119:0/4009266524 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a90003530 con 0x7f9aa80735c0 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient(hunting): handle_auth_done global_id 34144 payload 274 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 1 -- 172.21.15.119:0/4009266524 >> v1:172.21.15.119:6789/0 conn(0x7f9aa80735c0 legacy=0x7f9aa8137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 1 -- 172.21.15.119:0/4009266524 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 118093343 0 0) 0x7f9a90003530 con 0x7f9aa80735c0 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 1 -- 172.21.15.119:0/4009266524 >> v1:172.21.15.17:6789/0 conn(0x7f9aa80739a0 legacy=0x7f9aa813cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 1 monclient: found mon.noname-c 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aacfc8640 1 -- 172.21.15.119:0/4009266524 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa806f940 con 0x7f9aa813e240 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.247+0000 7f9aa6ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aacfc8640 1 --2- 172.21.15.119:0/4009266524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa813e240 0x7f9aa8140630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009740 tx=0x7f9a9801be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e713083508b73d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 1 -- 172.21.15.119:0/4009266524 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980af070 con 0x7f9aa813e240 2024-09-15T21:19:55.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: dump: 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.265 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 1 -- 172.21.15.119:0/4009266524 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a980aa400 con 0x7f9aa813e240 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 5 monclient: authenticate success, global_id 34144 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 1 -- 172.21.15.119:0/4009266524 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0001a80 con 0x7f9aa813e240 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 4 set_mon_vals no callback set 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aa6ffd640 10 monclient: dump: 2024-09-15T21:19:55.266 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: shutdown 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- 172.21.15.119:0/4009266524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa813e240 msgr2=0x7f9aa8140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/4009266524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa813e240 0x7f9aa8140630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009740 tx=0x7f9a9801be20 comp rx=0 tx=0).stop 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- 172.21.15.119:0/4009266524 shutdown_connections 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/4009266524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa813e240 0x7f9aa8140630 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- 172.21.15.119:0/4009266524 >> 172.21.15.119:0/4009266524 conn(0x7f9aa806cbf0 msgr2=0x7f9aa806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- 172.21.15.119:0/4009266524 shutdown_connections 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- 172.21.15.119:0/4009266524 wait complete. 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: monmap: 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.268 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 Processor -- start 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- start start 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: init 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9aa80735c0 0x7f9aa81d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): picked mon.noname-a con 0x7f9aa80735c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 0x7f9aa81d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): picked mon.noname-b con 0x7f9aa80739a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9aa813e240 0x7f9aa81337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): picked mon.noname-c con 0x7f9aa813e240 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa8143d30 con 0x7f9aa80735c0 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa8143bb0 con 0x7f9aa80739a0 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa8143eb0 con 0x7f9aa813e240 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.251+0000 7f9aaea2b640 10 monclient(hunting): authenticate will time out at 1114.051902s 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 0x7f9aa81d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 0x7f9aa81d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38752/0 (socket says 172.21.15.119:38752) 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 -- 172.21.15.119:0/641455703 learned_addr learned my addr 172.21.15.119:0/641455703 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:55.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): get_auth_request con 0x7f9aa80739a0 auth_method 0 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aacfc8640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9aa813e240 0x7f9aa81337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9aa80735c0 0x7f9aa81d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aacfc8640 10 monclient(hunting): get_auth_request con 0x7f9aa813e240 auth_method 0 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aacfc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aacfc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aacfc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 10 monclient(hunting): get_auth_request con 0x7f9aa80735c0 auth_method 0 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient(hunting): handle_auth_done global_id 34147 payload 824 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 -- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9aa813e240 msgr2=0x7f9aa81337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9aa813e240 0x7f9aa81337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 -- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9aa80735c0 msgr2=0x7f9aa81d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9aa80735c0 0x7f9aa81d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 monclient: found mon.noname-b 2024-09-15T21:19:55.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 -- 172.21.15.119:0/641455703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa813a1b0 con 0x7f9aa80739a0 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa77fe640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 0x7f9aa81d07a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c760 tx=0x7f9a9c00cb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c017070 con 0x7f9aa80739a0 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: dump: 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.273 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a9c005270 con 0x7f9aa80739a0 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 5 monclient: authenticate success, global_id 34147 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 monclient: _renew_subs 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa813de50 con 0x7f9aa80739a0 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c003600 con 0x7f9aa80739a0 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 10 monclient: dump: 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.275 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 monclient: _renew_subs 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa81d0ce0 con 0x7f9aa80739a0 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa67fc640 4 set_mon_vals no callback set 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa80746f0 con 0x7f9aa80739a0 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9a9c01a070 con 0x7f9aa80739a0 2024-09-15T21:19:55.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a9403b9c0 0x7f9a9403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7f9a9c04f180 con 0x7f9aa80739a0 2024-09-15T21:19:55.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a9403b9c0 0x7f9a9403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 10 monclient: get_auth_request con 0x7f9a9403b9c0 auth_method 0 2024-09-15T21:19:55.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.255+0000 7f9aa7fff640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a9403b9c0 0x7f9a9403de80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0001750 tx=0x7f9aa0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.259+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9a9c011d40 con 0x7f9aa80739a0 2024-09-15T21:19:55.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.259+0000 7f9aa4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.259+0000 7f9aa4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient: get_monmap_and_config 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient: monmap: 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:55.295132+0000 2024-09-15T21:19:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:55.295132+0000 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:55.298 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 Processor -- start 2024-09-15T21:19:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 -- start start 2024-09-15T21:19:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient: init 2024-09-15T21:19:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2024-09-15T21:19:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe70073640 0x7fbe70073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): picked mon.noname-a con 0x7fbe70073640 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 0x7fbe70137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): picked mon.noname-b con 0x7fbe70073f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbe70137bd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe700a0890 con 0x7fbe70137bd0 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbe700746e0 con 0x7fbe70073640 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbe700a1640 con 0x7fbe70073f60 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe75dff640 10 monclient(hunting): authenticate will time out at 1118.690569s 2024-09-15T21:19:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 0x7fbe70137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 0x7fbe70137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59432/0 (socket says 172.21.15.17:59432) 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe70073640 0x7fbe70073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 -- 172.21.15.17:0/2906142169 learned_addr learned my addr 172.21.15.17:0/2906142169 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): get_auth_request con 0x7fbe70073f60 auth_method 0 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): get_auth_request con 0x7fbe70073640 auth_method 0 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient(hunting): handle_auth_done global_id 24368 payload 274 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 -- 172.21.15.17:0/2906142169 >> v2:172.21.15.17:3300/0 conn(0x7fbe70073640 msgr2=0x7fbe70073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 --2- 172.21.15.17:0/2906142169 >> v2:172.21.15.17:3300/0 conn(0x7fbe70073640 0x7fbe70073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 -- 172.21.15.17:0/2906142169 >> v1:172.21.15.17:6789/0 conn(0x7fbe70137bd0 legacy=0x7fbe7013ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 monclient: found mon.noname-b 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6effd640 1 -- 172.21.15.17:0/2906142169 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe700a1920 con 0x7fbe70073f60 2024-09-15T21:19:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.294+0000 7fbe6f7fe640 1 --2- 172.21.15.17:0/2906142169 >> v2:172.21.15.17:3300/0 conn(0x7fbe70073640 0x7fbe70073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 1 --2- 172.21.15.17:0/2906142169 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 0x7fbe70137690 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009870 tx=0x7fbe5801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f07d5c7faa95c42c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 1 -- 172.21.15.17:0/2906142169 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe580b1070 con 0x7fbe70073f60 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: dump: 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 1 -- 172.21.15.17:0/2906142169 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe580ac620 con 0x7fbe70073f60 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 1 -- 172.21.15.17:0/2906142169 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe580ac9f0 con 0x7fbe70073f60 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 5 monclient: authenticate success, global_id 24368 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 4 set_mon_vals no callback set 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6e7fc640 10 monclient: dump: 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: shutdown 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- 172.21.15.17:0/2906142169 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 msgr2=0x7fbe70137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 --2- 172.21.15.17:0/2906142169 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 0x7fbe70137690 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009870 tx=0x7fbe5801c4b0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- 172.21.15.17:0/2906142169 shutdown_connections 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 --2- 172.21.15.17:0/2906142169 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073f60 0x7fbe70137690 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 --2- 172.21.15.17:0/2906142169 >> v2:172.21.15.17:3300/0 conn(0x7fbe70073640 0x7fbe70073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- 172.21.15.17:0/2906142169 >> 172.21.15.17:0/2906142169 conn(0x7fbe700a66a0 msgr2=0x7fbe700a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- 172.21.15.17:0/2906142169 shutdown_connections 2024-09-15T21:19:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- 172.21.15.17:0/2906142169 wait complete. 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: monmap: 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 Processor -- start 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- start start 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: init 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073640 0x7fbe7011fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): picked mon.noname-c con 0x7fbe70073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe70073f60 0x7fbe70136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): picked mon.noname-b con 0x7fbe70073f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70137bd0 0x7fbe7011d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): picked mon.noname-a con 0x7fbe70137bd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7009ce50 con 0x7fbe70137bd0 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7009ccd0 con 0x7fbe70073f60 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbe700a0d00 con 0x7fbe70073640 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe75dff640 10 monclient(hunting): authenticate will time out at 1118.695469s 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70137bd0 0x7fbe7011d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073640 0x7fbe7011fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073640 0x7fbe7011fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59446/0 (socket says 172.21.15.17:59446) 2024-09-15T21:19:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 1 -- 172.21.15.17:0/3474065633 learned_addr learned my addr 172.21.15.17:0/3474065633 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): get_auth_request con 0x7fbe70073640 auth_method 0 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): get_auth_request con 0x7fbe70137bd0 auth_method 0 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe70073f60 0x7fbe70136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): get_auth_request con 0x7fbe70073f60 auth_method 0 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.298+0000 7fbe6ffff640 10 monclient(hunting): handle_auth_done global_id 5022 payload 824 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073640 msgr2=0x7fbe7011fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073640 0x7fbe7011fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe70073f60 msgr2=0x7fbe70136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe70073f60 0x7fbe70136900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 monclient: found mon.noname-a 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe70120fb0 con 0x7fbe70137bd0 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbe70073640 0x7fbe7011fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6ffff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70137bd0 0x7fbe7011d470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800c3c0 tx=0x7fbe6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68017070 con 0x7fbe70137bd0 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: dump: 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 5 monclient: authenticate success, global_id 5022 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe680052d0 con 0x7fbe70137bd0 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 10 monclient: _renew_subs 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68003820 con 0x7fbe70137bd0 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe701375a0 con 0x7fbe70137bd0 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6dffb640 4 set_mon_vals no callback set 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: dump: 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 10 monclient: _renew_subs 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe70121290 con 0x7fbe70137bd0 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 0x7fbe7011d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7009d150 con 0x7fbe70136e40 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 0x7fbe7011d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient: get_auth_request con 0x7fbe70136e40 auth_method 0 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbe6801a070 con 0x7fbe70137bd0 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_done global_id 5025 payload 824 2024-09-15T21:19:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe70073a20 con 0x7fbe70136e40 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6f7fe640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 0x7fbe7011d9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbe60004980 tx=0x7fbe6000e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9f0 0x7fbe5003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7fbe6804f050 con 0x7fbe70137bd0 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60019070 con 0x7fbe70136e40 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: dump: 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6effd640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9f0 0x7fbe5003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6effd640 10 monclient: get_auth_request con 0x7fbe5003b9f0 auth_method 0 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbe60015920 con 0x7fbe70136e40 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 msgr2=0x7fbe7011d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 0x7fbe7011d9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbe60004980 tx=0x7fbe6000e8c0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 msgr2=0x7fbe7011d9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe67fff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70136e40 0x7fbe7011d9b0 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbe60004980 tx=0x7fbe6000e8c0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.302+0000 7fbe6effd640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9f0 0x7fbe5003deb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbe580ae000 tx=0x7fbe58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:55.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 0x7fbe7011def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70120ab0 con 0x7fbe701c7980 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 0x7fbe7011def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient: get_auth_request con 0x7fbe701c7980 auth_method 0 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 10 monclient(hunting): handle_auth_done global_id 5028 payload 824 2024-09-15T21:19:55.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 1 -- 172.21.15.17:0/3474065633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbe70120c60 con 0x7fbe701c7980 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6f7fe640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 0x7fbe7011def0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe60004980 tx=0x7fbe60004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600295f0 con 0x7fbe701c7980 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 10 monclient: dump: 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.331 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7fbe600143b0 con 0x7fbe701c7980 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 msgr2=0x7fbe7011def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 0x7fbe7011def0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe60004980 tx=0x7fbe60004ac0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 msgr2=0x7fbe7011def0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:55.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe67fff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe701c7980 0x7fbe7011def0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 32, 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 160931, 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:19:55.333 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.335 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:55.347 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9f0 msgr2=0x7fbe5003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbe5003b9f0 0x7fbe5003deb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbe580ae000 tx=0x7fbe58008040 comp rx=0 tx=0).stop 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 10 monclient: shutdown 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70137bd0 msgr2=0x7fbe7011d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 --2- 172.21.15.17:0/3474065633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe70137bd0 0x7fbe7011d470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800c3c0 tx=0x7fbe6800c880 comp rx=0 tx=0).stop 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6ffff640 1 -- 172.21.15.17:0/3474065633 reap_dead start 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe6ffff640 1 -- 172.21.15.17:0/3474065633 reap_dead start 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 shutdown_connections 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 >> 172.21.15.17:0/3474065633 conn(0x7fbe700a66a0 msgr2=0x7fbe700a5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 shutdown_connections 2024-09-15T21:19:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.310+0000 7fbe75dff640 1 -- 172.21.15.17:0/3474065633 wait complete. 2024-09-15T21:19:55.349 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:55.350 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:19:55.350 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:19:55.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aaea2b640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aaea2b640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aaea2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f9aa8137ca0 con 0x7f9aa80739a0 2024-09-15T21:19:55.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) ==== 65+0+60 (secure 0 0 0) 0x7f9a9c00a5e0 con 0x7f9aa80739a0 2024-09-15T21:19:55.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aa4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.427+0000 7f9aa4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.470+0000 7fcd2e979640 10 monclient: get_monmap_and_config 2024-09-15T21:19:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.470+0000 7fcd2e979640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.470+0000 7fcd2e979640 10 monclient: monmap: 2024-09-15T21:19:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:55.474125+0000 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:55.474125+0000 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:55.477 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 1 Processor -- start 2024-09-15T21:19:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 1 -- start start 2024-09-15T21:19:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient: init 2024-09-15T21:19:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcd280736a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcd280a21e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd2813c0b0 0x7fcd2813e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient(hunting): picked mon.noname-b con 0x7fcd2813c0b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd28074190 con 0x7fcd280a21e0 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd280a1520 con 0x7fcd280736a0 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcd280a1800 con 0x7fcd2813c0b0 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2e979640 10 monclient(hunting): authenticate will time out at 1118.869691s 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd277fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcd280a21e0 0x7fcd2813a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39950/0 (socket says 172.21.15.17:39950) 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd277fe640 1 -- 172.21.15.17:0/2764475238 learned_addr learned my addr 172.21.15.17:0/2764475238 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 1 --2- 172.21.15.17:0/2764475238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd2813c0b0 0x7fcd2813e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): get_auth_request con 0x7fcd2813c0b0 auth_method 0 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd26ffd640 1 -- 172.21.15.17:0/2764475238 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 637399300 0 0) 0x7fcd28074190 con 0x7fcd280a21e0 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd26ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:19:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd26ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd26ffd640 10 monclient(hunting): my global_id is 5031 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd26ffd640 1 -- 172.21.15.17:0/2764475238 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd10003530 con 0x7fcd280a21e0 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient(hunting): handle_auth_done global_id 24374 payload 274 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 1 -- 172.21.15.17:0/2764475238 >> v1:172.21.15.119:6789/0 conn(0x7fcd280736a0 legacy=0x7fcd28073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 1 -- 172.21.15.17:0/2764475238 >> v1:172.21.15.17:6789/0 conn(0x7fcd280a21e0 legacy=0x7fcd2813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 1 monclient: found mon.noname-b 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:19:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.474+0000 7fcd2cf16640 1 -- 172.21.15.17:0/2764475238 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd280a1ae0 con 0x7fcd2813c0b0 2024-09-15T21:19:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2cf16640 1 --2- 172.21.15.17:0/2764475238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd2813c0b0 0x7fcd2813e4a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd180097f0 tx=0x7fcd1801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b05c634241a412d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 1 -- 172.21.15.17:0/2764475238 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180b0070 con 0x7fcd2813c0b0 2024-09-15T21:19:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: dump: 2024-09-15T21:19:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 5 monclient: authenticate success, global_id 24374 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 1 -- 172.21.15.17:0/2764475238 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd180ab600 con 0x7fcd2813c0b0 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 4 set_mon_vals no callback set 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 1 -- 172.21.15.17:0/2764475238 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180ab9b0 con 0x7fcd2813c0b0 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd26ffd640 10 monclient: dump: 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.493 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: shutdown 2024-09-15T21:19:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- 172.21.15.17:0/2764475238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd2813c0b0 msgr2=0x7fcd2813e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 --2- 172.21.15.17:0/2764475238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd2813c0b0 0x7fcd2813e4a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd180097f0 tx=0x7fcd1801c3a0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- 172.21.15.17:0/2764475238 shutdown_connections 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 --2- 172.21.15.17:0/2764475238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd2813c0b0 0x7fcd2813e4a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- 172.21.15.17:0/2764475238 >> 172.21.15.17:0/2764475238 conn(0x7fcd280a4e90 msgr2=0x7fcd280a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- 172.21.15.17:0/2764475238 shutdown_connections 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- 172.21.15.17:0/2764475238 wait complete. 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: monmap: 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:55.495 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 Processor -- start 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- start start 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: init 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd280736a0 0x7fcd2813ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): picked mon.noname-c con 0x7fcd280736a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd280a21e0 0x7fcd281ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): picked mon.noname-a con 0x7fcd280a21e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2813c0b0 0x7fcd281ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): picked mon.noname-b con 0x7fcd2813c0b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281419a0 con 0x7fcd280a21e0 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcd28141820 con 0x7fcd2813c0b0 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcd28141b20 con 0x7fcd280736a0 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2e979640 10 monclient(hunting): authenticate will time out at 1118.875475s 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd280736a0 0x7fcd2813ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd280a21e0 0x7fcd281ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd280a21e0 0x7fcd281ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47592/0 (socket says 172.21.15.17:47592) 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 1 -- 172.21.15.17:0/710187137 learned_addr learned my addr 172.21.15.17:0/710187137 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): get_auth_request con 0x7fcd280a21e0 auth_method 0 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): get_auth_request con 0x7fcd280736a0 auth_method 0 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2cf16640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2813c0b0 0x7fcd281ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2cf16640 10 monclient(hunting): get_auth_request con 0x7fcd2813c0b0 auth_method 0 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2cf16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2cf16640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd2cf16640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.478+0000 7fcd277fe640 10 monclient(hunting): handle_auth_done global_id 5034 payload 824 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd280736a0 msgr2=0x7fcd2813ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd280736a0 0x7fcd2813ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2813c0b0 msgr2=0x7fcd281ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2813c0b0 0x7fcd281ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 monclient: found mon.noname-a 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2811ebd0 con 0x7fcd280a21e0 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2cf16640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2813c0b0 0x7fcd281ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd277fe640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd280a21e0 0x7fcd281ce090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c4d0 tx=0x7fcd1c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c017070 con 0x7fcd280a21e0 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: dump: 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c005d10 con 0x7fcd280a21e0 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 5 monclient: authenticate success, global_id 5034 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 10 monclient: _renew_subs 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2811eeb0 con 0x7fcd280a21e0 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0072b0 con 0x7fcd280a21e0 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: dump: 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd25ffb640 4 set_mon_vals no callback set 2024-09-15T21:19:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 10 monclient: _renew_subs 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2811e8f0 con 0x7fcd280a21e0 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 0x7fcd2802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcd28136660 con 0x7fcd28138e10 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 0x7fcd2802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient: get_auth_request con 0x7fcd28138e10 auth_method 0 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcd1c01a070 con 0x7fcd280a21e0 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 10 monclient(hunting): handle_auth_done global_id 24380 payload 824 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd28072a70 con 0x7fcd28138e10 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd1403b9c0 0x7fcd1403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd27fff640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 0x7fcd2802d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd2000ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7fcd1c04ef40 con 0x7fcd280a21e0 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20003b90 con 0x7fcd28138e10 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: dump: 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2cf16640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd1403b9c0 0x7fcd1403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2cf16640 10 monclient: get_auth_request con 0x7fcd1403b9c0 auth_method 0 2024-09-15T21:19:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcd20018070 con 0x7fcd28138e10 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 msgr2=0x7fcd2802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 0x7fcd2802d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd2000ff70 comp rx=0 tx=0).stop 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 msgr2=0x7fcd2802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd24ff9640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd28138e10 0x7fcd2802d450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.482+0000 7fcd2cf16640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd1403b9c0 0x7fcd1403de80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd180ad000 tx=0x7fcd180a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 0x7fcd2802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcd28073a80 con 0x7fcd281391f0 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 0x7fcd2802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient: get_auth_request con 0x7fcd281391f0 auth_method 0 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 10 monclient(hunting): handle_auth_done global_id 24383 payload 824 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 1 -- 172.21.15.17:0/710187137 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcd280a04a0 con 0x7fcd281391f0 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd27fff640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 0x7fcd2802d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd200014b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000db10 con 0x7fcd281391f0 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 10 monclient: dump: 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7fcd2000db10 con 0x7fcd281391f0 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 msgr2=0x7fcd2802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 0x7fcd2802d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd200014b0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 msgr2=0x7fcd2802d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd24ff9640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd281391f0 0x7fcd2802d990 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd200014b0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 32, 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 51295, 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:55.527 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd1403b9c0 msgr2=0x7fcd1403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd1403b9c0 0x7fcd1403de80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd180ad000 tx=0x7fcd180a6000 comp rx=0 tx=0).stop 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 10 monclient: shutdown 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd280a21e0 msgr2=0x7fcd281ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 --2- 172.21.15.17:0/710187137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd280a21e0 0x7fcd281ce090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c4d0 tx=0x7fcd1c00c990 comp rx=0 tx=0).stop 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2cf16640 1 -- 172.21.15.17:0/710187137 reap_dead start 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2cf16640 1 -- 172.21.15.17:0/710187137 reap_dead start 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 shutdown_connections 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 >> 172.21.15.17:0/710187137 conn(0x7fcd280a4e90 msgr2=0x7fcd280a65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 shutdown_connections 2024-09-15T21:19:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.490+0000 7fcd2e979640 1 -- 172.21.15.17:0/710187137 wait complete. 2024-09-15T21:19:55.530 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:55.530 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:19:55.530 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.523+0000 7f9aaea2b640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.523+0000 7f9aaea2b640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.523+0000 7f9aaea2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.523+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f9aa806c4b0 con 0x7f9aa80739a0 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aa4ff9640 1 -- 172.21.15.119:0/641455703 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) ==== 65+0+60 (secure 0 0 0) 0x7f9a9c051100 con 0x7f9aa80739a0 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aa4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T21:19:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aa4ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:19:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a9403b9c0 msgr2=0x7f9a9403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a9403b9c0 0x7f9a9403de80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0001750 tx=0x7f9aa0009040 comp rx=0 tx=0).stop 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 10 monclient: shutdown 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 msgr2=0x7f9aa81d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 0x7f9aa81d07a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c760 tx=0x7f9a9c00cb60 comp rx=0 tx=0).stop 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 shutdown_connections 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a9403b9c0 0x7f9a9403de80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9aa813e240 0x7f9aa81337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9aa80739a0 0x7f9aa81d07a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 --2- 172.21.15.119:0/641455703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9aa80735c0 0x7f9aa81d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 >> 172.21.15.119:0/641455703 conn(0x7f9aa806cbf0 msgr2=0x7f9aa8136c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 shutdown_connections 2024-09-15T21:19:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.527+0000 7f9aaea2b640 1 -- 172.21.15.119:0/641455703 wait complete. 2024-09-15T21:19:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:19:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm foo 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient: get_monmap_and_config 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient: monmap: 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:55.647258+0000 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:55.647258+0000 2024-09-15T21:19:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:55.650 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 Processor -- start 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 -- start start 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient: init 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f46e41378c0 0x7f46e4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): picked mon.noname-c con 0x7f46e41378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:19:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f46e41381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46e40a5210 0x7f46e40a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): picked mon.noname-a con 0x7f46e40a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e40a1d00 con 0x7f46e41381e0 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f46e40a1fe0 con 0x7f46e40a5210 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f46e40a22c0 con 0x7f46e41378c0 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e8b8b640 10 monclient(hunting): authenticate will time out at 1119.043337s 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46e40a5210 0x7f46e40a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46e40a5210 0x7f46e40a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47598/0 (socket says 172.21.15.17:47598) 2024-09-15T21:19:55.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 1 -- 172.21.15.17:0/1537090020 learned_addr learned my addr 172.21.15.17:0/1537090020 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): get_auth_request con 0x7f46e40a5210 auth_method 0 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e259c640 1 --2- 172.21.15.17:0/1537090020 >> v2:172.21.15.119:3300/0 conn(0x7f46e41378c0 0x7f46e4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e259c640 10 monclient(hunting): get_auth_request con 0x7f46e41378c0 auth_method 0 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 10 monclient(hunting): handle_auth_done global_id 5037 payload 274 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 1 -- 172.21.15.17:0/1537090020 >> v2:172.21.15.119:3300/0 conn(0x7f46e41378c0 msgr2=0x7f46e4137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.646+0000 7f46e2d9d640 1 --2- 172.21.15.17:0/1537090020 >> v2:172.21.15.119:3300/0 conn(0x7f46e41378c0 0x7f46e4137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 1 -- 172.21.15.17:0/1537090020 >> v1:172.21.15.119:6789/0 conn(0x7f46e41381e0 legacy=0x7f46e40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 1 monclient: found mon.noname-a 2024-09-15T21:19:55.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 1 -- 172.21.15.17:0/1537090020 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e40a4490 con 0x7f46e40a5210 2024-09-15T21:19:55.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e2d9d640 1 --2- 172.21.15.17:0/1537090020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e40a5210 0x7f46e40a56a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009e50 tx=0x7f46dc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53635ce9f370042 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 1 -- 172.21.15.17:0/1537090020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0a5790 con 0x7f46e40a5210 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: dump: 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.664 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 1 -- 172.21.15.17:0/1537090020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46dc0a5930 con 0x7f46e40a5210 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 1 -- 172.21.15.17:0/1537090020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0a5d00 con 0x7f46e40a5210 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e1d9b640 10 monclient: dump: 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 5 monclient: authenticate success, global_id 5037 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 4 set_mon_vals no callback set 2024-09-15T21:19:55.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 10 monclient: shutdown 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 -- 172.21.15.17:0/1537090020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e40a5210 msgr2=0x7f46e40a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/1537090020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e40a5210 0x7f46e40a56a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009e50 tx=0x7f46dc01c8b0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 -- 172.21.15.17:0/1537090020 shutdown_connections 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/1537090020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e40a5210 0x7f46e40a56a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/1537090020 >> v2:172.21.15.119:3300/0 conn(0x7f46e41378c0 0x7f46e4137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 -- 172.21.15.17:0/1537090020 >> 172.21.15.17:0/1537090020 conn(0x7f46e40aac40 msgr2=0x7f46e40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 -- 172.21.15.17:0/1537090020 shutdown_connections 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 1 -- 172.21.15.17:0/1537090020 wait complete. 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.650+0000 7f46e8b8b640 10 monclient: monmap: 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:55.667 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 Processor -- start 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- start start 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: init 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46e40a5210 0x7f46e41cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): picked mon.noname-c con 0x7f46e40a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 0x7f46e4136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): picked mon.noname-a con 0x7f46e41378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e41381e0 0x7f46e413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): picked mon.noname-b con 0x7f46e41381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f46e406f340 con 0x7f46e41378c0 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46e406f1c0 con 0x7f46e41381e0 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f46e41425c0 con 0x7f46e40a5210 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): authenticate will time out at 1119.048650s 2024-09-15T21:19:55.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 0x7f46e4136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46e40a5210 0x7f46e41cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 0x7f46e4136a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47602/0 (socket says 172.21.15.17:47602) 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 -- 172.21.15.17:0/9496546 learned_addr learned my addr 172.21.15.17:0/9496546 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): get_auth_request con 0x7f46e41378c0 auth_method 0 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): get_auth_request con 0x7f46e40a5210 auth_method 0 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e41381e0 0x7f46e413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 10 monclient(hunting): get_auth_request con 0x7f46e41381e0 auth_method 0 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient(hunting): handle_auth_done global_id 5040 payload 824 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46e40a5210 msgr2=0x7f46e41cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46e40a5210 0x7f46e41cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e41381e0 msgr2=0x7f46e413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e41381e0 0x7f46e413cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 monclient: found mon.noname-a 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e4120dd0 con 0x7f46e41378c0 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e41381e0 0x7f46e413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46e40a5210 0x7f46e41cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46dbfff640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 0x7f46e4136a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c500 tx=0x7f46cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc017070 con 0x7f46e41378c0 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: dump: 2024-09-15T21:19:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46cc0052d0 con 0x7f46e41378c0 2024-09-15T21:19:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 5 monclient: authenticate success, global_id 5040 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: _renew_subs 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e41210b0 con 0x7f46e41378c0 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc012870 con 0x7f46e41378c0 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: dump: 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e159a640 4 set_mon_vals no callback set 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: _renew_subs 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e4120af0 con 0x7f46e41378c0 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 0x7f46e413d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46e40a89c0 con 0x7f46e4136f40 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 0x7f46e413d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f46cc01a070 con 0x7f46e41378c0 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient: get_auth_request con 0x7f46e4136f40 auth_method 0 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46c403b9c0 0x7f46c403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7f46cc04f050 con 0x7f46e41378c0 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46c403b9c0 0x7f46c403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 10 monclient: get_auth_request con 0x7f46c403b9c0 auth_method 0 2024-09-15T21:19:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 10 monclient(hunting): handle_auth_done global_id 34168 payload 824 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46e40a56a0 con 0x7f46e4136f40 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e259c640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 0x7f46e413d100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006aa0 tx=0x7f46d400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46e2d9d640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46c403b9c0 0x7f46c403de80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0ae000 tx=0x7f46dc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4018070 con 0x7f46e4136f40 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.654+0000 7f46daffd640 10 monclient: dump: 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f46d4010c90 con 0x7f46e4136f40 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 msgr2=0x7f46e413d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 0x7f46e413d100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006aa0 tx=0x7f46d400e5a0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 msgr2=0x7f46e413d100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.658+0000 7f46daffd640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e4136f40 0x7f46e413d100 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006aa0 tx=0x7f46d400e5a0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e8b8b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 0x7f46e4138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e8b8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46e413da50 con 0x7f46e413d640 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 0x7f46e4138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient: get_auth_request con 0x7f46e413d640 auth_method 0 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 10 monclient(hunting): handle_auth_done global_id 34174 payload 824 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 1 -- 172.21.15.17:0/9496546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f46e41d0630 con 0x7f46e413d640 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46e259c640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 0x7f46e4138d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006aa0 tx=0x7f46d40046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d402b020 con 0x7f46e413d640 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 10 monclient: dump: 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f46d4014840 con 0x7f46e413d640 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 msgr2=0x7f46e4138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 0x7f46e4138d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006aa0 tx=0x7f46d40046d0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 msgr2=0x7f46e4138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.662+0000 7f46daffd640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e413d640 0x7f46e4138d60 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006aa0 tx=0x7f46d40046d0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.684 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 32, 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 8163, 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:19:55.685 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.686 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.687 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:19:55.688 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:19:55.689 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:19:55.691 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:19:55.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.695 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:19:55.696 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:19:55.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46c403b9c0 msgr2=0x7f46c403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f46c403b9c0 0x7f46c403de80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0ae000 tx=0x7f46dc0a7070 comp rx=0 tx=0).stop 2024-09-15T21:19:55.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 10 monclient: shutdown 2024-09-15T21:19:55.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 msgr2=0x7f46e4136a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 0x7f46e4136a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c500 tx=0x7f46cc00c9c0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e2d9d640 1 -- 172.21.15.17:0/9496546 reap_dead start 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 shutdown_connections 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 --2- 172.21.15.17:0/9496546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e41378c0 0x7f46e4136a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 >> 172.21.15.17:0/9496546 conn(0x7f46e40aac40 msgr2=0x7f46e409da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 shutdown_connections 2024-09-15T21:19:55.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.666+0000 7f46e8b8b640 1 -- 172.21.15.17:0/9496546 wait complete. 2024-09-15T21:19:55.700 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:19:55.701 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T21:19:55.701 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:19:55.701 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T21:19:55.701 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T21:19:55.701 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:19:55.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:19:55.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.655+0000 7ffb80197640 10 monclient: get_monmap_and_config 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.655+0000 7ffb80197640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.655+0000 7ffb80197640 10 monclient: monmap: 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:19:55.659755+0000 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:19:55.659755+0000 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:55.706 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.655+0000 7ffb80197640 1 Processor -- start 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.655+0000 7ffb80197640 1 -- start start 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.655+0000 7ffb80197640 10 monclient: init 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffb78137fe0 0x7ffb781383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient(hunting): picked mon.noname-a con 0x7ffb78137fe0 addr v2:172.21.15.17:3300/0 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb780a2320 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffb780a5570 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb780a3e10 con 0x7ffb780a2320 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb780a1960 con 0x7ffb780a5570 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ffb780a1400 con 0x7ffb78137fe0 2024-09-15T21:19:55.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient(hunting): authenticate will time out at 1114.456315s 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7e734640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffb780a5570 0x7ffb7813fe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33568/0 (socket says 172.21.15.119:33568) 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7e734640 1 -- 172.21.15.119:0/3607498406 learned_addr learned my addr 172.21.15.119:0/3607498406 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 1 --2- 172.21.15.119:0/3607498406 >> v2:172.21.15.17:3300/0 conn(0x7ffb78137fe0 0x7ffb781383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): get_auth_request con 0x7ffb78137fe0 auth_method 0 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598207546 0 0) 0x7ffb780a1960 con 0x7ffb780a5570 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): _negotiate 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): my global_id is 34171 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb64003530 con 0x7ffb780a5570 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491805782 0 0) 0x7ffb780a3e10 con 0x7ffb780a2320 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): _negotiate 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient(hunting): my global_id is 5046 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb780a1960 con 0x7ffb780a2320 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient(hunting): handle_auth_done global_id 5043 payload 274 2024-09-15T21:19:55.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 1 -- 172.21.15.119:0/3607498406 >> v1:172.21.15.119:6789/0 conn(0x7ffb780a5570 legacy=0x7ffb7813fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 1 -- 172.21.15.119:0/3607498406 >> v1:172.21.15.17:6789/0 conn(0x7ffb780a2320 legacy=0x7ffb780a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 1 monclient: found mon.noname-a 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 1 -- 172.21.15.119:0/3607498406 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb780a6bf0 con 0x7ffb78137fe0 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7df33640 1 --2- 172.21.15.119:0/3607498406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb78137fe0 0x7ffb781383c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffb740097c0 tx=0x7ffb7401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8a4e79bc2481c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb740b2070 con 0x7ffb78137fe0 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: dump: 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.710 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb740ad680 con 0x7ffb78137fe0 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 1 -- 172.21.15.119:0/3607498406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb740ada50 con 0x7ffb78137fe0 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 5 monclient: authenticate success, global_id 5043 2024-09-15T21:19:55.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 4 set_mon_vals no callback set 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb7cf31640 10 monclient: dump: 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.712 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 10 monclient: shutdown 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- 172.21.15.119:0/3607498406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb78137fe0 msgr2=0x7ffb781383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 --2- 172.21.15.119:0/3607498406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb78137fe0 0x7ffb781383c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffb740097c0 tx=0x7ffb7401c360 comp rx=0 tx=0).stop 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- 172.21.15.119:0/3607498406 shutdown_connections 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 --2- 172.21.15.119:0/3607498406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb78137fe0 0x7ffb781383c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- 172.21.15.119:0/3607498406 >> 172.21.15.119:0/3607498406 conn(0x7ffb780a6f60 msgr2=0x7ffb780a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- 172.21.15.119:0/3607498406 shutdown_connections 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.659+0000 7ffb80197640 1 -- 172.21.15.119:0/3607498406 wait complete. 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: monmap: 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:19:55.713 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 Processor -- start 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- start start 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: init 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:19:55.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 0x7ffb7813c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): picked mon.noname-b con 0x7ffb780a2320 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780a5570 0x7ffb7813cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): picked mon.noname-a con 0x7ffb780a5570 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb78137fe0 0x7ffb7813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): picked mon.noname-c con 0x7ffb78137fe0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffb780a64d0 con 0x7ffb780a5570 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb780a6350 con 0x7ffb780a2320 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb78143dc0 con 0x7ffb78137fe0 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient(hunting): authenticate will time out at 1114.461132s 2024-09-15T21:19:55.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 0x7ffb7813c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 0x7ffb7813c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38760/0 (socket says 172.21.15.119:38760) 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 -- 172.21.15.119:0/2827022470 learned_addr learned my addr 172.21.15.119:0/2827022470 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): get_auth_request con 0x7ffb780a2320 auth_method 0 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780a5570 0x7ffb7813cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb78137fe0 0x7ffb7813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): get_auth_request con 0x7ffb78137fe0 auth_method 0 2024-09-15T21:19:55.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): get_auth_request con 0x7ffb780a5570 auth_method 0 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7e734640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient(hunting): handle_auth_done global_id 34177 payload 824 2024-09-15T21:19:55.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 -- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb78137fe0 msgr2=0x7ffb7813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb78137fe0 0x7ffb7813d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 -- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780a5570 msgr2=0x7ffb7813cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780a5570 0x7ffb7813cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 monclient: found mon.noname-b 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 -- 172.21.15.119:0/2827022470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb78140610 con 0x7ffb780a2320 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7df33640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 0x7ffb7813c7f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffb74019f90 tx=0x7ffb7401aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb740b2040 con 0x7ffb780a2320 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:19:55.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: dump: 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb740ad8f0 con 0x7ffb780a2320 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 5 monclient: authenticate success, global_id 34177 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _renew_subs 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb78141270 con 0x7ffb780a2320 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb740adcc0 con 0x7ffb780a2320 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 10 monclient: dump: 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.720 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _renew_subs 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6ffff640 4 set_mon_vals no callback set 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7813e2b0 con 0x7ffb780a2320 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb781383c0 con 0x7ffb780a2320 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffb740bf030 con 0x7ffb780a2320 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5c03b9f0 0x7ffb5c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7ffb740e9410 con 0x7ffb780a2320 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5c03b9f0 0x7ffb5c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 10 monclient: get_auth_request con 0x7ffb5c03b9f0 auth_method 0 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.663+0000 7ffb7d732640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5c03b9f0 0x7ffb5c03deb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb78137590 tx=0x7ffb68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.667+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffb74016070 con 0x7ffb780a2320 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.667+0000 7ffb6e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.667+0000 7ffb6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:55.723 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:19:55.723 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:19:55.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:55.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.843+0000 7ffb80197640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:19:55.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.843+0000 7ffb80197640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:19:55.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.843+0000 7ffb80197640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:19:55.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:55.843+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7ffb780a10a0 con 0x7ffb780a2320 2024-09-15T21:19:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient: get_monmap_and_config 2024-09-15T21:19:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient: monmap: 2024-09-15T21:19:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:55.883440+0000 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:55.883440+0000 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:55.886 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 1 Processor -- start 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 1 -- start start 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient: init 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 0x7fb7240a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient(hunting): picked mon.noname-b con 0x7fb7240a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb72409ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb7240a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb724136490 con 0x7fb7240a11b0 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7240a3130 con 0x7fb72409ffc0 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb724073df0 con 0x7fb7240a0890 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72be5f640 10 monclient(hunting): authenticate will time out at 1119.278838s 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb729bfb640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 msgr2=0x7fb7240a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb729bfb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 0x7fb7240a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:55.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72a3fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb7240a11b0 0x7fb724135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39962/0 (socket says 172.21.15.17:39962) 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72a3fc640 1 -- 172.21.15.17:0/1201089392 learned_addr learned my addr 172.21.15.17:0/1201089392 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 178445702 0 0) 0x7fb724136490 con 0x7fb7240a11b0 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): _negotiate 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): my global_id is 5052 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb710003530 con 0x7fb7240a11b0 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1374257973 0 0) 0x7fb710003530 con 0x7fb7240a11b0 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 msgr2=0x7fb7240a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 0x7fb7240a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 >> v1:172.21.15.119:6789/0 conn(0x7fb72409ffc0 legacy=0x7fb724135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7240a1bf0 con 0x7fb7240a11b0 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb71c001a80 con 0x7fb7240a11b0 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: dump: 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 >> v1:172.21.15.17:6789/0 conn(0x7fb7240a11b0 legacy=0x7fb724135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7100039f0 0x7fb710003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): picked mon.a con 0x7fb7100039f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb710004810 0x7fb710004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): picked mon.b con 0x7fb710004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 0x7fb71000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): picked mon.c con 0x7fb7240a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb724073df0 con 0x7fb7100039f0 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100056a0 con 0x7fb710004810 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb71000bf90 con 0x7fb7240a0890 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb728bf9640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72a3fc640 1 -- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 msgr2=0x7fb71000b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb72a3fc640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 0x7fb71000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.882+0000 7fb729bfb640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7100039f0 0x7fb710003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): get_auth_request con 0x7fb7100039f0 auth_method 0 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient(hunting): handle_auth_done global_id 5052 payload 293 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 -- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 msgr2=0x7fb71000b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7240a0890 0x7fb71000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 -- 172.21.15.17:0/1201089392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb710004810 msgr2=0x7fb710004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb710004810 0x7fb710004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 monclient: found mon.a 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 -- 172.21.15.17:0/1201089392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb710003530 con 0x7fb7100039f0 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 5 monclient: authenticate success, global_id 5052 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:55.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb729bfb640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7100039f0 0x7fb710003e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb720003990 tx=0x7fb72001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d37ea27eefe83f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720009520 con 0x7fb7100039f0 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: dump: 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.901 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720003bc0 con 0x7fb7100039f0 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 1 -- 172.21.15.17:0/1201089392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720005070 con 0x7fb7100039f0 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 4 set_mon_vals no callback set 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb728bf9640 10 monclient: dump: 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 10 monclient: shutdown 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 -- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7100039f0 msgr2=0x7fb710003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 --2- 172.21.15.17:0/1201089392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7100039f0 0x7fb710003e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb720003990 tx=0x7fb72001c420 comp rx=0 tx=0).stop 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72a3fc640 1 -- 172.21.15.17:0/1201089392 reap_dead start 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 -- 172.21.15.17:0/1201089392 shutdown_connections 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 -- 172.21.15.17:0/1201089392 >> 172.21.15.17:0/1201089392 conn(0x7fb7240a66a0 msgr2=0x7fb7240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 -- 172.21.15.17:0/1201089392 shutdown_connections 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 -- 172.21.15.17:0/1201089392 wait complete. 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 10 monclient: build_initial_monmap 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 10 monclient: monmap: 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.886+0000 7fb72be5f640 1 Processor -- start 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- start start 2024-09-15T21:19:55.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: init 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7100039f0 0x7fb7241cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): picked mon.noname-c con 0x7fb7100039f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb724134870 0x7fb724134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): picked mon.noname-b con 0x7fb724134870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 0x7fb72409d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): picked mon.noname-a con 0x7fb72409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:55.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb72413f690 con 0x7fb72409d400 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb72413f510 con 0x7fb724134870 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb72413f810 con 0x7fb7100039f0 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient(hunting): authenticate will time out at 1119.284517s 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb729bfb640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7100039f0 msgr2=0x7fb7241cf9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb729bfb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7100039f0 0x7fb7241cf9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 0x7fb72409d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 0x7fb72409d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47618/0 (socket says 172.21.15.17:47618) 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 -- 172.21.15.17:0/2634422698 learned_addr learned my addr 172.21.15.17:0/2634422698 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): get_auth_request con 0x7fb72409d400 auth_method 0 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb7293fa640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb724134870 0x7fb724134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient(hunting): handle_auth_done global_id 5055 payload 824 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 -- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7100039f0 msgr2=0x7fb7241cf9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7100039f0 0x7fb7241cf9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 -- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb724134870 msgr2=0x7fb724134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb724134870 0x7fb724134cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient: _finish_hunting 0 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 monclient: found mon.noname-a 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 -- 172.21.15.17:0/2634422698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb724122050 con 0x7fb72409d400 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb7293fa640 10 monclient: get_auth_request con 0x7fb724134870 auth_method 0 2024-09-15T21:19:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb7293fa640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb724134870 0x7fb724134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72a3fc640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 0x7fb72409d880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb7240a1590 tx=0x7fb71c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c004d70 con 0x7fb72409d400 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: dump: 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c00a5b0 con 0x7fb72409d400 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00a960 con 0x7fb72409d400 2024-09-15T21:19:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 10 monclient: dump: 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 5 monclient: authenticate success, global_id 5055 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _renew_subs 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71bfff640 4 set_mon_vals no callback set 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb724122330 con 0x7fb72409d400 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _renew_subs 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72409d0f0 con 0x7fb72409d400 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb71c011070 con 0x7fb72409d400 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb724135670 con 0x7fb72409d400 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb70803b970 0x7fb70803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7fb71c04f370 con 0x7fb72409d400 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb729bfb640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb70803b970 0x7fb70803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb729bfb640 10 monclient: get_auth_request con 0x7fb70803b970 auth_method 0 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.890+0000 7fb729bfb640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb70803b970 0x7fb70803de30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb7200aa000 tx=0x7fb720005a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.894+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb71c012d90 con 0x7fb72409d400 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.894+0000 7fb71a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:55.894+0000 7fb71a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:56.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb724135270 con 0x7fb72409d400 2024-09-15T21:19:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb71a7fc640 1 -- 172.21.15.17:0/2634422698 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fb71c04f0d0 con 0x7fb72409d400 2024-09-15T21:19:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb71a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb71a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:56.142 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb70803b970 msgr2=0x7fb70803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb70803b970 0x7fb70803de30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb7200aa000 tx=0x7fb720005a90 comp rx=0 tx=0).stop 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 10 monclient: shutdown 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 msgr2=0x7fb72409d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 0x7fb72409d880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb7240a1590 tx=0x7fb71c00cb90 comp rx=0 tx=0).stop 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 shutdown_connections 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb70803b970 0x7fb70803de30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb72409d400 0x7fb72409d880 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb724134870 0x7fb724134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 --2- 172.21.15.17:0/2634422698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb7100039f0 0x7fb7241cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 >> 172.21.15.17:0/2634422698 conn(0x7fb7240a66a0 msgr2=0x7fb7240a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.138+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 shutdown_connections 2024-09-15T21:19:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:56.142+0000 7fb72be5f640 1 -- 172.21.15.17:0/2634422698 wait complete. 2024-09-15T21:19:58.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:58.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:19:58.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:58.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:58.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:58.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:58.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:58.663+0000 7ffb6d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:59.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.290+0000 7f1b5ec88640 10 monclient: get_monmap_and_config 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.290+0000 7f1b5ec88640 10 monclient: build_initial_monmap 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.290+0000 7f1b5ec88640 10 monclient: monmap: 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:19:59.293111+0000 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:19:59.293111+0000 2024-09-15T21:19:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:19:59.296 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.290+0000 7f1b5ec88640 1 Processor -- start 2024-09-15T21:19:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.290+0000 7f1b5ec88640 1 -- start start 2024-09-15T21:19:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.290+0000 7f1b5ec88640 10 monclient: init 2024-09-15T21:19:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1b580a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1b5809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 0x7f1b5813e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient(hunting): picked mon.noname-b con 0x7f1b580a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b580a3130 con 0x7f1b580a0890 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b58073df0 con 0x7f1b5809ffc0 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1b580a1bf0 con 0x7f1b580a1380 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 10 monclient(hunting): authenticate will time out at 1122.688743s 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 msgr2=0x7f1b5813e1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 0x7f1b5813e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ca24640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1b580a0890 0x7f1b580a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42068/0 (socket says 172.21.15.17:42068) 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ca24640 1 -- 172.21.15.17:0/378514012 learned_addr learned my addr 172.21.15.17:0/378514012 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3850195022 0 0) 0x7f1b580a3130 con 0x7f1b580a0890 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): _negotiate 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): my global_id is 5058 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b40003530 con 0x7f1b580a0890 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4035831226 0 0) 0x7f1b40003530 con 0x7f1b580a0890 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 msgr2=0x7f1b5813e1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 0x7f1b5813e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 >> v1:172.21.15.119:6789/0 conn(0x7f1b5809ffc0 legacy=0x7f1b5813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: _finish_hunting 0 2024-09-15T21:19:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5809ccd0 con 0x7f1b580a0890 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: _finish_auth 0 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: _check_auth_tickets 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f1b48001700 con 0x7f1b580a0890 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 5 monclient: authenticate success, global_id 5058 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: dump: 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 >> v1:172.21.15.17:6789/0 conn(0x7f1b580a0890 legacy=0x7f1b580a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:19:59.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 0x7f1b40003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): picked mon.c con 0x7f1b580a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b40004170 0x7f1b40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): picked mon.b con 0x7f1b40004170 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b40004f50 0x7f1b4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): picked mon.a con 0x7f1b40004f50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1b580a1bf0 con 0x7f1b40004f50 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000bf10 con 0x7f1b40004170 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4000c090 con 0x7f1b580a1380 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ca24640 1 -- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 msgr2=0x7f1b40003a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b577fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ec88640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5ca24640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 0x7f1b40003a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b40004f50 0x7f1b4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): get_auth_request con 0x7f1b40004f50 auth_method 0 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b57fff640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b40004170 0x7f1b40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b57fff640 10 monclient(hunting): get_auth_request con 0x7f1b40004170 auth_method 0 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b57fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b57fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient(hunting): handle_auth_done global_id 5058 payload 293 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 -- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 msgr2=0x7f1b40003a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b580a1380 0x7f1b40003a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 -- 172.21.15.17:0/378514012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b40004170 msgr2=0x7f1b40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b40004170 0x7f1b40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient: _finish_hunting 0 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 monclient: found mon.a 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 1 -- 172.21.15.17:0/378514012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b40003530 con 0x7f1b40004f50 2024-09-15T21:19:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient: _finish_auth 0 2024-09-15T21:19:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 10 monclient: _check_auth_tickets 2024-09-15T21:19:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b5d225640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.294+0000 7f1b57fff640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b40004170 0x7f1b40004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b40004f50 0x7f1b4000b9d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009e30 tx=0x7f1b5001caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e8670bf9eb3e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50009a30 con 0x7f1b40004f50 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: dump: 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b500a9070 con 0x7f1b40004f50 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: get_monmap_and_config success 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 1 -- 172.21.15.17:0/378514012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b500a5b20 con 0x7f1b40004f50 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 4 set_mon_vals no callback set 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b577fe640 10 monclient: dump: 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: shutdown 2024-09-15T21:19:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b40004f50 msgr2=0x7f1b4000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/378514012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b40004f50 0x7f1b4000b9d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009e30 tx=0x7f1b5001caf0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 1 -- 172.21.15.17:0/378514012 reap_dead start 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- 172.21.15.17:0/378514012 shutdown_connections 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- 172.21.15.17:0/378514012 >> 172.21.15.17:0/378514012 conn(0x7f1b580a66a0 msgr2=0x7f1b580a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- 172.21.15.17:0/378514012 shutdown_connections 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- 172.21.15.17:0/378514012 wait complete. 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: build_initial_monmap 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: monmap: 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:19:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 Processor -- start 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- start start 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: init 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: _reopen_session rank -1 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 0x7f1b581cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): picked mon.noname-a con 0x7f1b581cbf90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:19:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b58133cf0 0x7f1b581cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): picked mon.noname-c con 0x7f1b58133cf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58134150 0x7f1b581cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): picked mon.noname-b con 0x7f1b58134150 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5813ed90 con 0x7f1b581cbf90 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5813ec10 con 0x7f1b58134150 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): start opening mon connection 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5813ef10 con 0x7f1b58133cf0 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): _renew_subs 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ec88640 10 monclient(hunting): authenticate will time out at 1122.694646s 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b57fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b58133cf0 msgr2=0x7f1b581cf4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b58133cf0 0x7f1b581cf4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:19:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 0x7f1b581cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 0x7f1b581cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59980/0 (socket says 172.21.15.17:59980) 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 -- 172.21.15.17:0/1033764336 learned_addr learned my addr 172.21.15.17:0/1033764336 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): get_auth_request con 0x7f1b581cbf90 auth_method 0 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58134150 0x7f1b581cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 10 monclient(hunting): get_auth_request con 0x7f1b58134150 auth_method 0 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:19:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5d225640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient(hunting): handle_auth_done global_id 5061 payload 824 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 -- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b58133cf0 msgr2=0x7f1b581cf4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b58133cf0 0x7f1b581cf4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 -- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58134150 msgr2=0x7f1b581cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58134150 0x7f1b581cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient: _finish_hunting 0 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 monclient: found mon.noname-a 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 -- 172.21.15.17:0/1033764336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5813b570 con 0x7f1b581cbf90 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b5ca24640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 0x7f1b581cc370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c6b0 tx=0x7f1b4800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48018070 con 0x7f1b581cbf90 2024-09-15T21:19:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: dump: 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: _finish_auth 0 2024-09-15T21:19:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b48004ad0 con 0x7f1b581cbf90 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480070d0 con 0x7f1b581cbf90 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b557fa640 10 monclient: dump: 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:19:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.298+0000 7f1b56ffd640 4 set_mon_vals no callback set 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 5 monclient: authenticate success, global_id 5061 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 monclient: _renew_subs 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b580a4ca0 con 0x7f1b581cbf90 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 monclient: _renew_subs 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b581cff70 con 0x7f1b581cbf90 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b580a0c70 con 0x7f1b581cbf90 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1b48007270 con 0x7f1b581cbf90 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b557fa640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b4403b9a0 0x7f1b4403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7f1b4804f1a0 con 0x7f1b581cbf90 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b57fff640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b4403b9a0 0x7f1b4403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b57fff640 10 monclient: get_auth_request con 0x7f1b4403b9a0 auth_method 0 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.302+0000 7f1b57fff640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b4403b9a0 0x7f1b4403de60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b5811fdb0 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.306+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1b580a4ca0 con 0x7f1b581cbf90 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.306+0000 7f1b557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:19:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.306+0000 7f1b557fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:19:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b5ec88640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:19:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b5ec88640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b5ec88640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:19:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1b5806d020 con 0x7f1b581cbf90 2024-09-15T21:19:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b557fa640 1 -- 172.21.15.17:0/1033764336 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f1b4800ab10 con 0x7f1b581cbf90 2024-09-15T21:19:59.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b557fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:19:59.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.546+0000 7f1b557fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:19:59.585 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b4403b9a0 msgr2=0x7f1b4403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b4403b9a0 0x7f1b4403de60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b5811fdb0 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 10 monclient: shutdown 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 msgr2=0x7f1b581cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 0x7f1b581cc370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c6b0 tx=0x7f1b4800cb70 comp rx=0 tx=0).stop 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 shutdown_connections 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b4403b9a0 0x7f1b4403de60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58134150 0x7f1b581cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b58133cf0 0x7f1b581cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 --2- 172.21.15.17:0/1033764336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b581cbf90 0x7f1b581cc370 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:19:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 >> 172.21.15.17:0/1033764336 conn(0x7f1b580a66a0 msgr2=0x7f1b58134a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:19:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 shutdown_connections 2024-09-15T21:19:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:19:59.550+0000 7f1b5ec88640 1 -- 172.21.15.17:0/1033764336 wait complete. 2024-09-15T21:19:59.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:59.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:19:59.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:59.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:19:59.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:19:59.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:00.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:00.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:00.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:00.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:00.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:00.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:01.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:01.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:01.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:01.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:01.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:01.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:02.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:02.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:02.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:02.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:02.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:02.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:02.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient: build_initial_monmap 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient: monmap: 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:02.720013+0000 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:02.720013+0000 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:02.723 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 Processor -- start 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 -- start start 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient: init 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5d40a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5d409ffc0 0x7ff5d413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): picked mon.noname-a con 0x7ff5d409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d40a1380 0x7ff5d413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): picked mon.noname-c con 0x7ff5d40a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5d40a3130 con 0x7ff5d40a0890 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d4073df0 con 0x7ff5d409ffc0 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d40a1bf0 con 0x7ff5d40a1380 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d948f640 10 monclient(hunting): authenticate will time out at 1126.115385s 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff5d40a0890 0x7ff5d40a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42076/0 (socket says 172.21.15.17:42076) 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5d409ffc0 0x7ff5d413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d2ffd640 1 -- 172.21.15.17:0/467577592 learned_addr learned my addr 172.21.15.17:0/467577592 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): get_auth_request con 0x7ff5d409ffc0 auth_method 0 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d37fe640 1 --2- 172.21.15.17:0/467577592 >> v2:172.21.15.119:3300/0 conn(0x7ff5d40a1380 0x7ff5d413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d37fe640 10 monclient(hunting): get_auth_request con 0x7ff5d40a1380 auth_method 0 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient(hunting): handle_auth_done global_id 5064 payload 274 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 1 -- 172.21.15.17:0/467577592 >> v2:172.21.15.119:3300/0 conn(0x7ff5d40a1380 msgr2=0x7ff5d413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 1 --2- 172.21.15.17:0/467577592 >> v2:172.21.15.119:3300/0 conn(0x7ff5d40a1380 0x7ff5d413d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 1 -- 172.21.15.17:0/467577592 >> v1:172.21.15.17:6789/0 conn(0x7ff5d40a0890 legacy=0x7ff5d40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 1 monclient: found mon.noname-a 2024-09-15T21:20:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.717+0000 7ff5d27fc640 1 -- 172.21.15.17:0/467577592 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d409ccd0 con 0x7ff5d409ffc0 2024-09-15T21:20:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d27fc640 1 --2- 172.21.15.17:0/467577592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d413aac0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bbd38be78e5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 1 -- 172.21.15.17:0/467577592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b0070 con 0x7ff5d409ffc0 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: dump: 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 1 -- 172.21.15.17:0/467577592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c40ab600 con 0x7ff5d409ffc0 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 5 monclient: authenticate success, global_id 5064 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 4 set_mon_vals no callback set 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 1 -- 172.21.15.17:0/467577592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40ab9d0 con 0x7ff5d409ffc0 2024-09-15T21:20:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d1ffb640 10 monclient: dump: 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 10 monclient: shutdown 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 -- 172.21.15.17:0/467577592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 msgr2=0x7ff5d413aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 --2- 172.21.15.17:0/467577592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d413aac0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c320 comp rx=0 tx=0).stop 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 -- 172.21.15.17:0/467577592 shutdown_connections 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 --2- 172.21.15.17:0/467577592 >> v2:172.21.15.119:3300/0 conn(0x7ff5d40a1380 0x7ff5d413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 --2- 172.21.15.17:0/467577592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d413aac0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 -- 172.21.15.17:0/467577592 >> 172.21.15.17:0/467577592 conn(0x7ff5d40a66a0 msgr2=0x7ff5d40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 -- 172.21.15.17:0/467577592 shutdown_connections 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 -- 172.21.15.17:0/467577592 wait complete. 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 10 monclient: build_initial_monmap 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 10 monclient: monmap: 2024-09-15T21:20:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.721+0000 7ff5d948f640 1 Processor -- start 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- start start 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: init 2024-09-15T21:20:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d4133150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): picked mon.noname-a con 0x7ff5d409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40a0890 0x7ff5d4133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): picked mon.noname-b con 0x7ff5d40a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5d40a1380 0x7ff5d4133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): picked mon.noname-c con 0x7ff5d40a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d409d790 con 0x7ff5d409ffc0 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d409d610 con 0x7ff5d40a0890 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d409d910 con 0x7ff5d40a1380 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient(hunting): authenticate will time out at 1126.120478s 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d37fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5d40a1380 msgr2=0x7ff5d4133bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d37fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5d40a1380 0x7ff5d4133bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d4133150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d4133150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59992/0 (socket says 172.21.15.17:59992) 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 -- 172.21.15.17:0/641523105 learned_addr learned my addr 172.21.15.17:0/641523105 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): get_auth_request con 0x7ff5d409ffc0 auth_method 0 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d27fc640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40a0890 0x7ff5d4133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient(hunting): handle_auth_done global_id 5067 payload 824 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 -- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5d40a1380 msgr2=0x7ff5d4133bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5d40a1380 0x7ff5d4133bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 -- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40a0890 msgr2=0x7ff5d4133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40a0890 0x7ff5d4133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 monclient: found mon.noname-a 2024-09-15T21:20:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 -- 172.21.15.17:0/641523105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d4136c40 con 0x7ff5d409ffc0 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d27fc640 10 monclient: get_auth_request con 0x7ff5d40a0890 auth_method 0 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d27fc640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40a0890 0x7ff5d4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d2ffd640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d4133150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800c480 tx=0x7ff5c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8017070 con 0x7ff5d409ffc0 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: dump: 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c8005d10 con 0x7ff5d409ffc0 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 5 monclient: authenticate success, global_id 5067 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _renew_subs 2024-09-15T21:20:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d4136f20 con 0x7ff5d409ffc0 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80072b0 con 0x7ff5d409ffc0 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 10 monclient: dump: 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _renew_subs 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d413d1c0 con 0x7ff5d409ffc0 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d17fa640 4 set_mon_vals no callback set 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d40a0c70 con 0x7ff5d409ffc0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff5c801a070 con 0x7ff5d409ffc0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5c003b970 0x7ff5c003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7ff5c804f100 con 0x7ff5d409ffc0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d27fc640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5c003b970 0x7ff5c003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d27fc640 10 monclient: get_auth_request con 0x7ff5c003b970 auth_method 0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.725+0000 7ff5d27fc640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5c003b970 0x7ff5c003de30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40ad000 tx=0x7ff5c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.729+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff5c8011d80 con 0x7ff5d409ffc0 2024-09-15T21:20:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.729+0000 7ff5bb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.729+0000 7ff5bb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5d948f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5d948f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5d948f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff5d4134960 con 0x7ff5d409ffc0 2024-09-15T21:20:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5bb7fe640 1 -- 172.21.15.17:0/641523105 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7ff5c8007630 con 0x7ff5d409ffc0 2024-09-15T21:20:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5bb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.965+0000 7ff5bb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:02.972 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5c003b970 msgr2=0x7ff5c003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5c003b970 0x7ff5c003de30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40ad000 tx=0x7ff5c4008040 comp rx=0 tx=0).stop 2024-09-15T21:20:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 10 monclient: shutdown 2024-09-15T21:20:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 msgr2=0x7ff5d4133150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d4133150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800c480 tx=0x7ff5c800c940 comp rx=0 tx=0).stop 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 shutdown_connections 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5c003b970 0x7ff5c003de30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5d40a1380 0x7ff5d4133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40a0890 0x7ff5d4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 --2- 172.21.15.17:0/641523105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d409ffc0 0x7ff5d4133150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 >> 172.21.15.17:0/641523105 conn(0x7ff5d40a66a0 msgr2=0x7ff5d4130820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 shutdown_connections 2024-09-15T21:20:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:02.969+0000 7ff5d948f640 1 -- 172.21.15.17:0/641523105 wait complete. 2024-09-15T21:20:03.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:03.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:03.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:03.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:03.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:03.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:04.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:04.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:04.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:04.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:04.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:04.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:05.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:05.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:05.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:05.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:05.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:05.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:06.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.117+0000 7fba2b8fb640 10 monclient: get_monmap_and_config 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.117+0000 7fba2b8fb640 10 monclient: build_initial_monmap 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.117+0000 7fba2b8fb640 10 monclient: monmap: 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:06.120483+0000 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:06.120483+0000 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:06.123 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:06.124 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:06.124 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:06.124 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:06.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.117+0000 7fba2b8fb640 1 Processor -- start 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.117+0000 7fba2b8fb640 1 -- start start 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.117+0000 7fba2b8fb640 10 monclient: init 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba2409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): picked mon.noname-b con 0x7fba2409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fba2409dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): picked mon.noname-c con 0x7fba24134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba240a3490 con 0x7fba2409dd20 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fba240a1bc0 con 0x7fba24134d70 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fba240a0320 con 0x7fba2409d400 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29697640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba2409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba2b8fb640 10 monclient(hunting): authenticate will time out at 1129.516343s 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29697640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba2409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba28e96640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fba2409dd20 0x7fba24134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42086/0 (socket says 172.21.15.17:42086) 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 learned_addr learned my addr 172.21.15.17:0/3327408645 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 10 monclient(hunting): get_auth_request con 0x7fba24134d70 auth_method 0 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:20:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.121+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29697640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba2409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba2409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 10 monclient(hunting): get_auth_request con 0x7fba24134d70 auth_method 0 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:06.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:06.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:06.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:06.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.325+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:06.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:06.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:06.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:06.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:06.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:06.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 10 monclient(hunting): get_auth_request con 0x7fba24134d70 auth_method 0 2024-09-15T21:20:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29697640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba2409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba2409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:06.729+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29697640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba2409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba2409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:20:07.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:07.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 10 monclient(hunting): get_auth_request con 0x7fba24134d70 auth_method 0 2024-09-15T21:20:07.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:07.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:07.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:07.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:07.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:07.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:07.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:07.533+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:20:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:07.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:07.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:07.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:08.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:08.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:08.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:08.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:08.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:08.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:08.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:08.663+0000 7ffb6d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:09.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): tick 2024-09-15T21:20:09.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:20:09.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 monclient(hunting): continuing hunt 2024-09-15T21:20:09.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba2409d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba2409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 msgr2=0x7fba2413c8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.119:3300/0 conn(0x7fba24134d70 0x7fba2413c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 -- 172.21.15.17:0/3327408645 >> v1:172.21.15.17:6789/0 conn(0x7fba2409dd20 legacy=0x7fba24134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient: _add_conns ranks=[1,3] 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fba080036f0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): picked mon.noname-a con 0x7fba08007420 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 -- 172.21.15.17:0/3327408645 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba2409e7a0 con 0x7fba080036f0 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 1 -- 172.21.15.17:0/3327408645 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fba240a1bc0 con 0x7fba08007420 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba1affd640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 10 monclient(hunting): get_auth_request con 0x7fba08007420 auth_method 0 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:09.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.121+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:09.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:09.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 10 monclient(hunting): get_auth_request con 0x7fba08007420 auth_method 0 2024-09-15T21:20:09.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:09.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:09.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:09.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:20:09.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:09.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:09.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.325+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:09.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:09.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:09.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 10 monclient(hunting): get_auth_request con 0x7fba08007420 auth_method 0 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:09.729+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 10 monclient(hunting): get_auth_request con 0x7fba08007420 auth_method 0 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.533+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:20:10.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:10.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:10.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:10.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:10.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:10.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:10.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.681+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336752034 0 0) 0x7fba2409e7a0 con 0x7fba080036f0 2024-09-15T21:20:10.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.681+0000 7fba1bfff640 10 monclient(hunting): _negotiate 2024-09-15T21:20:10.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.681+0000 7fba1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:10.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.681+0000 7fba1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:10.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.681+0000 7fba1bfff640 10 monclient(hunting): my global_id is 34189 2024-09-15T21:20:10.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.681+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba10003530 con 0x7fba080036f0 2024-09-15T21:20:10.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1988026344 0 0) 0x7fba10003530 con 0x7fba080036f0 2024-09-15T21:20:10.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 msgr2=0x7fba080098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 1 --2- 172.21.15.17:0/3327408645 >> v2:172.21.15.17:3300/0 conn(0x7fba08007420 0x7fba080098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0800b010 con 0x7fba080036f0 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 10 monclient: _finish_auth 0 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.689+0000 7fba1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fba20004390 con 0x7fba080036f0 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba2b8fb640 5 monclient: authenticate success, global_id 34189 2024-09-15T21:20:10.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba2b8fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient: dump: 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:10.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 >> v1:172.21.15.119:6789/0 conn(0x7fba080036f0 legacy=0x7fba08005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 reap_dead start 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba100039c0 0x7fba10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): picked mon.a con 0x7fba100039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba10004810 0x7fba10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): picked mon.b con 0x7fba10004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): picked mon.c con 0x7fba2409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba240a1bc0 con 0x7fba100039c0 2024-09-15T21:20:10.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fba10005520 con 0x7fba10004810 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fba1000bf10 con 0x7fba2409d400 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba1bfff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba2b8fb640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29e98640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba1000b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba100039c0 0x7fba10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:10.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29e98640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba1000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba10004810 0x7fba10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): get_auth_request con 0x7fba100039c0 auth_method 0 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): get_auth_request con 0x7fba10004810 auth_method 0 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient(hunting): handle_auth_done global_id 34189 payload 293 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 msgr2=0x7fba1000b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba1000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba10004810 msgr2=0x7fba10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba10004810 0x7fba10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient: _finish_hunting 0 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 monclient: found mon.a 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 1 -- 172.21.15.17:0/3327408645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba10003530 con 0x7fba100039c0 2024-09-15T21:20:10.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient: _finish_auth 0 2024-09-15T21:20:10.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 10 monclient: _check_auth_tickets 2024-09-15T21:20:10.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba29697640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:10.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.693+0000 7fba28e96640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba10004810 0x7fba10004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba29697640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba100039c0 0x7fba10003dc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c490 tx=0x7fba2001d730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4060caeb23d23190 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000c0e0 con 0x7fba100039c0 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: dump: 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:10.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba2b8fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba200a9030 con 0x7fba100039c0 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba2b8fb640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba2b8fb640 4 set_mon_vals no callback set 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 1 -- 172.21.15.17:0/3327408645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20007020 con 0x7fba100039c0 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.697+0000 7fba1bfff640 10 monclient: dump: 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.708 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 10 monclient: shutdown 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 -- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba100039c0 msgr2=0x7fba10003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba100039c0 0x7fba10003dc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c490 tx=0x7fba2001d730 comp rx=0 tx=0).stop 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 -- 172.21.15.17:0/3327408645 shutdown_connections 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba10004810 0x7fba10004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba100039c0 0x7fba10003dc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/3327408645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba2409d400 0x7fba1000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 -- 172.21.15.17:0/3327408645 >> 172.21.15.17:0/3327408645 conn(0x7fba240a66a0 msgr2=0x7fba240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 -- 172.21.15.17:0/3327408645 shutdown_connections 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 -- 172.21.15.17:0/3327408645 wait complete. 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 10 monclient: build_initial_monmap 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 10 monclient: monmap: 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:10.710 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 Processor -- start 2024-09-15T21:20:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 1 -- start start 2024-09-15T21:20:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.701+0000 7fba2b8fb640 10 monclient: init 2024-09-15T21:20:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba100039c0 0x7fba241cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): picked mon.noname-b con 0x7fba100039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba10004810 0x7fba241cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): picked mon.noname-c con 0x7fba10004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 0x7fba24121010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): picked mon.noname-a con 0x7fba2409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba240a32b0 con 0x7fba2409d400 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fba240a3130 con 0x7fba100039c0 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fba2409ccd0 con 0x7fba10004810 2024-09-15T21:20:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient(hunting): authenticate will time out at 1134.100870s 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba28e96640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba10004810 msgr2=0x7fba241cbe30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba28e96640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba10004810 0x7fba241cbe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 0x7fba24121010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 0x7fba24121010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44282/0 (socket says 172.21.15.17:44282) 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 -- 172.21.15.17:0/552639133 learned_addr learned my addr 172.21.15.17:0/552639133 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): get_auth_request con 0x7fba2409d400 auth_method 0 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29697640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba100039c0 0x7fba241cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29697640 10 monclient(hunting): get_auth_request con 0x7fba100039c0 auth_method 0 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29697640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29697640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29697640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient(hunting): handle_auth_done global_id 5070 payload 824 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 -- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba10004810 msgr2=0x7fba241cbe30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba10004810 0x7fba241cbe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 -- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba100039c0 msgr2=0x7fba241cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba100039c0 0x7fba241cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient: _finish_hunting 0 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 monclient: found mon.noname-a 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 -- 172.21.15.17:0/552639133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba241343f0 con 0x7fba2409d400 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba29e98640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 0x7fba24121010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009610 tx=0x7fba1c0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c019020 con 0x7fba2409d400 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: dump: 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c007cd0 con 0x7fba2409d400 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c015630 con 0x7fba2409d400 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 5 monclient: authenticate success, global_id 5070 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _renew_subs 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2409d0f0 con 0x7fba2409d400 2024-09-15T21:20:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba1b7fe640 4 set_mon_vals no callback set 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 10 monclient: dump: 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _renew_subs 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba24124e30 con 0x7fba2409d400 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fba1c01c020 con 0x7fba2409d400 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba24134660 con 0x7fba2409d400 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fba0803c2f0 0x7fba0803e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.705+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7fba1c050340 con 0x7fba2409d400 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.709+0000 7fba29697640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fba0803c2f0 0x7fba0803e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.709+0000 7fba29697640 10 monclient: get_auth_request con 0x7fba0803c2f0 auth_method 0 2024-09-15T21:20:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.709+0000 7fba29697640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fba0803c2f0 0x7fba0803e7b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba200a9000 tx=0x7fba20006550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.713+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fba1c01bcd0 con 0x7fba2409d400 2024-09-15T21:20:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.713+0000 7fba19ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.713+0000 7fba19ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba2b8fb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba2b8fb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba2b8fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fba24124950 con 0x7fba2409d400 2024-09-15T21:20:10.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba19ffb640 1 -- 172.21.15.17:0/552639133 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7fba1c0500d0 con 0x7fba2409d400 2024-09-15T21:20:10.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba19ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:10.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.953+0000 7fba19ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:10.960 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":36,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fba0803c2f0 msgr2=0x7fba0803e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fba0803c2f0 0x7fba0803e7b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba200a9000 tx=0x7fba20006550 comp rx=0 tx=0).stop 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 10 monclient: shutdown 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 msgr2=0x7fba24121010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 0x7fba24121010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009610 tx=0x7fba1c0075c0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 shutdown_connections 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fba0803c2f0 0x7fba0803e7b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba2409d400 0x7fba24121010 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba10004810 0x7fba241cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 --2- 172.21.15.17:0/552639133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba100039c0 0x7fba241cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 >> 172.21.15.17:0/552639133 conn(0x7fba240a66a0 msgr2=0x7fba240a44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 shutdown_connections 2024-09-15T21:20:10.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:10.957+0000 7fba2b8fb640 1 -- 172.21.15.17:0/552639133 wait complete. 2024-09-15T21:20:10.980 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:20:10.980 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:20:10.981 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient: get_monmap_and_config 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient: build_initial_monmap 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient: monmap: 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:11.098930+0000 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:11.098930+0000 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:11.101 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:11.102 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:11.102 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:11.102 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:11.102 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 1 Processor -- start 2024-09-15T21:20:11.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 1 -- start start 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient: init 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5440a0890 0x7ff5440a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient(hunting): picked mon.noname-a con 0x7ff5440a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff54409ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff5440a11b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff544136490 con 0x7ff54409ffc0 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5440a1bf0 con 0x7ff5440a11b0 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff544073a50 con 0x7ff5440a0890 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54b903640 10 monclient(hunting): authenticate will time out at 1134.494631s 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5440a0890 0x7ff5440a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff548e9e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff54409ffc0 0x7ff544135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32924/0 (socket says 172.21.15.17:32924) 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff548e9e640 1 -- 172.21.15.17:0/3231221771 learned_addr learned my addr 172.21.15.17:0/3231221771 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): get_auth_request con 0x7ff5440a0890 auth_method 0 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient(hunting): handle_auth_done global_id 5073 payload 274 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 1 -- 172.21.15.17:0/3231221771 >> v1:172.21.15.119:6789/0 conn(0x7ff5440a11b0 legacy=0x7ff544135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 1 -- 172.21.15.17:0/3231221771 >> v1:172.21.15.17:6789/0 conn(0x7ff54409ffc0 legacy=0x7ff544135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 1 monclient: found mon.noname-a 2024-09-15T21:20:11.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.097+0000 7ff54969f640 1 -- 172.21.15.17:0/3231221771 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5440a1ed0 con 0x7ff5440a0890 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 1 --2- 172.21.15.17:0/3231221771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5440a0890 0x7ff5440a0c70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5400097c0 tx=0x7ff54001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14847af839390c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 1 -- 172.21.15.17:0/3231221771 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400b0070 con 0x7ff5440a0890 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: dump: 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.112 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 1 -- 172.21.15.17:0/3231221771 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5400ab600 con 0x7ff5440a0890 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 5 monclient: authenticate success, global_id 5073 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 4 set_mon_vals no callback set 2024-09-15T21:20:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 1 -- 172.21.15.17:0/3231221771 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400ab9d0 con 0x7ff5440a0890 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff53bfff640 10 monclient: dump: 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.114 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: shutdown 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- 172.21.15.17:0/3231221771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5440a0890 msgr2=0x7ff5440a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 --2- 172.21.15.17:0/3231221771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5440a0890 0x7ff5440a0c70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5400097c0 tx=0x7ff54001c360 comp rx=0 tx=0).stop 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- 172.21.15.17:0/3231221771 shutdown_connections 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 --2- 172.21.15.17:0/3231221771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5440a0890 0x7ff5440a0c70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- 172.21.15.17:0/3231221771 >> 172.21.15.17:0/3231221771 conn(0x7ff5440a66a0 msgr2=0x7ff5440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- 172.21.15.17:0/3231221771 shutdown_connections 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- 172.21.15.17:0/3231221771 wait complete. 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: build_initial_monmap 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: monmap: 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:11.115 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 Processor -- start 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- start start 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: init 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:11.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 0x7ff54411d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): picked mon.noname-a con 0x7ff54409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5440a0890 0x7ff5441352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): picked mon.noname-c con 0x7ff5440a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5440a11b0 0x7ff54413d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): picked mon.noname-b con 0x7ff5440a11b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441418a0 con 0x7ff54409ffc0 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff544141720 con 0x7ff5440a11b0 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff544141a20 con 0x7ff5440a0890 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54b903640 10 monclient(hunting): authenticate will time out at 1134.499686s 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff548e9e640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5440a0890 msgr2=0x7ff5441352b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff548e9e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5440a0890 0x7ff5441352b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 0x7ff54411d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 0x7ff54411d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44314/0 (socket says 172.21.15.17:44314) 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 1 -- 172.21.15.17:0/3028420795 learned_addr learned my addr 172.21.15.17:0/3028420795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): get_auth_request con 0x7ff54409ffc0 auth_method 0 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.101+0000 7ff54969f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 10 monclient(hunting): handle_auth_done global_id 5079 payload 824 2024-09-15T21:20:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff549ea0640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5440a11b0 0x7ff54413d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5440a0890 msgr2=0x7ff5441352b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5440a0890 0x7ff5441352b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5440a11b0 msgr2=0x7ff54413d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5440a11b0 0x7ff54413d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 monclient: found mon.noname-a 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54411ec30 con 0x7ff54409ffc0 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54969f640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 0x7ff54411d430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff5400ae040 tx=0x7ff540019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400b8030 con 0x7ff54409ffc0 2024-09-15T21:20:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: dump: 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5400b4410 con 0x7ff54409ffc0 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400b0040 con 0x7ff54409ffc0 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: dump: 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 5 monclient: authenticate success, global_id 5079 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 10 monclient: _renew_subs 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff53b7fe640 4 set_mon_vals no callback set 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5441ce850 con 0x7ff54409ffc0 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 10 monclient: _renew_subs 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54411ddc0 con 0x7ff54409ffc0 2024-09-15T21:20:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 0x7ff54402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440a0c70 con 0x7ff54402d450 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 0x7ff54402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient: get_auth_request con 0x7ff54402d450 auth_method 0 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff5400bb070 con 0x7ff54409ffc0 2024-09-15T21:20:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 10 monclient(hunting): handle_auth_done global_id 5082 payload 824 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff54411c8d0 con 0x7ff54402d450 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff52803b9a0 0x7ff52803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7ff5400e9760 con 0x7ff54409ffc0 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff548e9e640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 0x7ff54402d830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff534001ec0 tx=0x7ff53400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534017070 con 0x7ff54402d450 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: dump: 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff549ea0640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff52803b9a0 0x7ff52803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff549ea0640 10 monclient: get_auth_request con 0x7ff52803b9a0 auth_method 0 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff534005cf0 con 0x7ff54402d450 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 msgr2=0x7ff54402d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 0x7ff54402d830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff534001ec0 tx=0x7ff53400c810 comp rx=0 tx=0).stop 2024-09-15T21:20:11.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 msgr2=0x7ff54402d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:11.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff539ffb640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402d450 0x7ff54402d830 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff534001ec0 tx=0x7ff53400c810 comp rx=0 tx=0).stop 2024-09-15T21:20:11.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.105+0000 7ff549ea0640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff52803b9a0 0x7ff52803de60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff54411e5a0 tx=0x7ff53c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 0x7ff54402e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff54413a1c0 con 0x7ff54402dd70 2024-09-15T21:20:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 0x7ff54402e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient: get_auth_request con 0x7ff54402dd70 auth_method 0 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 10 monclient(hunting): handle_auth_done global_id 5085 payload 824 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 1 -- 172.21.15.17:0/3028420795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff54413e120 con 0x7ff54402dd70 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff548e9e640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 0x7ff54402e180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff534001ec0 tx=0x7ff534000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534017030 con 0x7ff54402dd70 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 10 monclient: dump: 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4703 (secure 0 0 0) 0x7ff534022450 con 0x7ff54402dd70 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 msgr2=0x7ff54402e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 0x7ff54402e180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff534001ec0 tx=0x7ff534000bd0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 msgr2=0x7ff54402e180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff539ffb640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54402dd70 0x7ff54402e180 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff534001ec0 tx=0x7ff534000bd0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:20:11.130 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 36, 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 176735, 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:11.131 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:11.132 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:11.133 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:11.134 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:11.135 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:11.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.141 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.142 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:11.143 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff52803b9a0 msgr2=0x7ff52803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff52803b9a0 0x7ff52803de60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff54411e5a0 tx=0x7ff53c009040 comp rx=0 tx=0).stop 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 10 monclient: shutdown 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 msgr2=0x7ff54411d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 0x7ff54411d430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff5400ae040 tx=0x7ff540019100 comp rx=0 tx=0).stop 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff549ea0640 1 -- 172.21.15.17:0/3028420795 reap_dead start 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 shutdown_connections 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 --2- 172.21.15.17:0/3028420795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff54409ffc0 0x7ff54411d430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 >> 172.21.15.17:0/3028420795 conn(0x7ff5440a66a0 msgr2=0x7ff5440a5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 shutdown_connections 2024-09-15T21:20:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.113+0000 7ff54b903640 1 -- 172.21.15.17:0/3028420795 wait complete. 2024-09-15T21:20:11.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:20:11.146 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:20:11.146 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:20:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient: get_monmap_and_config 2024-09-15T21:20:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient: build_initial_monmap 2024-09-15T21:20:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient: monmap: 2024-09-15T21:20:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:11.259394+0000 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:11.259394+0000 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:11.262 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 Processor -- start 2024-09-15T21:20:11.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 -- start start 2024-09-15T21:20:11.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient: init 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa8540a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 0x7fa85413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): picked mon.noname-b con 0x7fa85409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8540a1380 0x7fa85413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): picked mon.noname-c con 0x7fa8540a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540a3130 con 0x7fa8540a0890 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa854073df0 con 0x7fa8540a1380 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa8540a1bf0 con 0x7fa85409ffc0 2024-09-15T21:20:11.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85c142640 10 monclient(hunting): authenticate will time out at 1134.654764s 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa8596dd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 msgr2=0x7fa85413aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa8596dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 0x7fa85413aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa859ede640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa8540a0890 0x7fa8540a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32930/0 (socket says 172.21.15.17:32930) 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa859ede640 1 -- 172.21.15.17:0/4263382921 learned_addr learned my addr 172.21.15.17:0/4263382921 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 1 --2- 172.21.15.17:0/4263382921 >> v2:172.21.15.119:3300/0 conn(0x7fa8540a1380 0x7fa85413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): get_auth_request con 0x7fa8540a1380 auth_method 0 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1802713500 0 0) 0x7fa8540a3130 con 0x7fa8540a0890 2024-09-15T21:20:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): _negotiate 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): my global_id is 5088 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa840003530 con 0x7fa8540a0890 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2248703794 0 0) 0x7fa840003530 con 0x7fa8540a0890 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 msgr2=0x7fa85413aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 0x7fa85413aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 >> v2:172.21.15.119:3300/0 conn(0x7fa8540a1380 msgr2=0x7fa85413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 --2- 172.21.15.17:0/4263382921 >> v2:172.21.15.119:3300/0 conn(0x7fa8540a1380 0x7fa85413d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85409ccd0 con 0x7fa8540a0890 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa844001700 con 0x7fa8540a0890 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: dump: 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 >> v1:172.21.15.17:6789/0 conn(0x7fa8540a0890 legacy=0x7fa8540a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8400039c0 0x7fa840003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): picked mon.b con 0x7fa8400039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa840004810 0x7fa840004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): picked mon.a con 0x7fa840004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 0x7fa84000fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): picked mon.c con 0x7fa85409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa854073df0 con 0x7fa840004810 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540a1bf0 con 0x7fa8400039c0 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa8400055d0 con 0x7fa85409ffc0 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa858edc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 1 -- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 msgr2=0x7fa84000fde0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.257+0000 7fa85a6df640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 0x7fa84000fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa840004810 0x7fa840004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8400039c0 0x7fa840003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): get_auth_request con 0x7fa840004810 auth_method 0 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): get_auth_request con 0x7fa8400039c0 auth_method 0 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient(hunting): handle_auth_done global_id 5088 payload 293 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 -- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 msgr2=0x7fa84000fde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa85409ffc0 0x7fa84000fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 -- 172.21.15.17:0/4263382921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8400039c0 msgr2=0x7fa840003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8400039c0 0x7fa840003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 monclient: found mon.a 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 -- 172.21.15.17:0/4263382921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa840003530 con 0x7fa840004810 2024-09-15T21:20:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 5 monclient: authenticate success, global_id 5088 2024-09-15T21:20:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa8596dd640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa840004810 0x7fa840004c90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa850003990 tx=0x7fa85001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381f29b2ab59639c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850009520 con 0x7fa840004810 2024-09-15T21:20:11.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: dump: 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:11.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500039c0 con 0x7fa840004810 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 4 set_mon_vals no callback set 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 1 -- 172.21.15.17:0/4263382921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850003db0 con 0x7fa840004810 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa858edc640 10 monclient: dump: 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 10 monclient: shutdown 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 1 -- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa840004810 msgr2=0x7fa840004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 1 --2- 172.21.15.17:0/4263382921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa840004810 0x7fa840004c90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa850003990 tx=0x7fa85001c420 comp rx=0 tx=0).stop 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85a6df640 1 -- 172.21.15.17:0/4263382921 reap_dead start 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 1 -- 172.21.15.17:0/4263382921 shutdown_connections 2024-09-15T21:20:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 1 -- 172.21.15.17:0/4263382921 >> 172.21.15.17:0/4263382921 conn(0x7fa8540a66a0 msgr2=0x7fa8540a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 1 -- 172.21.15.17:0/4263382921 shutdown_connections 2024-09-15T21:20:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.261+0000 7fa85c142640 1 -- 172.21.15.17:0/4263382921 wait complete. 2024-09-15T21:20:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: build_initial_monmap 2024-09-15T21:20:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: monmap: 2024-09-15T21:20:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 Processor -- start 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- start start 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: init 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa840004810 0x7fa854120c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): picked mon.noname-b con 0x7fa840004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa854121170 0x7fa8541215b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): picked mon.noname-c con 0x7fa854121170 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa854121af0 0x7fa8541d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): picked mon.noname-a con 0x7fa854121af0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa854139650 con 0x7fa854121af0 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541394d0 con 0x7fa840004810 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa8541397d0 con 0x7fa854121170 2024-09-15T21:20:11.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): authenticate will time out at 1134.661599s 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa8596dd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa854121170 msgr2=0x7fa8541215b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa8596dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa854121170 0x7fa8541215b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa854121af0 0x7fa8541d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa854121af0 0x7fa8541d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44344/0 (socket says 172.21.15.17:44344) 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 -- 172.21.15.17:0/2853849137 learned_addr learned my addr 172.21.15.17:0/2853849137 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): get_auth_request con 0x7fa854121af0 auth_method 0 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa840004810 0x7fa854120c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): get_auth_request con 0x7fa840004810 auth_method 0 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient(hunting): handle_auth_done global_id 5091 payload 824 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa854121170 msgr2=0x7fa8541215b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa854121170 0x7fa8541215b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa840004810 msgr2=0x7fa854120c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa840004810 0x7fa854120c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 monclient: found mon.noname-a 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8541233c0 con 0x7fa854121af0 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa840004810 0x7fa854120c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85a6df640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa854121af0 0x7fa8541d19a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8540a1760 tx=0x7fa84c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c018020 con 0x7fa854121af0 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: dump: 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c014480 con 0x7fa854121af0 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 5 monclient: authenticate success, global_id 5091 2024-09-15T21:20:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c014850 con 0x7fa854121af0 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 10 monclient: dump: 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.293 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: _renew_subs 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84bfff640 4 set_mon_vals no callback set 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa854124510 con 0x7fa854121af0 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: _renew_subs 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85409d0f0 con 0x7fa854121af0 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 0x7fa85411c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa854138150 con 0x7fa85411c4e0 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 0x7fa85411c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient: get_auth_request con 0x7fa85411c4e0 auth_method 0 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa84c01b020 con 0x7fa854121af0 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.265+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa83803b9a0 0x7fa83803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7fa84c0507b0 con 0x7fa854121af0 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa859ede640 10 monclient(hunting): handle_auth_done global_id 34201 payload 824 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa859ede640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8540a3ed0 con 0x7fa85411c4e0 2024-09-15T21:20:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa8596dd640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa83803b9a0 0x7fa83803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa8596dd640 10 monclient: get_auth_request con 0x7fa83803b9a0 auth_method 0 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa859ede640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 0x7fa85411c8c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8440072c0 tx=0x7fa844007780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84401a070 con 0x7fa85411c4e0 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 10 monclient: dump: 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa8596dd640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa83803b9a0 0x7fa83803de60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa8500aa000 tx=0x7fa850005a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa8440119c0 con 0x7fa85411c4e0 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 msgr2=0x7fa85411c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 0x7fa85411c8c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8440072c0 tx=0x7fa844007780 comp rx=0 tx=0).stop 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 msgr2=0x7fa85411c8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.269+0000 7fa84a7fc640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411c4e0 0x7fa85411c8c0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8440072c0 tx=0x7fa844007780 comp rx=0 tx=0).stop 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.273+0000 7fa85c142640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.273+0000 7fa85c142640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 0x7fa8541d5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.273+0000 7fa85c142640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.273+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa854122490 con 0x7fa85411ce00 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 0x7fa8541d5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient: get_auth_request con 0x7fa85411ce00 auth_method 0 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 10 monclient(hunting): handle_auth_done global_id 34204 payload 824 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 1 -- 172.21.15.17:0/2853849137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa85406d020 con 0x7fa85411ce00 2024-09-15T21:20:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa859ede640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 0x7fa8541d5f00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa844017000 tx=0x7fa844001a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84401a050 con 0x7fa85411ce00 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 10 monclient: dump: 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.299 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7fa84401b040 con 0x7fa85411ce00 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 msgr2=0x7fa8541d5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 0x7fa8541d5f00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa844017000 tx=0x7fa844001a00 comp rx=0 tx=0).stop 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 msgr2=0x7fa8541d5f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa84a7fc640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85411ce00 0x7fa8541d5f00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 36, 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 23775, 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:11.302 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:11.303 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:11.304 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:11.305 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:11.308 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:11.309 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.311 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.312 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:11.313 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa83803b9a0 msgr2=0x7fa83803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa83803b9a0 0x7fa83803de60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa8500aa000 tx=0x7fa850005a90 comp rx=0 tx=0).stop 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 10 monclient: shutdown 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa854121af0 msgr2=0x7fa8541d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 --2- 172.21.15.17:0/2853849137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa854121af0 0x7fa8541d19a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8540a1760 tx=0x7fa84c00c490 comp rx=0 tx=0).stop 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85a6df640 1 -- 172.21.15.17:0/2853849137 reap_dead start 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85a6df640 1 -- 172.21.15.17:0/2853849137 reap_dead start 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 shutdown_connections 2024-09-15T21:20:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 >> 172.21.15.17:0/2853849137 conn(0x7fa8540a66a0 msgr2=0x7fa854136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 shutdown_connections 2024-09-15T21:20:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:11.277+0000 7fa85c142640 1 -- 172.21.15.17:0/2853849137 wait complete. 2024-09-15T21:20:11.316 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:20:11.316 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:20:11.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.663+0000 7ffb6d7fa640 10 monclient: tick 2024-09-15T21:20:11.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.663+0000 7ffb6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.663+0000 7ffb6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.667+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 v61) ==== 80+0+0 (secure 0 0 0) 0x7ffb740ab380 con 0x7ffb780a2320 2024-09-15T21:20:11.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.667+0000 7ffb6e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:11.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.667+0000 7ffb6e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:11.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb80197640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:11.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb80197640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:11.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb80197640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:11.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7ffb780008e0 con 0x7ffb780a2320 2024-09-15T21:20:11.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb6e7fc640 1 -- 172.21.15.119:0/2827022470 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v61) ==== 103+0+0 (secure 0 0 0) 0x7ffb740c0d80 con 0x7ffb780a2320 2024-09-15T21:20:11.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb6e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:11.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.775+0000 7ffb6e7fc640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-09-15T21:20:11.782 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo does not exist 2024-09-15T21:20:11.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5c03b9f0 msgr2=0x7ffb5c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5c03b9f0 0x7ffb5c03deb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb78137590 tx=0x7ffb68009040 comp rx=0 tx=0).stop 2024-09-15T21:20:11.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 10 monclient: shutdown 2024-09-15T21:20:11.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:11.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 msgr2=0x7ffb7813c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 0x7ffb7813c7f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffb74019f90 tx=0x7ffb7401aba0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 shutdown_connections 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5c03b9f0 0x7ffb5c03deb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb78137fe0 0x7ffb7813d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780a5570 0x7ffb7813cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 --2- 172.21.15.119:0/2827022470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb780a2320 0x7ffb7813c7f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 >> 172.21.15.119:0/2827022470 conn(0x7ffb780a6f60 msgr2=0x7ffb780a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 shutdown_connections 2024-09-15T21:20:11.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.779+0000 7ffb80197640 1 -- 172.21.15.119:0/2827022470 wait complete. 2024-09-15T21:20:11.799 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm foo 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.915+0000 7f2ca2274640 10 monclient: get_monmap_and_config 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.915+0000 7f2ca2274640 10 monclient: build_initial_monmap 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.915+0000 7f2ca2274640 10 monclient: monmap: 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:11.920680+0000 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:11.920680+0000 2024-09-15T21:20:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:11.923 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:11.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 Processor -- start 2024-09-15T21:20:11.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 -- start start 2024-09-15T21:20:11.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient: init 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c9c0a0900 0x7f2c9c0a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): picked mon.noname-a con 0x7f2c9c0a0900 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c9c0a1220 0x7f2c9c1356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): picked mon.noname-c con 0x7f2c9c0a1220 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2c9c135c20 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c9c0a2240 con 0x7f2c9c135c20 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9c09fdd0 con 0x7f2c9c0a0900 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9c0a1c80 con 0x7f2c9c0a1220 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2ca2274640 10 monclient(hunting): authenticate will time out at 1130.717316s 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c9c0a1220 0x7f2c9c1356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c9c0a1220 0x7f2c9c1356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33030/0 (socket says 172.21.15.119:33030) 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/3278418261 learned_addr learned my addr 172.21.15.119:0/3278418261 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): get_auth_request con 0x7f2c9c0a1220 auth_method 0 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9bfff640 1 --2- 172.21.15.119:0/3278418261 >> v2:172.21.15.17:3300/0 conn(0x7f2c9c0a0900 0x7f2c9c0a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9bfff640 10 monclient(hunting): get_auth_request con 0x7f2c9c0a0900 auth_method 0 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_done global_id 34207 payload 274 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/3278418261 >> v2:172.21.15.17:3300/0 conn(0x7f2c9c0a0900 msgr2=0x7f2c9c0a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 --2- 172.21.15.119:0/3278418261 >> v2:172.21.15.17:3300/0 conn(0x7f2c9c0a0900 0x7f2c9c0a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/3278418261 >> v1:172.21.15.17:6789/0 conn(0x7f2c9c135c20 legacy=0x7f2c9c136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:11.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.919+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/3278418261 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c0a1f60 con 0x7f2c9c0a1220 2024-09-15T21:20:11.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 --2- 172.21.15.119:0/3278418261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c1356e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c073620 tx=0x7f2c9001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1060d6c649e67eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 1 -- 172.21.15.119:0/3278418261 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900b1070 con 0x7f2c9c0a1220 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: dump: 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 1 -- 172.21.15.119:0/3278418261 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c900ac620 con 0x7f2c9c0a1220 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 1 -- 172.21.15.119:0/3278418261 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900ac9f0 con 0x7f2c9c0a1220 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 5 monclient: authenticate success, global_id 34207 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 4 set_mon_vals no callback set 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9affd640 10 monclient: dump: 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: shutdown 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:11.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- 172.21.15.119:0/3278418261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 msgr2=0x7f2c9c1356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 --2- 172.21.15.119:0/3278418261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c1356e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c073620 tx=0x7f2c9001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- 172.21.15.119:0/3278418261 shutdown_connections 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 --2- 172.21.15.119:0/3278418261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c1356e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 --2- 172.21.15.119:0/3278418261 >> v2:172.21.15.17:3300/0 conn(0x7f2c9c0a0900 0x7f2c9c0a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- 172.21.15.119:0/3278418261 >> 172.21.15.119:0/3278418261 conn(0x7f2c9c09be60 msgr2=0x7f2c9c09c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- 172.21.15.119:0/3278418261 shutdown_connections 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- 172.21.15.119:0/3278418261 wait complete. 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: build_initial_monmap 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: monmap: 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:11.937 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 Processor -- start 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- start start 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: init 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:11.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c9c0a0900 0x7f2c9c134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): picked mon.noname-a con 0x7f2c9c0a0900 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c13be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): picked mon.noname-b con 0x7f2c9c0a1220 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2c9c135c20 0x7f2c9c13c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): picked mon.noname-c con 0x7f2c9c135c20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c06ee90 con 0x7f2c9c0a0900 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c06ed10 con 0x7f2c9c0a1220 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9c0a0050 con 0x7f2c9c135c20 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:11.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca2274640 10 monclient(hunting): authenticate will time out at 1130.723344s 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c13be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca0811640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2c9c135c20 msgr2=0x7f2c9c13c340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c13be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33036/0 (socket says 172.21.15.119:33036) 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/1845447819 learned_addr learned my addr 172.21.15.119:0/1845447819 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): get_auth_request con 0x7f2c9c0a1220 auth_method 0 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2ca0811640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2c9c135c20 0x7f2c9c13c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:11.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9bfff640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c9c0a0900 0x7f2c9c134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9bfff640 10 monclient(hunting): get_auth_request con 0x7f2c9c0a0900 auth_method 0 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient(hunting): handle_auth_done global_id 34210 payload 824 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2c9c135c20 msgr2=0x7f2c9c13c340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2c9c135c20 0x7f2c9c13c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c9c0a0900 msgr2=0x7f2c9c134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c9c0a0900 0x7f2c9c134a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:20:11.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 -- 172.21.15.119:0/1845447819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c139020 con 0x7f2c9c0a1220 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.923+0000 7f2c9b7fe640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c13be00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c900af040 tx=0x7f2c90019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900bc070 con 0x7f2c9c0a1220 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: dump: 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:11.942 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: _finish_auth 0 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 5 monclient: authenticate success, global_id 34210 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 monclient: _renew_subs 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c900b7430 con 0x7f2c9c0a1220 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c13c880 con 0x7f2c9c0a1220 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:11.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900b1040 con 0x7f2c9c0a1220 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 10 monclient: dump: 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:11.944 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 monclient: _renew_subs 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c9a7fc640 4 set_mon_vals no callback set 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c135530 con 0x7f2c9c0a1220 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c0a0ce0 con 0x7f2c9c0a1220 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2c900bf070 con 0x7f2c9c0a1220 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c8803b970 0x7f2c8803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5162+0+0 (secure 0 0 0) 0x7f2c900e9590 con 0x7f2c9c0a1220 2024-09-15T21:20:11.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c9bfff640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c8803b970 0x7f2c8803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:11.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c9bfff640 10 monclient: get_auth_request con 0x7f2c8803b970 auth_method 0 2024-09-15T21:20:11.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.927+0000 7f2c9bfff640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c8803b970 0x7f2c8803de30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b930 tx=0x7f2c8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:11.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.931+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2c9c139020 con 0x7f2c9c0a1220 2024-09-15T21:20:11.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.931+0000 7f2c98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:11.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:11.931+0000 7f2c98ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:12.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.103+0000 7f2ca2274640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:12.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.103+0000 7f2ca2274640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:12.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.103+0000 7f2ca2274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.103+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7f2c9c139430 con 0x7f2c9c0a1220 2024-09-15T21:20:12.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.107+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v61) ==== 103+0+0 (secure 0 0 0) 0x7f2c900160f0 con 0x7f2c9c0a1220 2024-09-15T21:20:12.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.107+0000 7f2c98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:12.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.107+0000 7f2c98ff9640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-09-15T21:20:12.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2ca2274640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:12.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2ca2274640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:12.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2ca2274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7f2c9c06a810 con 0x7f2c9c0a1220 2024-09-15T21:20:12.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2c98ff9640 1 -- 172.21.15.119:0/1845447819 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v61) ==== 103+0+0 (secure 0 0 0) 0x7f2c900b6b80 con 0x7f2c9c0a1220 2024-09-15T21:20:12.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2c98ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T21:20:12.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.207+0000 7f2c98ff9640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-09-15T21:20:12.213 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo does not exist 2024-09-15T21:20:12.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c8803b970 msgr2=0x7f2c8803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c8803b970 0x7f2c8803de30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b930 tx=0x7f2c8c008040 comp rx=0 tx=0).stop 2024-09-15T21:20:12.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 10 monclient: shutdown 2024-09-15T21:20:12.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:12.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 msgr2=0x7f2c9c13be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c13be00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c900af040 tx=0x7f2c90019180 comp rx=0 tx=0).stop 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 shutdown_connections 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c8803b970 0x7f2c8803de30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2c9c135c20 0x7f2c9c13c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c9c0a1220 0x7f2c9c13be00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 --2- 172.21.15.119:0/1845447819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c9c0a0900 0x7f2c9c134a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 >> 172.21.15.119:0/1845447819 conn(0x7f2c9c09be60 msgr2=0x7f2c9c0a3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 shutdown_connections 2024-09-15T21:20:12.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.211+0000 7f2ca2274640 1 -- 172.21.15.119:0/1845447819 wait complete. 2024-09-15T21:20:12.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm bar 2024-09-15T21:20:12.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.339+0000 7f055d24b640 10 monclient: get_monmap_and_config 2024-09-15T21:20:12.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.339+0000 7f055d24b640 10 monclient: build_initial_monmap 2024-09-15T21:20:12.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.339+0000 7f055d24b640 10 monclient: monmap: 2024-09-15T21:20:12.345 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:12.345 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:12.343305+0000 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:12.343305+0000 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:12.346 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.339+0000 7f055d24b640 1 Processor -- start 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.339+0000 7f055d24b640 1 -- start start 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.339+0000 7f055d24b640 10 monclient: init 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05580a79e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05580a1100 0x7f055813cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): picked mon.noname-a con 0x7f05580a1100 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f055813d0a0 0x7f055813f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): picked mon.noname-c con 0x7f055813d0a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0558135180 con 0x7f05580a79e0 2024-09-15T21:20:12.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f05580a7dc0 con 0x7f05580a1100 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f05580a6920 con 0x7f055813d0a0 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055d24b640 10 monclient(hunting): authenticate will time out at 1131.140722s 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f055813d0a0 0x7f055813f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f055813d0a0 0x7f055813f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32856/0 (socket says 172.21.15.119:32856) 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 -- 172.21.15.119:0/1810433963 learned_addr learned my addr 172.21.15.119:0/1810433963 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): get_auth_request con 0x7f055813d0a0 auth_method 0 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:12.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055659c640 1 --2- 172.21.15.119:0/1810433963 >> v2:172.21.15.17:3300/0 conn(0x7f05580a1100 0x7f055813cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055659c640 10 monclient(hunting): get_auth_request con 0x7f05580a1100 auth_method 0 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient(hunting): handle_auth_done global_id 34213 payload 274 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 -- 172.21.15.119:0/1810433963 >> v2:172.21.15.17:3300/0 conn(0x7f05580a1100 msgr2=0x7f055813cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 --2- 172.21.15.119:0/1810433963 >> v2:172.21.15.17:3300/0 conn(0x7f05580a1100 0x7f055813cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 -- 172.21.15.119:0/1810433963 >> v1:172.21.15.17:6789/0 conn(0x7f05580a79e0 legacy=0x7f05580a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:12.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 monclient: found mon.noname-c 2024-09-15T21:20:12.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:12.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055759e640 1 -- 172.21.15.119:0/1810433963 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05580a1c30 con 0x7f055813d0a0 2024-09-15T21:20:12.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.343+0000 7f055659c640 1 --2- 172.21.15.119:0/1810433963 >> v2:172.21.15.17:3300/0 conn(0x7f05580a1100 0x7f055813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:12.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 --2- 172.21.15.119:0/1810433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f055813f490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05580a20e0 tx=0x7f055001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506730ed8bdf0639 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:12.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 1 -- 172.21.15.119:0/1810433963 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500a5820 con 0x7f055813d0a0 2024-09-15T21:20:12.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: dump: 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: _finish_auth 0 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 1 -- 172.21.15.119:0/1810433963 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05500a59c0 con 0x7f055813d0a0 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 1 -- 172.21.15.119:0/1810433963 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500a5d90 con 0x7f055813d0a0 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 5 monclient: authenticate success, global_id 34213 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 4 set_mon_vals no callback set 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0555d9b640 10 monclient: dump: 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: shutdown 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/1810433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 msgr2=0x7f055813f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 --2- 172.21.15.119:0/1810433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f055813f490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05580a20e0 tx=0x7f055001c940 comp rx=0 tx=0).stop 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/1810433963 shutdown_connections 2024-09-15T21:20:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 --2- 172.21.15.119:0/1810433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f055813f490 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 --2- 172.21.15.119:0/1810433963 >> v2:172.21.15.17:3300/0 conn(0x7f05580a1100 0x7f055813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/1810433963 >> 172.21.15.119:0/1810433963 conn(0x7f05580a6ef0 msgr2=0x7f05580a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/1810433963 shutdown_connections 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/1810433963 wait complete. 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: build_initial_monmap 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: monmap: 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 Processor -- start 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- start start 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: init 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05580a1100 0x7f0558072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): picked mon.noname-c con 0x7f05580a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05580a79e0 0x7f05581358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): picked mon.noname-a con 0x7f05580a79e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f0558135e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): picked mon.noname-b con 0x7f055813d0a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f05581348a0 con 0x7f05580a79e0 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0558134720 con 0x7f055813d0a0 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f05581421f0 con 0x7f05580a1100 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient(hunting): authenticate will time out at 1131.146725s 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f0558135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f0558135e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32860/0 (socket says 172.21.15.119:32860) 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 -- 172.21.15.119:0/3037047010 learned_addr learned my addr 172.21.15.119:0/3037047010 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): get_auth_request con 0x7f055813d0a0 auth_method 0 2024-09-15T21:20:12.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0556d9d640 1 -- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05580a1100 msgr2=0x7f0558072fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f0556d9d640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05580a1100 0x7f0558072fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055659c640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05580a79e0 0x7f05581358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient(hunting): handle_auth_done global_id 34216 payload 824 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 -- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05580a1100 msgr2=0x7f0558072fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05580a1100 0x7f0558072fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 -- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05580a79e0 msgr2=0x7f05581358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05580a79e0 0x7f05581358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 monclient: found mon.noname-b 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 -- 172.21.15.119:0/3037047010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0558137dc0 con 0x7f055813d0a0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055659c640 10 monclient: get_auth_request con 0x7f05580a79e0 auth_method 0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055659c640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05580a79e0 0x7f05581358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055759e640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f0558135e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05500af040 tx=0x7f05500196e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500bc020 con 0x7f055813d0a0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: dump: 2024-09-15T21:20:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05500b8690 con 0x7f055813d0a0 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 5 monclient: authenticate success, global_id 34216 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: _renew_subs 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0558133bc0 con 0x7f055813d0a0 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500b8a80 con 0x7f055813d0a0 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f053f7fe640 10 monclient: dump: 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.367 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: _renew_subs 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0558073a70 con 0x7f055813d0a0 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.347+0000 7f055559a640 4 set_mon_vals no callback set 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f055d24b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f055d24b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f055d24b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f05500bf020 con 0x7f055813d0a0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05580a4170 con 0x7f055813d0a0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f053f7fe640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f054403b970 0x7f054403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5162+0+0 (secure 0 0 0) 0x7f05500ead40 con 0x7f055813d0a0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f0556d9d640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f054403b970 0x7f054403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f0556d9d640 10 monclient: get_auth_request con 0x7f054403b970 auth_method 0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.351+0000 7f0556d9d640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f054403b970 0x7f054403de30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0548000bd0 tx=0x7f0548007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.355+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f05500c6030 con 0x7f055813d0a0 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.355+0000 7f053f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:12.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.355+0000 7f053f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.527+0000 7f055d24b640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T21:20:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.527+0000 7f055d24b640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T21:20:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.527+0000 7f055d24b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.527+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) -- 0x7f0558136340 con 0x7f055813d0a0 2024-09-15T21:20:12.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.667+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 v62) ==== 80+0+0 (secure 0 0 0) 0x7f05500165c0 con 0x7f055813d0a0 2024-09-15T21:20:12.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.667+0000 7f053f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T21:20:12.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.667+0000 7f053f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:12.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f055d24b640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T21:20:12.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f055d24b640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T21:20:12.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f055d24b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) -- 0x7f0558137a40 con 0x7f055813d0a0 2024-09-15T21:20:12.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f053f7fe640 1 -- 172.21.15.119:0/3037047010 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v62) ==== 103+0+0 (secure 0 0 0) 0x7f05500c0e20 con 0x7f055813d0a0 2024-09-15T21:20:12.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f053f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T21:20:12.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.767+0000 7f053f7fe640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-09-15T21:20:12.773 INFO:tasks.workunit.client.0.smithi119.stderr:rule bar does not exist 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f054403b970 msgr2=0x7f054403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f054403b970 0x7f054403de30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0548000bd0 tx=0x7f0548007a00 comp rx=0 tx=0).stop 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 10 monclient: shutdown 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 msgr2=0x7f0558135e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f0558135e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05500af040 tx=0x7f05500196e0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 shutdown_connections 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f054403b970 0x7f054403de30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f055813d0a0 0x7f0558135e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05580a79e0 0x7f05581358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 --2- 172.21.15.119:0/3037047010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05580a1100 0x7f0558072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 >> 172.21.15.119:0/3037047010 conn(0x7f05580a6ef0 msgr2=0x7f05580a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:12.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 shutdown_connections 2024-09-15T21:20:12.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.771+0000 7f055d24b640 1 -- 172.21.15.119:0/3037047010 wait complete. 2024-09-15T21:20:12.789 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create pinning_pool 1 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient: get_monmap_and_config 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient: build_initial_monmap 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient: monmap: 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:12.905469+0000 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:12.905469+0000 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:12.907 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.908 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:12.908 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:12.908 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:12.908 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:12.908 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:12.908 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 Processor -- start 2024-09-15T21:20:12.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 -- start start 2024-09-15T21:20:12.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient: init 2024-09-15T21:20:12.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:12.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2e00a2b00 0x7fb2e006ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): picked mon.noname-c con 0x7fb2e00a2b00 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e00a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): picked mon.noname-b con 0x7fb2e00a2ee0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb2e0137d90 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e0074bd0 con 0x7fb2e0137d90 2024-09-15T21:20:12.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e00a4160 con 0x7fb2e00a2b00 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e006ff70 con 0x7fb2e00a2ee0 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e6d11640 10 monclient(hunting): authenticate will time out at 1131.702985s 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e52ae640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb2e0137d90 0x7fb2e0138220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50140/0 (socket says 172.21.15.119:50140) 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e52ae640 1 -- 172.21.15.119:0/131373337 learned_addr learned my addr 172.21.15.119:0/131373337 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2d7fff640 1 -- 172.21.15.119:0/131373337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 msgr2=0x7fb2e00a32c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 1 --2- 172.21.15.119:0/131373337 >> v2:172.21.15.119:3300/0 conn(0x7fb2e00a2b00 0x7fb2e006ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2d7fff640 1 --2- 172.21.15.119:0/131373337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e00a32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): get_auth_request con 0x7fb2e00a2b00 auth_method 0 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:12.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient(hunting): handle_auth_done global_id 34219 payload 274 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 1 -- 172.21.15.119:0/131373337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 msgr2=0x7fb2e00a32c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 1 --2- 172.21.15.119:0/131373337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e00a32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 1 -- 172.21.15.119:0/131373337 >> v1:172.21.15.119:6789/0 conn(0x7fb2e0137d90 legacy=0x7fb2e0138220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient: _finish_hunting 0 2024-09-15T21:20:12.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 1 monclient: found mon.noname-c 2024-09-15T21:20:12.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:12.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.903+0000 7fb2e4aad640 1 -- 172.21.15.119:0/131373337 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00a4570 con 0x7fb2e00a2b00 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e4aad640 1 --2- 172.21.15.119:0/131373337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e00a2b00 0x7fb2e006ec30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0097c0 tx=0x7fb2dc01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832caf7348b4e256 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 1 -- 172.21.15.119:0/131373337 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0b0070 con 0x7fb2e00a2b00 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: dump: 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.919 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 1 -- 172.21.15.119:0/131373337 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0ab600 con 0x7fb2e00a2b00 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:12.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 1 -- 172.21.15.119:0/131373337 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0ab9d0 con 0x7fb2e00a2b00 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 5 monclient: authenticate success, global_id 34219 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 4 set_mon_vals no callback set 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2d77fe640 10 monclient: dump: 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.921 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 10 monclient: shutdown 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 -- 172.21.15.119:0/131373337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e00a2b00 msgr2=0x7fb2e006ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/131373337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e00a2b00 0x7fb2e006ec30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0097c0 tx=0x7fb2dc01c380 comp rx=0 tx=0).stop 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 -- 172.21.15.119:0/131373337 shutdown_connections 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/131373337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e00a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/131373337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e00a2b00 0x7fb2e006ec30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 -- 172.21.15.119:0/131373337 >> 172.21.15.119:0/131373337 conn(0x7fb2e00a8100 msgr2=0x7fb2e00a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 -- 172.21.15.119:0/131373337 shutdown_connections 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 -- 172.21.15.119:0/131373337 wait complete. 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 10 monclient: build_initial_monmap 2024-09-15T21:20:12.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 10 monclient: monmap: 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 Processor -- start 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 1 -- start start 2024-09-15T21:20:12.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 10 monclient: init 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.907+0000 7fb2e6d11640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2e00a2b00 0x7fb2e0136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): picked mon.noname-a con 0x7fb2e00a2b00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e0137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): picked mon.noname-c con 0x7fb2e00a2ee0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 0x7fb2e0139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): picked mon.noname-b con 0x7fb2e0137d90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0142a00 con 0x7fb2e00a2b00 2024-09-15T21:20:12.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0142880 con 0x7fb2e0137d90 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e0142b80 con 0x7fb2e00a2ee0 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient(hunting): authenticate will time out at 1131.708095s 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 0x7fb2e0139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d7fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 msgr2=0x7fb2e0137530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 0x7fb2e0139e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32880/0 (socket says 172.21.15.119:32880) 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e0137530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 -- 172.21.15.119:0/162496789 learned_addr learned my addr 172.21.15.119:0/162496789 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): get_auth_request con 0x7fb2e0137d90 auth_method 0 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:12.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2e00a2b00 0x7fb2e0136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 10 monclient(hunting): get_auth_request con 0x7fb2e00a2b00 auth_method 0 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient(hunting): handle_auth_done global_id 34225 payload 824 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 -- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 msgr2=0x7fb2e0137530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e0137530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 -- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2e00a2b00 msgr2=0x7fb2e0136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:12.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2e00a2b00 0x7fb2e0136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient: _finish_hunting 0 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 monclient: found mon.noname-b 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 -- 172.21.15.119:0/162496789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e009f2f0 con 0x7fb2e0137d90 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e52ae640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 0x7fb2e0139e90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800cb20 tx=0x7fb2d800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8017070 con 0x7fb2e0137d90 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: dump: 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.927 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 5 monclient: authenticate success, global_id 34225 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d800aca0 con 0x7fb2e0137d90 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:12.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8012600 con 0x7fb2e0137d90 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _renew_subs 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e013b810 con 0x7fb2e0137d90 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d67fc640 4 set_mon_vals no callback set 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 10 monclient: dump: 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:12.929 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _renew_subs 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e0137a70 con 0x7fb2e0137d90 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb2d801a070 con 0x7fb2e0137d90 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e006ec30 con 0x7fb2e0137d90 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb2cc03b970 0x7fb2cc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5107+0+0 (secure 0 0 0) 0x7fb2d804f870 con 0x7fb2e0137d90 2024-09-15T21:20:12.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb2cc03b970 0x7fb2cc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:12.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 10 monclient: get_auth_request con 0x7fb2cc03b970 auth_method 0 2024-09-15T21:20:12.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.911+0000 7fb2e4aad640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb2cc03b970 0x7fb2cc03de30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0ad000 tx=0x7fb2dc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:12.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.915+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb2d8010300 con 0x7fb2e0137d90 2024-09-15T21:20:12.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.915+0000 7fb2d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:12.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:12.915+0000 7fb2d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:13.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.075+0000 7fb2e6d11640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T21:20:13.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.075+0000 7fb2e6d11640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T21:20:13.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.075+0000 7fb2e6d11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.075+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) -- 0x7fb2e006a810 con 0x7fb2e0137d90 2024-09-15T21:20:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.691+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' created v63) ==== 127+0+0 (secure 0 0 0) 0x7fb2d80127a0 con 0x7fb2e0137d90 2024-09-15T21:20:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.691+0000 7fb2d57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T21:20:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.691+0000 7fb2d57fa640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' created 2024-09-15T21:20:13.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.787+0000 7fb2e6d11640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T21:20:13.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.787+0000 7fb2e6d11640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T21:20:13.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.787+0000 7fb2e6d11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.787+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) -- 0x7fb2e013e2f0 con 0x7fb2e0137d90 2024-09-15T21:20:13.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2d57fa640 1 -- 172.21.15.119:0/162496789 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' already exists v63) ==== 134+0+0 (secure 0 0 0) 0x7fb2d80113a0 con 0x7fb2e0137d90 2024-09-15T21:20:13.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2d57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T21:20:13.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2d57fa640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' already exists 2024-09-15T21:20:13.794 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'pinning_pool' already exists 2024-09-15T21:20:13.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb2cc03b970 msgr2=0x7fb2cc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:13.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb2cc03b970 0x7fb2cc03de30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0ad000 tx=0x7fb2dc0a6000 comp rx=0 tx=0).stop 2024-09-15T21:20:13.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 10 monclient: shutdown 2024-09-15T21:20:13.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:13.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 msgr2=0x7fb2e0139e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 0x7fb2e0139e90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800cb20 tx=0x7fb2d800a060 comp rx=0 tx=0).stop 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 shutdown_connections 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb2cc03b970 0x7fb2cc03de30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e0137d90 0x7fb2e0139e90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb2e00a2ee0 0x7fb2e0137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 --2- 172.21.15.119:0/162496789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2e00a2b00 0x7fb2e0136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 >> 172.21.15.119:0/162496789 conn(0x7fb2e00a8100 msgr2=0x7fb2e00a1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:13.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 shutdown_connections 2024-09-15T21:20:13.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.791+0000 7fb2e6d11640 1 -- 172.21.15.119:0/162496789 wait complete. 2024-09-15T21:20:13.811 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush rule rm replicated_rule 2024-09-15T21:20:13.811 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:13.811 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm replicated_rule 2024-09-15T21:20:13.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient: get_monmap_and_config 2024-09-15T21:20:13.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient: build_initial_monmap 2024-09-15T21:20:13.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient: monmap: 2024-09-15T21:20:13.918 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:13.918 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:13.917098+0000 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:13.917098+0000 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:13.919 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 Processor -- start 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 -- start start 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient: init 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e900a2b00 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:13.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e900a35f0 0x7f6e9006ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): picked mon.noname-b con 0x7f6e900a35f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e9013d040 0x7f6e9013f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): picked mon.noname-c con 0x7f6e9013d040 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e90074bd0 con 0x7f6e900a2b00 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6e900a4160 con 0x7f6e9013d040 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9006ff70 con 0x7f6e900a35f0 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e974f4640 10 monclient(hunting): authenticate will time out at 1132.713567s 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e94a8f640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e900a35f0 msgr2=0x7f6e9006ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e94a8f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e900a35f0 0x7f6e9006ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e9013d040 0x7f6e9013f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:13.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e9013d040 0x7f6e9013f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32884/0 (socket says 172.21.15.119:32884) 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 -- 172.21.15.119:0/2228924242 learned_addr learned my addr 172.21.15.119:0/2228924242 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): get_auth_request con 0x7f6e9013d040 auth_method 0 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient(hunting): handle_auth_done global_id 34228 payload 274 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 -- 172.21.15.119:0/2228924242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e900a35f0 msgr2=0x7f6e9006ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 --2- 172.21.15.119:0/2228924242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e900a35f0 0x7f6e9006ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 -- 172.21.15.119:0/2228924242 >> v1:172.21.15.119:6789/0 conn(0x7f6e900a2b00 legacy=0x7f6e900a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:13.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient: _finish_hunting 0 2024-09-15T21:20:13.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 monclient: found mon.noname-c 2024-09-15T21:20:13.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:13.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.915+0000 7f6e95a91640 1 -- 172.21.15.119:0/2228924242 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e900a4570 con 0x7f6e9013d040 2024-09-15T21:20:13.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95a91640 1 --2- 172.21.15.119:0/2228924242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9013d040 0x7f6e9013f430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009e50 tx=0x7f6e8801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de81216e6575ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:13.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 1 -- 172.21.15.119:0/2228924242 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a55d0 con 0x7f6e9013d040 2024-09-15T21:20:13.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:13.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:13.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: dump: 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:13.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 1 -- 172.21.15.119:0/2228924242 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e880a5770 con 0x7f6e9013d040 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 5 monclient: authenticate success, global_id 34228 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 1 -- 172.21.15.119:0/2228924242 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a5b40 con 0x7f6e9013d040 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 4 set_mon_vals no callback set 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e87fff640 10 monclient: dump: 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.930 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: shutdown 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/2228924242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9013d040 msgr2=0x7f6e9013f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 --2- 172.21.15.119:0/2228924242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9013d040 0x7f6e9013f430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009e50 tx=0x7f6e8801c910 comp rx=0 tx=0).stop 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/2228924242 shutdown_connections 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 --2- 172.21.15.119:0/2228924242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9013d040 0x7f6e9013f430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 --2- 172.21.15.119:0/2228924242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e900a35f0 0x7f6e9006ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/2228924242 >> 172.21.15.119:0/2228924242 conn(0x7f6e900a8100 msgr2=0x7f6e900a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/2228924242 shutdown_connections 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/2228924242 wait complete. 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: build_initial_monmap 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: monmap: 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:13.932 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 Processor -- start 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- start start 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: init 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e900a2b00 0x7f6e90139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): picked mon.noname-b con 0x7f6e900a2b00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e900a35f0 0x7f6e90139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): picked mon.noname-a con 0x7f6e900a35f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e9013d040 0x7f6e90139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): picked mon.noname-c con 0x7f6e9013d040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:13.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90142310 con 0x7f6e900a35f0 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90142190 con 0x7f6e900a2b00 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6e90142490 con 0x7f6e9013d040 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient(hunting): authenticate will time out at 1132.718348s 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e900a2b00 0x7f6e90139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e900a2b00 0x7f6e90139020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32890/0 (socket says 172.21.15.119:32890) 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 -- 172.21.15.119:0/1579459098 learned_addr learned my addr 172.21.15.119:0/1579459098 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): get_auth_request con 0x7f6e900a2b00 auth_method 0 2024-09-15T21:20:13.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95a91640 1 -- 172.21.15.119:0/1579459098 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e9013d040 msgr2=0x7f6e90139aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95a91640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e9013d040 0x7f6e90139aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient(hunting): handle_auth_done global_id 34231 payload 824 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 -- 172.21.15.119:0/1579459098 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e9013d040 msgr2=0x7f6e90139aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e9013d040 0x7f6e90139aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 -- 172.21.15.119:0/1579459098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e900a35f0 msgr2=0x7f6e90139560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e900a35f0 0x7f6e90139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient: _finish_hunting 0 2024-09-15T21:20:13.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 monclient: found mon.noname-b 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 -- 172.21.15.119:0/1579459098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9013bd60 con 0x7f6e900a2b00 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e95290640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e900a2b00 0x7f6e90139020 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c3c0 tx=0x7f6e8c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c017070 con 0x7f6e900a2b00 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: dump: 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.936 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e8c0052d0 con 0x7f6e900a2b00 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 5 monclient: authenticate success, global_id 34231 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _renew_subs 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/1579459098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9013f6d0 con 0x7f6e900a2b00 2024-09-15T21:20:13.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0054c0 con 0x7f6e900a2b00 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e85ffb640 10 monclient: dump: 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:13.938 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _renew_subs 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e877fe640 4 set_mon_vals no callback set 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/1579459098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9013ba80 con 0x7f6e900a2b00 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.919+0000 7f6e974f4640 1 -- 172.21.15.119:0/1579459098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9006ec30 con 0x7f6e900a2b00 2024-09-15T21:20:13.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.923+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6e8c01a070 con 0x7f6e900a2b00 2024-09-15T21:20:13.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.923+0000 7f6e85ffb640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e7403b970 0x7f6e7403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:13.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.923+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5458+0+0 (secure 0 0 0) 0x7f6e8c04efd0 con 0x7f6e900a2b00 2024-09-15T21:20:13.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.923+0000 7f6e94a8f640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e7403b970 0x7f6e7403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:13.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.923+0000 7f6e94a8f640 10 monclient: get_auth_request con 0x7f6e7403b970 auth_method 0 2024-09-15T21:20:13.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.923+0000 7f6e94a8f640 1 --2- 172.21.15.119:0/1579459098 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e7403b970 0x7f6e7403de30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e80006fd0 tx=0x7f6e80006d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:13.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.927+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6e8c011680 con 0x7f6e900a2b00 2024-09-15T21:20:13.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.927+0000 7f6e85ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:13.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:13.927+0000 7f6e85ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:14.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.083+0000 7f6e974f4640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-09-15T21:20:14.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.083+0000 7f6e974f4640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-09-15T21:20:14.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.083+0000 7f6e974f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.083+0000 7f6e974f4640 1 -- 172.21.15.119:0/1579459098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "replicated_rule"} v 0) -- 0x7f6e9013cb80 con 0x7f6e900a2b00 2024-09-15T21:20:14.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.087+0000 7f6e85ffb640 1 -- 172.21.15.119:0/1579459098 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "replicated_rule"}]=-16 crush rule replicated_rule (0) is in use v63) ==== 132+0+0 (secure 0 0 0) 0x7f6e8c01bbf0 con 0x7f6e900a2b00 2024-09-15T21:20:14.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.087+0000 7f6e85ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-09-15T21:20:14.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.087+0000 7f6e85ffb640 10 monclient: _finish_command 2 = mon:16 crush rule replicated_rule (0) is in use 2024-09-15T21:20:14.091 INFO:tasks.workunit.client.0.smithi119.stderr:Error EBUSY: crush rule replicated_rule (0) is in use 2024-09-15T21:20:14.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:14.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool rm pinning_pool pinning_pool --yes-i-really-really-mean-it 2024-09-15T21:20:14.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient: build_initial_monmap 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient: monmap: 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:14.201373+0000 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:14.201373+0000 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:14.203 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:14.204 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:14.204 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 Processor -- start 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 -- start start 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient: init 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7930135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f793009fe70 0x7f793013aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): picked mon.noname-a con 0x7f793009fe70 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793013d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): picked mon.noname-b con 0x7f793013afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:14.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79300a0300 con 0x7f7930135bf0 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f79300a3600 con 0x7f793009fe70 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f79300a1d80 con 0x7f793013afb0 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f793607f640 10 monclient(hunting): authenticate will time out at 1132.997932s 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792ffff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 msgr2=0x7f793013d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793013d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f793009fe70 0x7f793013aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7930135bf0 0x7f7930135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:45010/0 (socket says 172.21.15.119:45010) 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792f7fe640 1 -- 172.21.15.119:0/3506506905 learned_addr learned my addr 172.21.15.119:0/3506506905 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): get_auth_request con 0x7f793009fe70 auth_method 0 2024-09-15T21:20:14.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient(hunting): handle_auth_done global_id 5109 payload 274 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 1 -- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 msgr2=0x7f793013d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 1 --2- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793013d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 1 -- 172.21.15.119:0/3506506905 >> v1:172.21.15.17:6789/0 conn(0x7f7930135bf0 legacy=0x7f7930135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 1 monclient: found mon.noname-a 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:14.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:14.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.199+0000 7f792effd640 1 -- 172.21.15.119:0/3506506905 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79300a2060 con 0x7f793009fe70 2024-09-15T21:20:14.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 1 --2- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f793013aa70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79180097f0 tx=0x7f791801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398854d764d38b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:14.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 1 -- 172.21.15.119:0/3506506905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180b0070 con 0x7f793009fe70 2024-09-15T21:20:14.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: dump: 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:14.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 1 -- 172.21.15.119:0/3506506905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79180ab600 con 0x7f793009fe70 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 5 monclient: authenticate success, global_id 5109 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 1 -- 172.21.15.119:0/3506506905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180ab9d0 con 0x7f793009fe70 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 4 set_mon_vals no callback set 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792e7fc640 10 monclient: dump: 2024-09-15T21:20:14.218 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: shutdown 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 msgr2=0x7f793013aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 --2- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f793013aa70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79180097f0 tx=0x7f791801c380 comp rx=0 tx=0).stop 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- 172.21.15.119:0/3506506905 shutdown_connections 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 --2- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793013d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 --2- 172.21.15.119:0/3506506905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f793013aa70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- 172.21.15.119:0/3506506905 >> 172.21.15.119:0/3506506905 conn(0x7f79300a66e0 msgr2=0x7f79300a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- 172.21.15.119:0/3506506905 shutdown_connections 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- 172.21.15.119:0/3506506905 wait complete. 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: build_initial_monmap 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: monmap: 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:14.220 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 Processor -- start 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- start start 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: init 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f79301321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): picked mon.noname-a con 0x7f793009fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 0x7f7930139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): picked mon.noname-b con 0x7f7930135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793009d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): picked mon.noname-c con 0x7f793013afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7930140100 con 0x7f793009fe70 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f793006ed50 con 0x7f7930135bf0 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7930140280 con 0x7f793013afb0 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f793607f640 10 monclient(hunting): authenticate will time out at 1133.003574s 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792ffff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 msgr2=0x7f793009d550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:14.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793009d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 0x7f7930139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 0x7f7930139eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32892/0 (socket says 172.21.15.119:32892) 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 1 -- 172.21.15.119:0/1474872865 learned_addr learned my addr 172.21.15.119:0/1474872865 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): get_auth_request con 0x7f7930135bf0 auth_method 0 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792f7fe640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f79301321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792f7fe640 10 monclient(hunting): get_auth_request con 0x7f793009fe70 auth_method 0 2024-09-15T21:20:14.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.203+0000 7f792f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 10 monclient(hunting): handle_auth_done global_id 34234 payload 824 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 -- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 msgr2=0x7f793009d550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793009d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 -- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 msgr2=0x7f79301321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f79301321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 monclient: found mon.noname-b 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 -- 172.21.15.119:0/1474872865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793009f010 con 0x7f7930135bf0 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792f7fe640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f79301321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792effd640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 0x7f7930139eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79180ae040 tx=0x7f7918019120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180b8020 con 0x7f7930135bf0 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: dump: 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.225 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79180b4410 con 0x7f7930135bf0 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180b0040 con 0x7f7930135bf0 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 5 monclient: authenticate success, global_id 34234 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 monclient: _renew_subs 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793009d240 con 0x7f7930135bf0 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792dffb640 4 set_mon_vals no callback set 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 10 monclient: dump: 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 monclient: _renew_subs 2024-09-15T21:20:14.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7930137c00 con 0x7f7930135bf0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f79180bb020 con 0x7f7930135bf0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793013a450 con 0x7f7930135bf0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f791003b9a0 0x7f791003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5458+0+0 (secure 0 0 0) 0x7f79180e9af0 con 0x7f7930135bf0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792f7fe640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f791003b9a0 0x7f791003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792f7fe640 10 monclient: get_auth_request con 0x7f791003b9a0 auth_method 0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.207+0000 7f792f7fe640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f791003b9a0 0x7f791003de60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79200012f0 tx=0x7f7920009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.211+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f79180c2030 con 0x7f7930135bf0 2024-09-15T21:20:14.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.211+0000 7f7927fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:14.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.211+0000 7f7927fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:14.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.379+0000 7f793607f640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:20:14.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.379+0000 7f793607f640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:20:14.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.379+0000 7f793607f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.379+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7930137fc0 con 0x7f7930135bf0 2024-09-15T21:20:14.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.687+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' removed v64) ==== 172+0+0 (secure 0 0 0) 0x7f7918016090 con 0x7f7930135bf0 2024-09-15T21:20:14.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.687+0000 7f7927fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:20:14.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.687+0000 7f7927fff640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' removed 2024-09-15T21:20:14.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f793607f640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:20:14.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f793607f640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:20:14.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f793607f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7930136ca0 con 0x7f7930135bf0 2024-09-15T21:20:14.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f7927fff640 1 -- 172.21.15.119:0/1474872865 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' does not exist v64) ==== 179+0+0 (secure 0 0 0) 0x7f79180aa3a0 con 0x7f7930135bf0 2024-09-15T21:20:14.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f7927fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:20:14.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.783+0000 7f7927fff640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' does not exist 2024-09-15T21:20:14.790 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'pinning_pool' does not exist 2024-09-15T21:20:14.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f791003b9a0 msgr2=0x7f791003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f791003b9a0 0x7f791003de60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79200012f0 tx=0x7f7920009040 comp rx=0 tx=0).stop 2024-09-15T21:20:14.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 10 monclient: shutdown 2024-09-15T21:20:14.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 msgr2=0x7f7930139eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 0x7f7930139eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79180ae040 tx=0x7f7918019120 comp rx=0 tx=0).stop 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 shutdown_connections 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f791003b9a0 0x7f791003de60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f793013afb0 0x7f793009d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7930135bf0 0x7f7930139eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 --2- 172.21.15.119:0/1474872865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793009fe70 0x7f79301321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 >> 172.21.15.119:0/1474872865 conn(0x7f79300a66e0 msgr2=0x7f7930134ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 shutdown_connections 2024-09-15T21:20:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.787+0000 7f793607f640 1 -- 172.21.15.119:0/1474872865 wait complete. 2024-09-15T21:20:14.806 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush add-bucket foo osd 2024-09-15T21:20:14.806 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:14.806 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket foo osd 2024-09-15T21:20:14.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.919+0000 7f6a07bcc640 10 monclient: get_monmap_and_config 2024-09-15T21:20:14.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.919+0000 7f6a07bcc640 10 monclient: build_initial_monmap 2024-09-15T21:20:14.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.919+0000 7f6a07bcc640 10 monclient: monmap: 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:14.924137+0000 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:14.924137+0000 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:14.926 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 Processor -- start 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 -- start start 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient: init 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a000a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): picked mon.noname-b con 0x7f6a000a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a000a1220 0x7f6a001356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): picked mon.noname-c con 0x7f6a000a1220 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6a00135c20 0x7f6a001360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): picked mon.noname-a con 0x7f6a00135c20 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:14.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0009fdd0 con 0x7f6a00135c20 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6a00136610 con 0x7f6a000a1220 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6a000a2240 con 0x7f6a000a0900 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient(hunting): authenticate will time out at 1133.720751s 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05968640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 msgr2=0x7f6a000a0ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a000a1220 0x7f6a001356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05968640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a000a0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a000a1220 0x7f6a001356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32906/0 (socket says 172.21.15.119:32906) 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 -- 172.21.15.119:0/3605157728 learned_addr learned my addr 172.21.15.119:0/3605157728 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:14.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): get_auth_request con 0x7f6a000a1220 auth_method 0 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a06169640 1 --2- 172.21.15.119:0/3605157728 >> v2:172.21.15.17:3300/0 conn(0x7f6a00135c20 0x7f6a001360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a06169640 10 monclient(hunting): get_auth_request con 0x7f6a00135c20 auth_method 0 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a06169640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a06169640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a06169640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient(hunting): handle_auth_done global_id 34237 payload 274 2024-09-15T21:20:14.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 -- 172.21.15.119:0/3605157728 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 msgr2=0x7f6a000a0ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 --2- 172.21.15.119:0/3605157728 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a000a0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 -- 172.21.15.119:0/3605157728 >> v2:172.21.15.17:3300/0 conn(0x7f6a00135c20 msgr2=0x7f6a001360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 --2- 172.21.15.119:0/3605157728 >> v2:172.21.15.17:3300/0 conn(0x7f6a00135c20 0x7f6a001360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient: _finish_hunting 0 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 monclient: found mon.noname-c 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 -- 172.21.15.119:0/3605157728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a000a1c80 con 0x7f6a000a1220 2024-09-15T21:20:14.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a06169640 1 --2- 172.21.15.119:0/3605157728 >> v2:172.21.15.17:3300/0 conn(0x7f6a00135c20 0x7f6a001360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a05167640 1 --2- 172.21.15.119:0/3605157728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a000a1220 0x7f6a001356e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a0006a0c0 tx=0x7f69fc01c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=890dbcc6a5947d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 1 -- 172.21.15.119:0/3605157728 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0b1070 con 0x7f6a000a1220 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: dump: 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.938 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: _finish_auth 0 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: _check_auth_tickets 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 1 -- 172.21.15.119:0/3605157728 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc0ac410 con 0x7f6a000a1220 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 5 monclient: authenticate success, global_id 34237 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:14.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a07bcc640 4 set_mon_vals no callback set 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 1 -- 172.21.15.119:0/3605157728 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0ac7e0 con 0x7f6a000a1220 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.923+0000 7f6a04966640 10 monclient: dump: 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.940 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: shutdown 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/3605157728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a000a1220 msgr2=0x7f6a001356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- 172.21.15.119:0/3605157728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a000a1220 0x7f6a001356e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a0006a0c0 tx=0x7f69fc01c3d0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/3605157728 shutdown_connections 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- 172.21.15.119:0/3605157728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a000a1220 0x7f6a001356e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- 172.21.15.119:0/3605157728 >> v2:172.21.15.17:3300/0 conn(0x7f6a00135c20 0x7f6a001360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- 172.21.15.119:0/3605157728 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a000a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/3605157728 >> 172.21.15.119:0/3605157728 conn(0x7f6a0009be60 msgr2=0x7f6a0009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/3605157728 shutdown_connections 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/3605157728 wait complete. 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: build_initial_monmap 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: monmap: 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:14.942 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 Processor -- start 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- start start 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: init 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a0009e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): picked mon.noname-c con 0x7f6a000a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a000a1220 0x7f6a00139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): picked mon.noname-a con 0x7f6a000a1220 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a00135c20 0x7f6a0013a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): picked mon.noname-b con 0x7f6a00135c20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0006de90 con 0x7f6a000a1220 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0006ed10 con 0x7f6a00135c20 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0009ede0 con 0x7f6a000a0900 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient(hunting): authenticate will time out at 1133.725851s 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a00135c20 0x7f6a0013a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05968640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 msgr2=0x7f6a0009e140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a00135c20 0x7f6a0013a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32912/0 (socket says 172.21.15.119:32912) 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 -- 172.21.15.119:0/4262946521 learned_addr learned my addr 172.21.15.119:0/4262946521 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): get_auth_request con 0x7f6a00135c20 auth_method 0 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05968640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a0009e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05167640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a000a1220 0x7f6a00139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05167640 10 monclient(hunting): get_auth_request con 0x7f6a000a1220 auth_method 0 2024-09-15T21:20:14.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05167640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05167640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a05167640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient(hunting): handle_auth_done global_id 34240 payload 824 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 -- 172.21.15.119:0/4262946521 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 msgr2=0x7f6a0009e140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a000a0900 0x7f6a0009e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 -- 172.21.15.119:0/4262946521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a000a1220 msgr2=0x7f6a00139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a000a1220 0x7f6a00139d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient: _finish_hunting 0 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 monclient: found mon.noname-b 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 -- 172.21.15.119:0/4262946521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0009eb00 con 0x7f6a00135c20 2024-09-15T21:20:14.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a06169640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a00135c20 0x7f6a0013a2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009fa0 tx=0x7f69f800d560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f801a070 con 0x7f6a00135c20 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: dump: 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.947 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 5 monclient: authenticate success, global_id 34240 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f8003bc0 con 0x7f6a00135c20 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _renew_subs 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/4262946521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0013bc50 con 0x7f6a00135c20 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _renew_subs 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/4262946521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a00072d50 con 0x7f6a00135c20 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8004030 con 0x7f6a00135c20 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 10 monclient: dump: 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:14.949 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f7fff640 4 set_mon_vals no callback set 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f6a07bcc640 1 -- 172.21.15.119:0/4262946521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e8005090 con 0x7f6a00135c20 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f69f801d070 con 0x7f6a00135c20 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.927+0000 7f69f67fc640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69e403b970 0x7f69e403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.931+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5107+0+0 (secure 0 0 0) 0x7f69f804fff0 con 0x7f6a00135c20 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.931+0000 7f6a05968640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69e403b970 0x7f69e403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:14.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.931+0000 7f6a05968640 10 monclient: get_auth_request con 0x7f69e403b970 auth_method 0 2024-09-15T21:20:14.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.931+0000 7f6a05968640 1 --2- 172.21.15.119:0/4262946521 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69e403b970 0x7f69e403de30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69f0003940 tx=0x7f69f0006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:14.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.935+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f69f8010070 con 0x7f6a00135c20 2024-09-15T21:20:14.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.935+0000 7f69f67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:14.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:14.935+0000 7f69f67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:15.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f6a07bcc640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-09-15T21:20:15.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f6a07bcc640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-09-15T21:20:15.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f6a07bcc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f6a07bcc640 1 -- 172.21.15.119:0/4262946521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"} v 0) -- 0x7f69e8005650 con 0x7f6a00135c20 2024-09-15T21:20:15.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f69f67fc640 1 -- 172.21.15.119:0/4262946521 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}]=-22 type 'osd' is for devices, not buckets v64) ==== 136+0+0 (secure 0 0 0) 0x7f69f8013d10 con 0x7f6a00135c20 2024-09-15T21:20:15.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f69f67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-09-15T21:20:15.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.095+0000 7f69f67fc640 10 monclient: _finish_command 2 = mon:22 type 'osd' is for devices, not buckets 2024-09-15T21:20:15.101 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: type 'osd' is for devices, not buckets 2024-09-15T21:20:15.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:15.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket foo root 2024-09-15T21:20:15.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient: get_monmap_and_config 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient: build_initial_monmap 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient: monmap: 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:15.211291+0000 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:15.211291+0000 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:15.213 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 Processor -- start 2024-09-15T21:20:15.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 -- start start 2024-09-15T21:20:15.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient: init 2024-09-15T21:20:15.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:15.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2024-09-15T21:20:15.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c300a0900 0x7f4c300a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): picked mon.noname-c con 0x7f4c300a0900 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c300a1220 0x7f4c301356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): picked mon.noname-a con 0x7f4c300a1220 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c301360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): picked mon.noname-b con 0x7f4c30135c20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3009fdd0 con 0x7f4c300a1220 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4c301365f0 con 0x7f4c300a0900 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4c300a2240 con 0x7f4c30135c20 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:15.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.207+0000 7f4c37fb4640 10 monclient(hunting): authenticate will time out at 1134.007880s 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c300a0900 0x7f4c300a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c300a0900 0x7f4c300a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32914/0 (socket says 172.21.15.119:32914) 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c36551640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 msgr2=0x7f4c301360b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 -- 172.21.15.119:0/3468750767 learned_addr learned my addr 172.21.15.119:0/3468750767 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): get_auth_request con 0x7f4c300a0900 auth_method 0 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c36551640 1 --2- 172.21.15.119:0/3468750767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c301360b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:15.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c3554f640 1 --2- 172.21.15.119:0/3468750767 >> v2:172.21.15.17:3300/0 conn(0x7f4c300a1220 0x7f4c301356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c3554f640 10 monclient(hunting): get_auth_request con 0x7f4c300a1220 auth_method 0 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c3554f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c3554f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c3554f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient(hunting): handle_auth_done global_id 34243 payload 274 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 -- 172.21.15.119:0/3468750767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 msgr2=0x7f4c301360b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 --2- 172.21.15.119:0/3468750767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c301360b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 -- 172.21.15.119:0/3468750767 >> v2:172.21.15.17:3300/0 conn(0x7f4c300a1220 msgr2=0x7f4c301356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 --2- 172.21.15.119:0/3468750767 >> v2:172.21.15.17:3300/0 conn(0x7f4c300a1220 0x7f4c301356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient: _finish_hunting 0 2024-09-15T21:20:15.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 monclient: found mon.noname-c 2024-09-15T21:20:15.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:15.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 -- 172.21.15.119:0/3468750767 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c300a1c80 con 0x7f4c300a0900 2024-09-15T21:20:15.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c35d50640 1 --2- 172.21.15.119:0/3468750767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c300a0ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c3006a030 tx=0x7f4c2c01c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513d4c0fd9a5868e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:15.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 1 -- 172.21.15.119:0/3468750767 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b1070 con 0x7f4c300a0900 2024-09-15T21:20:15.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: dump: 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.224 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: _finish_auth 0 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: _check_auth_tickets 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 5 monclient: authenticate success, global_id 34243 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 1 -- 172.21.15.119:0/3468750767 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0ac410 con 0x7f4c300a0900 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 1 -- 172.21.15.119:0/3468750767 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0ac7e0 con 0x7f4c300a0900 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c34d4e640 10 monclient: dump: 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.225 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 4 set_mon_vals no callback set 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 10 monclient: shutdown 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 -- 172.21.15.119:0/3468750767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 msgr2=0x7f4c300a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/3468750767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c300a0ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c3006a030 tx=0x7f4c2c01c3d0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 -- 172.21.15.119:0/3468750767 shutdown_connections 2024-09-15T21:20:15.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/3468750767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c301360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/3468750767 >> v2:172.21.15.17:3300/0 conn(0x7f4c300a1220 0x7f4c301356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/3468750767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c300a0ce0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 -- 172.21.15.119:0/3468750767 >> 172.21.15.119:0/3468750767 conn(0x7f4c3009be60 msgr2=0x7f4c3009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 -- 172.21.15.119:0/3468750767 shutdown_connections 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 1 -- 172.21.15.119:0/3468750767 wait complete. 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 10 monclient: build_initial_monmap 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.211+0000 7f4c37fb4640 10 monclient: monmap: 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:15.227 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 Processor -- start 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- start start 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: init 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c3013ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): picked mon.noname-b con 0x7f4c300a0900 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c300a1220 0x7f4c30132060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): picked mon.noname-a con 0x7f4c300a1220 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c3009d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): picked mon.noname-c con 0x7f4c30135c20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3006ed10 con 0x7f4c300a1220 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3006df10 con 0x7f4c300a0900 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3006ee90 con 0x7f4c30135c20 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient(hunting): authenticate will time out at 1134.012759s 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c3013ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c3013ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32918/0 (socket says 172.21.15.119:32918) 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 -- 172.21.15.119:0/1037514118 learned_addr learned my addr 172.21.15.119:0/1037514118 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): get_auth_request con 0x7f4c300a0900 auth_method 0 2024-09-15T21:20:15.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c36551640 1 -- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 msgr2=0x7f4c3009d430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c36551640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c3009d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient(hunting): handle_auth_done global_id 34246 payload 824 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c3554f640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c300a1220 0x7f4c30132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 -- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 msgr2=0x7f4c3009d430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c3009d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 -- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c300a1220 msgr2=0x7f4c30132060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:15.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c300a1220 0x7f4c30132060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient: _finish_hunting 0 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 monclient: found mon.noname-b 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 -- 172.21.15.119:0/1037514118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3013b580 con 0x7f4c300a0900 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c35d50640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c3013ade0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0af040 tx=0x7f4c2c019170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0bc070 con 0x7f4c300a0900 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: dump: 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.231 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0ace20 con 0x7f4c300a0900 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 5 monclient: authenticate success, global_id 34246 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _renew_subs 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3013a660 con 0x7f4c300a0900 2024-09-15T21:20:15.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b1070 con 0x7f4c300a0900 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 10 monclient: dump: 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.233 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c27fff640 4 set_mon_vals no callback set 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _renew_subs 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3009d0f0 con 0x7f4c300a0900 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4c2c0ab7e0 con 0x7f4c300a0900 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c300a01c0 con 0x7f4c300a0900 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c1403b970 0x7f4c1403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5107+0+0 (secure 0 0 0) 0x7f4c2c0e95c0 con 0x7f4c300a0900 2024-09-15T21:20:15.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c3554f640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c1403b970 0x7f4c1403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c3554f640 10 monclient: get_auth_request con 0x7f4c1403b970 auth_method 0 2024-09-15T21:20:15.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.215+0000 7f4c3554f640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c1403b970 0x7f4c1403de30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c30139270 tx=0x7f4c20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:15.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.219+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4c2c016110 con 0x7f4c300a0900 2024-09-15T21:20:15.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.219+0000 7f4c267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:15.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.219+0000 7f4c267fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:15.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.379+0000 7f4c37fb4640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T21:20:15.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.379+0000 7f4c37fb4640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T21:20:15.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.379+0000 7f4c37fb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.379+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) -- 0x7f4c3009eef0 con 0x7f4c300a0900 2024-09-15T21:20:15.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.703+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 added bucket foo type root to crush map v65) ==== 138+0+0 (secure 0 0 0) 0x7f4c2c0b3d20 con 0x7f4c300a0900 2024-09-15T21:20:15.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.703+0000 7f4c267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T21:20:15.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.703+0000 7f4c267fc640 10 monclient: _finish_command 2 = system:0 added bucket foo type root to crush map 2024-09-15T21:20:15.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c37fb4640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T21:20:15.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c37fb4640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T21:20:15.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c37fb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) -- 0x7f4c3006a810 con 0x7f4c300a0900 2024-09-15T21:20:15.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c267fc640 1 -- 172.21.15.119:0/1037514118 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 bucket 'foo' already exists v65) ==== 126+0+0 (secure 0 0 0) 0x7f4c2c0c50a0 con 0x7f4c300a0900 2024-09-15T21:20:15.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c267fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T21:20:15.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.803+0000 7f4c267fc640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-09-15T21:20:15.808 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'foo' already exists 2024-09-15T21:20:15.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c1403b970 msgr2=0x7f4c1403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:15.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c1403b970 0x7f4c1403de30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c30139270 tx=0x7f4c20008040 comp rx=0 tx=0).stop 2024-09-15T21:20:15.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 10 monclient: shutdown 2024-09-15T21:20:15.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:15.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 msgr2=0x7f4c3013ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c3013ade0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0af040 tx=0x7f4c2c019170 comp rx=0 tx=0).stop 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 shutdown_connections 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c1403b970 0x7f4c1403de30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c30135c20 0x7f4c3009d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c300a1220 0x7f4c30132060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 --2- 172.21.15.119:0/1037514118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c300a0900 0x7f4c3013ade0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 >> 172.21.15.119:0/1037514118 conn(0x7f4c3009be60 msgr2=0x7f4c300a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 shutdown_connections 2024-09-15T21:20:15.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.807+0000 7f4c37fb4640 1 -- 172.21.15.119:0/1037514118 wait complete. 2024-09-15T21:20:15.825 INFO:tasks.workunit.client.0.smithi119.stderr:++ ceph osd create 2024-09-15T21:20:15.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient: get_monmap_and_config 2024-09-15T21:20:15.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient: build_initial_monmap 2024-09-15T21:20:15.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient: monmap: 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:15.936862+0000 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:15.936862+0000 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:15.940 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:15.941 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:15.941 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:15.941 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:15.941 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 Processor -- start 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 -- start start 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient: init 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45a009d4e0 0x7f45a009f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): picked mon.noname-c con 0x7f45a009d4e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d8c0 0x7f45a009dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): picked mon.noname-b con 0x7f45a009d8c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:15.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f45a01347b0 0x7f45a0134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): picked mon.noname-a con 0x7f45a01347b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f45a0074760 con 0x7f45a01347b0 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f45a00a09e0 con 0x7f45a009d4e0 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45a00a0f30 con 0x7f45a009d8c0 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a62ca640 10 monclient(hunting): authenticate will time out at 1134.733462s 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45a009d4e0 0x7f45a009f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459f7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d8c0 msgr2=0x7f45a009dcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45a009d4e0 0x7f45a009f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32922/0 (socket says 172.21.15.119:32922) 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 -- 172.21.15.119:0/3836919057 learned_addr learned my addr 172.21.15.119:0/3836919057 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): get_auth_request con 0x7f45a009d4e0 auth_method 0 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459f7fe640 1 --2- 172.21.15.119:0/3836919057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d8c0 0x7f45a009dcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a4867640 1 --2- 172.21.15.119:0/3836919057 >> v2:172.21.15.17:3300/0 conn(0x7f45a01347b0 0x7f45a0134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a4867640 10 monclient(hunting): get_auth_request con 0x7f45a01347b0 auth_method 0 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a4867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a4867640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f45a4867640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient(hunting): handle_auth_done global_id 34249 payload 274 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 -- 172.21.15.119:0/3836919057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d8c0 msgr2=0x7f45a009dcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 --2- 172.21.15.119:0/3836919057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d8c0 0x7f45a009dcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 -- 172.21.15.119:0/3836919057 >> v2:172.21.15.17:3300/0 conn(0x7f45a01347b0 msgr2=0x7f45a0134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 --2- 172.21.15.119:0/3836919057 >> v2:172.21.15.17:3300/0 conn(0x7f45a01347b0 0x7f45a0134bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 monclient: found mon.noname-c 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.935+0000 7f459ffff640 1 -- 172.21.15.119:0/3836919057 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a009e6e0 con 0x7f45a009d4e0 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459ffff640 1 --2- 172.21.15.119:0/3836919057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a009d4e0 0x7f45a009f410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45a00a3b60 tx=0x7f459401ead0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c8fb6f1cc3210a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 1 -- 172.21.15.119:0/3836919057 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940b4070 con 0x7f45a009d4e0 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: dump: 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.953 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: _finish_auth 0 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 1 -- 172.21.15.119:0/3836919057 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45940a7970 con 0x7f45a009d4e0 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 5 monclient: authenticate success, global_id 34249 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 1 -- 172.21.15.119:0/3836919057 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940a7d40 con 0x7f45a009d4e0 2024-09-15T21:20:15.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 4 set_mon_vals no callback set 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459effd640 10 monclient: dump: 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.955 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: shutdown 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/3836919057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a009d4e0 msgr2=0x7f45a009f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- 172.21.15.119:0/3836919057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a009d4e0 0x7f45a009f410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45a00a3b60 tx=0x7f459401ead0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/3836919057 shutdown_connections 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- 172.21.15.119:0/3836919057 >> v2:172.21.15.17:3300/0 conn(0x7f45a01347b0 0x7f45a0134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- 172.21.15.119:0/3836919057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d8c0 0x7f45a009dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- 172.21.15.119:0/3836919057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a009d4e0 0x7f45a009f410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/3836919057 >> 172.21.15.119:0/3836919057 conn(0x7f45a006cdc0 msgr2=0x7f45a006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/3836919057 shutdown_connections 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/3836919057 wait complete. 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: build_initial_monmap 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: monmap: 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:15.957 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 Processor -- start 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- start start 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: init 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d4e0 0x7f45a01336d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): picked mon.noname-c con 0x7f45a009d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45a009d8c0 0x7f45a0133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): picked mon.noname-a con 0x7f45a009d8c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 0x7f45a0139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): picked mon.noname-b con 0x7f45a01347b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:15.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0135520 con 0x7f45a009d8c0 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45a00a26d0 con 0x7f45a01347b0 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45a01356a0 con 0x7f45a009d4e0 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient(hunting): authenticate will time out at 1134.738625s 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 0x7f45a0139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459ffff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d4e0 msgr2=0x7f45a01336d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 0x7f45a0139650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32926/0 (socket says 172.21.15.119:32926) 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 -- 172.21.15.119:0/4271537215 learned_addr learned my addr 172.21.15.119:0/4271537215 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:15.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): get_auth_request con 0x7f45a01347b0 auth_method 0 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459ffff640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d4e0 0x7f45a01336d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459f7fe640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45a009d8c0 0x7f45a0133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459f7fe640 10 monclient(hunting): get_auth_request con 0x7f45a009d8c0 auth_method 0 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:15.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient(hunting): handle_auth_done global_id 34252 payload 824 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 -- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d4e0 msgr2=0x7f45a01336d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d4e0 0x7f45a01336d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 -- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45a009d8c0 msgr2=0x7f45a0133c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45a009d8c0 0x7f45a0133c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient: _finish_hunting 0 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 monclient: found mon.noname-b 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 -- 172.21.15.119:0/4271537215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a0120d50 con 0x7f45a01347b0 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a4867640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 0x7f45a0139650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f459800d580 tx=0x7f459800da40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980125f0 con 0x7f45a01347b0 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: dump: 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: _finish_auth 0 2024-09-15T21:20:15.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 5 monclient: authenticate success, global_id 34252 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598019070 con 0x7f45a01347b0 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: _renew_subs 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a01c2f20 con 0x7f45a01347b0 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980129d0 con 0x7f45a01347b0 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459cff9640 10 monclient: dump: 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:15.963 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f459e7fc640 4 set_mon_vals no callback set 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: _renew_subs 2024-09-15T21:20:15.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.939+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a01230b0 con 0x7f45a01347b0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f45a62ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f45a62ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f45a62ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a009dcd0 con 0x7f45a01347b0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4598012b70 con 0x7f45a01347b0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f459cff9640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f458c03b970 0x7f458c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5134+0+0 (secure 0 0 0) 0x7f4598050100 con 0x7f45a01347b0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f459ffff640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f458c03b970 0x7f458c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f459ffff640 10 monclient: get_auth_request con 0x7f458c03b970 auth_method 0 2024-09-15T21:20:15.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.943+0000 7f459ffff640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f458c03b970 0x7f458c03de30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f45940b1000 tx=0x7f4594008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:15.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.947+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f45a01c2f20 con 0x7f45a01347b0 2024-09-15T21:20:15.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.947+0000 7f459cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:15.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:15.947+0000 7f459cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:16.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.123+0000 7f45a62ca640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:20:16.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.123+0000 7f45a62ca640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T21:20:16.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.123+0000 7f45a62ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:16.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.123+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f45a006c440 con 0x7f45a01347b0 2024-09-15T21:20:16.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.703+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v66) ==== 58+0+2 (secure 0 0 0) 0x7f459800b360 con 0x7f45a01347b0 2024-09-15T21:20:16.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.703+0000 7f459cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T21:20:16.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.703+0000 7f459cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:16.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.819+0000 7f45a62ca640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:20:16.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.819+0000 7f45a62ca640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T21:20:16.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.819+0000 7f45a62ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:16.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:16.819+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f45a0122810 con 0x7f45a01347b0 2024-09-15T21:20:17.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.719+0000 7f459cff9640 1 -- 172.21.15.119:0/4271537215 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v67) ==== 58+0+2 (secure 0 0 0) 0x7f4598013390 con 0x7f45a01347b0 2024-09-15T21:20:17.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.719+0000 7f459cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T21:20:17.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.719+0000 7f459cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f458c03b970 msgr2=0x7f458c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f458c03b970 0x7f458c03de30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f45940b1000 tx=0x7f4594008040 comp rx=0 tx=0).stop 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 10 monclient: shutdown 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 msgr2=0x7f45a0139650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 0x7f45a0139650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f459800d580 tx=0x7f459800da40 comp rx=0 tx=0).stop 2024-09-15T21:20:17.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 shutdown_connections 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f458c03b970 0x7f458c03de30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a01347b0 0x7f45a0139650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45a009d8c0 0x7f45a0133c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 --2- 172.21.15.119:0/4271537215 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45a009d4e0 0x7f45a01336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 >> 172.21.15.119:0/4271537215 conn(0x7f45a006cdc0 msgr2=0x7f45a0073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 shutdown_connections 2024-09-15T21:20:17.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.723+0000 7f45a62ca640 1 -- 172.21.15.119:0/4271537215 wait complete. 2024-09-15T21:20:17.745 INFO:tasks.workunit.client.0.smithi119.stderr:+ o1=7 2024-09-15T21:20:17.745 INFO:tasks.workunit.client.0.smithi119.stderr:++ ceph osd create 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient: get_monmap_and_config 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient: build_initial_monmap 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient: monmap: 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:17.861524+0000 2024-09-15T21:20:17.863 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:17.861524+0000 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:17.864 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:17.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 Processor -- start 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 -- start start 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient: init 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f843009d4e0 0x7f843009f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): picked mon.noname-a con 0x7f843009d4e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f843009d8c0 0x7f843009dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:17.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): picked mon.noname-b con 0x7f843009d8c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f84301347b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84300a09e0 con 0x7f84301347b0 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8430074760 con 0x7f843009d4e0 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f84300a0f30 con 0x7f843009d8c0 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8437f29640 10 monclient(hunting): authenticate will time out at 1136.658742s 2024-09-15T21:20:17.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f84354c4640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f843009d8c0 msgr2=0x7f843009dcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f84354c4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f843009d8c0 0x7f843009dcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f84364c6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f84301347b0 0x7f8430134bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50152/0 (socket says 172.21.15.119:50152) 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f84364c6640 1 -- 172.21.15.119:0/452180035 learned_addr learned my addr 172.21.15.119:0/452180035 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 1 --2- 172.21.15.119:0/452180035 >> v2:172.21.15.17:3300/0 conn(0x7f843009d4e0 0x7f843009f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): get_auth_request con 0x7f843009d4e0 auth_method 0 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:17.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 1 -- 172.21.15.119:0/452180035 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551884071 0 0) 0x7f84300a09e0 con 0x7f84301347b0 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 10 monclient(hunting): _negotiate 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 10 monclient(hunting): my global_id is 34255 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 1 -- 172.21.15.119:0/452180035 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f841c003530 con 0x7f84301347b0 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient(hunting): handle_auth_done global_id 5133 payload 274 2024-09-15T21:20:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 1 -- 172.21.15.119:0/452180035 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 464085554 0 0) 0x7f841c003530 con 0x7f84301347b0 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 1 -- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f843009d8c0 msgr2=0x7f843009dcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 1 --2- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f843009d8c0 0x7f843009dcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 1 -- 172.21.15.119:0/452180035 >> v1:172.21.15.119:6789/0 conn(0x7f84301347b0 legacy=0x7f8430134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient: _finish_hunting 0 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 1 monclient: found mon.noname-a 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8435cc5640 1 -- 172.21.15.119:0/452180035 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843009e6e0 con 0x7f843009d4e0 2024-09-15T21:20:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.859+0000 7f8434cc3640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8435cc5640 1 --2- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843009f410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f842c0097c0 tx=0x7f842c01c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd3220f3d793d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 1 -- 172.21.15.119:0/452180035 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0b0070 con 0x7f843009d4e0 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: dump: 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: _finish_auth 0 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: _check_auth_tickets 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:17.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 5 monclient: authenticate success, global_id 5133 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 1 -- 172.21.15.119:0/452180035 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c0ab400 con 0x7f843009d4e0 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 4 set_mon_vals no callback set 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 1 -- 172.21.15.119:0/452180035 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428001a80 con 0x7f843009d4e0 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:17.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8434cc3640 10 monclient: dump: 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:17.884 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 10 monclient: shutdown 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 -- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 msgr2=0x7f843009f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 --2- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843009f410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f842c0097c0 tx=0x7f842c01c2a0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 -- 172.21.15.119:0/452180035 shutdown_connections 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 --2- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f843009d8c0 0x7f843009dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 --2- 172.21.15.119:0/452180035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843009f410 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 -- 172.21.15.119:0/452180035 >> 172.21.15.119:0/452180035 conn(0x7f843006cdc0 msgr2=0x7f843006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:17.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 -- 172.21.15.119:0/452180035 shutdown_connections 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 1 -- 172.21.15.119:0/452180035 wait complete. 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 10 monclient: build_initial_monmap 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.863+0000 7f8437f29640 10 monclient: monmap: 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:17.886 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 Processor -- start 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- start start 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: init 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:17.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843013ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): picked mon.noname-a con 0x7f843009d4e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 0x7f843011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): picked mon.noname-b con 0x7f843009d8c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84301347b0 0x7f8430133870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): picked mon.noname-c con 0x7f84301347b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:17.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f84300a2760 con 0x7f843009d4e0 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84300a25e0 con 0x7f843009d8c0 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f843013f520 con 0x7f84301347b0 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient(hunting): authenticate will time out at 1136.664688s 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 0x7f843011c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84364c6640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84301347b0 msgr2=0x7f8430133870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84364c6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84301347b0 0x7f8430133870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 0x7f843011c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32942/0 (socket says 172.21.15.119:32942) 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 -- 172.21.15.119:0/3130734577 learned_addr learned my addr 172.21.15.119:0/3130734577 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:17.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): get_auth_request con 0x7f843009d8c0 auth_method 0 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843013ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): get_auth_request con 0x7f843009d4e0 auth_method 0 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:17.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient(hunting): handle_auth_done global_id 34258 payload 824 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 -- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84301347b0 msgr2=0x7f8430133870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84301347b0 0x7f8430133870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 -- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 msgr2=0x7f843013ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843013ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient: _finish_hunting 0 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 monclient: found mon.noname-b 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 -- 172.21.15.119:0/3130734577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843013c540 con 0x7f843009d8c0 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84354c4640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 0x7f843011c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f842000b530 tx=0x7f842000b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420013020 con 0x7f843009d8c0 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: dump: 2024-09-15T21:20:17.892 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8420003df0 con 0x7f843009d8c0 2024-09-15T21:20:17.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84200052b0 con 0x7f843009d8c0 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 10 monclient: dump: 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:17.894 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 5 monclient: authenticate success, global_id 34258 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _renew_subs 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8427fff640 4 set_mon_vals no callback set 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84300a4740 con 0x7f843009d8c0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _renew_subs 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843011e970 con 0x7f843009d8c0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843011edb0 con 0x7f843009d8c0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8420016020 con 0x7f843009d8c0 2024-09-15T21:20:17.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f841403b970 0x7f841403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:17.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5364+0+0 (secure 0 0 0) 0x7f842004f8e0 con 0x7f843009d8c0 2024-09-15T21:20:17.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f841403b970 0x7f841403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:17.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 10 monclient: get_auth_request con 0x7f841403b970 auth_method 0 2024-09-15T21:20:17.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.867+0000 7f8435cc5640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f841403b970 0x7f841403de30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f842c0ad000 tx=0x7f842c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:17.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.871+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f842001d020 con 0x7f843009d8c0 2024-09-15T21:20:17.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.871+0000 7f84267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:17.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:17.871+0000 7f84267fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:18.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.039+0000 7f8437f29640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:20:18.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.039+0000 7f8437f29640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T21:20:18.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.039+0000 7f8437f29640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:18.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.039+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f843009f410 con 0x7f843009d8c0 2024-09-15T21:20:18.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.719+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v68) ==== 58+0+2 (secure 0 0 0) 0x7f842000d380 con 0x7f843009d8c0 2024-09-15T21:20:18.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.719+0000 7f84267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T21:20:18.725 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.719+0000 7f84267fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:18.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.831+0000 7f8437f29640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:20:18.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.831+0000 7f8437f29640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T21:20:18.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.831+0000 7f8437f29640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:18.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:18.831+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f84300008e0 con 0x7f843009d8c0 2024-09-15T21:20:19.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.427+0000 7f84267fc640 1 -- 172.21.15.119:0/3130734577 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v69) ==== 58+0+2 (secure 0 0 0) 0x7f842000fe10 con 0x7f843009d8c0 2024-09-15T21:20:19.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.427+0000 7f84267fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T21:20:19.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.427+0000 7f84267fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:19.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f841403b970 msgr2=0x7f841403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:19.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f841403b970 0x7f841403de30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f842c0ad000 tx=0x7f842c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:20:19.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 10 monclient: shutdown 2024-09-15T21:20:19.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 msgr2=0x7f843011c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 0x7f843011c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f842000b530 tx=0x7f842000b9f0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 shutdown_connections 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f841403b970 0x7f841403de30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84301347b0 0x7f8430133870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843009d8c0 0x7f843011c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 --2- 172.21.15.119:0/3130734577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f843009d4e0 0x7f843013ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 >> 172.21.15.119:0/3130734577 conn(0x7f843006cdc0 msgr2=0x7f8430073770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 shutdown_connections 2024-09-15T21:20:19.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.431+0000 7f8437f29640 1 -- 172.21.15.119:0/3130734577 wait complete. 2024-09-15T21:20:19.457 INFO:tasks.workunit.client.0.smithi119.stderr:+ o2=9 2024-09-15T21:20:19.457 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.571+0000 7fa1f0c1e640 10 monclient: get_monmap_and_config 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.571+0000 7fa1f0c1e640 10 monclient: build_initial_monmap 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.571+0000 7fa1f0c1e640 10 monclient: monmap: 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:19.575847+0000 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:19.575847+0000 2024-09-15T21:20:19.577 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:19.578 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 1 Processor -- start 2024-09-15T21:20:19.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 1 -- start start 2024-09-15T21:20:19.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient: init 2024-09-15T21:20:19.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:19.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-09-15T21:20:19.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 0x7fa1ec09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient(hunting): picked mon.noname-b con 0x7fa1ec135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa1ec135fd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa1ec136430 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ec0a1870 con 0x7fa1ec136430 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ec0a1d80 con 0x7fa1ec135fd0 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec0a3600 con 0x7fa1ec135bf0 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 10 monclient(hunting): authenticate will time out at 1138.372754s 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1ec135fd0 0x7fa1ec0a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50160/0 (socket says 172.21.15.119:50160) 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 -- 172.21.15.119:0/3759345070 learned_addr learned my addr 172.21.15.119:0/3759345070 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 1 -- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 msgr2=0x7fa1ec09fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 0x7fa1ec09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196794088 0 0) 0x7fa1ec0a1d80 con 0x7fa1ec135fd0 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): my global_id is 34261 2024-09-15T21:20:19.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1d8003530 con 0x7fa1ec135fd0 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3746188754 0 0) 0x7fa1d8003530 con 0x7fa1ec135fd0 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 msgr2=0x7fa1ec09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 0x7fa1ec09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 >> v1:172.21.15.17:6789/0 conn(0x7fa1ec136430 legacy=0x7fa1ec0a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec0a2060 con 0x7fa1ec135fd0 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: _finish_auth 0 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:19.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa1d4001700 con 0x7fa1ec135fd0 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 5 monclient: authenticate success, global_id 34261 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: dump: 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:19.586 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 >> v1:172.21.15.119:6789/0 conn(0x7fa1ec135fd0 legacy=0x7fa1ec0a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80039c0 0x7fa1d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): picked mon.a con 0x7fa1d80039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): picked mon.b con 0x7fa1d8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 0x7fa1d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): picked mon.c con 0x7fa1ec135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0a3600 con 0x7fa1d80039c0 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8005680 con 0x7fa1d8004810 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d800bf90 con 0x7fa1ec135bf0 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e9d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1f0c1e640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ead9d640 1 -- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 msgr2=0x7fa1d800b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ead9d640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 0x7fa1d800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): get_auth_request con 0x7fa1d8004810 auth_method 0 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:19.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80039c0 0x7fa1d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): get_auth_request con 0x7fa1d80039c0 auth_method 0 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient(hunting): handle_auth_done global_id 34261 payload 293 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 -- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 msgr2=0x7fa1d800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:19.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec135bf0 0x7fa1d800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 -- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80039c0 msgr2=0x7fa1d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80039c0 0x7fa1d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 monclient: found mon.b 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 1 -- 172.21.15.119:0/3759345070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d8003530 con 0x7fa1d8004810 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:19.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.575+0000 7fa1e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e3fff640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1d8004c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c240 tx=0x7fa1d401cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b25097e8c9240a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4007cf0 con 0x7fa1d8004810 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: dump: 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.596 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d40a9070 con 0x7fa1d8004810 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 4 set_mon_vals no callback set 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 1 -- 172.21.15.119:0/3759345070 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4005d20 con 0x7fa1d8004810 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:19.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e9d9b640 10 monclient: dump: 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: shutdown 2024-09-15T21:20:19.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/3759345070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 msgr2=0x7fa1d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/3759345070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1d8004c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c240 tx=0x7fa1d401cb50 comp rx=0 tx=0).stop 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ead9d640 1 -- 172.21.15.119:0/3759345070 reap_dead start 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/3759345070 shutdown_connections 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/3759345070 >> 172.21.15.119:0/3759345070 conn(0x7fa1ec0a66e0 msgr2=0x7fa1ec0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/3759345070 shutdown_connections 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/3759345070 wait complete. 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: build_initial_monmap 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: monmap: 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:19.599 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 Processor -- start 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- start start 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: init 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1ec1cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): picked mon.noname-b con 0x7fa1d8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:19.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec131cb0 0x7fa1ec132110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): picked mon.noname-c con 0x7fa1ec131cb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1ec13af00 0x7fa1ec13b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): picked mon.noname-a con 0x7fa1ec13af00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec06edb0 con 0x7fa1ec13af00 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec06ec30 con 0x7fa1d8004810 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec13f2b0 con 0x7fa1ec131cb0 2024-09-15T21:20:19.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1f0c1e640 10 monclient(hunting): authenticate will time out at 1138.379648s 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e3fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec131cb0 msgr2=0x7fa1ec132110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1ec1cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1e3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec131cb0 0x7fa1ec132110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1ec1cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32964/0 (socket says 172.21.15.119:32964) 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 1 -- 172.21.15.119:0/1749806536 learned_addr learned my addr 172.21.15.119:0/1749806536 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): get_auth_request con 0x7fa1d8004810 auth_method 0 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:19.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.579+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ead9d640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1ec13af00 0x7fa1ec13b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ead9d640 10 monclient(hunting): get_auth_request con 0x7fa1ec13af00 auth_method 0 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 10 monclient(hunting): handle_auth_done global_id 34264 payload 824 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 -- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec131cb0 msgr2=0x7fa1ec132110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec131cb0 0x7fa1ec132110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 -- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1ec13af00 msgr2=0x7fa1ec13b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1ec13af00 0x7fa1ec13b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 monclient: found mon.noname-b 2024-09-15T21:20:19.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:19.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:19.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 -- 172.21.15.119:0/1749806536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec11d750 con 0x7fa1d8004810 2024-09-15T21:20:19.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1ea59c640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1ec1cd830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0096b0 tx=0x7fa1dc009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:19.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc016070 con 0x7fa1d8004810 2024-09-15T21:20:19.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:19.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: dump: 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:19.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 5 monclient: authenticate success, global_id 34264 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1dc00e7b0 con 0x7fa1d8004810 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc00eba0 con 0x7fa1d8004810 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 10 monclient: dump: 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:19.606 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e959a640 4 set_mon_vals no callback set 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 monclient: _renew_subs 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec11d470 con 0x7fa1d8004810 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 monclient: _renew_subs 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec13d520 con 0x7fa1d8004810 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:19.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec0a0a50 con 0x7fa1d8004810 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa1dc019070 con 0x7fa1d8004810 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa1cc03b9a0 0x7fa1cc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5594+0+0 (secure 0 0 0) 0x7fa1dc04f870 con 0x7fa1d8004810 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e3fff640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa1cc03b9a0 0x7fa1cc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e3fff640 10 monclient: get_auth_request con 0x7fa1cc03b9a0 auth_method 0 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.583+0000 7fa1e3fff640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa1cc03b9a0 0x7fa1cc03de60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40ab000 tx=0x7fa1d40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.587+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa1dc019d80 con 0x7fa1d8004810 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.587+0000 7fa1e2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:19.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.587+0000 7fa1e2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:19.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.751+0000 7fa1f0c1e640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:19.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.751+0000 7fa1f0c1e640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:19.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.751+0000 7fa1f0c1e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:19.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:19.751+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7fa1ec06a830 con 0x7fa1d8004810 2024-09-15T21:20:20.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.735+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map v70) ==== 203+0+0 (secure 0 0 0) 0x7fa1dc022d20 con 0x7fa1d8004810 2024-09-15T21:20:20.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.735+0000 7fa1e2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:20.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.735+0000 7fa1e2ffd640 10 monclient: _finish_command 2 = system:0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map 2024-09-15T21:20:20.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1f0c1e640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:20.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1f0c1e640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:20.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1f0c1e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:20.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7fa1ec13b8d0 con 0x7fa1d8004810 2024-09-15T21:20:20.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1e2ffd640 1 -- 172.21.15.119:0/1749806536 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v70) ==== 201+0+0 (secure 0 0 0) 0x7fa1dc00ed40 con 0x7fa1d8004810 2024-09-15T21:20:20.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1e2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:20.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.843+0000 7fa1e2ffd640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T21:20:20.850 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa1cc03b9a0 msgr2=0x7fa1cc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa1cc03b9a0 0x7fa1cc03de60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40ab000 tx=0x7fa1d40a7070 comp rx=0 tx=0).stop 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 10 monclient: shutdown 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 msgr2=0x7fa1ec1cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1ec1cd830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0096b0 tx=0x7fa1dc009b70 comp rx=0 tx=0).stop 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 shutdown_connections 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa1cc03b9a0 0x7fa1cc03de60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1ec13af00 0x7fa1ec13b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:20.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa1ec131cb0 0x7fa1ec132110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:20.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 --2- 172.21.15.119:0/1749806536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1d8004810 0x7fa1ec1cd830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:20.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 >> 172.21.15.119:0/1749806536 conn(0x7fa1ec0a66e0 msgr2=0x7fa1ec0720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:20.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 shutdown_connections 2024-09-15T21:20:20.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.847+0000 7fa1f0c1e640 1 -- 172.21.15.119:0/1749806536 wait complete. 2024-09-15T21:20:20.867 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-09-15T21:20:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.983+0000 7ff3545f5640 10 monclient: get_monmap_and_config 2024-09-15T21:20:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.983+0000 7ff3545f5640 10 monclient: build_initial_monmap 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.983+0000 7ff3545f5640 10 monclient: monmap: 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:20.988294+0000 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:20.988294+0000 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:20.990 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:20.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 1 Processor -- start 2024-09-15T21:20:20.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 1 -- start start 2024-09-15T21:20:20.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient: init 2024-09-15T21:20:20.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:20.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2024-09-15T21:20:20.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 0x7ff34c137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient(hunting): picked mon.noname-b con 0x7ff34c0a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff34c0a41b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff34c0a4590 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c135120 con 0x7ff34c0a4590 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0a19d0 con 0x7ff34c0a41b0 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c0a4a40 con 0x7ff34c0a7ac0 2024-09-15T21:20:20.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 10 monclient(hunting): authenticate will time out at 1139.785400s 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff351b90640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff34c0a41b0 0x7ff34c134300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50164/0 (socket says 172.21.15.119:50164) 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff351b90640 1 -- 172.21.15.119:0/3934874409 learned_addr learned my addr 172.21.15.119:0/3934874409 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 msgr2=0x7ff34c137cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 0x7ff34c137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689383837 0 0) 0x7ff34c0a19d0 con 0x7ff34c0a41b0 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): _negotiate 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:20.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): my global_id is 34267 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff338003530 con 0x7ff34c0a41b0 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3680046358 0 0) 0x7ff338003530 con 0x7ff34c0a41b0 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 msgr2=0x7ff34c137cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 0x7ff34c137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 >> v1:172.21.15.17:6789/0 conn(0x7ff34c0a4590 legacy=0x7ff34c134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: _finish_hunting 0 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0a6960 con 0x7ff34c0a41b0 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: _finish_auth 0 2024-09-15T21:20:20.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: _check_auth_tickets 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff33c001700 con 0x7ff34c0a41b0 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: dump: 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.000 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 >> v1:172.21.15.119:6789/0 conn(0x7ff34c0a41b0 legacy=0x7ff34c134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff338003a10 0x7ff338003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): picked mon.a con 0x7ff338003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 0x7ff338004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): picked mon.c con 0x7ff34c0a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff338004f00 0x7ff33800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): picked mon.b con 0x7ff338004f00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0a4a40 con 0x7ff338003a10 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800bf10 con 0x7ff338004f00 2024-09-15T21:20:21.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff33800c090 con 0x7ff34c0a7ac0 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff35138f640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff338004f00 0x7ff33800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff351b90640 1 -- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 msgr2=0x7ff338004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): get_auth_request con 0x7ff338004f00 auth_method 0 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff351b90640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 0x7ff338004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:21.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff338003a10 0x7ff338003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 10 monclient(hunting): get_auth_request con 0x7ff338003a10 auth_method 0 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352391640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient(hunting): handle_auth_done global_id 34267 payload 293 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 -- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 msgr2=0x7ff338004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c0a7ac0 0x7ff338004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 -- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff338003a10 msgr2=0x7ff338003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff338003a10 0x7ff338003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient: _finish_hunting 0 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 monclient: found mon.b 2024-09-15T21:20:21.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 1 -- 172.21.15.119:0/3934874409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338003530 con 0x7ff338004f00 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient: _finish_auth 0 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 10 monclient: _check_auth_tickets 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff352b92640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 5 monclient: authenticate success, global_id 34267 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.987+0000 7ff3545f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352b92640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff338004f00 0x7ff33800b9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff344001e90 tx=0x7ff34401c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38009489fc2e4aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344001ae0 con 0x7ff338004f00 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: dump: 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.006 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.007 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.007 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.007 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.007 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.007 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff344003b80 con 0x7ff338004f00 2024-09-15T21:20:21.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:21.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:21.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:21.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 4 set_mon_vals no callback set 2024-09-15T21:20:21.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 1 -- 172.21.15.119:0/3934874409 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344005080 con 0x7ff338004f00 2024-09-15T21:20:21.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:21.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff35138f640 10 monclient: dump: 2024-09-15T21:20:21.009 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.009 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.009 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.010 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: shutdown 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3934874409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff338004f00 msgr2=0x7ff33800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3934874409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff338004f00 0x7ff33800b9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff344001e90 tx=0x7ff34401c750 comp rx=0 tx=0).stop 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352b92640 1 -- 172.21.15.119:0/3934874409 reap_dead start 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3934874409 shutdown_connections 2024-09-15T21:20:21.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3934874409 >> 172.21.15.119:0/3934874409 conn(0x7ff34c0a6fd0 msgr2=0x7ff34c0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:21.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3934874409 shutdown_connections 2024-09-15T21:20:21.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3934874409 wait complete. 2024-09-15T21:20:21.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: build_initial_monmap 2024-09-15T21:20:21.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: monmap: 2024-09-15T21:20:21.013 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:21.013 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:21.013 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:21.014 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 Processor -- start 2024-09-15T21:20:21.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- start start 2024-09-15T21:20:21.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: init 2024-09-15T21:20:21.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:21.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:21.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff34c070500 0x7ff34c073d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): picked mon.noname-a con 0x7ff34c070500 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:21.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 0x7ff34c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): picked mon.noname-b con 0x7ff34c06f390 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:21.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c06fd10 0x7ff34c1d6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): picked mon.noname-c con 0x7ff34c06fd10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:21.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c140d90 con 0x7ff34c070500 2024-09-15T21:20:21.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c140c10 con 0x7ff34c06f390 2024-09-15T21:20:21.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c140f10 con 0x7ff34c06fd10 2024-09-15T21:20:21.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:21.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient(hunting): authenticate will time out at 1139.791049s 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 0x7ff34c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 0x7ff34c06f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32974/0 (socket says 172.21.15.119:32974) 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 -- 172.21.15.119:0/3688091414 learned_addr learned my addr 172.21.15.119:0/3688091414 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): get_auth_request con 0x7ff34c06f390 auth_method 0 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352b92640 1 -- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c06fd10 msgr2=0x7ff34c1d6100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352b92640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c06fd10 0x7ff34c1d6100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:21.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient(hunting): handle_auth_done global_id 34270 payload 824 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352391640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff34c070500 0x7ff34c073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 -- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c06fd10 msgr2=0x7ff34c1d6100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c06fd10 0x7ff34c1d6100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 -- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff34c070500 msgr2=0x7ff34c073d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff34c070500 0x7ff34c073d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient: _finish_hunting 0 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 monclient: found mon.noname-b 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:21.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 -- 172.21.15.119:0/3688091414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c073450 con 0x7ff34c06f390 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352391640 10 monclient: get_auth_request con 0x7ff34c070500 auth_method 0 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff352391640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff34c070500 0x7ff34c073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff351b90640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 0x7ff34c06f7d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c580 tx=0x7ff33c00ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0075a0 con 0x7ff34c06f390 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: dump: 2024-09-15T21:20:21.027 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.028 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.028 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.028 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.028 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.028 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.028 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:21.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 5 monclient: authenticate success, global_id 34270 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33c0048b0 con 0x7ff34c06f390 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c007740 con 0x7ff34c06f390 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: _renew_subs 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c0701a0 con 0x7ff34c06f390 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff350b8e640 4 set_mon_vals no callback set 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:21.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff342ffd640 10 monclient: dump: 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: _renew_subs 2024-09-15T21:20:21.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.991+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c0721e0 con 0x7ff34c06f390 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff3545f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff3545f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff3545f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c134300 con 0x7ff34c06f390 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff33c011070 con 0x7ff34c06f390 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff342ffd640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff33003b970 0x7ff33003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5876+0+0 (secure 0 0 0) 0x7ff33c04f5c0 con 0x7ff34c06f390 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff352391640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff33003b970 0x7ff33003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff352391640 10 monclient: get_auth_request con 0x7ff33003b970 auth_method 0 2024-09-15T21:20:21.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.995+0000 7ff352391640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff33003b970 0x7ff33003de30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff348003940 tx=0x7ff348008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.999+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff34c073450 con 0x7ff34c06f390 2024-09-15T21:20:21.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.999+0000 7ff342ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:21.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:20.999+0000 7ff342ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:21.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.175+0000 7ff3545f5640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:21.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.175+0000 7ff3545f5640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:21.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.175+0000 7ff3545f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.175+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7ff34c06a760 con 0x7ff34c06f390 2024-09-15T21:20:21.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.179+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v70) ==== 201+0+0 (secure 0 0 0) 0x7ff33c01cbf0 con 0x7ff34c06f390 2024-09-15T21:20:21.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.179+0000 7ff342ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:21.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.179+0000 7ff342ffd640 10 monclient: _finish_command 2 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T21:20:21.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.279+0000 7ff3545f5640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:21.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.279+0000 7ff3545f5640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:21.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.279+0000 7ff3545f5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.279+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7ff34c072520 con 0x7ff34c06f390 2024-09-15T21:20:21.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff342ffd640 1 -- 172.21.15.119:0/3688091414 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v70) ==== 201+0+0 (secure 0 0 0) 0x7ff33c0192b0 con 0x7ff34c06f390 2024-09-15T21:20:21.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff342ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T21:20:21.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff342ffd640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T21:20:21.287 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T21:20:21.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff33003b970 msgr2=0x7ff33003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff33003b970 0x7ff33003de30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff348003940 tx=0x7ff348008040 comp rx=0 tx=0).stop 2024-09-15T21:20:21.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 10 monclient: shutdown 2024-09-15T21:20:21.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 msgr2=0x7ff34c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 0x7ff34c06f7d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c580 tx=0x7ff33c00ca40 comp rx=0 tx=0).stop 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 shutdown_connections 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff33003b970 0x7ff33003de30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff34c06fd10 0x7ff34c1d6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff34c06f390 0x7ff34c06f7d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 --2- 172.21.15.119:0/3688091414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff34c070500 0x7ff34c073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 >> 172.21.15.119:0/3688091414 conn(0x7ff34c0a6fd0 msgr2=0x7ff34c09fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 shutdown_connections 2024-09-15T21:20:21.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.283+0000 7ff3545f5640 1 -- 172.21.15.119:0/3688091414 wait complete. 2024-09-15T21:20:21.302 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-09-15T21:20:21.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient: get_monmap_and_config 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient: build_initial_monmap 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient: monmap: 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:21.422263+0000 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:21.422263+0000 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:21.424 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:21.425 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:21.425 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 Processor -- start 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 -- start start 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient: init 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f937c09d550 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): picked mon.noname-b con 0x7f937c09fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f937c09e040 0x7f937c13c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): picked mon.noname-c con 0x7f937c09e040 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f937c0a3600 con 0x7f937c09d550 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f937c09f1b0 con 0x7f937c09e040 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f937c0a2920 con 0x7f937c09fae0 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f9382f23640 10 monclient(hunting): authenticate will time out at 1140.219025s 2024-09-15T21:20:21.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f937bfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 msgr2=0x7f937c139c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f937c09e040 0x7f937c13c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f937bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f937c09e040 0x7f937c13c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32990/0 (socket says 172.21.15.119:32990) 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 -- 172.21.15.119:0/3081951360 learned_addr learned my addr 172.21.15.119:0/3081951360 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): get_auth_request con 0x7f937c09e040 auth_method 0 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient(hunting): handle_auth_done global_id 34273 payload 274 2024-09-15T21:20:21.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 -- 172.21.15.119:0/3081951360 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 msgr2=0x7f937c139c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 --2- 172.21.15.119:0/3081951360 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 -- 172.21.15.119:0/3081951360 >> v1:172.21.15.17:6789/0 conn(0x7f937c09d550 legacy=0x7f937c09d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient: _finish_hunting 0 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 monclient: found mon.noname-c 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.419+0000 7f93814c0640 1 -- 172.21.15.119:0/3081951360 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c0a1d60 con 0x7f937c09e040 2024-09-15T21:20:21.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f93814c0640 1 --2- 172.21.15.119:0/3081951360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09e040 0x7f937c13c1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f937c0a3550 tx=0x7f937401c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4303bfd7ca420621 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 1 -- 172.21.15.119:0/3081951360 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93740a5620 con 0x7f937c09e040 2024-09-15T21:20:21.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: dump: 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 5 monclient: authenticate success, global_id 34273 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 1 -- 172.21.15.119:0/3081951360 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93740a57c0 con 0x7f937c09e040 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 4 set_mon_vals no callback set 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 1 -- 172.21.15.119:0/3081951360 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93740a5b90 con 0x7f937c09e040 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f937b7fe640 10 monclient: dump: 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.438 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: shutdown 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 -- 172.21.15.119:0/3081951360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09e040 msgr2=0x7f937c13c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 --2- 172.21.15.119:0/3081951360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09e040 0x7f937c13c1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f937c0a3550 tx=0x7f937401c8d0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 -- 172.21.15.119:0/3081951360 shutdown_connections 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 --2- 172.21.15.119:0/3081951360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09e040 0x7f937c13c1c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 --2- 172.21.15.119:0/3081951360 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 -- 172.21.15.119:0/3081951360 >> 172.21.15.119:0/3081951360 conn(0x7f937c0a66e0 msgr2=0x7f937c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 -- 172.21.15.119:0/3081951360 shutdown_connections 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 -- 172.21.15.119:0/3081951360 wait complete. 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: build_initial_monmap 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: monmap: 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:21.440 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 Processor -- start 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 -- start start 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: init 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 0x7f937c1cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient(hunting): picked mon.noname-b con 0x7f937c09d550 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f937c09e040 0x7f937c132800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 10 monclient(hunting): picked mon.noname-a con 0x7f937c09e040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:21.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.423+0000 7f9382f23640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient(hunting): picked mon.noname-c con 0x7f937c09fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f937c13f0d0 con 0x7f937c09e040 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f937c13ef50 con 0x7f937c09d550 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f937c13f250 con 0x7f937c09fae0 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient(hunting): authenticate will time out at 1140.224031s 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 0x7f937c1cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 0x7f937c1cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33004/0 (socket says 172.21.15.119:33004) 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 -- 172.21.15.119:0/2877197637 learned_addr learned my addr 172.21.15.119:0/2877197637 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): get_auth_request con 0x7f937c09d550 auth_method 0 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93814c0640 1 -- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 msgr2=0x7f937c139a00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93814c0640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:21.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient(hunting): handle_auth_done global_id 34276 payload 824 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 -- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 msgr2=0x7f937c139a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 -- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f937c09e040 msgr2=0x7f937c132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f937bfff640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f937c09e040 0x7f937c132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f937c09e040 0x7f937c132800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient: _finish_hunting 0 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 monclient: found mon.noname-b 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 -- 172.21.15.119:0/2877197637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c09d240 con 0x7f937c09d550 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9380cbf640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 0x7f937c1cd4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f936c00c480 tx=0x7f936c00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c012830 con 0x7f937c09d550 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: dump: 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.445 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c017070 con 0x7f937c09d550 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c012c30 con 0x7f937c09d550 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 5 monclient: authenticate success, global_id 34276 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient: _renew_subs 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c13c3a0 con 0x7f937c09d550 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f937affd640 4 set_mon_vals no callback set 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:21.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 10 monclient: dump: 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient: _renew_subs 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c11e1c0 con 0x7f937c09d550 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c11c700 con 0x7f937c09d550 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f936c01c460 con 0x7f937c09d550 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f936803b970 0x7f936803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5876+0+0 (secure 0 0 0) 0x7f936c04fcb0 con 0x7f937c09d550 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f937bfff640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f936803b970 0x7f936803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f937bfff640 10 monclient: get_auth_request con 0x7f936803b970 auth_method 0 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.427+0000 7f937bfff640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f936803b970 0x7f936803de30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f937c11c920 tx=0x7f9370008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.431+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f936c01b3c0 con 0x7f937c09d550 2024-09-15T21:20:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.431+0000 7f93797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:21.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.431+0000 7f93797fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:21.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.599+0000 7f9382f23640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:21.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.599+0000 7f9382f23640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:21.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.599+0000 7f9382f23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.599+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7f937c09d930 con 0x7f937c09d550 2024-09-15T21:20:21.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.747+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map v71) ==== 203+0+0 (secure 0 0 0) 0x7f936c01ac80 con 0x7f937c09d550 2024-09-15T21:20:21.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.747+0000 7f93797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:21.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.747+0000 7f93797fa640 10 monclient: _finish_command 2 = system:0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map 2024-09-15T21:20:21.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.847+0000 7f9382f23640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:21.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.847+0000 7f9382f23640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:21.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.847+0000 7f9382f23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.847+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7f937c06a830 con 0x7f937c09d550 2024-09-15T21:20:21.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f93797fa640 1 -- 172.21.15.119:0/2877197637 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v71) ==== 201+0+0 (secure 0 0 0) 0x7f936c0102e0 con 0x7f937c09d550 2024-09-15T21:20:21.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f93797fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:21.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f93797fa640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T21:20:21.856 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f936803b970 msgr2=0x7f936803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f936803b970 0x7f936803de30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f937c11c920 tx=0x7f9370008040 comp rx=0 tx=0).stop 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 10 monclient: shutdown 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 msgr2=0x7f937c1cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 0x7f937c1cd4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f936c00c480 tx=0x7f936c00c940 comp rx=0 tx=0).stop 2024-09-15T21:20:21.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 shutdown_connections 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.851+0000 7f9382f23640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f936803b970 0x7f936803de30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.855+0000 7f9382f23640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f937c09fae0 0x7f937c139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.855+0000 7f9382f23640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f937c09e040 0x7f937c132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.855+0000 7f9382f23640 1 --2- 172.21.15.119:0/2877197637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f937c09d550 0x7f937c1cd4a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.855+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 >> 172.21.15.119:0/2877197637 conn(0x7f937c0a66e0 msgr2=0x7f937c0a56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.855+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 shutdown_connections 2024-09-15T21:20:21.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.855+0000 7f9382f23640 1 -- 172.21.15.119:0/2877197637 wait complete. 2024-09-15T21:20:21.872 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-09-15T21:20:21.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient: get_monmap_and_config 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient: build_initial_monmap 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient: monmap: 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:21.986501+0000 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:21.986501+0000 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:21.989 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:21.989 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 Processor -- start 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 -- start start 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient: init 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8eb40a7ac0 0x7f8eb4137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): picked mon.noname-c con 0x7f8eb40a7ac0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8eb40a41b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8eb40a45c0 0x7f8eb4134a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:21.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): picked mon.noname-a con 0x7f8eb40a45c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb40a19d0 con 0x7f8eb40a41b0 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40a7ea0 con 0x7f8eb40a45c0 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40a6960 con 0x7f8eb40a7ac0 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.983+0000 7f8eba3e6640 10 monclient(hunting): authenticate will time out at 1140.783995s 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8eb40a7ac0 0x7f8eb4137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8eb40a7ac0 0x7f8eb4137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33010/0 (socket says 172.21.15.119:33010) 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eab7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8eb40a41b0 0x7f8eb4134300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50176/0 (socket says 172.21.15.119:50176) 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 -- 172.21.15.119:0/2369655220 learned_addr learned my addr 172.21.15.119:0/2369655220 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): get_auth_request con 0x7f8eb40a7ac0 auth_method 0 2024-09-15T21:20:21.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb8983640 1 --2- 172.21.15.119:0/2369655220 >> v2:172.21.15.17:3300/0 conn(0x7f8eb40a45c0 0x7f8eb4134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb8983640 10 monclient(hunting): get_auth_request con 0x7f8eb40a45c0 auth_method 0 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb8983640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb8983640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb8983640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient(hunting): handle_auth_done global_id 34279 payload 274 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 -- 172.21.15.119:0/2369655220 >> v2:172.21.15.17:3300/0 conn(0x7f8eb40a45c0 msgr2=0x7f8eb4134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 --2- 172.21.15.119:0/2369655220 >> v2:172.21.15.17:3300/0 conn(0x7f8eb40a45c0 0x7f8eb4134a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:21.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 -- 172.21.15.119:0/2369655220 >> v1:172.21.15.119:6789/0 conn(0x7f8eb40a41b0 legacy=0x7f8eb4134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:21.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:21.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 monclient: found mon.noname-c 2024-09-15T21:20:21.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:21.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:21.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 -- 172.21.15.119:0/2369655220 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb40a6c40 con 0x7f8eb40a7ac0 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb3fff640 1 --2- 172.21.15.119:0/2369655220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a7ac0 0x7f8eb4137cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009870 tx=0x7f8ea401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be55812245cecbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 1 -- 172.21.15.119:0/2369655220 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40b0070 con 0x7f8eb40a7ac0 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: dump: 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:21.999 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 5 monclient: authenticate success, global_id 34279 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 1 -- 172.21.15.119:0/2369655220 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea40ab600 con 0x7f8eb40a7ac0 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 1 -- 172.21.15.119:0/2369655220 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40ab9d0 con 0x7f8eb40a7ac0 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 4 set_mon_vals no callback set 2024-09-15T21:20:22.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eb37fe640 10 monclient: dump: 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 10 monclient: shutdown 2024-09-15T21:20:22.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 -- 172.21.15.119:0/2369655220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a7ac0 msgr2=0x7f8eb4137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/2369655220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a7ac0 0x7f8eb4137cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009870 tx=0x7f8ea401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 -- 172.21.15.119:0/2369655220 shutdown_connections 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/2369655220 >> v2:172.21.15.17:3300/0 conn(0x7f8eb40a45c0 0x7f8eb4134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/2369655220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a7ac0 0x7f8eb4137cd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 -- 172.21.15.119:0/2369655220 >> 172.21.15.119:0/2369655220 conn(0x7f8eb40a6fd0 msgr2=0x7f8eb40a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 -- 172.21.15.119:0/2369655220 shutdown_connections 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.987+0000 7f8eba3e6640 1 -- 172.21.15.119:0/2369655220 wait complete. 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: build_initial_monmap 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: monmap: 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:22.002 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 Processor -- start 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- start start 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: init 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8eb40a41b0 0x7f8eb406f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): picked mon.noname-a con 0x7f8eb40a41b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 0x7f8eb406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): picked mon.noname-b con 0x7f8eb40a45c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8eb40a7ac0 0x7f8eb406fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): picked mon.noname-c con 0x7f8eb40a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41350d0 con 0x7f8eb40a41b0 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4134f50 con 0x7f8eb40a45c0 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb41423c0 con 0x7f8eb40a7ac0 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient(hunting): authenticate will time out at 1140.788916s 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 0x7f8eb406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 0x7f8eb406f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33020/0 (socket says 172.21.15.119:33020) 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 -- 172.21.15.119:0/3051075705 learned_addr learned my addr 172.21.15.119:0/3051075705 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): get_auth_request con 0x7f8eb40a45c0 auth_method 0 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb8983640 1 -- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8eb40a7ac0 msgr2=0x7f8eb406fe10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb8983640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8eb40a7ac0 0x7f8eb406fe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb3fff640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8eb40a41b0 0x7f8eb406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient(hunting): handle_auth_done global_id 34285 payload 824 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 -- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8eb40a7ac0 msgr2=0x7f8eb406fe10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8eb40a7ac0 0x7f8eb406fe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 -- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8eb40a41b0 msgr2=0x7f8eb406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8eb40a41b0 0x7f8eb406f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 monclient: found mon.noname-b 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 -- 172.21.15.119:0/3051075705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb4070ba0 con 0x7f8eb40a45c0 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eab7fe640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 0x7f8eb406f8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00c480 tx=0x7f8e9c00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c018070 con 0x7f8eb40a45c0 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: dump: 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:22.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e9c007480 con 0x7f8eb40a45c0 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c004a10 con 0x7f8eb40a45c0 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 10 monclient: dump: 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 5 monclient: authenticate success, global_id 34285 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _renew_subs 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb2ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb4072f00 con 0x7f8eb40a45c0 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _renew_subs 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb4071700 con 0x7f8eb40a45c0 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:22.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb4077180 con 0x7f8eb40a45c0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8e9c004bb0 con 0x7f8eb40a45c0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8e9403b970 0x7f8e9403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6120+0+0 (secure 0 0 0) 0x7f8e9c04f230 con 0x7f8eb40a45c0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb3fff640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8e9403b970 0x7f8e9403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb3fff640 10 monclient: get_auth_request con 0x7f8e9403b970 auth_method 0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.991+0000 7f8eb3fff640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8e9403b970 0x7f8e9403de30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40ad000 tx=0x7f8ea4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:22.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.995+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8e9c01bb10 con 0x7f8eb40a45c0 2024-09-15T21:20:22.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.995+0000 7f8eb17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:22.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:21.995+0000 7f8eb17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:22.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eba3e6640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:22.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eba3e6640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:22.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eba3e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7f8eb406a760 con 0x7f8eb40a45c0 2024-09-15T21:20:22.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v71) ==== 201+0+0 (secure 0 0 0) 0x7f8e9c010300 con 0x7f8eb40a45c0 2024-09-15T21:20:22.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eb17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:22.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.155+0000 7f8eb17fa640 10 monclient: _finish_command 2 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T21:20:22.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eba3e6640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:22.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eba3e6640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:22.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eba3e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7f8eb4073660 con 0x7f8eb40a45c0 2024-09-15T21:20:22.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eb17fa640 1 -- 172.21.15.119:0/3051075705 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v71) ==== 201+0+0 (secure 0 0 0) 0x7f8e9c00e9d0 con 0x7f8eb40a45c0 2024-09-15T21:20:22.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eb17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T21:20:22.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.259+0000 7f8eb17fa640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T21:20:22.264 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T21:20:22.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8e9403b970 msgr2=0x7f8e9403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8e9403b970 0x7f8e9403de30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40ad000 tx=0x7f8ea4008040 comp rx=0 tx=0).stop 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 10 monclient: shutdown 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 msgr2=0x7f8eb406f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 0x7f8eb406f8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00c480 tx=0x7f8e9c00c940 comp rx=0 tx=0).stop 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 shutdown_connections 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8e9403b970 0x7f8e9403de30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8eb40a7ac0 0x7f8eb406fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40a45c0 0x7f8eb406f8d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 --2- 172.21.15.119:0/3051075705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8eb40a41b0 0x7f8eb406f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 >> 172.21.15.119:0/3051075705 conn(0x7f8eb40a6fd0 msgr2=0x7f8eb4132200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 shutdown_connections 2024-09-15T21:20:22.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.263+0000 7f8eba3e6640 1 -- 172.21.15.119:0/3051075705 wait complete. 2024-09-15T21:20:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket bar root 2024-09-15T21:20:22.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.383+0000 7f8ff8ab7640 10 monclient: get_monmap_and_config 2024-09-15T21:20:22.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.383+0000 7f8ff8ab7640 10 monclient: build_initial_monmap 2024-09-15T21:20:22.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.383+0000 7f8ff8ab7640 10 monclient: monmap: 2024-09-15T21:20:22.389 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:22.389 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.389 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:22.388049+0000 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:22.388049+0000 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:22.390 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 1 Processor -- start 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 1 -- start start 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient: init 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ff4132750 0x7f8ff4132b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff4132750 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8ff406f8c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ff4133070 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:22.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff40a3170 con 0x7f8ff4133070 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff406dbf0 con 0x7f8ff406f8c0 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff406f580 con 0x7f8ff4132750 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient(hunting): authenticate will time out at 1141.185239s 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ff4132750 0x7f8ff4132b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ff4132750 0x7f8ff4132b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48972/0 (socket says 172.21.15.119:48972) 2024-09-15T21:20:22.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 -- 172.21.15.119:0/1688841502 learned_addr learned my addr 172.21.15.119:0/1688841502 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): get_auth_request con 0x7f8ff4132750 auth_method 0 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient(hunting): handle_auth_done global_id 34288 payload 274 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 -- 172.21.15.119:0/1688841502 >> v1:172.21.15.119:6789/0 conn(0x7f8ff406f8c0 legacy=0x7f8ff40a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 -- 172.21.15.119:0/1688841502 >> v1:172.21.15.17:6789/0 conn(0x7f8ff4133070 legacy=0x7f8ff40a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 monclient: found mon.noname-c 2024-09-15T21:20:22.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:22.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 -- 172.21.15.119:0/1688841502 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff4072840 con 0x7f8ff4132750 2024-09-15T21:20:22.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff259c640 1 --2- 172.21.15.119:0/1688841502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff4132750 0x7f8ff4132b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098a0 tx=0x7f8fe401c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c52638c590019c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 1 -- 172.21.15.119:0/1688841502 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40ab5f0 con 0x7f8ff4132750 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: dump: 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.400 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: _finish_auth 0 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: _check_auth_tickets 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 1 -- 172.21.15.119:0/1688841502 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe40b0070 con 0x7f8ff4132750 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 5 monclient: authenticate success, global_id 34288 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 1 -- 172.21.15.119:0/1688841502 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40ab9b0 con 0x7f8ff4132750 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff8ab7640 4 set_mon_vals no callback set 2024-09-15T21:20:22.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.387+0000 7f8ff159a640 10 monclient: dump: 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: shutdown 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:22.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/1688841502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff4132750 msgr2=0x7f8ff4132b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/1688841502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff4132750 0x7f8ff4132b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098a0 tx=0x7f8fe401c3d0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/1688841502 shutdown_connections 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/1688841502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff4132750 0x7f8ff4132b30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/1688841502 >> 172.21.15.119:0/1688841502 conn(0x7f8ff406cbf0 msgr2=0x7f8ff406cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/1688841502 shutdown_connections 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/1688841502 wait complete. 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: build_initial_monmap 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: monmap: 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:22.403 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 Processor -- start 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- start start 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: init 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff4132750 0x7f8ff412c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff4132750 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:22.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff4133070 0x7f8ff412c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): picked mon.noname-a con 0x7f8ff4133070 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 0x7f8ff4133970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): picked mon.noname-b con 0x7f8ff41334e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41414e0 con 0x7f8ff4133070 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41324f0 con 0x7f8ff41334e0 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff4141660 con 0x7f8ff4132750 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient(hunting): authenticate will time out at 1141.189826s 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff259c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff4132750 msgr2=0x7f8ff412c2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:22.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff4132750 0x7f8ff412c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 0x7f8ff4133970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 0x7f8ff4133970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48976/0 (socket says 172.21.15.119:48976) 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 -- 172.21.15.119:0/3075593148 learned_addr learned my addr 172.21.15.119:0/3075593148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): get_auth_request con 0x7f8ff41334e0 auth_method 0 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff1d9b640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff4133070 0x7f8ff412c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff1d9b640 10 monclient(hunting): get_auth_request con 0x7f8ff4133070 auth_method 0 2024-09-15T21:20:22.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient(hunting): handle_auth_done global_id 34291 payload 824 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 -- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff4132750 msgr2=0x7f8ff412c2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff4132750 0x7f8ff412c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 -- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff4133070 msgr2=0x7f8ff412c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff4133070 0x7f8ff412c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 monclient: found mon.noname-b 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 -- 172.21.15.119:0/3075593148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff4131ff0 con 0x7f8ff41334e0 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff2d9d640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 0x7f8ff4133970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c7c0 tx=0x7f8fec00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:22.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec003b80 con 0x7f8ff41334e0 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: dump: 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.408 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 5 monclient: authenticate success, global_id 34291 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec003d20 con 0x7f8ff41334e0 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00aa90 con 0x7f8ff41334e0 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _renew_subs 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff4136c80 con 0x7f8ff41334e0 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 10 monclient: dump: 2024-09-15T21:20:22.409 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe3fff640 4 set_mon_vals no callback set 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _renew_subs 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41369a0 con 0x7f8ff41334e0 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:22.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff4132b30 con 0x7f8ff41334e0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8fec00ac30 con 0x7f8ff41334e0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6120+0+0 (secure 0 0 0) 0x7f8fec04fff0 con 0x7f8ff41334e0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff259c640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.391+0000 7f8ff259c640 10 monclient: get_auth_request con 0x7f8fd403b970 auth_method 0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.395+0000 7f8ff259c640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40af000 tx=0x7f8fe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:22.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.399+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8ff4131ff0 con 0x7f8ff41334e0 2024-09-15T21:20:22.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.399+0000 7f8fe2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:22.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.399+0000 7f8fe2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:22.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.559+0000 7f8ff8ab7640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:22.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.559+0000 7f8ff8ab7640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:22.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.559+0000 7f8ff8ab7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.559+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7f8ff406a810 con 0x7f8ff41334e0 2024-09-15T21:20:22.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.751+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 added bucket bar type root to crush map v72) ==== 138+0+0 (secure 0 0 0) 0x7f8fec005080 con 0x7f8ff41334e0 2024-09-15T21:20:22.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.751+0000 7f8fe2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:22.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.751+0000 7f8fe2ffd640 10 monclient: _finish_command 2 = system:0 added bucket bar type root to crush map 2024-09-15T21:20:22.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8ff8ab7640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:22.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8ff8ab7640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:22.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8ff8ab7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7f8ff412cd60 con 0x7f8ff41334e0 2024-09-15T21:20:22.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8fe2ffd640 1 -- 172.21.15.119:0/3075593148 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v72) ==== 126+0+0 (secure 0 0 0) 0x7f8fec04e4d0 con 0x7f8ff41334e0 2024-09-15T21:20:22.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8fe2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:22.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.851+0000 7f8fe2ffd640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-09-15T21:20:22.856 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'bar' already exists 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 msgr2=0x7f8fd403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40af000 tx=0x7f8fe4008040 comp rx=0 tx=0).stop 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 10 monclient: shutdown 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 msgr2=0x7f8ff4133970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 0x7f8ff4133970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c7c0 tx=0x7f8fec00cc80 comp rx=0 tx=0).stop 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 shutdown_connections 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff41334e0 0x7f8ff4133970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff4133070 0x7f8ff412c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 --2- 172.21.15.119:0/3075593148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff4132750 0x7f8ff412c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 >> 172.21.15.119:0/3075593148 conn(0x7f8ff406cbf0 msgr2=0x7f8ff409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:22.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 shutdown_connections 2024-09-15T21:20:22.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.855+0000 7f8ff8ab7640 1 -- 172.21.15.119:0/3075593148 wait complete. 2024-09-15T21:20:22.872 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket bar root 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient: get_monmap_and_config 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient: build_initial_monmap 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient: monmap: 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:22.985686+0000 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:22.985686+0000 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:22.987 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.988 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:22.988 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:22.988 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:22.988 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:22.988 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:22.988 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 Processor -- start 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 -- start start 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient: init 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd4c009d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:22.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009def0 0x7fd4c0139980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): picked mon.noname-b con 0x7fd4c009def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4c0139ec0 0x7fd4c013c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): picked mon.noname-c con 0x7fd4c0139ec0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c006e240 con 0x7fd4c009d400 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c006e000 con 0x7fd4c0139ec0 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c009e7f0 con 0x7fd4c009def0 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4c5be8640 10 monclient(hunting): authenticate will time out at 1141.782872s 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4c0139ec0 0x7fd4c013c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4beffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009def0 msgr2=0x7fd4c0139980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:22.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4beffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009def0 0x7fd4c0139980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4c0139ec0 0x7fd4c013c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48978/0 (socket says 172.21.15.119:48978) 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 -- 172.21.15.119:0/3838341555 learned_addr learned my addr 172.21.15.119:0/3838341555 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): get_auth_request con 0x7fd4c0139ec0 auth_method 0 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient(hunting): handle_auth_done global_id 34294 payload 274 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 -- 172.21.15.119:0/3838341555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009def0 msgr2=0x7fd4c0139980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 --2- 172.21.15.119:0/3838341555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009def0 0x7fd4c0139980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 -- 172.21.15.119:0/3838341555 >> v1:172.21.15.119:6789/0 conn(0x7fd4c009d400 legacy=0x7fd4c009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:22.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:22.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 monclient: found mon.noname-c 2024-09-15T21:20:22.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:22.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:22.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.983+0000 7fd4bffff640 1 -- 172.21.15.119:0/3838341555 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c00a2500 con 0x7fd4c0139ec0 2024-09-15T21:20:22.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4bffff640 1 --2- 172.21.15.119:0/3838341555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c013c2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009e50 tx=0x7fd4b801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92755a7163369b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:22.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 1 -- 172.21.15.119:0/3838341555 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80a55d0 con 0x7fd4c0139ec0 2024-09-15T21:20:22.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:22.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:22.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: dump: 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:22.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 5 monclient: authenticate success, global_id 34294 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 1 -- 172.21.15.119:0/3838341555 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80a5770 con 0x7fd4c0139ec0 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 4 set_mon_vals no callback set 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 1 -- 172.21.15.119:0/3838341555 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80a5b40 con 0x7fd4c0139ec0 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:23.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4be7fc640 10 monclient: dump: 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: shutdown 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:23.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- 172.21.15.119:0/3838341555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 msgr2=0x7fd4c013c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/3838341555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c013c2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009e50 tx=0x7fd4b801c910 comp rx=0 tx=0).stop 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- 172.21.15.119:0/3838341555 shutdown_connections 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/3838341555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c013c2b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/3838341555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009def0 0x7fd4c0139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- 172.21.15.119:0/3838341555 >> 172.21.15.119:0/3838341555 conn(0x7fd4c009be60 msgr2=0x7fd4c009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- 172.21.15.119:0/3838341555 shutdown_connections 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- 172.21.15.119:0/3838341555 wait complete. 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: build_initial_monmap 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: monmap: 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:23.002 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 Processor -- start 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- start start 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: init 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009d400 0x7fd4c011eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): picked mon.noname-c con 0x7fd4c009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c009def0 0x7fd4c011cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): picked mon.noname-a con 0x7fd4c009def0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c01349d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): picked mon.noname-b con 0x7fd4c0139ec0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c013f190 con 0x7fd4c009def0 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c013f010 con 0x7fd4c0139ec0 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c013f310 con 0x7fd4c009d400 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.987+0000 7fd4c5be8640 10 monclient(hunting): authenticate will time out at 1141.787928s 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c01349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c01349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48980/0 (socket says 172.21.15.119:48980) 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bf7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009d400 msgr2=0x7fd4c011eb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 -- 172.21.15.119:0/2380427504 learned_addr learned my addr 172.21.15.119:0/2380427504 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): get_auth_request con 0x7fd4c0139ec0 auth_method 0 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bf7fe640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009d400 0x7fd4c011eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4beffd640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c009def0 0x7fd4c011cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4beffd640 10 monclient(hunting): get_auth_request con 0x7fd4c009def0 auth_method 0 2024-09-15T21:20:23.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4beffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient(hunting): handle_auth_done global_id 34297 payload 824 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 -- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009d400 msgr2=0x7fd4c011eb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009d400 0x7fd4c011eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 -- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c009def0 msgr2=0x7fd4c011cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c009def0 0x7fd4c011cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 monclient: found mon.noname-b 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 -- 172.21.15.119:0/2380427504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c011e030 con 0x7fd4c0139ec0 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bffff640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c01349d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd4b801a540 tx=0x7fd4b80196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:23.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80bc070 con 0x7fd4c0139ec0 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: dump: 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.007 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80b8660 con 0x7fd4c0139ec0 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 5 monclient: authenticate success, global_id 34297 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 monclient: _renew_subs 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c013a530 con 0x7fd4c0139ec0 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b8a50 con 0x7fd4c0139ec0 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 10 monclient: dump: 2024-09-15T21:20:23.008 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 monclient: _renew_subs 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bdffb640 4 set_mon_vals no callback set 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c0120090 con 0x7fd4c0139ec0 2024-09-15T21:20:23.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c009d7e0 con 0x7fd4c0139ec0 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd4b80bf070 con 0x7fd4c0139ec0 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4ac03b970 0x7fd4ac03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6147+0+0 (secure 0 0 0) 0x7fd4b80eaab0 con 0x7fd4c0139ec0 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bf7fe640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4ac03b970 0x7fd4ac03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bf7fe640 10 monclient: get_auth_request con 0x7fd4ac03b970 auth_method 0 2024-09-15T21:20:23.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.991+0000 7fd4bf7fe640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4ac03b970 0x7fd4ac03de30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0000bd0 tx=0x7fd4b0007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:23.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.995+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd4b80c6070 con 0x7fd4c0139ec0 2024-09-15T21:20:23.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.995+0000 7fd4a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:23.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:22.995+0000 7fd4a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:23.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.167+0000 7fd4c5be8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:23.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.167+0000 7fd4c5be8640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:23.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.167+0000 7fd4c5be8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.167+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7fd4c011fbd0 con 0x7fd4c0139ec0 2024-09-15T21:20:23.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.171+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v72) ==== 126+0+0 (secure 0 0 0) 0x7fd4b8016620 con 0x7fd4c0139ec0 2024-09-15T21:20:23.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.171+0000 7fd4a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:23.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.171+0000 7fd4a7fff640 10 monclient: _finish_command 2 = system:0 bucket 'bar' already exists 2024-09-15T21:20:23.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4c5be8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:23.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4c5be8640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:23.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4c5be8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7fd4c0134f10 con 0x7fd4c0139ec0 2024-09-15T21:20:23.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4a7fff640 1 -- 172.21.15.119:0/2380427504 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v72) ==== 126+0+0 (secure 0 0 0) 0x7fd4b80ac3a0 con 0x7fd4c0139ec0 2024-09-15T21:20:23.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4a7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T21:20:23.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.275+0000 7fd4a7fff640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-09-15T21:20:23.280 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'bar' already exists 2024-09-15T21:20:23.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4ac03b970 msgr2=0x7fd4ac03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4ac03b970 0x7fd4ac03de30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0000bd0 tx=0x7fd4b0007a00 comp rx=0 tx=0).stop 2024-09-15T21:20:23.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 10 monclient: shutdown 2024-09-15T21:20:23.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 msgr2=0x7fd4c01349d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c01349d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd4b801a540 tx=0x7fd4b80196b0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 shutdown_connections 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4ac03b970 0x7fd4ac03de30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4c0139ec0 0x7fd4c01349d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c009def0 0x7fd4c011cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 --2- 172.21.15.119:0/2380427504 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4c009d400 0x7fd4c011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 >> 172.21.15.119:0/2380427504 conn(0x7fd4c009be60 msgr2=0x7fd4c013ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 shutdown_connections 2024-09-15T21:20:23.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.279+0000 7fd4c5be8640 1 -- 172.21.15.119:0/2380427504 wait complete. 2024-09-15T21:20:23.297 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush link host1 root=bar 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.411+0000 7fa7cb313640 10 monclient: get_monmap_and_config 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.411+0000 7fa7cb313640 10 monclient: build_initial_monmap 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.411+0000 7fa7cb313640 10 monclient: monmap: 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:23.415611+0000 2024-09-15T21:20:23.417 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:23.415611+0000 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:23.418 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.411+0000 7fa7cb313640 1 Processor -- start 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.411+0000 7fa7cb313640 1 -- start start 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.411+0000 7fa7cb313640 10 monclient: init 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa7c40a0900 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa7c40a13f0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa7c413c330 0x7fa7c413e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient(hunting): picked mon.noname-a con 0x7fa7c413c330 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c40a2240 con 0x7fa7c40a13f0 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c40a1c80 con 0x7fa7c40a0900 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c409fdd0 con 0x7fa7c413c330 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 10 monclient(hunting): authenticate will time out at 1142.212584s 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c90af640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa7c40a0900 0x7fa7c40a0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39160/0 (socket says 172.21.15.119:39160) 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c90af640 1 -- 172.21.15.119:0/2881764058 learned_addr learned my addr 172.21.15.119:0/2881764058 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:23.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/2881764058 >> v2:172.21.15.17:3300/0 conn(0x7fa7c413c330 0x7fa7c413e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): get_auth_request con 0x7fa7c413c330 auth_method 0 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 481561316 0 0) 0x7fa7c40a1c80 con 0x7fa7c40a0900 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): _negotiate 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): my global_id is 34300 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7ac003530 con 0x7fa7c40a0900 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:23.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1397612156 0 0) 0x7fa7ac003530 con 0x7fa7c40a0900 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 >> v2:172.21.15.17:3300/0 conn(0x7fa7c413c330 msgr2=0x7fa7c413e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 --2- 172.21.15.119:0/2881764058 >> v2:172.21.15.17:3300/0 conn(0x7fa7c413c330 0x7fa7c413e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 >> v1:172.21.15.17:6789/0 conn(0x7fa7c40a13f0 legacy=0x7fa7c413ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c40a1f60 con 0x7fa7c40a0900 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:23.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 5 monclient: authenticate success, global_id 34300 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa7bc001700 con 0x7fa7c40a0900 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: dump: 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.433 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 >> v1:172.21.15.119:6789/0 conn(0x7fa7c40a0900 legacy=0x7fa7c40a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7ac003a60 0x7fa7ac003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): picked mon.c con 0x7fa7ac003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:23.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7ac004810 0x7fa7ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): picked mon.a con 0x7fa7ac004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7ac00fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): picked mon.b con 0x7fa7ac005320 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c409fdd0 con 0x7fa7ac004810 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac010670 con 0x7fa7ac005320 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac010870 con 0x7fa7ac003a60 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c3fff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7cb313640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:23.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c90af640 1 -- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7ac003a60 msgr2=0x7fa7ac003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7ac00fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c90af640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7ac003a60 0x7fa7ac003ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): get_auth_request con 0x7fa7ac005320 auth_method 0 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c88ae640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7ac004810 0x7fa7ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient(hunting): handle_auth_done global_id 34300 payload 293 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 -- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7ac003a60 msgr2=0x7fa7ac003ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:23.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7ac003a60 0x7fa7ac003ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 -- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7ac004810 msgr2=0x7fa7ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7ac004810 0x7fa7ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient: _finish_hunting 0 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 monclient: found mon.b 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 1 -- 172.21.15.119:0/2881764058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac003530 con 0x7fa7ac005320 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient: _finish_auth 0 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 10 monclient: _check_auth_tickets 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c98b0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c88ae640 10 monclient: get_auth_request con 0x7fa7ac004810 auth_method 0 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.415+0000 7fa7c88ae640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7ac004810 0x7fa7ac004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:23.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 1 -- 172.21.15.119:0/2881764058 reap_dead start 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7ac00fde0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009ab0 tx=0x7fa7b401ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87984d81a1b20290 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40096b0 con 0x7fa7ac005320 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: dump: 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.438 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.439 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.439 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.439 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.439 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b4004020 con 0x7fa7ac005320 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 1 -- 172.21.15.119:0/2881764058 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004210 con 0x7fa7ac005320 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 4 set_mon_vals no callback set 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c3fff640 10 monclient: dump: 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.440 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: shutdown 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/2881764058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 msgr2=0x7fa7ac00fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7ac00fde0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009ab0 tx=0x7fa7b401ca70 comp rx=0 tx=0).stop 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/2881764058 shutdown_connections 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 --2- 172.21.15.119:0/2881764058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7ac00fde0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/2881764058 >> 172.21.15.119:0/2881764058 conn(0x7fa7c409be60 msgr2=0x7fa7c409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/2881764058 shutdown_connections 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/2881764058 wait complete. 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: build_initial_monmap 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: monmap: 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:23.442 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 Processor -- start 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- start start 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: init 2024-09-15T21:20:23.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7c41cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): picked mon.noname-b con 0x7fa7ac005320 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7c41cb7a0 0x7fa7c41cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): picked mon.noname-c con 0x7fa7c41cb7a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c411c550 0x7fa7c411c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): picked mon.noname-a con 0x7fa7c411c550 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40a01d0 con 0x7fa7c411c550 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40a0050 con 0x7fa7ac005320 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c40a0ce0 con 0x7fa7c41cb7a0 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient(hunting): authenticate will time out at 1142.218438s 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7c41cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c88ae640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7c41cb7a0 msgr2=0x7fa7c41cbc00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c88ae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7c41cb7a0 0x7fa7c41cbc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7c41cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49000/0 (socket says 172.21.15.119:49000) 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 -- 172.21.15.119:0/528107387 learned_addr learned my addr 172.21.15.119:0/528107387 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): get_auth_request con 0x7fa7ac005320 auth_method 0 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:23.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c411c550 0x7fa7c411c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 10 monclient(hunting): get_auth_request con 0x7fa7c411c550 auth_method 0 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient(hunting): handle_auth_done global_id 34303 payload 824 2024-09-15T21:20:23.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 -- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7c41cb7a0 msgr2=0x7fa7c41cbc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7c41cb7a0 0x7fa7c41cbc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 -- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c411c550 msgr2=0x7fa7c411c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c411c550 0x7fa7c411c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient: _finish_hunting 0 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 monclient: found mon.noname-b 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 -- 172.21.15.119:0/528107387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c411e150 con 0x7fa7ac005320 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c98b0640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c411c550 0x7fa7c411c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c90af640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7c41cb260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c730 tx=0x7fa7bc00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:23.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc018070 con 0x7fa7ac005320 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: dump: 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.448 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc0072d0 con 0x7fa7ac005320 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0047f0 con 0x7fa7ac005320 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 5 monclient: authenticate success, global_id 34303 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: _renew_subs 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c411efc0 con 0x7fa7ac005320 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:23.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c37fe640 4 set_mon_vals no callback set 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7c1ffb640 10 monclient: dump: 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:23.450 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: _renew_subs 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.419+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c413b520 con 0x7fa7ac005320 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7cb313640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7cb313640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7cb313640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c40a1070 con 0x7fa7ac005320 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa7bc01b070 con 0x7fa7ac005320 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7c1ffb640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa7b003b970 0x7fa7b003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:23.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6147+0+0 (secure 0 0 0) 0x7fa7bc04f380 con 0x7fa7ac005320 2024-09-15T21:20:23.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7c88ae640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa7b003b970 0x7fa7b003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:23.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7c88ae640 10 monclient: get_auth_request con 0x7fa7b003b970 auth_method 0 2024-09-15T21:20:23.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.423+0000 7fa7c88ae640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa7b003b970 0x7fa7b003de30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80012f0 tx=0x7fa7b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:23.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.427+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa7bc010300 con 0x7fa7ac005320 2024-09-15T21:20:23.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.427+0000 7fa7c1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:23.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.427+0000 7fa7c1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:23.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.599+0000 7fa7cb313640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:23.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.599+0000 7fa7cb313640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:23.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.599+0000 7fa7cb313640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.599+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7fa7c411da20 con 0x7fa7ac005320 2024-09-15T21:20:23.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.763+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 linked item id -11 name 'host1' to location {root=bar} in crush map v73) ==== 168+0+0 (secure 0 0 0) 0x7fa7bc00ae00 con 0x7fa7ac005320 2024-09-15T21:20:23.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.767+0000 7fa7c1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:23.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.767+0000 7fa7c1ffb640 10 monclient: _finish_command 2 = system:0 linked item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T21:20:23.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7cb313640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:23.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7cb313640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:23.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7cb313640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:23.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7fa7c411ec40 con 0x7fa7ac005320 2024-09-15T21:20:23.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7c1ffb640 1 -- 172.21.15.119:0/528107387 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v73) ==== 177+0+0 (secure 0 0 0) 0x7fa7bc010df0 con 0x7fa7ac005320 2024-09-15T21:20:23.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7c1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:23.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.871+0000 7fa7c1ffb640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T21:20:23.878 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T21:20:23.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa7b003b970 msgr2=0x7fa7b003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa7b003b970 0x7fa7b003de30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80012f0 tx=0x7fa7b8009040 comp rx=0 tx=0).stop 2024-09-15T21:20:23.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 10 monclient: shutdown 2024-09-15T21:20:23.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:23.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 msgr2=0x7fa7c41cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7c41cb260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c730 tx=0x7fa7bc00cbf0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 shutdown_connections 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa7b003b970 0x7fa7b003de30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c411c550 0x7fa7c411c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa7c41cb7a0 0x7fa7c41cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 --2- 172.21.15.119:0/528107387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ac005320 0x7fa7c41cb260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 >> 172.21.15.119:0/528107387 conn(0x7fa7c409be60 msgr2=0x7fa7c40a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:23.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 shutdown_connections 2024-09-15T21:20:23.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:23.875+0000 7fa7cb313640 1 -- 172.21.15.119:0/528107387 wait complete. 2024-09-15T21:20:23.895 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush link host1 root=bar 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.011+0000 7f2cf1320640 10 monclient: get_monmap_and_config 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.011+0000 7f2cf1320640 10 monclient: build_initial_monmap 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.011+0000 7f2cf1320640 10 monclient: monmap: 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:24.015447+0000 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:24.015447+0000 2024-09-15T21:20:24.017 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:24.018 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.018 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:24.018 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:24.018 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:24.018 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:24.018 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:24.019 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.011+0000 7f2cf1320640 1 Processor -- start 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.011+0000 7f2cf1320640 1 -- start start 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.011+0000 7f2cf1320640 10 monclient: init 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cec0a0900 0x7f2cec0a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): picked mon.noname-c con 0x7f2cec0a0900 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec1356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): picked mon.noname-b con 0x7f2cec0a1220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2cec135c20 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cec0a2240 con 0x7f2cec135c20 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2cec09fdd0 con 0x7f2cec0a0900 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec0a1c80 con 0x7f2cec0a1220 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient(hunting): authenticate will time out at 1142.812549s 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cec0a0900 0x7f2cec0a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cea7fc640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 msgr2=0x7f2cec1356e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:24.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cec0a0900 0x7f2cec0a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49004/0 (socket says 172.21.15.119:49004) 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cea7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec1356e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 -- 172.21.15.119:0/1871979998 learned_addr learned my addr 172.21.15.119:0/1871979998 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): get_auth_request con 0x7f2cec0a0900 auth_method 0 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient(hunting): handle_auth_done global_id 34306 payload 274 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 -- 172.21.15.119:0/1871979998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 msgr2=0x7f2cec1356e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 --2- 172.21.15.119:0/1871979998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec1356e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 -- 172.21.15.119:0/1871979998 >> v1:172.21.15.17:6789/0 conn(0x7f2cec135c20 legacy=0x7f2cec13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 monclient: found mon.noname-c 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 -- 172.21.15.119:0/1871979998 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec0a1f60 con 0x7f2cec0a0900 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ceaffd640 1 --2- 172.21.15.119:0/1871979998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec0a0900 0x7f2cec0a0ce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a84889a9c373b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 1 -- 172.21.15.119:0/1871979998 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40ab660 con 0x7f2cec0a0900 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: dump: 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.030 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 1 -- 172.21.15.119:0/1871979998 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd40b0070 con 0x7f2cec0a0900 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 1 -- 172.21.15.119:0/1871979998 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40aba20 con 0x7f2cec0a0900 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 5 monclient: authenticate success, global_id 34306 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:24.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2cf1320640 4 set_mon_vals no callback set 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.015+0000 7f2ce9ffb640 10 monclient: dump: 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.032 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: shutdown 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/1871979998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec0a0900 msgr2=0x7f2cec0a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 --2- 172.21.15.119:0/1871979998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec0a0900 0x7f2cec0a0ce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd401c440 comp rx=0 tx=0).stop 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/1871979998 shutdown_connections 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 --2- 172.21.15.119:0/1871979998 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec1356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 --2- 172.21.15.119:0/1871979998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec0a0900 0x7f2cec0a0ce0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/1871979998 >> 172.21.15.119:0/1871979998 conn(0x7f2cec09be60 msgr2=0x7f2cec09c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/1871979998 shutdown_connections 2024-09-15T21:20:24.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/1871979998 wait complete. 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: build_initial_monmap 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: monmap: 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.034 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 Processor -- start 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- start start 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: init 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cec0a0900 0x7f2cec1320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): picked mon.noname-a con 0x7f2cec0a0900 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec132630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): picked mon.noname-c con 0x7f2cec0a1220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 0x7f2cec132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): picked mon.noname-b con 0x7f2cec135c20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec06ed10 con 0x7f2cec0a0900 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec136510 con 0x7f2cec135c20 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec06ee90 con 0x7f2cec0a1220 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient(hunting): authenticate will time out at 1142.818332s 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 0x7f2cec132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cea7fc640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 msgr2=0x7f2cec132630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cea7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec132630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 0x7f2cec132b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49016/0 (socket says 172.21.15.119:49016) 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 -- 172.21.15.119:0/2715775880 learned_addr learned my addr 172.21.15.119:0/2715775880 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:24.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): get_auth_request con 0x7f2cec135c20 auth_method 0 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceaffd640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cec0a0900 0x7f2cec1320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceaffd640 10 monclient(hunting): get_auth_request con 0x7f2cec0a0900 auth_method 0 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient(hunting): handle_auth_done global_id 34309 payload 824 2024-09-15T21:20:24.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 -- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 msgr2=0x7f2cec132630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec132630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 -- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cec0a0900 msgr2=0x7f2cec1320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cec0a0900 0x7f2cec1320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 monclient: found mon.noname-b 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 -- 172.21.15.119:0/2715775880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec09d0f0 con 0x7f2cec135c20 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceaffd640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cec0a0900 0x7f2cec1320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ceb7fe640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 0x7f2cec132b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c7c0 tx=0x7f2ce400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4003b80 con 0x7f2cec135c20 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: dump: 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:24.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 5 monclient: authenticate success, global_id 34309 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce4003d20 con 0x7f2cec135c20 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: _renew_subs 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec072880 con 0x7f2cec135c20 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400aa90 con 0x7f2cec135c20 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce37fe640 10 monclient: dump: 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.040 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2ce97fa640 4 set_mon_vals no callback set 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: _renew_subs 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.019+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec072030 con 0x7f2cec135c20 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2cf1320640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2cf1320640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:24.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2cf1320640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec0a0170 con 0x7f2cec135c20 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2ce400ac30 con 0x7f2cec135c20 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2ce37fe640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ccc03b970 0x7f2ccc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6273+0+0 (secure 0 0 0) 0x7f2ce4050080 con 0x7f2cec135c20 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2ceaffd640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ccc03b970 0x7f2ccc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2ceaffd640 10 monclient: get_auth_request con 0x7f2ccc03b970 auth_method 0 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.023+0000 7f2ceaffd640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ccc03b970 0x7f2ccc03de30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40af000 tx=0x7f2cd40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.027+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2cec09d0f0 con 0x7f2cec135c20 2024-09-15T21:20:24.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.027+0000 7f2ce37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:24.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.027+0000 7f2ce37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:24.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.203+0000 7f2cf1320640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:24.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.203+0000 7f2cf1320640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:24.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.203+0000 7f2cf1320640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.203+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7f2cec11e890 con 0x7f2cec135c20 2024-09-15T21:20:24.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.207+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v73) ==== 177+0+0 (secure 0 0 0) 0x7f2ce4012e00 con 0x7f2cec135c20 2024-09-15T21:20:24.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.207+0000 7f2ce37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:24.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.207+0000 7f2ce37fe640 10 monclient: _finish_command 2 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T21:20:24.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.311+0000 7f2cf1320640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:24.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.311+0000 7f2cf1320640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:24.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.311+0000 7f2cf1320640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.311+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7f2cec11d880 con 0x7f2cec135c20 2024-09-15T21:20:24.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2ce37fe640 1 -- 172.21.15.119:0/2715775880 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v73) ==== 177+0+0 (secure 0 0 0) 0x7f2ce4019d10 con 0x7f2cec135c20 2024-09-15T21:20:24.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2ce37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T21:20:24.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2ce37fe640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T21:20:24.318 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T21:20:24.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ccc03b970 msgr2=0x7f2ccc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ccc03b970 0x7f2ccc03de30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40af000 tx=0x7f2cd40a7000 comp rx=0 tx=0).stop 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 10 monclient: shutdown 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 msgr2=0x7f2cec132b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 0x7f2cec132b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c7c0 tx=0x7f2ce400cc80 comp rx=0 tx=0).stop 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 shutdown_connections 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ccc03b970 0x7f2ccc03de30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cec135c20 0x7f2cec132b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cec0a1220 0x7f2cec132630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 --2- 172.21.15.119:0/2715775880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cec0a0900 0x7f2cec1320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 >> 172.21.15.119:0/2715775880 conn(0x7f2cec09be60 msgr2=0x7f2cec0a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:24.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 shutdown_connections 2024-09-15T21:20:24.322 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.315+0000 7f2cf1320640 1 -- 172.21.15.119:0/2715775880 wait complete. 2024-09-15T21:20:24.334 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush link host2 root=bar 2024-09-15T21:20:24.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.435+0000 7fb9da8ef640 10 monclient: get_monmap_and_config 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.435+0000 7fb9da8ef640 10 monclient: build_initial_monmap 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.435+0000 7fb9da8ef640 10 monclient: monmap: 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:24.440557+0000 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:24.440557+0000 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:24.442 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:24.443 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:24.443 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 1 Processor -- start 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 1 -- start start 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient: init 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 0x7fb9d40a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient(hunting): picked mon.noname-b con 0x7fb9d40a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb9d40a46b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:24.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb9d40a1100 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d4138600 con 0x7fb9d40a46b0 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d4135180 con 0x7fb9d40a1100 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d40a7dc0 con 0x7fb9d40a79e0 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 10 monclient(hunting): authenticate will time out at 1143.237263s 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d3fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 msgr2=0x7fb9d40a4170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 0x7fb9d40a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d8e8c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb9d40a1100 0x7fb9d4137ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39162/0 (socket says 172.21.15.119:39162) 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d8e8c640 1 -- 172.21.15.119:0/3238639082 learned_addr learned my addr 172.21.15.119:0/3238639082 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1598102371 0 0) 0x7fb9d4135180 con 0x7fb9d40a1100 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): my global_id is 34312 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9c0003530 con 0x7fb9d40a1100 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358517994 0 0) 0x7fb9d4138600 con 0x7fb9d40a46b0 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): my global_id is 5169 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d4135180 con 0x7fb9d40a46b0 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3331621802 0 0) 0x7fb9c0003530 con 0x7fb9d40a1100 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 msgr2=0x7fb9d40a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 0x7fb9d40a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 >> v1:172.21.15.17:6789/0 conn(0x7fb9d40a46b0 legacy=0x7fb9d41377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:24.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d40a6920 con 0x7fb9d40a1100 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb9cc001700 con 0x7fb9d40a1100 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: dump: 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.455 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 >> v1:172.21.15.119:6789/0 conn(0x7fb9d40a1100 legacy=0x7fb9d4137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 0x7fb9c0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): picked mon.c con 0x7fb9d40a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:24.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9c00041c0 0x7fb9c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): picked mon.b con 0x7fb9c00041c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0004f50 0x7fb9c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): picked mon.a con 0x7fb9c0004f50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40a7dc0 con 0x7fb9c0004f50 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c000bf10 con 0x7fb9c00041c0 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c000c090 con 0x7fb9d40a79e0 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d3fff640 1 -- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 msgr2=0x7fb9c0003bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:24.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d3fff640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 0x7fb9c0003bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9c00041c0 0x7fb9c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): get_auth_request con 0x7fb9c00041c0 auth_method 0 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d8e8c640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0004f50 0x7fb9c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_done global_id 34312 payload 293 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 -- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 msgr2=0x7fb9c0003bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40a79e0 0x7fb9c0003bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 -- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0004f50 msgr2=0x7fb9c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0004f50 0x7fb9c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 monclient: found mon.b 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 1 -- 172.21.15.119:0/3238639082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c0003530 con 0x7fb9c00041c0 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d8e8c640 10 monclient: get_auth_request con 0x7fb9c0004f50 auth_method 0 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9d8e8c640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0004f50 0x7fb9c000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 5 monclient: authenticate success, global_id 34312 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.439+0000 7fb9da8ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9c00041c0 0x7fb9c0004810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4131de0 tx=0x7fb9bc01eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f5473fa65fc57c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:24.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc001530 con 0x7fb9c00041c0 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: dump: 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.460 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0052d0 con 0x7fb9c00041c0 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 4 set_mon_vals no callback set 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 1 -- 172.21.15.119:0/3238639082 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0a8870 con 0x7fb9c00041c0 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d2ffd640 10 monclient: dump: 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.461 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: shutdown 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- 172.21.15.119:0/3238639082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9c00041c0 msgr2=0x7fb9c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/3238639082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9c00041c0 0x7fb9c0004810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4131de0 tx=0x7fb9bc01eb90 comp rx=0 tx=0).stop 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d8e8c640 1 -- 172.21.15.119:0/3238639082 reap_dead start 2024-09-15T21:20:24.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- 172.21.15.119:0/3238639082 shutdown_connections 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- 172.21.15.119:0/3238639082 >> 172.21.15.119:0/3238639082 conn(0x7fb9d40a6ef0 msgr2=0x7fb9d40a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- 172.21.15.119:0/3238639082 shutdown_connections 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- 172.21.15.119:0/3238639082 wait complete. 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: build_initial_monmap 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: monmap: 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:24.463 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 Processor -- start 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- start start 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: init 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c00041c0 0x7fb9d4130ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): picked mon.noname-a con 0x7fb9c00041c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 0x7fb9d4131400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): picked mon.noname-b con 0x7fb9d41ccbd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40723f0 0x7fb9d4072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): picked mon.noname-c con 0x7fb9d40723f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:24.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40a6c00 con 0x7fb9c00041c0 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40a1c30 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d4134720 con 0x7fb9d40723f0 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9da8ef640 10 monclient(hunting): authenticate will time out at 1143.243271s 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d8e8c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40723f0 msgr2=0x7fb9d4072870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 0x7fb9d4131400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d8e8c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40723f0 0x7fb9d4072870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:24.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 0x7fb9d4131400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49024/0 (socket says 172.21.15.119:49024) 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 -- 172.21.15.119:0/1490260901 learned_addr learned my addr 172.21.15.119:0/1490260901 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): get_auth_request con 0x7fb9d41ccbd0 auth_method 0 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d3fff640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c00041c0 0x7fb9d4130ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d3fff640 10 monclient(hunting): get_auth_request con 0x7fb9c00041c0 auth_method 0 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:24.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient(hunting): handle_auth_done global_id 34315 payload 824 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 -- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40723f0 msgr2=0x7fb9d4072870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40723f0 0x7fb9d4072870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 -- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c00041c0 msgr2=0x7fb9d4130ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c00041c0 0x7fb9d4130ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 monclient: found mon.noname-b 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 -- 172.21.15.119:0/1490260901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d4073e60 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.443+0000 7fb9d37fe640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 0x7fb9d4131400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005040 tx=0x7fb9c0005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0b5040 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: dump: 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:20:24.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 5 monclient: authenticate success, global_id 34315 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9bc001510 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc005480 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 monclient: _renew_subs 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d41225e0 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d27fc640 4 set_mon_vals no callback set 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 10 monclient: dump: 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:24.469 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 monclient: _renew_subs 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d40a3860 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:24.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d4131940 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb9bc0067e0 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9b403b9a0 0x7fb9b403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6273+0+0 (secure 0 0 0) 0x7fb9bc0ed930 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d3fff640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9b403b9a0 0x7fb9b403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d3fff640 10 monclient: get_auth_request con 0x7fb9b403b9a0 auth_method 0 2024-09-15T21:20:24.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.447+0000 7fb9d3fff640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9b403b9a0 0x7fb9b403de60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4003cc0 tx=0x7fb9c4009530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:24.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.451+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb9bc018940 con 0x7fb9d41ccbd0 2024-09-15T21:20:24.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.451+0000 7fb9d0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:24.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.451+0000 7fb9d0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:24.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.623+0000 7fb9da8ef640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:24.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.623+0000 7fb9da8ef640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:24.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.623+0000 7fb9da8ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:24.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:24.623+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7fb9d40a4170 con 0x7fb9d41ccbd0 2024-09-15T21:20:25.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.431+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 linked item id -16 name 'host2' to location {root=bar} in crush map v74) ==== 168+0+0 (secure 0 0 0) 0x7fb9bc007030 con 0x7fb9d41ccbd0 2024-09-15T21:20:25.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.431+0000 7fb9d0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.431+0000 7fb9d0ff9640 10 monclient: _finish_command 2 = system:0 linked item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T21:20:25.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.539+0000 7fb9da8ef640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.539+0000 7fb9da8ef640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.539+0000 7fb9da8ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.539+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7fb9d40a4170 con 0x7fb9d41ccbd0 2024-09-15T21:20:25.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9d0ff9640 1 -- 172.21.15.119:0/1490260901 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v74) ==== 177+0+0 (secure 0 0 0) 0x7fb9bc0b0e00 con 0x7fb9d41ccbd0 2024-09-15T21:20:25.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9d0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9d0ff9640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T21:20:25.546 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T21:20:25.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9b403b9a0 msgr2=0x7fb9b403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9b403b9a0 0x7fb9b403de60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4003cc0 tx=0x7fb9c4009530 comp rx=0 tx=0).stop 2024-09-15T21:20:25.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 10 monclient: shutdown 2024-09-15T21:20:25.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:25.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 msgr2=0x7fb9d4131400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 0x7fb9d4131400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005040 tx=0x7fb9c0005840 comp rx=0 tx=0).stop 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 shutdown_connections 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9b403b9a0 0x7fb9b403de60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9d40723f0 0x7fb9d4072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9d41ccbd0 0x7fb9d4131400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 --2- 172.21.15.119:0/1490260901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c00041c0 0x7fb9d4130ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 >> 172.21.15.119:0/1490260901 conn(0x7fb9d40a6ef0 msgr2=0x7fb9d40a8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 shutdown_connections 2024-09-15T21:20:25.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.543+0000 7fb9da8ef640 1 -- 172.21.15.119:0/1490260901 wait complete. 2024-09-15T21:20:25.563 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush link host2 root=bar 2024-09-15T21:20:25.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.675+0000 7f067a5e1640 10 monclient: get_monmap_and_config 2024-09-15T21:20:25.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.675+0000 7f067a5e1640 10 monclient: build_initial_monmap 2024-09-15T21:20:25.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.675+0000 7f067a5e1640 10 monclient: monmap: 2024-09-15T21:20:25.681 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:25.681 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:25.679976+0000 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:25.679976+0000 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:25.682 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.675+0000 7f067a5e1640 1 Processor -- start 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.675+0000 7f067a5e1640 1 -- start start 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.675+0000 7f067a5e1640 10 monclient: init 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f067409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f067409def0 0x7f0674139980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient(hunting): picked mon.noname-a con 0x7f067409def0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0674139ec0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f067406e240 con 0x7f0674139ec0 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f067409e7f0 con 0x7f067409d400 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f067406e000 con 0x7f067409def0 2024-09-15T21:20:25.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 10 monclient(hunting): authenticate will time out at 1144.476604s 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0673fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f067409d400 0x7f067409d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39166/0 (socket says 172.21.15.119:39166) 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0673fff640 1 -- 172.21.15.119:0/1814021078 learned_addr learned my addr 172.21.15.119:0/1814021078 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809967663 0 0) 0x7f067409e7f0 con 0x7f067409d400 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): _negotiate 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): my global_id is 34318 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0660003530 con 0x7f067409d400 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 1 --2- 172.21.15.119:0/1814021078 >> v2:172.21.15.17:3300/0 conn(0x7f067409def0 0x7f0674139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): get_auth_request con 0x7f067409def0 auth_method 0 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:25.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493634608 0 0) 0x7f0660003530 con 0x7f067409d400 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 >> v2:172.21.15.17:3300/0 conn(0x7f067409def0 msgr2=0x7f0674139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 --2- 172.21.15.119:0/1814021078 >> v2:172.21.15.17:3300/0 conn(0x7f067409def0 0x7f0674139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 >> v1:172.21.15.17:6789/0 conn(0x7f0674139ec0 legacy=0x7f067413c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06740a2500 con 0x7f067409d400 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:25.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0664001700 con 0x7f067409d400 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: dump: 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:25.695 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 >> v1:172.21.15.119:6789/0 conn(0x7f067409d400 legacy=0x7f067409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0660003a60 0x7f0660003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): picked mon.c con 0x7f0660003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0660004810 0x7f0660004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): picked mon.a con 0x7f0660004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:25.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f066000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): picked mon.b con 0x7f06600053b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f067406e000 con 0x7f0660004810 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f066000e500 con 0x7f06600053b0 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f066000e700 con 0x7f0660003a60 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f06737fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0673fff640 1 -- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0660003a60 msgr2=0x7f0660003e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0673fff640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0660003a60 0x7f0660003e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f066000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:25.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): get_auth_request con 0x7f06600053b0 auth_method 0 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0660004810 0x7f0660004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): get_auth_request con 0x7f0660004810 auth_method 0 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f066b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:25.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient(hunting): handle_auth_done global_id 34318 payload 293 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 -- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0660003a60 msgr2=0x7f0660003e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0660003a60 0x7f0660003e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 -- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0660004810 msgr2=0x7f0660004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0660004810 0x7f0660004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 monclient: found mon.b 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 -- 172.21.15.119:0/1814021078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0660003530 con 0x7f06600053b0 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient: _finish_auth 0 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 10 monclient: _check_auth_tickets 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:25.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 5 monclient: authenticate success, global_id 34318 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f067a5e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.679+0000 7f0678b7e640 1 -- 172.21.15.119:0/1814021078 reap_dead start 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0678b7e640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f066000dc20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0674069fa0 tx=0x7f066c01c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886faff897596d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c001ae0 con 0x7f06600053b0 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: dump: 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.700 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066c003b80 con 0x7f06600053b0 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 1 -- 172.21.15.119:0/1814021078 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c0050d0 con 0x7f06600053b0 2024-09-15T21:20:25.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06737fe640 10 monclient: dump: 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 4 set_mon_vals no callback set 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: shutdown 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/1814021078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 msgr2=0x7f066000dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f066000dc20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0674069fa0 tx=0x7f066c01c6f0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/1814021078 shutdown_connections 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 --2- 172.21.15.119:0/1814021078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f066000dc20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/1814021078 >> 172.21.15.119:0/1814021078 conn(0x7f067409be60 msgr2=0x7f067409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/1814021078 shutdown_connections 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/1814021078 wait complete. 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: build_initial_monmap 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: monmap: 2024-09-15T21:20:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 Processor -- start 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- start start 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: init 2024-09-15T21:20:25.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f06741cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): picked mon.noname-b con 0x7f06600053b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06741cbca0 0x7f06741cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): picked mon.noname-c con 0x7f06741cbca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0674071620 0x7f0674071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): picked mon.noname-a con 0x7f0674071620 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f067409d960 con 0x7f0674071620 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f067409d7e0 con 0x7f06600053b0 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:25.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f067409dae0 con 0x7f06741cbca0 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient(hunting): authenticate will time out at 1144.482372s 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f06741cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f06741cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49050/0 (socket says 172.21.15.119:49050) 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 -- 172.21.15.119:0/12525782 learned_addr learned my addr 172.21.15.119:0/12525782 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): get_auth_request con 0x7f06600053b0 auth_method 0 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f066b7fe640 1 -- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06741cbca0 msgr2=0x7f06741cc0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f066b7fe640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06741cbca0 0x7f06741cc0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:25.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient(hunting): handle_auth_done global_id 34321 payload 824 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 -- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06741cbca0 msgr2=0x7f06741cc0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06741cbca0 0x7f06741cc0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0678b7e640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0674071620 0x7f0674071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 -- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0674071620 msgr2=0x7f0674071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0674071620 0x7f0674071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 monclient: found mon.noname-b 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 -- 172.21.15.119:0/12525782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067411d590 con 0x7f06600053b0 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0673fff640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f06741cb760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f066400c6b0 tx=0x7f066400cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664018070 con 0x7f06600053b0 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: dump: 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.708 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06640072f0 con 0x7f06600053b0 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 5 monclient: authenticate success, global_id 34321 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640047f0 con 0x7f06600053b0 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _renew_subs 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067413a4c0 con 0x7f06600053b0 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f0672ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 10 monclient: dump: 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:25.710 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _renew_subs 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067411d2b0 con 0x7f06600053b0 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06740a19a0 con 0x7f06600053b0 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.683+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f066401b070 con 0x7f06600053b0 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.687+0000 7f06717fa640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f065403b970 0x7f065403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:25.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.687+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7f066404f4c0 con 0x7f06600053b0 2024-09-15T21:20:25.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.687+0000 7f066b7fe640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f065403b970 0x7f065403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:25.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.687+0000 7f066b7fe640 10 monclient: get_auth_request con 0x7f065403b970 auth_method 0 2024-09-15T21:20:25.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.687+0000 7f066b7fe640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f065403b970 0x7f065403de30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f067411d7a0 tx=0x7f065c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:25.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.691+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f066401b360 con 0x7f06600053b0 2024-09-15T21:20:25.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.691+0000 7f06717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:25.712 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.691+0000 7f06717fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:25.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f067a5e1640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f067a5e1640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f067a5e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7f06740737f0 con 0x7f06600053b0 2024-09-15T21:20:25.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v74) ==== 177+0+0 (secure 0 0 0) 0x7f066400ae00 con 0x7f06600053b0 2024-09-15T21:20:25.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f06717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.855+0000 7f06717fa640 10 monclient: _finish_command 2 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T21:20:25.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.951+0000 7f067a5e1640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.951+0000 7f067a5e1640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.951+0000 7f067a5e1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:25.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.951+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7f06740008e0 con 0x7f06600053b0 2024-09-15T21:20:25.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f06717fa640 1 -- 172.21.15.119:0/12525782 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v74) ==== 177+0+0 (secure 0 0 0) 0x7f066400a5e0 con 0x7f06600053b0 2024-09-15T21:20:25.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f06717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T21:20:25.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f06717fa640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T21:20:25.958 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T21:20:25.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f065403b970 msgr2=0x7f065403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f065403b970 0x7f065403de30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f067411d7a0 tx=0x7f065c008040 comp rx=0 tx=0).stop 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 10 monclient: shutdown 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 msgr2=0x7f06741cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f06741cb760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f066400c6b0 tx=0x7f066400cb70 comp rx=0 tx=0).stop 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 shutdown_connections 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f065403b970 0x7f065403de30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0674071620 0x7f0674071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f06741cbca0 0x7f06741cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 --2- 172.21.15.119:0/12525782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06600053b0 0x7f06741cb760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 >> 172.21.15.119:0/12525782 conn(0x7f067409be60 msgr2=0x7f067413a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 shutdown_connections 2024-09-15T21:20:25.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:25.955+0000 7f067a5e1640 1 -- 172.21.15.119:0/12525782 wait complete. 2024-09-15T21:20:25.974 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:25.974 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c osd.7 2024-09-15T21:20:25.975 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 2 2024-09-15T21:20:26.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient: get_monmap_and_config 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient: build_initial_monmap 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient: monmap: 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:26.087316+0000 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:26.087316+0000 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:26.089 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:26.090 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:26.090 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:26.090 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 1 Processor -- start 2024-09-15T21:20:26.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 1 -- start start 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient: init 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 0x7f737806ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.083+0000 7f7380101640 10 monclient(hunting): picked mon.noname-b con 0x7f7378135d10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f73781360f0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f73780a10f0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73780a1b90 con 0x7f73780a10f0 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73780a1e70 con 0x7f73781360f0 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f737809f6f0 con 0x7f7378135d10 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 10 monclient(hunting): authenticate will time out at 1144.884112s 2024-09-15T21:20:26.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 msgr2=0x7f737806ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f73781360f0 0x7f73780a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39180/0 (socket says 172.21.15.119:39180) 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 0x7f737806ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 -- 172.21.15.119:0/2951389663 learned_addr learned my addr 172.21.15.119:0/2951389663 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525622464 0 0) 0x7f73780a1e70 con 0x7f73781360f0 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): _negotiate 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): my global_id is 34324 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7364003530 con 0x7f73781360f0 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2002317391 0 0) 0x7f7364003530 con 0x7f73781360f0 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 msgr2=0x7f737806ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:26.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 0x7f737806ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 >> v1:172.21.15.17:6789/0 conn(0x7f73780a10f0 legacy=0x7f737813db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73780a2150 con 0x7f73781360f0 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7368001700 con 0x7f73781360f0 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:26.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: dump: 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.103 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 >> v1:172.21.15.119:6789/0 conn(0x7f73781360f0 legacy=0x7f73780a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73640039c0 0x7f7364003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): picked mon.a con 0x7f73640039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f7364004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): picked mon.b con 0x7f7364004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 0x7f736400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): picked mon.c con 0x7f7378135d10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f737809f6f0 con 0x7f73640039c0 2024-09-15T21:20:26.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7364005680 con 0x7f7364004810 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f736400bf90 con 0x7f7378135d10 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f7364004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): get_auth_request con 0x7f7364004810 auth_method 0 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737e69e640 1 -- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 msgr2=0x7f736400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737e69e640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 0x7f736400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73640039c0 0x7f7364003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 10 monclient(hunting): get_auth_request con 0x7f73640039c0 auth_method 0 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient(hunting): handle_auth_done global_id 34324 payload 293 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 -- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 msgr2=0x7f736400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7378135d10 0x7f736400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 -- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73640039c0 msgr2=0x7f7364003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73640039c0 0x7f7364003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 monclient: found mon.b 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 -- 172.21.15.119:0/2951389663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7364003530 con 0x7f7364004810 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 5 monclient: authenticate success, global_id 34324 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737de9d640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73640039c0 0x7f7364003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737d69c640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f7364004c90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f736800c160 tx=0x7f736801cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4a8fa1430d38dee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368007c70 con 0x7f7364004810 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: dump: 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73680a9020 con 0x7f7364004810 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:26.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 1 -- 172.21.15.119:0/2951389663 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368005b00 con 0x7f7364004810 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f7380101640 4 set_mon_vals no callback set 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.087+0000 7f737ce9b640 10 monclient: dump: 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.109 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: shutdown 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2951389663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 msgr2=0x7f7364004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 --2- 172.21.15.119:0/2951389663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f7364004c90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f736800c160 tx=0x7f736801cb10 comp rx=0 tx=0).stop 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737e69e640 1 -- 172.21.15.119:0/2951389663 reap_dead start 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2951389663 shutdown_connections 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2951389663 >> 172.21.15.119:0/2951389663 conn(0x7f737806cdc0 msgr2=0x7f737806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2951389663 shutdown_connections 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2951389663 wait complete. 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: build_initial_monmap 2024-09-15T21:20:26.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: monmap: 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 Processor -- start 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- start start 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: init 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f737812d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): picked mon.noname-b con 0x7f7364004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f737812dc80 0x7f737812e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): picked mon.noname-a con 0x7f737812dc80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f737813cee0 0x7f737813d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): picked mon.noname-c con 0x7f737813cee0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7378136670 con 0x7f737812dc80 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73781364f0 con 0x7f7364004810 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f737813fa30 con 0x7f737813cee0 2024-09-15T21:20:26.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient(hunting): authenticate will time out at 1144.889788s 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f737812d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737e69e640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f737813cee0 msgr2=0x7f737813d2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f737812d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49054/0 (socket says 172.21.15.119:49054) 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737e69e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f737813cee0 0x7f737813d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 -- 172.21.15.119:0/2029303938 learned_addr learned my addr 172.21.15.119:0/2029303938 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): get_auth_request con 0x7f7364004810 auth_method 0 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:26.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737d69c640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f737812dc80 0x7f737812e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737d69c640 10 monclient(hunting): get_auth_request con 0x7f737812dc80 auth_method 0 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737d69c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737d69c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737d69c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient(hunting): handle_auth_done global_id 34327 payload 824 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 -- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f737813cee0 msgr2=0x7f737813d2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f737813cee0 0x7f737813d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 -- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f737812dc80 msgr2=0x7f737812e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f737812dc80 0x7f737812e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 monclient: found mon.noname-b 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 -- 172.21.15.119:0/2029303938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737809e870 con 0x7f7364004810 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f737de9d640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f737812d740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f737400d620 tx=0x7f737400dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374003bd0 con 0x7f7364004810 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: dump: 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.115 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7374017070 con 0x7f7364004810 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 5 monclient: authenticate success, global_id 34327 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73740050d0 con 0x7f7364004810 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736e7fc640 10 monclient: dump: 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f736ffff640 4 set_mon_vals no callback set 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _renew_subs 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737809d1d0 con 0x7f7364004810 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _renew_subs 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737809f130 con 0x7f7364004810 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.091+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73780a09e0 con 0x7f7364004810 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.095+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7374005270 con 0x7f7364004810 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.095+0000 7f736e7fc640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f735c03b9a0 0x7f735c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.095+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7f737404fec0 con 0x7f7364004810 2024-09-15T21:20:26.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.095+0000 7f737d69c640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f735c03b9a0 0x7f735c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.095+0000 7f737d69c640 10 monclient: get_auth_request con 0x7f735c03b9a0 auth_method 0 2024-09-15T21:20:26.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.095+0000 7f737d69c640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f735c03b9a0 0x7f735c03de60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f73680aa000 tx=0x7f73780a2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.099+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f737401c980 con 0x7f7364004810 2024-09-15T21:20:26.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.099+0000 7f736e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.099+0000 7f736e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:26.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.255+0000 7f7380101640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:26.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.255+0000 7f7380101640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.255+0000 7f7380101640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.255+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f73781324b0 con 0x7f7364004810 2024-09-15T21:20:26.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.259+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7f737401a540 con 0x7f7364004810 2024-09-15T21:20:26.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.259+0000 7f736e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.259+0000 7f736e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:26.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f7380101640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:26.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f7380101640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f7380101640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f73780726f0 con 0x7f7364004810 2024-09-15T21:20:26.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f736e7fc640 1 -- 172.21.15.119:0/2029303938 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7f737401c460 con 0x7f7364004810 2024-09-15T21:20:26.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f736e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.351+0000 7f736e7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f735c03b9a0 msgr2=0x7f735c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f735c03b9a0 0x7f735c03de60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f73680aa000 tx=0x7f73780a2040 comp rx=0 tx=0).stop 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 10 monclient: shutdown 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 msgr2=0x7f737812d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f737812d740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f737400d620 tx=0x7f737400dae0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 shutdown_connections 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f735c03b9a0 0x7f735c03de60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f737813cee0 0x7f737813d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f737812dc80 0x7f737812e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 --2- 172.21.15.119:0/2029303938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7364004810 0x7f737812d740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 >> 172.21.15.119:0/2029303938 conn(0x7f737806cdc0 msgr2=0x7f73780a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 shutdown_connections 2024-09-15T21:20:26.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.355+0000 7f7380101640 1 -- 172.21.15.119:0/2029303938 wait complete. 2024-09-15T21:20:26.373 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:26.373 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c host1 2024-09-15T21:20:26.373 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 2 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient: get_monmap_and_config 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient: build_initial_monmap 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient: monmap: 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:26.474901+0000 2024-09-15T21:20:26.476 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:26.474901+0000 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:26.477 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 Processor -- start 2024-09-15T21:20:26.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 -- start start 2024-09-15T21:20:26.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient: init 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efde40a4d80 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efde40a6250 0x7efde413cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): picked mon.noname-c con 0x7efde40a6250 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efde40a5870 0x7efde413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): picked mon.noname-a con 0x7efde40a5870 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde40a2600 con 0x7efde40a4d80 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7efde40a1880 con 0x7efde40a5870 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7efde40a5fd0 con 0x7efde40a6250 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efdeb261640 10 monclient(hunting): authenticate will time out at 1145.271502s 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde8ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efde40a4d80 0x7efde40a5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39184/0 (socket says 172.21.15.119:39184) 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde8ffd640 1 -- 172.21.15.119:0/4179275023 learned_addr learned my addr 172.21.15.119:0/4179275023 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:26.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 1 --2- 172.21.15.119:0/4179275023 >> v2:172.21.15.119:3300/0 conn(0x7efde40a6250 0x7efde413cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): get_auth_request con 0x7efde40a6250 auth_method 0 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde97fe640 1 --2- 172.21.15.119:0/4179275023 >> v2:172.21.15.17:3300/0 conn(0x7efde40a5870 0x7efde413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde97fe640 10 monclient(hunting): get_auth_request con 0x7efde40a5870 auth_method 0 2024-09-15T21:20:26.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde97fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde97fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.471+0000 7efde97fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 10 monclient(hunting): handle_auth_done global_id 34330 payload 274 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 1 -- 172.21.15.119:0/4179275023 >> v2:172.21.15.17:3300/0 conn(0x7efde40a5870 msgr2=0x7efde413f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 1 --2- 172.21.15.119:0/4179275023 >> v2:172.21.15.17:3300/0 conn(0x7efde40a5870 0x7efde413f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 1 -- 172.21.15.119:0/4179275023 >> v1:172.21.15.119:6789/0 conn(0x7efde40a4d80 legacy=0x7efde40a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 1 monclient: found mon.noname-c 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 1 -- 172.21.15.119:0/4179275023 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde40a1d60 con 0x7efde40a6250 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde3fff640 1 --2- 172.21.15.119:0/4179275023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a6250 0x7efde413cf90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efde4074580 tx=0x7efdd001c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c8c526f6ecf9ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 1 -- 172.21.15.119:0/4179275023 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b0070 con 0x7efde40a6250 2024-09-15T21:20:26.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: dump: 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 1 -- 172.21.15.119:0/4179275023 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00ab600 con 0x7efde40a6250 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 5 monclient: authenticate success, global_id 34330 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 1 -- 172.21.15.119:0/4179275023 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab9d0 con 0x7efde40a6250 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 4 set_mon_vals no callback set 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efde37fe640 10 monclient: dump: 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.487 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 10 monclient: shutdown 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 -- 172.21.15.119:0/4179275023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a6250 msgr2=0x7efde413cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 --2- 172.21.15.119:0/4179275023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a6250 0x7efde413cf90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efde4074580 tx=0x7efdd001c310 comp rx=0 tx=0).stop 2024-09-15T21:20:26.488 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 -- 172.21.15.119:0/4179275023 shutdown_connections 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 --2- 172.21.15.119:0/4179275023 >> v2:172.21.15.17:3300/0 conn(0x7efde40a5870 0x7efde413f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 --2- 172.21.15.119:0/4179275023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a6250 0x7efde413cf90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 -- 172.21.15.119:0/4179275023 >> 172.21.15.119:0/4179275023 conn(0x7efde40a7640 msgr2=0x7efde406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 -- 172.21.15.119:0/4179275023 shutdown_connections 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 -- 172.21.15.119:0/4179275023 wait complete. 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 10 monclient: build_initial_monmap 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 10 monclient: monmap: 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.489 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:26.490 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:26.490 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:26.490 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 Processor -- start 2024-09-15T21:20:26.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 1 -- start start 2024-09-15T21:20:26.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.475+0000 7efdeb261640 10 monclient: init 2024-09-15T21:20:26.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:26.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efde40a4d80 0x7efde413a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): picked mon.noname-c con 0x7efde40a4d80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:26.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 0x7efde413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): picked mon.noname-b con 0x7efde40a5870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde40a6250 0x7efde413aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): picked mon.noname-a con 0x7efde40a6250 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efde4142240 con 0x7efde40a6250 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efde406d620 con 0x7efde40a5870 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efde41423c0 con 0x7efde40a4d80 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient(hunting): authenticate will time out at 1145.276395s 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 0x7efde413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 0x7efde413a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49076/0 (socket says 172.21.15.119:49076) 2024-09-15T21:20:26.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 -- 172.21.15.119:0/2685916489 learned_addr learned my addr 172.21.15.119:0/2685916489 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): get_auth_request con 0x7efde40a5870 auth_method 0 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde8ffd640 1 -- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efde40a4d80 msgr2=0x7efde413a120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde8ffd640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efde40a4d80 0x7efde413a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient(hunting): handle_auth_done global_id 34336 payload 824 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde97fe640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde40a6250 0x7efde413aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 -- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efde40a4d80 msgr2=0x7efde413a120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:26.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efde40a4d80 0x7efde413a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 -- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde40a6250 msgr2=0x7efde413aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde40a6250 0x7efde413aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 monclient: found mon.noname-b 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 -- 172.21.15.119:0/2685916489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde413dbe0 con 0x7efde40a5870 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde97fe640 10 monclient: get_auth_request con 0x7efde40a6250 auth_method 0 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde97fe640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde40a6250 0x7efde413aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde3fff640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 0x7efde413a660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ae040 tx=0x7efdd00190b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b0040 con 0x7efde40a5870 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: dump: 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00b9070 con 0x7efde40a5870 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b4690 con 0x7efde40a5870 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 10 monclient: dump: 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.498 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde2ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 5 monclient: authenticate success, global_id 34336 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _renew_subs 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde413f640 con 0x7efde40a5870 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _renew_subs 2024-09-15T21:20:26.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde413d2f0 con 0x7efde40a5870 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde40a5160 con 0x7efde40a5870 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7efdd00b4830 con 0x7efde40a5870 2024-09-15T21:20:26.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdcc03b970 0x7efdcc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7efdd00ea040 con 0x7efde40a5870 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde8ffd640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdcc03b970 0x7efdcc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde8ffd640 10 monclient: get_auth_request con 0x7efdcc03b970 auth_method 0 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.479+0000 7efde8ffd640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdcc03b970 0x7efdcc03de30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdd4004820 tx=0x7efdd4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.483+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7efdd0016050 con 0x7efde40a5870 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.483+0000 7efde17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.483+0000 7efde17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:26.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.639+0000 7efdeb261640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:26.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.639+0000 7efdeb261640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.639+0000 7efdeb261640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.639+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7efde406c480 con 0x7efde40a5870 2024-09-15T21:20:26.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.643+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7efdd00b3390 con 0x7efde40a5870 2024-09-15T21:20:26.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.643+0000 7efde17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.643+0000 7efde17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:26.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:26.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7efde413c6c0 con 0x7efde40a5870 2024-09-15T21:20:26.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efde17fa640 1 -- 172.21.15.119:0/2685916489 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7efdd00be460 con 0x7efde40a5870 2024-09-15T21:20:26.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efde17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:26.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efde17fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdcc03b970 msgr2=0x7efdcc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdcc03b970 0x7efdcc03de30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdd4004820 tx=0x7efdd4007b50 comp rx=0 tx=0).stop 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 10 monclient: shutdown 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 msgr2=0x7efde413a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 0x7efde413a660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ae040 tx=0x7efdd00190b0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 shutdown_connections 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdcc03b970 0x7efdcc03de30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde40a6250 0x7efde413aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efde40a5870 0x7efde413a660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 --2- 172.21.15.119:0/2685916489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efde40a4d80 0x7efde413a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 >> 172.21.15.119:0/2685916489 conn(0x7efde40a7640 msgr2=0x7efde4137e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 shutdown_connections 2024-09-15T21:20:26.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.735+0000 7efdeb261640 1 -- 172.21.15.119:0/2685916489 wait complete. 2024-09-15T21:20:26.755 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:26.755 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c osd.9 2024-09-15T21:20:26.755 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 2 2024-09-15T21:20:26.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient: get_monmap_and_config 2024-09-15T21:20:26.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient: build_initial_monmap 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient: monmap: 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:26.867312+0000 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:26.867312+0000 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:26.869 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 1 Processor -- start 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 1 -- start start 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient: init 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fccdc09d4e0 0x7fccdc09f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient(hunting): picked mon.noname-c con 0x7fccdc09d4e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fccdc09d8c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:26.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fccdc139ce0 0x7fccdc13c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 10 monclient(hunting): picked mon.noname-a con 0x7fccdc139ce0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.863+0000 7fcce3fc3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccdc0a09e0 con 0x7fccdc09d8c0 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fccdc074760 con 0x7fccdc139ce0 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fccdc0a0f30 con 0x7fccdc09d4e0 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient(hunting): authenticate will time out at 1145.664034s 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fccdc09d4e0 0x7fccdc09f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fccdc09d4e0 0x7fccdc09f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49090/0 (socket says 172.21.15.119:49090) 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 -- 172.21.15.119:0/3476901869 learned_addr learned my addr 172.21.15.119:0/3476901869 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): get_auth_request con 0x7fccdc09d4e0 auth_method 0 2024-09-15T21:20:26.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce2560640 1 --2- 172.21.15.119:0/3476901869 >> v2:172.21.15.17:3300/0 conn(0x7fccdc139ce0 0x7fccdc13c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_done global_id 34339 payload 274 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 -- 172.21.15.119:0/3476901869 >> v2:172.21.15.17:3300/0 conn(0x7fccdc139ce0 msgr2=0x7fccdc13c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 --2- 172.21.15.119:0/3476901869 >> v2:172.21.15.17:3300/0 conn(0x7fccdc139ce0 0x7fccdc13c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 -- 172.21.15.119:0/3476901869 >> v1:172.21.15.17:6789/0 conn(0x7fccdc09d8c0 legacy=0x7fccdc09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 monclient: found mon.noname-c 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 -- 172.21.15.119:0/3476901869 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc09e6e0 con 0x7fccdc09d4e0 2024-09-15T21:20:26.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce2560640 10 monclient: get_auth_request con 0x7fccdc139ce0 auth_method 0 2024-09-15T21:20:26.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce2560640 1 --2- 172.21.15.119:0/3476901869 >> v2:172.21.15.17:3300/0 conn(0x7fccdc139ce0 0x7fccdc13c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:26.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce1d5f640 1 --2- 172.21.15.119:0/3476901869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc09f410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0a5f20 tx=0x7fccd801ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1c7cdd74243eac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 1 -- 172.21.15.119:0/3476901869 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b4070 con 0x7fccdc09d4e0 2024-09-15T21:20:26.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: dump: 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 1 -- 172.21.15.119:0/3476901869 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd80a7800 con 0x7fccdc09d4e0 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 5 monclient: authenticate success, global_id 34339 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 1 -- 172.21.15.119:0/3476901869 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80a7bd0 con 0x7fccdc09d4e0 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 4 set_mon_vals no callback set 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce0d5d640 10 monclient: dump: 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.881 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient: shutdown 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- 172.21.15.119:0/3476901869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 msgr2=0x7fccdc09f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/3476901869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc09f410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0a5f20 tx=0x7fccd801ea70 comp rx=0 tx=0).stop 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- 172.21.15.119:0/3476901869 shutdown_connections 2024-09-15T21:20:26.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/3476901869 >> v2:172.21.15.17:3300/0 conn(0x7fccdc139ce0 0x7fccdc13c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/3476901869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc09f410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- 172.21.15.119:0/3476901869 >> 172.21.15.119:0/3476901869 conn(0x7fccdc06cdc0 msgr2=0x7fccdc06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- 172.21.15.119:0/3476901869 shutdown_connections 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 1 -- 172.21.15.119:0/3476901869 wait complete. 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient: build_initial_monmap 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.867+0000 7fcce3fc3640 10 monclient: monmap: 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:26.883 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 Processor -- start 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- start start 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: init 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): picked mon.noname-b con 0x7fccdc09d4e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccdc09d8c0 0x7fccdc134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): picked mon.noname-a con 0x7fccdc09d8c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fccdc139ce0 0x7fccdc135220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): picked mon.noname-c con 0x7fccdc139ce0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc0a1c00 con 0x7fccdc09d8c0 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc0a1a80 con 0x7fccdc09d4e0 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fccdc0a1d80 con 0x7fccdc139ce0 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient(hunting): authenticate will time out at 1145.668910s 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc072b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49094/0 (socket says 172.21.15.119:49094) 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 -- 172.21.15.119:0/4029909959 learned_addr learned my addr 172.21.15.119:0/4029909959 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): get_auth_request con 0x7fccdc09d4e0 auth_method 0 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce2560640 1 -- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fccdc139ce0 msgr2=0x7fccdc135220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce2560640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fccdc139ce0 0x7fccdc135220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient(hunting): handle_auth_done global_id 34342 payload 824 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 -- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fccdc139ce0 msgr2=0x7fccdc135220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fccdc139ce0 0x7fccdc135220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 -- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccdc09d8c0 msgr2=0x7fccdc134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccdc09d8c0 0x7fccdc134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient: _finish_hunting 0 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 monclient: found mon.noname-b 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 -- 172.21.15.119:0/4029909959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc122690 con 0x7fccdc09d4e0 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce1d5f640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc072b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fccd800b2c0 tx=0x7fccd801d2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80bf070 con 0x7fccdc09d4e0 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: dump: 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd80af9d0 con 0x7fccdc09d4e0 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 5 monclient: authenticate success, global_id 34342 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _renew_subs 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc0735c0 con 0x7fccdc09d4e0 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b4070 con 0x7fccdc09d4e0 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 10 monclient: dump: 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd3fff640 4 set_mon_vals no callback set 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _renew_subs 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc122e10 con 0x7fccdc09d4e0 2024-09-15T21:20:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc09f410 con 0x7fccdc09d4e0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fccd80c2070 con 0x7fccdc09d4e0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fccc003b970 0x7fccc003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7fccd80ea770 con 0x7fccdc09d4e0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce155e640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fccc003b970 0x7fccc003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce155e640 10 monclient: get_auth_request con 0x7fccc003b970 auth_method 0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.871+0000 7fcce155e640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fccc003b970 0x7fccc003de30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fccdc13a510 tx=0x7fcccc0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:26.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.875+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fccd8018780 con 0x7fccdc09d4e0 2024-09-15T21:20:26.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.875+0000 7fccd27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:26.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:26.875+0000 7fccd27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:27.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fcce3fc3640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:27.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fcce3fc3640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fcce3fc3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fccdc073ce0 con 0x7fccdc09d4e0 2024-09-15T21:20:27.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7fccd80afdc0 con 0x7fccdc09d4e0 2024-09-15T21:20:27.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fccd27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.035+0000 7fccd27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:27.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fcce3fc3640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:27.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fcce3fc3640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fcce3fc3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fccdc06c440 con 0x7fccdc09d4e0 2024-09-15T21:20:27.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fccd27fc640 1 -- 172.21.15.119:0/4029909959 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7fccd80c3d70 con 0x7fccdc09d4e0 2024-09-15T21:20:27.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fccd27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.127+0000 7fccd27fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fccc003b970 msgr2=0x7fccc003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fccc003b970 0x7fccc003de30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fccdc13a510 tx=0x7fcccc0079c0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 10 monclient: shutdown 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 msgr2=0x7fccdc072b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc072b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fccd800b2c0 tx=0x7fccd801d2b0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 shutdown_connections 2024-09-15T21:20:27.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fccc003b970 0x7fccc003de30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fccdc139ce0 0x7fccdc135220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccdc09d8c0 0x7fccdc134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 --2- 172.21.15.119:0/4029909959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccdc09d4e0 0x7fccdc072b10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 >> 172.21.15.119:0/4029909959 conn(0x7fccdc06cdc0 msgr2=0x7fccdc0a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:27.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 shutdown_connections 2024-09-15T21:20:27.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.131+0000 7fcce3fc3640 1 -- 172.21.15.119:0/4029909959 wait complete. 2024-09-15T21:20:27.150 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:27.150 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c host2 2024-09-15T21:20:27.150 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 2 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.251+0000 7f56b5d48640 10 monclient: get_monmap_and_config 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.251+0000 7f56b5d48640 10 monclient: build_initial_monmap 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.251+0000 7f56b5d48640 10 monclient: monmap: 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:27.255656+0000 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:27.255656+0000 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:27.257 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.258 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:27.258 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:27.258 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:27.258 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:27.258 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:27.258 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.251+0000 7f56b5d48640 1 Processor -- start 2024-09-15T21:20:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.251+0000 7f56b5d48640 1 -- start start 2024-09-15T21:20:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.251+0000 7f56b5d48640 10 monclient: init 2024-09-15T21:20:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2024-09-15T21:20:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f56b006d8e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 0x7f56b00a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient(hunting): picked mon.noname-b con 0x7f56b009fc70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f56b0139100 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b01317b0 con 0x7f56b0139100 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00a28a0 con 0x7f56b006d8e0 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f56b009d6f0 con 0x7f56b009fc70 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 10 monclient(hunting): authenticate will time out at 1146.052565s 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f56b006d8e0 0x7f56b01310a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39198/0 (socket says 172.21.15.119:39198) 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 -- 172.21.15.119:0/4076679695 learned_addr learned my addr 172.21.15.119:0/4076679695 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 1 -- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 msgr2=0x7f56b00a00c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:27.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 0x7f56b00a00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495108212 0 0) 0x7f56b00a28a0 con 0x7f56b006d8e0 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): my global_id is 34345 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5698003450 con 0x7f56b006d8e0 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3633878093 0 0) 0x7f5698003450 con 0x7f56b006d8e0 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 msgr2=0x7f56b00a00c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 0x7f56b00a00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 >> v1:172.21.15.17:6789/0 conn(0x7f56b0139100 legacy=0x7f56b013b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:27.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b009dea0 con 0x7f56b006d8e0 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f56a0001700 con 0x7f56b006d8e0 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 5 monclient: authenticate success, global_id 34345 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: dump: 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.271 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 >> v1:172.21.15.119:6789/0 conn(0x7f56b006d8e0 legacy=0x7f56b01310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:27.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5698003ab0 0x7f5698003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): picked mon.b con 0x7f5698003ab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56980089f0 0x7f569800ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): picked mon.a con 0x7f56980089f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 0x7f569800d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): picked mon.c con 0x7f56b009fc70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f56b009d6f0 con 0x7f56980089f0 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f569800df30 con 0x7f5698003ab0 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f569800e130 con 0x7f56b009fc70 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56ae7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56b5d48640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5698003ab0 0x7f5698003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): get_auth_request con 0x7f5698003ab0 auth_method 0 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56affff640 1 -- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 msgr2=0x7f569800d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56affff640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 0x7f569800d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56980089f0 0x7f569800ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 10 monclient(hunting): get_auth_request con 0x7f56980089f0 auth_method 0 2024-09-15T21:20:27.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56aeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient(hunting): handle_auth_done global_id 34345 payload 293 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 -- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 msgr2=0x7f569800d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56b009fc70 0x7f569800d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 -- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56980089f0 msgr2=0x7f569800ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56980089f0 0x7f569800ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 monclient: found mon.b 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 1 -- 172.21.15.119:0/4076679695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5698003450 con 0x7f5698003ab0 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.255+0000 7f56af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56af7fe640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5698003ab0 0x7f5698003ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c190 tx=0x7f56a001cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c09b3b631d0d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007cd0 con 0x7f5698003ab0 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: dump: 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.276 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a00a9020 con 0x7f5698003ab0 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 1 -- 172.21.15.119:0/4076679695 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005b20 con 0x7f5698003ab0 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 4 set_mon_vals no callback set 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56ae7fc640 10 monclient: dump: 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: shutdown 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:27.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/4076679695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5698003ab0 msgr2=0x7f5698003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 --2- 172.21.15.119:0/4076679695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5698003ab0 0x7f5698003ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c190 tx=0x7f56a001cb30 comp rx=0 tx=0).stop 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56affff640 1 -- 172.21.15.119:0/4076679695 reap_dead start 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/4076679695 shutdown_connections 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/4076679695 >> 172.21.15.119:0/4076679695 conn(0x7f56b006cdc0 msgr2=0x7f56b006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/4076679695 shutdown_connections 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/4076679695 wait complete. 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: build_initial_monmap 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: monmap: 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:27.279 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 Processor -- start 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- start start 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: init 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5698003ab0 0x7f56b006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): picked mon.noname-c con 0x7f5698003ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:27.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 0x7f56b013aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): picked mon.noname-b con 0x7f56b013aab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56b013b430 0x7f56b013b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): picked mon.noname-a con 0x7f56b013b430 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f56b013cd90 con 0x7f56b013b430 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56b013cc10 con 0x7f56b013aab0 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f56b013cf10 con 0x7f5698003ab0 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient(hunting): authenticate will time out at 1146.058516s 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 0x7f56b013aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56af7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5698003ab0 msgr2=0x7f56b006f6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:27.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 0x7f56b013aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49116/0 (socket says 172.21.15.119:49116) 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 -- 172.21.15.119:0/723142867 learned_addr learned my addr 172.21.15.119:0/723142867 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56af7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5698003ab0 0x7f56b006f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): get_auth_request con 0x7f56b013aab0 auth_method 0 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56affff640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56b013b430 0x7f56b013b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56affff640 10 monclient(hunting): get_auth_request con 0x7f56b013b430 auth_method 0 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient(hunting): handle_auth_done global_id 34348 payload 824 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 -- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5698003ab0 msgr2=0x7f56b006f6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5698003ab0 0x7f56b006f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 -- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56b013b430 msgr2=0x7f56b013b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56b013b430 0x7f56b013b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 monclient: found mon.noname-b 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 -- 172.21.15.119:0/723142867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b009d1d0 con 0x7f56b013aab0 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56aeffd640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 0x7f56b013aef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56a400d7d0 tx=0x7f56a400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4016020 con 0x7f56b013aab0 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: dump: 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 5 monclient: authenticate success, global_id 34348 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: _renew_subs 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a40039b0 con 0x7f56b013aab0 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b0070030 con 0x7f56b013aab0 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003da0 con 0x7f56b013aab0 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f5697fff640 10 monclient: dump: 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.285 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: _renew_subs 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56adffb640 4 set_mon_vals no callback set 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.259+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b006ebe0 con 0x7f56b013aab0 2024-09-15T21:20:27.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56b5d48640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56b5d48640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56b5d48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b01310a0 con 0x7f56b013aab0 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f56a4019020 con 0x7f56b013aab0 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f5697fff640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f569c03b970 0x7f569c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7f56a404fcb0 con 0x7f56b013aab0 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56af7fe640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f569c03b970 0x7f569c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56af7fe640 10 monclient: get_auth_request con 0x7f569c03b970 auth_method 0 2024-09-15T21:20:27.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.263+0000 7f56af7fe640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f569c03b970 0x7f569c03de30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f56a00aa000 tx=0x7f56b009d8c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.267+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f56a4020030 con 0x7f56b013aab0 2024-09-15T21:20:27.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.267+0000 7f5697fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.267+0000 7f5697fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:27.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f56b5d48640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:27.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f56b5d48640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f56b5d48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f56b006c440 con 0x7f56b013aab0 2024-09-15T21:20:27.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7f56a40096a0 con 0x7f56b013aab0 2024-09-15T21:20:27.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f5697fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.423+0000 7f5697fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:27.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f56b5d48640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:27.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f56b5d48640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f56b5d48640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f56b01d20b0 con 0x7f56b013aab0 2024-09-15T21:20:27.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f5697fff640 1 -- 172.21.15.119:0/723142867 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v74) ==== 56+0+1474 (secure 0 0 0) 0x7f56a40055a0 con 0x7f56b013aab0 2024-09-15T21:20:27.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f5697fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:27.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.519+0000 7f5697fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:27.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f569c03b970 msgr2=0x7f569c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f569c03b970 0x7f569c03de30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f56a00aa000 tx=0x7f56b009d8c0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 10 monclient: shutdown 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 msgr2=0x7f56b013aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 0x7f56b013aef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56a400d7d0 tx=0x7f56a400dc90 comp rx=0 tx=0).stop 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 shutdown_connections 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f569c03b970 0x7f569c03de30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56b013b430 0x7f56b013b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b013aab0 0x7f56b013aef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 --2- 172.21.15.119:0/723142867 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5698003ab0 0x7f56b006f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 >> 172.21.15.119:0/723142867 conn(0x7f56b006cdc0 msgr2=0x7f56b0136490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 shutdown_connections 2024-09-15T21:20:27.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.523+0000 7f56b5d48640 1 -- 172.21.15.119:0/723142867 wait complete. 2024-09-15T21:20:27.542 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush rm host1 foo 2024-09-15T21:20:27.542 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:27.542 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm host1 foo 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.651+0000 7fc675f81640 10 monclient: get_monmap_and_config 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.651+0000 7fc675f81640 10 monclient: build_initial_monmap 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.651+0000 7fc675f81640 10 monclient: monmap: 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:27.656577+0000 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:27.656577+0000 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.658 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:27.659 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:27.659 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:27.659 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:27.659 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:27.659 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 1 Processor -- start 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 1 -- start start 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient: init 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6700a0900 0x7fc6700a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient(hunting): picked mon.noname-c con 0x7fc6700a0900 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc6700a1220 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc670135df0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700a2240 con 0x7fc6700a1220 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700a1c80 con 0x7fc670135df0 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc67009fdd0 con 0x7fc6700a0900 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc675f81640 10 monclient(hunting): authenticate will time out at 1146.453500s 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6700a0900 0x7fc6700a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6700a0900 0x7fc6700a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49126/0 (socket says 172.21.15.119:49126) 2024-09-15T21:20:27.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc670135df0 0x7fc67013dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39200/0 (socket says 172.21.15.119:39200) 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 -- 172.21.15.119:0/240321775 learned_addr learned my addr 172.21.15.119:0/240321775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): get_auth_request con 0x7fc6700a0900 auth_method 0 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient(hunting): handle_auth_done global_id 34351 payload 274 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 -- 172.21.15.119:0/240321775 >> v1:172.21.15.119:6789/0 conn(0x7fc670135df0 legacy=0x7fc67013dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 -- 172.21.15.119:0/240321775 >> v1:172.21.15.17:6789/0 conn(0x7fc6700a1220 legacy=0x7fc6701356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:20:27.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.655+0000 7fc66f7fe640 1 -- 172.21.15.119:0/240321775 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6700a1f60 con 0x7fc6700a0900 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66f7fe640 1 --2- 172.21.15.119:0/240321775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6700a0900 0x7fc6700a0ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc67009ff50 tx=0x7fc66001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a482d3de80520b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 1 -- 172.21.15.119:0/240321775 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6600b1070 con 0x7fc6700a0900 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: dump: 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.670 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 5 monclient: authenticate success, global_id 34351 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 1 -- 172.21.15.119:0/240321775 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6600ac640 con 0x7fc6700a0900 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:27.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 4 set_mon_vals no callback set 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 1 -- 172.21.15.119:0/240321775 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6600aca10 con 0x7fc6700a0900 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66e7fc640 10 monclient: dump: 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.672 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: shutdown 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/240321775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6700a0900 msgr2=0x7fc6700a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 --2- 172.21.15.119:0/240321775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6700a0900 0x7fc6700a0ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc67009ff50 tx=0x7fc66001c400 comp rx=0 tx=0).stop 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/240321775 shutdown_connections 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 --2- 172.21.15.119:0/240321775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6700a0900 0x7fc6700a0ce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/240321775 >> 172.21.15.119:0/240321775 conn(0x7fc67009be60 msgr2=0x7fc67009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/240321775 shutdown_connections 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/240321775 wait complete. 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: build_initial_monmap 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: monmap: 2024-09-15T21:20:27.673 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 Processor -- start 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- start start 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: init 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6700a0900 0x7fc670130d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): picked mon.noname-c con 0x7fc6700a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6700a1220 0x7fc670138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): picked mon.noname-a con 0x7fc6700a1220 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc670135df0 0x7fc670138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): picked mon.noname-b con 0x7fc670135df0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc67006ee90 con 0x7fc6700a1220 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc67006ed10 con 0x7fc670135df0 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc6700a0050 con 0x7fc6700a0900 2024-09-15T21:20:27.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient(hunting): authenticate will time out at 1146.458385s 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66f7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6700a0900 msgr2=0x7fc670130d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6700a0900 0x7fc670130d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc670135df0 0x7fc670138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc670135df0 0x7fc670138ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49134/0 (socket says 172.21.15.119:49134) 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 -- 172.21.15.119:0/969225243 learned_addr learned my addr 172.21.15.119:0/969225243 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): get_auth_request con 0x7fc670135df0 auth_method 0 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:27.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66effd640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6700a1220 0x7fc670138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66effd640 10 monclient(hunting): get_auth_request con 0x7fc6700a1220 auth_method 0 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient(hunting): handle_auth_done global_id 34357 payload 824 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 -- 172.21.15.119:0/969225243 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6700a0900 msgr2=0x7fc670130d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6700a0900 0x7fc670130d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 -- 172.21.15.119:0/969225243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6700a1220 msgr2=0x7fc670138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6700a1220 0x7fc670138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 monclient: found mon.noname-b 2024-09-15T21:20:27.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 -- 172.21.15.119:0/969225243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67013a450 con 0x7fc670135df0 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66ffff640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc670135df0 0x7fc670138ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c8a0 tx=0x7fc66800cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668017070 con 0x7fc670135df0 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: dump: 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.678 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66800a640 con 0x7fc670135df0 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 5 monclient: authenticate success, global_id 34357 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: _renew_subs 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/969225243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67006f0b0 con 0x7fc670135df0 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66800a9f0 con 0x7fc670135df0 2024-09-15T21:20:27.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc657fff640 10 monclient: dump: 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: _renew_subs 2024-09-15T21:20:27.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc675f81640 1 -- 172.21.15.119:0/969225243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67013bf10 con 0x7fc670135df0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.659+0000 7fc66dffb640 4 set_mon_vals no callback set 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc675f81640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc675f81640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc675f81640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc675f81640 1 -- 172.21.15.119:0/969225243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6700a0ce0 con 0x7fc670135df0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc66801a070 con 0x7fc670135df0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc657fff640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc65c03b9a0 0x7fc65c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7fc66804f670 con 0x7fc670135df0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc66f7fe640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc65c03b9a0 0x7fc65c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc66f7fe640 10 monclient: get_auth_request con 0x7fc65c03b9a0 auth_method 0 2024-09-15T21:20:27.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.663+0000 7fc66f7fe640 1 --2- 172.21.15.119:0/969225243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc65c03b9a0 0x7fc65c03de60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6600ae000 tx=0x7fc660008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.667+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc6680113a0 con 0x7fc670135df0 2024-09-15T21:20:27.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.667+0000 7fc657fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.667+0000 7fc657fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:27.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc675f81640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:27.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc675f81640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:27.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc675f81640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc675f81640 1 -- 172.21.15.119:0/969225243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"} v 0) -- 0x7fc67006a810 con 0x7fc670135df0 2024-09-15T21:20:27.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc657fff640 1 -- 172.21.15.119:0/969225243 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}]=-39 (39) Directory not empty v74) ==== 120+0+0 (secure 0 0 0) 0x7fc668012df0 con 0x7fc670135df0 2024-09-15T21:20:27.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc657fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:27.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.835+0000 7fc657fff640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T21:20:27.841 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T21:20:27.847 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:27.847 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush unlink host1 foo 2024-09-15T21:20:27.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.959+0000 7f30217e6640 10 monclient: get_monmap_and_config 2024-09-15T21:20:27.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.959+0000 7f30217e6640 10 monclient: build_initial_monmap 2024-09-15T21:20:27.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.959+0000 7f30217e6640 10 monclient: monmap: 2024-09-15T21:20:27.965 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:27.965 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.965 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:27.963650+0000 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:27.963650+0000 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:27.966 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.959+0000 7f30217e6640 1 Processor -- start 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.959+0000 7f30217e6640 1 -- start start 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.959+0000 7f30217e6640 10 monclient: init 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a79e0 0x7f301c0a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): picked mon.noname-b con 0x7f301c0a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f301c0a46b0 0x7f301c1377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): picked mon.noname-c con 0x7f301c0a46b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f301c0a1100 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f301c135180 con 0x7f301c0a1100 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f301c0a7dc0 con 0x7f301c0a46b0 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f301c0a6920 con 0x7f301c0a79e0 2024-09-15T21:20:27.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient(hunting): authenticate will time out at 1146.760364s 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f301c0a46b0 0x7f301c1377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301affd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a79e0 msgr2=0x7f301c0a4170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f301c0a46b0 0x7f301c1377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49136/0 (socket says 172.21.15.119:49136) 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 -- 172.21.15.119:0/1207997946 learned_addr learned my addr 172.21.15.119:0/1207997946 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): get_auth_request con 0x7f301c0a46b0 auth_method 0 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301affd640 1 --2- 172.21.15.119:0/1207997946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a79e0 0x7f301c0a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient(hunting): handle_auth_done global_id 34360 payload 274 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 -- 172.21.15.119:0/1207997946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a79e0 msgr2=0x7f301c0a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 --2- 172.21.15.119:0/1207997946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a79e0 0x7f301c0a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 -- 172.21.15.119:0/1207997946 >> v1:172.21.15.119:6789/0 conn(0x7f301c0a1100 legacy=0x7f301c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 monclient: found mon.noname-c 2024-09-15T21:20:27.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 -- 172.21.15.119:0/1207997946 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c0a1c30 con 0x7f301c0a46b0 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f301a7fc640 1 --2- 172.21.15.119:0/1207997946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1377e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3010009870 tx=0x7f301001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc032ffaba7cc1b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 1 -- 172.21.15.119:0/1207997946 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c134930 con 0x7f301c0a46b0 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: dump: 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.980 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 1 -- 172.21.15.119:0/1207997946 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30100af070 con 0x7f301c0a46b0 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 5 monclient: authenticate success, global_id 34360 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f30217e6640 4 set_mon_vals no callback set 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 1 -- 172.21.15.119:0/1207997946 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100aa870 con 0x7f301c0a46b0 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.963+0000 7f3019ffb640 10 monclient: dump: 2024-09-15T21:20:27.982 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: shutdown 2024-09-15T21:20:27.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/1207997946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 msgr2=0x7f301c1377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 --2- 172.21.15.119:0/1207997946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1377e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3010009870 tx=0x7f301001c4d0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/1207997946 shutdown_connections 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 --2- 172.21.15.119:0/1207997946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1377e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 --2- 172.21.15.119:0/1207997946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a79e0 0x7f301c0a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/1207997946 >> 172.21.15.119:0/1207997946 conn(0x7f301c0a6ef0 msgr2=0x7f301c0a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/1207997946 shutdown_connections 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/1207997946 wait complete. 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: build_initial_monmap 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: monmap: 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:27.984 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 Processor -- start 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- start start 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: init 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:27.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a1100 0x7f301c131160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): picked mon.noname-c con 0x7f301c0a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1316a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): picked mon.noname-b con 0x7f301c0a46b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f301c0a79e0 0x7f301c072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): picked mon.noname-a con 0x7f301c0a79e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1385b0 con 0x7f301c0a79e0 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f301c138430 con 0x7f301c0a46b0 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f301c1423a0 con 0x7f301c0a1100 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient(hunting): authenticate will time out at 1146.765731s 2024-09-15T21:20:27.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1316a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49148/0 (socket says 172.21.15.119:49148) 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 -- 172.21.15.119:0/4003314358 learned_addr learned my addr 172.21.15.119:0/4003314358 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): get_auth_request con 0x7f301c0a46b0 auth_method 0 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301affd640 1 -- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a1100 msgr2=0x7f301c131160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301affd640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a1100 0x7f301c131160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient(hunting): handle_auth_done global_id 34363 payload 824 2024-09-15T21:20:27.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 -- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a1100 msgr2=0x7f301c131160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301b7fe640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f301c0a79e0 0x7f301c072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a1100 0x7f301c131160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 -- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f301c0a79e0 msgr2=0x7f301c072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f301c0a79e0 0x7f301c072310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 monclient: found mon.noname-b 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 -- 172.21.15.119:0/4003314358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c073d80 con 0x7f301c0a46b0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f301a7fc640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1316a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f30100ac040 tx=0x7f301001ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100aae60 con 0x7f301c0a46b0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: dump: 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:27.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30100a6040 con 0x7f301c0a46b0 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 5 monclient: authenticate success, global_id 34363 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _renew_subs 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301c139b40 con 0x7f301c0a46b0 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100b3a60 con 0x7f301c0a46b0 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 10 monclient: dump: 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:27.990 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _renew_subs 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301c13af30 con 0x7f301c0a46b0 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f3018ff9640 4 set_mon_vals no callback set 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301c0a4170 con 0x7f301c0a46b0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.967+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f30100b3c00 con 0x7f301c0a46b0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f30037fe640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f300803b970 0x7f300803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6297+0+0 (secure 0 0 0) 0x7f30100eaf20 con 0x7f301c0a46b0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f301affd640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f300803b970 0x7f300803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f301affd640 10 monclient: get_auth_request con 0x7f300803b970 auth_method 0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f301affd640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f300803b970 0x7f300803de30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f300c003940 tx=0x7f300c006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:27.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f301c139b40 con 0x7f301c0a46b0 2024-09-15T21:20:27.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f30037fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:27.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:27.971+0000 7f30037fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:28.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.147+0000 7f30217e6640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.147+0000 7f30217e6640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.147+0000 7f30217e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.147+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7f301c13a000 con 0x7f301c0a46b0 2024-09-15T21:20:28.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.443+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 removed item id -11 name 'host1' from crush map v75) ==== 147+0+0 (secure 0 0 0) 0x7f3010016150 con 0x7f301c0a46b0 2024-09-15T21:20:28.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.443+0000 7f30037fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.443+0000 7f30037fe640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-09-15T21:20:28.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30217e6640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30217e6640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30217e6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7f301c0008e0 con 0x7f301c0a46b0 2024-09-15T21:20:28.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30037fe640 1 -- 172.21.15.119:0/4003314358 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v75) ==== 141+0+0 (secure 0 0 0) 0x7f30100bbd50 con 0x7f301c0a46b0 2024-09-15T21:20:28.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30037fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.555+0000 7f30037fe640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-09-15T21:20:28.561 INFO:tasks.workunit.client.0.smithi119.stderr:item -11 does not appear in that position 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f300803b970 msgr2=0x7f300803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f300803b970 0x7f300803de30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f300c003940 tx=0x7f300c006d90 comp rx=0 tx=0).stop 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 10 monclient: shutdown 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 msgr2=0x7f301c1316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1316a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f30100ac040 tx=0x7f301001ad10 comp rx=0 tx=0).stop 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 shutdown_connections 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f300803b970 0x7f300803de30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f301c0a79e0 0x7f301c072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0a46b0 0x7f301c1316a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 --2- 172.21.15.119:0/4003314358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f301c0a1100 0x7f301c131160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 >> 172.21.15.119:0/4003314358 conn(0x7f301c0a6ef0 msgr2=0x7f301c0a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:28.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 shutdown_connections 2024-09-15T21:20:28.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.559+0000 7f30217e6640 1 -- 172.21.15.119:0/4003314358 wait complete. 2024-09-15T21:20:28.577 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush unlink host1 foo 2024-09-15T21:20:28.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient: get_monmap_and_config 2024-09-15T21:20:28.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient: build_initial_monmap 2024-09-15T21:20:28.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient: monmap: 2024-09-15T21:20:28.682 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:28.681266+0000 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:28.681266+0000 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:28.683 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 Processor -- start 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 -- start start 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient: init 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3aec137870 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:28.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3aec138360 0x7f3aec0a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): picked mon.noname-a con 0x7f3aec138360 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3aec13cdc0 0x7f3aec13f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): picked mon.noname-c con 0x7f3aec13cdc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aec0a1b60 con 0x7f3aec137870 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3aec0a27f0 con 0x7f3aec138360 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3aec0a1e40 con 0x7f3aec13cdc0 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3af0a69640 10 monclient(hunting): authenticate will time out at 1147.477858s 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3aec13cdc0 0x7f3aec13f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3aec13cdc0 0x7f3aec13f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49150/0 (socket says 172.21.15.119:49150) 2024-09-15T21:20:28.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 -- 172.21.15.119:0/491373077 learned_addr learned my addr 172.21.15.119:0/491373077 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): get_auth_request con 0x7f3aec13cdc0 auth_method 0 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3ae9d9b640 1 --2- 172.21.15.119:0/491373077 >> v2:172.21.15.17:3300/0 conn(0x7f3aec138360 0x7f3aec0a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient(hunting): handle_auth_done global_id 34366 payload 274 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 -- 172.21.15.119:0/491373077 >> v2:172.21.15.17:3300/0 conn(0x7f3aec138360 msgr2=0x7f3aec0a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 --2- 172.21.15.119:0/491373077 >> v2:172.21.15.17:3300/0 conn(0x7f3aec138360 0x7f3aec0a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 -- 172.21.15.119:0/491373077 >> v1:172.21.15.17:6789/0 conn(0x7f3aec137870 legacy=0x7f3aec137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:28.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 monclient: found mon.noname-c 2024-09-15T21:20:28.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:28.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.679+0000 7f3aead9d640 1 -- 172.21.15.119:0/491373077 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec0a2120 con 0x7f3aec13cdc0 2024-09-15T21:20:28.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aead9d640 1 --2- 172.21.15.119:0/491373077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec13cdc0 0x7f3aec13f1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3aec1387d0 tx=0x7f3ae401c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c297fd7406ee50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 1 -- 172.21.15.119:0/491373077 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40b0070 con 0x7f3aec13cdc0 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: dump: 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:28.695 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: _finish_auth 0 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: _check_auth_tickets 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 1 -- 172.21.15.119:0/491373077 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae40ab600 con 0x7f3aec13cdc0 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 5 monclient: authenticate success, global_id 34366 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:28.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 1 -- 172.21.15.119:0/491373077 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40ab9d0 con 0x7f3aec13cdc0 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 4 set_mon_vals no callback set 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae959a640 10 monclient: dump: 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:28.697 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: shutdown 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/491373077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec13cdc0 msgr2=0x7f3aec13f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 --2- 172.21.15.119:0/491373077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec13cdc0 0x7f3aec13f1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3aec1387d0 tx=0x7f3ae401c1d0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/491373077 shutdown_connections 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 --2- 172.21.15.119:0/491373077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec13cdc0 0x7f3aec13f1b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 --2- 172.21.15.119:0/491373077 >> v2:172.21.15.17:3300/0 conn(0x7f3aec138360 0x7f3aec0a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/491373077 >> 172.21.15.119:0/491373077 conn(0x7f3aec09be60 msgr2=0x7f3aec09c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/491373077 shutdown_connections 2024-09-15T21:20:28.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/491373077 wait complete. 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: build_initial_monmap 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: monmap: 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:28.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 Processor -- start 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- start start 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: init 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 0x7f3aec11fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): picked mon.noname-b con 0x7f3aec137870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3aec138360 0x7f3aec11daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): picked mon.noname-c con 0x7f3aec138360 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aec13cdc0 0x7f3aec136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): picked mon.noname-a con 0x7f3aec13cdc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec142090 con 0x7f3aec13cdc0 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec141f10 con 0x7f3aec137870 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec142210 con 0x7f3aec138360 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient(hunting): authenticate will time out at 1147.482605s 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 0x7f3aec11fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 0x7f3aec11fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49152/0 (socket says 172.21.15.119:49152) 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 -- 172.21.15.119:0/781903044 learned_addr learned my addr 172.21.15.119:0/781903044 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): get_auth_request con 0x7f3aec137870 auth_method 0 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae9d9b640 1 -- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3aec138360 msgr2=0x7f3aec11daa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:28.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae9d9b640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3aec138360 0x7f3aec11daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aead9d640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aec13cdc0 0x7f3aec136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aead9d640 10 monclient(hunting): get_auth_request con 0x7f3aec13cdc0 auth_method 0 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient(hunting): handle_auth_done global_id 34369 payload 824 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 -- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3aec138360 msgr2=0x7f3aec11daa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3aec138360 0x7f3aec11daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 -- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aec13cdc0 msgr2=0x7f3aec136a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:28.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aec13cdc0 0x7f3aec136a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 monclient: found mon.noname-b 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 -- 172.21.15.119:0/781903044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec09d0f0 con 0x7f3aec137870 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3aea59c640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 0x7f3aec11fb90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c820 tx=0x7f3adc00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc016020 con 0x7f3aec137870 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: dump: 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.703 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3adc00aa50 con 0x7f3aec137870 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 5 monclient: authenticate success, global_id 34369 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: _renew_subs 2024-09-15T21:20:28.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec11ea40 con 0x7f3aec137870 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00ae40 con 0x7f3aec137870 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3adaffd640 10 monclient: dump: 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:28.705 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3ae8d99640 4 set_mon_vals no callback set 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: _renew_subs 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.683+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec120af0 con 0x7f3aec137870 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3af0a69640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3af0a69640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3af0a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aec0a0e10 con 0x7f3aec137870 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3adc019020 con 0x7f3aec137870 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3adaffd640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3acc03b970 0x7f3acc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6289+0+0 (secure 0 0 0) 0x7f3adc050180 con 0x7f3aec137870 2024-09-15T21:20:28.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3ae9d9b640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3acc03b970 0x7f3acc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:28.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3ae9d9b640 10 monclient: get_auth_request con 0x7f3acc03b970 auth_method 0 2024-09-15T21:20:28.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.687+0000 7f3ae9d9b640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3acc03b970 0x7f3acc03de30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3aec137800 tx=0x7f3ae0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:28.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.691+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3adc020030 con 0x7f3aec137870 2024-09-15T21:20:28.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.691+0000 7f3adaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:28.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.691+0000 7f3adaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:28.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3af0a69640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3af0a69640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3af0a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7f3aec06a810 con 0x7f3aec137870 2024-09-15T21:20:28.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v75) ==== 141+0+0 (secure 0 0 0) 0x7f3adc010cf0 con 0x7f3aec137870 2024-09-15T21:20:28.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3adaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.851+0000 7f3adaffd640 10 monclient: _finish_command 2 = system:0 item -11 does not appear in that position 2024-09-15T21:20:28.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3af0a69640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3af0a69640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3af0a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:28.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7f3aec136f40 con 0x7f3aec137870 2024-09-15T21:20:28.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3adaffd640 1 -- 172.21.15.119:0/781903044 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v75) ==== 141+0+0 (secure 0 0 0) 0x7f3adc022300 con 0x7f3aec137870 2024-09-15T21:20:28.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3adaffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T21:20:28.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.951+0000 7f3adaffd640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-09-15T21:20:28.957 INFO:tasks.workunit.client.0.smithi119.stderr:item -11 does not appear in that position 2024-09-15T21:20:28.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3acc03b970 msgr2=0x7f3acc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:28.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3acc03b970 0x7f3acc03de30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3aec137800 tx=0x7f3ae0008040 comp rx=0 tx=0).stop 2024-09-15T21:20:28.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 10 monclient: shutdown 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 msgr2=0x7f3aec11fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 0x7f3aec11fb90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c820 tx=0x7f3adc00cce0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 shutdown_connections 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3acc03b970 0x7f3acc03de30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aec13cdc0 0x7f3aec136a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3aec138360 0x7f3aec11daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 --2- 172.21.15.119:0/781903044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3aec137870 0x7f3aec11fb90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 >> 172.21.15.119:0/781903044 conn(0x7f3aec09be60 msgr2=0x7f3aec09de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 shutdown_connections 2024-09-15T21:20:28.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:28.955+0000 7f3af0a69640 1 -- 172.21.15.119:0/781903044 wait complete. 2024-09-15T21:20:28.974 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:28.974 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 1 2024-09-15T21:20:28.974 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c host1 2024-09-15T21:20:29.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient: build_initial_monmap 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient: monmap: 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:29.079341+0000 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:29.079341+0000 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.081 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:29.082 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:29.082 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:29.082 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:29.082 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:29.082 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 Processor -- start 2024-09-15T21:20:29.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 -- start start 2024-09-15T21:20:29.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient: init 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41c409d4e0 0x7f41c409f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): picked mon.noname-c con 0x7f41c409d4e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41c409d8c0 0x7f41c409dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): picked mon.noname-a con 0x7f41c409d8c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c41347b0 0x7f41c4134c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): picked mon.noname-b con 0x7f41c41347b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f41c4074760 con 0x7f41c409d8c0 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f41c40a09e0 con 0x7f41c409d4e0 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f41c40a0f30 con 0x7f41c41347b0 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.075+0000 7f41cc74f640 10 monclient(hunting): authenticate will time out at 1147.875872s 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41c409d4e0 0x7f41c409f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cacec640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c41347b0 msgr2=0x7f41c4134c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cacec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c41347b0 0x7f41c4134c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41c409d4e0 0x7f41c409f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49164/0 (socket says 172.21.15.119:49164) 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 -- 172.21.15.119:0/3729013690 learned_addr learned my addr 172.21.15.119:0/3729013690 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): get_auth_request con 0x7f41c409d4e0 auth_method 0 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c9cea640 1 --2- 172.21.15.119:0/3729013690 >> v2:172.21.15.17:3300/0 conn(0x7f41c409d8c0 0x7f41c409dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c9cea640 10 monclient(hunting): get_auth_request con 0x7f41c409d8c0 auth_method 0 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c9cea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c9cea640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c9cea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_done global_id 34372 payload 274 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 -- 172.21.15.119:0/3729013690 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c41347b0 msgr2=0x7f41c4134c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:29.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 --2- 172.21.15.119:0/3729013690 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c41347b0 0x7f41c4134c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 -- 172.21.15.119:0/3729013690 >> v2:172.21.15.17:3300/0 conn(0x7f41c409d8c0 msgr2=0x7f41c409dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 --2- 172.21.15.119:0/3729013690 >> v2:172.21.15.17:3300/0 conn(0x7f41c409d8c0 0x7f41c409dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 monclient: found mon.noname-c 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 -- 172.21.15.119:0/3729013690 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c409e6e0 con 0x7f41c409d4e0 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41ca4eb640 1 --2- 172.21.15.119:0/3729013690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c409f410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41c40a3b60 tx=0x7f41b801c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c21376e20f795ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 1 -- 172.21.15.119:0/3729013690 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80b0070 con 0x7f41c409d4e0 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:29.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: dump: 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 1 -- 172.21.15.119:0/3729013690 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b80ab600 con 0x7f41c409d4e0 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 5 monclient: authenticate success, global_id 34372 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 1 -- 172.21.15.119:0/3729013690 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80ab9d0 con 0x7f41c409d4e0 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 4 set_mon_vals no callback set 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41c94e9640 10 monclient: dump: 2024-09-15T21:20:29.096 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 10 monclient: shutdown 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 -- 172.21.15.119:0/3729013690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 msgr2=0x7f41c409f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 --2- 172.21.15.119:0/3729013690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c409f410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41c40a3b60 tx=0x7f41b801c400 comp rx=0 tx=0).stop 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 -- 172.21.15.119:0/3729013690 shutdown_connections 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 --2- 172.21.15.119:0/3729013690 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c41347b0 0x7f41c4134c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 --2- 172.21.15.119:0/3729013690 >> v2:172.21.15.17:3300/0 conn(0x7f41c409d8c0 0x7f41c409dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 --2- 172.21.15.119:0/3729013690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c409f410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 -- 172.21.15.119:0/3729013690 >> 172.21.15.119:0/3729013690 conn(0x7f41c406cdc0 msgr2=0x7f41c406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 -- 172.21.15.119:0/3729013690 shutdown_connections 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.079+0000 7f41cc74f640 1 -- 172.21.15.119:0/3729013690 wait complete. 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: build_initial_monmap 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: monmap: 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:29.098 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 Processor -- start 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- start start 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: init 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c4139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): picked mon.noname-b con 0x7f41c409d4e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c409d8c0 0x7f41c4133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): picked mon.noname-c con 0x7f41c409d8c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41c41347b0 0x7f41c41cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): picked mon.noname-a con 0x7f41c41347b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f41c40a25e0 con 0x7f41c41347b0 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41356c0 con 0x7f41c409d4e0 2024-09-15T21:20:29.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f41c40a2760 con 0x7f41c409d8c0 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient(hunting): authenticate will time out at 1147.880828s 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c4139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41c9cea640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c409d8c0 msgr2=0x7f41c4133b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c4139d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49172/0 (socket says 172.21.15.119:49172) 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 -- 172.21.15.119:0/1459684304 learned_addr learned my addr 172.21.15.119:0/1459684304 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): get_auth_request con 0x7f41c409d4e0 auth_method 0 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41c9cea640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c409d8c0 0x7f41c4133b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cacec640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41c41347b0 0x7f41c41cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient(hunting): handle_auth_done global_id 34375 payload 824 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 -- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c409d8c0 msgr2=0x7f41c4133b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c409d8c0 0x7f41c4133b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 -- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41c41347b0 msgr2=0x7f41c41cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41c41347b0 0x7f41c41cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 monclient: found mon.noname-b 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 -- 172.21.15.119:0/1459684304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c409d1d0 con 0x7f41c409d4e0 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cacec640 10 monclient: get_auth_request con 0x7f41c41347b0 auth_method 0 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cacec640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41c41347b0 0x7f41c41cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41ca4eb640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c4139d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41b801aab0 tx=0x7f41b80191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80b0040 con 0x7f41c409d4e0 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: dump: 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.103 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b80b4420 con 0x7f41c409d4e0 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:29.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80b47d0 con 0x7f41c409d4e0 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 5 monclient: authenticate success, global_id 34375 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _renew_subs 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c413a7e0 con 0x7f41c409d4e0 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41c8ce8640 4 set_mon_vals no callback set 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 10 monclient: dump: 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.105 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _renew_subs 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c411caf0 con 0x7f41c409d4e0 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c409dcd0 con 0x7f41c409d4e0 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f41b80b4970 con 0x7f41c409d4e0 2024-09-15T21:20:29.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41a803b970 0x7f41a803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6289+0+0 (secure 0 0 0) 0x7f41b80e9e00 con 0x7f41c409d4e0 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41c9cea640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41a803b970 0x7f41a803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41c9cea640 10 monclient: get_auth_request con 0x7f41a803b970 auth_method 0 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.083+0000 7f41c9cea640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41a803b970 0x7f41a803de30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41c0006e90 tx=0x7f41c0006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.087+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f41b8016110 con 0x7f41c409d4e0 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.087+0000 7f41b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.087+0000 7f41b6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:29.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41cc74f640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:29.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41cc74f640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:29.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41cc74f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f41c411f140 con 0x7f41c409d4e0 2024-09-15T21:20:29.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v75) ==== 56+0+1340 (secure 0 0 0) 0x7f41b80eb020 con 0x7f41c409d4e0 2024-09-15T21:20:29.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41b6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:29.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.259+0000 7f41b6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:29.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:29.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:29.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f41c40008e0 con 0x7f41c409d4e0 2024-09-15T21:20:29.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41b6ffd640 1 -- 172.21.15.119:0/1459684304 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v75) ==== 56+0+1340 (secure 0 0 0) 0x7f41b80bd920 con 0x7f41c409d4e0 2024-09-15T21:20:29.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41b6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:29.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41b6ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:29.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41a803b970 msgr2=0x7f41a803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41a803b970 0x7f41a803de30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41c0006e90 tx=0x7f41c0006e20 comp rx=0 tx=0).stop 2024-09-15T21:20:29.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 10 monclient: shutdown 2024-09-15T21:20:29.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:29.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 msgr2=0x7f41c4139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.355+0000 7f41cc74f640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c4139d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41b801aab0 tx=0x7f41b80191a0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 shutdown_connections 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41a803b970 0x7f41a803de30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41c41347b0 0x7f41c41cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41c409d8c0 0x7f41c4133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 --2- 172.21.15.119:0/1459684304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c409d4e0 0x7f41c4139d30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 >> 172.21.15.119:0/1459684304 conn(0x7f41c406cdc0 msgr2=0x7f41c40a4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 shutdown_connections 2024-09-15T21:20:29.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.359+0000 7f41cc74f640 1 -- 172.21.15.119:0/1459684304 wait complete. 2024-09-15T21:20:29.376 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush rm foo 2024-09-15T21:20:29.376 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:29.377 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm foo 2024-09-15T21:20:29.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient: get_monmap_and_config 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient: build_initial_monmap 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient: monmap: 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:29.490542+0000 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:29.490542+0000 2024-09-15T21:20:29.492 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:29.493 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 1 Processor -- start 2024-09-15T21:20:29.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 1 -- start start 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient: init 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b9c135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 0x7f5b9c13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient(hunting): picked mon.noname-b con 0x7f5b9c0a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b9c06ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9c09cf30 con 0x7f5b9c135ca0 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9c0a1ca0 con 0x7f5b9c06ec30 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c0a1f80 con 0x7f5b9c0a0160 2024-09-15T21:20:29.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5ba1ac1640 10 monclient(hunting): authenticate will time out at 1148.287781s 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5b92ffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 msgr2=0x7f5b9c13aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5b9bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5b9c06ec30 0x7f5b9c13cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39226/0 (socket says 172.21.15.119:39226) 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5b9bfff640 1 -- 172.21.15.119:0/2628877445 learned_addr learned my addr 172.21.15.119:0/2628877445 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.487+0000 7f5b92ffd640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 0x7f5b9c13aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392482709 0 0) 0x7f5b9c0a1ca0 con 0x7f5b9c06ec30 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _negotiate 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): my global_id is 34378 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b88003530 con 0x7f5b9c06ec30 2024-09-15T21:20:29.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437142276 0 0) 0x7f5b9c09cf30 con 0x7f5b9c135ca0 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _negotiate 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): my global_id is 5202 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b9c0a1ca0 con 0x7f5b9c135ca0 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1229195499 0 0) 0x7f5b88003530 con 0x7f5b9c06ec30 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 msgr2=0x7f5b9c13aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 0x7f5b9c13aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 >> v1:172.21.15.17:6789/0 conn(0x7f5b9c135ca0 legacy=0x7f5b9c136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c0a3860 con 0x7f5b9c06ec30 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5b94001a80 con 0x7f5b9c06ec30 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: dump: 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:29.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 >> v1:172.21.15.119:6789/0 conn(0x7f5b9c06ec30 legacy=0x7f5b9c13cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b88003bf0 0x7f5b88003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): picked mon.a con 0x7f5b88003bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 0x7f5b88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): picked mon.c con 0x7f5b9c0a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b88004f30 0x7f5b8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): picked mon.b con 0x7f5b88004f30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c0a1f80 con 0x7f5b88003bf0 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8800bf10 con 0x7f5b88004f30 2024-09-15T21:20:29.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8800c090 con 0x7f5b9c0a0160 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9affd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b92ffd640 1 -- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 msgr2=0x7f5b88004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b92ffd640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 0x7f5b88004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b88004f30 0x7f5b8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): get_auth_request con 0x7f5b88004f30 auth_method 0 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9b7fe640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b88003bf0 0x7f5b88003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9b7fe640 10 monclient(hunting): get_auth_request con 0x7f5b88003bf0 auth_method 0 2024-09-15T21:20:29.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient(hunting): handle_auth_done global_id 34378 payload 293 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 -- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 msgr2=0x7f5b88004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c0a0160 0x7f5b88004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 -- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b88003bf0 msgr2=0x7f5b88003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b88003bf0 0x7f5b88003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 monclient: found mon.b 2024-09-15T21:20:29.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 1 -- 172.21.15.119:0/2628877445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b88003530 con 0x7f5b88004f30 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5ba1ac1640 5 monclient: authenticate success, global_id 34378 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5ba1ac1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:29.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.491+0000 7f5b9b7fe640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b88003bf0 0x7f5b88003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:29.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9bfff640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b88004f30 0x7f5b8800b9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400a7e0 tx=0x7f5b9401d220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849d0266ad9f78b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400a3b0 con 0x7f5b88004f30 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: dump: 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.511 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b940ad650 con 0x7f5b88004f30 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 4 set_mon_vals no callback set 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 1 -- 172.21.15.119:0/2628877445 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ada40 con 0x7f5b88004f30 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9affd640 10 monclient: dump: 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.512 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: shutdown 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/2628877445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b88004f30 msgr2=0x7f5b8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 --2- 172.21.15.119:0/2628877445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b88004f30 0x7f5b8800b9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400a7e0 tx=0x7f5b9401d220 comp rx=0 tx=0).stop 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9bfff640 1 -- 172.21.15.119:0/2628877445 reap_dead start 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/2628877445 shutdown_connections 2024-09-15T21:20:29.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/2628877445 >> 172.21.15.119:0/2628877445 conn(0x7f5b9c0a2ae0 msgr2=0x7f5b9c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/2628877445 shutdown_connections 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/2628877445 wait complete. 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: build_initial_monmap 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: monmap: 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:29.514 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 Processor -- start 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- start start 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: init 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c09d4e0 0x7f5b9c09d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): picked mon.noname-a con 0x7f5b9c09d4e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c09de00 0x7f5b9c1d4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): picked mon.noname-b con 0x7f5b9c09de00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c1d4890 0x7f5b9c1d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): picked mon.noname-c con 0x7f5b9c1d4890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c13f000 con 0x7f5b9c09d4e0 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c13ee80 con 0x7f5b9c09de00 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c13f180 con 0x7f5b9c1d4890 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient(hunting): authenticate will time out at 1148.294374s 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c09de00 0x7f5b9c1d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c09de00 0x7f5b9c1d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49192/0 (socket says 172.21.15.119:49192) 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 -- 172.21.15.119:0/1135637023 learned_addr learned my addr 172.21.15.119:0/1135637023 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): get_auth_request con 0x7f5b9c09de00 auth_method 0 2024-09-15T21:20:29.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9bfff640 1 -- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c1d4890 msgr2=0x7f5b9c1d6c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9bfff640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c1d4890 0x7f5b9c1d6c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9b7fe640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c09d4e0 0x7f5b9c09d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9b7fe640 10 monclient(hunting): get_auth_request con 0x7f5b9c09d4e0 auth_method 0 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient(hunting): handle_auth_done global_id 34381 payload 824 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 -- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c1d4890 msgr2=0x7f5b9c1d6c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b9c1d4890 0x7f5b9c1d6c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 -- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c09d4e0 msgr2=0x7f5b9c09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c09d4e0 0x7f5b9c09d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 monclient: found mon.noname-b 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 -- 172.21.15.119:0/1135637023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c121a60 con 0x7f5b9c09de00 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b92ffd640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c09de00 0x7f5b9c1d4350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400b560 tx=0x7f5b8400ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84013020 con 0x7f5b9c09de00 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: dump: 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.519 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 5 monclient: authenticate success, global_id 34381 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b84003df0 con 0x7f5b9c09de00 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: _renew_subs 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/1135637023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c121d40 con 0x7f5b9c09de00 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b840052b0 con 0x7f5b9c09de00 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b98ff9640 10 monclient: dump: 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5b9a7fc640 4 set_mon_vals no callback set 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: _renew_subs 2024-09-15T21:20:29.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.495+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/1135637023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c13e300 con 0x7f5b9c09de00 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5ba1ac1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5ba1ac1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5ba1ac1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/1135637023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c09d1d0 con 0x7f5b9c09de00 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5b84016020 con 0x7f5b9c09de00 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5b98ff9640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b7c03b970 0x7f5b7c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6289+0+0 (secure 0 0 0) 0x7f5b8404fe20 con 0x7f5b9c09de00 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5b9b7fe640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b7c03b970 0x7f5b7c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5b9b7fe640 10 monclient: get_auth_request con 0x7f5b7c03b970 auth_method 0 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.499+0000 7f5b9b7fe640 1 --2- 172.21.15.119:0/1135637023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b7c03b970 0x7f5b7c03de30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c001110 tx=0x7f5b8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.503+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5b8401d020 con 0x7f5b9c09de00 2024-09-15T21:20:29.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.503+0000 7f5b98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.503+0000 7f5b98ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:29.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.675+0000 7f5ba1ac1640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:29.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.675+0000 7f5ba1ac1640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:29.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.675+0000 7f5ba1ac1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.675+0000 7f5ba1ac1640 1 -- 172.21.15.119:0/1135637023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7f5b9c136080 con 0x7f5b9c09de00 2024-09-15T21:20:29.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.679+0000 7f5b98ff9640 1 -- 172.21.15.119:0/1135637023 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-39 (39) Directory not empty v75) ==== 99+0+0 (secure 0 0 0) 0x7f5b8400fdc0 con 0x7f5b9c09de00 2024-09-15T21:20:29.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.679+0000 7f5b98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:29.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.679+0000 7f5b98ff9640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T21:20:29.684 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T21:20:29.689 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:29.690 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush rm bar 2024-09-15T21:20:29.690 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:29.690 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm bar 2024-09-15T21:20:29.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient: get_monmap_and_config 2024-09-15T21:20:29.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient: build_initial_monmap 2024-09-15T21:20:29.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient: monmap: 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:29.806251+0000 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:29.806251+0000 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:29.808 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:29.809 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:29.809 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 Processor -- start 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 -- start start 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient: init 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8bcc09f640 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc0a0130 0x7f8bcc09d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): picked mon.noname-b con 0x7f8bcc0a0130 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bcc139c00 0x7f8bcc13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): picked mon.noname-c con 0x7f8bcc139c00 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc09e880 con 0x7f8bcc09f640 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc09eb60 con 0x7f8bcc139c00 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc09ee40 con 0x7f8bcc0a0130 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd35bf640 10 monclient(hunting): authenticate will time out at 1148.603030s 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd135b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8bcc09f640 0x7f8bcc09fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39230/0 (socket says 172.21.15.119:39230) 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd135b640 1 -- 172.21.15.119:0/1345035181 learned_addr learned my addr 172.21.15.119:0/1345035181 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 1 --2- 172.21.15.119:0/1345035181 >> v2:172.21.15.119:3300/0 conn(0x7f8bcc139c00 0x7f8bcc13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd0b5a640 1 -- 172.21.15.119:0/1345035181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc0a0130 msgr2=0x7f8bcc09d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): get_auth_request con 0x7f8bcc139c00 auth_method 0 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd0b5a640 1 --2- 172.21.15.119:0/1345035181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc0a0130 0x7f8bcc09d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:29.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient(hunting): handle_auth_done global_id 34384 payload 274 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 1 -- 172.21.15.119:0/1345035181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc0a0130 msgr2=0x7f8bcc09d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 1 --2- 172.21.15.119:0/1345035181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc0a0130 0x7f8bcc09d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 1 -- 172.21.15.119:0/1345035181 >> v1:172.21.15.119:6789/0 conn(0x7f8bcc09f640 legacy=0x7f8bcc09fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 1 monclient: found mon.noname-c 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.803+0000 7f8bd1b5c640 1 -- 172.21.15.119:0/1345035181 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0a2840 con 0x7f8bcc139c00 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd1b5c640 1 --2- 172.21.15.119:0/1345035181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc139c00 0x7f8bcc13bff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e66986b09a70ff00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 1 -- 172.21.15.119:0/1345035181 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40b2070 con 0x7f8bcc139c00 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: dump: 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 1 -- 172.21.15.119:0/1345035181 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc40a57a0 con 0x7f8bcc139c00 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 1 -- 172.21.15.119:0/1345035181 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40a5b70 con 0x7f8bcc139c00 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 5 monclient: authenticate success, global_id 34384 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 4 set_mon_vals no callback set 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bc3fff640 10 monclient: dump: 2024-09-15T21:20:29.818 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 10 monclient: shutdown 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 -- 172.21.15.119:0/1345035181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc139c00 msgr2=0x7f8bcc13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 --2- 172.21.15.119:0/1345035181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc139c00 0x7f8bcc13bff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc401c8e0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 -- 172.21.15.119:0/1345035181 shutdown_connections 2024-09-15T21:20:29.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 --2- 172.21.15.119:0/1345035181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc139c00 0x7f8bcc13bff0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 --2- 172.21.15.119:0/1345035181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc0a0130 0x7f8bcc09d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 -- 172.21.15.119:0/1345035181 >> 172.21.15.119:0/1345035181 conn(0x7f8bcc0a2ae0 msgr2=0x7f8bcc0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:29.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 -- 172.21.15.119:0/1345035181 shutdown_connections 2024-09-15T21:20:29.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 1 -- 172.21.15.119:0/1345035181 wait complete. 2024-09-15T21:20:29.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 10 monclient: build_initial_monmap 2024-09-15T21:20:29.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.807+0000 7f8bd35bf640 10 monclient: monmap: 2024-09-15T21:20:29.824 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 Processor -- start 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- start start 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: init 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc09f640 0x7f8bcc1cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): picked mon.noname-b con 0x7f8bcc09f640 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc0a0130 0x7f8bcc0733a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): picked mon.noname-a con 0x7f8bcc0a0130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc139c00 0x7f8bcc0738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): picked mon.noname-c con 0x7f8bcc139c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0a13f0 con 0x7f8bcc0a0130 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0a1270 con 0x7f8bcc09f640 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0a1570 con 0x7f8bcc139c00 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient(hunting): authenticate will time out at 1148.608635s 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc09f640 0x7f8bcc1cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc09f640 0x7f8bcc1cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49210/0 (socket says 172.21.15.119:49210) 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 -- 172.21.15.119:0/3323348018 learned_addr learned my addr 172.21.15.119:0/3323348018 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): get_auth_request con 0x7f8bcc09f640 auth_method 0 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd1b5c640 1 -- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc139c00 msgr2=0x7f8bcc0738e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd1b5c640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc139c00 0x7f8bcc0738e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:29.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient(hunting): handle_auth_done global_id 34390 payload 824 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd0b5a640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc0a0130 0x7f8bcc0733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 -- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc139c00 msgr2=0x7f8bcc0738e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8bcc139c00 0x7f8bcc0738e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 -- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc0a0130 msgr2=0x7f8bcc0733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc0a0130 0x7f8bcc0733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient: _finish_hunting 0 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 monclient: found mon.noname-b 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 -- 172.21.15.119:0/3323348018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc09d1d0 con 0x7f8bcc09f640 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd0b5a640 10 monclient: get_auth_request con 0x7f8bcc0a0130 auth_method 0 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd0b5a640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc0a0130 0x7f8bcc0733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd135b640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc09f640 0x7f8bcc1cb010 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c4b0 tx=0x7f8bbc00c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc017070 con 0x7f8bcc09f640 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: dump: 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.829 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc005d10 con 0x7f8bcc09f640 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0072b0 con 0x7f8bcc09f640 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 5 monclient: authenticate success, global_id 34390 2024-09-15T21:20:29.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 10 monclient: dump: 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc37fe640 4 set_mon_vals no callback set 2024-09-15T21:20:29.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _renew_subs 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- 172.21.15.119:0/3323348018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1cbbd0 con 0x7f8bcc09f640 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _renew_subs 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- 172.21.15.119:0/3323348018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc134400 con 0x7f8bcc09f640 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd35bf640 1 -- 172.21.15.119:0/3323348018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc09d9a0 con 0x7f8bcc09f640 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8bbc01a070 con 0x7f8bcc09f640 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8bb003b970 0x7f8bb003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6289+0+0 (secure 0 0 0) 0x7f8bbc04f7f0 con 0x7f8bcc09f640 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd0b5a640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8bb003b970 0x7f8bb003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:29.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd0b5a640 10 monclient: get_auth_request con 0x7f8bb003b970 auth_method 0 2024-09-15T21:20:29.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.811+0000 7f8bd0b5a640 1 --2- 172.21.15.119:0/3323348018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8bb003b970 0x7f8bb003de30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06ecc0 tx=0x7f8bc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:29.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.815+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8bbc021020 con 0x7f8bcc09f640 2024-09-15T21:20:29.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.815+0000 7f8bc1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:29.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.815+0000 7f8bc1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:29.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bd35bf640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:29.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bd35bf640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:29.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bd35bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:29.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bd35bf640 1 -- 172.21.15.119:0/3323348018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7f8bcc0a4ff0 con 0x7f8bcc09f640 2024-09-15T21:20:30.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bc1ffb640 1 -- 172.21.15.119:0/3323348018 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v75) ==== 99+0+0 (secure 0 0 0) 0x7f8bbc010300 con 0x7f8bcc09f640 2024-09-15T21:20:30.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bc1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:30.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:29.995+0000 7f8bc1ffb640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T21:20:30.001 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T21:20:30.005 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:30.005 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush unlink host1 bar 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.119+0000 7fd81aec0640 10 monclient: get_monmap_and_config 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.119+0000 7fd81aec0640 10 monclient: build_initial_monmap 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.119+0000 7fd81aec0640 10 monclient: monmap: 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:30.123496+0000 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:30.123496+0000 2024-09-15T21:20:30.125 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:30.126 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.119+0000 7fd81aec0640 1 Processor -- start 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.119+0000 7fd81aec0640 1 -- start start 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.119+0000 7fd81aec0640 10 monclient: init 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8140a0900 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8140a13f0 0x7fd81413ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient(hunting): picked mon.noname-a con 0x7fd8140a13f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd81413b150 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8140a2240 con 0x7fd8140a0900 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8140a1c80 con 0x7fd81413b150 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd81409fdd0 con 0x7fd8140a13f0 2024-09-15T21:20:30.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 10 monclient(hunting): authenticate will time out at 1148.920410s 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd81413b150 0x7fd81413d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39238/0 (socket says 172.21.15.119:39238) 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 -- 172.21.15.119:0/2319002268 learned_addr learned my addr 172.21.15.119:0/2319002268 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088885470 0 0) 0x7fd8140a1c80 con 0x7fd81413b150 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): _negotiate 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): my global_id is 34393 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7fc003530 con 0x7fd81413b150 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 1 --2- 172.21.15.119:0/2319002268 >> v2:172.21.15.17:3300/0 conn(0x7fd8140a13f0 0x7fd81413ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): get_auth_request con 0x7fd8140a13f0 auth_method 0 2024-09-15T21:20:30.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3587980505 0 0) 0x7fd7fc003530 con 0x7fd81413b150 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 >> v2:172.21.15.17:3300/0 conn(0x7fd8140a13f0 msgr2=0x7fd81413ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 --2- 172.21.15.119:0/2319002268 >> v2:172.21.15.17:3300/0 conn(0x7fd8140a13f0 0x7fd81413ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 >> v1:172.21.15.17:6789/0 conn(0x7fd8140a0900 legacy=0x7fd8140a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:30.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8140a1f60 con 0x7fd81413b150 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 5 monclient: authenticate success, global_id 34393 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fd80c001a80 con 0x7fd81413b150 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: dump: 2024-09-15T21:20:30.141 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 >> v1:172.21.15.119:6789/0 conn(0x7fd81413b150 legacy=0x7fd81413d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:30.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7fc003a60 0x7fd7fc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): picked mon.c con 0x7fd7fc003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7fc004810 0x7fd7fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): picked mon.a con 0x7fd7fc004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd7fc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): picked mon.b con 0x7fd7fc0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd81409fdd0 con 0x7fd7fc004810 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc00c310 con 0x7fd7fc0053d0 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc00c510 con 0x7fd7fc003a60 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd8137fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81aec0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 1 --2- 172.21.15.119:0/2319002268 >> v2:172.21.15.17:3300/0 conn(0x7fd8140a13f0 0x7fd81413ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd818c5c640 1 -- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7fc003a60 msgr2=0x7fd7fc003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd7fc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd818c5c640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7fc003a60 0x7fd7fc003ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): get_auth_request con 0x7fd7fc0053d0 auth_method 0 2024-09-15T21:20:30.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7fc004810 0x7fd7fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): get_auth_request con 0x7fd7fc004810 auth_method 0 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd813fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient(hunting): handle_auth_done global_id 34393 payload 293 2024-09-15T21:20:30.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 -- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7fc003a60 msgr2=0x7fd7fc003ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7fc003a60 0x7fd7fc003ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 -- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7fc004810 msgr2=0x7fd7fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7fc004810 0x7fd7fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 monclient: found mon.b 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:30.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 -- 172.21.15.119:0/2319002268 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc003530 con 0x7fd7fc0053d0 2024-09-15T21:20:30.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient: _finish_auth 0 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 10 monclient: _check_auth_tickets 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.123+0000 7fd81945d640 1 -- 172.21.15.119:0/2319002268 reap_dead start 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81945d640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd7fc00ba30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00a880 tx=0x7fd80c01d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ced7f4264c6300c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00a4d0 con 0x7fd7fc0053d0 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: dump: 2024-09-15T21:20:30.157 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.158 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80c0ac620 con 0x7fd7fc0053d0 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 4 set_mon_vals no callback set 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 1 -- 172.21.15.119:0/2319002268 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0aca10 con 0x7fd7fc0053d0 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8137fe640 10 monclient: dump: 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.159 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: shutdown 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2319002268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 msgr2=0x7fd7fc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd7fc00ba30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00a880 tx=0x7fd80c01d260 comp rx=0 tx=0).stop 2024-09-15T21:20:30.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2319002268 shutdown_connections 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2319002268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd7fc00ba30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2319002268 >> 172.21.15.119:0/2319002268 conn(0x7fd81409be60 msgr2=0x7fd81409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2319002268 shutdown_connections 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2319002268 wait complete. 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: build_initial_monmap 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: monmap: 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 Processor -- start 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- start start 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: init 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd814131b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): picked mon.noname-b con 0x7fd7fc0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:30.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8141320c0 0x7fd81406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): picked mon.noname-c con 0x7fd8141320c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81406f4c0 0x7fd81406f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): picked mon.noname-a con 0x7fd81406f4c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd8140a01d0 con 0x7fd81406f4c0 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8140a0050 con 0x7fd7fc0053d0 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd8140a0ce0 con 0x7fd8141320c0 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient(hunting): authenticate will time out at 1148.926386s 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd813fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8141320c0 msgr2=0x7fd81406ef80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:30.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd814131b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd813fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8141320c0 0x7fd81406ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd814131b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49222/0 (socket says 172.21.15.119:49222) 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 -- 172.21.15.119:0/2946223331 learned_addr learned my addr 172.21.15.119:0/2946223331 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): get_auth_request con 0x7fd7fc0053d0 auth_method 0 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81945d640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81406f4c0 0x7fd81406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81945d640 10 monclient(hunting): get_auth_request con 0x7fd81406f4c0 auth_method 0 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81945d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81945d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81945d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient(hunting): handle_auth_done global_id 34396 payload 824 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 -- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8141320c0 msgr2=0x7fd81406ef80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8141320c0 0x7fd81406ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 -- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81406f4c0 msgr2=0x7fd81406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81406f4c0 0x7fd81406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 monclient: found mon.noname-b 2024-09-15T21:20:30.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 -- 172.21.15.119:0/2946223331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8140714c0 con 0x7fd7fc0053d0 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd818c5c640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd814131b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c480 tx=0x7fd80400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804017070 con 0x7fd7fc0053d0 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: dump: 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.166 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 5 monclient: authenticate success, global_id 34396 2024-09-15T21:20:30.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd804007480 con 0x7fd7fc0053d0 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804005a10 con 0x7fd7fc0053d0 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: _renew_subs 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8140717a0 con 0x7fd7fc0053d0 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd812ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:30.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd8117fa640 10 monclient: dump: 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.169 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: _renew_subs 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.127+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81409d0f0 con 0x7fd7fc0053d0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd81aec0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd81aec0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd81aec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81413ba50 con 0x7fd7fc0053d0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd80401a070 con 0x7fd7fc0053d0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd8117fa640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd80003b9a0 0x7fd80003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6289+0+0 (secure 0 0 0) 0x7fd80404f3d0 con 0x7fd7fc0053d0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd813fff640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd80003b9a0 0x7fd80003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd813fff640 10 monclient: get_auth_request con 0x7fd80003b9a0 auth_method 0 2024-09-15T21:20:30.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.131+0000 7fd813fff640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd80003b9a0 0x7fd80003de60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd814070590 tx=0x7fd808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:30.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.135+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd804010300 con 0x7fd7fc0053d0 2024-09-15T21:20:30.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.135+0000 7fd8117fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:30.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.135+0000 7fd8117fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:30.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.311+0000 7fd81aec0640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T21:20:30.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.311+0000 7fd81aec0640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T21:20:30.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.311+0000 7fd81aec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.311+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) -- 0x7fd814136140 con 0x7fd7fc0053d0 2024-09-15T21:20:30.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.463+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 removed item id -11 name 'host1' from crush map v76) ==== 147+0+0 (secure 0 0 0) 0x7fd804011d90 con 0x7fd7fc0053d0 2024-09-15T21:20:30.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.463+0000 7fd8117fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T21:20:30.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.463+0000 7fd8117fa640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-09-15T21:20:30.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.567+0000 7fd81aec0640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T21:20:30.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.571+0000 7fd81aec0640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T21:20:30.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.571+0000 7fd81aec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.571+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) -- 0x7fd81413a310 con 0x7fd7fc0053d0 2024-09-15T21:20:30.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.571+0000 7fd8117fa640 1 -- 172.21.15.119:0/2946223331 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 item -11 does not appear in that position v76) ==== 141+0+0 (secure 0 0 0) 0x7fd804019d30 con 0x7fd7fc0053d0 2024-09-15T21:20:30.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.571+0000 7fd8117fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T21:20:30.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.571+0000 7fd8117fa640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:item -11 does not appear in that position 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd80003b9a0 msgr2=0x7fd80003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd80003b9a0 0x7fd80003de60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd814070590 tx=0x7fd808008040 comp rx=0 tx=0).stop 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 10 monclient: shutdown 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 msgr2=0x7fd814131b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd814131b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c480 tx=0x7fd80400c940 comp rx=0 tx=0).stop 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 shutdown_connections 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd80003b9a0 0x7fd80003de60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81406f4c0 0x7fd81406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8141320c0 0x7fd81406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 --2- 172.21.15.119:0/2946223331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7fc0053d0 0x7fd814131b80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 >> 172.21.15.119:0/2946223331 conn(0x7fd81409be60 msgr2=0x7fd81413dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:30.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 shutdown_connections 2024-09-15T21:20:30.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.575+0000 7fd81aec0640 1 -- 172.21.15.119:0/2946223331 wait complete. 2024-09-15T21:20:30.593 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:30.594 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 1 2024-09-15T21:20:30.594 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c host1 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient: get_monmap_and_config 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient: build_initial_monmap 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient: monmap: 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:30.706698+0000 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:30.706698+0000 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.709 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:30.709 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:30.709 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:30.709 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:30.709 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:30.709 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 1 Processor -- start 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 1 -- start start 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient: init 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e74135d10 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 0x7f5e7413aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient(hunting): picked mon.noname-b con 0x7f5e7406ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e7413b020 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e740a1b90 con 0x7f5e7413b020 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e740a1e70 con 0x7f5e74135d10 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7409f6f0 con 0x7f5e7406ec30 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:30.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e79b13640 10 monclient(hunting): authenticate will time out at 1149.503497s 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e737fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5e74135d10 0x7f5e741360f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39242/0 (socket says 172.21.15.119:39242) 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e737fe640 1 -- 172.21.15.119:0/183395428 learned_addr learned my addr 172.21.15.119:0/183395428 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e72ffd640 1 -- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 msgr2=0x7f5e7413aae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e72ffd640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 0x7f5e7413aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889589689 0 0) 0x7f5e740a1e70 con 0x7f5e74135d10 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e727fc640 10 monclient(hunting): _negotiate 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e727fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e727fc640 10 monclient(hunting): my global_id is 34399 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.703+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e5c003530 con 0x7f5e74135d10 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1474499372 0 0) 0x7f5e5c003530 con 0x7f5e74135d10 2024-09-15T21:20:30.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 msgr2=0x7f5e7413aae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 0x7f5e7413aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 >> v1:172.21.15.17:6789/0 conn(0x7f5e7413b020 legacy=0x7f5e7413d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e740a2150 con 0x7f5e74135d10 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 5 monclient: authenticate success, global_id 34399 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5e64001a80 con 0x7f5e74135d10 2024-09-15T21:20:30.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: dump: 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:30.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 >> v1:172.21.15.119:6789/0 conn(0x7f5e74135d10 legacy=0x7f5e741360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c003a10 0x7f5e5c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): picked mon.b con 0x7f5e5c003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e5c004810 0x7f5e5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): picked mon.a con 0x7f5e5c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 0x7f5e5c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): picked mon.c con 0x7f5e7406ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7409f6f0 con 0x7f5e5c004810 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c00bf10 con 0x7f5e5c003a10 2024-09-15T21:20:30.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e5c00c090 con 0x7f5e7406ec30 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c003a10 0x7f5e5c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): get_auth_request con 0x7f5e5c003a10 auth_method 0 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e73fff640 1 -- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 msgr2=0x7f5e5c00b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e73fff640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 0x7f5e5c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:30.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e72ffd640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e5c004810 0x7f5e5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient(hunting): handle_auth_done global_id 34399 payload 293 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 -- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 msgr2=0x7f5e5c00b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e7406ec30 0x7f5e5c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 -- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e5c004810 msgr2=0x7f5e5c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e5c004810 0x7f5e5c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 monclient: found mon.b 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.720 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 -- 172.21.15.119:0/183395428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c003530 con 0x7f5e5c003a10 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e72ffd640 10 monclient: get_auth_request con 0x7f5e5c004810 auth_method 0 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e72ffd640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e5c004810 0x7f5e5c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e737fe640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c003a10 0x7f5e5c003e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400a820 tx=0x7f5e6401d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68412eff7e1a57e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400a450 con 0x7f5e5c003a10 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: dump: 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.721 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e640072f0 con 0x7f5e5c003a10 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 1 -- 172.21.15.119:0/183395428 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e640076e0 con 0x7f5e5c003a10 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 4 set_mon_vals no callback set 2024-09-15T21:20:30.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e727fc640 10 monclient: dump: 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 10 monclient: shutdown 2024-09-15T21:20:30.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 1 -- 172.21.15.119:0/183395428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c003a10 msgr2=0x7f5e5c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 1 --2- 172.21.15.119:0/183395428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c003a10 0x7f5e5c003e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400a820 tx=0x7f5e6401d260 comp rx=0 tx=0).stop 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e73fff640 1 -- 172.21.15.119:0/183395428 reap_dead start 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 1 -- 172.21.15.119:0/183395428 shutdown_connections 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 1 -- 172.21.15.119:0/183395428 >> 172.21.15.119:0/183395428 conn(0x7f5e7406cdc0 msgr2=0x7f5e7406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 1 -- 172.21.15.119:0/183395428 shutdown_connections 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 1 -- 172.21.15.119:0/183395428 wait complete. 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 10 monclient: build_initial_monmap 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.707+0000 7f5e79b13640 10 monclient: monmap: 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:30.724 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:30.725 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:30.725 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.725 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:30.725 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:30.725 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:30.725 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 Processor -- start 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- start start 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: init 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e5c003a10 0x7f5e74072890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): picked mon.noname-c con 0x7f5e5c003a10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 0x7f5e74134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): picked mon.noname-b con 0x7f5e7409f050 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:30.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e74134750 0x7f5e741cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): picked mon.noname-a con 0x7f5e74134750 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7413f4c0 con 0x7f5e74134750 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7413f340 con 0x7f5e7409f050 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7413f640 con 0x7f5e5c003a10 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient(hunting): authenticate will time out at 1149.508867s 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 0x7f5e74134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 0x7f5e74134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49244/0 (socket says 172.21.15.119:49244) 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e737fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e5c003a10 msgr2=0x7f5e74072890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 -- 172.21.15.119:0/1963168172 learned_addr learned my addr 172.21.15.119:0/1963168172 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:30.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): get_auth_request con 0x7f5e7409f050 auth_method 0 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e737fe640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e5c003a10 0x7f5e74072890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient(hunting): handle_auth_done global_id 34402 payload 824 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e73fff640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e74134750 0x7f5e741cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 -- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e5c003a10 msgr2=0x7f5e74072890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e5c003a10 0x7f5e74072890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 -- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e74134750 msgr2=0x7f5e741cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e74134750 0x7f5e741cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 monclient: found mon.noname-b 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 -- 172.21.15.119:0/1963168172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7409d1d0 con 0x7f5e7409f050 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e72ffd640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 0x7f5e74134210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c820 tx=0x7f5e6800cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68003d00 con 0x7f5e7409f050 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: dump: 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.732 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: _finish_auth 0 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e68016070 con 0x7f5e7409f050 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68005270 con 0x7f5e7409f050 2024-09-15T21:20:30.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 5 monclient: authenticate success, global_id 34402 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _renew_subs 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e74121330 con 0x7f5e7409f050 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 10 monclient: dump: 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:30.734 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e71ffb640 4 set_mon_vals no callback set 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _renew_subs 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e741d0120 con 0x7f5e7409f050 2024-09-15T21:20:30.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e68005410 con 0x7f5e7409f050 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e741360f0 con 0x7f5e7409f050 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6003b9a0 0x7f5e6003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6249+0+0 (secure 0 0 0) 0x7f5e680500a0 con 0x7f5e7409f050 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e737fe640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6003b9a0 0x7f5e6003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e737fe640 10 monclient: get_auth_request con 0x7f5e6003b9a0 auth_method 0 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.711+0000 7f5e737fe640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6003b9a0 0x7f5e6003de60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e640ad000 tx=0x7f5e64001d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.715+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5e6800ce10 con 0x7f5e7409f050 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.715+0000 7f5e5bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:30.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.715+0000 7f5e5bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:30.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.875+0000 7f5e79b13640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:30.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.875+0000 7f5e79b13640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:30.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.875+0000 7f5e79b13640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.875+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f5e74121500 con 0x7f5e7409f050 2024-09-15T21:20:30.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.879+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v76) ==== 56+0+1340 (secure 0 0 0) 0x7f5e68005700 con 0x7f5e7409f050 2024-09-15T21:20:30.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.879+0000 7f5e5bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:30.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.879+0000 7f5e5bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:30.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.971+0000 7f5e79b13640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:30.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.971+0000 7f5e79b13640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:30.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.971+0000 7f5e79b13640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:30.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.971+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f5e740008e0 con 0x7f5e7409f050 2024-09-15T21:20:30.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e5bfff640 1 -- 172.21.15.119:0/1963168172 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v76) ==== 56+0+1340 (secure 0 0 0) 0x7f5e68007b40 con 0x7f5e7409f050 2024-09-15T21:20:30.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e5bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:30.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e5bfff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:30.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6003b9a0 msgr2=0x7f5e6003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6003b9a0 0x7f5e6003de60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e640ad000 tx=0x7f5e64001d40 comp rx=0 tx=0).stop 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 10 monclient: shutdown 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 msgr2=0x7f5e74134210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 0x7f5e74134210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c820 tx=0x7f5e6800cce0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 shutdown_connections 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6003b9a0 0x7f5e6003de60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e74134750 0x7f5e741cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7409f050 0x7f5e74134210 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 --2- 172.21.15.119:0/1963168172 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e5c003a10 0x7f5e74072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 >> 172.21.15.119:0/1963168172 conn(0x7f5e7406cdc0 msgr2=0x7f5e7409c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:30.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 shutdown_connections 2024-09-15T21:20:30.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:30.975+0000 7f5e79b13640 1 -- 172.21.15.119:0/1963168172 wait complete. 2024-09-15T21:20:30.994 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.7 host1 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.103+0000 7f7215f3f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.103+0000 7f7215f3f640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.103+0000 7f7215f3f640 10 monclient: monmap: 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:31.107808+0000 2024-09-15T21:20:31.109 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:31.107808+0000 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:31.110 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.103+0000 7f7215f3f640 1 Processor -- start 2024-09-15T21:20:31.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.103+0000 7f7215f3f640 1 -- start start 2024-09-15T21:20:31.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.103+0000 7f7215f3f640 10 monclient: init 2024-09-15T21:20:31.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f721009d400 0x7f721009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): picked mon.noname-b con 0x7f721009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f721009dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7210134de0 0x7f721013c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): picked mon.noname-a con 0x7f7210134de0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721006e240 con 0x7f721009dd20 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f721006e000 con 0x7f7210134de0 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f721009e7f0 con 0x7f721009d400 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 10 monclient(hunting): authenticate will time out at 1149.904419s 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720f7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f721009d400 msgr2=0x7f721009d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:31.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f721009d400 0x7f721009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7210134de0 0x7f721013c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7210134de0 0x7f721013c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:45398/0 (socket says 172.21.15.119:45398) 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 -- 172.21.15.119:0/891738459 learned_addr learned my addr 172.21.15.119:0/891738459 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): get_auth_request con 0x7f7210134de0 auth_method 0 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient(hunting): handle_auth_done global_id 5220 payload 274 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 -- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f721009d400 msgr2=0x7f721009d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:31.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 --2- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f721009d400 0x7f721009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 -- 172.21.15.119:0/891738459 >> v1:172.21.15.17:6789/0 conn(0x7f721009dd20 legacy=0x7f72101346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 monclient: found mon.noname-a 2024-09-15T21:20:31.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 -- 172.21.15.119:0/891738459 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721009e140 con 0x7f7210134de0 2024-09-15T21:20:31.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720ffff640 1 --2- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7210134de0 0x7f721013c9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7200009870 tx=0x7f720001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362022ca971868da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 1 -- 172.21.15.119:0/891738459 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000b0070 con 0x7f7210134de0 2024-09-15T21:20:31.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:31.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 10 monclient: dump: 2024-09-15T21:20:31.120 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.121 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 5 monclient: authenticate success, global_id 5220 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f7215f3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.107+0000 7f720e7fc640 1 -- 172.21.15.119:0/891738459 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72000ab600 con 0x7f7210134de0 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720e7fc640 1 -- 172.21.15.119:0/891738459 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000ab9d0 con 0x7f7210134de0 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 4 set_mon_vals no callback set 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720e7fc640 10 monclient: dump: 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: shutdown 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7210134de0 msgr2=0x7f721013c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 --2- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7210134de0 0x7f721013c9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7200009870 tx=0x7f720001c440 comp rx=0 tx=0).stop 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/891738459 shutdown_connections 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 --2- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7210134de0 0x7f721013c9a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 --2- 172.21.15.119:0/891738459 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f721009d400 0x7f721009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/891738459 >> 172.21.15.119:0/891738459 conn(0x7f721009be60 msgr2=0x7f721009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/891738459 shutdown_connections 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/891738459 wait complete. 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: monmap: 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:31.124 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 Processor -- start 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- start start 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: init 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f721009d400 0x7f721013def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): picked mon.noname-a con 0x7f721009d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:31.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 0x7f721013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): picked mon.noname-b con 0x7f721009dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7210134de0 0x7f721013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): picked mon.noname-c con 0x7f7210134de0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f72100a0360 con 0x7f721009d400 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f72100a01e0 con 0x7f721009dd20 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f721013d6f0 con 0x7f7210134de0 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient(hunting): authenticate will time out at 1149.910047s 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720ffff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7210134de0 msgr2=0x7f721013e970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 0x7f721013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7210134de0 0x7f721013e970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 0x7f721013e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49246/0 (socket says 172.21.15.119:49246) 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 -- 172.21.15.119:0/1016324651 learned_addr learned my addr 172.21.15.119:0/1016324651 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): get_auth_request con 0x7f721009dd20 auth_method 0 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720f7fe640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f721009d400 0x7f721013def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720f7fe640 10 monclient(hunting): get_auth_request con 0x7f721009d400 auth_method 0 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient(hunting): handle_auth_done global_id 34405 payload 824 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 -- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7210134de0 msgr2=0x7f721013e970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7210134de0 0x7f721013e970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 -- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f721009d400 msgr2=0x7f721013def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f721009d400 0x7f721013def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 monclient: found mon.noname-b 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 -- 172.21.15.119:0/1016324651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7210139eb0 con 0x7f721009dd20 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720f7fe640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f721009d400 0x7f721013def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:31.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720effd640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 0x7f721013e430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f720400c4b0 tx=0x7f720400c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040052d0 con 0x7f721009dd20 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: dump: 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.129 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7204005470 con 0x7f721009dd20 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 5 monclient: authenticate success, global_id 34405 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _renew_subs 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721013cb60 con 0x7f721009dd20 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204012850 con 0x7f721009dd20 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 10 monclient: dump: 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _renew_subs 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f720dffb640 4 set_mon_vals no callback set 2024-09-15T21:20:31.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721013a480 con 0x7f721009dd20 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721013db80 con 0x7f721009dd20 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.111+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f72040129f0 con 0x7f721009dd20 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.115+0000 7f71f7fff640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f71fc03b970 0x7f71fc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.115+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6249+0+0 (secure 0 0 0) 0x7f720404fca0 con 0x7f721009dd20 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.115+0000 7f720f7fe640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f71fc03b970 0x7f71fc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.115+0000 7f720f7fe640 10 monclient: get_auth_request con 0x7f71fc03b970 auth_method 0 2024-09-15T21:20:31.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.115+0000 7f720f7fe640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f71fc03b970 0x7f71fc03de30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7208003cc0 tx=0x7f72080095f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.119+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7204019d10 con 0x7f721009dd20 2024-09-15T21:20:31.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.119+0000 7f71f7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.119+0000 7f71f7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:31.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.291+0000 7f7215f3f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T21:20:31.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.291+0000 7f7215f3f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T21:20:31.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.291+0000 7f7215f3f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.297 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.291+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) -- 0x7f721013fd90 con 0x7f721009dd20 2024-09-15T21:20:31.317 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T21:20:31.317 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:20:31.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:20:31.320 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:20:31.320 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:20:31.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:31.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient: get_monmap_and_config 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient: monmap: 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:31.443878+0000 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:31.443878+0000 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:31.446 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:31.447 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:31.447 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 1 Processor -- start 2024-09-15T21:20:31.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 1 -- start start 2024-09-15T21:20:31.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient: init 2024-09-15T21:20:31.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad401378c0 0x7fad40137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient(hunting): picked mon.noname-c con 0x7fad401378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fad401381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fad400a5210 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad400a1a20 con 0x7fad401381e0 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad400a1d00 con 0x7fad400a5210 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fad400a1fe0 con 0x7fad401378c0 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 10 monclient(hunting): authenticate will time out at 1154.839011s 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fad401381e0 0x7fad400a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49034/0 (socket says 172.21.15.17:49034) 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3effd640 1 -- 172.21.15.17:0/2475820671 learned_addr learned my addr 172.21.15.17:0/2475820671 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310994260 0 0) 0x7fad30001120 con 0x7fad401381e0 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): my global_id is 5229 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/2475820671 >> v2:172.21.15.119:3300/0 conn(0x7fad401378c0 0x7fad40137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad28003530 con 0x7fad401381e0 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): get_auth_request con 0x7fad401378c0 auth_method 0 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2791990007 0 0) 0x7fad28003530 con 0x7fad401381e0 2024-09-15T21:20:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 >> v2:172.21.15.119:3300/0 conn(0x7fad401378c0 msgr2=0x7fad40137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 --2- 172.21.15.17:0/2475820671 >> v2:172.21.15.119:3300/0 conn(0x7fad401378c0 0x7fad40137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 >> v1:172.21.15.119:6789/0 conn(0x7fad400a5210 legacy=0x7fad400a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad400a22c0 con 0x7fad401381e0 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fad300018d0 con 0x7fad401381e0 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 5 monclient: authenticate success, global_id 5229 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: dump: 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 >> v1:172.21.15.17:6789/0 conn(0x7fad401381e0 legacy=0x7fad400a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad28003a60 0x7fad28003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): picked mon.a con 0x7fad28003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28004810 0x7fad28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): picked mon.c con 0x7fad28004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad28005400 0x7fad2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): picked mon.b con 0x7fad28005400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad400a1fe0 con 0x7fad28003a60 2024-09-15T21:20:31.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad2800c290 con 0x7fad28005400 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad2800c490 con 0x7fad28004810 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad3e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.441+0000 7fad45efd640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad28003a60 0x7fad28003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): get_auth_request con 0x7fad28003a60 auth_method 0 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28004810 0x7fad28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): get_auth_request con 0x7fad28004810 auth_method 0 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3ffff640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad28005400 0x7fad2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3ffff640 10 monclient(hunting): get_auth_request con 0x7fad28005400 auth_method 0 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient(hunting): handle_auth_done global_id 5229 payload 293 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 -- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28004810 msgr2=0x7fad28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28004810 0x7fad28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 -- 172.21.15.17:0/2475820671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad28005400 msgr2=0x7fad2800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad28005400 0x7fad2800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 monclient: found mon.a 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3ffff640 1 -- 172.21.15.17:0/2475820671 reap_dead start 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 -- 172.21.15.17:0/2475820671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad28003530 con 0x7fad28003a60 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad28003a60 0x7fad28003e80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad34009ae0 tx=0x7fad3401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f588a1a9e6e8f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340b1070 con 0x7fad28003a60 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: dump: 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.465 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad340a4990 con 0x7fad28003a60 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 1 -- 172.21.15.17:0/2475820671 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340096b0 con 0x7fad28003a60 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 4 set_mon_vals no callback set 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad3e7fc640 10 monclient: dump: 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 10 monclient: shutdown 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 -- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad28003a60 msgr2=0x7fad28003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad28003a60 0x7fad28003e80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad34009ae0 tx=0x7fad3401c730 comp rx=0 tx=0).stop 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 -- 172.21.15.17:0/2475820671 shutdown_connections 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 --2- 172.21.15.17:0/2475820671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad28003a60 0x7fad28003e80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 -- 172.21.15.17:0/2475820671 >> 172.21.15.17:0/2475820671 conn(0x7fad400aac40 msgr2=0x7fad400a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 -- 172.21.15.17:0/2475820671 shutdown_connections 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 1 -- 172.21.15.17:0/2475820671 wait complete. 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.445+0000 7fad45efd640 10 monclient: monmap: 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 Processor -- start 2024-09-15T21:20:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- start start 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: init 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): picked mon.noname-c con 0x7fad28003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 0x7fad40120750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): picked mon.noname-a con 0x7fad40137150 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad40120c90 0x7fad40121140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): picked mon.noname-b con 0x7fad40120c90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad400a5b80 con 0x7fad40137150 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad400a5a00 con 0x7fad40120c90 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad4013b580 con 0x7fad28003a60 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient(hunting): authenticate will time out at 1154.844721s 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 0x7fad40120750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57618/0 (socket says 172.21.15.17:57618) 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 -- 172.21.15.17:0/1017904081 learned_addr learned my addr 172.21.15.17:0/1017904081 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 0x7fad40120750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41980/0 (socket says 172.21.15.17:41980) 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 10 monclient(hunting): get_auth_request con 0x7fad28003a60 auth_method 0 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): get_auth_request con 0x7fad40137150 auth_method 0 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 -- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 msgr2=0x7fad40136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 -- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 msgr2=0x7fad40136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3ffff640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad40120c90 0x7fad40121140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient(hunting): handle_auth_done global_id 5232 payload 824 2024-09-15T21:20:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 -- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 msgr2=0x7fad40136c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 -- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad40120c90 msgr2=0x7fad40121140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad40120c90 0x7fad40121140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 monclient: found mon.noname-a 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 -- 172.21.15.17:0/1017904081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad401375d0 con 0x7fad40137150 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3ffff640 10 monclient: get_auth_request con 0x7fad40120c90 auth_method 0 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3ffff640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad40120c90 0x7fad40121140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3effd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 0x7fad40120750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fad4006a210 tx=0x7fad30007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3000f780 con 0x7fad40137150 2024-09-15T21:20:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: dump: 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 5 monclient: authenticate success, global_id 5232 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad30018070 con 0x7fad40137150 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _renew_subs 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3000fb60 con 0x7fad40137150 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad401245d0 con 0x7fad40137150 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 10 monclient: dump: 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3dffb640 4 set_mon_vals no callback set 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _renew_subs 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad40123390 con 0x7fad40137150 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4013b820 con 0x7fad40137150 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fad3000fd00 con 0x7fad40137150 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad2c03b9c0 0x7fad2c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6249+0+0 (secure 0 0 0) 0x7fad3004fd90 con 0x7fad40137150 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad2c03b9c0 0x7fad2c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 10 monclient: get_auth_request con 0x7fad2c03b9c0 auth_method 0 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.449+0000 7fad3f7fe640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad2c03b9c0 0x7fad2c03de80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad3401c770 tx=0x7fad340046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.453+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fad400a1a20 con 0x7fad40137150 2024-09-15T21:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.453+0000 7fad27fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.453+0000 7fad27fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:31.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.475+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 removed item id 7 name 'osd.7' from crush map v77) ==== 143+0+0 (secure 0 0 0) 0x7f72040102e0 con 0x7f721009dd20 2024-09-15T21:20:31.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.475+0000 7f71f7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T21:20:31.480 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.475+0000 7f71f7fff640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-09-15T21:20:31.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f7215f3f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T21:20:31.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f7215f3f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T21:20:31.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f7215f3f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) -- 0x7f721013fd90 con 0x7f721009dd20 2024-09-15T21:20:31.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f71f7fff640 1 -- 172.21.15.119:0/1016324651 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 device 'osd.7' does not appear in the crush map v77) ==== 145+0+0 (secure 0 0 0) 0x7f7204019300 con 0x7f721009dd20 2024-09-15T21:20:31.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f71f7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T21:20:31.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.583+0000 7f71f7fff640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-09-15T21:20:31.589 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.7' does not appear in the crush map 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f71fc03b970 msgr2=0x7f71fc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f71fc03b970 0x7f71fc03de30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7208003cc0 tx=0x7f72080095f0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 10 monclient: shutdown 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 msgr2=0x7f721013e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 0x7f721013e430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f720400c4b0 tx=0x7f720400c970 comp rx=0 tx=0).stop 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 shutdown_connections 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f71fc03b970 0x7f71fc03de30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7210134de0 0x7f721013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f721009dd20 0x7f721013e430 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 --2- 172.21.15.119:0/1016324651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f721009d400 0x7f721013def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 >> 172.21.15.119:0/1016324651 conn(0x7f721009be60 msgr2=0x7f72100a3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 shutdown_connections 2024-09-15T21:20:31.592 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.587+0000 7f7215f3f640 1 -- 172.21.15.119:0/1016324651 wait complete. 2024-09-15T21:20:31.605 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm host1 2024-09-15T21:20:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fad40122270 con 0x7fad40137150 2024-09-15T21:20:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad27fff640 1 -- 172.21.15.17:0/1017904081 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fad30012d60 con 0x7fad40137150 2024-09-15T21:20:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad27fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad27fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:31.698 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad2c03b9c0 msgr2=0x7fad2c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad2c03b9c0 0x7fad2c03de80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad3401c770 tx=0x7fad340046d0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 10 monclient: shutdown 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 msgr2=0x7fad40120750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 0x7fad40120750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fad4006a210 tx=0x7fad30007430 comp rx=0 tx=0).stop 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 shutdown_connections 2024-09-15T21:20:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad2c03b9c0 0x7fad2c03de80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad40120c90 0x7fad40121140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad40137150 0x7fad40120750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.693+0000 7fad45efd640 1 --2- 172.21.15.17:0/1017904081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad28003a60 0x7fad40136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.697+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 >> 172.21.15.17:0/1017904081 conn(0x7fad400aac40 msgr2=0x7fad4009c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.697+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 shutdown_connections 2024-09-15T21:20:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.697+0000 7fad45efd640 1 -- 172.21.15.17:0/1017904081 wait complete. 2024-09-15T21:20:31.715 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:20:31.715 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:20:31.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient: get_monmap_and_config 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient: monmap: 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:31.721868+0000 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:31.721868+0000 2024-09-15T21:20:31.723 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:31.724 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 Processor -- start 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 -- start start 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient: init 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2678137bc0 0x7f2678137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): picked mon.noname-c con 0x7f2678137bc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 0x7f2678073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): picked mon.noname-b con 0x7f26780a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f26780a0ea0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267809cfa0 con 0x7f26780a0ea0 2024-09-15T21:20:31.728 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f267809cd20 con 0x7f2678137bc0 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f26780a1820 con 0x7f26780a09e0 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f26800cb640 10 monclient(hunting): authenticate will time out at 1150.518418s 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2678137bc0 0x7f2678137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2678137bc0 0x7f2678137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49252/0 (socket says 172.21.15.119:49252) 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 -- 172.21.15.119:0/2428486325 learned_addr learned my addr 172.21.15.119:0/2428486325 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): get_auth_request con 0x7f2678137bc0 auth_method 0 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.729 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 0x7f2678073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 10 monclient(hunting): get_auth_request con 0x7f26780a09e0 auth_method 0 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient(hunting): handle_auth_done global_id 34414 payload 274 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 1 -- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 msgr2=0x7f2678073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 1 -- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 msgr2=0x7f2678073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 -- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 msgr2=0x7f2678073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:31.730 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 0x7f2678073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267d666640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 0x7f2678073040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 0x7f2678073040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 -- 172.21.15.119:0/2428486325 >> v1:172.21.15.17:6789/0 conn(0x7f26780a0ea0 legacy=0x7f2678073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 monclient: found mon.noname-c 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.719+0000 7f267de67640 1 -- 172.21.15.119:0/2428486325 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26780a1b00 con 0x7f2678137bc0 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2678137bc0 0x7f2678137fa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2674009870 tx=0x7f267401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22701b42525b4db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 1 -- 172.21.15.119:0/2428486325 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740b1070 con 0x7f2678137bc0 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: dump: 2024-09-15T21:20:31.731 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 1 -- 172.21.15.119:0/2428486325 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26740ac640 con 0x7f2678137bc0 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 1 -- 172.21.15.119:0/2428486325 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740aca10 con 0x7f2678137bc0 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267ce65640 10 monclient: dump: 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.733 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 5 monclient: authenticate success, global_id 34414 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 4 set_mon_vals no callback set 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: shutdown 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- 172.21.15.119:0/2428486325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2678137bc0 msgr2=0x7f2678137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2678137bc0 0x7f2678137fa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2674009870 tx=0x7f267401c510 comp rx=0 tx=0).stop 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- 172.21.15.119:0/2428486325 shutdown_connections 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26780a09e0 0x7f2678073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 --2- 172.21.15.119:0/2428486325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2678137bc0 0x7f2678137fa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- 172.21.15.119:0/2428486325 >> 172.21.15.119:0/2428486325 conn(0x7f26780a2ae0 msgr2=0x7f26780a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- 172.21.15.119:0/2428486325 shutdown_connections 2024-09-15T21:20:31.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- 172.21.15.119:0/2428486325 wait complete. 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: monmap: 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.739 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 Processor -- start 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- start start 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: init 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26780a09e0 0x7f267811f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): picked mon.noname-a con 0x7f26780a09e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:31.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 0x7f267811d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): picked mon.noname-b con 0x7f26780a0ea0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 0x7f2678136860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): picked mon.noname-c con 0x7f2678137bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2678073e10 con 0x7f26780a09e0 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2678073c90 con 0x7f26780a0ea0 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f26781422d0 con 0x7f2678137bc0 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f26800cb640 10 monclient(hunting): authenticate will time out at 1150.523506s 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 0x7f267811d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 0x7f267811d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49266/0 (socket says 172.21.15.119:49266) 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 -- 172.21.15.119:0/1007991610 learned_addr learned my addr 172.21.15.119:0/1007991610 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): get_auth_request con 0x7f26780a0ea0 auth_method 0 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26780a09e0 0x7f267811f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): get_auth_request con 0x7f26780a09e0 auth_method 0 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 0x7f2678136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 10 monclient(hunting): get_auth_request con 0x7f2678137bc0 auth_method 0 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267de67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 1 -- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 msgr2=0x7f2678136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 1 -- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 msgr2=0x7f2678136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 0x7f2678136860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267e668640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 0x7f2678136860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:31.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient(hunting): handle_auth_done global_id 34417 payload 824 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 -- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 msgr2=0x7f2678136860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 0x7f2678136860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 -- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26780a09e0 msgr2=0x7f267811f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26780a09e0 0x7f267811f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 monclient: found mon.noname-b 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.723+0000 7f267d666640 1 -- 172.21.15.119:0/1007991610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267809d1d0 con 0x7f26780a0ea0 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f267d666640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 0x7f267811d8b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f266800e810 tx=0x7f266800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668017020 con 0x7f26780a0ea0 2024-09-15T21:20:31.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: dump: 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266800c6d0 con 0x7f26780a0ea0 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266800f040 con 0x7f26780a0ea0 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 10 monclient: dump: 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.746 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 5 monclient: authenticate success, global_id 34417 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 monclient: _renew_subs 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2678137500 con 0x7f26780a0ea0 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266ffff640 4 set_mon_vals no callback set 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 monclient: _renew_subs 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2678120f10 con 0x7f26780a0ea0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2678137fa0 con 0x7f26780a0ea0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f266801a020 con 0x7f26780a0ea0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f265c03b9c0 0x7f265c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6228+0+0 (secure 0 0 0) 0x7f266804ffe0 con 0x7f26780a0ea0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f267de67640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f265c03b9c0 0x7f265c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f267de67640 10 monclient: get_auth_request con 0x7f265c03b9c0 auth_method 0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.727+0000 7f267de67640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f265c03b9c0 0x7f265c03de80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26740ae000 tx=0x7f2674008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.731+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2668021020 con 0x7f26780a0ea0 2024-09-15T21:20:31.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.731+0000 7f266e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.731+0000 7f266e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:31.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient: get_monmap_and_config 2024-09-15T21:20:31.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient: monmap: 2024-09-15T21:20:31.833 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:31.831608+0000 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:31.831608+0000 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:31.834 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:31.835 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 Processor -- start 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 -- start start 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient: init 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f312c0a0890 0x7f312c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): picked mon.noname-a con 0x7f312c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f312c09ffc0 0x7f312c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): picked mon.noname-c con 0x7f312c09ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f312c135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c0a3130 con 0x7f312c135670 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f312c0a16f0 con 0x7f312c0a0890 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f312c073df0 con 0x7f312c09ffc0 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f31331bf640 10 monclient(hunting): authenticate will time out at 1155.226934s 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f312c0a0890 0x7f312c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f312c0a0890 0x7f312c0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42004/0 (socket says 172.21.15.17:42004) 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f313175c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f312c135670 0x7f312c135aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49040/0 (socket says 172.21.15.17:49040) 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 -- 172.21.15.17:0/1108746952 learned_addr learned my addr 172.21.15.17:0/1108746952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): get_auth_request con 0x7f312c0a0890 auth_method 0 2024-09-15T21:20:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 1 --2- 172.21.15.17:0/1108746952 >> v2:172.21.15.119:3300/0 conn(0x7f312c09ffc0 0x7f312c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): get_auth_request con 0x7f312c09ffc0 auth_method 0 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient(hunting): handle_auth_done global_id 5241 payload 274 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 -- 172.21.15.17:0/1108746952 >> v2:172.21.15.119:3300/0 conn(0x7f312c09ffc0 msgr2=0x7f312c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 --2- 172.21.15.17:0/1108746952 >> v2:172.21.15.119:3300/0 conn(0x7f312c09ffc0 0x7f312c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 -- 172.21.15.17:0/1108746952 >> v1:172.21.15.17:6789/0 conn(0x7f312c135670 legacy=0x7f312c135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 monclient: found mon.noname-a 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f3130f5b640 1 -- 172.21.15.17:0/1108746952 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c0a1bf0 con 0x7f312c0a0890 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312b7fe640 1 -- 172.21.15.17:0/1108746952 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3652983342 0 0) 0x7f312c0a3130 con 0x7f312c135670 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.829+0000 7f312b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f3130f5b640 1 --2- 172.21.15.17:0/1108746952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c0a0890 0x7f312c0a0c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f311c009870 tx=0x7f311c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2713415225fe8315 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 1 -- 172.21.15.17:0/1108746952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b0070 con 0x7f312c0a0890 2024-09-15T21:20:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: dump: 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 5 monclient: authenticate success, global_id 5241 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 1 -- 172.21.15.17:0/1108746952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c0ab600 con 0x7f312c0a0890 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 4 set_mon_vals no callback set 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 1 -- 172.21.15.17:0/1108746952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0ab9d0 con 0x7f312c0a0890 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f312b7fe640 10 monclient: dump: 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.844 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 10 monclient: shutdown 2024-09-15T21:20:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 -- 172.21.15.17:0/1108746952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c0a0890 msgr2=0x7f312c0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 --2- 172.21.15.17:0/1108746952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c0a0890 0x7f312c0a0c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f311c009870 tx=0x7f311c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 -- 172.21.15.17:0/1108746952 shutdown_connections 2024-09-15T21:20:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 --2- 172.21.15.17:0/1108746952 >> v2:172.21.15.119:3300/0 conn(0x7f312c09ffc0 0x7f312c0a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 --2- 172.21.15.17:0/1108746952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c0a0890 0x7f312c0a0c70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 -- 172.21.15.17:0/1108746952 >> 172.21.15.17:0/1108746952 conn(0x7f312c0a66a0 msgr2=0x7f312c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 -- 172.21.15.17:0/1108746952 shutdown_connections 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 -- 172.21.15.17:0/1108746952 wait complete. 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 10 monclient: build_initial_monmap 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 10 monclient: monmap: 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 Processor -- start 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 1 -- start start 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.833+0000 7f31331bf640 10 monclient: init 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312c09ffc0 0x7f312c13c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): picked mon.noname-b con 0x7f312c09ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 0x7f312c134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): picked mon.noname-c con 0x7f312c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c135670 0x7f312c134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): picked mon.noname-a con 0x7f312c135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f312c140690 con 0x7f312c135670 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f312c140510 con 0x7f312c09ffc0 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f312c140810 con 0x7f312c0a0890 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): authenticate will time out at 1155.232392s 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c135670 0x7f312c134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 0x7f312c134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c135670 0x7f312c134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42016/0 (socket says 172.21.15.17:42016) 2024-09-15T21:20:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 0x7f312c134730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57632/0 (socket says 172.21.15.17:57632) 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 -- 172.21.15.17:0/784022898 learned_addr learned my addr 172.21.15.17:0/784022898 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): get_auth_request con 0x7f312c135670 auth_method 0 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 10 monclient(hunting): get_auth_request con 0x7f312c0a0890 auth_method 0 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312c09ffc0 0x7f312c13c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 msgr2=0x7f312c134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 msgr2=0x7f312c134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 0x7f312c134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): get_auth_request con 0x7f312c09ffc0 auth_method 0 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 0x7f312c134730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient(hunting): handle_auth_done global_id 5247 payload 824 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 msgr2=0x7f312c134730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f312c0a0890 0x7f312c134730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312c09ffc0 msgr2=0x7f312c13c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312c09ffc0 0x7f312c13c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 monclient: found mon.noname-a 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c13d130 con 0x7f312c135670 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f313175c640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c135670 0x7f312c134c70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f312400c7a0 tx=0x7f312400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124018070 con 0x7f312c135670 2024-09-15T21:20:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: dump: 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:31.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312400a640 con 0x7f312c135670 2024-09-15T21:20:31.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 5 monclient: authenticate success, global_id 5247 2024-09-15T21:20:31.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient: _renew_subs 2024-09-15T21:20:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c121100 con 0x7f312c135670 2024-09-15T21:20:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400aa70 con 0x7f312c135670 2024-09-15T21:20:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: dump: 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312affd640 4 set_mon_vals no callback set 2024-09-15T21:20:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient: _renew_subs 2024-09-15T21:20:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c09d910 con 0x7f312c135670 2024-09-15T21:20:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 0x7f312c1cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f312c0a5620 con 0x7f312c1cf820 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 0x7f312c1cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient: get_auth_request con 0x7f312c1cf820 auth_method 0 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f312401b070 con 0x7f312c135670 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 10 monclient(hunting): handle_auth_done global_id 5250 payload 824 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f312c09d0f0 con 0x7f312c1cf820 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f311803b9c0 0x7f311803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6228+0+0 (secure 0 0 0) 0x7f312404f620 con 0x7f312c135670 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f3130f5b640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 0x7f312c1cfc00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f312c073b00 tx=0x7f311c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b0040 con 0x7f312c1cf820 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: dump: 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f311803b9c0 0x7f311803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 10 monclient: get_auth_request con 0x7f311803b9c0 auth_method 0 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f311c009490 con 0x7f312c1cf820 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 msgr2=0x7f312c1cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 0x7f312c1cfc00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f312c073b00 tx=0x7f311c019180 comp rx=0 tx=0).stop 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 msgr2=0x7f312c1cfc00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f31297fa640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1cf820 0x7f312c1cfc00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.837+0000 7f312bfff640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f311803b9c0 0x7f311803de80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f312c09f5e0 tx=0x7f3120008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 0x7f312c138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f312c06d020 con 0x7f312c1d0140 2024-09-15T21:20:31.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 0x7f312c138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient: get_auth_request con 0x7f312c1d0140 auth_method 0 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 10 monclient(hunting): handle_auth_done global_id 5253 payload 824 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 1 -- 172.21.15.17:0/784022898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f312c1d0550 con 0x7f312c1d0140 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f3130f5b640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 0x7f312c138ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f311c018ff0 tx=0x7f311c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0c0b60 con 0x7f312c1d0140 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:31.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 10 monclient: dump: 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f311c0c0b60 con 0x7f312c1d0140 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:31.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 msgr2=0x7f312c138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 0x7f312c138ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f311c018ff0 tx=0x7f311c009240 comp rx=0 tx=0).stop 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 msgr2=0x7f312c138ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31297fa640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c1d0140 0x7f312c138ab0 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f311c018ff0 tx=0x7f311c009240 comp rx=0 tx=0).stop 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:20:31.869 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 38, 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 197467, 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:31.870 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:31.871 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:31.872 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:31.873 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:31.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:31.874 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:31.874 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:31.874 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:31.874 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:31.874 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:31.875 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:31.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.877 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:31.877 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:31.877 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:31.877 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:31.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:31.878 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:31.878 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:31.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:31.878 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:31.879 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.880 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:31.882 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.883 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:20:31.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:31.885 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:31.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:31.885 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:31.885 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f311803b9c0 msgr2=0x7f311803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f311803b9c0 0x7f311803de80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f312c09f5e0 tx=0x7f3120008040 comp rx=0 tx=0).stop 2024-09-15T21:20:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 10 monclient: shutdown 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c135670 msgr2=0x7f312c134c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 --2- 172.21.15.17:0/784022898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312c135670 0x7f312c134c70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f312400c7a0 tx=0x7f312400cc60 comp rx=0 tx=0).stop 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f313175c640 1 -- 172.21.15.17:0/784022898 reap_dead start 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f313175c640 1 -- 172.21.15.17:0/784022898 reap_dead start 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 shutdown_connections 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 >> 172.21.15.17:0/784022898 conn(0x7f312c0a66a0 msgr2=0x7f312c072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 shutdown_connections 2024-09-15T21:20:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.845+0000 7f31331bf640 1 -- 172.21.15.17:0/784022898 wait complete. 2024-09-15T21:20:31.887 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:20:31.888 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:20:31.888 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:20:31.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.911+0000 7f26800cb640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T21:20:31.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.911+0000 7f26800cb640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T21:20:31.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.911+0000 7f26800cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:31.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:31.911+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) -- 0x7f267806c460 con 0x7f26780a0ea0 2024-09-15T21:20:32.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient: get_monmap_and_config 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient: build_initial_monmap 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient: monmap: 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:31.999871+0000 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:31.999871+0000 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:32.002 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:32.003 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:32.003 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:32.003 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 1 Processor -- start 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 1 -- start start 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient: init 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f13f01378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13f013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient(hunting): picked mon.noname-b con 0x7f13f01383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f13f013d2c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13f00a1a20 con 0x7f13f013d2c0 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13f00a1d00 con 0x7f13f01378c0 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f13f00a1fe0 con 0x7f13f01383b0 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f6ea6640 10 monclient(hunting): authenticate will time out at 1155.395438s 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13f013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f5443640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f13f013d2c0 0x7f13f013f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49048/0 (socket says 172.21.15.17:49048) 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13f013cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57642/0 (socket says 172.21.15.17:57642) 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13f5443640 1 -- 172.21.15.17:0/678676237 learned_addr learned my addr 172.21.15.17:0/678676237 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:32.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 10 monclient(hunting): get_auth_request con 0x7f13f01383b0 auth_method 0 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 1 -- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 msgr2=0x7f13f013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 1 -- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 msgr2=0x7f13f013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13f013cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:31.997+0000 7f13effff640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13f013cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750299852 0 0) 0x7f13e8001120 con 0x7f13f013d2c0 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): my global_id is 5256 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13dc003530 con 0x7f13f013d2c0 2024-09-15T21:20:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906358823 0 0) 0x7f13f00a1d00 con 0x7f13f01378c0 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): my global_id is 34426 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e8001120 con 0x7f13f01378c0 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 519007104 0 0) 0x7f13dc003530 con 0x7f13f013d2c0 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 msgr2=0x7f13f013cd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13f013cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 >> v1:172.21.15.119:6789/0 conn(0x7f13f01378c0 legacy=0x7f13f0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f00a22c0 con 0x7f13f013d2c0 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 5 monclient: authenticate success, global_id 5256 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f13e80018d0 con 0x7f13f013d2c0 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: dump: 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 >> v1:172.21.15.17:6789/0 conn(0x7f13f013d2c0 legacy=0x7f13f013f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc003c20 0x7f13dc004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): picked mon.b con 0x7f13dc003c20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): picked mon.c con 0x7f13f01383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc004e70 0x7f13dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): picked mon.a con 0x7f13dc004e70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13f00a1fe0 con 0x7f13dc004e70 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc005660 con 0x7f13dc003c20 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f13dc00bf60 con 0x7f13f01383b0 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): get_auth_request con 0x7f13f01383b0 auth_method 0 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc004e70 0x7f13dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc003c20 0x7f13dc004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): get_auth_request con 0x7f13dc003c20 auth_method 0 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): get_auth_request con 0x7f13dc004e70 auth_method 0 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f4c42640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient(hunting): handle_auth_done global_id 5256 payload 293 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 -- 172.21.15.17:0/678676237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc003c20 msgr2=0x7f13dc004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc003c20 0x7f13dc004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 -- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc004e70 msgr2=0x7f13dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc004e70 0x7f13dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 monclient: found mon.c 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 -- 172.21.15.17:0/678676237 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc003530 con 0x7f13f01383b0 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient: _finish_auth 0 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13effff640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13dc004810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c850 tx=0x7f13d801c1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5adc87c31d71529 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80b1070 con 0x7f13f01383b0 2024-09-15T21:20:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: dump: 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d80a47b0 con 0x7f13f01383b0 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 4 set_mon_vals no callback set 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 1 -- 172.21.15.17:0/678676237 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d800f040 con 0x7f13f01383b0 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13ef7fe640 10 monclient: dump: 2024-09-15T21:20:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 10 monclient: shutdown 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 1 -- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 msgr2=0x7f13dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/678676237 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f01383b0 0x7f13dc004810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c850 tx=0x7f13d801c1f0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f5443640 1 -- 172.21.15.17:0/678676237 reap_dead start 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 1 -- 172.21.15.17:0/678676237 shutdown_connections 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.001+0000 7f13f6ea6640 1 -- 172.21.15.17:0/678676237 >> 172.21.15.17:0/678676237 conn(0x7f13f00aac40 msgr2=0x7f13f00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- 172.21.15.17:0/678676237 shutdown_connections 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- 172.21.15.17:0/678676237 wait complete. 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: build_initial_monmap 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: monmap: 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 Processor -- start 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- start start 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: init 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13f01383b0 0x7f13f009d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): picked mon.noname-a con 0x7f13f01383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13f009d880 0x7f13f009dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): picked mon.noname-b con 0x7f13f009d880 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 0x7f13f01d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): picked mon.noname-c con 0x7f13f009e200 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0141d80 con 0x7f13f01383b0 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0141c00 con 0x7f13f009d880 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f13f0141f00 con 0x7f13f009e200 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): authenticate will time out at 1155.401732s 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 0x7f13f01d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13f01383b0 0x7f13f009d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 0x7f13f01d70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57646/0 (socket says 172.21.15.17:57646) 2024-09-15T21:20:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 -- 172.21.15.17:0/1757161647 learned_addr learned my addr 172.21.15.17:0/1757161647 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 10 monclient(hunting): get_auth_request con 0x7f13f009e200 auth_method 0 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): get_auth_request con 0x7f13f01383b0 auth_method 0 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 msgr2=0x7f13f01d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 msgr2=0x7f13f01d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 0x7f13f01d70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13f009d880 0x7f13f009dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f5443640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 0x7f13f01d70c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): get_auth_request con 0x7f13f009d880 auth_method 0 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient(hunting): handle_auth_done global_id 5259 payload 824 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 msgr2=0x7f13f01d70c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f009e200 0x7f13f01d70c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13f009d880 msgr2=0x7f13f009dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13f009d880 0x7f13f009dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient: _finish_hunting 0 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 monclient: found mon.noname-a 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f0120880 con 0x7f13f01383b0 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f4c42640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13f01383b0 0x7f13f009d340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f13e000fb90 tx=0x7f13e000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e000fe90 con 0x7f13f01383b0 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: dump: 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e0018070 con 0x7f13f01383b0 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e000de50 con 0x7f13f01383b0 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 5 monclient: authenticate success, global_id 5259 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: _renew_subs 2024-09-15T21:20:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f0120b60 con 0x7f13f01383b0 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13eeffd640 4 set_mon_vals no callback set 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 10 monclient: dump: 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: _renew_subs 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f00a12f0 con 0x7f13f01383b0 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f13f009f700 con 0x7f13f002e5d0 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13f002e5d0 auth_method 0 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f13e0003980 con 0x7f13f01383b0 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13d003bd30 0x7f13d003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.005+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6228+0+0 (secure 0 0 0) 0x7f13e0050250 con 0x7f13f01383b0 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.009+0000 7f13f5443640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13d003bd30 0x7f13d003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.009+0000 7f13f5443640 10 monclient: get_auth_request con 0x7f13d003bd30 auth_method 0 2024-09-15T21:20:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.009+0000 7f13f5443640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13d003bd30 0x7f13d003e1f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13e8000c10 tx=0x7f13e8007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13f002e5d0 auth_method 0 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:32.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.209+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13f002e5d0 auth_method 0 2024-09-15T21:20:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:32.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:32.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:32.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:32.613+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:32.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.687+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 removed item id -11 name 'host1' from crush map v78) ==== 124+0+0 (secure 0 0 0) 0x7f2668012390 con 0x7f26780a0ea0 2024-09-15T21:20:32.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.687+0000 7f266e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T21:20:32.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.687+0000 7f266e7fc640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-09-15T21:20:32.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.791+0000 7f26800cb640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T21:20:32.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.791+0000 7f26800cb640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T21:20:32.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.791+0000 7f26800cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:32.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.791+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) -- 0x7f2678136ec0 con 0x7f26780a0ea0 2024-09-15T21:20:32.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f266e7fc640 1 -- 172.21.15.119:0/1007991610 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 device 'host1' does not appear in the crush map v78) ==== 124+0+0 (secure 0 0 0) 0x7f26680234c0 con 0x7f26780a0ea0 2024-09-15T21:20:32.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f266e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T21:20:32.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f266e7fc640 10 monclient: _finish_command 3 = system:0 device 'host1' does not appear in the crush map 2024-09-15T21:20:32.799 INFO:tasks.workunit.client.0.smithi119.stderr:device 'host1' does not appear in the crush map 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f265c03b9c0 msgr2=0x7f265c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f265c03b9c0 0x7f265c03de80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26740ae000 tx=0x7f2674008040 comp rx=0 tx=0).stop 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 10 monclient: shutdown 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 msgr2=0x7f267811d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 0x7f267811d8b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f266800e810 tx=0x7f266800ecd0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 shutdown_connections 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f265c03b9c0 0x7f265c03de80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2678137bc0 0x7f2678136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26780a0ea0 0x7f267811d8b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 --2- 172.21.15.119:0/1007991610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26780a09e0 0x7f267811f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 >> 172.21.15.119:0/1007991610 conn(0x7f26780a2ae0 msgr2=0x7f267809d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:32.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 shutdown_connections 2024-09-15T21:20:32.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.795+0000 7f26800cb640 1 -- 172.21.15.119:0/1007991610 wait complete. 2024-09-15T21:20:32.817 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:32.817 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c host1 2024-09-15T21:20:32.817 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 0 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient: get_monmap_and_config 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient: build_initial_monmap 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient: monmap: 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:32.926756+0000 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:32.926756+0000 2024-09-15T21:20:32.928 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:32.929 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 Processor -- start 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 -- start start 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient: init 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1498135d10 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f149806ec30 0x7f149813aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): picked mon.noname-a con 0x7f149806ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f149813d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): picked mon.noname-b con 0x7f149813b020 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14980a1b90 con 0x7f1498135d10 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f149806f080 con 0x7f149806ec30 2024-09-15T21:20:32.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f149809f6f0 con 0x7f149813b020 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f149d4bb640 10 monclient(hunting): authenticate will time out at 1151.723498s 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1496ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1498135d10 0x7f14981360f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46818/0 (socket says 172.21.15.119:46818) 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1496ffd640 1 -- 172.21.15.119:0/2664191692 learned_addr learned my addr 172.21.15.119:0/2664191692 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31912061 0 0) 0x7f14980a1b90 con 0x7f1498135d10 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1495ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1495ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1495ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1495ffb640 10 monclient(hunting): my global_id is 34432 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1480003530 con 0x7f1498135d10 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f14967fc640 1 --2- 172.21.15.119:0/2664191692 >> v2:172.21.15.17:3300/0 conn(0x7f149806ec30 0x7f149813aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f14967fc640 10 monclient(hunting): get_auth_request con 0x7f149806ec30 auth_method 0 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f14967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f14967fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.923+0000 7f14967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216005607 0 0) 0x7f1480003530 con 0x7f1498135d10 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 msgr2=0x7f149813d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14977fe640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f149813d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f149813d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 >> v2:172.21.15.17:3300/0 conn(0x7f149806ec30 msgr2=0x7f149813aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 --2- 172.21.15.119:0/2664191692 >> v2:172.21.15.17:3300/0 conn(0x7f149806ec30 0x7f149813aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:32.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980a1e70 con 0x7f1498135d10 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f1488001700 con 0x7f1498135d10 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 --2- 172.21.15.119:0/2664191692 >> v2:172.21.15.17:3300/0 conn(0x7f149806ec30 0x7f149813aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 5 monclient: authenticate success, global_id 34432 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: dump: 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:32.941 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 >> v1:172.21.15.119:6789/0 conn(0x7f1498135d10 legacy=0x7f14981360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1480003a60 0x7f1480003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): picked mon.a con 0x7f1480003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f1480004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): picked mon.b con 0x7f1480004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f148000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): picked mon.c con 0x7f149813b020 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f149806f080 con 0x7f1480003a60 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f149809f6f0 con 0x7f1480004810 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1480005690 con 0x7f149813b020 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:32.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f1480004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): get_auth_request con 0x7f1480004810 auth_method 0 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1496ffd640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1480003a60 0x7f1480003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1496ffd640 10 monclient(hunting): get_auth_request con 0x7f1480003a60 auth_method 0 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1496ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1496ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14977fe640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f148000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient(hunting): handle_auth_done global_id 34432 payload 293 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 -- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 msgr2=0x7f148000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f148000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 -- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1480003a60 msgr2=0x7f1480003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1480003a60 0x7f1480003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 monclient: found mon.b 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 -- 172.21.15.119:0/2664191692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1480003530 con 0x7f1480004810 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:32.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14977fe640 10 monclient: get_auth_request con 0x7f149813b020 auth_method 0 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14977fe640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f149813b020 0x7f148000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14967fc640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f1480004c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f148c0098d0 tx=0x7f148c01c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b5807609374f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c00f040 con 0x7f1480004810 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: dump: 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.946 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148c0b0070 con 0x7f1480004810 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 1 -- 172.21.15.119:0/2664191692 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c0a7070 con 0x7f1480004810 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 4 set_mon_vals no callback set 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f1495ffb640 10 monclient: dump: 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:32.947 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 10 monclient: shutdown 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 1 -- 172.21.15.119:0/2664191692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 msgr2=0x7f1480004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 1 --2- 172.21.15.119:0/2664191692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f1480004c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f148c0098d0 tx=0x7f148c01c630 comp rx=0 tx=0).stop 2024-09-15T21:20:32.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f14977fe640 1 -- 172.21.15.119:0/2664191692 reap_dead start 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 1 -- 172.21.15.119:0/2664191692 shutdown_connections 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 1 -- 172.21.15.119:0/2664191692 >> 172.21.15.119:0/2664191692 conn(0x7f149806cdc0 msgr2=0x7f149806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 1 -- 172.21.15.119:0/2664191692 shutdown_connections 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.927+0000 7f149d4bb640 1 -- 172.21.15.119:0/2664191692 wait complete. 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: build_initial_monmap 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: monmap: 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 Processor -- start 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- start start 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: init 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f14981cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): picked mon.noname-b con 0x7f1480004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f149811ef10 0x7f1498133d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): picked mon.noname-a con 0x7f149811ef10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1498134280 0x7f149811ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): picked mon.noname-c con 0x7f1498134280 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f149813dc20 con 0x7f149811ef10 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f149806dc30 con 0x7f1480004810 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f149813dda0 con 0x7f1498134280 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient(hunting): authenticate will time out at 1151.729054s 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f14981cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f14981cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39926/0 (socket says 172.21.15.119:39926) 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 -- 172.21.15.119:0/669542639 learned_addr learned my addr 172.21.15.119:0/669542639 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): get_auth_request con 0x7f1480004810 auth_method 0 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f149811ef10 0x7f1498133d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14977fe640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1498134280 0x7f149811ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 10 monclient(hunting): get_auth_request con 0x7f149811ef10 auth_method 0 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient(hunting): handle_auth_done global_id 34435 payload 824 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 -- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1498134280 msgr2=0x7f149811ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1498134280 0x7f149811ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 -- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f149811ef10 msgr2=0x7f1498133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f149811ef10 0x7f1498133d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 monclient: found mon.noname-b 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 -- 172.21.15.119:0/669542639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14981cd8f0 con 0x7f1480004810 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14977fe640 10 monclient: get_auth_request con 0x7f1498134280 auth_method 0 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14977fe640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1498134280 0x7f149811ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f1496ffd640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f14981cd3b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1488005fd0 tx=0x7f148800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148800c8d0 con 0x7f1480004810 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: dump: 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1488018070 con 0x7f1480004810 2024-09-15T21:20:32.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148800ccb0 con 0x7f1480004810 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 5 monclient: authenticate success, global_id 34435 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _renew_subs 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149809d1d0 con 0x7f1480004810 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14957fa640 4 set_mon_vals no callback set 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:32.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 10 monclient: dump: 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:32.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _renew_subs 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1498120430 con 0x7f1480004810 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f148801d460 con 0x7f1480004810 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14981360f0 con 0x7f1480004810 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f148403b9c0 0x7f148403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:32.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6093+0+0 (secure 0 0 0) 0x7f148804fe50 con 0x7f1480004810 2024-09-15T21:20:32.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f148403b9c0 0x7f148403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:32.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 10 monclient: get_auth_request con 0x7f148403b9c0 auth_method 0 2024-09-15T21:20:32.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.931+0000 7f14967fc640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f148403b9c0 0x7f148403de80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f148c0af000 tx=0x7f148c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:32.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.935+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1488013420 con 0x7f1480004810 2024-09-15T21:20:32.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.935+0000 7f147f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:32.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:32.935+0000 7f147f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.107+0000 7f149d4bb640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.107+0000 7f149d4bb640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.107+0000 7f149d4bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.107+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f149806c440 con 0x7f1480004810 2024-09-15T21:20:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.111+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v78) ==== 56+0+1273 (secure 0 0 0) 0x7f148801bc80 con 0x7f1480004810 2024-09-15T21:20:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.111+0000 7f147f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.111+0000 7f147f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.207+0000 7f149d4bb640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.207+0000 7f149d4bb640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.207+0000 7f149d4bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.207+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f149811d730 con 0x7f1480004810 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f147f7fe640 1 -- 172.21.15.119:0/669542639 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v78) ==== 56+0+1273 (secure 0 0 0) 0x7f1488011b70 con 0x7f1480004810 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f147f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:33.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f147f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:33.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f148403b9c0 msgr2=0x7f148403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f148403b9c0 0x7f148403de80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f148c0af000 tx=0x7f148c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 10 monclient: shutdown 2024-09-15T21:20:33.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:33.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 msgr2=0x7f14981cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f14981cd3b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1488005fd0 tx=0x7f148800ec80 comp rx=0 tx=0).stop 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 shutdown_connections 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f148403b9c0 0x7f148403de80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1498134280 0x7f149811ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f149811ef10 0x7f1498133d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 --2- 172.21.15.119:0/669542639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480004810 0x7f14981cd3b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 >> 172.21.15.119:0/669542639 conn(0x7f149806cdc0 msgr2=0x7f149813a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 shutdown_connections 2024-09-15T21:20:33.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.211+0000 7f149d4bb640 1 -- 172.21.15.119:0/669542639 wait complete. 2024-09-15T21:20:33.232 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd tree-from host1 2024-09-15T21:20:33.232 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:33.232 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree-from host1 2024-09-15T21:20:33.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient: get_monmap_and_config 2024-09-15T21:20:33.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient: build_initial_monmap 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient: monmap: 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:33.346305+0000 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:33.346305+0000 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:33.348 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 1 Processor -- start 2024-09-15T21:20:33.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 1 -- start start 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient: init 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe450135b80 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4500a0030 0x7fe45013aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient(hunting): picked mon.noname-c con 0x7fe4500a0030 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe45013b020 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe45009fdb0 con 0x7fe45013b020 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4500a1b60 con 0x7fe450135b80 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe4500a1e40 con 0x7fe4500a0030 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:33.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe456db5640 10 monclient(hunting): authenticate will time out at 1152.142954s 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe454b51640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe450135b80 0x7fe450135f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46832/0 (socket says 172.21.15.119:46832) 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe454b51640 1 -- 172.21.15.119:0/2801837075 learned_addr learned my addr 172.21.15.119:0/2801837075 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 1 --2- 172.21.15.119:0/2801837075 >> v2:172.21.15.119:3300/0 conn(0x7fe4500a0030 0x7fe45013aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): get_auth_request con 0x7fe4500a0030 auth_method 0 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient(hunting): handle_auth_done global_id 34438 payload 274 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 1 -- 172.21.15.119:0/2801837075 >> v1:172.21.15.119:6789/0 conn(0x7fe450135b80 legacy=0x7fe450135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 1 -- 172.21.15.119:0/2801837075 >> v1:172.21.15.17:6789/0 conn(0x7fe45013b020 legacy=0x7fe45013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:33.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 1 monclient: found mon.noname-c 2024-09-15T21:20:33.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:33.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.343+0000 7fe44ffff640 1 -- 172.21.15.119:0/2801837075 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500a2120 con 0x7fe4500a0030 2024-09-15T21:20:33.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44ffff640 1 --2- 172.21.15.119:0/2801837075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4500a0030 0x7fe45013aae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097c0 tx=0x7fe43c01ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c0c9f09f121c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 1 -- 172.21.15.119:0/2801837075 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0b0070 con 0x7fe4500a0030 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: dump: 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.360 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 1 -- 172.21.15.119:0/2801837075 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c0ab600 con 0x7fe4500a0030 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 5 monclient: authenticate success, global_id 34438 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 1 -- 172.21.15.119:0/2801837075 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0ab9d0 con 0x7fe4500a0030 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe44f7fe640 10 monclient: dump: 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 4 set_mon_vals no callback set 2024-09-15T21:20:33.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 10 monclient: shutdown 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 -- 172.21.15.119:0/2801837075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4500a0030 msgr2=0x7fe45013aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 --2- 172.21.15.119:0/2801837075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4500a0030 0x7fe45013aae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097c0 tx=0x7fe43c01ce50 comp rx=0 tx=0).stop 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 -- 172.21.15.119:0/2801837075 shutdown_connections 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 --2- 172.21.15.119:0/2801837075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4500a0030 0x7fe45013aae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 -- 172.21.15.119:0/2801837075 >> 172.21.15.119:0/2801837075 conn(0x7fe45006cdc0 msgr2=0x7fe45006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 -- 172.21.15.119:0/2801837075 shutdown_connections 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 -- 172.21.15.119:0/2801837075 wait complete. 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 10 monclient: build_initial_monmap 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 10 monclient: monmap: 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:33.363 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 Processor -- start 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 1 -- start start 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.347+0000 7fe456db5640 10 monclient: init 2024-09-15T21:20:33.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:33.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:33.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4500a0030 0x7fe450134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): picked mon.noname-c con 0x7fe4500a0030 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe450135b80 0x7fe4501350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): picked mon.noname-b con 0x7fe450135b80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe45013b020 0x7fe450138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): picked mon.noname-a con 0x7fe45013b020 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe450141760 con 0x7fe45013b020 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe45006dcb0 con 0x7fe450135b80 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe4501418e0 con 0x7fe4500a0030 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient(hunting): authenticate will time out at 1152.148388s 2024-09-15T21:20:33.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe450135b80 0x7fe4501350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe450135b80 0x7fe4501350c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39944/0 (socket says 172.21.15.119:39944) 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 -- 172.21.15.119:0/3181939271 learned_addr learned my addr 172.21.15.119:0/3181939271 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): get_auth_request con 0x7fe450135b80 auth_method 0 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe455352640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe45013b020 0x7fe450138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient(hunting): handle_auth_done global_id 34444 payload 824 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe454b51640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4500a0030 0x7fe450134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 -- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4500a0030 msgr2=0x7fe450134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe4500a0030 0x7fe450134b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 -- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe45013b020 msgr2=0x7fe450138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe45013b020 0x7fe450138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 monclient: found mon.noname-b 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 -- 172.21.15.119:0/3181939271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45009d1d0 con 0x7fe450135b80 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe455352640 10 monclient: get_auth_request con 0x7fe45013b020 auth_method 0 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe455352640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe45013b020 0x7fe450138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44ffff640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe450135b80 0x7fe4501350c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe43c019000 tx=0x7fe43c019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0b0050 con 0x7fe450135b80 2024-09-15T21:20:33.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: dump: 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c0bc070 con 0x7fe450135b80 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 5 monclient: authenticate success, global_id 34444 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _renew_subs 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 -- 172.21.15.119:0/3181939271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe450073da0 con 0x7fe450135b80 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0b5a10 con 0x7fe450135b80 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 10 monclient: dump: 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44effd640 4 set_mon_vals no callback set 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _renew_subs 2024-09-15T21:20:33.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 -- 172.21.15.119:0/3181939271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45009f1f0 con 0x7fe450135b80 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe43c0ab4c0 con 0x7fe450135b80 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe456db5640 1 -- 172.21.15.119:0/3181939271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe450135610 con 0x7fe450135b80 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe43403b9c0 0x7fe43403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6093+0+0 (secure 0 0 0) 0x7fe43c0ea9e0 con 0x7fe450135b80 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe454b51640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe43403b9c0 0x7fe43403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe454b51640 10 monclient: get_auth_request con 0x7fe43403b9c0 auth_method 0 2024-09-15T21:20:33.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.351+0000 7fe454b51640 1 --2- 172.21.15.119:0/3181939271 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe43403b9c0 0x7fe43403de80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe444000f70 tx=0x7fe444009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:33.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.355+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe43c016070 con 0x7fe450135b80 2024-09-15T21:20:33.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.355+0000 7fe44d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:33.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.355+0000 7fe44d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13f002e5d0 auth_method 0 2024-09-15T21:20:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:33.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:33.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:33.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:33.417+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:20:33.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe456db5640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host1"}] 2024-09-15T21:20:33.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe456db5640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-09-15T21:20:33.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe456db5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe456db5640 1 -- 172.21.15.119:0/3181939271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host1"} v 0) -- 0x7fe45013ae00 con 0x7fe450135b80 2024-09-15T21:20:33.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe44d7fa640 1 -- 172.21.15.119:0/3181939271 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host1"}]=-2 bucket 'host1' does not exist v78) ==== 109+0+0 (secure 0 0 0) 0x7fe43c0c1dc0 con 0x7fe450135b80 2024-09-15T21:20:33.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe44d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-09-15T21:20:33.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.515+0000 7fe44d7fa640 10 monclient: _finish_command 2 = mon:2 bucket 'host1' does not exist 2024-09-15T21:20:33.520 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOENT: bucket 'host1' does not exist 2024-09-15T21:20:33.524 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:33.524 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree-from host2 2024-09-15T21:20:33.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.635+0000 7f17102bc640 10 monclient: get_monmap_and_config 2024-09-15T21:20:33.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.635+0000 7f17102bc640 10 monclient: build_initial_monmap 2024-09-15T21:20:33.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.635+0000 7f17102bc640 10 monclient: monmap: 2024-09-15T21:20:33.640 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:33.638911+0000 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:33.638911+0000 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:33.641 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.635+0000 7f17102bc640 1 Processor -- start 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.635+0000 7f17102bc640 1 -- start start 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.635+0000 7f17102bc640 10 monclient: init 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1708135b80 0x7f1708135f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient(hunting): picked mon.noname-a con 0x7f1708135b80 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f17081364a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:33.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f17080a0030 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17080a1800 con 0x7f17080a0030 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170809fdb0 con 0x7f17081364a0 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f17080a1b60 con 0x7f1708135b80 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 10 monclient(hunting): authenticate will time out at 1152.436493s 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f17081364a0 0x7f17080a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46848/0 (socket says 172.21.15.119:46848) 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 1 -- 172.21.15.119:0/2620588303 learned_addr learned my addr 172.21.15.119:0/2620588303 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 --2- 172.21.15.119:0/2620588303 >> v2:172.21.15.17:3300/0 conn(0x7f1708135b80 0x7f1708135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243963135 0 0) 0x7f170809fdb0 con 0x7f17081364a0 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): _negotiate 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:33.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): my global_id is 34447 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f4003530 con 0x7f17081364a0 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): get_auth_request con 0x7f1708135b80 auth_method 0 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3219931545 0 0) 0x7f16f4003530 con 0x7f17081364a0 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 >> v2:172.21.15.17:3300/0 conn(0x7f1708135b80 msgr2=0x7f1708135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 --2- 172.21.15.119:0/2620588303 >> v2:172.21.15.17:3300/0 conn(0x7f1708135b80 0x7f1708135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 >> v1:172.21.15.17:6789/0 conn(0x7f17080a0030 legacy=0x7f17080a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: _finish_hunting 0 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17080a1e40 con 0x7f17081364a0 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: _finish_auth 0 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: _check_auth_tickets 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f16f8001700 con 0x7f17081364a0 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: dump: 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.648 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 >> v1:172.21.15.119:6789/0 conn(0x7f17081364a0 legacy=0x7f17080a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f16f4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): picked mon.b con 0x7f16f4003ab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): picked mon.a con 0x7f16f4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f16f40053d0 0x7f16f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): picked mon.c con 0x7f16f40053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f17080a1b60 con 0x7f16f4004810 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16f400c290 con 0x7f16f4003ab0 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f16f400c490 con 0x7f16f40053d0 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d056640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:33.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f16f4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): get_auth_request con 0x7f16f4003ab0 auth_method 0 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): get_auth_request con 0x7f16f4004810 auth_method 0 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e859640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f16f40053d0 0x7f16f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e859640 10 monclient(hunting): get_auth_request con 0x7f16f40053d0 auth_method 0 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e859640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e859640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e859640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170d857640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient(hunting): handle_auth_done global_id 34447 payload 293 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 -- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f16f40053d0 msgr2=0x7f16f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f16f40053d0 0x7f16f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 -- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f4004810 msgr2=0x7f16f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient: _finish_hunting 0 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 monclient: found mon.b 2024-09-15T21:20:33.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 1 -- 172.21.15.119:0/2620588303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f4003530 con 0x7f16f4003ab0 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e859640 1 -- 172.21.15.119:0/2620588303 reap_dead start 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient: _finish_auth 0 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 10 monclient: _check_auth_tickets 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f170e058640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 5 monclient: authenticate success, global_id 34447 2024-09-15T21:20:33.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.639+0000 7f17102bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:33.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f16f4003ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ae0 tx=0x7f170401ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f801572e600d0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:33.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040b2070 con 0x7f16f4003ab0 2024-09-15T21:20:33.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: dump: 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17040a5530 con 0x7f16f4003ab0 2024-09-15T21:20:33.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 1 -- 172.21.15.119:0/2620588303 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040096b0 con 0x7f16f4003ab0 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 4 set_mon_vals no callback set 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d056640 10 monclient: dump: 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.660 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: shutdown 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/2620588303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 msgr2=0x7f16f4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f16f4003ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ae0 tx=0x7f170401ca50 comp rx=0 tx=0).stop 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/2620588303 shutdown_connections 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 --2- 172.21.15.119:0/2620588303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f16f4003ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/2620588303 >> 172.21.15.119:0/2620588303 conn(0x7f170806cdc0 msgr2=0x7f170806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/2620588303 shutdown_connections 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/2620588303 wait complete. 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: build_initial_monmap 2024-09-15T21:20:33.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: monmap: 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 Processor -- start 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- start start 2024-09-15T21:20:33.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: init 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f17081cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): picked mon.noname-b con 0x7f16f4003ab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17081ce290 0x7f1708123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): picked mon.noname-c con 0x7f17081ce290 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17081ce6f0 0x7f1708123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): picked mon.noname-a con 0x7f17081ce6f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f170806dcc0 con 0x7f17081ce6f0 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f170806db40 con 0x7f16f4003ab0 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:33.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f17080a11f0 con 0x7f17081ce290 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient(hunting): authenticate will time out at 1152.442447s 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f17081cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f17081cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39964/0 (socket says 172.21.15.119:39964) 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 -- 172.21.15.119:0/1540157699 learned_addr learned my addr 172.21.15.119:0/1540157699 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): get_auth_request con 0x7f16f4003ab0 auth_method 0 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:33.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient(hunting): handle_auth_done global_id 34450 payload 824 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e859640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17081ce6f0 0x7f1708123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 -- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17081ce290 msgr2=0x7f1708123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170d857640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17081ce290 0x7f1708123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17081ce290 0x7f1708123030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 -- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17081ce6f0 msgr2=0x7f1708123570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17081ce6f0 0x7f1708123570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient: _finish_hunting 0 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 monclient: found mon.noname-b 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 -- 172.21.15.119:0/1540157699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1708126e50 con 0x7f16f4003ab0 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e859640 10 monclient: get_auth_request con 0x7f17081ce6f0 auth_method 0 2024-09-15T21:20:33.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e859640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17081ce6f0 0x7f1708123570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170e058640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f17081cdd50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f170401c880 tx=0x7f17040197f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040b2040 con 0x7f16f4003ab0 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: dump: 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.666 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17040a5e00 con 0x7f16f4003ab0 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 5 monclient: authenticate success, global_id 34450 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: _renew_subs 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170809d1d0 con 0x7f16f4003ab0 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704004020 con 0x7f16f4003ab0 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:33.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f16feffd640 10 monclient: dump: 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f170c855640 4 set_mon_vals no callback set 2024-09-15T21:20:33.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: _renew_subs 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.643+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1708126b70 con 0x7f16f4003ab0 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f17102bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f17102bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f17102bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1708138ad0 con 0x7f16f4003ab0 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f17040bf030 con 0x7f16f4003ab0 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f16feffd640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f16ec03b9c0 0x7f16ec03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6093+0+0 (secure 0 0 0) 0x7f17040e9a10 con 0x7f16f4003ab0 2024-09-15T21:20:33.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f170d857640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f16ec03b9c0 0x7f16ec03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:33.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f170d857640 10 monclient: get_auth_request con 0x7f16ec03b9c0 auth_method 0 2024-09-15T21:20:33.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.647+0000 7f170d857640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f16ec03b9c0 0x7f16ec03de80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16f8001170 tx=0x7f16f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:33.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.651+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f17040c6060 con 0x7f16f4003ab0 2024-09-15T21:20:33.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.651+0000 7f16feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:33.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.651+0000 7f16feffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:33.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.815+0000 7f17102bc640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T21:20:33.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.815+0000 7f17102bc640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T21:20:33.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.815+0000 7f17102bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.815+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) -- 0x7f170806f660 con 0x7f16f4003ab0 2024-09-15T21:20:33.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.819+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v78) ==== 80+0+177 (secure 0 0 0) 0x7f170806f660 con 0x7f16f4003ab0 2024-09-15T21:20:33.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.819+0000 7f16feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T21:20:33.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.819+0000 7f16feffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:33.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.907+0000 7f17102bc640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T21:20:33.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.907+0000 7f17102bc640 10 monclient: _send_command 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T21:20:33.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.907+0000 7f17102bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:33.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.907+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) -- 0x7f1708125650 con 0x7f16f4003ab0 2024-09-15T21:20:33.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f16feffd640 1 -- 172.21.15.119:0/1540157699 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v78) ==== 80+0+177 (secure 0 0 0) 0x7f1704016760 con 0x7f16f4003ab0 2024-09-15T21:20:33.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f16feffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T21:20:33.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f16feffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:33.915 INFO:tasks.workunit.client.0.smithi119.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2024-09-15T21:20:33.915 INFO:tasks.workunit.client.0.smithi119.stdout:-16 1.00000 host host2 2024-09-15T21:20:33.915 INFO:tasks.workunit.client.0.smithi119.stdout: 9 1.00000 osd.9 down 1.00000 1.00000 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f16ec03b9c0 msgr2=0x7f16ec03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f16ec03b9c0 0x7f16ec03de80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16f8001170 tx=0x7f16f8009040 comp rx=0 tx=0).stop 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 10 monclient: shutdown 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 msgr2=0x7f17081cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f17081cdd50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f170401c880 tx=0x7f17040197f0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 shutdown_connections 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f16ec03b9c0 0x7f16ec03de80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17081ce6f0 0x7f1708123570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17081ce290 0x7f1708123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 --2- 172.21.15.119:0/1540157699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f4003ab0 0x7f17081cdd50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 >> 172.21.15.119:0/1540157699 conn(0x7f170806cdc0 msgr2=0x7f170806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 shutdown_connections 2024-09-15T21:20:33.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:33.911+0000 7f17102bc640 1 -- 172.21.15.119:0/1540157699 wait complete. 2024-09-15T21:20:33.931 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd tree-from osd.9 2024-09-15T21:20:33.931 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:33.931 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree-from osd.9 2024-09-15T21:20:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient: build_initial_monmap 2024-09-15T21:20:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient: monmap: 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:34.041177+0000 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:34.041177+0000 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:34.043 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 Processor -- start 2024-09-15T21:20:34.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 -- start start 2024-09-15T21:20:34.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient: init 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc69c09d4e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09f960 0x7fc69c139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): picked mon.noname-b con 0x7fc69c09f960 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc69c09dfd0 0x7fc69c13c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): picked mon.noname-c con 0x7fc69c09dfd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c09ef80 con 0x7fc69c09d4e0 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c06da70 con 0x7fc69c09dfd0 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0746f0 con 0x7fc69c09f960 2024-09-15T21:20:34.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a352f640 10 monclient(hunting): authenticate will time out at 1152.837847s 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a12cb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc69c09d4e0 0x7fc69c09d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46852/0 (socket says 172.21.15.119:46852) 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a12cb640 1 -- 172.21.15.119:0/203185292 learned_addr learned my addr 172.21.15.119:0/203185292 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 1 --2- 172.21.15.119:0/203185292 >> v2:172.21.15.119:3300/0 conn(0x7fc69c09dfd0 0x7fc69c13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): get_auth_request con 0x7fc69c09dfd0 auth_method 0 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a0aca640 1 --2- 172.21.15.119:0/203185292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09f960 0x7fc69c139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient(hunting): handle_auth_done global_id 34453 payload 274 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 1 -- 172.21.15.119:0/203185292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09f960 msgr2=0x7fc69c139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 1 --2- 172.21.15.119:0/203185292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09f960 0x7fc69c139c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 1 -- 172.21.15.119:0/203185292 >> v1:172.21.15.119:6789/0 conn(0x7fc69c09d4e0 legacy=0x7fc69c09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 1 monclient: found mon.noname-c 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.039+0000 7fc6a1acc640 1 -- 172.21.15.119:0/203185292 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c09e6e0 con 0x7fc69c09dfd0 2024-09-15T21:20:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a1acc640 1 --2- 172.21.15.119:0/203185292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09dfd0 0x7fc69c13c150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc6940097c0 tx=0x7fc69401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7719e6ea2ea881b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 1 -- 172.21.15.119:0/203185292 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6940b0070 con 0x7fc69c09dfd0 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: dump: 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 1 -- 172.21.15.119:0/203185292 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc694004230 con 0x7fc69c09dfd0 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 5 monclient: authenticate success, global_id 34453 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 1 -- 172.21.15.119:0/203185292 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694003620 con 0x7fc69c09dfd0 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 4 set_mon_vals no callback set 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc693fff640 10 monclient: dump: 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: shutdown 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/203185292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09dfd0 msgr2=0x7fc69c13c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 --2- 172.21.15.119:0/203185292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09dfd0 0x7fc69c13c150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc6940097c0 tx=0x7fc69401c330 comp rx=0 tx=0).stop 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/203185292 shutdown_connections 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 --2- 172.21.15.119:0/203185292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09dfd0 0x7fc69c13c150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 --2- 172.21.15.119:0/203185292 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09f960 0x7fc69c139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/203185292 >> 172.21.15.119:0/203185292 conn(0x7fc69c06cdc0 msgr2=0x7fc69c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/203185292 shutdown_connections 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/203185292 wait complete. 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: build_initial_monmap 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: monmap: 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 Processor -- start 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- start start 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: init 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09d4e0 0x7fc69c1309d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): picked mon.noname-b con 0x7fc69c09d4e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09dfd0 0x7fc69c130f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): picked mon.noname-c con 0x7fc69c09dfd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc69c09f960 0x7fc69c135e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): picked mon.noname-a con 0x7fc69c09f960 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c13eeb0 con 0x7fc69c09f960 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0a26d0 con 0x7fc69c09d4e0 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c13f030 con 0x7fc69c09dfd0 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient(hunting): authenticate will time out at 1152.842826s 2024-09-15T21:20:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09d4e0 0x7fc69c1309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09d4e0 0x7fc69c1309d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39984/0 (socket says 172.21.15.119:39984) 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 -- 172.21.15.119:0/2730196015 learned_addr learned my addr 172.21.15.119:0/2730196015 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): get_auth_request con 0x7fc69c09d4e0 auth_method 0 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a0aca640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09dfd0 0x7fc69c130f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient(hunting): handle_auth_done global_id 34459 payload 824 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a1acc640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc69c09f960 0x7fc69c135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 -- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09dfd0 msgr2=0x7fc69c130f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09dfd0 0x7fc69c130f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 -- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc69c09f960 msgr2=0x7fc69c135e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc69c09f960 0x7fc69c135e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 monclient: found mon.noname-b 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 -- 172.21.15.119:0/2730196015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c09d1d0 con 0x7fc69c09d4e0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a0aca640 10 monclient: get_auth_request con 0x7fc69c09dfd0 auth_method 0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a0aca640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc69c09dfd0 0x7fc69c130f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a12cb640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c09d4e0 0x7fc69c1309d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c400 tx=0x7fc68c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c017070 con 0x7fc69c09d4e0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: dump: 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c005cf0 con 0x7fc69c09d4e0 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 5 monclient: authenticate success, global_id 34459 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: _renew_subs 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/2730196015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c13c260 con 0x7fc69c09d4e0 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0072d0 con 0x7fc69c09d4e0 2024-09-15T21:20:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc691ffb640 10 monclient: dump: 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.063 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: _renew_subs 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6937fe640 4 set_mon_vals no callback set 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.043+0000 7fc6a352f640 1 -- 172.21.15.119:0/2730196015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c138130 con 0x7fc69c09d4e0 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc68c01a070 con 0x7fc69c09d4e0 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a352f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a352f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a352f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a352f640 1 -- 172.21.15.119:0/2730196015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c09d8c0 con 0x7fc69c09d4e0 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc691ffb640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc68003b9c0 0x7fc68003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6093+0+0 (secure 0 0 0) 0x7fc68c04f450 con 0x7fc69c09d4e0 2024-09-15T21:20:34.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a0aca640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc68003b9c0 0x7fc68003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a0aca640 10 monclient: get_auth_request con 0x7fc68003b9c0 auth_method 0 2024-09-15T21:20:34.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.047+0000 7fc6a0aca640 1 --2- 172.21.15.119:0/2730196015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc68003b9c0 0x7fc68003de80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc69c138f40 tx=0x7fc6980074a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.051+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc68c012400 con 0x7fc69c09d4e0 2024-09-15T21:20:34.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.051+0000 7fc691ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.051+0000 7fc691ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:34.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc6a352f640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-09-15T21:20:34.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc6a352f640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-09-15T21:20:34.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc6a352f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc6a352f640 1 -- 172.21.15.119:0/2730196015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "osd.9"} v 0) -- 0x7fc69c1397e0 con 0x7fc69c09d4e0 2024-09-15T21:20:34.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc691ffb640 1 -- 172.21.15.119:0/2730196015 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "osd.9"}]=-22 "osd.9" is not a bucket v78) ==== 103+0+0 (secure 0 0 0) 0x7fc68c04d870 con 0x7fc69c09d4e0 2024-09-15T21:20:34.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc691ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-09-15T21:20:34.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.207+0000 7fc691ffb640 10 monclient: _finish_command 2 = mon:22 "osd.9" is not a bucket 2024-09-15T21:20:34.214 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: "osd.9" is not a bucket 2024-09-15T21:20:34.219 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:34.219 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush rm bar 2024-09-15T21:20:34.219 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:34.219 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm bar 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient: get_monmap_and_config 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient: build_initial_monmap 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient: monmap: 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:34.321969+0000 2024-09-15T21:20:34.323 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:34.321969+0000 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:34.324 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 1 Processor -- start 2024-09-15T21:20:34.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 1 -- start start 2024-09-15T21:20:34.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient: init 2024-09-15T21:20:34.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:20:34.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f79480a4ed0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f79480a59c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 0x7f7948140600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient(hunting): picked mon.noname-b con 0x7f79480a5dd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79480a6db0 con 0x7f79480a4ed0 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79480a6ad0 con 0x7f79480a59c0 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f79480a1210 con 0x7f79480a5dd0 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 10 monclient(hunting): authenticate will time out at 1153.119054s 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f7947fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f79480a59c0 0x7f794813ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46860/0 (socket says 172.21.15.119:46860) 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f7947fff640 1 -- 172.21.15.119:0/4226999308 learned_addr learned my addr 172.21.15.119:0/4226999308 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655604512 0 0) 0x7f79480a6ad0 con 0x7f79480a59c0 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): _negotiate 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): my global_id is 34462 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7938003530 con 0x7f79480a59c0 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794d019640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 0x7f7948140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794d019640 10 monclient(hunting): get_auth_request con 0x7f79480a5dd0 auth_method 0 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794d019640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794d019640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794d019640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 709629712 0 0) 0x7f7938003530 con 0x7f79480a59c0 2024-09-15T21:20:34.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 msgr2=0x7f7948140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 0x7f7948140600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 >> v1:172.21.15.17:6789/0 conn(0x7f79480a4ed0 legacy=0x7f79480a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79480a1450 con 0x7f79480a59c0 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7934001700 con 0x7f79480a59c0 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 5 monclient: authenticate success, global_id 34462 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:34.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: dump: 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.337 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 >> v1:172.21.15.119:6789/0 conn(0x7f79480a59c0 legacy=0x7f794813ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7938003a10 0x7f7938003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): picked mon.a con 0x7f7938003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 0x7f7938004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): picked mon.c con 0x7f79480a5dd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f793800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): picked mon.b con 0x7f7938004e70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7938005630 con 0x7f7938003a10 2024-09-15T21:20:34.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f793800bf10 con 0x7f7938004e70 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f793800c090 con 0x7f79480a5dd0 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f79477fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.319+0000 7f794ea7c640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f793800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): get_auth_request con 0x7f7938004e70 auth_method 0 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7938003a10 0x7f7938003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f7947fff640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 0x7f7938004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): get_auth_request con 0x7f7938003a10 auth_method 0 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f7947fff640 10 monclient(hunting): get_auth_request con 0x7f79480a5dd0 auth_method 0 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f7947fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f7947fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f7947fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:34.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794c818640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient(hunting): handle_auth_done global_id 34462 payload 293 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 -- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 msgr2=0x7f7938004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79480a5dd0 0x7f7938004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 -- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7938003a10 msgr2=0x7f7938003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7938003a10 0x7f7938003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 monclient: found mon.b 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 -- 172.21.15.119:0/4226999308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7938003530 con 0x7f7938004e70 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f793800b9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7940007330 tx=0x7f794001ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c4e180624e4486 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79400b2070 con 0x7f7938004e70 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: dump: 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.342 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79400a5bf0 con 0x7f7938004e70 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 1 -- 172.21.15.119:0/4226999308 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940009be0 con 0x7f7938004e70 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 4 set_mon_vals no callback set 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f79477fe640 10 monclient: dump: 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 10 monclient: shutdown 2024-09-15T21:20:34.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 -- 172.21.15.119:0/4226999308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 msgr2=0x7f793800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 --2- 172.21.15.119:0/4226999308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f793800b9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7940007330 tx=0x7f794001ce00 comp rx=0 tx=0).stop 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794d019640 1 -- 172.21.15.119:0/4226999308 reap_dead start 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 -- 172.21.15.119:0/4226999308 shutdown_connections 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 -- 172.21.15.119:0/4226999308 >> 172.21.15.119:0/4226999308 conn(0x7f79480a6fd0 msgr2=0x7f79480a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 -- 172.21.15.119:0/4226999308 shutdown_connections 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 -- 172.21.15.119:0/4226999308 wait complete. 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 10 monclient: build_initial_monmap 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 10 monclient: monmap: 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:34.345 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.323+0000 7f794ea7c640 1 Processor -- start 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- start start 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: init 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f7948073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): picked mon.noname-b con 0x7f7938004e70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7948138400 0x7f7948138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): picked mon.noname-c con 0x7f7948138400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7948120b20 0x7f7948120f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): picked mon.noname-a con 0x7f7948120b20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7948140f90 con 0x7f7948120b20 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7948140e10 con 0x7f7938004e70 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7948141110 con 0x7f7948138400 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient(hunting): authenticate will time out at 1153.124486s 2024-09-15T21:20:34.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f7948073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f7948073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40004/0 (socket says 172.21.15.119:40004) 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 -- 172.21.15.119:0/3806858283 learned_addr learned my addr 172.21.15.119:0/3806858283 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): get_auth_request con 0x7f7938004e70 auth_method 0 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f7947fff640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7948138400 0x7f7948138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient(hunting): handle_auth_done global_id 34465 payload 824 2024-09-15T21:20:34.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794d019640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7948120b20 0x7f7948120f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 -- 172.21.15.119:0/3806858283 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7948138400 msgr2=0x7f7948138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7948138400 0x7f7948138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 -- 172.21.15.119:0/3806858283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7948120b20 msgr2=0x7f7948120f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7948120b20 0x7f7948120f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 monclient: found mon.noname-b 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 -- 172.21.15.119:0/3806858283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79480a9b10 con 0x7f7938004e70 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794c818640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7938004e70 0x7f7948073ae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f793c00f7f0 tx=0x7f793c00fcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0050d0 con 0x7f7938004e70 2024-09-15T21:20:34.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: dump: 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.350 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793c018070 con 0x7f7938004e70 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 5 monclient: authenticate success, global_id 34465 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _renew_subs 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- 172.21.15.119:0/3806858283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7948123ae0 con 0x7f7938004e70 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00d730 con 0x7f7938004e70 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 10 monclient: dump: 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _renew_subs 2024-09-15T21:20:34.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- 172.21.15.119:0/3806858283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79480a16c0 con 0x7f7938004e70 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f7946ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f794ea7c640 1 -- 172.21.15.119:0/3806858283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79480a52b0 con 0x7f7938004e70 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f793c00d8d0 con 0x7f7938004e70 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f792c03b9c0 0x7f792c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6093+0+0 (secure 0 0 0) 0x7f793c04fe60 con 0x7f7938004e70 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f7947fff640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f792c03b9c0 0x7f792c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f7947fff640 10 monclient: get_auth_request con 0x7f792c03b9c0 auth_method 0 2024-09-15T21:20:34.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.327+0000 7f7947fff640 1 --2- 172.21.15.119:0/3806858283 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f792c03b9c0 0x7f792c03de80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7934001250 tx=0x7f7934001db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.331+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f79480a9b10 con 0x7f7938004e70 2024-09-15T21:20:34.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.331+0000 7f79457fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.331+0000 7f79457fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:34.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.495+0000 7f794ea7c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:34.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.495+0000 7f794ea7c640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:34.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.495+0000 7f794ea7c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.495+0000 7f794ea7c640 1 -- 172.21.15.119:0/3806858283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7f79481d0050 con 0x7f7938004e70 2024-09-15T21:20:34.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.499+0000 7f79457fa640 1 -- 172.21.15.119:0/3806858283 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v79) ==== 99+0+0 (secure 0 0 0) 0x7f793c00dda0 con 0x7f7938004e70 2024-09-15T21:20:34.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.499+0000 7f79457fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:34.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.499+0000 7f79457fa640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T21:20:34.503 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T21:20:34.507 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:34.508 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush unlink host2 2024-09-15T21:20:34.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:34.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient: build_initial_monmap 2024-09-15T21:20:34.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient: monmap: 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:34.610116+0000 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:34.610116+0000 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:34.612 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:34.613 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:34.613 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:34.613 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 Processor -- start 2024-09-15T21:20:34.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 -- start start 2024-09-15T21:20:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient: init 2024-09-15T21:20:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:20:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c137bc0 0x7f406c137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): picked mon.noname-b con 0x7f406c137bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f406c0a09e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f406c0a0ea0 0x7f406c073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): picked mon.noname-c con 0x7f406c0a0ea0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c09cfa0 con 0x7f406c0a09e0 2024-09-15T21:20:34.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f406c09cd20 con 0x7f406c0a0ea0 2024-09-15T21:20:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f406c0a1820 con 0x7f406c137bc0 2024-09-15T21:20:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f407166f640 10 monclient(hunting): authenticate will time out at 1153.406699s 2024-09-15T21:20:34.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f406c0a0ea0 0x7f406c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f406c0a0ea0 0x7f406c073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40016/0 (socket says 172.21.15.119:40016) 2024-09-15T21:20:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 -- 172.21.15.119:0/1601949775 learned_addr learned my addr 172.21.15.119:0/1601949775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): get_auth_request con 0x7f406c0a0ea0 auth_method 0 2024-09-15T21:20:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406a7fc640 1 --1- 172.21.15.119:0/1601949775 >> v1:172.21.15.119:6789/0 conn(0x7f406c0a09e0 0x7f406c073040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46870/0 (socket says 172.21.15.119:46870) 2024-09-15T21:20:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406affd640 1 --2- 172.21.15.119:0/1601949775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c137bc0 0x7f406c137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406affd640 10 monclient(hunting): get_auth_request con 0x7f406c137bc0 auth_method 0 2024-09-15T21:20:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient(hunting): handle_auth_done global_id 34468 payload 274 2024-09-15T21:20:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 -- 172.21.15.119:0/1601949775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c137bc0 msgr2=0x7f406c137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 --2- 172.21.15.119:0/1601949775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c137bc0 0x7f406c137fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 -- 172.21.15.119:0/1601949775 >> v1:172.21.15.119:6789/0 conn(0x7f406c0a09e0 legacy=0x7f406c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.607+0000 7f406b7fe640 1 -- 172.21.15.119:0/1601949775 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0a1b00 con 0x7f406c0a0ea0 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406b7fe640 1 --2- 172.21.15.119:0/1601949775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a0ea0 0x7f406c073750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4064009870 tx=0x7f406401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b53ccb347fa6aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 1 -- 172.21.15.119:0/1601949775 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640b1070 con 0x7f406c0a0ea0 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: dump: 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.633 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 1 -- 172.21.15.119:0/1601949775 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40640ac640 con 0x7f406c0a0ea0 2024-09-15T21:20:34.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 5 monclient: authenticate success, global_id 34468 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 1 -- 172.21.15.119:0/1601949775 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640aca10 con 0x7f406c0a0ea0 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 4 set_mon_vals no callback set 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f4069ffb640 10 monclient: dump: 2024-09-15T21:20:34.636 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.636 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.636 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: shutdown 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- 172.21.15.119:0/1601949775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a0ea0 msgr2=0x7f406c073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 --2- 172.21.15.119:0/1601949775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a0ea0 0x7f406c073750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4064009870 tx=0x7f406401c510 comp rx=0 tx=0).stop 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- 172.21.15.119:0/1601949775 shutdown_connections 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 --2- 172.21.15.119:0/1601949775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a0ea0 0x7f406c073750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 --2- 172.21.15.119:0/1601949775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c137bc0 0x7f406c137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- 172.21.15.119:0/1601949775 >> 172.21.15.119:0/1601949775 conn(0x7f406c0a2ae0 msgr2=0x7f406c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- 172.21.15.119:0/1601949775 shutdown_connections 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- 172.21.15.119:0/1601949775 wait complete. 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: build_initial_monmap 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: monmap: 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:34.638 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 Processor -- start 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- start start 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: init 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:34.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 0x7f406c11f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): picked mon.noname-b con 0x7f406c0a09e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c0a0ea0 0x7f406c1366e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): picked mon.noname-c con 0x7f406c0a0ea0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c137bc0 0x7f406c11d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): picked mon.noname-a con 0x7f406c137bc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f406c073e10 con 0x7f406c137bc0 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f406c073c90 con 0x7f406c0a09e0 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f406c1422d0 con 0x7f406c0a0ea0 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:34.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f407166f640 10 monclient(hunting): authenticate will time out at 1153.411444s 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 0x7f406c11f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 0x7f406c11f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40022/0 (socket says 172.21.15.119:40022) 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 -- 172.21.15.119:0/741964302 learned_addr learned my addr 172.21.15.119:0/741964302 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): get_auth_request con 0x7f406c0a09e0 auth_method 0 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:34.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient(hunting): handle_auth_done global_id 34474 payload 824 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 -- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c0a0ea0 msgr2=0x7f406c1366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c0a0ea0 0x7f406c1366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 -- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c137bc0 msgr2=0x7f406c11d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406b7fe640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c137bc0 0x7f406c11d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c137bc0 0x7f406c11d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 monclient: found mon.noname-b 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.611+0000 7f406affd640 1 -- 172.21.15.119:0/741964302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c120200 con 0x7f406c0a09e0 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f406affd640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 0x7f406c11f870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f406000d800 tx=0x7f406000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000b6c0 con 0x7f406c0a09e0 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: dump: 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4060019070 con 0x7f406c0a09e0 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000ba60 con 0x7f406c0a09e0 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 5 monclient: authenticate success, global_id 34474 2024-09-15T21:20:34.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 monclient: _renew_subs 2024-09-15T21:20:34.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c0a5860 con 0x7f406c0a09e0 2024-09-15T21:20:34.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:34.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:34.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 10 monclient: dump: 2024-09-15T21:20:34.645 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:34.646 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:34.646 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.646 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:34.646 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:34.646 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:34.646 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:34.647 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:34.647 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:34.647 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:34.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40697fa640 4 set_mon_vals no callback set 2024-09-15T21:20:34.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 monclient: _renew_subs 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c120d90 con 0x7f406c0a09e0 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c120940 con 0x7f406c0a09e0 2024-09-15T21:20:34.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f406000bc00 con 0x7f406c0a09e0 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405803b9f0 0x7f405803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6093+0+0 (secure 0 0 0) 0x7f40600504c0 con 0x7f406c0a09e0 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f406a7fc640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405803b9f0 0x7f405803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f406a7fc640 10 monclient: get_auth_request con 0x7f405803b9f0 auth_method 0 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.615+0000 7f406a7fc640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405803b9f0 0x7f405803deb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f406c1377c0 tx=0x7f405c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.619+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f406c120940 con 0x7f406c0a09e0 2024-09-15T21:20:34.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.619+0000 7f40537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:34.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.619+0000 7f40537fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:34.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.783+0000 7f407166f640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T21:20:34.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.783+0000 7f407166f640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T21:20:34.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.783+0000 7f407166f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:34.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:34.783+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) -- 0x7f406c06c460 con 0x7f406c0a09e0 2024-09-15T21:20:35.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.005+0000 7f13e7fff640 10 monclient: tick 2024-09-15T21:20:35.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.005+0000 7f13e7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:35.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.005+0000 7f13e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:35.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.005+0000 7f13e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13f002e5d0 auth_method 0 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:35.021+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-15T21:20:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.703+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v80) ==== 128+0+0 (secure 0 0 0) 0x7f406001cd70 con 0x7f406c0a09e0 2024-09-15T21:20:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.703+0000 7f40537fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T21:20:35.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.703+0000 7f40537fe640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-09-15T21:20:35.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.806+0000 7f407166f640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T21:20:35.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.806+0000 7f407166f640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T21:20:35.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.806+0000 7f407166f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:35.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:35.806+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) -- 0x7f406c1c73b0 con 0x7f406c0a09e0 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 10 monclient: tick 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 5 monclient: _check_tell_commands timeout tell command 1 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 msgr2=0x7f13f0063e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f002e5d0 0x7f13f0063e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13e7fff640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f13c80060e0 con 0x7f13c80035a0 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13c80035a0 auth_method 0 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 msgr2=0x7f13c8005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 msgr2=0x7f13c8005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.005+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13c80035a0 auth_method 0 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 msgr2=0x7f13c8005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:20:36.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 msgr2=0x7f13c8005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:20:36.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:36.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.209+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13c80035a0 auth_method 0 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 10 monclient(hunting): handle_auth_done global_id 44099 payload 824 2024-09-15T21:20:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13c8006290 con 0x7f13c80035a0 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c820 tx=0x7f13d80192b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80b1040 con 0x7f13c80035a0 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 10 monclient: dump: 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f13d800f000 con 0x7f13c80035a0 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 msgr2=0x7f13c8005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c820 tx=0x7f13d80192b0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 msgr2=0x7f13c8005a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.613+0000 7f13ed7fa640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13c80035a0 0x7f13c8005a30 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c820 tx=0x7f13d80192b0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13f6ea6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 0x7f13f01db620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13f6ea6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f13f0140bb0 con 0x7f13f0064340 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 0x7f13f01db620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient: get_auth_request con 0x7f13f0064340 auth_method 0 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.621+0000 7f13effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13effff640 10 monclient(hunting): handle_auth_done global_id 44102 payload 824 2024-09-15T21:20:36.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13effff640 1 -- 172.21.15.17:0/1757161647 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13f013edb0 con 0x7f13f0064340 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13effff640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 0x7f13f01db620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000ee0 tx=0x7f13dc005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80c5070 con 0x7f13f0064340 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 10 monclient: dump: 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f13d80c5570 con 0x7f13f0064340 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 msgr2=0x7f13f01db620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 0x7f13f01db620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000ee0 tx=0x7f13dc005840 comp rx=0 tx=0).stop 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 msgr2=0x7f13f01db620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13ed7fa640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13f0064340 0x7f13f01db620 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000ee0 tx=0x7f13dc005840 comp rx=0 tx=0).stop 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f5443640 1 -- 172.21.15.17:0/1757161647 reap_dead start 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 38, 2024-09-15T21:20:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 5223, 2024-09-15T21:20:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:36.648 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:36.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13d003bd30 msgr2=0x7f13d003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13d003bd30 0x7f13d003e1f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13e8000c10 tx=0x7f13e8007a00 comp rx=0 tx=0).stop 2024-09-15T21:20:36.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 10 monclient: shutdown 2024-09-15T21:20:36.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13f01383b0 msgr2=0x7f13f009d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13f01383b0 0x7f13f009d340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f13e000fb90 tx=0x7f13e000d520 comp rx=0 tx=0).stop 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 shutdown_connections 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13d003bd30 0x7f13d003e1f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 --2- 172.21.15.17:0/1757161647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13f01383b0 0x7f13f009d340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 >> 172.21.15.17:0/1757161647 conn(0x7f13f00aac40 msgr2=0x7f13f0135bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 shutdown_connections 2024-09-15T21:20:36.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.625+0000 7f13f6ea6640 1 -- 172.21.15.17:0/1757161647 wait complete. 2024-09-15T21:20:36.655 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:20:36.655 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:20:36.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:20:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.706+0000 7f40537fe640 1 -- 172.21.15.119:0/741964302 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v81) ==== 128+0+0 (secure 0 0 0) 0x7f406001cad0 con 0x7f406c0a09e0 2024-09-15T21:20:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.706+0000 7f40537fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T21:20:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.706+0000 7f40537fe640 10 monclient: _finish_command 3 = system:0 removed item id -16 name 'host2' from crush map 2024-09-15T21:20:36.711 INFO:tasks.workunit.client.0.smithi119.stderr:removed item id -16 name 'host2' from crush map 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405803b9f0 msgr2=0x7f405803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405803b9f0 0x7f405803deb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f406c1377c0 tx=0x7f405c007d50 comp rx=0 tx=0).stop 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 10 monclient: shutdown 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 msgr2=0x7f406c11f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 0x7f406c11f870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f406000d800 tx=0x7f406000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 shutdown_connections 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405803b9f0 0x7f405803deb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c137bc0 0x7f406c11d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f406c0a0ea0 0x7f406c1366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 --2- 172.21.15.119:0/741964302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406c0a09e0 0x7f406c11f870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 >> 172.21.15.119:0/741964302 conn(0x7f406c0a2ae0 msgr2=0x7f406c09d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 shutdown_connections 2024-09-15T21:20:36.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.710+0000 7f407166f640 1 -- 172.21.15.119:0/741964302 wait complete. 2024-09-15T21:20:36.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket host-for-test host root=root-for-test rack=rack-for-test 2024-09-15T21:20:36.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient: get_monmap_and_config 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient: build_initial_monmap 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient: monmap: 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:36.783851+0000 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:36.783851+0000 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:36.787 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:36.787 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 Processor -- start 2024-09-15T21:20:36.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 -- start start 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient: init 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fae941340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae9409d6c0 0x7fae94138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): picked mon.noname-c con 0x7fae9409d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 0x7fae9413b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): picked mon.noname-b con 0x7fae94139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9409e1b0 con 0x7fae941340c0 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fae9409d520 con 0x7fae9409d6c0 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fae9409e490 con 0x7fae94139390 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae9ad05640 10 monclient(hunting): authenticate will time out at 1160.179186s 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 0x7fae9413b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 0x7fae9413b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57768/0 (socket says 172.21.15.17:57768) 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 -- 172.21.15.17:0/297469347 learned_addr learned my addr 172.21.15.17:0/297469347 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): get_auth_request con 0x7fae94139390 auth_method 0 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae93fff640 1 --2- 172.21.15.17:0/297469347 >> v2:172.21.15.119:3300/0 conn(0x7fae9409d6c0 0x7fae94138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae93fff640 10 monclient(hunting): get_auth_request con 0x7fae9409d6c0 auth_method 0 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient(hunting): handle_auth_done global_id 44105 payload 274 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 -- 172.21.15.17:0/297469347 >> v2:172.21.15.119:3300/0 conn(0x7fae9409d6c0 msgr2=0x7fae94138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 --2- 172.21.15.17:0/297469347 >> v2:172.21.15.119:3300/0 conn(0x7fae9409d6c0 0x7fae94138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 -- 172.21.15.17:0/297469347 >> v1:172.21.15.119:6789/0 conn(0x7fae941340c0 legacy=0x7fae941344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient: _finish_hunting 0 2024-09-15T21:20:36.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 monclient: found mon.noname-b 2024-09-15T21:20:36.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:36.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.781+0000 7fae992a2640 1 -- 172.21.15.17:0/297469347 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9409e770 con 0x7fae94139390 2024-09-15T21:20:36.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae992a2640 1 --2- 172.21.15.17:0/297469347 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 0x7fae9413b780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009e50 tx=0x7fae8c01c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec92faa198c0dbfc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 1 -- 172.21.15.17:0/297469347 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0a5530 con 0x7fae94139390 2024-09-15T21:20:36.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:36.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: dump: 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.799 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 1 -- 172.21.15.17:0/297469347 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8c0a56d0 con 0x7fae94139390 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 1 -- 172.21.15.17:0/297469347 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0a5aa0 con 0x7fae94139390 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 5 monclient: authenticate success, global_id 44105 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 4 set_mon_vals no callback set 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:36.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae937fe640 10 monclient: dump: 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 10 monclient: shutdown 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:36.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 -- 172.21.15.17:0/297469347 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 msgr2=0x7fae9413b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 --2- 172.21.15.17:0/297469347 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 0x7fae9413b780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009e50 tx=0x7fae8c01c8f0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 -- 172.21.15.17:0/297469347 shutdown_connections 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 --2- 172.21.15.17:0/297469347 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae94139390 0x7fae9413b780 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 --2- 172.21.15.17:0/297469347 >> v2:172.21.15.119:3300/0 conn(0x7fae9409d6c0 0x7fae94138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 -- 172.21.15.17:0/297469347 >> 172.21.15.17:0/297469347 conn(0x7fae940a66a0 msgr2=0x7fae940a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 -- 172.21.15.17:0/297469347 shutdown_connections 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.785+0000 7fae9ad05640 1 -- 172.21.15.17:0/297469347 wait complete. 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: build_initial_monmap 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: monmap: 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:36.802 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 Processor -- start 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- start start 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: init 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9409d6c0 0x7fae941cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): picked mon.noname-a con 0x7fae9409d6c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae941340c0 0x7fae941309f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): picked mon.noname-c con 0x7fae941340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae94139390 0x7fae94071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): picked mon.noname-b con 0x7fae94139390 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fae9413e060 con 0x7fae9409d6c0 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae9413dee0 con 0x7fae94139390 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fae9413e1e0 con 0x7fae941340c0 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:36.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): authenticate will time out at 1160.185073s 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae941340c0 0x7fae941309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9409d6c0 0x7fae941cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae941340c0 0x7fae941309f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57778/0 (socket says 172.21.15.17:57778) 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 -- 172.21.15.17:0/487538757 learned_addr learned my addr 172.21.15.17:0/487538757 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): get_auth_request con 0x7fae941340c0 auth_method 0 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): get_auth_request con 0x7fae9409d6c0 auth_method 0 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae94139390 0x7fae94071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): get_auth_request con 0x7fae94139390 auth_method 0 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient(hunting): handle_auth_done global_id 44108 payload 824 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae94139390 msgr2=0x7fae94071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae94139390 0x7fae94071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9409d6c0 msgr2=0x7fae941cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9409d6c0 0x7fae941cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 monclient: found mon.noname-c 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae940730f0 con 0x7fae941340c0 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9409d6c0 0x7fae941cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae93fff640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae941340c0 0x7fae941309f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d700 tx=0x7fae8400dbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84015070 con 0x7fae941340c0 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: dump: 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:36.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae84007900 con 0x7fae941340c0 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8400b040 con 0x7fae941340c0 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 5 monclient: authenticate success, global_id 44108 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: _renew_subs 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9409d0f0 con 0x7fae941340c0 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 10 monclient: dump: 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.809 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae92ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: _renew_subs 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9411c4e0 con 0x7fae941340c0 2024-09-15T21:20:36.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 0x7fae9402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae9411d4b0 con 0x7fae9402d450 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fae84018070 con 0x7fae941340c0 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 0x7fae9402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient: get_auth_request con 0x7fae9402d450 auth_method 0 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fae8003b9c0 0x7fae8003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(81..81 src has 1..81) ==== 6045+0+0 (secure 0 0 0) 0x7fae84050190 con 0x7fae941340c0 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 10 monclient(hunting): handle_auth_done global_id 34483 payload 824 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae941344a0 con 0x7fae9402d450 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fae8003b9c0 0x7fae8003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae992a2640 10 monclient: get_auth_request con 0x7fae8003b9c0 auth_method 0 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.789+0000 7fae98aa1640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 0x7fae9402d830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae88000f70 tx=0x7fae8800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae880052e0 con 0x7fae9402d450 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 10 monclient: dump: 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae992a2640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fae8003b9c0 0x7fae8003de80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0ae000 tx=0x7fae8c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fae88018070 con 0x7fae9402d450 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 msgr2=0x7fae9402d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 0x7fae9402d830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae88000f70 tx=0x7fae8800ff70 comp rx=0 tx=0).stop 2024-09-15T21:20:36.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 msgr2=0x7fae9402d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:36.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.793+0000 7fae917fa640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402d450 0x7fae9402d830 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae88000f70 tx=0x7fae8800ff70 comp rx=0 tx=0).stop 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae9ad05640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae9ad05640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 0x7fae9402e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae9ad05640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae94137dd0 con 0x7fae9402dd70 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 0x7fae9402e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient: get_auth_request con 0x7fae9402dd70 auth_method 0 2024-09-15T21:20:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 10 monclient(hunting): handle_auth_done global_id 34486 payload 824 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 1 -- 172.21.15.17:0/487538757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fae941ce540 con 0x7fae9402dd70 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.797+0000 7fae98aa1640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 0x7fae9402e180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae88017000 tx=0x7fae88001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88005480 con 0x7fae9402dd70 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 10 monclient: dump: 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7fae88005480 con 0x7fae9402dd70 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 msgr2=0x7fae9402e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 0x7fae9402e180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae88017000 tx=0x7fae88001130 comp rx=0 tx=0).stop 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 msgr2=0x7fae9402e180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae917fa640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae9402dd70 0x7fae9402e180 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae88017000 tx=0x7fae88001130 comp rx=0 tx=0).stop 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:36.818 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 38, 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 49298, 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:36.819 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.820 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:36.821 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:36.824 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:36.825 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.826 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.828 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.829 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:36.830 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:36.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:36.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:36.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:36.831 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:36.832 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:36.833 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.834 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:36.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:36.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:36.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:36.836 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:36.836 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:36.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fae8003b9c0 msgr2=0x7fae8003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fae8003b9c0 0x7fae8003de80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0ae000 tx=0x7fae8c0a7070 comp rx=0 tx=0).stop 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 10 monclient: shutdown 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae941340c0 msgr2=0x7fae941309f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 --2- 172.21.15.17:0/487538757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fae941340c0 0x7fae941309f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d700 tx=0x7fae8400dbc0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae992a2640 1 -- 172.21.15.17:0/487538757 reap_dead start 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae992a2640 1 -- 172.21.15.17:0/487538757 reap_dead start 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 shutdown_connections 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 >> 172.21.15.17:0/487538757 conn(0x7fae940a66a0 msgr2=0x7fae94138a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:36.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 shutdown_connections 2024-09-15T21:20:36.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.801+0000 7fae9ad05640 1 -- 172.21.15.17:0/487538757 wait complete. 2024-09-15T21:20:36.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:20:36.839 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:20:36.839 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:20:36.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient: get_monmap_and_config 2024-09-15T21:20:36.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient: build_initial_monmap 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient: monmap: 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:36.858309+0000 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:36.858309+0000 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:36.860 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:36.861 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:36.861 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:36.861 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:36.861 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:36.861 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:36.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 1 Processor -- start 2024-09-15T21:20:36.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 1 -- start start 2024-09-15T21:20:36.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient: init 2024-09-15T21:20:36.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:36.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:20:36.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2e68135bf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e6809fe70 0x7f2e6813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient(hunting): picked mon.noname-b con 0x7f2e6809fe70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2e6813afb0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680a0300 con 0x7f2e6813afb0 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680a1d80 con 0x7f2e68135bf0 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2e680a3600 con 0x7f2e6809fe70 2024-09-15T21:20:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e6db79640 10 monclient(hunting): authenticate will time out at 1155.654925s 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e677fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2e68135bf0 0x7f2e68135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46872/0 (socket says 172.21.15.119:46872) 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e677fe640 1 -- 172.21.15.119:0/2254936175 learned_addr learned my addr 172.21.15.119:0/2254936175 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 1 --2- 172.21.15.119:0/2254936175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e6809fe70 0x7f2e6813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): get_auth_request con 0x7f2e6809fe70 auth_method 0 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334535642 0 0) 0x7f2e680a1d80 con 0x7f2e68135bf0 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): my global_id is 34489 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e54003530 con 0x7f2e68135bf0 2024-09-15T21:20:36.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209949981 0 0) 0x7f2e600014a0 con 0x7f2e6813afb0 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 10 monclient(hunting): my global_id is 5283 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e680a1d80 con 0x7f2e6813afb0 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient(hunting): handle_auth_done global_id 44111 payload 274 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 1 -- 172.21.15.119:0/2254936175 >> v1:172.21.15.119:6789/0 conn(0x7f2e68135bf0 legacy=0x7f2e68135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 1 -- 172.21.15.119:0/2254936175 >> v1:172.21.15.17:6789/0 conn(0x7f2e6813afb0 legacy=0x7f2e6813d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 1 monclient: found mon.noname-b 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.854+0000 7f2e5effd640 1 -- 172.21.15.119:0/2254936175 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680a2060 con 0x7f2e6809fe70 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e5effd640 1 --2- 172.21.15.119:0/2254936175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e6809fe70 0x7f2e6813aa70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e500097f0 tx=0x7f2e5001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eadfe34a7626482 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b1070 con 0x7f2e6809fe70 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:36.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: dump: 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.878 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 5 monclient: authenticate success, global_id 44111 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e500ac5f0 con 0x7f2e6809fe70 2024-09-15T21:20:36.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:36.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:36.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 1 -- 172.21.15.119:0/2254936175 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500ac9a0 con 0x7f2e6809fe70 2024-09-15T21:20:36.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 4 set_mon_vals no callback set 2024-09-15T21:20:36.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e66ffd640 10 monclient: dump: 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.881 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: shutdown 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- 172.21.15.119:0/2254936175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e6809fe70 msgr2=0x7f2e6813aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 --2- 172.21.15.119:0/2254936175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e6809fe70 0x7f2e6813aa70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e500097f0 tx=0x7f2e5001c370 comp rx=0 tx=0).stop 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- 172.21.15.119:0/2254936175 shutdown_connections 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 --2- 172.21.15.119:0/2254936175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e6809fe70 0x7f2e6813aa70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- 172.21.15.119:0/2254936175 >> 172.21.15.119:0/2254936175 conn(0x7f2e680a66e0 msgr2=0x7f2e680a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- 172.21.15.119:0/2254936175 shutdown_connections 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- 172.21.15.119:0/2254936175 wait complete. 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: build_initial_monmap 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: monmap: 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:36.883 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 Processor -- start 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- start start 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: init 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:36.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e6809fe70 0x7f2e681cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): picked mon.noname-a con 0x7f2e6809fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e68135bf0 0x7f2e68134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): picked mon.noname-c con 0x7f2e68135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 0x7f2e6809d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): picked mon.noname-b con 0x7f2e6813afb0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6806edb0 con 0x7f2e6809fe70 2024-09-15T21:20:36.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6806ec30 con 0x7f2e6813afb0 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2e681416f0 con 0x7f2e68135bf0 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e6db79640 10 monclient(hunting): authenticate will time out at 1155.659894s 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.858+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 0x7f2e6809d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 0x7f2e6809d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40036/0 (socket says 172.21.15.119:40036) 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 -- 172.21.15.119:0/1087655161 learned_addr learned my addr 172.21.15.119:0/1087655161 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:36.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): get_auth_request con 0x7f2e6813afb0 auth_method 0 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e68135bf0 0x7f2e68134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e6809fe70 0x7f2e681cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): get_auth_request con 0x7f2e68135bf0 auth_method 0 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): get_auth_request con 0x7f2e6809fe70 auth_method 0 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient(hunting): handle_auth_done global_id 34492 payload 824 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 -- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e68135bf0 msgr2=0x7f2e68134bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e68135bf0 0x7f2e68134bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 -- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e6809fe70 msgr2=0x7f2e681cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e6809fe70 0x7f2e681cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 monclient: found mon.noname-b 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 -- 172.21.15.119:0/1087655161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e68072a20 con 0x7f2e6813afb0 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e67fff640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 0x7f2e6809d580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c7a0 tx=0x7f2e6000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60003b80 con 0x7f2e6813afb0 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: dump: 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.891 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: _finish_auth 0 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e60003d20 con 0x7f2e6813afb0 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 5 monclient: authenticate success, global_id 34492 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 monclient: _renew_subs 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6809d240 con 0x7f2e6813afb0 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:36.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000aa90 con 0x7f2e6813afb0 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 10 monclient: dump: 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.894 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e667fc640 4 set_mon_vals no callback set 2024-09-15T21:20:36.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 monclient: _renew_subs 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6809f040 con 0x7f2e6813afb0 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e68135fd0 con 0x7f2e6813afb0 2024-09-15T21:20:36.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2e6000ac30 con 0x7f2e6813afb0 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e4803b9c0 0x7f2e4803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6045+0+0 (secure 0 0 0) 0x7f2e6004fdd0 con 0x7f2e6813afb0 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e4803b9c0 0x7f2e4803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 10 monclient: get_auth_request con 0x7f2e4803b9c0 auth_method 0 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.862+0000 7f2e677fe640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e4803b9c0 0x7f2e4803de80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e58007fd0 tx=0x7f2e58007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.866+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2e680a0300 con 0x7f2e6813afb0 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.866+0000 7f2e64ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:36.866+0000 7f2e64ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:36.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.949+0000 7fbc640a1640 10 monclient: get_monmap_and_config 2024-09-15T21:20:36.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.949+0000 7fbc640a1640 10 monclient: build_initial_monmap 2024-09-15T21:20:36.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.949+0000 7fbc640a1640 10 monclient: monmap: 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:36.953739+0000 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:36.953739+0000 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:36.956 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 1 Processor -- start 2024-09-15T21:20:36.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 1 -- start start 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient: init 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbc5c0a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbc5c0a5580 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbc5c0a59d0 0x7fbc5c140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient(hunting): picked mon.noname-a con 0x7fbc5c0a59d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5c0a2c30 con 0x7fbc5c0a4a90 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5c0a19b0 con 0x7fbc5c0a5580 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5c0a0f40 con 0x7fbc5c0a59d0 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc640a1640 10 monclient(hunting): authenticate will time out at 1160.349376s 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc61e3d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbc5c0a4a90 0x7fbc5c0a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49058/0 (socket says 172.21.15.17:49058) 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc61e3d640 1 -- 172.21.15.17:0/3961443437 learned_addr learned my addr 172.21.15.17:0/3961443437 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 1 --2- 172.21.15.17:0/3961443437 >> v2:172.21.15.17:3300/0 conn(0x7fbc5c0a59d0 0x7fbc5c140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): get_auth_request con 0x7fbc5c0a59d0 auth_method 0 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient(hunting): handle_auth_done global_id 5289 payload 274 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc60e3b640 1 -- 172.21.15.17:0/3961443437 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3763578578 0 0) 0x7fbc58001120 con 0x7fbc5c0a4a90 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 1 -- 172.21.15.17:0/3961443437 >> v1:172.21.15.119:6789/0 conn(0x7fbc5c0a5580 legacy=0x7fbc5c13cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 1 -- 172.21.15.17:0/3961443437 >> v1:172.21.15.17:6789/0 conn(0x7fbc5c0a4a90 legacy=0x7fbc5c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:36.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 1 monclient: found mon.noname-a 2024-09-15T21:20:36.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:36.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:36.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc6263e640 1 -- 172.21.15.17:0/3961443437 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0a1ce0 con 0x7fbc5c0a59d0 2024-09-15T21:20:36.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.953+0000 7fbc60e3b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:20:36.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6263e640 1 --2- 172.21.15.17:0/3961443437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c140440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009dc0 tx=0x7fbc5401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a5d245c689a7a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 1 -- 172.21.15.17:0/3961443437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540b2070 con 0x7fbc5c0a59d0 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: dump: 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.966 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: _finish_auth 0 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 1 -- 172.21.15.17:0/3961443437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc540a5850 con 0x7fbc5c0a59d0 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 5 monclient: authenticate success, global_id 5289 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 4 set_mon_vals no callback set 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 1 -- 172.21.15.17:0/3961443437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540a5c20 con 0x7fbc5c0a59d0 2024-09-15T21:20:36.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc60e3b640 10 monclient: dump: 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.968 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: shutdown 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/3961443437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a59d0 msgr2=0x7fbc5c140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 --2- 172.21.15.17:0/3961443437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c140440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009dc0 tx=0x7fbc5401c870 comp rx=0 tx=0).stop 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/3961443437 shutdown_connections 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 --2- 172.21.15.17:0/3961443437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c140440 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/3961443437 >> 172.21.15.17:0/3961443437 conn(0x7fbc5c0aabb0 msgr2=0x7fbc5c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/3961443437 shutdown_connections 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/3961443437 wait complete. 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: build_initial_monmap 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: monmap: 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:36.969 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 Processor -- start 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- start start 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: init 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:36.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 0x7fbc5c133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): picked mon.noname-a con 0x7fbc5c0a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 0x7fbc5c133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): picked mon.noname-c con 0x7fbc5c0a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): picked mon.noname-b con 0x7fbc5c0a59d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1437c0 con 0x7fbc5c0a4a90 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c06f2f0 con 0x7fbc5c0a59d0 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c143940 con 0x7fbc5c0a5580 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:36.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient(hunting): authenticate will time out at 1160.354450s 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 0x7fbc5c133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 0x7fbc5c133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 0x7fbc5c133a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42106/0 (socket says 172.21.15.17:42106) 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 0x7fbc5c133fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57780/0 (socket says 172.21.15.17:57780) 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 1 -- 172.21.15.17:0/2706100568 learned_addr learned my addr 172.21.15.17:0/2706100568 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): get_auth_request con 0x7fbc5c0a4a90 auth_method 0 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): get_auth_request con 0x7fbc5c0a5580 auth_method 0 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6263e640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6263e640 10 monclient(hunting): get_auth_request con 0x7fbc5c0a59d0 auth_method 0 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6263e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6263e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6263e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient(hunting): handle_auth_done global_id 44117 payload 824 2024-09-15T21:20:36.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 -- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c0a59d0 msgr2=0x7fbc5c138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 -- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 msgr2=0x7fbc5c133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 0x7fbc5c133a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 monclient: found mon.noname-c 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 -- 172.21.15.17:0/2706100568 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c13aa80 con 0x7fbc5c0a5580 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc61e3d640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 0x7fbc5c133a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc6163c640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 0x7fbc5c133fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c400 tx=0x7fbc4c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c017070 con 0x7fbc5c0a5580 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: dump: 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0052d0 con 0x7fbc5c0a5580 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 5 monclient: authenticate success, global_id 44117 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: _renew_subs 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c13f5b0 con 0x7fbc5c0a5580 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c003820 con 0x7fbc5c0a5580 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc527fc640 10 monclient: dump: 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:36.976 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc53fff640 4 set_mon_vals no callback set 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: _renew_subs 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.957+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c0a1280 con 0x7fbc5c0a5580 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc640a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc640a1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc640a1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c0a4e70 con 0x7fbc5c0a5580 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbc4c01a070 con 0x7fbc5c0a5580 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc527fc640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbc4003b9f0 0x7fbc4003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(81..81 src has 1..81) ==== 6045+0+0 (secure 0 0 0) 0x7fbc4c04f410 con 0x7fbc5c0a5580 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc61e3d640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbc4003b9f0 0x7fbc4003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc61e3d640 10 monclient: get_auth_request con 0x7fbc4003b9f0 auth_method 0 2024-09-15T21:20:36.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.961+0000 7fbc61e3d640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbc4003b9f0 0x7fbc4003deb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc58000f70 tx=0x7fbc580018e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:36.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.965+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbc5c13f5b0 con 0x7fbc5c0a5580 2024-09-15T21:20:36.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.965+0000 7fbc527fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:36.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:36.965+0000 7fbc527fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:37.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.026+0000 7f2e6db79640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T21:20:37.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.026+0000 7f2e6db79640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T21:20:37.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.026+0000 7f2e6db79640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:37.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.026+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) -- 0x7f2e68071df0 con 0x7f2e6813afb0 2024-09-15T21:20:37.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.201+0000 7fbc640a1640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:20:37.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.201+0000 7fbc640a1640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:20:37.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.201+0000 7fbc640a1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:37.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fbc5c069200 con 0x7fbc5c0a5580 2024-09-15T21:20:37.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fbc527fc640 1 -- 172.21.15.17:0/2706100568 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fbc4c005480 con 0x7fbc5c0a5580 2024-09-15T21:20:37.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fbc527fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:20:37.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fbc527fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:37.211 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.211 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=11,kv=12,logm=64} crc {auth=2215358404,config=3541143443,health=1096256309,kv=3079713637,logm=3054134746}) 2024-09-15T21:20:37.212 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=2215358404,config=3541143443,health=1535303502,kv=3079713637,logm=1942757572}) 2024-09-15T21:20:37.212 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.212 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=11,kv=12,logm=64} crc {auth=2215358404,config=3541143443,health=1096256309,kv=3079713637,logm=3054134746}) 2024-09-15T21:20:37.212 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.205+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2215358404,config=3541143443,health=1535303502,kv=363428264,logm=1902456147}) 2024-09-15T21:20:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbc4003b9f0 msgr2=0x7fbc4003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbc4003b9f0 0x7fbc4003deb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc58000f70 tx=0x7fbc580018e0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 10 monclient: shutdown 2024-09-15T21:20:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 msgr2=0x7fbc5c133fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 0x7fbc5c133fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c400 tx=0x7fbc4c00c8c0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 shutdown_connections 2024-09-15T21:20:37.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbc4003b9f0 0x7fbc4003deb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc5c0a59d0 0x7fbc5c138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbc5c0a5580 0x7fbc5c133fb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 --2- 172.21.15.17:0/2706100568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc5c0a4a90 0x7fbc5c133a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 >> 172.21.15.17:0/2706100568 conn(0x7fbc5c0aabb0 msgr2=0x7fbc5c136a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:37.214 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.214 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3274989835}) 2024-09-15T21:20:37.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 shutdown_connections 2024-09-15T21:20:37.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fbc640a1640 1 -- 172.21.15.17:0/2706100568 wait complete. 2024-09-15T21:20:37.215 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=974024663}) 2024-09-15T21:20:37.215 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.215 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3274989835}) 2024-09-15T21:20:37.215 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3339359417}) 2024-09-15T21:20:37.216 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.209+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.216 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176238291}) 2024-09-15T21:20:37.216 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=20975204}) 2024-09-15T21:20:37.216 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.216 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176238291}) 2024-09-15T21:20:37.216 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3688139801}) 2024-09-15T21:20:37.218 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.218 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1301502590}) 2024-09-15T21:20:37.218 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632768166}) 2024-09-15T21:20:37.218 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.218 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1301502590}) 2024-09-15T21:20:37.218 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.213+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4142633485}) 2024-09-15T21:20:37.220 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.221 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1719988852}) 2024-09-15T21:20:37.221 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=275316520}) 2024-09-15T21:20:37.221 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.221 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1719988852}) 2024-09-15T21:20:37.221 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=97915153}) 2024-09-15T21:20:37.223 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.223 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=313941800}) 2024-09-15T21:20:37.223 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1835630480}) 2024-09-15T21:20:37.223 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.223 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=313941800}) 2024-09-15T21:20:37.223 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.217+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2624669877}) 2024-09-15T21:20:37.225 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.225 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3581128075}) 2024-09-15T21:20:37.225 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1480799785}) 2024-09-15T21:20:37.225 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.225 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3581128075}) 2024-09-15T21:20:37.226 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3984273026}) 2024-09-15T21:20:37.227 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.227 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3251763283}) 2024-09-15T21:20:37.227 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=316800938}) 2024-09-15T21:20:37.227 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.227 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3251763283}) 2024-09-15T21:20:37.227 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.221+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=50874794}) 2024-09-15T21:20:37.232 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.225+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.232 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.225+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207357051}) 2024-09-15T21:20:37.232 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.225+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1874395739}) 2024-09-15T21:20:37.232 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.225+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.232 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.225+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207357051}) 2024-09-15T21:20:37.233 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.225+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=149751028}) 2024-09-15T21:20:37.234 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:20:37.235 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.235 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636556554}) 2024-09-15T21:20:37.235 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3143717714}) 2024-09-15T21:20:37.235 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.236 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=636556554}) 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3265154608}) 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=639587590}) 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4163343090}) 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=639587590}) 2024-09-15T21:20:37.237 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.229+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2965148513}) 2024-09-15T21:20:37.238 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.233+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.238 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.233+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2057578194}) 2024-09-15T21:20:37.238 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.233+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1741100397}) 2024-09-15T21:20:37.239 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.233+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.239 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.233+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2057578194}) 2024-09-15T21:20:37.239 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.233+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1399741844}) 2024-09-15T21:20:37.240 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.237+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.240 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.237+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=96,mdsmap=3,mgr=1} crc {logm=2780602932,mdsmap=4166369859,mgr=3189685207}) 2024-09-15T21:20:37.240 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.237+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=1630929975,mdsmap=173658045,mgr=259257734}) 2024-09-15T21:20:37.240 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.237+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.240 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.237+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=96,mdsmap=3,mgr=1} crc {logm=2780602932,mdsmap=4166369859,mgr=3189685207}) 2024-09-15T21:20:37.240 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.237+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=2396017080,mdsmap=173658045,mgr=259257734}) 2024-09-15T21:20:37.244 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.241+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.245 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.241+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {mgr=4266173344,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3809222421,mon_config_key=1008760200,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=4057149032,osd_snap=841190635,osdmap=989104815}) 2024-09-15T21:20:37.245 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.241+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=5,osd_snap=2,osdmap=57} crc {mgr=4266173344,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1902416853,mon_config_key=3106313821,monmap=186996059,nvmeofgw=598337686,osd_metadata=3068027012,osd_snap=841190635,osdmap=1961366833}) 2024-09-15T21:20:37.245 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.241+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.245 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.241+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {mgr=4266173344,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3809222421,mon_config_key=1008760200,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=4057149032,osd_snap=841190635,osdmap=989104815}) 2024-09-15T21:20:37.245 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.241+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=5,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=60} crc {mgr=4270402007,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2735313488,mon_config_key=3670636429,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=4057149032,osd_snap=3823785925,osdmap=408937868}) 2024-09-15T21:20:37.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.253+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.253+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=71} crc {osdmap=1273764978}) 2024-09-15T21:20:37.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.253+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=75} crc {osdmap=3675921954}) 2024-09-15T21:20:37.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.253+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:20:37.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.253+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=71} crc {osdmap=1273764978}) 2024-09-15T21:20:37.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:20:37.253+0000 7fa914b53640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=75} crc {osdmap=1858945530}) 2024-09-15T21:20:37.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.726+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} v82) ==== 251+0+0 (secure 0 0 0) 0x7f2e60012d70 con 0x7f2e6813afb0 2024-09-15T21:20:37.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.726+0000 7f2e64ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T21:20:37.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.726+0000 7f2e64ff9640 10 monclient: _finish_command 2 = system:0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} 2024-09-15T21:20:37.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.830+0000 7f2e6db79640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T21:20:37.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.830+0000 7f2e6db79640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T21:20:37.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.830+0000 7f2e6db79640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:37.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.830+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) -- 0x7f2e6806a830 con 0x7f2e6813afb0 2024-09-15T21:20:37.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.834+0000 7f2e64ff9640 1 -- 172.21.15.119:0/1087655161 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 bucket 'host-for-test' already exists v82) ==== 200+0+0 (secure 0 0 0) 0x7f2e60010cf0 con 0x7f2e6813afb0 2024-09-15T21:20:37.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.834+0000 7f2e64ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T21:20:37.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.834+0000 7f2e64ff9640 10 monclient: _finish_command 3 = system:0 bucket 'host-for-test' already exists 2024-09-15T21:20:37.840 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'host-for-test' already exists 2024-09-15T21:20:37.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e4803b9c0 msgr2=0x7f2e4803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:37.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e4803b9c0 0x7f2e4803de80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e58007fd0 tx=0x7f2e58007ef0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 10 monclient: shutdown 2024-09-15T21:20:37.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 msgr2=0x7f2e6809d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 0x7f2e6809d580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c7a0 tx=0x7f2e6000cc60 comp rx=0 tx=0).stop 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 shutdown_connections 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e4803b9c0 0x7f2e4803de80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6813afb0 0x7f2e6809d580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e68135bf0 0x7f2e68134bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 --2- 172.21.15.119:0/1087655161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e6809fe70 0x7f2e681cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 >> 172.21.15.119:0/1087655161 conn(0x7f2e680a66e0 msgr2=0x7f2e68136840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 shutdown_connections 2024-09-15T21:20:37.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.838+0000 7f2e6db79640 1 -- 172.21.15.119:0/1087655161 wait complete. 2024-09-15T21:20:37.858 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:37.858 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep host-for-test 2024-09-15T21:20:37.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.966+0000 7f2cbcd9b640 10 monclient: get_monmap_and_config 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.966+0000 7f2cbcd9b640 10 monclient: build_initial_monmap 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.966+0000 7f2cbcd9b640 10 monclient: monmap: 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:37.971362+0000 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:37.971362+0000 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:37.973 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:37.974 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.966+0000 7f2cbcd9b640 1 Processor -- start 2024-09-15T21:20:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.966+0000 7f2cbcd9b640 1 -- start start 2024-09-15T21:20:37.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.966+0000 7f2cbcd9b640 10 monclient: init 2024-09-15T21:20:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:37.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2024-09-15T21:20:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb80a4d80 0x7f2cb80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): picked mon.noname-b con 0x7f2cb80a4d80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2cb80a56a0 0x7f2cb8137d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): picked mon.noname-a con 0x7f2cb80a56a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cb8138260 0x7f2cb81386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:37.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): picked mon.noname-c con 0x7f2cb8138260 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb80a4650 con 0x7f2cb80a56a0 2024-09-15T21:20:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb80a2250 con 0x7f2cb8138260 2024-09-15T21:20:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:37.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb806dab0 con 0x7f2cb80a4d80 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cbcd9b640 10 monclient(hunting): authenticate will time out at 1156.768361s 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cb8138260 0x7f2cb81386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cb8138260 0x7f2cb81386d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40040/0 (socket says 172.21.15.119:40040) 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 -- 172.21.15.119:0/2311952561 learned_addr learned my addr 172.21.15.119:0/2311952561 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): get_auth_request con 0x7f2cb8138260 auth_method 0 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:37.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:37.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:37.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:37.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:37.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb5d9b640 1 --2- 172.21.15.119:0/2311952561 >> v2:172.21.15.17:3300/0 conn(0x7f2cb80a56a0 0x7f2cb8137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:37.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb659c640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb80a4d80 0x7f2cb80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:37.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb5d9b640 10 monclient(hunting): get_auth_request con 0x7f2cb80a56a0 auth_method 0 2024-09-15T21:20:37.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:37.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:37.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:37.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient(hunting): handle_auth_done global_id 34501 payload 274 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 -- 172.21.15.119:0/2311952561 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb80a4d80 msgr2=0x7f2cb80a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb80a4d80 0x7f2cb80a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 -- 172.21.15.119:0/2311952561 >> v2:172.21.15.17:3300/0 conn(0x7f2cb80a56a0 msgr2=0x7f2cb8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 --2- 172.21.15.119:0/2311952561 >> v2:172.21.15.17:3300/0 conn(0x7f2cb80a56a0 0x7f2cb8137d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 monclient: found mon.noname-c 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 -- 172.21.15.119:0/2311952561 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb80a19b0 con 0x7f2cb8138260 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb659c640 10 monclient: get_auth_request con 0x7f2cb80a4d80 auth_method 0 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb659c640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb80a4d80 0x7f2cb80a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.970+0000 7f2cb6d9d640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb8138260 0x7f2cb81386d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8069d10 tx=0x7f2ca801c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52dbba8a643356c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:37.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 1 -- 172.21.15.119:0/2311952561 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80b0070 con 0x7f2cb8138260 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: dump: 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:37.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: _finish_auth 0 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: _check_auth_tickets 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 5 monclient: authenticate success, global_id 34501 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 1 -- 172.21.15.119:0/2311952561 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca80ab600 con 0x7f2cb8138260 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 4 set_mon_vals no callback set 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 1 -- 172.21.15.119:0/2311952561 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80ab9d0 con 0x7f2cb8138260 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb559a640 10 monclient: dump: 2024-09-15T21:20:37.999 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: shutdown 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2311952561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb8138260 msgr2=0x7f2cb81386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb8138260 0x7f2cb81386d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8069d10 tx=0x7f2ca801c3b0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2311952561 shutdown_connections 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb8138260 0x7f2cb81386d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2311952561 >> v2:172.21.15.17:3300/0 conn(0x7f2cb80a56a0 0x7f2cb8137d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2311952561 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb80a4d80 0x7f2cb80a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2311952561 >> 172.21.15.119:0/2311952561 conn(0x7f2cb80a75c0 msgr2=0x7f2cb806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2311952561 shutdown_connections 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2311952561 wait complete. 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: monmap: 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:38.001 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 Processor -- start 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- start start 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: init 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cb80a4d80 0x7f2cb8136310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): picked mon.noname-a con 0x7f2cb80a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 0x7f2cb8071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): picked mon.noname-b con 0x7f2cb80a56a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb8138260 0x7f2cb8071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): picked mon.noname-c con 0x7f2cb8138260 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80a1110 con 0x7f2cb80a4d80 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80a5ba0 con 0x7f2cb80a56a0 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb80a1290 con 0x7f2cb8138260 2024-09-15T21:20:38.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient(hunting): authenticate will time out at 1156.774677s 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 0x7f2cb8071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 0x7f2cb8071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40050/0 (socket says 172.21.15.119:40050) 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 -- 172.21.15.119:0/2620800388 learned_addr learned my addr 172.21.15.119:0/2620800388 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): get_auth_request con 0x7f2cb80a56a0 auth_method 0 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb659c640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cb80a4d80 0x7f2cb8136310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb659c640 10 monclient(hunting): get_auth_request con 0x7f2cb80a4d80 auth_method 0 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb6d9d640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb8138260 0x7f2cb8071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb6d9d640 10 monclient(hunting): get_auth_request con 0x7f2cb8138260 auth_method 0 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient(hunting): handle_auth_done global_id 34504 payload 824 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 -- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb8138260 msgr2=0x7f2cb8071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb8138260 0x7f2cb8071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 -- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cb80a4d80 msgr2=0x7f2cb8136310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cb80a4d80 0x7f2cb8136310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 monclient: found mon.noname-b 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 -- 172.21.15.119:0/2620800388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb81219d0 con 0x7f2cb80a56a0 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb659c640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cb80a4d80 0x7f2cb8136310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb5d9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 0x7f2cb8071350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d800 tx=0x7f2ca000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000b6c0 con 0x7f2cb80a56a0 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: dump: 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0019070 con 0x7f2cb80a56a0 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 5 monclient: authenticate success, global_id 34504 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: _renew_subs 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb80a1570 con 0x7f2cb80a56a0 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000ba60 con 0x7f2cb80a56a0 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2caeffd640 10 monclient: dump: 2024-09-15T21:20:38.008 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: _renew_subs 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cb4d99640 4 set_mon_vals no callback set 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.974+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb81d02b0 con 0x7f2cb80a56a0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cbcd9b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cbcd9b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cbcd9b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb8137d20 con 0x7f2cb80a56a0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2ca000bc00 con 0x7f2cb80a56a0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2caeffd640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c9803b9c0 0x7f2c9803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f2ca0050650 con 0x7f2cb80a56a0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cb659c640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c9803b9c0 0x7f2c9803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cb659c640 10 monclient: get_auth_request con 0x7f2c9803b9c0 auth_method 0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.978+0000 7f2cb659c640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c9803b9c0 0x7f2c9803de80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000e550 tx=0x7f2cb0007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.982+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2ca001cd70 con 0x7f2cb80a56a0 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.982+0000 7f2caeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:37.982+0000 7f2caeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:38.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2cbcd9b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:38.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2cbcd9b640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2cbcd9b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f2cb80a4850 con 0x7f2cb80a56a0 2024-09-15T21:20:38.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1500 (secure 0 0 0) 0x7f2ca0013390 con 0x7f2cb80a56a0 2024-09-15T21:20:38.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2caeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.150+0000 7f2caeffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:38.235 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:20:38.236 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2cbcd9b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2cbcd9b640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2cbcd9b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f2cb80008e0 con 0x7f2cb80a56a0 2024-09-15T21:20:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2caeffd640 1 -- 172.21.15.119:0/2620800388 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1500 (secure 0 0 0) 0x7f2ca0052ba0 con 0x7f2cb80a56a0 2024-09-15T21:20:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2caeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.242+0000 7f2caeffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:38.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c9803b9c0 msgr2=0x7f2c9803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c9803b9c0 0x7f2c9803de80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000e550 tx=0x7f2cb0007ab0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 10 monclient: shutdown 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 msgr2=0x7f2cb8071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 0x7f2cb8071350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d800 tx=0x7f2ca000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 shutdown_connections 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2c9803b9c0 0x7f2c9803de80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2cb8138260 0x7f2cb8071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cb80a56a0 0x7f2cb8071350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 --2- 172.21.15.119:0/2620800388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cb80a4d80 0x7f2cb8136310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 >> 172.21.15.119:0/2620800388 conn(0x7f2cb80a75c0 msgr2=0x7f2cb80a8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 shutdown_connections 2024-09-15T21:20:38.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.246+0000 7f2cbcd9b640 1 -- 172.21.15.119:0/2620800388 wait complete. 2024-09-15T21:20:38.265 INFO:tasks.workunit.client.0.smithi119.stdout:-11 0 host host-for-test 2024-09-15T21:20:38.265 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:38.266 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep rack-for-test 2024-09-15T21:20:38.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.361+0000 7f239ed5a640 10 monclient: get_monmap_and_config 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.361+0000 7f239ed5a640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.361+0000 7f239ed5a640 10 monclient: monmap: 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:38.366018+0000 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:38.366018+0000 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:38.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:38.369 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:38.369 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:38.369 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:38.369 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 1 Processor -- start 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 1 -- start start 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient: init 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f23980a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f239809e8c0 0x7f23980a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient(hunting): picked mon.noname-b con 0x7f239809e8c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f23981410a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23980a31c0 con 0x7f23981410a0 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239809d640 con 0x7f23980a0890 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f23980a1ac0 con 0x7f239809e8c0 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239ed5a640 10 monclient(hunting): authenticate will time out at 1161.761489s 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239d2f7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f23981410a0 0x7f2398143490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60792/0 (socket says 172.21.15.17:60792) 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f239d2f7640 1 -- 172.21.15.17:0/1125319589 learned_addr learned my addr 172.21.15.17:0/1125319589 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 1 --2- 172.21.15.17:0/1125319589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f239809e8c0 0x7f23980a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): get_auth_request con 0x7f239809e8c0 auth_method 0 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient(hunting): handle_auth_done global_id 44123 payload 274 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 1 -- 172.21.15.17:0/1125319589 >> v1:172.21.15.119:6789/0 conn(0x7f23980a0890 legacy=0x7f23980a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 1 -- 172.21.15.17:0/1125319589 >> v1:172.21.15.17:6789/0 conn(0x7f23981410a0 legacy=0x7f2398143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 1 monclient: found mon.noname-b 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:38.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.365+0000 7f2397fff640 1 -- 172.21.15.17:0/1125319589 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239809d970 con 0x7f239809e8c0 2024-09-15T21:20:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 --2- 172.21.15.17:0/1125319589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f239809e8c0 0x7f23980a1380 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23800097f0 tx=0x7f238001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d0420ed6017ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 1 -- 172.21.15.17:0/1125319589 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800b0070 con 0x7f239809e8c0 2024-09-15T21:20:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:38.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: dump: 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 1 -- 172.21.15.17:0/1125319589 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23800ab600 con 0x7f239809e8c0 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 1 -- 172.21.15.17:0/1125319589 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800ab9d0 con 0x7f239809e8c0 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 5 monclient: authenticate success, global_id 44123 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 4 set_mon_vals no callback set 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23977fe640 10 monclient: dump: 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: shutdown 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/1125319589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f239809e8c0 msgr2=0x7f23980a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 --2- 172.21.15.17:0/1125319589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f239809e8c0 0x7f23980a1380 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23800097f0 tx=0x7f238001c3a0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/1125319589 shutdown_connections 2024-09-15T21:20:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 --2- 172.21.15.17:0/1125319589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f239809e8c0 0x7f23980a1380 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/1125319589 >> 172.21.15.17:0/1125319589 conn(0x7f23980a66a0 msgr2=0x7f23980a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/1125319589 shutdown_connections 2024-09-15T21:20:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/1125319589 wait complete. 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: monmap: 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 Processor -- start 2024-09-15T21:20:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- start start 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: init 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23980a0890 0x7f23981d4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): picked mon.noname-b con 0x7f23980a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 0x7f239813c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): picked mon.noname-a con 0x7f23981410a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2398071330 0x7f2398071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): picked mon.noname-c con 0x7f2398071330 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2398147ba0 con 0x7f23981410a0 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2398147a20 con 0x7f23980a0890 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2398147d20 con 0x7f2398071330 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient(hunting): authenticate will time out at 1161.766634s 2024-09-15T21:20:38.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 0x7f239813c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2398071330 0x7f2398071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 0x7f239813c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47580/0 (socket says 172.21.15.17:47580) 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 -- 172.21.15.17:0/86442356 learned_addr learned my addr 172.21.15.17:0/86442356 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): get_auth_request con 0x7f23981410a0 auth_method 0 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): get_auth_request con 0x7f2398071330 auth_method 0 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23980a0890 0x7f23981d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): get_auth_request con 0x7f23980a0890 auth_method 0 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239caf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient(hunting): handle_auth_done global_id 5307 payload 824 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 -- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2398071330 msgr2=0x7f2398071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2398071330 0x7f2398071790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 -- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23980a0890 msgr2=0x7f23981d4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23980a0890 0x7f23981d4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 monclient: found mon.noname-a 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 -- 172.21.15.17:0/86442356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2398073f60 con 0x7f23981410a0 2024-09-15T21:20:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239d2f7640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2398071330 0x7f2398071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f2397fff640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 0x7f239813c060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2380018fb0 tx=0x7f2380019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800b8020 con 0x7f23981410a0 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: dump: 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23800b4630 con 0x7f23981410a0 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 5 monclient: authenticate success, global_id 5307 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: _renew_subs 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2398073860 con 0x7f23981410a0 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800b0040 con 0x7f23981410a0 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23957fa640 10 monclient: dump: 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f23967fc640 4 set_mon_vals no callback set 2024-09-15T21:20:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: _renew_subs 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.369+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2398072d00 con 0x7f23981410a0 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239ed5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239ed5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239ed5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23980a0c70 con 0x7f23981410a0 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f23800bb020 con 0x7f23981410a0 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f23957fa640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f237803b9f0 0x7f237803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f23800ea030 con 0x7f23981410a0 2024-09-15T21:20:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239caf6640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f237803b9f0 0x7f237803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239caf6640 10 monclient: get_auth_request con 0x7f237803b9f0 auth_method 0 2024-09-15T21:20:38.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.373+0000 7f239caf6640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f237803b9f0 0x7f237803deb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2388007fd0 tx=0x7f2388009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.377+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f23800c2030 con 0x7f23981410a0 2024-09-15T21:20:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.377+0000 7f23957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.377+0000 7f23957fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient: get_monmap_and_config 2024-09-15T21:20:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient: monmap: 2024-09-15T21:20:38.402 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:38.377172+0000 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:38.377172+0000 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 Processor -- start 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 -- start start 2024-09-15T21:20:38.403 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient: init 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0700a4d80 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a5870 0x7fe07013cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): picked mon.noname-b con 0x7fe0700a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe07013d4d0 0x7fe07013f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): picked mon.noname-a con 0x7fe07013d4d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0700a26e0 con 0x7fe0700a4d80 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe0700a6330 con 0x7fe07013d4d0 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe0700a30a0 con 0x7fe0700a5870 2024-09-15T21:20:38.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe075a0b640 10 monclient(hunting): authenticate will time out at 1157.173769s 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe0700a4d80 0x7fe0700a5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:34678/0 (socket says 172.21.15.119:34678) 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06effd640 1 -- 172.21.15.119:0/2852610971 learned_addr learned my addr 172.21.15.119:0/2852610971 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 1 --2- 172.21.15.119:0/2852610971 >> v2:172.21.15.17:3300/0 conn(0x7fe07013d4d0 0x7fe07013f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a5870 0x7fe07013cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): get_auth_request con 0x7fe07013d4d0 auth_method 0 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): get_auth_request con 0x7fe0700a5870 auth_method 0 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_done global_id 5310 payload 274 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 1 -- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a5870 msgr2=0x7fe07013cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a5870 0x7fe07013cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 1 -- 172.21.15.119:0/2852610971 >> v1:172.21.15.17:6789/0 conn(0x7fe0700a4d80 legacy=0x7fe0700a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:20:38.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06f7fe640 1 -- 172.21.15.119:0/2852610971 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0700a6a30 con 0x7fe07013d4d0 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.374+0000 7fe06e7fc640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a5870 0x7fe07013cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe07013f8c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe068009dc0 tx=0x7fe06801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6375accff7e8c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 1 -- 172.21.15.119:0/2852610971 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680b2070 con 0x7fe07013d4d0 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: dump: 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.407 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 1 -- 172.21.15.119:0/2852610971 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0680a5870 con 0x7fe07013d4d0 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 1 -- 172.21.15.119:0/2852610971 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680a5c40 con 0x7fe07013d4d0 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 5 monclient: authenticate success, global_id 5310 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:38.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 4 set_mon_vals no callback set 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06dffb640 10 monclient: dump: 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.409 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: shutdown 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 msgr2=0x7fe07013f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe07013f8c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe068009dc0 tx=0x7fe06801c8a0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- 172.21.15.119:0/2852610971 shutdown_connections 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe07013f8c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 --2- 172.21.15.119:0/2852610971 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a5870 0x7fe07013cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- 172.21.15.119:0/2852610971 >> 172.21.15.119:0/2852610971 conn(0x7fe0700a7a00 msgr2=0x7fe07006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- 172.21.15.119:0/2852610971 shutdown_connections 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- 172.21.15.119:0/2852610971 wait complete. 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: monmap: 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:38.410 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 Processor -- start 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- start start 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: init 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a4d80 0x7fe070134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): picked mon.noname-c con 0x7fe0700a4d80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 0x7fe070135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): picked mon.noname-b con 0x7fe0700a5870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe0701cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): picked mon.noname-a con 0x7fe07013d4d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe0700a5f60 con 0x7fe07013d4d0 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe07006d660 con 0x7fe0700a5870 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe0700a60e0 con 0x7fe0700a4d80 2024-09-15T21:20:38.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient(hunting): authenticate will time out at 1157.179087s 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 0x7fe070135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 0x7fe070135430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40056/0 (socket says 172.21.15.119:40056) 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 -- 172.21.15.119:0/705085329 learned_addr learned my addr 172.21.15.119:0/705085329 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): get_auth_request con 0x7fe0700a5870 auth_method 0 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a4d80 0x7fe070134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): get_auth_request con 0x7fe0700a4d80 auth_method 0 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe0701cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): get_auth_request con 0x7fe07013d4d0 auth_method 0 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient(hunting): handle_auth_done global_id 34513 payload 824 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 -- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a4d80 msgr2=0x7fe070134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a4d80 0x7fe070134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 -- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 msgr2=0x7fe0701cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe0701cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 monclient: found mon.noname-b 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 -- 172.21.15.119:0/705085329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe070070740 con 0x7fe0700a5870 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06e7fc640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 0x7fe070135430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe06400ec10 tx=0x7fe06400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064017070 con 0x7fe0700a5870 2024-09-15T21:20:38.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: dump: 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe06400cc30 con 0x7fe0700a5870 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064013470 con 0x7fe0700a5870 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe057fff640 10 monclient: dump: 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.417 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 5 monclient: authenticate success, global_id 34513 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: _renew_subs 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0700a1570 con 0x7fe0700a5870 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.378+0000 7fe06d7fa640 4 set_mon_vals no callback set 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 10 monclient: _renew_subs 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe070072aa0 con 0x7fe0700a5870 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe06401a070 con 0x7fe0700a5870 2024-09-15T21:20:38.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0700a5160 con 0x7fe0700a5870 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe057fff640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe05c03b9f0 0x7fe05c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7fe064050160 con 0x7fe0700a5870 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe06effd640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe05c03b9f0 0x7fe05c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe06effd640 10 monclient: get_auth_request con 0x7fe05c03b9f0 auth_method 0 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.382+0000 7fe06effd640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe05c03b9f0 0x7fe05c03deb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe060000f70 tx=0x7fe060009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.386+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe064013ae0 con 0x7fe0700a5870 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.386+0000 7fe057fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.386+0000 7fe057fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:38.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe075a0b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:38.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe075a0b640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe075a0b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fe0700712a0 con 0x7fe0700a5870 2024-09-15T21:20:38.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1500 (secure 0 0 0) 0x7fe064013610 con 0x7fe0700a5870 2024-09-15T21:20:38.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe057fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.558+0000 7fe057fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:38.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f239ed5a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:38.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f239ed5a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:38.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f239ed5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2398125aa0 con 0x7f23981410a0 2024-09-15T21:20:38.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f23957fa640 1 -- 172.21.15.17:0/86442356 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f2398073860 con 0x7f23981410a0 2024-09-15T21:20:38.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f23957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:38.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.617+0000 7f23957fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:38.624 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f237803b9f0 msgr2=0x7f237803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f237803b9f0 0x7f237803deb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2388007fd0 tx=0x7f2388009040 comp rx=0 tx=0).stop 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 10 monclient: shutdown 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 msgr2=0x7f239813c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 0x7f239813c060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2380018fb0 tx=0x7f2380019140 comp rx=0 tx=0).stop 2024-09-15T21:20:38.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 shutdown_connections 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f237803b9f0 0x7f237803deb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2398071330 0x7f2398071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23981410a0 0x7f239813c060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 --2- 172.21.15.17:0/86442356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23980a0890 0x7f23981d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 >> 172.21.15.17:0/86442356 conn(0x7f23980a66a0 msgr2=0x7f23981402c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 shutdown_connections 2024-09-15T21:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.621+0000 7f239ed5a640 1 -- 172.21.15.17:0/86442356 wait complete. 2024-09-15T21:20:38.640 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:20:38.640 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:20:38.641 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:20:38.641 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:20:38.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:38.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fe0700008e0 con 0x7fe0700a5870 2024-09-15T21:20:38.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe057fff640 1 -- 172.21.15.119:0/705085329 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1500 (secure 0 0 0) 0x7fe064011350 con 0x7fe0700a5870 2024-09-15T21:20:38.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe057fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe057fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe05c03b9f0 msgr2=0x7fe05c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe05c03b9f0 0x7fe05c03deb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe060000f70 tx=0x7fe060009040 comp rx=0 tx=0).stop 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 10 monclient: shutdown 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 msgr2=0x7fe070135430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 0x7fe070135430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe06400ec10 tx=0x7fe06400c520 comp rx=0 tx=0).stop 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 shutdown_connections 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe05c03b9f0 0x7fe05c03deb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07013d4d0 0x7fe0701cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0700a5870 0x7fe070135430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 --2- 172.21.15.119:0/705085329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0700a4d80 0x7fe070134ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.654+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 >> 172.21.15.119:0/705085329 conn(0x7fe0700a7a00 msgr2=0x7fe070137dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.658+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 shutdown_connections 2024-09-15T21:20:38.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.658+0000 7fe075a0b640 1 -- 172.21.15.119:0/705085329 wait complete. 2024-09-15T21:20:38.676 INFO:tasks.workunit.client.0.smithi119.stdout:-12 0 rack rack-for-test 2024-09-15T21:20:38.676 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:20:38.676 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep root-for-test 2024-09-15T21:20:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient: get_monmap_and_config 2024-09-15T21:20:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient: monmap: 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:38.766603+0000 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:38.766603+0000 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:38.769 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 Processor -- start 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 -- start start 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient: init 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0f340a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0f3409ffc0 0x7f0f3413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): picked mon.noname-a con 0x7f0f3409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f340a1380 0x7f0f3413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): picked mon.noname-c con 0x7f0f340a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:38.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f340a3130 con 0x7f0f340a0890 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0f34073df0 con 0x7f0f3409ffc0 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0f340a1bf0 con 0x7f0f340a1380 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f3b166640 10 monclient(hunting): authenticate will time out at 1162.162037s 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f38f02640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0f340a0890 0x7f0f340a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60794/0 (socket says 172.21.15.17:60794) 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f38f02640 1 -- 172.21.15.17:0/2149333512 learned_addr learned my addr 172.21.15.17:0/2149333512 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2149333512 >> v2:172.21.15.17:3300/0 conn(0x7f0f3409ffc0 0x7f0f3413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 1 --2- 172.21.15.17:0/2149333512 >> v2:172.21.15.119:3300/0 conn(0x7f0f340a1380 0x7f0f3413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): get_auth_request con 0x7f0f3409ffc0 auth_method 0 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): get_auth_request con 0x7f0f340a1380 auth_method 0 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2b7fe640 1 -- 172.21.15.17:0/2149333512 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006760343 0 0) 0x7f0f340a3130 con 0x7f0f340a0890 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f39703640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2b7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:20:38.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2b7fe640 10 monclient(hunting): my global_id is 5316 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2b7fe640 1 -- 172.21.15.17:0/2149333512 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f1c003530 con 0x7f0f340a0890 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_done global_id 5319 payload 274 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 1 -- 172.21.15.17:0/2149333512 >> v2:172.21.15.119:3300/0 conn(0x7f0f340a1380 msgr2=0x7f0f3413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2149333512 >> v2:172.21.15.119:3300/0 conn(0x7f0f340a1380 0x7f0f3413d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 1 -- 172.21.15.17:0/2149333512 >> v1:172.21.15.17:6789/0 conn(0x7f0f340a0890 legacy=0x7f0f340a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 1 monclient: found mon.noname-a 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.765+0000 7f0f2bfff640 1 -- 172.21.15.17:0/2149333512 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3409ccd0 con 0x7f0f3409ffc0 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2149333512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3413aac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0f240097c0 tx=0x7f0f2401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31f4f9caccccd40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 1 -- 172.21.15.17:0/2149333512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240b0070 con 0x7f0f3409ffc0 2024-09-15T21:20:38.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:38.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: dump: 2024-09-15T21:20:38.778 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.778 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.779 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.779 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.779 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.779 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.779 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.780 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 1 -- 172.21.15.17:0/2149333512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f240ab600 con 0x7f0f3409ffc0 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 1 -- 172.21.15.17:0/2149333512 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240ab9b0 con 0x7f0f3409ffc0 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 5 monclient: authenticate success, global_id 5319 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:38.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 4 set_mon_vals no callback set 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2b7fe640 10 monclient: dump: 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.782 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: shutdown 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- 172.21.15.17:0/2149333512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 msgr2=0x7f0f3413aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2149333512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3413aac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0f240097c0 tx=0x7f0f2401c320 comp rx=0 tx=0).stop 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- 172.21.15.17:0/2149333512 shutdown_connections 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2149333512 >> v2:172.21.15.119:3300/0 conn(0x7f0f340a1380 0x7f0f3413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2149333512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3413aac0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- 172.21.15.17:0/2149333512 >> 172.21.15.17:0/2149333512 conn(0x7f0f340a66a0 msgr2=0x7f0f340a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- 172.21.15.17:0/2149333512 shutdown_connections 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- 172.21.15.17:0/2149333512 wait complete. 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: monmap: 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:38.784 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 Processor -- start 2024-09-15T21:20:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- start start 2024-09-15T21:20:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: init 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): picked mon.noname-a con 0x7f0f3409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f340a0890 0x7f0f3411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): picked mon.noname-c con 0x7f0f340a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340a1380 0x7f0f3411cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): picked mon.noname-b con 0x7f0f340a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3413e0f0 con 0x7f0f3409ffc0 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3413df70 con 0x7f0f340a1380 2024-09-15T21:20:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3413e270 con 0x7f0f340a0890 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f3b166640 10 monclient(hunting): authenticate will time out at 1162.167341s 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3411c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47610/0 (socket says 172.21.15.17:47610) 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f340a0890 0x7f0f3411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 -- 172.21.15.17:0/2755842356 learned_addr learned my addr 172.21.15.17:0/2755842356 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): get_auth_request con 0x7f0f3409ffc0 auth_method 0 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): get_auth_request con 0x7f0f340a0890 auth_method 0 2024-09-15T21:20:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340a1380 0x7f0f3411cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): get_auth_request con 0x7f0f340a1380 auth_method 0 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f39703640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient(hunting): handle_auth_done global_id 5322 payload 824 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f340a0890 msgr2=0x7f0f3411ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f340a0890 0x7f0f3411ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340a1380 msgr2=0x7f0f3411cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340a1380 0x7f0f3411cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 monclient: found mon.noname-a 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f38f02640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3411fb70 con 0x7f0f3409ffc0 2024-09-15T21:20:38.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.769+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f340a0890 0x7f0f3411ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f38f02640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3411c4e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c4d0 tx=0x7f0f3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30017070 con 0x7f0f3409ffc0 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: dump: 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.797 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30005cf0 con 0x7f0f3409ffc0 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30007270 con 0x7f0f3409ffc0 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: dump: 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.798 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 5 monclient: authenticate success, global_id 5322 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 10 monclient: _renew_subs 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2affd640 4 set_mon_vals no callback set 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f34073050 con 0x7f0f3409ffc0 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 10 monclient: _renew_subs 2024-09-15T21:20:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3411d810 con 0x7f0f3409ffc0 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 0x7f0f34063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3402e9b0 con 0x7f0f3402e5d0 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0f3001a070 con 0x7f0f3409ffc0 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 0x7f0f34063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient: get_auth_request con 0x7f0f3402e5d0 auth_method 0 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f2003b9c0 0x7f0f2003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f0f3004f480 con 0x7f0f3409ffc0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_done global_id 5325 payload 824 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f340a0c70 con 0x7f0f3402e5d0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f39703640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f2003b9c0 0x7f0f2003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 0x7f0f34063e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f3409d0a0 tx=0x7f0f240190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f39703640 10 monclient: get_auth_request con 0x7f0f2003b9c0 auth_method 0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240bc070 con 0x7f0f3402e5d0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: dump: 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0f240b1040 con 0x7f0f3402e5d0 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 msgr2=0x7f0f34063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 0x7f0f34063e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f3409d0a0 tx=0x7f0f240190c0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 msgr2=0x7f0f34063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f297fa640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3402e5d0 0x7f0f34063e00 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f3409d0a0 tx=0x7f0f240190c0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.773+0000 7f0f39703640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f2003b9c0 0x7f0f2003de80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f340706b0 tx=0x7f0f2c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 0x7f0f341d9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34071e80 con 0x7f0f34064340 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 0x7f0f341d9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient: get_auth_request con 0x7f0f34064340 auth_method 0 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 10 monclient(hunting): handle_auth_done global_id 5328 payload 824 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 1 -- 172.21.15.17:0/2755842356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0f3406d020 con 0x7f0f34064340 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f2bfff640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 0x7f0f341d9750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f24019f50 tx=0x7f0f24009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240b0040 con 0x7f0f34064340 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 10 monclient: dump: 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f0f24003620 con 0x7f0f34064340 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 msgr2=0x7f0f341d9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 0x7f0f341d9750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f24019f50 tx=0x7f0f24009190 comp rx=0 tx=0).stop 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 msgr2=0x7f0f341d9750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:38.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f297fa640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f34064340 0x7f0f341d9750 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f24019f50 tx=0x7f0f24009190 comp rx=0 tx=0).stop 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 38, 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 204403, 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 7, 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:38.818 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f2003b9c0 msgr2=0x7f0f2003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f2003b9c0 0x7f0f2003de80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f340706b0 tx=0x7f0f2c00b040 comp rx=0 tx=0).stop 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 10 monclient: shutdown 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 msgr2=0x7f0f3411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 --2- 172.21.15.17:0/2755842356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3409ffc0 0x7f0f3411c4e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c4d0 tx=0x7f0f3000c990 comp rx=0 tx=0).stop 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f39703640 1 -- 172.21.15.17:0/2755842356 reap_dead start 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f39703640 1 -- 172.21.15.17:0/2755842356 reap_dead start 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 shutdown_connections 2024-09-15T21:20:38.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 >> 172.21.15.17:0/2755842356 conn(0x7f0f340a66a0 msgr2=0x7f0f340728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 shutdown_connections 2024-09-15T21:20:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.781+0000 7f0f3b166640 1 -- 172.21.15.17:0/2755842356 wait complete. 2024-09-15T21:20:38.822 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:20:38.822 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:20:38.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:20:38.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.782+0000 7f7f4773d640 10 monclient: get_monmap_and_config 2024-09-15T21:20:38.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.782+0000 7f7f4773d640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.782+0000 7f7f4773d640 10 monclient: monmap: 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:38.787639+0000 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:38.787639+0000 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.782+0000 7f7f4773d640 1 Processor -- start 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.782+0000 7f7f4773d640 1 -- start start 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.782+0000 7f7f4773d640 10 monclient: init 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7f400a4d80 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 0x7f7f4013cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient(hunting): picked mon.noname-b con 0x7f7f400a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7f4013d4d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f400a26e0 con 0x7f7f4013d4d0 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f400a6a30 con 0x7f7f400a4d80 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7f400a6330 con 0x7f7f400a5870 2024-09-15T21:20:38.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7f400a4d80 0x7f7f400a5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46884/0 (socket says 172.21.15.119:46884) 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 1 -- 172.21.15.119:0/325221259 learned_addr learned my addr 172.21.15.119:0/325221259 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient(hunting): authenticate will time out at 1157.584419s 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 0x7f7f4013cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567509771 0 0) 0x7f7f400a6a30 con 0x7f7f400a4d80 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): get_auth_request con 0x7f7f400a5870 auth_method 0 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): _negotiate 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): my global_id is 34522 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f2c003530 con 0x7f7f400a4d80 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 275249230 0 0) 0x7f7f2c003530 con 0x7f7f400a4d80 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 msgr2=0x7f7f4013cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 0x7f7f4013cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 >> v1:172.21.15.17:6789/0 conn(0x7f7f4013d4d0 legacy=0x7f7f4013f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f400a2400 con 0x7f7f400a4d80 2024-09-15T21:20:38.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 5 monclient: authenticate success, global_id 34522 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7f3c001700 con 0x7f7f400a4d80 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: dump: 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.834 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 >> v1:172.21.15.119:6789/0 conn(0x7f7f400a4d80 legacy=0x7f7f400a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 0x7f7f2c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): picked mon.c con 0x7f7f400a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2c003f50 0x7f7f2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): picked mon.b con 0x7f7f2c003f50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): picked mon.a con 0x7f7f2c004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c00bb60 con 0x7f7f2c004d50 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c00bce0 con 0x7f7f2c003f50 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c00beb0 con 0x7f7f400a5870 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2c003f50 0x7f7f2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 0x7f7f2c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): get_auth_request con 0x7f7f2c003f50 auth_method 0 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): get_auth_request con 0x7f7f400a5870 auth_method 0 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): get_auth_request con 0x7f7f2c004d50 auth_method 0 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:38.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f454d9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f45cda640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_done global_id 34522 payload 293 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 -- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 msgr2=0x7f7f2c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f400a5870 0x7f7f2c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 -- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f2c004d50 msgr2=0x7f7f2c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:38.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f2c004d50 0x7f7f2c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 monclient: found mon.b 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 -- 172.21.15.119:0/325221259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c003530 con 0x7f7f2c003f50 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2c003f50 0x7f7f2c004810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009ae0 tx=0x7f7f3001ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a85a6b80f1e8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300096b0 con 0x7f7f2c003f50 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: dump: 2024-09-15T21:20:38.839 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f300a6430 con 0x7f7f2c003f50 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f4773d640 4 set_mon_vals no callback set 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 1 -- 172.21.15.119:0/325221259 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300a6820 con 0x7f7f2c003f50 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.786+0000 7f7f37fff640 10 monclient: dump: 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.841 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: shutdown 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/325221259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2c003f50 msgr2=0x7f7f2c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 --2- 172.21.15.119:0/325221259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f2c003f50 0x7f7f2c004810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009ae0 tx=0x7f7f3001ca50 comp rx=0 tx=0).stop 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 1 -- 172.21.15.119:0/325221259 reap_dead start 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/325221259 shutdown_connections 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/325221259 >> 172.21.15.119:0/325221259 conn(0x7f7f400a7a00 msgr2=0x7f7f4006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/325221259 shutdown_connections 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/325221259 wait complete. 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: monmap: 2024-09-15T21:20:38.842 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 Processor -- start 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- start start 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: init 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f40072d20 0x7f7f40137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): picked mon.noname-a con 0x7f7f40072d20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 0x7f7f401cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): picked mon.noname-b con 0x7f7f40122040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f401383e0 0x7f7f401cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): picked mon.noname-c con 0x7f7f401383e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7f400a60e0 con 0x7f7f40072d20 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f400a5f60 con 0x7f7f40122040 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7f401417f0 con 0x7f7f401383e0 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient(hunting): authenticate will time out at 1157.589901s 2024-09-15T21:20:38.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 0x7f7f401cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 0x7f7f401cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40074/0 (socket says 172.21.15.119:40074) 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 -- 172.21.15.119:0/2216588163 learned_addr learned my addr 172.21.15.119:0/2216588163 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): get_auth_request con 0x7f7f40122040 auth_method 0 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f401383e0 0x7f7f401cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f40072d20 0x7f7f40137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): get_auth_request con 0x7f7f401383e0 auth_method 0 2024-09-15T21:20:38.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): get_auth_request con 0x7f7f40072d20 auth_method 0 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f45cda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f454d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient(hunting): handle_auth_done global_id 34525 payload 824 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 -- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f401383e0 msgr2=0x7f7f401cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f401383e0 0x7f7f401cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 -- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f40072d20 msgr2=0x7f7f40137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f40072d20 0x7f7f40137ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 monclient: found mon.noname-b 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 -- 172.21.15.119:0/2216588163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f400737d0 con 0x7f7f40122040 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f44cd8640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 0x7f7f401cf880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009ae0 tx=0x7f7f300197f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300be070 con 0x7f7f40122040 2024-09-15T21:20:38.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: dump: 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 5 monclient: authenticate success, global_id 34525 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3000f000 con 0x7f7f40122040 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300a6e00 con 0x7f7f40122040 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f35ffb640 10 monclient: dump: 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.849 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f36ffd640 4 set_mon_vals no callback set 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: _renew_subs 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f400a1570 con 0x7f7f40122040 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: _renew_subs 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.790+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f40123840 con 0x7f7f40122040 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f4773d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f4773d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f4773d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f401219a0 con 0x7f7f40122040 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7f300a7410 con 0x7f7f40122040 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f35ffb640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f2403b9f0 0x7f7f2403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f7f300e9be0 con 0x7f7f40122040 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f454d9640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f2403b9f0 0x7f7f2403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f454d9640 10 monclient: get_auth_request con 0x7f7f2403b9f0 auth_method 0 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.794+0000 7f7f454d9640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f2403b9f0 0x7f7f2403deb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004850 tx=0x7f7f3c001e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.798+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7f400a1570 con 0x7f7f40122040 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.798+0000 7f7f35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.798+0000 7f7f35ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:38.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient: get_monmap_and_config 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient: monmap: 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:38.942646+0000 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:38.942646+0000 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:38.945 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:38.946 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 Processor -- start 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 -- start start 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient: init 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01d809d400 0x7f01d809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): picked mon.noname-a con 0x7f01d809d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01d809dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d813cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): picked mon.noname-b con 0x7f01d8134d70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d80a3130 con 0x7f01d809dd20 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f01d80a1bc0 con 0x7f01d809d400 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01d80a3490 con 0x7f01d8134d70 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01dd98b640 10 monclient(hunting): authenticate will time out at 1162.338619s 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01d809d400 0x7f01d809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d813cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01d809dd20 0x7f01d8134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60800/0 (socket says 172.21.15.17:60800) 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d67fc640 1 -- 172.21.15.17:0/3337740420 learned_addr learned my addr 172.21.15.17:0/3337740420 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): get_auth_request con 0x7f01d809d400 auth_method 0 2024-09-15T21:20:38.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): get_auth_request con 0x7f01d8134d70 auth_method 0 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_done global_id 5337 payload 274 2024-09-15T21:20:38.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 1 -- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 msgr2=0x7f01d813cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d813cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 1 -- 172.21.15.17:0/3337740420 >> v1:172.21.15.17:6789/0 conn(0x7f01d809dd20 legacy=0x7f01d8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 1 monclient: found mon.noname-a 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:38.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.941+0000 7f01d6ffd640 1 -- 172.21.15.17:0/3337740420 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d809e7a0 con 0x7f01d809d400 2024-09-15T21:20:38.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d809d7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097c0 tx=0x7f01c801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8aa8311e25f87f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 1 -- 172.21.15.17:0/3337740420 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80b0070 con 0x7f01d809d400 2024-09-15T21:20:38.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:38.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: dump: 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 5 monclient: authenticate success, global_id 5337 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 1 -- 172.21.15.17:0/3337740420 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c80ab600 con 0x7f01d809d400 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 4 set_mon_vals no callback set 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 1 -- 172.21.15.17:0/3337740420 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80ab9d0 con 0x7f01d809d400 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d5ffb640 10 monclient: dump: 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.959 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: shutdown 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 msgr2=0x7f01d809d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 --2- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d809d7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097c0 tx=0x7f01c801c300 comp rx=0 tx=0).stop 2024-09-15T21:20:38.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- 172.21.15.17:0/3337740420 shutdown_connections 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 --2- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d813cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 --2- 172.21.15.17:0/3337740420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d809d7e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- 172.21.15.17:0/3337740420 >> 172.21.15.17:0/3337740420 conn(0x7f01d80a66a0 msgr2=0x7f01d80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- 172.21.15.17:0/3337740420 shutdown_connections 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- 172.21.15.17:0/3337740420 wait complete. 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: build_initial_monmap 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: monmap: 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.961 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 Processor -- start 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- start start 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: init 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d81cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): picked mon.noname-a con 0x7f01d809d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01d809dd20 0x7f01d81d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): picked mon.noname-b con 0x7f01d809dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d81d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): picked mon.noname-c con 0x7f01d8134d70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f01d809fba0 con 0x7f01d809d400 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f01d809fa20 con 0x7f01d809dd20 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01d813f830 con 0x7f01d8134d70 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01dd98b640 10 monclient(hunting): authenticate will time out at 1162.343714s 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d81cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d81d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d81cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47652/0 (socket says 172.21.15.17:47652) 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 1 -- 172.21.15.17:0/2556943421 learned_addr learned my addr 172.21.15.17:0/2556943421 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:38.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d81d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39774/0 (socket says 172.21.15.17:39774) 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): get_auth_request con 0x7f01d809d400 auth_method 0 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 10 monclient(hunting): get_auth_request con 0x7f01d8134d70 auth_method 0 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.945+0000 7f01d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01d809dd20 0x7f01d81d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 10 monclient(hunting): get_auth_request con 0x7f01d809dd20 auth_method 0 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 10 monclient(hunting): handle_auth_done global_id 44147 payload 824 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01d809dd20 msgr2=0x7f01d81d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01d809dd20 0x7f01d81d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 msgr2=0x7f01d81cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d81cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 monclient: found mon.noname-c 2024-09-15T21:20:38.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d8121710 con 0x7f01d8134d70 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d809d400 0x7f01d81cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d77fe640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d81d0690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01d000efc0 tx=0x7f01d000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0018020 con 0x7f01d8134d70 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: dump: 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.966 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: _finish_auth 0 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0014480 con 0x7f01d8134d70 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0014870 con 0x7f01d8134d70 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: dump: 2024-09-15T21:20:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 5 monclient: authenticate success, global_id 44147 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 10 monclient: _renew_subs 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d57fa640 4 set_mon_vals no callback set 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d80a4e40 con 0x7f01d8134d70 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 10 monclient: _renew_subs 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d80a5390 con 0x7f01d8134d70 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 0x7f01d8133c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01d809d7e0 con 0x7f01d81338a0 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 0x7f01d8133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient: get_auth_request con 0x7f01d81338a0 auth_method 0 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f01d001b020 con 0x7f01d8134d70 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_done global_id 44150 payload 824 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01d80008e0 con 0x7f01d81338a0 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01c403b9c0 0x7f01c403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f01d0051010 con 0x7f01d8134d70 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 0x7f01d8133c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f01c8019f90 tx=0x7f01c801aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80bb020 con 0x7f01d81338a0 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01c403b9c0 0x7f01c403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: dump: 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 10 monclient: get_auth_request con 0x7f01c403b9c0 auth_method 0 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01c80abe10 con 0x7f01d81338a0 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 msgr2=0x7f01d8133c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 0x7f01d8133c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f01c8019f90 tx=0x7f01c801aba0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 msgr2=0x7f01d8133c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01bffff640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81338a0 0x7f01d8133c80 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f01c8019f90 tx=0x7f01c801aba0 comp rx=0 tx=0).stop 2024-09-15T21:20:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.949+0000 7f01d67fc640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01c403b9c0 0x7f01c403de80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01cc007fd0 tx=0x7f01cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 0x7f01d8137aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01d813c260 con 0x7f01d81341c0 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 0x7f01d8137aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient: get_auth_request con 0x7f01d81341c0 auth_method 0 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 10 monclient(hunting): handle_auth_done global_id 44153 payload 824 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 1 -- 172.21.15.17:0/2556943421 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01d81345d0 con 0x7f01d81341c0 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d6ffd640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 0x7f01d8137aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c10 tx=0x7f01c8019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80b0040 con 0x7f01d81341c0 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 10 monclient: dump: 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f01c80b6870 con 0x7f01d81341c0 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 msgr2=0x7f01d8137aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 0x7f01d8137aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c10 tx=0x7f01c8019100 comp rx=0 tx=0).stop 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 msgr2=0x7f01d8137aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01bffff640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d81341c0 0x7f01d8137aa0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c10 tx=0x7f01c8019100 comp rx=0 tx=0).stop 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:38.976 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 38, 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7555, 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 7, 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:38.978 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:38.979 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:38.980 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:38.981 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:38.983 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:38.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:38.986 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:38.987 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:38.988 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:38.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:38.989 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:38.989 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:38.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01c403b9c0 msgr2=0x7f01c403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01c403b9c0 0x7f01c403de80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01cc007fd0 tx=0x7f01cc009040 comp rx=0 tx=0).stop 2024-09-15T21:20:38.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 10 monclient: shutdown 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 msgr2=0x7f01d81d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 --2- 172.21.15.17:0/2556943421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01d8134d70 0x7f01d81d0690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01d000efc0 tx=0x7f01d000c490 comp rx=0 tx=0).stop 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d77fe640 1 -- 172.21.15.17:0/2556943421 reap_dead start 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01d77fe640 1 -- 172.21.15.17:0/2556943421 reap_dead start 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 shutdown_connections 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 >> 172.21.15.17:0/2556943421 conn(0x7f01d80a66a0 msgr2=0x7f01d8071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 shutdown_connections 2024-09-15T21:20:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:38.957+0000 7f01dd98b640 1 -- 172.21.15.17:0/2556943421 wait complete. 2024-09-15T21:20:38.991 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:20:38.991 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:20:38.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:20:38.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f4773d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:38.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f4773d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f4773d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:38.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f7f40140a50 con 0x7f7f40122040 2024-09-15T21:20:38.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1500 (secure 0 0 0) 0x7f7f300ac020 con 0x7f7f40122040 2024-09-15T21:20:38.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:20:38.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:38.970+0000 7f7f35ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:39.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f4773d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:20:39.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f4773d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:39.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f4773d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f7f400a5160 con 0x7f7f40122040 2024-09-15T21:20:39.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f35ffb640 1 -- 172.21.15.119:0/2216588163 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1500 (secure 0 0 0) 0x7f7f30016760 con 0x7f7f40122040 2024-09-15T21:20:39.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f35ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:20:39.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.070+0000 7f7f35ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f2403b9f0 msgr2=0x7f7f2403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f2403b9f0 0x7f7f2403deb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004850 tx=0x7f7f3c001e30 comp rx=0 tx=0).stop 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 10 monclient: shutdown 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 msgr2=0x7f7f401cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 0x7f7f401cf880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009ae0 tx=0x7f7f300197f0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 shutdown_connections 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f2403b9f0 0x7f7f2403deb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f401383e0 0x7f7f401cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40122040 0x7f7f401cf880 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 --2- 172.21.15.119:0/2216588163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f40072d20 0x7f7f40137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 >> 172.21.15.119:0/2216588163 conn(0x7f7f400a7a00 msgr2=0x7f7f400a97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:39.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 shutdown_connections 2024-09-15T21:20:39.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.074+0000 7f7f4773d640 1 -- 172.21.15.119:0/2216588163 wait complete. 2024-09-15T21:20:39.094 INFO:tasks.workunit.client.0.smithi119.stdout:-14 0 root root-for-test 2024-09-15T21:20:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm host-for-test 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient: get_monmap_and_config 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient: build_initial_monmap 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient: monmap: 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:39.102275+0000 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:39.102275+0000 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:39.104 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:39.105 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:39.105 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:39.105 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:39.105 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 Processor -- start 2024-09-15T21:20:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 -- start start 2024-09-15T21:20:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient: init 2024-09-15T21:20:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2024-09-15T21:20:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce8409d400 0x7fce8409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): picked mon.noname-a con 0x7fce8409d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce8409dd20 0x7fce84134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): picked mon.noname-c con 0x7fce8409dd20 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fce84134ba0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce840a3130 con 0x7fce84134ba0 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fce840a1bc0 con 0x7fce8409d400 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fce840a3490 con 0x7fce8409dd20 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce8962f640 10 monclient(hunting): authenticate will time out at 1162.497575s 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce8409d400 0x7fce8409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce837fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fce84134ba0 0x7fce8413bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60804/0 (socket says 172.21.15.17:60804) 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce837fe640 1 -- 172.21.15.17:0/2993322462 learned_addr learned my addr 172.21.15.17:0/2993322462 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): get_auth_request con 0x7fce8409d400 auth_method 0 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 1 --2- 172.21.15.17:0/2993322462 >> v2:172.21.15.119:3300/0 conn(0x7fce8409dd20 0x7fce84134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): get_auth_request con 0x7fce8409dd20 auth_method 0 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient(hunting): handle_auth_done global_id 5346 payload 274 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 1 -- 172.21.15.17:0/2993322462 >> v2:172.21.15.119:3300/0 conn(0x7fce8409dd20 msgr2=0x7fce84134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 1 --2- 172.21.15.17:0/2993322462 >> v2:172.21.15.119:3300/0 conn(0x7fce8409dd20 0x7fce84134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 1 -- 172.21.15.17:0/2993322462 >> v1:172.21.15.17:6789/0 conn(0x7fce84134ba0 legacy=0x7fce8413bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 1 monclient: found mon.noname-a 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce81ffb640 1 -- 172.21.15.17:0/2993322462 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 857819556 0 0) 0x7fce840a3130 con 0x7fce84134ba0 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce82ffd640 1 -- 172.21.15.17:0/2993322462 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8409e7a0 con 0x7fce8409d400 2024-09-15T21:20:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.101+0000 7fce81ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:20:39.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 --2- 172.21.15.17:0/2993322462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8409d7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fce84074430 tx=0x7fce7801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2bffdc26a63d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 1 -- 172.21.15.17:0/2993322462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780b0070 con 0x7fce8409d400 2024-09-15T21:20:39.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:39.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: dump: 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:39.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 1 -- 172.21.15.17:0/2993322462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce780ab600 con 0x7fce8409d400 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 5 monclient: authenticate success, global_id 5346 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 1 -- 172.21.15.17:0/2993322462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780ab9d0 con 0x7fce8409d400 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 4 set_mon_vals no callback set 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce81ffb640 10 monclient: dump: 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.118 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: shutdown 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/2993322462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 msgr2=0x7fce8409d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 --2- 172.21.15.17:0/2993322462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8409d7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fce84074430 tx=0x7fce7801c340 comp rx=0 tx=0).stop 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/2993322462 shutdown_connections 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 --2- 172.21.15.17:0/2993322462 >> v2:172.21.15.119:3300/0 conn(0x7fce8409dd20 0x7fce84134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 --2- 172.21.15.17:0/2993322462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8409d7e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/2993322462 >> 172.21.15.17:0/2993322462 conn(0x7fce840a66a0 msgr2=0x7fce840a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/2993322462 shutdown_connections 2024-09-15T21:20:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/2993322462 wait complete. 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: build_initial_monmap 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: monmap: 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 Processor -- start 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- start start 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: init 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8412fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): picked mon.noname-a con 0x7fce8409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8409dd20 0x7fce84135a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): picked mon.noname-b con 0x7fce8409dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce84134ba0 0x7fce84135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): picked mon.noname-c con 0x7fce84134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fce8409ce50 con 0x7fce8409d400 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fce8409ccd0 con 0x7fce8409dd20 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fce8409fa20 con 0x7fce84134ba0 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient(hunting): authenticate will time out at 1162.502597s 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8412fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce84134ba0 0x7fce84135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8412fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47666/0 (socket says 172.21.15.17:47666) 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 -- 172.21.15.17:0/4292949245 learned_addr learned my addr 172.21.15.17:0/4292949245 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:39.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): get_auth_request con 0x7fce8409d400 auth_method 0 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): get_auth_request con 0x7fce84134ba0 auth_method 0 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8409dd20 0x7fce84135a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): get_auth_request con 0x7fce8409dd20 auth_method 0 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient(hunting): handle_auth_done global_id 5352 payload 824 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce84134ba0 msgr2=0x7fce84135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce84134ba0 0x7fce84135fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8409dd20 msgr2=0x7fce84135a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8409dd20 0x7fce84135a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 monclient: found mon.noname-a 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce841393a0 con 0x7fce8409d400 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce837fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce84134ba0 0x7fce84135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce82ffd640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8412fbc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce78018f50 tx=0x7fce780190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780b0040 con 0x7fce8409d400 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: dump: 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 5 monclient: authenticate success, global_id 5352 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce780b9070 con 0x7fce8409d400 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: _renew_subs 2024-09-15T21:20:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce84139680 con 0x7fce8409d400 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780b4650 con 0x7fce8409d400 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce6b7fe640 10 monclient: dump: 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: _renew_subs 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8409d0f0 con 0x7fce8409d400 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.105+0000 7fce80ff9640 4 set_mon_vals no callback set 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce8962f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce8962f640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 0x7fce84136510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fce8409d7e0 con 0x7fce84130100 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fce780b47f0 con 0x7fce8409d400 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 0x7fce84136510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient: get_auth_request con 0x7fce84130100 auth_method 0 2024-09-15T21:20:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce7003b9f0 0x7fce7003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 10 monclient(hunting): handle_auth_done global_id 34537 payload 824 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce8413c4c0 con 0x7fce84130100 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7fce780ea060 con 0x7fce8409d400 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce837fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce7003b9f0 0x7fce7003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce837fe640 10 monclient: get_auth_request con 0x7fce7003b9f0 auth_method 0 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce827fc640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 0x7fce84136510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce84073de0 tx=0x7fce74011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74019070 con 0x7fce84130100 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 10 monclient: dump: 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce837fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce7003b9f0 0x7fce7003deb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fce7c007dd0 tx=0x7fce7c007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fce74004240 con 0x7fce84130100 2024-09-15T21:20:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 msgr2=0x7fce84136510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 0x7fce84136510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce84073de0 tx=0x7fce74011e30 comp rx=0 tx=0).stop 2024-09-15T21:20:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 msgr2=0x7fce84136510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.109+0000 7fce6b7fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84130100 0x7fce84136510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce8962f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce8962f640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 0x7fce8406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce8962f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fce84136e60 con 0x7fce84136a50 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 0x7fce8406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient: get_auth_request con 0x7fce84136a50 auth_method 0 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.113+0000 7fce827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce827fc640 10 monclient(hunting): handle_auth_done global_id 34540 payload 824 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce827fc640 1 -- 172.21.15.17:0/4292949245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fce8406d020 con 0x7fce84136a50 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce827fc640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 0x7fce8406f980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce84073de0 tx=0x7fce740046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7401e650 con 0x7fce84136a50 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 10 monclient: dump: 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7fce740283a0 con 0x7fce84136a50 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 msgr2=0x7fce8406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 0x7fce8406f980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce84073de0 tx=0x7fce740046d0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 msgr2=0x7fce8406f980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce6b7fe640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce84136a50 0x7fce8406f980 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce84073de0 tx=0x7fce740046d0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 38, 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 51614, 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 7, 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:39.137 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:39.138 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:39.139 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:39.140 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:39.142 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:39.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:39.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:39.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:39.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.147 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:20:39.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:39.149 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce7003b9f0 msgr2=0x7fce7003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce7003b9f0 0x7fce7003deb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fce7c007dd0 tx=0x7fce7c007d20 comp rx=0 tx=0).stop 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 10 monclient: shutdown 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 msgr2=0x7fce8412fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 --2- 172.21.15.17:0/4292949245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8409d400 0x7fce8412fbc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce78018f50 tx=0x7fce780190e0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce837fe640 1 -- 172.21.15.17:0/4292949245 reap_dead start 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce837fe640 1 -- 172.21.15.17:0/4292949245 reap_dead start 2024-09-15T21:20:39.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 shutdown_connections 2024-09-15T21:20:39.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 >> 172.21.15.17:0/4292949245 conn(0x7fce840a66a0 msgr2=0x7fce84132840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:39.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 shutdown_connections 2024-09-15T21:20:39.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.117+0000 7fce8962f640 1 -- 172.21.15.17:0/4292949245 wait complete. 2024-09-15T21:20:39.152 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:20:39.152 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:20:39.152 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:20:39.152 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:20:39.152 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:20:39.153 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:20:39.153 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:20:39.165 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:20:39.165 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:20:39.166 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:39.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient: get_monmap_and_config 2024-09-15T21:20:39.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient: build_initial_monmap 2024-09-15T21:20:39.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient: monmap: 2024-09-15T21:20:39.207 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:39.206157+0000 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:39.206157+0000 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:39.208 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:39.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 1 Processor -- start 2024-09-15T21:20:39.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 1 -- start start 2024-09-15T21:20:39.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient: init 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f782409f640 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78240a0130 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f782409d9a0 0x7f782413d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient(hunting): picked mon.noname-c con 0x7f782409d9a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782409ddc0 con 0x7f78240a0130 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782409e880 con 0x7f782409f640 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f782409eb60 con 0x7f782409d9a0 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78294b2640 10 monclient(hunting): authenticate will time out at 1158.003346s 2024-09-15T21:20:39.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f782409d9a0 0x7f782413d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78227fc640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f78240a0130 legacy=0x7f7824139c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f7822ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f782409f640 0x7f782409fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46888/0 (socket says 172.21.15.119:46888) 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f7822ffd640 1 -- 172.21.15.119:0/1578724888 learned_addr learned my addr 172.21.15.119:0/1578724888 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): get_auth_request con 0x7f782409d9a0 auth_method 0 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient(hunting): handle_auth_done global_id 34543 payload 274 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 1 -- 172.21.15.119:0/1578724888 >> v1:172.21.15.119:6789/0 conn(0x7f782409f640 legacy=0x7f782409fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 1 -- 172.21.15.119:0/1578724888 >> v1:172.21.15.17:6789/0 conn(0x7f78240a0130 legacy=0x7f7824139c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:39.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:39.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 1 monclient: found mon.noname-c 2024-09-15T21:20:39.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:39.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.202+0000 7f78237fe640 1 -- 172.21.15.119:0/1578724888 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782409ee40 con 0x7f782409d9a0 2024-09-15T21:20:39.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78237fe640 1 --2- 172.21.15.119:0/1578724888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409d9a0 0x7f782413d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f781c009d90 tx=0x7f781c01c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a271f067431a4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 1 -- 172.21.15.119:0/1578724888 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c0b2070 con 0x7f782409d9a0 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: dump: 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.219 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 5 monclient: authenticate success, global_id 34543 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 1 -- 172.21.15.119:0/1578724888 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c0a5780 con 0x7f782409d9a0 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 1 -- 172.21.15.119:0/1578724888 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c0a5b50 con 0x7f782409d9a0 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 4 set_mon_vals no callback set 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f7821ffb640 10 monclient: dump: 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:39.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: shutdown 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 -- 172.21.15.119:0/1578724888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409d9a0 msgr2=0x7f782413d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 --2- 172.21.15.119:0/1578724888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409d9a0 0x7f782413d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f781c009d90 tx=0x7f781c01c8c0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 -- 172.21.15.119:0/1578724888 shutdown_connections 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 --2- 172.21.15.119:0/1578724888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409d9a0 0x7f782413d330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 -- 172.21.15.119:0/1578724888 >> 172.21.15.119:0/1578724888 conn(0x7f78240a2ae0 msgr2=0x7f78240a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 -- 172.21.15.119:0/1578724888 shutdown_connections 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 -- 172.21.15.119:0/1578724888 wait complete. 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: build_initial_monmap 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: monmap: 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:39.222 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 Processor -- start 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 1 -- start start 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: init 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.206+0000 7f78294b2640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f782409d9a0 0x7f782413bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): picked mon.noname-a con 0x7f782409d9a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 0x7f7824134ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): picked mon.noname-b con 0x7f782409f640 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f78240a0130 0x7f7824135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): picked mon.noname-c con 0x7f78240a0130 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7824140620 con 0x7f782409d9a0 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f78241404a0 con 0x7f782409f640 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f78241407a0 con 0x7f78240a0130 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient(hunting): authenticate will time out at 1158.008090s 2024-09-15T21:20:39.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f7822ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f782409d9a0 msgr2=0x7f782413bc00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 0x7f7824134ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f7822ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f782409d9a0 0x7f782413bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 0x7f7824134ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40136/0 (socket says 172.21.15.119:40136) 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 -- 172.21.15.119:0/163387512 learned_addr learned my addr 172.21.15.119:0/163387512 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): get_auth_request con 0x7f782409f640 auth_method 0 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78237fe640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f78240a0130 0x7f7824135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78237fe640 10 monclient(hunting): get_auth_request con 0x7f78240a0130 auth_method 0 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient(hunting): handle_auth_done global_id 34549 payload 824 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 -- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f78240a0130 msgr2=0x7f7824135010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f78240a0130 0x7f7824135010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 -- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f782409d9a0 msgr2=0x7f782413bc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f782409d9a0 0x7f782413bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 monclient: found mon.noname-b 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:39.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 -- 172.21.15.119:0/163387512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782413c5c0 con 0x7f782409f640 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78237fe640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f78240a0130 0x7f7824135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78227fc640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 0x7f7824134ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f781800b560 tx=0x7f781800ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818013020 con 0x7f782409f640 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: dump: 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818003df0 con 0x7f782409f640 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 5 monclient: authenticate success, global_id 34549 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient: _renew_subs 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782406f230 con 0x7f782409f640 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78180052b0 con 0x7f782409f640 2024-09-15T21:20:39.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 10 monclient: dump: 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:39.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient: _renew_subs 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7824120c00 con 0x7f782409f640 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78217fa640 4 set_mon_vals no callback set 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782409fa20 con 0x7f782409f640 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7818016020 con 0x7f782409f640 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f781003b970 0x7f781003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f781804ff80 con 0x7f782409f640 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f7822ffd640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f781003b970 0x7f781003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f7822ffd640 10 monclient: get_auth_request con 0x7f781003b970 auth_method 0 2024-09-15T21:20:39.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.210+0000 7f7822ffd640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f781003b970 0x7f781003de30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7814004820 tx=0x7f7814007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.214+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f781801d020 con 0x7f782409f640 2024-09-15T21:20:39.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.214+0000 7f780b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.214+0000 7f780b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient: get_monmap_and_config 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient: build_initial_monmap 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient: monmap: 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:39.324176+0000 2024-09-15T21:20:39.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:39.324176+0000 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:39.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 1 Processor -- start 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 1 -- start start 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient: init 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 0x7fe3a009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient(hunting): picked mon.noname-b con 0x7fe3a009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe3a009dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe3a0134d70 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:39.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a00a3490 con 0x7fe3a009dd20 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a00a0320 con 0x7fe3a0134d70 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a00a1bc0 con 0x7fe3a009d400 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe39ffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fe3a009dd20 legacy=0x7fe3a0134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a6f5d640 10 monclient(hunting): authenticate will time out at 1162.719687s 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 0x7fe3a009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 0x7fe3a009d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39824/0 (socket says 172.21.15.17:39824) 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 1 -- 172.21.15.17:0/2449988076 learned_addr learned my addr 172.21.15.17:0/2449988076 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): get_auth_request con 0x7fe3a009d400 auth_method 0 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.321+0000 7fe3a4cf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 10 monclient(hunting): handle_auth_done global_id 44162 payload 274 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 1 -- 172.21.15.17:0/2449988076 >> v1:172.21.15.119:6789/0 conn(0x7fe3a0134d70 legacy=0x7fe3a013c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 1 -- 172.21.15.17:0/2449988076 >> v1:172.21.15.17:6789/0 conn(0x7fe3a009dd20 legacy=0x7fe3a0134660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 10 monclient: _finish_hunting 0 2024-09-15T21:20:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 1 monclient: found mon.noname-b 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 1 -- 172.21.15.17:0/2449988076 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a009e7a0 con 0x7fe3a009d400 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a4cf9640 1 --2- 172.21.15.17:0/2449988076 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 0x7fe3a009d7e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe3900098a0 tx=0x7fe39001ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbd2dc97b1fc824 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 1 -- 172.21.15.17:0/2449988076 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900b0070 con 0x7fe3a009d400 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: dump: 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 1 -- 172.21.15.17:0/2449988076 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3900ab600 con 0x7fe3a009d400 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 1 -- 172.21.15.17:0/2449988076 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900ab9d0 con 0x7fe3a009d400 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 5 monclient: authenticate success, global_id 44162 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 4 set_mon_vals no callback set 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe39f7fe640 10 monclient: dump: 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 10 monclient: shutdown 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/2449988076 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 msgr2=0x7fe3a009d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/2449988076 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 0x7fe3a009d7e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe3900098a0 tx=0x7fe39001ce70 comp rx=0 tx=0).stop 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/2449988076 shutdown_connections 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/2449988076 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a009d400 0x7fe3a009d7e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/2449988076 >> 172.21.15.17:0/2449988076 conn(0x7fe3a00a66a0 msgr2=0x7fe3a00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:39.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/2449988076 shutdown_connections 2024-09-15T21:20:39.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/2449988076 wait complete. 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 10 monclient: build_initial_monmap 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.325+0000 7fe3a6f5d640 10 monclient: monmap: 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:39.343 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 Processor -- start 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- start start 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: init 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a009d400 0x7fe3a01cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): picked mon.noname-b con 0x7fe3a009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a009dd20 0x7fe3a01cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): picked mon.noname-a con 0x7fe3a009dd20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 0x7fe3a013a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): picked mon.noname-c con 0x7fe3a0134d70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a00a32b0 con 0x7fe3a009dd20 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a00a3130 con 0x7fe3a009d400 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a009ccd0 con 0x7fe3a0134d70 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:39.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient(hunting): authenticate will time out at 1162.724680s 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39ffff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a009dd20 msgr2=0x7fe3a01cc510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a009dd20 0x7fe3a01cc510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 0x7fe3a013a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 0x7fe3a013a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39834/0 (socket says 172.21.15.17:39834) 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 -- 172.21.15.17:0/3188013272 learned_addr learned my addr 172.21.15.17:0/3188013272 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): get_auth_request con 0x7fe3a0134d70 auth_method 0 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a009d400 0x7fe3a01cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 10 monclient(hunting): get_auth_request con 0x7fe3a009d400 auth_method 0 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient(hunting): handle_auth_done global_id 44165 payload 824 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 -- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a009d400 msgr2=0x7fe3a01cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a009d400 0x7fe3a01cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 -- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a009dd20 msgr2=0x7fe3a01cc510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a009dd20 0x7fe3a01cc510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient: _finish_hunting 0 2024-09-15T21:20:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 monclient: found mon.noname-c 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 -- 172.21.15.17:0/3188013272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a011cfb0 con 0x7fe3a0134d70 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a54fa640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 0x7fe3a013a490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe39800c7a0 tx=0x7fe39800cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398003b80 con 0x7fe3a0134d70 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: dump: 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:20:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe398003d20 con 0x7fe3a0134d70 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39800aa90 con 0x7fe3a0134d70 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 10 monclient: dump: 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39effd640 4 set_mon_vals no callback set 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 5 monclient: authenticate success, global_id 44165 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _renew_subs 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a006f2c0 con 0x7fe3a0134d70 2024-09-15T21:20:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _renew_subs 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a013cc90 con 0x7fe3a0134d70 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a009d7e0 con 0x7fe3a0134d70 2024-09-15T21:20:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe39800ac30 con 0x7fe3a0134d70 2024-09-15T21:20:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe38c03b9a0 0x7fe38c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7fe398050110 con 0x7fe3a0134d70 2024-09-15T21:20:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe38c03b9a0 0x7fe38c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.329+0000 7fe3a4cf9640 10 monclient: get_auth_request con 0x7fe38c03b9a0 auth_method 0 2024-09-15T21:20:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.333+0000 7fe3a4cf9640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe38c03b9a0 0x7fe38c03de60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ad000 tx=0x7fe390004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.337+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe398011c60 con 0x7fe3a0134d70 2024-09-15T21:20:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.337+0000 7fe39d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.337+0000 7fe39d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:39.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.374+0000 7f78294b2640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T21:20:39.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.374+0000 7f78294b2640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T21:20:39.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.374+0000 7f78294b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:39.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:39.374+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) -- 0x7f782406c460 con 0x7f782409f640 2024-09-15T21:20:39.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.565+0000 7fe3a6f5d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:39.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.565+0000 7fe3a6f5d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:39.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.565+0000 7fe3a6f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:39.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.565+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe3a013d490 con 0x7fe3a0134d70 2024-09-15T21:20:39.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe39d7fa640 1 -- 172.21.15.17:0/3188013272 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fe398005080 con 0x7fe3a0134d70 2024-09-15T21:20:39.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe39d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:39.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe39d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:39.572 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:39.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe38c03b9a0 msgr2=0x7fe38c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe38c03b9a0 0x7fe38c03de60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ad000 tx=0x7fe390004740 comp rx=0 tx=0).stop 2024-09-15T21:20:39.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 10 monclient: shutdown 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 msgr2=0x7fe3a013a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 0x7fe3a013a490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe39800c7a0 tx=0x7fe39800cc60 comp rx=0 tx=0).stop 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 shutdown_connections 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe38c03b9a0 0x7fe38c03de60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3a0134d70 0x7fe3a013a490 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a009dd20 0x7fe3a01cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 --2- 172.21.15.17:0/3188013272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a009d400 0x7fe3a01cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 >> 172.21.15.17:0/3188013272 conn(0x7fe3a00a66a0 msgr2=0x7fe3a00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 shutdown_connections 2024-09-15T21:20:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:39.569+0000 7fe3a6f5d640 1 -- 172.21.15.17:0/3188013272 wait complete. 2024-09-15T21:20:42.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:42.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:42.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:42.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:42.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:42.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:42.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:42.210+0000 7f780a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:42.593 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient: get_monmap_and_config 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient: build_initial_monmap 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient: monmap: 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:42.724034+0000 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:42.724034+0000 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:42.727 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 1 Processor -- start 2024-09-15T21:20:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 1 -- start start 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient: init 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f98840a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f98840a5580 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f9884140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient(hunting): picked mon.noname-b con 0x7f98840a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98840a2c30 con 0x7f98840a5580 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98840a19b0 con 0x7f98840a4a90 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f98837fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f98840a5580 legacy=0x7f988413cd10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f98840a1c90 con 0x7f98840a59d0 2024-09-15T21:20:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f988a87b640 10 monclient(hunting): authenticate will time out at 1166.119572s 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f9884140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f9884140440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39846/0 (socket says 172.21.15.17:39846) 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 1 -- 172.21.15.17:0/3496168970 learned_addr learned my addr 172.21.15.17:0/3496168970 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): get_auth_request con 0x7f98840a59d0 auth_method 0 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.721+0000 7f9888e18640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 10 monclient(hunting): handle_auth_done global_id 44168 payload 274 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 1 -- 172.21.15.17:0/3496168970 >> v1:172.21.15.119:6789/0 conn(0x7f98840a4a90 legacy=0x7f98840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 1 -- 172.21.15.17:0/3496168970 >> v1:172.21.15.17:6789/0 conn(0x7f98840a5580 legacy=0x7f988413cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 10 monclient: _finish_hunting 0 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 1 monclient: found mon.noname-b 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 1 -- 172.21.15.17:0/3496168970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98840a1f70 con 0x7f98840a59d0 2024-09-15T21:20:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9888e18640 1 --2- 172.21.15.17:0/3496168970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f9884140440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f987c009eb0 tx=0x7f987c01c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb191bc1ee7a1fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 1 -- 172.21.15.17:0/3496168970 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0ac020 con 0x7f98840a59d0 2024-09-15T21:20:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: dump: 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: _finish_auth 0 2024-09-15T21:20:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 5 monclient: authenticate success, global_id 44168 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 1 -- 172.21.15.17:0/3496168970 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c0a6410 con 0x7f98840a59d0 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 1 -- 172.21.15.17:0/3496168970 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a67e0 con 0x7f98840a59d0 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f9882ffd640 10 monclient: dump: 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 4 set_mon_vals no callback set 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 10 monclient: shutdown 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 -- 172.21.15.17:0/3496168970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 msgr2=0x7f9884140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 --2- 172.21.15.17:0/3496168970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f9884140440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f987c009eb0 tx=0x7f987c01c710 comp rx=0 tx=0).stop 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 -- 172.21.15.17:0/3496168970 shutdown_connections 2024-09-15T21:20:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 --2- 172.21.15.17:0/3496168970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f9884140440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 -- 172.21.15.17:0/3496168970 >> 172.21.15.17:0/3496168970 conn(0x7f98840aabb0 msgr2=0x7f98840a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 -- 172.21.15.17:0/3496168970 shutdown_connections 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 1 -- 172.21.15.17:0/3496168970 wait complete. 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 10 monclient: build_initial_monmap 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.725+0000 7f988a87b640 10 monclient: monmap: 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 Processor -- start 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- start start 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: init 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98840a4a90 0x7f9884137800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): picked mon.noname-b con 0x7f98840a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840a5580 0x7f9884137d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): picked mon.noname-a con 0x7f98840a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f988413cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): picked mon.noname-c con 0x7f98840a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9884143970 con 0x7f98840a5580 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98841437f0 con 0x7f98840a4a90 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9884143af0 con 0x7f98840a59d0 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient(hunting): authenticate will time out at 1166.124709s 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f988413cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f98837fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840a5580 msgr2=0x7f9884137d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f988413cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39862/0 (socket says 172.21.15.17:39862) 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 -- 172.21.15.17:0/3631497134 learned_addr learned my addr 172.21.15.17:0/3631497134 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): get_auth_request con 0x7f98840a59d0 auth_method 0 2024-09-15T21:20:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f98837fe640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840a5580 0x7f9884137d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient(hunting): handle_auth_done global_id 44171 payload 824 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 -- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98840a4a90 msgr2=0x7f9884137800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9883fff640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98840a4a90 0x7f9884137800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98840a4a90 0x7f9884137800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 -- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840a5580 msgr2=0x7f9884137d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840a5580 0x7f9884137d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient: _finish_hunting 0 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 monclient: found mon.noname-c 2024-09-15T21:20:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 -- 172.21.15.17:0/3631497134 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98841206e0 con 0x7f98840a59d0 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9883fff640 10 monclient: get_auth_request con 0x7f98840a4a90 auth_method 0 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9883fff640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98840a4a90 0x7f9884137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9888e18640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f988413cce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f987c01cbf0 tx=0x7f987c0194b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c009510 con 0x7f98840a59d0 2024-09-15T21:20:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: dump: 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: _finish_auth 0 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c0b1620 con 0x7f98840a59d0 2024-09-15T21:20:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0b19f0 con 0x7f98840a59d0 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 10 monclient: dump: 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 5 monclient: authenticate success, global_id 44171 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _renew_subs 2024-09-15T21:20:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9881ffb640 4 set_mon_vals no callback set 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9884121360 con 0x7f98840a59d0 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _renew_subs 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f988413fa70 con 0x7f98840a59d0 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98840a4e70 con 0x7f98840a59d0 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f987c0af020 con 0x7f98840a59d0 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f987003b970 0x7f987003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f987c0ea0a0 con 0x7f98840a59d0 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9883fff640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f987003b970 0x7f987003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9883fff640 10 monclient: get_auth_request con 0x7f987003b970 auth_method 0 2024-09-15T21:20:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.729+0000 7f9883fff640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f987003b970 0x7f987003de30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9874003e40 tx=0x7f9874009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.733+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f98840a2c30 con 0x7f98840a59d0 2024-09-15T21:20:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.733+0000 7f9880ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.733+0000 7f9880ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.965+0000 7f988a87b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.965+0000 7f988a87b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.965+0000 7f988a87b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:42.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.965+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9884069200 con 0x7f98840a59d0 2024-09-15T21:20:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f9880ff9640 1 -- 172.21.15.17:0/3631497134 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f987c0c2c60 con 0x7f98840a59d0 2024-09-15T21:20:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f9880ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:42.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f9880ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:42.972 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f987003b970 msgr2=0x7f987003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f987003b970 0x7f987003de30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9874003e40 tx=0x7f9874009040 comp rx=0 tx=0).stop 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 10 monclient: shutdown 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 msgr2=0x7f988413cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f988413cce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f987c01cbf0 tx=0x7f987c0194b0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 shutdown_connections 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f987003b970 0x7f987003de30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98840a59d0 0x7f988413cce0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840a5580 0x7f9884137d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 --2- 172.21.15.17:0/3631497134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98840a4a90 0x7f9884137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:42.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 >> 172.21.15.17:0/3631497134 conn(0x7f98840aabb0 msgr2=0x7f98840a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:42.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 shutdown_connections 2024-09-15T21:20:42.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:42.969+0000 7f988a87b640 1 -- 172.21.15.17:0/3631497134 wait complete. 2024-09-15T21:20:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:43.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:43.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:43.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:44.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:44.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:44.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:44.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:44.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:45.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:45.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:45.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:45.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:45.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:46.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient: get_monmap_and_config 2024-09-15T21:20:46.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient: build_initial_monmap 2024-09-15T21:20:46.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient: monmap: 2024-09-15T21:20:46.121 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:46.119674+0000 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:46.119674+0000 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:46.122 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:46.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 1 Processor -- start 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 1 -- start start 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient: init 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b8809d400 0x7f1b8809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient(hunting): picked mon.noname-c con 0x7f1b8809d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1b8809dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1b88134d70 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b880a3490 con 0x7f1b88134d70 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b880a0320 con 0x7f1b8809dd20 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1b880a1bc0 con 0x7f1b8809d400 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b8d758640 10 monclient(hunting): authenticate will time out at 1169.515584s 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b877fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f1b88134d70 legacy=0x7f1b8813c8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b86ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b8809d400 0x7f1b8809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b8809d400 0x7f1b8809d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:37934/0 (socket says 172.21.15.17:37934) 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.117+0000 7f1b867fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1b8809dd20 0x7f1b88134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:52628/0 (socket says 172.21.15.17:52628) 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1455181312 learned_addr learned my addr 172.21.15.17:0/1455181312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): get_auth_request con 0x7f1b8809d400 auth_method 0 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_done global_id 34555 payload 274 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1455181312 >> v1:172.21.15.119:6789/0 conn(0x7f1b8809dd20 legacy=0x7f1b88134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1455181312 >> v1:172.21.15.17:6789/0 conn(0x7f1b88134d70 legacy=0x7f1b8813c8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 1 monclient: found mon.noname-c 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.121+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1455181312 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8809e7a0 con 0x7f1b8809d400 2024-09-15T21:20:46.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 1 --2- 172.21.15.17:0/1455181312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b8809d400 0x7f1b8809d7e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b880a21c0 tx=0x7f1b7801f500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791ad19ebb06b96e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 1 -- 172.21.15.17:0/1455181312 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780b4070 con 0x7f1b8809d400 2024-09-15T21:20:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: dump: 2024-09-15T21:20:46.129 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 1 -- 172.21.15.17:0/1455181312 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b780a7a90 con 0x7f1b8809d400 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 1 -- 172.21.15.17:0/1455181312 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780a7e60 con 0x7f1b8809d400 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 5 monclient: authenticate success, global_id 34555 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 4 set_mon_vals no callback set 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b85ffb640 10 monclient: dump: 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.131 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: shutdown 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- 172.21.15.17:0/1455181312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b8809d400 msgr2=0x7f1b8809d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1455181312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b8809d400 0x7f1b8809d7e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b880a21c0 tx=0x7f1b7801f500 comp rx=0 tx=0).stop 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- 172.21.15.17:0/1455181312 shutdown_connections 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1455181312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b8809d400 0x7f1b8809d7e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- 172.21.15.17:0/1455181312 >> 172.21.15.17:0/1455181312 conn(0x7f1b880a66a0 msgr2=0x7f1b880a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- 172.21.15.17:0/1455181312 shutdown_connections 2024-09-15T21:20:46.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- 172.21.15.17:0/1455181312 wait complete. 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: build_initial_monmap 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: monmap: 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:46.137 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 Processor -- start 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- start start 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: init 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 0x7f1b881d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): picked mon.noname-c con 0x7f1b8809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b8809dd20 0x7f1b881d0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): picked mon.noname-a con 0x7f1b8809dd20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b88134d70 0x7f1b8811ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): picked mon.noname-b con 0x7f1b88134d70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880a32b0 con 0x7f1b8809dd20 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880a3130 con 0x7f1b88134d70 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8809ccd0 con 0x7f1b8809d400 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b8d758640 10 monclient(hunting): authenticate will time out at 1169.523616s 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b867fc640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b8809dd20 msgr2=0x7f1b881d0630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b8809dd20 0x7f1b881d0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 0x7f1b881d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 0x7f1b881d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39876/0 (socket says 172.21.15.17:39876) 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1229046220 learned_addr learned my addr 172.21.15.17:0/1229046220 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 10 monclient(hunting): get_auth_request con 0x7f1b8809d400 auth_method 0 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.125+0000 7f1b86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b877fe640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b88134d70 0x7f1b8811ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b877fe640 10 monclient(hunting): get_auth_request con 0x7f1b88134d70 auth_method 0 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 10 monclient(hunting): handle_auth_done global_id 44174 payload 824 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b88134d70 msgr2=0x7f1b8811ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b88134d70 0x7f1b8811ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b8809dd20 msgr2=0x7f1b881d0630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b8809dd20 0x7f1b881d0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 monclient: found mon.noname-c 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 -- 172.21.15.17:0/1229046220 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b881209d0 con 0x7f1b8809d400 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b877fe640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b88134d70 0x7f1b8811ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b86ffd640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 0x7f1b881d00f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b7801c6d0 tx=0x7f1b7801b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780bf070 con 0x7f1b8809d400 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: dump: 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:46.142 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b780b6050 con 0x7f1b8809d400 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 5 monclient: authenticate success, global_id 44174 2024-09-15T21:20:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 monclient: _renew_subs 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b88120cb0 con 0x7f1b8809d400 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780b4030 con 0x7f1b8809d400 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 10 monclient: dump: 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:46.144 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b857fa640 4 set_mon_vals no callback set 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 monclient: _renew_subs 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b881d0b70 con 0x7f1b8809d400 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8809d7e0 con 0x7f1b8809d400 2024-09-15T21:20:46.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1b780c2070 con 0x7f1b8809d400 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b7403b9a0 0x7f1b7403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 6570+0+0 (secure 0 0 0) 0x7f1b780ea720 con 0x7f1b8809d400 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b867fc640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b7403b9a0 0x7f1b7403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b867fc640 10 monclient: get_auth_request con 0x7f1b7403b9a0 auth_method 0 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.129+0000 7f1b867fc640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b7403b9a0 0x7f1b7403de60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b8811efc0 tx=0x7f1b7c007a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.133+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1b780187e0 con 0x7f1b8809d400 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.133+0000 7f1b6f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:46.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.133+0000 7f1b6f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:46.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:46.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:46.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:46.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:46.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:46.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:46.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:46.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:46.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:46.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1b8806f370 con 0x7f1b8809d400 2024-09-15T21:20:46.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b6f7fe640 1 -- 172.21.15.17:0/1229046220 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f1b780a7950 con 0x7f1b8809d400 2024-09-15T21:20:46.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b6f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:46.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b6f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:46.366 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b7403b9a0 msgr2=0x7f1b7403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b7403b9a0 0x7f1b7403de60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b8811efc0 tx=0x7f1b7c007a90 comp rx=0 tx=0).stop 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 10 monclient: shutdown 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 msgr2=0x7f1b881d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 0x7f1b881d00f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b7801c6d0 tx=0x7f1b7801b840 comp rx=0 tx=0).stop 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 shutdown_connections 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b7403b9a0 0x7f1b7403de60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b88134d70 0x7f1b8811ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b8809dd20 0x7f1b881d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.361+0000 7f1b8d758640 1 --2- 172.21.15.17:0/1229046220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b8809d400 0x7f1b881d00f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:46.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.365+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 >> 172.21.15.17:0/1229046220 conn(0x7f1b880a66a0 msgr2=0x7f1b880738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:46.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.365+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 shutdown_connections 2024-09-15T21:20:46.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:46.365+0000 7f1b8d758640 1 -- 172.21.15.17:0/1229046220 wait complete. 2024-09-15T21:20:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:47.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:47.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:47.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:48.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:48.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:48.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:49.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:49.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:49.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:49.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:49.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:49.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:49.384 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:20:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient: get_monmap_and_config 2024-09-15T21:20:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient: build_initial_monmap 2024-09-15T21:20:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient: monmap: 2024-09-15T21:20:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:49.515491+0000 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:49.515491+0000 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:49.518 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 Processor -- start 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 -- start start 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient: init 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2980a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 0x7ff29813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): picked mon.noname-a con 0x7ff29809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): picked mon.noname-c con 0x7ff2980a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2980a3130 con 0x7ff2980a0890 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff298073df0 con 0x7ff29809ffc0 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2980a1bf0 con 0x7ff2980a1380 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29fc3a640 10 monclient(hunting): authenticate will time out at 1172.911066s 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29d1d5640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 msgr2=0x7ff29813aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29d1d5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 0x7ff29813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:51140/0 (socket says 172.21.15.17:51140) 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 learned_addr learned my addr 172.21.15.17:0/3164251430 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request con 0x7ff2980a1380 auth_method 0 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:20:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.513+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:49.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request con 0x7ff2980a1380 auth_method 0 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 msgr2=0x7ff29813aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 0x7ff29813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:49.717+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:50.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 msgr2=0x7ff29813aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:50.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 0x7ff29813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request con 0x7ff2980a1380 auth_method 0 2024-09-15T21:20:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.121+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:50.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:50.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:50.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:50.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:50.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:50.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:50.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 msgr2=0x7ff29813aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:50.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 0x7ff29813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:20:50.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request con 0x7ff2980a1380 auth_method 0 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:50.925+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:20:51.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:51.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:51.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:51.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:51.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:51.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:52.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:52.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:52.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:52.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:52.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:52.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:52.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:52.210+0000 7f780a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): tick 2024-09-15T21:20:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:20:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:20:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:20:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 msgr2=0x7ff29813d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.119:3300/0 conn(0x7ff2980a1380 0x7ff29813d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 -- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 msgr2=0x7ff29813aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 --2- 172.21.15.17:0/3164251430 >> v2:172.21.15.17:3300/0 conn(0x7ff29809ffc0 0x7ff29813aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 -- 172.21.15.17:0/3164251430 >> v1:172.21.15.119:6789/0 conn(0x7ff2980a0890 legacy=0x7ff2980a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient: _add_conns ranks=[0,2] 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2840036d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): picked mon.noname-b con 0x7ff2840073c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 -- 172.21.15.17:0/3164251430 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29809ccd0 con 0x7ff2840036d0 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 1 -- 172.21.15.17:0/3164251430 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff298073df0 con 0x7ff2840073c0 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff2977fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d9d6640 1 -- 172.21.15.17:0/3164251430 >> v1:172.21.15.17:6789/0 conn(0x7ff2840036d0 legacy=0x7ff284005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request con 0x7ff2840073c0 auth_method 0 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.513+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.517+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.517+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:52.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.717+0000 7ff29d9d6640 1 -- 172.21.15.17:0/3164251430 >> v1:172.21.15.17:6789/0 conn(0x7ff2840036d0 legacy=0x7ff284005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request con 0x7ff2840073c0 auth_method 0 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:52.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:52.721+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:20:53.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.121+0000 7ff29d9d6640 1 -- 172.21.15.17:0/3164251430 >> v1:172.21.15.17:6789/0 conn(0x7ff2840036d0 legacy=0x7ff284005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:53.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:53.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request con 0x7ff2840073c0 auth_method 0 2024-09-15T21:20:53.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:53.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:20:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:20:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:20:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.125+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:20:53.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.210+0000 7f780a7fc640 10 monclient: tick 2024-09-15T21:20:53.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.210+0000 7f780a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:53.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.210+0000 7f780a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:53.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.830+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 removed item id -11 name 'host-for-test' from crush map v83) ==== 140+0+0 (secure 0 0 0) 0x7f781800fdc0 con 0x7f782409f640 2024-09-15T21:20:53.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.830+0000 7f780b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T21:20:53.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.830+0000 7f780b7fe640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host-for-test' from crush map 2024-09-15T21:20:53.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.925+0000 7ff29d9d6640 1 -- 172.21.15.17:0/3164251430 >> v1:172.21.15.17:6789/0 conn(0x7ff2840036d0 legacy=0x7ff284005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request con 0x7ff2840073c0 auth_method 0 2024-09-15T21:20:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient(hunting): handle_auth_done global_id 44177 payload 274 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 >> v1:172.21.15.17:6789/0 conn(0x7ff2840036d0 legacy=0x7ff284005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient: _finish_hunting 0 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 1 monclient: found mon.noname-b 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.929+0000 7ff29d1d5640 1 -- 172.21.15.17:0/3164251430 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28400b040 con 0x7ff2840073c0 2024-09-15T21:20:53.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.933+0000 7ff29d1d5640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009d30 tx=0x7ff28c01d1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7484382f572560f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.933+0000 7ff29c9d4640 1 -- 172.21.15.17:0/3164251430 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0b1070 con 0x7ff2840073c0 2024-09-15T21:20:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.933+0000 7ff29c9d4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: dump: 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: _finish_auth 0 2024-09-15T21:20:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: _check_auth_tickets 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 1 -- 172.21.15.17:0/3164251430 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28c0ac600 con 0x7ff2840073c0 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 5 monclient: authenticate success, global_id 44177 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 1 -- 172.21.15.17:0/3164251430 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0ac9d0 con 0x7ff2840073c0 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 4 set_mon_vals no callback set 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29c9d4640 10 monclient: dump: 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:53.943 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f78294b2640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f78294b2640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f78294b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) -- 0x7f78241cf840 con 0x7f782409f640 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f780b7fe640 1 -- 172.21.15.119:0/163387512 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 device 'host-for-test' does not appear in the crush map v83) ==== 140+0+0 (secure 0 0 0) 0x7f78180208e0 con 0x7f782409f640 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f780b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.938+0000 7f780b7fe640 10 monclient: _finish_command 3 = system:0 device 'host-for-test' does not appear in the crush map 2024-09-15T21:20:53.945 INFO:tasks.workunit.client.0.smithi119.stderr:device 'host-for-test' does not appear in the crush map 2024-09-15T21:20:53.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f781003b970 msgr2=0x7f781003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:53.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f781003b970 0x7f781003de30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7814004820 tx=0x7f7814007b50 comp rx=0 tx=0).stop 2024-09-15T21:20:53.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 10 monclient: shutdown 2024-09-15T21:20:53.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 msgr2=0x7f7824134ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 0x7f7824134ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f781800b560 tx=0x7f781800ba20 comp rx=0 tx=0).stop 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 shutdown_connections 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f781003b970 0x7f781003de30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f78240a0130 0x7f7824135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f782409f640 0x7f7824134ad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 --2- 172.21.15.119:0/163387512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f782409d9a0 0x7f782413bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 >> 172.21.15.119:0/163387512 conn(0x7f78240a2ae0 msgr2=0x7f78240a1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 shutdown_connections 2024-09-15T21:20:53.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:53.942+0000 7f78294b2640 1 -- 172.21.15.119:0/163387512 wait complete. 2024-09-15T21:20:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 10 monclient: shutdown 2024-09-15T21:20:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 1 -- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 msgr2=0x7ff284009860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/3164251430 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2840073c0 0x7ff284009860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009d30 tx=0x7ff28c01d1e0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29e1d7640 1 -- 172.21.15.17:0/3164251430 reap_dead start 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 1 -- 172.21.15.17:0/3164251430 shutdown_connections 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 1 -- 172.21.15.17:0/3164251430 >> 172.21.15.17:0/3164251430 conn(0x7ff2980a66a0 msgr2=0x7ff2980a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 1 -- 172.21.15.17:0/3164251430 shutdown_connections 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 1 -- 172.21.15.17:0/3164251430 wait complete. 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 10 monclient: build_initial_monmap 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.937+0000 7ff29fc3a640 10 monclient: monmap: 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 Processor -- start 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 -- start start 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: init 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:20:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff298135a30 0x7ff298135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): picked mon.noname-b con 0x7ff298135a30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2981cf610 0x7ff29809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): picked mon.noname-a con 0x7ff2981cf610 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 0x7ff29809ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): picked mon.noname-c con 0x7ff29809d940 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff298140100 con 0x7ff2981cf610 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff29813ff80 con 0x7ff298135a30 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff298140280 con 0x7ff29809d940 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d1d5640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2981cf610 msgr2=0x7ff29809d400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient(hunting): authenticate will time out at 1177.338143s 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d1d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2981cf610 0x7ff29809d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 0x7ff29809ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 0x7ff29809ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58500/0 (socket says 172.21.15.17:58500) 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 -- 172.21.15.17:0/2633678355 learned_addr learned my addr 172.21.15.17:0/2633678355 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request con 0x7ff29809d940 auth_method 0 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d9d6640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff298135a30 0x7ff298135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d9d6640 10 monclient(hunting): get_auth_request con 0x7ff298135a30 auth_method 0 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d9d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d9d6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d9d6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient(hunting): handle_auth_done global_id 44180 payload 824 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 -- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff298135a30 msgr2=0x7ff298135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff298135a30 0x7ff298135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 -- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2981cf610 msgr2=0x7ff29809d400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2981cf610 0x7ff29809d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient: _finish_hunting 0 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 monclient: found mon.noname-c 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 -- 172.21.15.17:0/2633678355 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29809ed90 con 0x7ff29809d940 2024-09-15T21:20:53.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29d9d6640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff298135a30 0x7ff298135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29e1d7640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 0x7ff29809ddc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff288009100 tx=0x7ff288014a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28801d070 con 0x7ff29809d940 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: dump: 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2880168a0 con 0x7ff29809d940 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 5 monclient: authenticate success, global_id 44180 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: _renew_subs 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff298121b40 con 0x7ff29809d940 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28800f040 con 0x7ff29809d940 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2967fc640 10 monclient: dump: 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff2977fe640 4 set_mon_vals no callback set 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: _renew_subs 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.941+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff298120fb0 con 0x7ff29809d940 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29fc3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29fc3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29fc3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2980a0c70 con 0x7ff29809d940 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff288020070 con 0x7ff29809d940 2024-09-15T21:20:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff2967fc640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28403c8a0 0x7ff28403ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 6387+0+0 (secure 0 0 0) 0x7ff288050950 con 0x7ff29809d940 2024-09-15T21:20:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29d9d6640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28403c8a0 0x7ff28403ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29d9d6640 10 monclient: get_auth_request con 0x7ff28403c8a0 auth_method 0 2024-09-15T21:20:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.945+0000 7ff29d9d6640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28403c8a0 0x7ff28403ed60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff2900012f0 tx=0x7ff290009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.949+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff288027020 con 0x7ff29809d940 2024-09-15T21:20:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.949+0000 7ff2967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:53.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:53.949+0000 7ff2967fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:53.963 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm rack-for-test 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient: get_monmap_and_config 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient: build_initial_monmap 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient: monmap: 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:54.077863+0000 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:54.077863+0000 2024-09-15T21:20:54.079 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:54.080 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 Processor -- start 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 -- start start 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient: init 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96b809f640 0x7f96b809d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): picked mon.noname-c con 0x7f96b809f640 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:54.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f96b809fa20 0x7f96b809feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): picked mon.noname-a con 0x7f96b809fa20 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f96b81347b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b809e880 con 0x7f96b81347b0 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f96b809eb60 con 0x7f96b809fa20 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f96b809ee40 con 0x7f96b809f640 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bf2c2640 10 monclient(hunting): authenticate will time out at 1172.874525s 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd85f640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f96b81347b0 legacy=0x7f96b8134bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bc85d640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f96b809fa20 msgr2=0x7f96b809feb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:54.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bc85d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f96b809fa20 0x7f96b809feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96b809f640 0x7f96b809d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96b809f640 0x7f96b809d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35594/0 (socket says 172.21.15.119:35594) 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 -- 172.21.15.119:0/774212269 learned_addr learned my addr 172.21.15.119:0/774212269 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): get_auth_request con 0x7f96b809f640 auth_method 0 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient(hunting): handle_auth_done global_id 34564 payload 274 2024-09-15T21:20:54.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 -- 172.21.15.119:0/774212269 >> v2:172.21.15.17:3300/0 conn(0x7f96b809fa20 msgr2=0x7f96b809feb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 --2- 172.21.15.119:0/774212269 >> v2:172.21.15.17:3300/0 conn(0x7f96b809fa20 0x7f96b809feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 -- 172.21.15.119:0/774212269 >> v1:172.21.15.17:6789/0 conn(0x7f96b81347b0 legacy=0x7f96b8134bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 monclient: found mon.noname-c 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.074+0000 7f96bd05e640 1 -- 172.21.15.119:0/774212269 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b80a03f0 con 0x7f96b809f640 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bd05e640 1 --2- 172.21.15.119:0/774212269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809f640 0x7f96b809d9a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f96b8076830 tx=0x7f96b401eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c29435de17830a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 1 -- 172.21.15.119:0/774212269 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b40b4070 con 0x7f96b809f640 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: dump: 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: _finish_auth 0 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 5 monclient: authenticate success, global_id 34564 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 1 -- 172.21.15.119:0/774212269 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b40a78f0 con 0x7f96b809f640 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 1 -- 172.21.15.119:0/774212269 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b40a7cc0 con 0x7f96b809f640 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 4 set_mon_vals no callback set 2024-09-15T21:20:54.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96affff640 10 monclient: dump: 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 10 monclient: shutdown 2024-09-15T21:20:54.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 -- 172.21.15.119:0/774212269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809f640 msgr2=0x7f96b809d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/774212269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809f640 0x7f96b809d9a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f96b8076830 tx=0x7f96b401eb60 comp rx=0 tx=0).stop 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 -- 172.21.15.119:0/774212269 shutdown_connections 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/774212269 >> v2:172.21.15.17:3300/0 conn(0x7f96b809fa20 0x7f96b809feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/774212269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809f640 0x7f96b809d9a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 -- 172.21.15.119:0/774212269 >> 172.21.15.119:0/774212269 conn(0x7f96b80a2ae0 msgr2=0x7f96b80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 -- 172.21.15.119:0/774212269 shutdown_connections 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 1 -- 172.21.15.119:0/774212269 wait complete. 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 10 monclient: build_initial_monmap 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.078+0000 7f96bf2c2640 10 monclient: monmap: 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:54.094 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 Processor -- start 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- start start 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: init 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:54.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b809f640 0x7f96b81cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): picked mon.noname-c con 0x7f96b809f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 0x7f96b8133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): picked mon.noname-b con 0x7f96b809fa20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b81347b0 0x7f96b8133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): picked mon.noname-a con 0x7f96b81347b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f96b80a1c00 con 0x7f96b81347b0 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f96b80a1a80 con 0x7f96b809fa20 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f96b80a1d80 con 0x7f96b809f640 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:54.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient(hunting): authenticate will time out at 1172.881200s 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd85f640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b81347b0 msgr2=0x7f96b8133d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 0x7f96b8133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd85f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b81347b0 0x7f96b8133d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 0x7f96b8133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35608/0 (socket says 172.21.15.119:35608) 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 -- 172.21.15.119:0/353572094 learned_addr learned my addr 172.21.15.119:0/353572094 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): get_auth_request con 0x7f96b809fa20 auth_method 0 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b809f640 0x7f96b81cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 10 monclient(hunting): get_auth_request con 0x7f96b809f640 auth_method 0 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient(hunting): handle_auth_done global_id 34567 payload 824 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 -- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b809f640 msgr2=0x7f96b81cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b809f640 0x7f96b81cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 -- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b81347b0 msgr2=0x7f96b8133d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b81347b0 0x7f96b8133d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 monclient: found mon.noname-b 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:54.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 -- 172.21.15.119:0/353572094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b809d1d0 con 0x7f96b809fa20 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bc85d640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 0x7f96b8133830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96a800b560 tx=0x7f96a800ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8013020 con 0x7f96b809fa20 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: dump: 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.099 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 5 monclient: authenticate success, global_id 34567 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a8003df0 con 0x7f96b809fa20 2024-09-15T21:20:54.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _renew_subs 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b8122200 con 0x7f96b809fa20 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80052b0 con 0x7f96b809fa20 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 10 monclient: dump: 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:54.101 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _renew_subs 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96af7fe640 4 set_mon_vals no callback set 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b81cffb0 con 0x7f96b809fa20 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b8121db0 con 0x7f96b809fa20 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f96a8016020 con 0x7f96b809fa20 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f969c03b9a0 0x7f969c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6387+0+0 (secure 0 0 0) 0x7f96a804fe80 con 0x7f96b809fa20 2024-09-15T21:20:54.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f969c03b9a0 0x7f969c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.082+0000 7f96bd05e640 10 monclient: get_auth_request con 0x7f969c03b9a0 auth_method 0 2024-09-15T21:20:54.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.086+0000 7f96bd05e640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f969c03b9a0 0x7f969c03de60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b1000 tx=0x7f96b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.090+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f96b8121db0 con 0x7f96b809fa20 2024-09-15T21:20:54.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.090+0000 7f96adffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.090+0000 7f96adffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:20:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff2981217b0 con 0x7ff29809d940 2024-09-15T21:20:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff2967fc640 1 -- 172.21.15.17:0/2633678355 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7ff288020360 con 0x7ff29809d940 2024-09-15T21:20:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff2967fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:20:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff2967fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:54.197 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:20:54.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28403c8a0 msgr2=0x7ff28403ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28403c8a0 0x7ff28403ed60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff2900012f0 tx=0x7ff290009040 comp rx=0 tx=0).stop 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 10 monclient: shutdown 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 msgr2=0x7ff29809ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 0x7ff29809ddc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff288009100 tx=0x7ff288014a30 comp rx=0 tx=0).stop 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 shutdown_connections 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28403c8a0 0x7ff28403ed60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff29809d940 0x7ff29809ddc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2981cf610 0x7ff29809d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 --2- 172.21.15.17:0/2633678355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff298135a30 0x7ff298135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 >> 172.21.15.17:0/2633678355 conn(0x7ff2980a66a0 msgr2=0x7ff2980a4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 shutdown_connections 2024-09-15T21:20:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.193+0000 7ff29fc3a640 1 -- 172.21.15.17:0/2633678355 wait complete. 2024-09-15T21:20:54.217 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:20:54.217 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:20:54.218 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:20:54.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.262+0000 7f96bf2c2640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T21:20:54.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.262+0000 7f96bf2c2640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T21:20:54.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.266+0000 7f96bf2c2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.266+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) -- 0x7f96b809d9a0 con 0x7f96b809fa20 2024-09-15T21:20:54.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient: get_monmap_and_config 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient: build_initial_monmap 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient: monmap: 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:54.342928+0000 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:54.342928+0000 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:54.345 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:54.346 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:54.346 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:54.346 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:54.346 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 Processor -- start 2024-09-15T21:20:54.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 -- start start 2024-09-15T21:20:54.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient: init 2024-09-15T21:20:54.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:54.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:20:54.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f223c133c30 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f223c134720 0x7f223c0a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): picked mon.noname-a con 0x7f223c134720 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f223c139160 0x7f223c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): picked mon.noname-c con 0x7f223c139160 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223b7fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f223c133c30 legacy=0x7f223c134010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f223c0a29a0 con 0x7f223c133c30 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f223c09f6d0 con 0x7f223c134720 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f223c0a25f0 con 0x7f223c139160 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f2241ef3640 10 monclient(hunting): authenticate will time out at 1177.739267s 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223affd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f223c134720 msgr2=0x7f223c0a03e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:54.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f223c134720 0x7f223c0a03e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f223c139160 0x7f223c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f223c139160 0x7f223c13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:51188/0 (socket says 172.21.15.17:51188) 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 1 -- 172.21.15.17:0/1047098037 learned_addr learned my addr 172.21.15.17:0/1047098037 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): get_auth_request con 0x7f223c139160 auth_method 0 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.341+0000 7f223bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 10 monclient(hunting): handle_auth_done global_id 34570 payload 274 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 1 -- 172.21.15.17:0/1047098037 >> v2:172.21.15.17:3300/0 conn(0x7f223c134720 msgr2=0x7f223c0a03e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:54.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 1 --2- 172.21.15.17:0/1047098037 >> v2:172.21.15.17:3300/0 conn(0x7f223c134720 0x7f223c0a03e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 1 -- 172.21.15.17:0/1047098037 >> v1:172.21.15.17:6789/0 conn(0x7f223c133c30 legacy=0x7f223c134010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:20:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 1 monclient: found mon.noname-c 2024-09-15T21:20:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 1 -- 172.21.15.17:0/1047098037 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c09f110 con 0x7f223c139160 2024-09-15T21:20:54.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223bfff640 1 --2- 172.21.15.17:0/1047098037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c139160 0x7f223c13b550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f223c09f850 tx=0x7f223401d9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621bd73a1b3091a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 1 -- 172.21.15.17:0/1047098037 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223400a2d0 con 0x7f223c139160 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: dump: 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 1 -- 172.21.15.17:0/1047098037 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22340a67e0 con 0x7f223c139160 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 1 -- 172.21.15.17:0/1047098037 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340a6bb0 con 0x7f223c139160 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f223a7fc640 10 monclient: dump: 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.354 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f2241ef3640 5 monclient: authenticate success, global_id 34570 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f2241ef3640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.345+0000 7f2241ef3640 4 set_mon_vals no callback set 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: shutdown 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- 172.21.15.17:0/1047098037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c139160 msgr2=0x7f223c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1047098037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c139160 0x7f223c13b550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f223c09f850 tx=0x7f223401d9a0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- 172.21.15.17:0/1047098037 shutdown_connections 2024-09-15T21:20:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1047098037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c139160 0x7f223c13b550 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1047098037 >> v2:172.21.15.17:3300/0 conn(0x7f223c134720 0x7f223c0a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- 172.21.15.17:0/1047098037 >> 172.21.15.17:0/1047098037 conn(0x7f223c0a2fb0 msgr2=0x7f223c0a3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- 172.21.15.17:0/1047098037 shutdown_connections 2024-09-15T21:20:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- 172.21.15.17:0/1047098037 wait complete. 2024-09-15T21:20:54.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: build_initial_monmap 2024-09-15T21:20:54.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: monmap: 2024-09-15T21:20:54.361 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 Processor -- start 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- start start 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: init 2024-09-15T21:20:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c133c30 0x7f223c130dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): picked mon.noname-c con 0x7f223c133c30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c134720 0x7f223c09b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): picked mon.noname-b con 0x7f223c134720 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c139160 0x7f223c09be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): picked mon.noname-a con 0x7f223c139160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f223c13de30 con 0x7f223c139160 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f223c13dcb0 con 0x7f223c134720 2024-09-15T21:20:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f223c13dfb0 con 0x7f223c133c30 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient(hunting): authenticate will time out at 1177.746540s 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c133c30 0x7f223c130dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223bfff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c139160 msgr2=0x7f223c09be60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c133c30 0x7f223c130dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58512/0 (socket says 172.21.15.17:58512) 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 -- 172.21.15.17:0/1802050635 learned_addr learned my addr 172.21.15.17:0/1802050635 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): get_auth_request con 0x7f223c133c30 auth_method 0 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223bfff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c139160 0x7f223c09be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223affd640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c134720 0x7f223c09b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223affd640 10 monclient(hunting): get_auth_request con 0x7f223c134720 auth_method 0 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient(hunting): handle_auth_done global_id 44186 payload 824 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c134720 msgr2=0x7f223c09b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223c134720 0x7f223c09b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c139160 msgr2=0x7f223c09be60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c139160 0x7f223c09be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c13b6f0 con 0x7f223c133c30 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f223b7fe640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c133c30 0x7f223c130dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b530 tx=0x7f222c00b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c013020 con 0x7f223c133c30 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: dump: 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.366 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: _finish_auth 0 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222c003df0 con 0x7f223c133c30 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c0052b0 con 0x7f223c133c30 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2223fff640 10 monclient: dump: 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2239ffb640 4 set_mon_vals no callback set 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 5 monclient: authenticate success, global_id 44186 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: _renew_subs 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.349+0000 7f2241ef3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c0a56a0 con 0x7f223c133c30 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 10 monclient: _renew_subs 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c12f570 con 0x7f223c133c30 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 0x7f223c09c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f223c09d700 con 0x7f223c02e5d0 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 0x7f223c09c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient: get_auth_request con 0x7f223c02e5d0 auth_method 0 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f222c016020 con 0x7f223c133c30 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 10 monclient(hunting): handle_auth_done global_id 44189 payload 824 2024-09-15T21:20:54.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f222803b970 0x7f222803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f223c02e9b0 con 0x7f223c02e5d0 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 6387+0+0 (secure 0 0 0) 0x7f222c04fde0 con 0x7f223c133c30 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223affd640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 0x7f223c09c3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f223c1335c0 tx=0x7f223000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223bfff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f222803b970 0x7f222803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300039c0 con 0x7f223c02e5d0 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 10 monclient: dump: 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.371 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223bfff640 10 monclient: get_auth_request con 0x7f222803b970 auth_method 0 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2230017070 con 0x7f223c02e5d0 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:54.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 msgr2=0x7f223c09c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 0x7f223c09c3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f223c1335c0 tx=0x7f223000db90 comp rx=0 tx=0).stop 2024-09-15T21:20:54.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 msgr2=0x7f223c09c3a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:54.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f2223fff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c02e5d0 0x7f223c09c3a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.353+0000 7f223bfff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f222803b970 0x7f222803de30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22340af000 tx=0x7f22340a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 0x7f223c064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f223c0a03e0 con 0x7f223c063e00 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 0x7f223c064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient: get_auth_request con 0x7f223c063e00 auth_method 0 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 10 monclient(hunting): handle_auth_done global_id 44192 payload 824 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 1 -- 172.21.15.17:0/1802050635 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f223c12cc80 con 0x7f223c063e00 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223affd640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 0x7f223c064210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2230016000 tx=0x7f22300059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300054e0 con 0x7f223c063e00 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 10 monclient: dump: 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4700 (secure 0 0 0) 0x7f22300054e0 con 0x7f223c063e00 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 msgr2=0x7f223c064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 0x7f223c064210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2230016000 tx=0x7f22300059e0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 msgr2=0x7f223c064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2223fff640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c063e00 0x7f223c064210 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2230016000 tx=0x7f22300059e0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22959, 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:54.378 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:54.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:54.380 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:54.381 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:54.382 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:54.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.384 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:54.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:54.386 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:54.387 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:54.388 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:54.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.390 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:54.391 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f222803b970 msgr2=0x7f222803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f222803b970 0x7f222803de30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22340af000 tx=0x7f22340a8070 comp rx=0 tx=0).stop 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 10 monclient: shutdown 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c133c30 msgr2=0x7f223c130dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 --2- 172.21.15.17:0/1802050635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f223c133c30 0x7f223c130dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b530 tx=0x7f222c00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223bfff640 1 -- 172.21.15.17:0/1802050635 reap_dead start 2024-09-15T21:20:54.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f223bfff640 1 -- 172.21.15.17:0/1802050635 reap_dead start 2024-09-15T21:20:54.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 shutdown_connections 2024-09-15T21:20:54.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 >> 172.21.15.17:0/1802050635 conn(0x7f223c0a2fb0 msgr2=0x7f223c074890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 shutdown_connections 2024-09-15T21:20:54.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.361+0000 7f2241ef3640 1 -- 172.21.15.17:0/1802050635 wait complete. 2024-09-15T21:20:54.393 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:20:54.393 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:20:54.393 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.505+0000 7f9d7a25d640 10 monclient: get_monmap_and_config 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.505+0000 7f9d7a25d640 10 monclient: build_initial_monmap 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.505+0000 7f9d7a25d640 10 monclient: monmap: 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:20:54.509009+0000 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:20:54.509009+0000 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:54.512 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.505+0000 7f9d7a25d640 1 Processor -- start 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.505+0000 7f9d7a25d640 1 -- start start 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.505+0000 7f9d7a25d640 10 monclient: init 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d740a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d740a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d7409ffc0 0x7f9d740a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d7409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9d74135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d740a3130 con 0x7f9d74135670 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9d740a16f0 con 0x7f9d7409ffc0 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9d74073df0 con 0x7f9d740a0890 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient(hunting): authenticate will time out at 1177.904401s 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d740a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d740a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58550/0 (socket says 172.21.15.17:58550) 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 -- 172.21.15.17:0/1984539115 learned_addr learned my addr 172.21.15.17:0/1984539115 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): get_auth_request con 0x7f9d740a0890 auth_method 0 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d72ffd640 1 --2- 172.21.15.17:0/1984539115 >> v2:172.21.15.119:3300/0 conn(0x7f9d7409ffc0 0x7f9d740a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d72ffd640 10 monclient(hunting): get_auth_request con 0x7f9d7409ffc0 auth_method 0 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient(hunting): handle_auth_done global_id 44195 payload 274 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 -- 172.21.15.17:0/1984539115 >> v2:172.21.15.119:3300/0 conn(0x7f9d7409ffc0 msgr2=0x7f9d740a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 --2- 172.21.15.17:0/1984539115 >> v2:172.21.15.119:3300/0 conn(0x7f9d7409ffc0 0x7f9d740a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 -- 172.21.15.17:0/1984539115 >> v1:172.21.15.119:6789/0 conn(0x7f9d74135670 legacy=0x7f9d74135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 monclient: found mon.noname-b 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 -- 172.21.15.17:0/1984539115 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d740a1bf0 con 0x7f9d740a0890 2024-09-15T21:20:54.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d737fe640 1 --2- 172.21.15.17:0/1984539115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d740a0c70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d740a1870 tx=0x7f9d6401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8bb101d9f908d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 1 -- 172.21.15.17:0/1984539115 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d640a5620 con 0x7f9d740a0890 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: dump: 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.524 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 5 monclient: authenticate success, global_id 44195 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 1 -- 172.21.15.17:0/1984539115 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d640a57c0 con 0x7f9d740a0890 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:54.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d7a25d640 4 set_mon_vals no callback set 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 1 -- 172.21.15.17:0/1984539115 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d640a5b90 con 0x7f9d740a0890 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.509+0000 7f9d727fc640 10 monclient: dump: 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.526 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: shutdown 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/1984539115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 msgr2=0x7f9d740a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/1984539115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d740a0c70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d740a1870 tx=0x7f9d6401c8d0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/1984539115 shutdown_connections 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/1984539115 >> v2:172.21.15.119:3300/0 conn(0x7f9d7409ffc0 0x7f9d740a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/1984539115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d740a0c70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/1984539115 >> 172.21.15.17:0/1984539115 conn(0x7f9d740a66a0 msgr2=0x7f9d740a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/1984539115 shutdown_connections 2024-09-15T21:20:54.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/1984539115 wait complete. 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: build_initial_monmap 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: monmap: 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 Processor -- start 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- start start 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: init 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7409ffc0 0x7f9d7411d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d7409ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d741345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d740a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d74135670 0x7f9d7411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): picked mon.noname-a con 0x7f9d74135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d74140690 con 0x7f9d74135670 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d74140510 con 0x7f9d7409ffc0 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9d74140810 con 0x7f9d740a0890 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): authenticate will time out at 1177.910188s 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d73fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d74135670 msgr2=0x7f9d7411c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d74135670 0x7f9d7411c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d741345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d741345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:58560/0 (socket says 172.21.15.17:58560) 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 -- 172.21.15.17:0/2374194644 learned_addr learned my addr 172.21.15.17:0/2374194644 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): get_auth_request con 0x7f9d740a0890 auth_method 0 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7409ffc0 0x7f9d7411d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): get_auth_request con 0x7f9d7409ffc0 auth_method 0 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient(hunting): handle_auth_done global_id 44198 payload 824 2024-09-15T21:20:54.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7409ffc0 msgr2=0x7f9d7411d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7409ffc0 0x7f9d7411d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d74135670 msgr2=0x7f9d7411c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d74135670 0x7f9d7411c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 monclient: found mon.noname-c 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d741ce630 con 0x7f9d740a0890 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d72ffd640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d741345c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800d800 tx=0x7f9d6800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800b6c0 con 0x7f9d740a0890 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: dump: 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: _finish_auth 0 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:54.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d68019070 con 0x7f9d740a0890 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800ba60 con 0x7f9d740a0890 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d5bfff640 10 monclient: dump: 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 5 monclient: authenticate success, global_id 44198 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: _renew_subs 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7411ef50 con 0x7f9d740a0890 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d71ffb640 4 set_mon_vals no callback set 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: _renew_subs 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7413d1e0 con 0x7f9d740a0890 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 0x7f9d7402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.513+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d74135aa0 con 0x7f9d74134b00 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9d6800bc00 con 0x7f9d740a0890 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 0x7f9d7402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient: get_auth_request con 0x7f9d74134b00 auth_method 0 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d6003b970 0x7f9d6003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 6387+0+0 (secure 0 0 0) 0x7f9d68050530 con 0x7f9d740a0890 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d73fff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d6003b970 0x7f9d6003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d73fff640 10 monclient: get_auth_request con 0x7f9d6003b970 auth_method 0 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 10 monclient(hunting): handle_auth_done global_id 34582 payload 824 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d741398d0 con 0x7f9d74134b00 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d737fe640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 0x7f9d7402d450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d7406f1c0 tx=0x7f9d64019670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d73fff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d6003b970 0x7f9d6003de30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007fd0 tx=0x7f9d6c0079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64009510 con 0x7f9d74134b00 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 10 monclient: dump: 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.537 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9d640c06a0 con 0x7f9d74134b00 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 msgr2=0x7f9d7402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 0x7f9d7402d450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d7406f1c0 tx=0x7f9d64019670 comp rx=0 tx=0).stop 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 msgr2=0x7f9d7402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.517+0000 7f9d5bfff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74134b00 0x7f9d7402d450 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d7406f1c0 tx=0x7f9d64019670 comp rx=0 tx=0).stop 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 0x7f9d7402dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7411ebd0 con 0x7f9d7402d990 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 0x7f9d7402dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient: get_auth_request con 0x7f9d7402d990 auth_method 0 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 10 monclient(hunting): handle_auth_done global_id 34585 payload 824 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 1 -- 172.21.15.17:0/2374194644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d7411ed80 con 0x7f9d7402d990 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d737fe640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 0x7f9d7402dda0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d6401d170 tx=0x7f9d640a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d640c9410 con 0x7f9d7402d990 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 10 monclient: dump: 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7f9d640b63f0 con 0x7f9d7402d990 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 msgr2=0x7f9d7402dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 0x7f9d7402dda0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d6401d170 tx=0x7f9d640a7000 comp rx=0 tx=0).stop 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 msgr2=0x7f9d7402dda0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:20:54.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d5bfff640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7402d990 0x7f9d7402dda0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 67026, 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:20:54.543 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:54.544 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:54.545 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:20:54.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:20:54.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:54.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:20:54.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.554 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:20:54.555 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:20:54.556 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:20:54.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:20:54.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:20:54.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:20:54.556 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:20:54.556 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d6003b970 msgr2=0x7f9d6003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d6003b970 0x7f9d6003de30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007fd0 tx=0x7f9d6c0079a0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 10 monclient: shutdown 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 msgr2=0x7f9d741345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 --2- 172.21.15.17:0/2374194644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d740a0890 0x7f9d741345c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800d800 tx=0x7f9d6800dcc0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d73fff640 1 -- 172.21.15.17:0/2374194644 reap_dead start 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d73fff640 1 -- 172.21.15.17:0/2374194644 reap_dead start 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 shutdown_connections 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 >> 172.21.15.17:0/2374194644 conn(0x7f9d740a66a0 msgr2=0x7f9d74136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 shutdown_connections 2024-09-15T21:20:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:20:54.525+0000 7f9d7a25d640 1 -- 172.21.15.17:0/2374194644 wait complete. 2024-09-15T21:20:54.559 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:20:54.559 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:20:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.834+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 removed item id -12 name 'rack-for-test' from crush map v84) ==== 140+0+0 (secure 0 0 0) 0x7f96a801d020 con 0x7f96b809fa20 2024-09-15T21:20:54.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.834+0000 7f96adffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T21:20:54.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.834+0000 7f96adffb640 10 monclient: _finish_command 2 = system:0 removed item id -12 name 'rack-for-test' from crush map 2024-09-15T21:20:54.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96bf2c2640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T21:20:54.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96bf2c2640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T21:20:54.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96bf2c2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:54.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) -- 0x7f96b8120a40 con 0x7f96b809fa20 2024-09-15T21:20:54.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96adffb640 1 -- 172.21.15.119:0/353572094 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 device 'rack-for-test' does not appear in the crush map v84) ==== 140+0+0 (secure 0 0 0) 0x7f96b809d1d0 con 0x7f96b809fa20 2024-09-15T21:20:54.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96adffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T21:20:54.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.942+0000 7f96adffb640 10 monclient: _finish_command 3 = system:0 device 'rack-for-test' does not appear in the crush map 2024-09-15T21:20:54.949 INFO:tasks.workunit.client.0.smithi119.stderr:device 'rack-for-test' does not appear in the crush map 2024-09-15T21:20:54.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f969c03b9a0 msgr2=0x7f969c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f969c03b9a0 0x7f969c03de60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b1000 tx=0x7f96b4008040 comp rx=0 tx=0).stop 2024-09-15T21:20:54.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 10 monclient: shutdown 2024-09-15T21:20:54.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:54.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 msgr2=0x7f96b8133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:54.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 0x7f96b8133830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96a800b560 tx=0x7f96a800ba20 comp rx=0 tx=0).stop 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 shutdown_connections 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f969c03b9a0 0x7f969c03de60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b81347b0 0x7f96b8133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b809fa20 0x7f96b8133830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 --2- 172.21.15.119:0/353572094 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b809f640 0x7f96b81cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 >> 172.21.15.119:0/353572094 conn(0x7f96b80a2ae0 msgr2=0x7f96b8071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 shutdown_connections 2024-09-15T21:20:54.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:54.946+0000 7f96bf2c2640 1 -- 172.21.15.119:0/353572094 wait complete. 2024-09-15T21:20:54.966 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm root-for-test 2024-09-15T21:20:55.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient: get_monmap_and_config 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient: build_initial_monmap 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient: monmap: 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:55.082324+0000 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:55.082324+0000 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:55.084 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:55.085 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 1 Processor -- start 2024-09-15T21:20:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 1 -- start start 2024-09-15T21:20:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient: init 2024-09-15T21:20:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:20:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0e1c137bc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 0x7f0e1c13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient(hunting): picked mon.noname-b con 0x7f0e1c0a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0e1c0a0ea0 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c09cfa0 con 0x7f0e1c137bc0 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c09cd20 con 0x7f0e1c0a0ea0 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c1386b0 con 0x7f0e1c0a09e0 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 10 monclient(hunting): authenticate will time out at 1173.878955s 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1ad9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0e1c0a0ea0 0x7f0e1c13efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39028/0 (socket says 172.21.15.119:39028) 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/2481797610 learned_addr learned my addr 172.21.15.119:0/2481797610 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1a59c640 1 -- 172.21.15.119:0/2481797610 >> v1:172.21.15.17:6789/0 conn(0x7f0e1c137bc0 legacy=0x7f0e1c137fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 0x7f0e1c13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): get_auth_request con 0x7f0e1c0a09e0 auth_method 0 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1408664190 0 0) 0x7f0e1c09cd20 con 0x7f0e1c0a0ea0 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): _negotiate 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): my global_id is 34588 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e04003530 con 0x7f0e1c0a0ea0 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:55.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e19d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3157598116 0 0) 0x7f0e04003530 con 0x7f0e1c0a0ea0 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 msgr2=0x7f0e1c13ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 0x7f0e1c13ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 >> v1:172.21.15.17:6789/0 conn(0x7f0e1c137bc0 legacy=0x7f0e1c137fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: _finish_hunting 0 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0a1820 con 0x7f0e1c0a0ea0 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: _finish_auth 0 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: _check_auth_tickets 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0e0c001700 con 0x7f0e1c0a0ea0 2024-09-15T21:20:55.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 5 monclient: authenticate success, global_id 34588 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: dump: 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:55.092 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 >> v1:172.21.15.119:6789/0 conn(0x7f0e1c0a0ea0 legacy=0x7f0e1c13efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e040039c0 0x7f0e04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): picked mon.b con 0x7f0e040039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 0x7f0e04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): picked mon.c con 0x7f0e1c0a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e04004d50 0x7f0e040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): picked mon.a con 0x7f0e04004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1386b0 con 0x7f0e04004d50 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0400bb00 con 0x7f0e040039c0 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0400bd00 con 0x7f0e1c0a09e0 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1959a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e20e23640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e040039c0 0x7f0e04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1a59c640 10 monclient(hunting): get_auth_request con 0x7f0e040039c0 auth_method 0 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:55.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.078+0000 7f0e1a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e04004d50 msgr2=0x7f0e040051d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1ad9d640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e04004d50 0x7f0e040051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient(hunting): handle_auth_done global_id 34588 payload 293 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 -- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 msgr2=0x7f0e04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e19d9b640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 0x7f0e04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a09e0 0x7f0e04004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 -- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e04004d50 msgr2=0x7f0e040051d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e04004d50 0x7f0e040051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:55.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 monclient: found mon.b 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 -- 172.21.15.119:0/2481797610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e04003530 con 0x7f0e040039c0 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient: _finish_auth 0 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 10 monclient: _check_auth_tickets 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e040039c0 0x7f0e04003de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009e30 tx=0x7f0e1401caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd3bc81ccaedc21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14009a30 con 0x7f0e040039c0 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: dump: 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:55.096 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e140a9070 con 0x7f0e040039c0 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 1 -- 172.21.15.119:0/2481797610 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140a5b40 con 0x7f0e040039c0 2024-09-15T21:20:55.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 4 set_mon_vals no callback set 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1959a640 10 monclient: dump: 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:55.098 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 10 monclient: shutdown 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 1 -- 172.21.15.119:0/2481797610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e040039c0 msgr2=0x7f0e04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 1 --2- 172.21.15.119:0/2481797610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e040039c0 0x7f0e04003de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009e30 tx=0x7f0e1401caf0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/2481797610 reap_dead start 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 1 -- 172.21.15.119:0/2481797610 shutdown_connections 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 1 -- 172.21.15.119:0/2481797610 >> 172.21.15.119:0/2481797610 conn(0x7f0e1c0a2ae0 msgr2=0x7f0e1c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 1 -- 172.21.15.119:0/2481797610 shutdown_connections 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 1 -- 172.21.15.119:0/2481797610 wait complete. 2024-09-15T21:20:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 10 monclient: build_initial_monmap 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.082+0000 7f0e20e23640 10 monclient: monmap: 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 Processor -- start 2024-09-15T21:20:55.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- start start 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: init 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e040039c0 0x7f0e1c1342e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): picked mon.noname-c con 0x7f0e040039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c121180 0x7f0e1c11eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): picked mon.noname-a con 0x7f0e1c121180 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 0x7f0e1c11f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): picked mon.noname-b con 0x7f0e1c11f0a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0a39e0 con 0x7f0e1c121180 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0a3860 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c13aa80 con 0x7f0e040039c0 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient(hunting): authenticate will time out at 1173.884870s 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 0x7f0e1c11f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e19d9b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c121180 msgr2=0x7f0e1c11eb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e19d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c121180 0x7f0e1c11eb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 0x7f0e1c11f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35626/0 (socket says 172.21.15.119:35626) 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/415068836 learned_addr learned my addr 172.21.15.119:0/415068836 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): get_auth_request con 0x7f0e1c11f0a0 auth_method 0 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:55.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e040039c0 0x7f0e1c1342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient(hunting): handle_auth_done global_id 34591 payload 824 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e040039c0 msgr2=0x7f0e1c1342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e040039c0 0x7f0e1c1342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c121180 msgr2=0x7f0e1c11eb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c121180 0x7f0e1c11eb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 -- 172.21.15.119:0/415068836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c121b10 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1a59c640 10 monclient: get_auth_request con 0x7f0e040039c0 auth_method 0 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e040039c0 0x7f0e1c1342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1ad9d640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 0x7f0e1c11f530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c09fb90 tx=0x7f0e0c00f760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c007d00 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: dump: 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:55.108 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: _finish_auth 0 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: _check_auth_tickets 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00a110 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00a500 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 5 monclient: authenticate success, global_id 34591 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:55.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 10 monclient: dump: 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _renew_subs 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e18d99640 4 set_mon_vals no callback set 2024-09-15T21:20:55.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c120620 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _renew_subs 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c1220e0 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0e0c00a6a0 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c09cd20 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6204+0+0 (secure 0 0 0) 0x7f0e0c0521f0 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:55.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1a59c640 10 monclient: get_auth_request con 0x7f0dfc03b970 auth_method 0 2024-09-15T21:20:55.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.086+0000 7f0e1a59c640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e140ad000 tx=0x7f0e14005e10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:55.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.090+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0e0c01e070 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.090+0000 7f0e0affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:55.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.090+0000 7f0e0affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.266+0000 7f0e20e23640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T21:20:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.270+0000 7f0e20e23640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T21:20:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.270+0000 7f0e20e23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.270+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) -- 0x7f0e1c06c460 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.850+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 removed item id -14 name 'root-for-test' from crush map v85) ==== 140+0+0 (secure 0 0 0) 0x7f0e0c015ca0 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.850+0000 7f0e0affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T21:20:55.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.850+0000 7f0e0affd640 10 monclient: _finish_command 2 = system:0 removed item id -14 name 'root-for-test' from crush map 2024-09-15T21:20:55.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e20e23640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T21:20:55.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e20e23640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T21:20:55.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e20e23640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:55.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) -- 0x7f0e1c1d2c50 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e0affd640 1 -- 172.21.15.119:0/415068836 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 device 'root-for-test' does not appear in the crush map v85) ==== 140+0+0 (secure 0 0 0) 0x7f0e0c01e7e0 con 0x7f0e1c11f0a0 2024-09-15T21:20:55.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e0affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T21:20:55.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.958+0000 7f0e0affd640 10 monclient: _finish_command 3 = system:0 device 'root-for-test' does not appear in the crush map 2024-09-15T21:20:55.964 INFO:tasks.workunit.client.0.smithi119.stderr:device 'root-for-test' does not appear in the crush map 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 msgr2=0x7f0dfc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e140ad000 tx=0x7f0e14005e10 comp rx=0 tx=0).stop 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 10 monclient: shutdown 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 msgr2=0x7f0e1c11f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 0x7f0e1c11f530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c09fb90 tx=0x7f0e0c00f760 comp rx=0 tx=0).stop 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 shutdown_connections 2024-09-15T21:20:55.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c11f0a0 0x7f0e1c11f530 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c121180 0x7f0e1c11eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 --2- 172.21.15.119:0/415068836 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e040039c0 0x7f0e1c1342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:55.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 >> 172.21.15.119:0/415068836 conn(0x7f0e1c0a2ae0 msgr2=0x7f0e1c09c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:55.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 shutdown_connections 2024-09-15T21:20:55.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:55.962+0000 7f0e20e23640 1 -- 172.21.15.119:0/415068836 wait complete. 2024-09-15T21:20:55.981 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule create-simple foo-rule foo host firstn 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.094+0000 7f5c594bc640 10 monclient: get_monmap_and_config 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.094+0000 7f5c594bc640 10 monclient: build_initial_monmap 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.094+0000 7f5c594bc640 10 monclient: monmap: 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:56.099599+0000 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:56.099599+0000 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:56.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:56.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:56.102 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:56.102 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:56.102 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:56.102 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:56.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:56.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.094+0000 7f5c594bc640 1 Processor -- start 2024-09-15T21:20:56.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.094+0000 7f5c594bc640 1 -- start start 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.094+0000 7f5c594bc640 10 monclient: init 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c54135bf0 0x7f5c5409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient(hunting): picked mon.noname-c con 0x7f5c54135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c54135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:56.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5c54136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540a1870 con 0x7f5c54135fd0 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540a1d80 con 0x7f5c54136460 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5c540a3600 con 0x7f5c54135bf0 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c594bc640 10 monclient(hunting): authenticate will time out at 1174.896274s 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c527fc640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f5c54135fd0 legacy=0x7f5c540a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c54135bf0 0x7f5c5409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:56.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c537fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5c54136460 0x7f5c540a1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39040/0 (socket says 172.21.15.119:39040) 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c537fe640 1 -- 172.21.15.119:0/3667650694 learned_addr learned my addr 172.21.15.119:0/3667650694 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): get_auth_request con 0x7f5c54135bf0 auth_method 0 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c51ffb640 1 -- 172.21.15.119:0/3667650694 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3115163392 0 0) 0x7f5c540a1d80 con 0x7f5c54136460 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c51ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c51ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c51ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c51ffb640 10 monclient(hunting): my global_id is 34597 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c51ffb640 1 -- 172.21.15.119:0/3667650694 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c3c003530 con 0x7f5c54136460 2024-09-15T21:20:56.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient(hunting): handle_auth_done global_id 34594 payload 274 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 1 -- 172.21.15.119:0/3667650694 >> v1:172.21.15.119:6789/0 conn(0x7f5c54136460 legacy=0x7f5c540a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 1 -- 172.21.15.119:0/3667650694 >> v1:172.21.15.17:6789/0 conn(0x7f5c54135fd0 legacy=0x7f5c540a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 1 monclient: found mon.noname-c 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.098+0000 7f5c52ffd640 1 -- 172.21.15.119:0/3667650694 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c540a2060 con 0x7f5c54135bf0 2024-09-15T21:20:56.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c52ffd640 1 --2- 172.21.15.119:0/3667650694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54135bf0 0x7f5c5409fe70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097f0 tx=0x7f5c4401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4b42b48ee75cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 1 -- 172.21.15.119:0/3667650694 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440b0070 con 0x7f5c54135bf0 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: dump: 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:56.114 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 1 -- 172.21.15.119:0/3667650694 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44004230 con 0x7f5c54135bf0 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 5 monclient: authenticate success, global_id 34594 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 1 -- 172.21.15.119:0/3667650694 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44003620 con 0x7f5c54135bf0 2024-09-15T21:20:56.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 4 set_mon_vals no callback set 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c51ffb640 10 monclient: dump: 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.116 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: shutdown 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- 172.21.15.119:0/3667650694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54135bf0 msgr2=0x7f5c5409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 --2- 172.21.15.119:0/3667650694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54135bf0 0x7f5c5409fe70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097f0 tx=0x7f5c4401c330 comp rx=0 tx=0).stop 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- 172.21.15.119:0/3667650694 shutdown_connections 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 --2- 172.21.15.119:0/3667650694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54135bf0 0x7f5c5409fe70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- 172.21.15.119:0/3667650694 >> 172.21.15.119:0/3667650694 conn(0x7f5c540a66e0 msgr2=0x7f5c540a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- 172.21.15.119:0/3667650694 shutdown_connections 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- 172.21.15.119:0/3667650694 wait complete. 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: build_initial_monmap 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: monmap: 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:56.118 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 Processor -- start 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- start start 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: init 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c54135bf0 0x7f5c5413c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): picked mon.noname-a con 0x7f5c54135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c54135fd0 0x7f5c5413cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): picked mon.noname-c con 0x7f5c54135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 0x7f5c5413d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:56.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): picked mon.noname-b con 0x7f5c54136460 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541416d0 con 0x7f5c54135bf0 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5406ed50 con 0x7f5c54136460 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c54141850 con 0x7f5c54135fd0 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c594bc640 10 monclient(hunting): authenticate will time out at 1174.903220s 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 0x7f5c5413d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c52ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c54135bf0 msgr2=0x7f5c5413c600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c54135bf0 0x7f5c5413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 0x7f5c5413d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35640/0 (socket says 172.21.15.119:35640) 2024-09-15T21:20:56.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 -- 172.21.15.119:0/2205570240 learned_addr learned my addr 172.21.15.119:0/2205570240 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): get_auth_request con 0x7f5c54136460 auth_method 0 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c527fc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c54135fd0 0x7f5c5413cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient(hunting): handle_auth_done global_id 34600 payload 824 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 -- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c54135fd0 msgr2=0x7f5c5413cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:56.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c54135fd0 0x7f5c5413cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 -- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c54135bf0 msgr2=0x7f5c5413c600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c54135bf0 0x7f5c5413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 monclient: found mon.noname-b 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 -- 172.21.15.119:0/2205570240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5411d030 con 0x7f5c54136460 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c527fc640 10 monclient: get_auth_request con 0x7f5c54135fd0 auth_method 0 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c527fc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c54135fd0 0x7f5c5413cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c537fe640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 0x7f5c5413d080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00cb40 tx=0x7f5c4c00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00aca0 con 0x7f5c54136460 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c3b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:56.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.102+0000 7f5c3b7fe640 10 monclient: dump: 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:56.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 5 monclient: authenticate success, global_id 34600 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c00ae40 con 0x7f5c54136460 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 monclient: _renew_subs 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5413def0 con 0x7f5c54136460 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c012600 con 0x7f5c54136460 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 10 monclient: dump: 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:56.124 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 monclient: _renew_subs 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5411d600 con 0x7f5c54136460 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c517fa640 4 set_mon_vals no callback set 2024-09-15T21:20:56.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5409fe70 con 0x7f5c54136460 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5c4c0127a0 con 0x7f5c54136460 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c4003b970 0x7f5c4003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6045+0+0 (secure 0 0 0) 0x7f5c4c050630 con 0x7f5c54136460 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c52ffd640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c4003b970 0x7f5c4003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c52ffd640 10 monclient: get_auth_request con 0x7f5c4003b970 auth_method 0 2024-09-15T21:20:56.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.106+0000 7f5c52ffd640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c4003b970 0x7f5c4003de30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5c440ad000 tx=0x7f5c44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:56.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.110+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5c4c01a360 con 0x7f5c54136460 2024-09-15T21:20:56.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.110+0000 7f5c3b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:56.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.110+0000 7f5c3b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:56.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.270+0000 7f5c594bc640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T21:20:56.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.270+0000 7f5c594bc640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T21:20:56.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.270+0000 7f5c594bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.270+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) -- 0x7f5c5406a830 con 0x7f5c54136460 2024-09-15T21:20:56.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.850+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 v86) ==== 145+0+0 (secure 0 0 0) 0x7f5c4c007a90 con 0x7f5c54136460 2024-09-15T21:20:56.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.850+0000 7f5c3b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T21:20:56.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.850+0000 7f5c3b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:56.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c594bc640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T21:20:56.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c594bc640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T21:20:56.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c594bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:56.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) -- 0x7f5c5411ca20 con 0x7f5c54136460 2024-09-15T21:20:56.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c3b7fe640 1 -- 172.21.15.119:0/2205570240 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 rule foo-rule already exists v86) ==== 173+0+0 (secure 0 0 0) 0x7f5c4c012d20 con 0x7f5c54136460 2024-09-15T21:20:56.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c3b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T21:20:56.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.954+0000 7f5c3b7fe640 10 monclient: _finish_command 3 = system:0 rule foo-rule already exists 2024-09-15T21:20:56.961 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo-rule already exists 2024-09-15T21:20:56.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c4003b970 msgr2=0x7f5c4003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:56.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c4003b970 0x7f5c4003de30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5c440ad000 tx=0x7f5c44008040 comp rx=0 tx=0).stop 2024-09-15T21:20:56.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 10 monclient: shutdown 2024-09-15T21:20:56.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 msgr2=0x7f5c5413d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 0x7f5c5413d080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00cb40 tx=0x7f5c4c00a060 comp rx=0 tx=0).stop 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 shutdown_connections 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c4003b970 0x7f5c4003de30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c54136460 0x7f5c5413d080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c54135fd0 0x7f5c5413cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 --2- 172.21.15.119:0/2205570240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c54135bf0 0x7f5c5413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 >> 172.21.15.119:0/2205570240 conn(0x7f5c540a66e0 msgr2=0x7f5c54072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 shutdown_connections 2024-09-15T21:20:56.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:56.958+0000 7f5c594bc640 1 -- 172.21.15.119:0/2205570240 wait complete. 2024-09-15T21:20:56.977 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush rm foo 2024-09-15T21:20:56.978 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:20:56.978 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm foo 2024-09-15T21:20:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient: get_monmap_and_config 2024-09-15T21:20:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient: build_initial_monmap 2024-09-15T21:20:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient: monmap: 2024-09-15T21:20:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:57.089933+0000 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:57.089933+0000 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:57.092 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 Processor -- start 2024-09-15T21:20:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 -- start start 2024-09-15T21:20:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient: init 2024-09-15T21:20:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:20:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf900a4ed0 addr v1:172.21.15.17:6789/0 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a59c0 0x7faf9013ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): picked mon.noname-b con 0x7faf900a59c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf900a5dd0 0x7faf9013f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): picked mon.noname-c con 0x7faf900a5dd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf900a6db0 con 0x7faf900a4ed0 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faf900a1210 con 0x7faf900a5dd0 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faf900a1b00 con 0x7faf900a59c0 2024-09-15T21:20:57.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf95015640 10 monclient(hunting): authenticate will time out at 1175.886675s 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8ed9d640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7faf900a4ed0 legacy=0x7faf900a52b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf900a5dd0 0x7faf9013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf900a5dd0 0x7faf9013f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35652/0 (socket says 172.21.15.119:35652) 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 -- 172.21.15.119:0/3214256440 learned_addr learned my addr 172.21.15.119:0/3214256440 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): get_auth_request con 0x7faf900a5dd0 auth_method 0 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:57.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8e59c640 1 --2- 172.21.15.119:0/3214256440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a59c0 0x7faf9013ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8e59c640 10 monclient(hunting): get_auth_request con 0x7faf900a59c0 auth_method 0 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient(hunting): handle_auth_done global_id 34603 payload 274 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 -- 172.21.15.119:0/3214256440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a59c0 msgr2=0x7faf9013ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:57.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 --2- 172.21.15.119:0/3214256440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a59c0 0x7faf9013ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 -- 172.21.15.119:0/3214256440 >> v1:172.21.15.17:6789/0 conn(0x7faf900a4ed0 legacy=0x7faf900a52b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 monclient: found mon.noname-c 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.086+0000 7faf8f59e640 1 -- 172.21.15.119:0/3214256440 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf900a6ad0 con 0x7faf900a5dd0 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8f59e640 1 --2- 172.21.15.119:0/3214256440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a5dd0 0x7faf9013f420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf88009e50 tx=0x7faf8801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971fe912037ffbc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 1 -- 172.21.15.119:0/3214256440 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880a5540 con 0x7faf900a5dd0 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: dump: 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.099 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 5 monclient: authenticate success, global_id 34603 2024-09-15T21:20:57.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 1 -- 172.21.15.119:0/3214256440 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf880a56e0 con 0x7faf900a5dd0 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 4 set_mon_vals no callback set 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 1 -- 172.21.15.119:0/3214256440 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880a5ab0 con 0x7faf900a5dd0 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8dd9b640 10 monclient: dump: 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.102 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.102 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.102 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.102 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: shutdown 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- 172.21.15.119:0/3214256440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a5dd0 msgr2=0x7faf9013f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 --2- 172.21.15.119:0/3214256440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a5dd0 0x7faf9013f420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf88009e50 tx=0x7faf8801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- 172.21.15.119:0/3214256440 shutdown_connections 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 --2- 172.21.15.119:0/3214256440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a5dd0 0x7faf9013f420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 --2- 172.21.15.119:0/3214256440 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a59c0 0x7faf9013ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- 172.21.15.119:0/3214256440 >> 172.21.15.119:0/3214256440 conn(0x7faf900a6fd0 msgr2=0x7faf900a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- 172.21.15.119:0/3214256440 shutdown_connections 2024-09-15T21:20:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- 172.21.15.119:0/3214256440 wait complete. 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: build_initial_monmap 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: monmap: 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:57.110 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 Processor -- start 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- start start 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: init 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a4ed0 0x7faf90127c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): picked mon.noname-c con 0x7faf900a4ed0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a59c0 0x7faf9013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): picked mon.noname-b con 0x7faf900a59c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf900a5dd0 0x7faf901cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): picked mon.noname-a con 0x7faf900a5dd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:57.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faf90142180 con 0x7faf900a5dd0 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faf900a7e70 con 0x7faf900a59c0 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faf90142300 con 0x7faf900a4ed0 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient(hunting): authenticate will time out at 1175.891271s 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a59c0 0x7faf9013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a59c0 0x7faf9013eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35666/0 (socket says 172.21.15.119:35666) 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 -- 172.21.15.119:0/4024331383 learned_addr learned my addr 172.21.15.119:0/4024331383 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): get_auth_request con 0x7faf900a59c0 auth_method 0 2024-09-15T21:20:57.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8f59e640 1 -- 172.21.15.119:0/4024331383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf900a5dd0 msgr2=0x7faf901cf650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8f59e640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf900a5dd0 0x7faf901cf650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient(hunting): handle_auth_done global_id 34606 payload 824 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 -- 172.21.15.119:0/4024331383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a4ed0 msgr2=0x7faf90127c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf900a4ed0 0x7faf90127c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 -- 172.21.15.119:0/4024331383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf900a5dd0 msgr2=0x7faf901cf650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:57.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf900a5dd0 0x7faf901cf650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 monclient: found mon.noname-b 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 -- 172.21.15.119:0/4024331383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf900a9d10 con 0x7faf900a59c0 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8e59c640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900a59c0 0x7faf9013eaf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf8400d800 tx=0x7faf8400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400b6c0 con 0x7faf900a59c0 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: dump: 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf84019070 con 0x7faf900a59c0 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 5 monclient: authenticate success, global_id 34606 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: _renew_subs 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf95015640 1 -- 172.21.15.119:0/4024331383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf90129480 con 0x7faf900a59c0 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400ba60 con 0x7faf900a59c0 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:57.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf777fe640 10 monclient: dump: 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 10 monclient: _renew_subs 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.090+0000 7faf8d59a640 4 set_mon_vals no callback set 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 1 -- 172.21.15.119:0/4024331383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90128be0 con 0x7faf900a59c0 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf95015640 1 -- 172.21.15.119:0/4024331383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf90075270 con 0x7faf900a59c0 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7faf8400bc00 con 0x7faf900a59c0 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf777fe640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf7c03b970 0x7faf7c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6105+0+0 (secure 0 0 0) 0x7faf840504d0 con 0x7faf900a59c0 2024-09-15T21:20:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf8ed9d640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf7c03b970 0x7faf7c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf8ed9d640 10 monclient: get_auth_request con 0x7faf7c03b970 auth_method 0 2024-09-15T21:20:57.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.094+0000 7faf8ed9d640 1 --2- 172.21.15.119:0/4024331383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf7c03b970 0x7faf7c03de30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf80006fd0 tx=0x7faf80006d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:57.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.098+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7faf900a9d10 con 0x7faf900a59c0 2024-09-15T21:20:57.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.098+0000 7faf777fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:57.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.098+0000 7faf777fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:57.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.270+0000 7faf95015640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:57.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.270+0000 7faf95015640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:57.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.270+0000 7faf95015640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.270+0000 7faf95015640 1 -- 172.21.15.119:0/4024331383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7faf900a52b0 con 0x7faf900a59c0 2024-09-15T21:20:57.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.274+0000 7faf777fe640 1 -- 172.21.15.119:0/4024331383 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-16 (16) Device or resource busy v86) ==== 103+0+0 (secure 0 0 0) 0x7faf8401cd70 con 0x7faf900a59c0 2024-09-15T21:20:57.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.274+0000 7faf777fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:57.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.274+0000 7faf777fe640 10 monclient: _finish_command 2 = mon:16 (16) Device or resource busy 2024-09-15T21:20:57.280 INFO:tasks.workunit.client.0.smithi119.stderr:Error EBUSY: (16) Device or resource busy 2024-09-15T21:20:57.284 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:20:57.284 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rule rm foo-rule 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.390+0000 7fde11462640 10 monclient: get_monmap_and_config 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.390+0000 7fde11462640 10 monclient: build_initial_monmap 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.390+0000 7fde11462640 10 monclient: monmap: 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:57.395881+0000 2024-09-15T21:20:57.397 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:57.395881+0000 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:57.398 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 Processor -- start 2024-09-15T21:20:57.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 -- start start 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient: init 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde0c0a79e0 0x7fde0c0a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): picked mon.noname-a con 0x7fde0c0a79e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 0x7fde0c1377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): picked mon.noname-b con 0x7fde0c0a46b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fde0c0a1100 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0c135180 con 0x7fde0c0a1100 2024-09-15T21:20:57.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c0a7dc0 con 0x7fde0c0a79e0 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c0a6920 con 0x7fde0c0a46b0 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 10 monclient(hunting): authenticate will time out at 1176.193077s 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fde0c0a79e0 msgr2=0x7fde0c0a4170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fde0c0a1100 0x7fde0c137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39046/0 (socket says 172.21.15.119:39046) 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 -- 172.21.15.119:0/74449637 learned_addr learned my addr 172.21.15.119:0/74449637 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 1 --2- 172.21.15.119:0/74449637 >> v2:172.21.15.17:3300/0 conn(0x7fde0c0a79e0 0x7fde0c0a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 0x7fde0c1377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): get_auth_request con 0x7fde0c0a46b0 auth_method 0 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157991053 0 0) 0x7fde0c135180 con 0x7fde0c0a1100 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): my global_id is 34609 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf4003530 con 0x7fde0c0a1100 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 313329521 0 0) 0x7fddf4003530 con 0x7fde0c0a1100 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 msgr2=0x7fde0c1377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 0x7fde0c1377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 >> v2:172.21.15.17:3300/0 conn(0x7fde0c0a79e0 msgr2=0x7fde0c0a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 --2- 172.21.15.119:0/74449637 >> v2:172.21.15.17:3300/0 conn(0x7fde0c0a79e0 0x7fde0c0a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c0a1c30 con 0x7fde0c0a1100 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: _finish_auth 0 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 5 monclient: authenticate success, global_id 34609 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fde04001a80 con 0x7fde0c0a1100 2024-09-15T21:20:57.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: dump: 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 >> v1:172.21.15.119:6789/0 conn(0x7fde0c0a1100 legacy=0x7fde0c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 0x7fddf40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): picked mon.c con 0x7fde0c0a46b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4003f90 0x7fddf4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): picked mon.a con 0x7fddf4003f90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fddf4004d50 0x7fddf40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): picked mon.b con 0x7fddf4004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c0a7dc0 con 0x7fddf4003f90 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c0a6920 con 0x7fddf4004d50 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fddf400bc10 con 0x7fde0c0a46b0 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde09ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde11462640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 1 -- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4003f90 msgr2=0x7fddf4004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0a7fc640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4003f90 0x7fddf4004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fddf4004d50 0x7fddf40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): get_auth_request con 0x7fddf4004d50 auth_method 0 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 0x7fddf40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): get_auth_request con 0x7fde0c0a46b0 auth_method 0 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient(hunting): handle_auth_done global_id 34609 payload 293 2024-09-15T21:20:57.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 -- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 msgr2=0x7fddf40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c0a46b0 0x7fddf40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 -- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4003f90 msgr2=0x7fddf4004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4003f90 0x7fddf4004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 monclient: found mon.b 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 1 -- 172.21.15.119:0/74449637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf4003530 con 0x7fddf4004d50 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.394+0000 7fde0b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0b7fe640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fddf4004d50 0x7fddf40051d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0a15b0 tx=0x7fde0401d280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84923911257a8571 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400a4a0 con 0x7fddf4004d50 2024-09-15T21:20:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: dump: 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde040ad650 con 0x7fddf4004d50 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 1 -- 172.21.15.119:0/74449637 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde040ada40 con 0x7fddf4004d50 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 4 set_mon_vals no callback set 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde09ffb640 10 monclient: dump: 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: shutdown 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- 172.21.15.119:0/74449637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fddf4004d50 msgr2=0x7fddf40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 --2- 172.21.15.119:0/74449637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fddf4004d50 0x7fddf40051d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0a15b0 tx=0x7fde0401d280 comp rx=0 tx=0).stop 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0b7fe640 1 -- 172.21.15.119:0/74449637 reap_dead start 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- 172.21.15.119:0/74449637 shutdown_connections 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- 172.21.15.119:0/74449637 >> 172.21.15.119:0/74449637 conn(0x7fde0c0a6ef0 msgr2=0x7fde0c0a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:57.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- 172.21.15.119:0/74449637 shutdown_connections 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- 172.21.15.119:0/74449637 wait complete. 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: build_initial_monmap 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: monmap: 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:57.421 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 Processor -- start 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- start start 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: init 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4004d50 0x7fde0c13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): picked mon.noname-a con 0x7fddf4004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 0x7fde0c13e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): picked mon.noname-b con 0x7fde0c13e390 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c1cf670 0x7fde0c1cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): picked mon.noname-c con 0x7fde0c1cf670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c138430 con 0x7fddf4004d50 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c134930 con 0x7fde0c13e390 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c1385b0 con 0x7fde0c1cf670 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde11462640 10 monclient(hunting): authenticate will time out at 1176.199485s 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 0x7fde0c13e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0affd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4004d50 msgr2=0x7fde0c13de50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4004d50 0x7fde0c13de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 0x7fde0c13e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35680/0 (socket says 172.21.15.119:35680) 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 -- 172.21.15.119:0/1647104142 learned_addr learned my addr 172.21.15.119:0/1647104142 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): get_auth_request con 0x7fde0c13e390 auth_method 0 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0b7fe640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c1cf670 0x7fde0c1cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient(hunting): handle_auth_done global_id 34612 payload 824 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 -- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c1cf670 msgr2=0x7fde0c1cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c1cf670 0x7fde0c1cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 -- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4004d50 msgr2=0x7fde0c13de50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4004d50 0x7fde0c13de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 monclient: found mon.noname-b 2024-09-15T21:20:57.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0a7fc640 1 -- 172.21.15.119:0/1647104142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c06fb30 con 0x7fde0c13e390 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0b7fe640 10 monclient: get_auth_request con 0x7fde0c1cf670 auth_method 0 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.398+0000 7fde0b7fe640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c1cf670 0x7fde0c1cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde0a7fc640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 0x7fde0c13e810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde0000e8c0 tx=0x7fde0000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00017070 con 0x7fde0c13e390 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: dump: 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.425 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde0000c940 con 0x7fde0c13e390 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0000cd10 con 0x7fde0c13e390 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 5 monclient: authenticate success, global_id 34612 2024-09-15T21:20:57.426 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 monclient: _renew_subs 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c070cd0 con 0x7fde0c13e390 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde097fa640 4 set_mon_vals no callback set 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 10 monclient: dump: 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:57.427 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 monclient: _renew_subs 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c0a15e0 con 0x7fde0c13e390 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fde0001a070 con 0x7fde0c13e390 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c0a4170 con 0x7fde0c13e390 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fddf803b970 0x7fddf803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6105+0+0 (secure 0 0 0) 0x7fde0004ffe0 con 0x7fde0c13e390 2024-09-15T21:20:57.428 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde0affd640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fddf803b970 0x7fddf803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:57.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde0affd640 10 monclient: get_auth_request con 0x7fddf803b970 auth_method 0 2024-09-15T21:20:57.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.402+0000 7fde0affd640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fddf803b970 0x7fddf803de30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fddfc006fd0 tx=0x7fddfc0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:57.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.406+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fde0c06fb30 con 0x7fde0c13e390 2024-09-15T21:20:57.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.406+0000 7fddf37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:57.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.406+0000 7fddf37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:57.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.582+0000 7fde11462640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T21:20:57.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.582+0000 7fde11462640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T21:20:57.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.582+0000 7fde11462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:57.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:57.582+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) -- 0x7fde0c06a760 con 0x7fde0c13e390 2024-09-15T21:20:58.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.514+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 v88) ==== 85+0+0 (secure 0 0 0) 0x7fde00013c70 con 0x7fde0c13e390 2024-09-15T21:20:58.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.514+0000 7fddf37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T21:20:58.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.514+0000 7fddf37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:20:58.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.618+0000 7fde11462640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T21:20:58.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.618+0000 7fde11462640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T21:20:58.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.618+0000 7fde11462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.618+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) -- 0x7fde0c071380 con 0x7fde0c13e390 2024-09-15T21:20:58.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fddf37fe640 1 -- 172.21.15.119:0/1647104142 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 rule foo-rule does not exist v88) ==== 113+0+0 (secure 0 0 0) 0x7fde000520e0 con 0x7fde0c13e390 2024-09-15T21:20:58.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fddf37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T21:20:58.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fddf37fe640 10 monclient: _finish_command 3 = system:0 rule foo-rule does not exist 2024-09-15T21:20:58.626 INFO:tasks.workunit.client.0.smithi119.stderr:rule foo-rule does not exist 2024-09-15T21:20:58.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fddf803b970 msgr2=0x7fddf803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fddf803b970 0x7fddf803de30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fddfc006fd0 tx=0x7fddfc0090e0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 10 monclient: shutdown 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 msgr2=0x7fde0c13e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 0x7fde0c13e810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde0000e8c0 tx=0x7fde0000ed80 comp rx=0 tx=0).stop 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 shutdown_connections 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fddf803b970 0x7fddf803de30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde0c1cf670 0x7fde0c1cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde0c13e390 0x7fde0c13e810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 --2- 172.21.15.119:0/1647104142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf4004d50 0x7fde0c13de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 >> 172.21.15.119:0/1647104142 conn(0x7fde0c0a6ef0 msgr2=0x7fde0c132bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:58.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 shutdown_connections 2024-09-15T21:20:58.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.622+0000 7fde11462640 1 -- 172.21.15.119:0/1647104142 wait complete. 2024-09-15T21:20:58.643 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm bar 2024-09-15T21:20:58.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.754+0000 7ffa349d6640 10 monclient: get_monmap_and_config 2024-09-15T21:20:58.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.754+0000 7ffa349d6640 10 monclient: build_initial_monmap 2024-09-15T21:20:58.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.754+0000 7ffa349d6640 10 monclient: monmap: 2024-09-15T21:20:58.761 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:58.761 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:58.761 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:58.759915+0000 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:58.759915+0000 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:58.762 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:58.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.754+0000 7ffa349d6640 1 Processor -- start 2024-09-15T21:20:58.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.754+0000 7ffa349d6640 1 -- start start 2024-09-15T21:20:58.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.754+0000 7ffa349d6640 10 monclient: init 2024-09-15T21:20:58.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffa3009f500 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffa3009eb70 0x7ffa3009f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): picked mon.noname-a con 0x7ffa3009eb70 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa300a10f0 0x7ffa30144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): picked mon.noname-c con 0x7ffa300a10f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa300a02f0 con 0x7ffa3009f500 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3009efa0 con 0x7ffa3009eb70 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3009fe20 con 0x7ffa300a10f0 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:58.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient(hunting): authenticate will time out at 1177.556585s 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2e59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffa3009f500 0x7ffa300a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39048/0 (socket says 172.21.15.119:39048) 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2e59c640 1 -- 172.21.15.119:0/3308989381 learned_addr learned my addr 172.21.15.119:0/3308989381 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa25d9b640 1 -- 172.21.15.119:0/3308989381 >> v2:172.21.15.17:3300/0 conn(0x7ffa3009eb70 msgr2=0x7ffa3009f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa25d9b640 1 --2- 172.21.15.119:0/3308989381 >> v2:172.21.15.17:3300/0 conn(0x7ffa3009eb70 0x7ffa3009f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 --2- 172.21.15.119:0/3308989381 >> v2:172.21.15.119:3300/0 conn(0x7ffa300a10f0 0x7ffa30144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): get_auth_request con 0x7ffa300a10f0 auth_method 0 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:58.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 1 -- 172.21.15.119:0/3308989381 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247844446 0 0) 0x7ffa300a02f0 con 0x7ffa3009f500 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient(hunting): _negotiate 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient(hunting): my global_id is 34615 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 1 -- 172.21.15.119:0/3308989381 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa1c003530 con 0x7ffa3009f500 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient(hunting): handle_auth_done global_id 34618 payload 274 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 -- 172.21.15.119:0/3308989381 >> v2:172.21.15.17:3300/0 conn(0x7ffa3009eb70 msgr2=0x7ffa3009f8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 --2- 172.21.15.119:0/3308989381 >> v2:172.21.15.17:3300/0 conn(0x7ffa3009eb70 0x7ffa3009f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 -- 172.21.15.119:0/3308989381 >> v1:172.21.15.119:6789/0 conn(0x7ffa3009f500 legacy=0x7ffa300a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 monclient: found mon.noname-c 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 -- 172.21.15.119:0/3308989381 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa30074260 con 0x7ffa300a10f0 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2ed9d640 1 --2- 172.21.15.119:0/3308989381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa300a10f0 0x7ffa30144000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009d90 tx=0x7ffa2801c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457c5d87b9d0ad8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 1 -- 172.21.15.119:0/3308989381 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa280b2070 con 0x7ffa300a10f0 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: dump: 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:58.769 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 1 -- 172.21.15.119:0/3308989381 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa280a57c0 con 0x7ffa300a10f0 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 1 -- 172.21.15.119:0/3308989381 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa280a5b90 con 0x7ffa300a10f0 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa2dd9b640 10 monclient: dump: 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:58.770 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 5 monclient: authenticate success, global_id 34618 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:58.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.758+0000 7ffa349d6640 4 set_mon_vals no callback set 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: shutdown 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/3308989381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa300a10f0 msgr2=0x7ffa30144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 --2- 172.21.15.119:0/3308989381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa300a10f0 0x7ffa30144000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009d90 tx=0x7ffa2801c810 comp rx=0 tx=0).stop 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/3308989381 shutdown_connections 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 --2- 172.21.15.119:0/3308989381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa300a10f0 0x7ffa30144000 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 --2- 172.21.15.119:0/3308989381 >> v2:172.21.15.17:3300/0 conn(0x7ffa3009eb70 0x7ffa3009f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/3308989381 >> 172.21.15.119:0/3308989381 conn(0x7ffa300a2ae0 msgr2=0x7ffa300a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/3308989381 shutdown_connections 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/3308989381 wait complete. 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: build_initial_monmap 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: monmap: 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:58.777 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 Processor -- start 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- start start 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: init 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 0x7ffa30128fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): picked mon.noname-b con 0x7ffa3009eb70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa3009f500 0x7ffa301d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): picked mon.noname-c con 0x7ffa3009f500 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa300a10f0 0x7ffa30126ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:58.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): picked mon.noname-a con 0x7ffa300a10f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300a39e0 con 0x7ffa300a10f0 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300a3860 con 0x7ffa3009eb70 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffa30146d60 con 0x7ffa3009f500 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient(hunting): authenticate will time out at 1177.561873s 2024-09-15T21:20:58.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 0x7ffa30128fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 0x7ffa30128fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35690/0 (socket says 172.21.15.119:35690) 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 -- 172.21.15.119:0/222549309 learned_addr learned my addr 172.21.15.119:0/222549309 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): get_auth_request con 0x7ffa3009eb70 auth_method 0 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2ed9d640 1 -- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa300a10f0 msgr2=0x7ffa30126ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2ed9d640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa300a10f0 0x7ffa30126ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:58.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa25d9b640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa3009f500 0x7ffa301d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient(hunting): handle_auth_done global_id 34621 payload 824 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 -- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa3009f500 msgr2=0x7ffa301d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa3009f500 0x7ffa301d45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 -- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa300a10f0 msgr2=0x7ffa30126ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa300a10f0 0x7ffa30126ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 monclient: found mon.noname-b 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 -- 172.21.15.119:0/222549309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3012a4f0 con 0x7ffa3009eb70 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2e59c640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 0x7ffa30128fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c4d0 tx=0x7ffa2000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20017070 con 0x7ffa3009eb70 2024-09-15T21:20:58.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: dump: 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:58.782 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: _finish_auth 0 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: _check_auth_tickets 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 5 monclient: authenticate success, global_id 34621 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa20005cf0 con 0x7ffa3009eb70 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _renew_subs 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3012a7d0 con 0x7ffa3009eb70 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:58.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20007270 con 0x7ffa3009eb70 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 10 monclient: dump: 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:58.784 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _renew_subs 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa2d59a640 4 set_mon_vals no callback set 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3012a210 con 0x7ffa3009eb70 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3009f8e0 con 0x7ffa3009eb70 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffa2001a070 con 0x7ffa3009eb70 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa1003b970 0x7ffa1003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.762+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6045+0+0 (secure 0 0 0) 0x7ffa2004f1d0 con 0x7ffa3009eb70 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.766+0000 7ffa25d9b640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa1003b970 0x7ffa1003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.766+0000 7ffa25d9b640 10 monclient: get_auth_request con 0x7ffa1003b970 auth_method 0 2024-09-15T21:20:58.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.766+0000 7ffa25d9b640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa1003b970 0x7ffa1003de30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa301409d0 tx=0x7ffa18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:58.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.770+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffa2001b3b0 con 0x7ffa3009eb70 2024-09-15T21:20:58.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.770+0000 7ffa277fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:58.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.770+0000 7ffa277fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:58.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.934+0000 7ffa349d6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.934+0000 7ffa349d6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.934+0000 7ffa349d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:58.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:58.934+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7ffa3009d1d0 con 0x7ffa3009eb70 2024-09-15T21:20:59.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.522+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 removed item id -19 name 'bar' from crush map v89) ==== 120+0+0 (secure 0 0 0) 0x7ffa2001b730 con 0x7ffa3009eb70 2024-09-15T21:20:59.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.522+0000 7ffa277fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:59.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.522+0000 7ffa277fe640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'bar' from crush map 2024-09-15T21:20:59.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:59.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:59.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7ffa300a1bf0 con 0x7ffa3009eb70 2024-09-15T21:20:59.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa277fe640 1 -- 172.21.15.119:0/222549309 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 device 'bar' does not appear in the crush map v89) ==== 120+0+0 (secure 0 0 0) 0x7ffa20010c00 con 0x7ffa3009eb70 2024-09-15T21:20:59.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa277fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T21:20:59.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa277fe640 10 monclient: _finish_command 3 = system:0 device 'bar' does not appear in the crush map 2024-09-15T21:20:59.635 INFO:tasks.workunit.client.0.smithi119.stderr:device 'bar' does not appear in the crush map 2024-09-15T21:20:59.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa1003b970 msgr2=0x7ffa1003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:59.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa1003b970 0x7ffa1003de30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa301409d0 tx=0x7ffa18008040 comp rx=0 tx=0).stop 2024-09-15T21:20:59.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 10 monclient: shutdown 2024-09-15T21:20:59.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 msgr2=0x7ffa30128fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 0x7ffa30128fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c4d0 tx=0x7ffa2000c990 comp rx=0 tx=0).stop 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 shutdown_connections 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa1003b970 0x7ffa1003de30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa300a10f0 0x7ffa30126ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa3009f500 0x7ffa301d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 --2- 172.21.15.119:0/222549309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3009eb70 0x7ffa30128fd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 >> 172.21.15.119:0/222549309 conn(0x7ffa300a2ae0 msgr2=0x7ffa300725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 shutdown_connections 2024-09-15T21:20:59.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.630+0000 7ffa349d6640 1 -- 172.21.15.119:0/222549309 wait complete. 2024-09-15T21:20:59.652 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm foo 2024-09-15T21:20:59.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.762+0000 7fe2e0202640 10 monclient: get_monmap_and_config 2024-09-15T21:20:59.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.762+0000 7fe2e0202640 10 monclient: build_initial_monmap 2024-09-15T21:20:59.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.762+0000 7fe2e0202640 10 monclient: monmap: 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:20:59.768024+0000 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:20:59.768024+0000 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:20:59.770 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:59.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.762+0000 7fe2e0202640 1 Processor -- start 2024-09-15T21:20:59.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 1 -- start start 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient: init 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe2d809f640 addr v1:172.21.15.119:6789/0 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2d80a0130 0x7fe2d809d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): picked mon.noname-c con 0x7fe2d80a0130 addr v2:172.21.15.119:3300/0 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2d8139c00 0x7fe2d813bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): picked mon.noname-a con 0x7fe2d8139c00 addr v2:172.21.15.17:3300/0 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d809e880 con 0x7fe2d809f640 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d809eb60 con 0x7fe2d8139c00 2024-09-15T21:20:59.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d809ee40 con 0x7fe2d80a0130 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient(hunting): authenticate will time out at 1178.564839s 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2ddf9e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2d809f640 0x7fe2d809fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39062/0 (socket says 172.21.15.119:39062) 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2ddf9e640 1 -- 172.21.15.119:0/2314583835 learned_addr learned my addr 172.21.15.119:0/2314583835 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 --2- 172.21.15.119:0/2314583835 >> v2:172.21.15.119:3300/0 conn(0x7fe2d80a0130 0x7fe2d809d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2de79f640 1 -- 172.21.15.119:0/2314583835 >> v2:172.21.15.17:3300/0 conn(0x7fe2d8139c00 msgr2=0x7fe2d813bff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2de79f640 1 --2- 172.21.15.119:0/2314583835 >> v2:172.21.15.17:3300/0 conn(0x7fe2d8139c00 0x7fe2d813bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): get_auth_request con 0x7fe2d80a0130 auth_method 0 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 1 -- 172.21.15.119:0/2314583835 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419762687 0 0) 0x7fe2d809e880 con 0x7fe2d809f640 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient(hunting): _negotiate 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient(hunting): my global_id is 34627 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 1 -- 172.21.15.119:0/2314583835 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2c4005740 con 0x7fe2d809f640 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient(hunting): handle_auth_done global_id 34624 payload 274 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 -- 172.21.15.119:0/2314583835 >> v2:172.21.15.17:3300/0 conn(0x7fe2d8139c00 msgr2=0x7fe2d813bff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 --2- 172.21.15.119:0/2314583835 >> v2:172.21.15.17:3300/0 conn(0x7fe2d8139c00 0x7fe2d813bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 -- 172.21.15.119:0/2314583835 >> v1:172.21.15.119:6789/0 conn(0x7fe2d809f640 legacy=0x7fe2d809fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient: _finish_hunting 0 2024-09-15T21:20:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 monclient: found mon.noname-c 2024-09-15T21:20:59.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:59.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 -- 172.21.15.119:0/2314583835 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d80a2840 con 0x7fe2d80a0130 2024-09-15T21:20:59.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dd79d640 1 --2- 172.21.15.119:0/2314583835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d80a0130 0x7fe2d809d9a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2d809e5a0 tx=0x7fe2c801c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5afbd62f2b92f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:59.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 1 -- 172.21.15.119:0/2314583835 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80b0070 con 0x7fe2d80a0130 2024-09-15T21:20:59.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:59.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: dump: 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: _finish_auth 0 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: _check_auth_tickets 2024-09-15T21:20:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 1 -- 172.21.15.119:0/2314583835 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c80ab600 con 0x7fe2d80a0130 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 5 monclient: authenticate success, global_id 34624 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 10 monclient: get_monmap_and_config success 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 1 -- 172.21.15.119:0/2314583835 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ab9d0 con 0x7fe2d80a0130 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2e0202640 4 set_mon_vals no callback set 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.766+0000 7fe2dcf9c640 10 monclient: dump: 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: shutdown 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/2314583835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d80a0130 msgr2=0x7fe2d809d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 --2- 172.21.15.119:0/2314583835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d80a0130 0x7fe2d809d9a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2d809e5a0 tx=0x7fe2c801c360 comp rx=0 tx=0).stop 2024-09-15T21:20:59.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/2314583835 shutdown_connections 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 --2- 172.21.15.119:0/2314583835 >> v2:172.21.15.17:3300/0 conn(0x7fe2d8139c00 0x7fe2d813bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 --2- 172.21.15.119:0/2314583835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d80a0130 0x7fe2d809d9a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/2314583835 >> 172.21.15.119:0/2314583835 conn(0x7fe2d80a2ae0 msgr2=0x7fe2d80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/2314583835 shutdown_connections 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/2314583835 wait complete. 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: build_initial_monmap 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: monmap: 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:20:59.784 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 Processor -- start 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- start start 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: init 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _reopen_session rank -1 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 0x7fe2d811c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): picked mon.noname-b con 0x7fe2d809f640 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2d80a0130 0x7fe2d811cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:59.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): picked mon.noname-c con 0x7fe2d80a0130 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d8139c00 0x7fe2d811d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): picked mon.noname-a con 0x7fe2d8139c00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d80a1c00 con 0x7fe2d8139c00 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d80a1a80 con 0x7fe2d809f640 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): start opening mon connection 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d80a1d80 con 0x7fe2d80a0130 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): _renew_subs 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient(hunting): authenticate will time out at 1178.569815s 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 0x7fe2d811c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 0x7fe2d811c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35712/0 (socket says 172.21.15.119:35712) 2024-09-15T21:20:59.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 -- 172.21.15.119:0/1796450395 learned_addr learned my addr 172.21.15.119:0/1796450395 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): get_auth_request con 0x7fe2d809f640 auth_method 0 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2de79f640 1 -- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d8139c00 msgr2=0x7fe2d811d0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2de79f640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d8139c00 0x7fe2d811d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient(hunting): handle_auth_done global_id 34630 payload 824 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 -- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2d80a0130 msgr2=0x7fe2d811cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2d80a0130 0x7fe2d811cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 -- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d8139c00 msgr2=0x7fe2d811d0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:20:59.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d8139c00 0x7fe2d811d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient: _finish_hunting 0 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 monclient: found mon.noname-b 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 -- 172.21.15.119:0/1796450395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d809c770 con 0x7fe2d809f640 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ddf9e640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 0x7fe2d811c630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c4d0 tx=0x7fe2d400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4017070 con 0x7fe2d809f640 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: dump: 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:59.788 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: _finish_auth 0 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d4005cf0 con 0x7fe2d809f640 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:20:59.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4007270 con 0x7fe2d809f640 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 10 monclient: dump: 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:20:59.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 5 monclient: authenticate success, global_id 34630 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2cffff640 4 set_mon_vals no callback set 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _renew_subs 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d811ea80 con 0x7fe2d809f640 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _renew_subs 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d809d1d0 con 0x7fe2d809f640 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d809d9a0 con 0x7fe2d809f640 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe2d401a070 con 0x7fe2d809f640 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe2bc03b970 0x7fe2bc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.770+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5916+0+0 (secure 0 0 0) 0x7fe2d404f1d0 con 0x7fe2d809f640 2024-09-15T21:20:59.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.774+0000 7fe2dd79d640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe2bc03b970 0x7fe2bc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:20:59.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.774+0000 7fe2dd79d640 10 monclient: get_auth_request con 0x7fe2bc03b970 auth_method 0 2024-09-15T21:20:59.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.774+0000 7fe2dd79d640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe2bc03b970 0x7fe2bc03de30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad000 tx=0x7fe2c8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:20:59.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.778+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe2d809c770 con 0x7fe2d809f640 2024-09-15T21:20:59.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.778+0000 7fe2ce7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:20:59.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.778+0000 7fe2ce7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:20:59.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.950+0000 7fe2e0202640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:59.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.950+0000 7fe2e0202640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:20:59.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.950+0000 7fe2e0202640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:20:59.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:20:59.950+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7fe2d811efa0 con 0x7fe2d809f640 2024-09-15T21:21:00.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.526+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v90) ==== 120+0+0 (secure 0 0 0) 0x7fe2d401b3b0 con 0x7fe2d809f640 2024-09-15T21:21:00.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.526+0000 7fe2ce7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:00.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.526+0000 7fe2ce7fc640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-09-15T21:21:00.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.634+0000 7fe2e0202640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:00.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.634+0000 7fe2e0202640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:00.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.634+0000 7fe2e0202640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.634+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7fe2d8072d00 con 0x7fe2d809f640 2024-09-15T21:21:00.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2ce7fc640 1 -- 172.21.15.119:0/1796450395 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v90) ==== 120+0+0 (secure 0 0 0) 0x7fe2d4019d60 con 0x7fe2d809f640 2024-09-15T21:21:00.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2ce7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:00.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2ce7fc640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-09-15T21:21:00.643 INFO:tasks.workunit.client.0.smithi119.stderr:device 'foo' does not appear in the crush map 2024-09-15T21:21:00.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe2bc03b970 msgr2=0x7fe2bc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:00.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe2bc03b970 0x7fe2bc03de30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad000 tx=0x7fe2c8004740 comp rx=0 tx=0).stop 2024-09-15T21:21:00.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 10 monclient: shutdown 2024-09-15T21:21:00.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:00.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 msgr2=0x7fe2d811c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 0x7fe2d811c630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c4d0 tx=0x7fe2d400c990 comp rx=0 tx=0).stop 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 shutdown_connections 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe2bc03b970 0x7fe2bc03de30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d8139c00 0x7fe2d811d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2d80a0130 0x7fe2d811cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 --2- 172.21.15.119:0/1796450395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d809f640 0x7fe2d811c630 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 >> 172.21.15.119:0/1796450395 conn(0x7fe2d80a2ae0 msgr2=0x7fe2d80a4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 shutdown_connections 2024-09-15T21:21:00.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.638+0000 7fe2e0202640 1 -- 172.21.15.119:0/1796450395 wait complete. 2024-09-15T21:21:00.661 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.9 host2 2024-09-15T21:21:00.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.770+0000 7f46454dc640 10 monclient: get_monmap_and_config 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.770+0000 7f46454dc640 10 monclient: build_initial_monmap 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.770+0000 7f46454dc640 10 monclient: monmap: 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:00.775539+0000 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:00.775539+0000 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:00.777 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:00.778 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:00.778 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.770+0000 7f46454dc640 1 Processor -- start 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.770+0000 7f46454dc640 1 -- start start 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.770+0000 7f46454dc640 10 monclient: init 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a0900 0x7f46400a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): picked mon.noname-b con 0x7f46400a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f46400a1220 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4640135df0 0x7f464013dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): picked mon.noname-c con 0x7f4640135df0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46400a2240 con 0x7f46400a1220 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f464009fdd0 con 0x7f4640135df0 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f46400a1c80 con 0x7f46400a0900 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient(hunting): authenticate will time out at 1179.572293s 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4640135df0 0x7f464013dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463e7fc640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f46400a1220 legacy=0x7f46401356e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4640135df0 0x7f464013dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35714/0 (socket says 172.21.15.119:35714) 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 -- 172.21.15.119:0/1033077049 learned_addr learned my addr 172.21.15.119:0/1033077049 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): get_auth_request con 0x7f4640135df0 auth_method 0 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463effd640 1 --2- 172.21.15.119:0/1033077049 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a0900 0x7f46400a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463effd640 10 monclient(hunting): get_auth_request con 0x7f46400a0900 auth_method 0 2024-09-15T21:21:00.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient(hunting): handle_auth_done global_id 34633 payload 274 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 -- 172.21.15.119:0/1033077049 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a0900 msgr2=0x7f46400a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 --2- 172.21.15.119:0/1033077049 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a0900 0x7f46400a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 -- 172.21.15.119:0/1033077049 >> v1:172.21.15.17:6789/0 conn(0x7f46400a1220 legacy=0x7f46401356e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:00.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:21:00.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:00.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 -- 172.21.15.119:0/1033077049 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46400a1f60 con 0x7f4640135df0 2024-09-15T21:21:00.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463f7fe640 1 --2- 172.21.15.119:0/1033077049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f464013dc30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4638009e50 tx=0x7f463801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43d55b21c98353c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 1 -- 172.21.15.119:0/1033077049 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380a5750 con 0x7f4640135df0 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: dump: 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:00.795 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 5 monclient: authenticate success, global_id 34633 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 1 -- 172.21.15.119:0/1033077049 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380a58f0 con 0x7f4640135df0 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:00.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 4 set_mon_vals no callback set 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 1 -- 172.21.15.119:0/1033077049 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380a5cc0 con 0x7f4640135df0 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f463dffb640 10 monclient: dump: 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:00.797 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 10 monclient: shutdown 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- 172.21.15.119:0/1033077049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 msgr2=0x7f464013dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 --2- 172.21.15.119:0/1033077049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f464013dc30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4638009e50 tx=0x7f463801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- 172.21.15.119:0/1033077049 shutdown_connections 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 --2- 172.21.15.119:0/1033077049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f464013dc30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 --2- 172.21.15.119:0/1033077049 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a0900 0x7f46400a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- 172.21.15.119:0/1033077049 >> 172.21.15.119:0/1033077049 conn(0x7f464009be60 msgr2=0x7f464009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- 172.21.15.119:0/1033077049 shutdown_connections 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.774+0000 7f46454dc640 1 -- 172.21.15.119:0/1033077049 wait complete. 2024-09-15T21:21:00.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: build_initial_monmap 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: monmap: 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 Processor -- start 2024-09-15T21:21:00.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 -- start start 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: init 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f464013cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): picked mon.noname-a con 0x7f46400a0900 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f4640134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): picked mon.noname-c con 0x7f46400a1220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): picked mon.noname-b con 0x7f4640135df0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f464006ee90 con 0x7f46400a0900 2024-09-15T21:21:00.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f464006ed10 con 0x7f4640135df0 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f46400a0050 con 0x7f46400a1220 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): authenticate will time out at 1179.577083s 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35718/0 (socket says 172.21.15.119:35718) 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 learned_addr learned my addr 172.21.15.119:0/4044709483 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): get_auth_request con 0x7f4640135df0 auth_method 0 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:00.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463effd640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 msgr2=0x7f464013cbd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463effd640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f464013cbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463e7fc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f4640134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_done global_id 34636 payload 824 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 msgr2=0x7f4640134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f4640134a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 msgr2=0x7f464013cbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f464013cbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:00.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4640120e90 con 0x7f4640135df0 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463e7fc640 10 monclient: get_auth_request con 0x7f46400a1220 auth_method 0 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463e7fc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f4640134a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46380af040 tx=0x7f4638019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380bc020 con 0x7f4640135df0 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: dump: 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:00.803 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380b8670 con 0x7f4640135df0 2024-09-15T21:21:00.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 5 monclient: authenticate success, global_id 34636 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: _renew_subs 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4640121170 con 0x7f4640135df0 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380b8a60 con 0x7f4640135df0 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: dump: 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.805 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463d7fa640 4 set_mon_vals no callback set 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: _renew_subs 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464006f0b0 con 0x7f4640135df0 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 msgr2=0x7f4640134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 msgr2=0x7f4640134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46380af040 tx=0x7f4638019690 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:00.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46380af040 tx=0x7f4638019690 comp rx=0 tx=0).stop 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 msgr2=0x7f4640134fc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4640135df0 0x7f4640134fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f462c005860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): picked mon.a con 0x7f46400a0900 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:00.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f462c008260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): picked mon.c con 0x7f46400a1220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f462c008be0 0x7f462c00b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): picked mon.b con 0x7f462c008be0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f462c00bb20 con 0x7f46400a0900 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f462c00bca0 con 0x7f462c008be0 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f462c00be20 con 0x7f46400a1220 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:00.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46454dc640 10 monclient(hunting): _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f462c008be0 0x7f462c00b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): get_auth_request con 0x7f462c008be0 auth_method 0 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463effd640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 msgr2=0x7f462c005860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463effd640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f462c005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient(hunting): handle_auth_done global_id 34636 payload 293 2024-09-15T21:21:00.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463e7fc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f462c008260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 msgr2=0x7f462c008260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f462c008260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 msgr2=0x7f462c005860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f462c005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 monclient: found mon.b 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f462c00c140 con 0x7f462c008be0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46400a0ce0 con 0x7f462c008be0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464006f0b0 con 0x7f462c008be0 2024-09-15T21:21:00.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463f7fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f462c008be0 0x7f462c00b070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46380b8fd0 tx=0x7f46380a7070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380b8c00 con 0x7f462c008be0 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: dump: 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:00.811 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380ca6c0 con 0x7f462c008be0 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f463cff9640 4 set_mon_vals no callback set 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.778+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f46380bf020 con 0x7f462c008be0 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.782+0000 7f46277fe640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4640135df0 0x7f462c045ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:00.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.782+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 4 ==== osd_map(90..90 src has 1..90) ==== 5787+0+0 (secure 0 0 0) 0x7f46380eca60 con 0x7f462c008be0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.782+0000 7f463effd640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4640135df0 0x7f462c045ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.782+0000 7f463effd640 10 monclient: get_auth_request con 0x7f4640135df0 auth_method 0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.782+0000 7f463effd640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4640135df0 0x7f462c045ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4634006aa0 tx=0x7f4634006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.786+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f46380c6030 con 0x7f462c008be0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.786+0000 7f46277fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.786+0000 7f46277fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.786+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4638016600 con 0x7f462c008be0 2024-09-15T21:21:00.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.786+0000 7f46277fe640 10 monclient: handle_mon_command_ack 1 not found 2024-09-15T21:21:00.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.954+0000 7f46454dc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T21:21:00.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.954+0000 7f46454dc640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T21:21:00.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.954+0000 7f46454dc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:00.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:00.954+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) -- 0x7f464013d590 con 0x7f462c008be0 2024-09-15T21:21:01.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.542+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 removed item id 9 name 'osd.9' from crush map v91) ==== 143+0+0 (secure 0 0 0) 0x7f46380ad3f0 con 0x7f462c008be0 2024-09-15T21:21:01.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.542+0000 7f46277fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T21:21:01.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.542+0000 7f46277fe640 10 monclient: _finish_command 2 = system:0 removed item id 9 name 'osd.9' from crush map 2024-09-15T21:21:01.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46454dc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T21:21:01.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46454dc640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T21:21:01.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46454dc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:01.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) -- 0x7f464013d590 con 0x7f462c008be0 2024-09-15T21:21:01.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46277fe640 1 -- 172.21.15.119:0/4044709483 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 device 'osd.9' does not appear in the crush map v91) ==== 145+0+0 (secure 0 0 0) 0x7f46380d4450 con 0x7f462c008be0 2024-09-15T21:21:01.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46277fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T21:21:01.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.646+0000 7f46277fe640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-09-15T21:21:01.654 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.9' does not appear in the crush map 2024-09-15T21:21:01.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4640135df0 msgr2=0x7f462c045ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:01.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4640135df0 0x7f462c045ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4634006aa0 tx=0x7f4634006a30 comp rx=0 tx=0).stop 2024-09-15T21:21:01.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 10 monclient: shutdown 2024-09-15T21:21:01.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:01.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f462c008be0 msgr2=0x7f462c00b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:01.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f462c008be0 0x7f462c00b070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46380b8fd0 tx=0x7f46380a7070 comp rx=0 tx=0).stop 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 shutdown_connections 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4640135df0 0x7f462c045ed0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f462c008be0 0x7f462c00b070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46400a1220 0x7f462c008260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 --2- 172.21.15.119:0/4044709483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46400a0900 0x7f462c005860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 >> 172.21.15.119:0/4044709483 conn(0x7f464009be60 msgr2=0x7f4640136bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 shutdown_connections 2024-09-15T21:21:01.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.650+0000 7f46454dc640 1 -- 172.21.15.119:0/4044709483 wait complete. 2024-09-15T21:21:01.671 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm host2 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient: get_monmap_and_config 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient: build_initial_monmap 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient: monmap: 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:01.786609+0000 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:01.786609+0000 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:01.788 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:01.789 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:01.789 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:01.789 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:01.789 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 1 Processor -- start 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 1 -- start start 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient: init 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2850135ca0 0x7f2850136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient(hunting): picked mon.noname-a con 0x7f2850135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f28500a0160 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f285006ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28500a1800 con 0x7f28500a0160 2024-09-15T21:21:01.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28501365c0 con 0x7f285006ec30 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f285009cf30 con 0x7f2850135ca0 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2856fd9640 10 monclient(hunting): authenticate will time out at 1180.583376s 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284ffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f28500a0160 legacy=0x7f28500a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2854d75640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f2850135ca0 msgr2=0x7f2850136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2855576640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f285006ec30 0x7f28500a10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39078/0 (socket says 172.21.15.119:39078) 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2855576640 1 -- 172.21.15.119:0/1935493031 learned_addr learned my addr 172.21.15.119:0/1935493031 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f2854d75640 1 --2- 172.21.15.119:0/1935493031 >> v2:172.21.15.17:3300/0 conn(0x7f2850135ca0 0x7f2850136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373252224 0 0) 0x7f28501365c0 con 0x7f285006ec30 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284f7fe640 10 monclient(hunting): my global_id is 34639 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.782+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2838003530 con 0x7f285006ec30 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3554123146 0 0) 0x7f2838003530 con 0x7f285006ec30 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 >> v2:172.21.15.17:3300/0 conn(0x7f2850135ca0 msgr2=0x7f2850136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 --2- 172.21.15.119:0/1935493031 >> v2:172.21.15.17:3300/0 conn(0x7f2850135ca0 0x7f2850136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 >> v1:172.21.15.17:6789/0 conn(0x7f28500a0160 legacy=0x7f28500a09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28500a1ca0 con 0x7f285006ec30 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:01.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f2848001a80 con 0x7f285006ec30 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: dump: 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:01.796 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 >> v1:172.21.15.119:6789/0 conn(0x7f285006ec30 legacy=0x7f28500a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28380039c0 0x7f2838003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): picked mon.c con 0x7f28380039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2838004810 0x7f2838004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): picked mon.a con 0x7f2838004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 0x7f283800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): picked mon.b con 0x7f28380053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f285009cf30 con 0x7f2838004810 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f283800c310 con 0x7f28380053d0 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f283800c510 con 0x7f28380039c0 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 0x7f283800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284ffff640 1 -- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2838004810 msgr2=0x7f2838004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f284ffff640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2838004810 0x7f2838004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 10 monclient(hunting): get_auth_request con 0x7f28380053d0 auth_method 0 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:01.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 1 -- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 msgr2=0x7f283800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 1 -- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 msgr2=0x7f283800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 0x7f283800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 0x7f283800ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28380039c0 0x7f2838003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): get_auth_request con 0x7f28380039c0 auth_method 0 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:01.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient(hunting): handle_auth_done global_id 34639 payload 293 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 -- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 msgr2=0x7f283800ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380053d0 0x7f283800ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 -- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2838004810 msgr2=0x7f2838004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2838004810 0x7f2838004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient: _finish_hunting 0 2024-09-15T21:21:01.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 monclient: found mon.c 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2855576640 1 -- 172.21.15.119:0/1935493031 reap_dead start 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 1 -- 172.21.15.119:0/1935493031 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2838003530 con 0x7f28380039c0 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient: _finish_auth 0 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 10 monclient: _check_auth_tickets 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2854d75640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2856fd9640 5 monclient: authenticate success, global_id 34639 2024-09-15T21:21:01.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.786+0000 7f2856fd9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:01.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28380039c0 0x7f2838003dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28400038c0 tx=0x7f284001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e580e9d3ceee8f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:01.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840009470 con 0x7f28380039c0 2024-09-15T21:21:01.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:01.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: dump: 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:01.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2840003af0 con 0x7f28380039c0 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 1 -- 172.21.15.119:0/1935493031 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400050d0 con 0x7f28380039c0 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 4 set_mon_vals no callback set 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284f7fe640 10 monclient: dump: 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.808 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: shutdown 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28380039c0 msgr2=0x7f2838003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28380039c0 0x7f2838003dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28400038c0 tx=0x7f284001c340 comp rx=0 tx=0).stop 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- 172.21.15.119:0/1935493031 shutdown_connections 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 --2- 172.21.15.119:0/1935493031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28380039c0 0x7f2838003dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- 172.21.15.119:0/1935493031 >> 172.21.15.119:0/1935493031 conn(0x7f28500a2ae0 msgr2=0x7f28500a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- 172.21.15.119:0/1935493031 shutdown_connections 2024-09-15T21:21:01.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- 172.21.15.119:0/1935493031 wait complete. 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: build_initial_monmap 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: monmap: 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 Processor -- start 2024-09-15T21:21:01.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- start start 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: init 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 0x7f285009d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): picked mon.noname-b con 0x7f28380039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285009da50 0x7f285009ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): picked mon.noname-a con 0x7f285009da50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28501cfaa0 0x7f28501cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): picked mon.noname-c con 0x7f28501cfaa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f28500a1370 con 0x7f285009da50 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f28500a11f0 con 0x7f28380039c0 2024-09-15T21:21:01.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f28500a14f0 con 0x7f28501cfaa0 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2856fd9640 10 monclient(hunting): authenticate will time out at 1180.591529s 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 0x7f285009d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284ffff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285009da50 msgr2=0x7f285009ded0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 0x7f285009d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35752/0 (socket says 172.21.15.119:35752) 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 -- 172.21.15.119:0/3686202776 learned_addr learned my addr 172.21.15.119:0/3686202776 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f284ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285009da50 0x7f285009ded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): get_auth_request con 0x7f28380039c0 auth_method 0 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:01.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:01.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:01.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:01.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2855576640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28501cfaa0 0x7f28501cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:01.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient(hunting): handle_auth_done global_id 34642 payload 824 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 -- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28501cfaa0 msgr2=0x7f28501cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28501cfaa0 0x7f28501cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 -- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285009da50 msgr2=0x7f285009ded0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285009da50 0x7f285009ded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient: _finish_hunting 0 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 monclient: found mon.noname-b 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:01.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2854d75640 1 -- 172.21.15.119:0/3686202776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850125ae0 con 0x7f28380039c0 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2855576640 10 monclient: get_auth_request con 0x7f28501cfaa0 auth_method 0 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.790+0000 7f2855576640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28501cfaa0 0x7f28501cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2854d75640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 0x7f285009d510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f284001c170 tx=0x7f2838005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400b9030 con 0x7f28380039c0 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: dump: 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:01.818 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28400050d0 con 0x7f28380039c0 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 5 monclient: authenticate success, global_id 34642 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 monclient: _renew_subs 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400052a0 con 0x7f28380039c0 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:01.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285009d1d0 con 0x7f28380039c0 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284effd640 4 set_mon_vals no callback set 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 monclient: _renew_subs 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28501271d0 con 0x7f28380039c0 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 10 monclient: dump: 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:01.820 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28501389f0 con 0x7f28380039c0 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f28400bc070 con 0x7f28380039c0 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283c03b9a0 0x7f283c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f28400e9d20 con 0x7f28380039c0 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284ffff640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283c03b9a0 0x7f283c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:01.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284ffff640 10 monclient: get_auth_request con 0x7f283c03b9a0 auth_method 0 2024-09-15T21:21:01.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.794+0000 7f284ffff640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283c03b9a0 0x7f283c03de60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2850072050 tx=0x7f2844006d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:01.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.798+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f28400c3070 con 0x7f28380039c0 2024-09-15T21:21:01.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.798+0000 7f284d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:01.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.798+0000 7f284d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:01.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.962+0000 7f2856fd9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T21:21:01.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.962+0000 7f2856fd9640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T21:21:01.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.962+0000 7f2856fd9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:01.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:01.962+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) -- 0x7f285009f8f0 con 0x7f28380039c0 2024-09-15T21:21:02.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.546+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v92) ==== 124+0+0 (secure 0 0 0) 0x7f2850125ae0 con 0x7f28380039c0 2024-09-15T21:21:02.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.546+0000 7f284d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T21:21:02.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.546+0000 7f284d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-09-15T21:21:02.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T21:21:02.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T21:21:02.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:02.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) -- 0x7f2850133210 con 0x7f28380039c0 2024-09-15T21:21:02.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f284d7fa640 1 -- 172.21.15.119:0/3686202776 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 device 'host2' does not appear in the crush map v92) ==== 124+0+0 (secure 0 0 0) 0x7f2840016050 con 0x7f28380039c0 2024-09-15T21:21:02.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f284d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T21:21:02.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f284d7fa640 10 monclient: _finish_command 3 = system:0 device 'host2' does not appear in the crush map 2024-09-15T21:21:02.664 INFO:tasks.workunit.client.0.smithi119.stderr:device 'host2' does not appear in the crush map 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283c03b9a0 msgr2=0x7f283c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283c03b9a0 0x7f283c03de60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2850072050 tx=0x7f2844006d50 comp rx=0 tx=0).stop 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 10 monclient: shutdown 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.658+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 msgr2=0x7f285009d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 0x7f285009d510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f284001c170 tx=0x7f2838005840 comp rx=0 tx=0).stop 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 shutdown_connections 2024-09-15T21:21:02.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f283c03b9a0 0x7f283c03de60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f28501cfaa0 0x7f28501cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f285009da50 0x7f285009ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 --2- 172.21.15.119:0/3686202776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28380039c0 0x7f285009d510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 >> 172.21.15.119:0/3686202776 conn(0x7f28500a2ae0 msgr2=0x7f28500a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:02.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 shutdown_connections 2024-09-15T21:21:02.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.662+0000 7f2856fd9640 1 -- 172.21.15.119:0/3686202776 wait complete. 2024-09-15T21:21:02.683 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket foo host 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.790+0000 7fe2b6e30640 10 monclient: get_monmap_and_config 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.790+0000 7fe2b6e30640 10 monclient: build_initial_monmap 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.790+0000 7fe2b6e30640 10 monclient: monmap: 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:02.795843+0000 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:02.795843+0000 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:02.798 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:02.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.790+0000 7fe2b6e30640 1 Processor -- start 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.790+0000 7fe2b6e30640 1 -- start start 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.790+0000 7fe2b6e30640 10 monclient: init 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b006dc00 0x7fe2b00a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient(hunting): picked mon.noname-b con 0x7fe2b006dc00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe2b009ec20 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe2b00a18a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b009e240 con 0x7fe2b009ec20 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b009e520 con 0x7fe2b00a18a0 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:02.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b009e800 con 0x7fe2b006dc00 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b6e30640 10 monclient(hunting): authenticate will time out at 1181.592543s 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2affff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fe2b009ec20 legacy=0x7fe2b00a1190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b53cd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2b00a18a0 0x7fe2b013b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42170/0 (socket says 172.21.15.119:42170) 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b53cd640 1 -- 172.21.15.119:0/179385809 learned_addr learned my addr 172.21.15.119:0/179385809 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 1 --2- 172.21.15.119:0/179385809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b006dc00 0x7fe2b00a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): get_auth_request con 0x7fe2b006dc00 auth_method 0 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:02.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2af7fe640 1 -- 172.21.15.119:0/179385809 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935615663 0 0) 0x7fe2b009e520 con 0x7fe2b00a18a0 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2af7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2af7fe640 10 monclient(hunting): my global_id is 34645 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2af7fe640 1 -- 172.21.15.119:0/179385809 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe298005740 con 0x7fe2b00a18a0 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_done global_id 44213 payload 274 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/179385809 >> v1:172.21.15.119:6789/0 conn(0x7fe2b00a18a0 legacy=0x7fe2b013b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/179385809 >> v1:172.21.15.17:6789/0 conn(0x7fe2b009ec20 legacy=0x7fe2b00a1190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:02.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 1 monclient: found mon.noname-b 2024-09-15T21:21:02.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:02.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:02.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/179385809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b0074110 con 0x7fe2b006dc00 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.794+0000 7fe2b4bcc640 1 --2- 172.21.15.119:0/179385809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b006dc00 0x7fe2b00a0010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00097f0 tx=0x7fe2a001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7dca3a98d89d548 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 1 -- 172.21.15.119:0/179385809 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b0070 con 0x7fe2b006dc00 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: dump: 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:02.812 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 5 monclient: authenticate success, global_id 44213 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 1 -- 172.21.15.119:0/179385809 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a00ab600 con 0x7fe2b006dc00 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:02.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 1 -- 172.21.15.119:0/179385809 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00ab9b0 con 0x7fe2b006dc00 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 4 set_mon_vals no callback set 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2af7fe640 10 monclient: dump: 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:02.817 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: shutdown 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/179385809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b006dc00 msgr2=0x7fe2b00a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/179385809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b006dc00 0x7fe2b00a0010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00097f0 tx=0x7fe2a001c3a0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/179385809 shutdown_connections 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/179385809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b006dc00 0x7fe2b00a0010 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/179385809 >> 172.21.15.119:0/179385809 conn(0x7fe2b009be60 msgr2=0x7fe2b009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/179385809 shutdown_connections 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/179385809 wait complete. 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: build_initial_monmap 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: monmap: 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:02.819 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:02.820 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:02.820 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:02.820 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:02.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 Processor -- start 2024-09-15T21:21:02.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- start start 2024-09-15T21:21:02.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: init 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 0x7fe2b01cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): picked mon.noname-b con 0x7fe2b006dc00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b009ec20 0x7fe2b0136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): picked mon.noname-c con 0x7fe2b009ec20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2b00a18a0 0x7fe2b0120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): picked mon.noname-a con 0x7fe2b00a18a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b013f000 con 0x7fe2b00a18a0 2024-09-15T21:21:02.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b013ee80 con 0x7fe2b006dc00 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b013f180 con 0x7fe2b009ec20 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient(hunting): authenticate will time out at 1181.598117s 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 0x7fe2b01cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 0x7fe2b01cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46762/0 (socket says 172.21.15.119:46762) 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/919524981 learned_addr learned my addr 172.21.15.119:0/919524981 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): get_auth_request con 0x7fe2b006dc00 auth_method 0 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:02.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b53cd640 1 -- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2b00a18a0 msgr2=0x7fe2b0120770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b53cd640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2b00a18a0 0x7fe2b0120770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2affff640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b009ec20 0x7fe2b0136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient(hunting): handle_auth_done global_id 34648 payload 824 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b009ec20 msgr2=0x7fe2b0136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b009ec20 0x7fe2b0136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2b00a18a0 msgr2=0x7fe2b0120770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2b00a18a0 0x7fe2b0120770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 monclient: found mon.noname-b 2024-09-15T21:21:02.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 -- 172.21.15.119:0/919524981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01242b0 con 0x7fe2b006dc00 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2affff640 10 monclient: get_auth_request con 0x7fe2b009ec20 auth_method 0 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2affff640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b009ec20 0x7fe2b0136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b4bcc640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 0x7fe2b01cf4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2a001aa50 tx=0x7fe2a0019140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b9070 con 0x7fe2b006dc00 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: dump: 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:02.824 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 5 monclient: authenticate success, global_id 34648 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a00abd70 con 0x7fe2b006dc00 2024-09-15T21:21:02.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _renew_subs 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b0124590 con 0x7fe2b006dc00 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b0040 con 0x7fe2b006dc00 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2ad7fa640 10 monclient: dump: 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.826 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2aeffd640 4 set_mon_vals no callback set 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _renew_subs 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b0122230 con 0x7fe2b006dc00 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:02.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.798+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b0137530 con 0x7fe2b006dc00 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.802+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe2a00bc070 con 0x7fe2b006dc00 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.802+0000 7fe2ad7fa640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29c03b970 0x7fe29c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.802+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5631+0+0 (secure 0 0 0) 0x7fe2a00ea450 con 0x7fe2b006dc00 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.802+0000 7fe2affff640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29c03b970 0x7fe29c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.802+0000 7fe2affff640 10 monclient: get_auth_request con 0x7fe29c03b970 auth_method 0 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.802+0000 7fe2affff640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29c03b970 0x7fe29c03de30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0122c70 tx=0x7fe2a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.806+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe2b0137530 con 0x7fe2b006dc00 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.806+0000 7fe2ad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:02.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.806+0000 7fe2ad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:02.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.978+0000 7fe2b6e30640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T21:21:02.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.978+0000 7fe2b6e30640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T21:21:02.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.978+0000 7fe2b6e30640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:02.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:02.978+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) -- 0x7fe2b00a0010 con 0x7fe2b006dc00 2024-09-15T21:21:03.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.562+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 added bucket foo type host to crush map v93) ==== 138+0+0 (secure 0 0 0) 0x7fe2a00160b0 con 0x7fe2b006dc00 2024-09-15T21:21:03.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.562+0000 7fe2ad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T21:21:03.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.562+0000 7fe2ad7fa640 10 monclient: _finish_command 2 = system:0 added bucket foo type host to crush map 2024-09-15T21:21:03.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2b6e30640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T21:21:03.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2b6e30640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T21:21:03.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2b6e30640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:03.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) -- 0x7fe2b0137110 con 0x7fe2b006dc00 2024-09-15T21:21:03.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2ad7fa640 1 -- 172.21.15.119:0/919524981 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 bucket 'foo' already exists v93) ==== 126+0+0 (secure 0 0 0) 0x7fe2a0003860 con 0x7fe2b006dc00 2024-09-15T21:21:03.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2ad7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T21:21:03.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.666+0000 7fe2ad7fa640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-09-15T21:21:03.673 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'foo' already exists 2024-09-15T21:21:03.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29c03b970 msgr2=0x7fe29c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:03.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29c03b970 0x7fe29c03de30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0122c70 tx=0x7fe2a4008040 comp rx=0 tx=0).stop 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 10 monclient: shutdown 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 msgr2=0x7fe2b01cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 0x7fe2b01cf4b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2a001aa50 tx=0x7fe2a0019140 comp rx=0 tx=0).stop 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 shutdown_connections 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29c03b970 0x7fe29c03de30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2b00a18a0 0x7fe2b0120770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2b009ec20 0x7fe2b0136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 --2- 172.21.15.119:0/919524981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2b006dc00 0x7fe2b01cf4b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 >> 172.21.15.119:0/919524981 conn(0x7fe2b009be60 msgr2=0x7fe2b013ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 shutdown_connections 2024-09-15T21:21:03.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.670+0000 7fe2b6e30640 1 -- 172.21.15.119:0/919524981 wait complete. 2024-09-15T21:21:03.689 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush move foo root=default rack=localrack 2024-09-15T21:21:03.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.806+0000 7fa27491b640 10 monclient: get_monmap_and_config 2024-09-15T21:21:03.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.806+0000 7fa27491b640 10 monclient: build_initial_monmap 2024-09-15T21:21:03.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.806+0000 7fa27491b640 10 monclient: monmap: 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:03.812207+0000 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:03.812207+0000 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:03.814 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:03.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 1 Processor -- start 2024-09-15T21:21:03.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 1 -- start start 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient: init 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa270135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa27009fe70 0x7fa27013aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient(hunting): picked mon.noname-c con 0x7fa27009fe70 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa27013afb0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2700a0300 con 0x7fa270135bf0 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2700a1d80 con 0x7fa27013afb0 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa2700a3600 con 0x7fa27009fe70 2024-09-15T21:21:03.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient(hunting): authenticate will time out at 1182.608825s 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa27009fe70 0x7fa27013aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa27013afb0 0x7fa27013d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42184/0 (socket says 172.21.15.119:42184) 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26ed9d640 1 -- 172.21.15.119:0/3161570967 learned_addr learned my addr 172.21.15.119:0/3161570967 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26e59c640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fa270135bf0 legacy=0x7fa270135fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): get_auth_request con 0x7fa27009fe70 auth_method 0 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient(hunting): handle_auth_done global_id 34651 payload 274 2024-09-15T21:21:03.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 1 -- 172.21.15.119:0/3161570967 >> v1:172.21.15.119:6789/0 conn(0x7fa27013afb0 legacy=0x7fa27013d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 1 -- 172.21.15.119:0/3161570967 >> v1:172.21.15.17:6789/0 conn(0x7fa270135bf0 legacy=0x7fa270135fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 1 monclient: found mon.noname-c 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 1 -- 172.21.15.119:0/3161570967 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173679613 0 0) 0x7fa2700a1d80 con 0x7fa27013afb0 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 1 -- 172.21.15.119:0/3161570967 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2700a2060 con 0x7fa27009fe70 2024-09-15T21:21:03.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:03.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26dd9b640 1 --2- 172.21.15.119:0/3161570967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa27013aa70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097f0 tx=0x7fa25801c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dbeea1daafe1b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:03.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 1 -- 172.21.15.119:0/3161570967 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580b0070 con 0x7fa27009fe70 2024-09-15T21:21:03.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:03.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: dump: 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: _finish_auth 0 2024-09-15T21:21:03.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 1 -- 172.21.15.119:0/3161570967 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2580ab400 con 0x7fa27009fe70 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 5 monclient: authenticate success, global_id 34651 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 1 -- 172.21.15.119:0/3161570967 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580ab7d0 con 0x7fa27009fe70 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa27491b640 4 set_mon_vals no callback set 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.810+0000 7fa26d59a640 10 monclient: dump: 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:03.826 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: shutdown 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3161570967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 msgr2=0x7fa27013aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 --2- 172.21.15.119:0/3161570967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa27013aa70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097f0 tx=0x7fa25801c150 comp rx=0 tx=0).stop 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3161570967 shutdown_connections 2024-09-15T21:21:03.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 --2- 172.21.15.119:0/3161570967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa27013aa70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3161570967 >> 172.21.15.119:0/3161570967 conn(0x7fa2700a66e0 msgr2=0x7fa2700a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3161570967 shutdown_connections 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3161570967 wait complete. 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: build_initial_monmap 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: monmap: 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:03.828 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 Processor -- start 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- start start 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: init 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa2701cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): picked mon.noname-b con 0x7fa27009fe70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa270135bf0 0x7fa2701ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): picked mon.noname-a con 0x7fa270135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa27013afb0 0x7fa2701381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:03.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): picked mon.noname-c con 0x7fa27013afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701416f0 con 0x7fa270135bf0 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa27006ed50 con 0x7fa27009fe70 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa270141870 con 0x7fa27013afb0 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient(hunting): authenticate will time out at 1182.613867s 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26dd9b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa270135bf0 msgr2=0x7fa2701ce180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa2701cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa270135bf0 0x7fa2701ce180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa2701cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46784/0 (socket says 172.21.15.119:46784) 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 -- 172.21.15.119:0/3926497615 learned_addr learned my addr 172.21.15.119:0/3926497615 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:03.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): get_auth_request con 0x7fa27009fe70 auth_method 0 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26ed9d640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa27013afb0 0x7fa2701381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient(hunting): handle_auth_done global_id 34657 payload 824 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 -- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa27013afb0 msgr2=0x7fa2701381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa27013afb0 0x7fa2701381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 -- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa270135bf0 msgr2=0x7fa2701ce180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa270135bf0 0x7fa2701ce180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:03.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 monclient: found mon.noname-b 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 -- 172.21.15.119:0/3926497615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27011dba0 con 0x7fa27009fe70 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26ed9d640 10 monclient: get_auth_request con 0x7fa27013afb0 auth_method 0 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26ed9d640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa27013afb0 0x7fa2701381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26e59c640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa2701cdc40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b800 tx=0x7fa26000bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26000c9c0 con 0x7fa27009fe70 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: dump: 2024-09-15T21:21:03.832 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa260010070 con 0x7fa27009fe70 2024-09-15T21:21:03.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 5 monclient: authenticate success, global_id 34657 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _renew_subs 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27011de80 con 0x7fa27009fe70 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26000cdc0 con 0x7fa27009fe70 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 10 monclient: dump: 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:03.834 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _renew_subs 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26cd99640 4 set_mon_vals no callback set 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27009d240 con 0x7fa27009fe70 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:03.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa26001e3f0 con 0x7fa27009fe70 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa270135fd0 con 0x7fa27009fe70 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa25003b970 0x7fa25003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5658+0+0 (secure 0 0 0) 0x7fa26004fec0 con 0x7fa27009fe70 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26dd9b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa25003b970 0x7fa25003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.814+0000 7fa26dd9b640 10 monclient: get_auth_request con 0x7fa25003b970 auth_method 0 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.818+0000 7fa26dd9b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa25003b970 0x7fa25003de30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2580ad000 tx=0x7fa258008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.822+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa260014a30 con 0x7fa27009fe70 2024-09-15T21:21:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.822+0000 7fa266ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:03.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.822+0000 7fa266ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:04.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.998+0000 7fa27491b640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T21:21:04.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.998+0000 7fa27491b640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T21:21:04.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.998+0000 7fa27491b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:03.998+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) -- 0x7fa27006a830 con 0x7fa27009fe70 2024-09-15T21:21:04.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.570+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v94) ==== 204+0+0 (secure 0 0 0) 0x7fa26001d3a0 con 0x7fa27009fe70 2024-09-15T21:21:04.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.570+0000 7fa266ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T21:21:04.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.570+0000 7fa266ffd640 10 monclient: _finish_command 2 = system:0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-09-15T21:21:04.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa27491b640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T21:21:04.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa27491b640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T21:21:04.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa27491b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) -- 0x7fa2701ce6c0 con 0x7fa27009fe70 2024-09-15T21:21:04.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa266ffd640 1 -- 172.21.15.119:0/3926497615 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v94) ==== 214+0+0 (secure 0 0 0) 0x7fa260014cf0 con 0x7fa27009fe70 2024-09-15T21:21:04.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa266ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T21:21:04.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.682+0000 7fa266ffd640 10 monclient: _finish_command 3 = system:0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-09-15T21:21:04.688 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-09-15T21:21:04.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa25003b970 msgr2=0x7fa25003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:04.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa25003b970 0x7fa25003de30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2580ad000 tx=0x7fa258008040 comp rx=0 tx=0).stop 2024-09-15T21:21:04.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 10 monclient: shutdown 2024-09-15T21:21:04.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 msgr2=0x7fa2701cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa2701cdc40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b800 tx=0x7fa26000bcc0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 shutdown_connections 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa25003b970 0x7fa25003de30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa27013afb0 0x7fa2701381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa270135bf0 0x7fa2701ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 --2- 172.21.15.119:0/3926497615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27009fe70 0x7fa2701cdc40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 >> 172.21.15.119:0/3926497615 conn(0x7fa2700a66e0 msgr2=0x7fa27013bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 shutdown_connections 2024-09-15T21:21:04.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.686+0000 7fa27491b640 1 -- 172.21.15.119:0/3926497615 wait complete. 2024-09-15T21:21:04.705 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush create-or-move osd.7 1.0 root=default 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient: get_monmap_and_config 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient: build_initial_monmap 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient: monmap: 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:04.824851+0000 2024-09-15T21:21:04.826 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:04.824851+0000 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:04.827 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:04.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 Processor -- start 2024-09-15T21:21:04.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 -- start start 2024-09-15T21:21:04.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient: init 2024-09-15T21:21:04.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7920135bf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f792009fe70 0x7f792013aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): picked mon.noname-c con 0x7f792009fe70 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f792013afb0 0x7f792013d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): picked mon.noname-b con 0x7f792013afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79200a0300 con 0x7f7920135bf0 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f79200a3600 con 0x7f792009fe70 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f79200a1d80 con 0x7f792013afb0 2024-09-15T21:21:04.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f7924871640 10 monclient(hunting): authenticate will time out at 1183.621643s 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791e59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7920135bf0 0x7f7920135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42198/0 (socket says 172.21.15.119:42198) 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791e59c640 1 -- 172.21.15.119:0/2032882863 learned_addr learned my addr 172.21.15.119:0/2032882863 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f792009fe70 0x7f792013aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): get_auth_request con 0x7f792009fe70 auth_method 0 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791ed9d640 1 --2- 172.21.15.119:0/2032882863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f792013afb0 0x7f792013d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791ed9d640 10 monclient(hunting): get_auth_request con 0x7f792013afb0 auth_method 0 2024-09-15T21:21:04.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient(hunting): handle_auth_done global_id 34660 payload 274 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 1 -- 172.21.15.119:0/2032882863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f792013afb0 msgr2=0x7f792013d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 1 --2- 172.21.15.119:0/2032882863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f792013afb0 0x7f792013d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 1 -- 172.21.15.119:0/2032882863 >> v1:172.21.15.119:6789/0 conn(0x7f7920135bf0 legacy=0x7f7920135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 1 monclient: found mon.noname-c 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791dd9b640 1 -- 172.21.15.119:0/2032882863 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79200a2060 con 0x7f792009fe70 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791d59a640 1 -- 172.21.15.119:0/2032882863 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513463251 0 0) 0x7f7910001120 con 0x7f7920135bf0 2024-09-15T21:21:04.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.822+0000 7f791d59a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:04.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791dd9b640 1 --2- 172.21.15.119:0/2032882863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792013aa70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f79140097f0 tx=0x7f791401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4694e5e598b033d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:04.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 1 -- 172.21.15.119:0/2032882863 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140b0070 con 0x7f792009fe70 2024-09-15T21:21:04.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: dump: 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: _finish_auth 0 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:21:04.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 5 monclient: authenticate success, global_id 34660 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 1 -- 172.21.15.119:0/2032882863 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79140ab600 con 0x7f792009fe70 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 1 -- 172.21.15.119:0/2032882863 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140ab9d0 con 0x7f792009fe70 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 4 set_mon_vals no callback set 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791d59a640 10 monclient: dump: 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.839 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: shutdown 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/2032882863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 msgr2=0x7f792013aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 --2- 172.21.15.119:0/2032882863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792013aa70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f79140097f0 tx=0x7f791401c360 comp rx=0 tx=0).stop 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/2032882863 shutdown_connections 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 --2- 172.21.15.119:0/2032882863 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f792013afb0 0x7f792013d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 --2- 172.21.15.119:0/2032882863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792013aa70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/2032882863 >> 172.21.15.119:0/2032882863 conn(0x7f79200a66e0 msgr2=0x7f79200a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:04.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/2032882863 shutdown_connections 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/2032882863 wait complete. 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: build_initial_monmap 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: monmap: 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 Processor -- start 2024-09-15T21:21:04.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- start start 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: init 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792011d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): picked mon.noname-b con 0x7f792009fe70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7920135bf0 0x7f7920134810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): picked mon.noname-c con 0x7f7920135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f792013afb0 0x7f792011c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): picked mon.noname-a con 0x7f792013afb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7920140100 con 0x7f792013afb0 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f792006ed50 con 0x7f792009fe70 2024-09-15T21:21:04.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7920140280 con 0x7f7920135bf0 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient(hunting): authenticate will time out at 1183.626746s 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792011d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792011d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46794/0 (socket says 172.21.15.119:46794) 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 -- 172.21.15.119:0/222054282 learned_addr learned my addr 172.21.15.119:0/222054282 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): get_auth_request con 0x7f792009fe70 auth_method 0 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791ed9d640 1 -- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f792013afb0 msgr2=0x7f792011c6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791ed9d640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f792013afb0 0x7f792011c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:04.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791dd9b640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7920135bf0 0x7f7920134810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient(hunting): handle_auth_done global_id 34666 payload 824 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 -- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7920135bf0 msgr2=0x7f7920134810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7920135bf0 0x7f7920134810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 -- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f792013afb0 msgr2=0x7f792011c6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f792013afb0 0x7f792011c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 monclient: found mon.noname-b 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:04.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 -- 172.21.15.119:0/222054282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792009d240 con 0x7f792009fe70 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791dd9b640 10 monclient: get_auth_request con 0x7f7920135bf0 auth_method 0 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791dd9b640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7920135bf0 0x7f7920134810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791e59c640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792011d9a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f791000c730 tx=0x7f791000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910018070 con 0x7f792009fe70 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: dump: 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.845 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: _finish_auth 0 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79100070d0 con 0x7f792009fe70 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910007500 con 0x7f792009fe70 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:04.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f790effd640 10 monclient: dump: 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 5 monclient: authenticate success, global_id 34666 2024-09-15T21:21:04.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: _renew_subs 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f791cd99640 4 set_mon_vals no callback set 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792013abd0 con 0x7f792009fe70 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: _renew_subs 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.826+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792011eec0 con 0x7f792009fe70 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f7924871640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f7924871640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f7924871640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7920135fd0 con 0x7f792009fe70 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f79100047f0 con 0x7f792009fe70 2024-09-15T21:21:04.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f790effd640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f790003b970 0x7f790003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:04.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5955+0+0 (secure 0 0 0) 0x7f791004f2d0 con 0x7f792009fe70 2024-09-15T21:21:04.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f791dd9b640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f790003b970 0x7f790003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:04.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f791dd9b640 10 monclient: get_auth_request con 0x7f790003b970 auth_method 0 2024-09-15T21:21:04.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.830+0000 7f791dd9b640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f790003b970 0x7f790003de30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f79140ad000 tx=0x7f7914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:04.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.834+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f79200a0300 con 0x7f792009fe70 2024-09-15T21:21:04.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.834+0000 7f790effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:04.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.834+0000 7f790effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:05.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.998+0000 7f7924871640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T21:21:05.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.998+0000 7f7924871640 10 monclient: _send_command 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T21:21:05.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.998+0000 7f7924871640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:04.998+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) -- 0x7f79201ce230 con 0x7f792009fe70 2024-09-15T21:21:05.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.574+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map v95) ==== 212+0+0 (secure 0 0 0) 0x7f791000a640 con 0x7f792009fe70 2024-09-15T21:21:05.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.574+0000 7f790effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T21:21:05.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.574+0000 7f790effd640 10 monclient: _finish_command 2 = system:0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map 2024-09-15T21:21:05.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f7924871640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T21:21:05.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f7924871640 10 monclient: _send_command 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T21:21:05.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f7924871640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) -- 0x7f792006a830 con 0x7f792009fe70 2024-09-15T21:21:05.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f790effd640 1 -- 172.21.15.119:0/222054282 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map v95) ==== 211+0+0 (secure 0 0 0) 0x7f791000ae00 con 0x7f792009fe70 2024-09-15T21:21:05.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f790effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T21:21:05.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.686+0000 7f790effd640 10 monclient: _finish_command 3 = system:0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-09-15T21:21:05.693 INFO:tasks.workunit.client.0.smithi119.stderr:create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f790003b970 msgr2=0x7f790003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f790003b970 0x7f790003de30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f79140ad000 tx=0x7f7914008040 comp rx=0 tx=0).stop 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 10 monclient: shutdown 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 msgr2=0x7f792011d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792011d9a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f791000c730 tx=0x7f791000cbf0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 shutdown_connections 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f790003b970 0x7f790003de30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f792013afb0 0x7f792011c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7920135bf0 0x7f7920134810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 --2- 172.21.15.119:0/222054282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f792009fe70 0x7f792011d9a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 >> 172.21.15.119:0/222054282 conn(0x7f79200a66e0 msgr2=0x7f792013bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:05.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 shutdown_connections 2024-09-15T21:21:05.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.690+0000 7f7924871640 1 -- 172.21.15.119:0/222054282 wait complete. 2024-09-15T21:21:05.710 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush move osd.7 host=foo 2024-09-15T21:21:05.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient: get_monmap_and_config 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient: build_initial_monmap 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient: monmap: 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:05.831383+0000 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:05.831383+0000 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:05.833 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:05.834 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:05.834 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 1 Processor -- start 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 1 -- start start 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient: init 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.826+0000 7f33df997640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f33d809d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f33d809def0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33d813b0a0 0x7f33d813d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient(hunting): picked mon.noname-c con 0x7f33d813b0a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d806e240 con 0x7f33d809d400 2024-09-15T21:21:05.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d809e7f0 con 0x7f33d809def0 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f33d806e000 con 0x7f33d813b0a0 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient(hunting): authenticate will time out at 1184.628127s 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33dd733640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f33d809d400 legacy=0x7f33d809d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33d813b0a0 0x7f33d813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33dcf32640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f33d809def0 0x7f33d8139980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42210/0 (socket says 172.21.15.119:42210) 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33dcf32640 1 -- 172.21.15.119:0/2961732929 learned_addr learned my addr 172.21.15.119:0/2961732929 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): get_auth_request con 0x7f33d813b0a0 auth_method 0 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:05.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient(hunting): handle_auth_done global_id 34669 payload 274 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 1 -- 172.21.15.119:0/2961732929 >> v1:172.21.15.119:6789/0 conn(0x7f33d809def0 legacy=0x7f33d8139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 1 -- 172.21.15.119:0/2961732929 >> v1:172.21.15.17:6789/0 conn(0x7f33d809d400 legacy=0x7f33d809d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient: _finish_hunting 0 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 1 monclient: found mon.noname-c 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 1 -- 172.21.15.119:0/2961732929 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d80a2500 con 0x7f33d813b0a0 2024-09-15T21:21:05.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33ddf34640 1 --2- 172.21.15.119:0/2961732929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d813b0a0 0x7f33d813d490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009dc0 tx=0x7f33d001c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5691c95da030aea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:05.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 1 -- 172.21.15.119:0/2961732929 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00a66e0 con 0x7f33d813b0a0 2024-09-15T21:21:05.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:05.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:05.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: dump: 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:05.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: _finish_auth 0 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 5 monclient: authenticate success, global_id 34669 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 1 -- 172.21.15.119:0/2961732929 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d00a6880 con 0x7f33d813b0a0 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 4 set_mon_vals no callback set 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 1 -- 172.21.15.119:0/2961732929 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00a6c50 con 0x7f33d813b0a0 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:05.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33cffff640 10 monclient: dump: 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 10 monclient: shutdown 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- 172.21.15.119:0/2961732929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d813b0a0 msgr2=0x7f33d813d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:05.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 --2- 172.21.15.119:0/2961732929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d813b0a0 0x7f33d813d490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009dc0 tx=0x7f33d001c810 comp rx=0 tx=0).stop 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- 172.21.15.119:0/2961732929 shutdown_connections 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 --2- 172.21.15.119:0/2961732929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d813b0a0 0x7f33d813d490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- 172.21.15.119:0/2961732929 >> 172.21.15.119:0/2961732929 conn(0x7f33d809be60 msgr2=0x7f33d809c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- 172.21.15.119:0/2961732929 shutdown_connections 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.830+0000 7f33df997640 1 -- 172.21.15.119:0/2961732929 wait complete. 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: build_initial_monmap 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: monmap: 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:05.847 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 Processor -- start 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- start start 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: init 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33d809d400 0x7f33d811f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): picked mon.noname-c con 0x7f33d809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:05.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 0x7f33d8121470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): picked mon.noname-b con 0x7f33d809def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33d813b0a0 0x7f33d813a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): picked mon.noname-a con 0x7f33d813b0a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80a1310 con 0x7f33d813b0a0 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80a1190 con 0x7f33d809def0 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f33d80a1490 con 0x7f33d809d400 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient(hunting): authenticate will time out at 1184.632935s 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 0x7f33d8121470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:05.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33ddf34640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33d813b0a0 msgr2=0x7f33d813a300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 0x7f33d8121470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46816/0 (socket says 172.21.15.119:46816) 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33ddf34640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33d813b0a0 0x7f33d813a300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 -- 172.21.15.119:0/1682440093 learned_addr learned my addr 172.21.15.119:0/1682440093 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): get_auth_request con 0x7f33d809def0 auth_method 0 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dd733640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33d809d400 0x7f33d811f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:05.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient(hunting): handle_auth_done global_id 34675 payload 824 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 -- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33d809d400 msgr2=0x7f33d811f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33d809d400 0x7f33d811f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 -- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33d813b0a0 msgr2=0x7f33d813a300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33d813b0a0 0x7f33d813a300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient: _finish_hunting 0 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 monclient: found mon.noname-b 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 -- 172.21.15.119:0/1682440093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d8121e00 con 0x7f33d809def0 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dcf32640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 0x7f33d8121470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c4e0 tx=0x7f33c800c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8017070 con 0x7f33d809def0 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:05.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: dump: 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:05.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c8005d10 con 0x7f33d809def0 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c80072b0 con 0x7f33d809def0 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 10 monclient: dump: 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.853 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 5 monclient: authenticate success, global_id 34675 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _renew_subs 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cf7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d809d0f0 con 0x7f33d809def0 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _renew_subs 2024-09-15T21:21:05.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d8122c70 con 0x7f33d809def0 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d809d7e0 con 0x7f33d809def0 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f33c801a070 con 0x7f33d809def0 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f33bc03b970 0x7f33bc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:05.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5976+0+0 (secure 0 0 0) 0x7f33c804f200 con 0x7f33d809def0 2024-09-15T21:21:05.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dd733640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f33bc03b970 0x7f33bc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:05.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dd733640 10 monclient: get_auth_request con 0x7f33bc03b970 auth_method 0 2024-09-15T21:21:05.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.834+0000 7f33dd733640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f33bc03b970 0x7f33bc03de30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f33d4003940 tx=0x7f33d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:05.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.838+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f33c8010300 con 0x7f33d809def0 2024-09-15T21:21:05.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.838+0000 7f33cdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:05.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:05.838+0000 7f33cdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:06.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.010+0000 7f33df997640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T21:21:06.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.010+0000 7f33df997640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T21:21:06.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.010+0000 7f33df997640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.010+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) -- 0x7f33d8122430 con 0x7f33d809def0 2024-09-15T21:21:06.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.574+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 moved item id 7 name 'osd.7' to location {host=foo} in crush map v96) ==== 165+0+0 (secure 0 0 0) 0x7f33c8010e20 con 0x7f33d809def0 2024-09-15T21:21:06.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.574+0000 7f33cdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T21:21:06.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.574+0000 7f33cdffb640 10 monclient: _finish_command 2 = system:0 moved item id 7 name 'osd.7' to location {host=foo} in crush map 2024-09-15T21:21:06.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T21:21:06.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T21:21:06.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) -- 0x7f33d813a840 con 0x7f33d809def0 2024-09-15T21:21:06.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33cdffb640 1 -- 172.21.15.119:0/1682440093 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map v96) ==== 175+0+0 (secure 0 0 0) 0x7f33c8011d80 con 0x7f33d809def0 2024-09-15T21:21:06.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33cdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T21:21:06.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33cdffb640 10 monclient: _finish_command 3 = system:0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-09-15T21:21:06.696 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-09-15T21:21:06.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f33bc03b970 msgr2=0x7f33bc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:06.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f33bc03b970 0x7f33bc03de30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f33d4003940 tx=0x7f33d4008040 comp rx=0 tx=0).stop 2024-09-15T21:21:06.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 10 monclient: shutdown 2024-09-15T21:21:06.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 msgr2=0x7f33d8121470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.690+0000 7f33df997640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 0x7f33d8121470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c4e0 tx=0x7f33c800c9a0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 shutdown_connections 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f33bc03b970 0x7f33bc03de30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33d813b0a0 0x7f33d813a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d809def0 0x7f33d8121470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 --2- 172.21.15.119:0/1682440093 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33d809d400 0x7f33d811f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 >> 172.21.15.119:0/1682440093 conn(0x7f33d809be60 msgr2=0x7f33d813c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 shutdown_connections 2024-09-15T21:21:06.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.694+0000 7f33df997640 1 -- 172.21.15.119:0/1682440093 wait complete. 2024-09-15T21:21:06.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep host 2024-09-15T21:21:06.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd find osd.7 2024-09-15T21:21:06.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep foo 2024-09-15T21:21:06.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient: get_monmap_and_config 2024-09-15T21:21:06.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient: build_initial_monmap 2024-09-15T21:21:06.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient: monmap: 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:06.822261+0000 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:06.822261+0000 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:06.824 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:06.825 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:06.825 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 1 Processor -- start 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 1 -- start start 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient: init 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbee409d4e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbee409f960 0x7fbee4139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient(hunting): picked mon.noname-c con 0x7fbee409f960 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:06.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbee409dfd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee409ef80 con 0x7fbee409dfd0 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee40746f0 con 0x7fbee409d4e0 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbee406da70 con 0x7fbee409f960 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbeeabcf640 10 monclient(hunting): authenticate will time out at 1185.619043s 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee896b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbee409d4e0 0x7fbee409d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42216/0 (socket says 172.21.15.119:42216) 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee896b640 1 -- 172.21.15.119:0/3789824281 learned_addr learned my addr 172.21.15.119:0/3789824281 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee916c640 1 -- 172.21.15.119:0/3789824281 >> v1:172.21.15.17:6789/0 conn(0x7fbee409dfd0 legacy=0x7fbee413c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 1 --2- 172.21.15.119:0/3789824281 >> v2:172.21.15.119:3300/0 conn(0x7fbee409f960 0x7fbee4139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:06.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): get_auth_request con 0x7fbee409f960 auth_method 0 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee37fe640 1 -- 172.21.15.119:0/3789824281 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546806080 0 0) 0x7fbee40746f0 con 0x7fbee409d4e0 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:06.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee37fe640 10 monclient(hunting): my global_id is 34681 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee37fe640 1 -- 172.21.15.119:0/3789824281 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbecc003530 con 0x7fbee409d4e0 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient(hunting): handle_auth_done global_id 34678 payload 274 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 1 -- 172.21.15.119:0/3789824281 >> v1:172.21.15.119:6789/0 conn(0x7fbee409d4e0 legacy=0x7fbee409d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 1 -- 172.21.15.119:0/3789824281 >> v1:172.21.15.17:6789/0 conn(0x7fbee409dfd0 legacy=0x7fbee413c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 1 monclient: found mon.noname-c 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.818+0000 7fbee3fff640 1 -- 172.21.15.119:0/3789824281 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee409e6e0 con 0x7fbee409f960 2024-09-15T21:21:06.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee3fff640 1 --2- 172.21.15.119:0/3789824281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee409f960 0x7fbee4139c00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbed80097c0 tx=0x7fbed801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72eb9b294882d49f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:06.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 1 -- 172.21.15.119:0/3789824281 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80b0070 con 0x7fbee409f960 2024-09-15T21:21:06.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:06.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:06.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: dump: 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:06.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 1 -- 172.21.15.119:0/3789824281 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8004230 con 0x7fbee409f960 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 5 monclient: authenticate success, global_id 34678 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 1 -- 172.21.15.119:0/3789824281 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003620 con 0x7fbee409f960 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbee37fe640 10 monclient: dump: 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:06.838 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 4 set_mon_vals no callback set 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 10 monclient: shutdown 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 -- 172.21.15.119:0/3789824281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee409f960 msgr2=0x7fbee4139c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/3789824281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee409f960 0x7fbee4139c00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbed80097c0 tx=0x7fbed801c330 comp rx=0 tx=0).stop 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 -- 172.21.15.119:0/3789824281 shutdown_connections 2024-09-15T21:21:06.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/3789824281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee409f960 0x7fbee4139c00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 -- 172.21.15.119:0/3789824281 >> 172.21.15.119:0/3789824281 conn(0x7fbee406cdc0 msgr2=0x7fbee406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 -- 172.21.15.119:0/3789824281 shutdown_connections 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 -- 172.21.15.119:0/3789824281 wait complete. 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 10 monclient: build_initial_monmap 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 10 monclient: monmap: 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:06.840 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 Processor -- start 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 1 -- start start 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.822+0000 7fbeeabcf640 10 monclient: init 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbee409d4e0 0x7fbee41cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): picked mon.noname-c con 0x7fbee409d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbee409dfd0 0x7fbee4133800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): picked mon.noname-a con 0x7fbee409dfd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 0x7fbee411c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:06.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): picked mon.noname-b con 0x7fbee4133d40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41404a0 con 0x7fbee409dfd0 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbee40a2750 con 0x7fbee4133d40 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbee4140620 con 0x7fbee409d4e0 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient(hunting): authenticate will time out at 1185.624167s 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 0x7fbee411c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 0x7fbee411c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46832/0 (socket says 172.21.15.119:46832) 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 -- 172.21.15.119:0/803015461 learned_addr learned my addr 172.21.15.119:0/803015461 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): get_auth_request con 0x7fbee4133d40 auth_method 0 2024-09-15T21:21:06.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee3fff640 1 -- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbee409dfd0 msgr2=0x7fbee4133800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee3fff640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbee409dfd0 0x7fbee4133800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient(hunting): handle_auth_done global_id 34684 payload 824 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee896b640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbee409d4e0 0x7fbee41cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 -- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbee409d4e0 msgr2=0x7fbee41cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbee409d4e0 0x7fbee41cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 -- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbee409dfd0 msgr2=0x7fbee4133800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:06.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbee409dfd0 0x7fbee4133800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 monclient: found mon.noname-b 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 -- 172.21.15.119:0/803015461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee411ecc0 con 0x7fbee4133d40 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee916c640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 0x7fbee411c630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00b9e0 tx=0x7fbedc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00caa0 con 0x7fbee4133d40 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: dump: 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:06.844 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 5 monclient: authenticate success, global_id 34684 2024-09-15T21:21:06.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc010070 con 0x7fbee4133d40 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc015470 con 0x7fbee4133d40 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _renew_subs 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee40a5200 con 0x7fbee4133d40 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee27fc640 4 set_mon_vals no callback set 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 10 monclient: dump: 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:06.846 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _renew_subs 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee409d1d0 con 0x7fbee4133d40 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee409d8c0 con 0x7fbee4133d40 2024-09-15T21:21:06.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbedc01e3f0 con 0x7fbee4133d40 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbed003b9a0 0x7fbed003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6150+0+0 (secure 0 0 0) 0x7fbedc0500a0 con 0x7fbee4133d40 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee896b640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbed003b9a0 0x7fbed003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee896b640 10 monclient: get_auth_request con 0x7fbed003b9a0 auth_method 0 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.826+0000 7fbee896b640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbed003b9a0 0x7fbed003de60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbed4007fd0 tx=0x7fbed4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.830+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbee40a5200 con 0x7fbee4133d40 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.830+0000 7fbee17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:06.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.830+0000 7fbee17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:06.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbeeabcf640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-09-15T21:21:06.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbeeabcf640 10 monclient: _send_command 2 [{"prefix": "osd find", "id": 7}] 2024-09-15T21:21:06.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbeeabcf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:06.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd find", "id": 7} v 0) -- 0x7fbee411e9b0 con 0x7fbee4133d40 2024-09-15T21:21:06.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v96) ==== 65+0+226 (secure 0 0 0) 0x7fbee411ecc0 con 0x7fbee4133d40 2024-09-15T21:21:06.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbee17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd find", "id": 7}] 2024-09-15T21:21:06.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:06.990+0000 7fbee17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:07.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-09-15T21:21:07.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 10 monclient: _send_command 3 [{"prefix": "osd find", "id": 7}] 2024-09-15T21:21:07.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd find", "id": 7} v 0) -- 0x7fbee411f910 con 0x7fbee4133d40 2024-09-15T21:21:07.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbee17fa640 1 -- 172.21.15.119:0/803015461 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v96) ==== 65+0+226 (secure 0 0 0) 0x7fbedc04e600 con 0x7fbee4133d40 2024-09-15T21:21:07.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbee17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd find", "id": 7}] 2024-09-15T21:21:07.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbee17fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:07.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbed003b9a0 msgr2=0x7fbed003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbed003b9a0 0x7fbed003de60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbed4007fd0 tx=0x7fbed4009040 comp rx=0 tx=0).stop 2024-09-15T21:21:07.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 10 monclient: shutdown 2024-09-15T21:21:07.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:07.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 msgr2=0x7fbee411c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 0x7fbee411c630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00b9e0 tx=0x7fbedc00bea0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 shutdown_connections 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbed003b9a0 0x7fbed003de60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee4133d40 0x7fbee411c630 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbee409dfd0 0x7fbee4133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 --2- 172.21.15.119:0/803015461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbee409d4e0 0x7fbee41cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 >> 172.21.15.119:0/803015461 conn(0x7fbee406cdc0 msgr2=0x7fbee40a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 shutdown_connections 2024-09-15T21:21:07.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.086+0000 7fbeeabcf640 1 -- 172.21.15.119:0/803015461 wait complete. 2024-09-15T21:21:07.107 INFO:tasks.workunit.client.0.smithi119.stdout: "host": "foo", 2024-09-15T21:21:07.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.7 2024-09-15T21:21:07.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient: get_monmap_and_config 2024-09-15T21:21:07.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient: build_initial_monmap 2024-09-15T21:21:07.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient: monmap: 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:07.222706+0000 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:07.222706+0000 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 Processor -- start 2024-09-15T21:21:07.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 -- start start 2024-09-15T21:21:07.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient: init 2024-09-15T21:21:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-09-15T21:21:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f274c135ca0 0x7f274c136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): picked mon.noname-a con 0x7f274c135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 0x7f274c0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): picked mon.noname-b con 0x7f274c0a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f274c06ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c1365c0 con 0x7f274c06ec30 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f274c09cf30 con 0x7f274c135ca0 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0a1ca0 con 0x7f274c0a0160 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f2751c5f640 10 monclient(hunting): authenticate will time out at 1186.019267s 2024-09-15T21:21:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f274c06ec30 0x7f274c0a0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42220/0 (socket says 172.21.15.119:42220) 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274bfff640 1 -- 172.21.15.119:0/804751358 learned_addr learned my addr 172.21.15.119:0/804751358 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274b7fe640 1 -- 172.21.15.119:0/804751358 >> v2:172.21.15.17:3300/0 conn(0x7f274c135ca0 msgr2=0x7f274c136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274b7fe640 1 --2- 172.21.15.119:0/804751358 >> v2:172.21.15.17:3300/0 conn(0x7f274c135ca0 0x7f274c136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 0x7f274c0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): get_auth_request con 0x7f274c0a0160 auth_method 0 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2032401481 0 0) 0x7f274c1365c0 con 0x7f274c06ec30 2024-09-15T21:21:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 10 monclient(hunting): my global_id is 34687 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2734003530 con 0x7f274c06ec30 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1447774012 0 0) 0x7f2734003530 con 0x7f274c06ec30 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 msgr2=0x7f274c0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 0x7f274c0a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 >> v2:172.21.15.17:3300/0 conn(0x7f274c135ca0 msgr2=0x7f274c136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 --2- 172.21.15.119:0/804751358 >> v2:172.21.15.17:3300/0 conn(0x7f274c135ca0 0x7f274c136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.218+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0a1f80 con 0x7f274c06ec30 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f2744001700 con 0x7f274c06ec30 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 5 monclient: authenticate success, global_id 34687 2024-09-15T21:21:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: dump: 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 >> v1:172.21.15.119:6789/0 conn(0x7f274c06ec30 legacy=0x7f274c0a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2734003a10 0x7f2734003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): picked mon.a con 0x7f2734003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 0x7f2734004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): picked mon.c con 0x7f274c0a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2734004d50 0x7f27340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): picked mon.b con 0x7f2734004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f274c09cf30 con 0x7f2734003a10 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0a1ca0 con 0x7f2734004d50 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f273400bc80 con 0x7f274c0a0160 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274b7fe640 1 -- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2734003a10 msgr2=0x7f2734003e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274b7fe640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2734003a10 0x7f2734003e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2734004d50 0x7f27340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): get_auth_request con 0x7f2734004d50 auth_method 0 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient(hunting): handle_auth_done global_id 34687 payload 293 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274affd640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 0x7f2734004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 -- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 msgr2=0x7f2734004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c0a0160 0x7f2734004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 -- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2734003a10 msgr2=0x7f2734003e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2734003a10 0x7f2734003e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 monclient: found mon.b 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 -- 172.21.15.119:0/804751358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2734003530 con 0x7f2734004d50 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient: _finish_auth 0 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2734004d50 0x7f27340051d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f274400c2f0 tx=0x7f274401cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fd5e78a295a299 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744007da0 con 0x7f2734004d50 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: dump: 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27440a9020 con 0x7f2734004d50 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 4 set_mon_vals no callback set 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 1 -- 172.21.15.119:0/804751358 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744005b20 con 0x7f2734004d50 2024-09-15T21:21:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274a7fc640 10 monclient: dump: 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.246 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 10 monclient: shutdown 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 1 -- 172.21.15.119:0/804751358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2734004d50 msgr2=0x7f27340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 1 --2- 172.21.15.119:0/804751358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2734004d50 0x7f27340051d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f274400c2f0 tx=0x7f274401cb80 comp rx=0 tx=0).stop 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f274bfff640 1 -- 172.21.15.119:0/804751358 reap_dead start 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 1 -- 172.21.15.119:0/804751358 shutdown_connections 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 1 -- 172.21.15.119:0/804751358 >> 172.21.15.119:0/804751358 conn(0x7f274c0a2ae0 msgr2=0x7f274c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 1 -- 172.21.15.119:0/804751358 shutdown_connections 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.222+0000 7f2751c5f640 1 -- 172.21.15.119:0/804751358 wait complete. 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: build_initial_monmap 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: monmap: 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 Processor -- start 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- start start 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: init 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 0x7f274c09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): picked mon.noname-b con 0x7f274c09d740 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c09e060 0x7f274c09e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): picked mon.noname-c con 0x7f274c09e060 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1d4450 0x7f274c1d6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): picked mon.noname-a con 0x7f274c1d4450 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f274c13e230 con 0x7f274c1d4450 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0a1840 con 0x7f274c09d740 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f274c13e3b0 con 0x7f274c09e060 2024-09-15T21:21:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient(hunting): authenticate will time out at 1186.025032s 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 0x7f274c09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 0x7f274c09db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46858/0 (socket says 172.21.15.119:46858) 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 -- 172.21.15.119:0/2943027383 learned_addr learned my addr 172.21.15.119:0/2943027383 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): get_auth_request con 0x7f274c09d740 auth_method 0 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274bfff640 1 -- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1d4450 msgr2=0x7f274c1d6840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274bfff640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1d4450 0x7f274c1d6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:07.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274affd640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c09e060 0x7f274c09e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient(hunting): handle_auth_done global_id 34690 payload 824 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 -- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c09e060 msgr2=0x7f274c09e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c09e060 0x7f274c09e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 -- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1d4450 msgr2=0x7f274c1d6840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1d4450 0x7f274c1d6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 -- 172.21.15.119:0/2943027383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c09ef20 con 0x7f274c09d740 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274affd640 10 monclient: get_auth_request con 0x7f274c09e060 auth_method 0 2024-09-15T21:21:07.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274affd640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c09e060 0x7f274c09e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274b7fe640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 0x7f274c09db20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f273c00b9e0 tx=0x7f273c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c00cba0 con 0x7f274c09d740 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: dump: 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.252 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c010070 con 0x7f274c09d740 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 5 monclient: authenticate success, global_id 34690 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _renew_subs 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c09d1d0 con 0x7f274c09d740 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c015470 con 0x7f274c09d740 2024-09-15T21:21:07.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 10 monclient: dump: 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.254 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _renew_subs 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2749ffb640 4 set_mon_vals no callback set 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c121200 con 0x7f274c09d740 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f273c01e3f0 con 0x7f274c09d740 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c136080 con 0x7f274c09d740 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f273803b9a0 0x7f273803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6150+0+0 (secure 0 0 0) 0x7f273c050350 con 0x7f274c09d740 2024-09-15T21:21:07.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274affd640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f273803b9a0 0x7f273803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274affd640 10 monclient: get_auth_request con 0x7f273803b9a0 auth_method 0 2024-09-15T21:21:07.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.226+0000 7f274affd640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f273803b9a0 0x7f273803de60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f274c121160 tx=0x7f2740008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:07.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.230+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f273c0139d0 con 0x7f274c09d740 2024-09-15T21:21:07.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.230+0000 7f2733fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:07.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.230+0000 7f2733fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:07.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.406+0000 7f2751c5f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T21:21:07.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.406+0000 7f2751c5f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T21:21:07.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.406+0000 7f2751c5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.406+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) -- 0x7f274c09f8f0 con 0x7f274c09d740 2024-09-15T21:21:07.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.518+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 removed item id 7 name 'osd.7' from crush map v97) ==== 122+0+0 (secure 0 0 0) 0x7f273c04e7b0 con 0x7f274c09d740 2024-09-15T21:21:07.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.518+0000 7f2733fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T21:21:07.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.518+0000 7f2733fff640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-09-15T21:21:07.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T21:21:07.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T21:21:07.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) -- 0x7f274c0008e0 con 0x7f274c09d740 2024-09-15T21:21:07.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2733fff640 1 -- 172.21.15.119:0/2943027383 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 device 'osd.7' does not appear in the crush map v97) ==== 124+0+0 (secure 0 0 0) 0x7f273c007dc0 con 0x7f274c09d740 2024-09-15T21:21:07.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2733fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T21:21:07.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2733fff640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-09-15T21:21:07.632 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.7' does not appear in the crush map 2024-09-15T21:21:07.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f273803b9a0 msgr2=0x7f273803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f273803b9a0 0x7f273803de60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f274c121160 tx=0x7f2740008040 comp rx=0 tx=0).stop 2024-09-15T21:21:07.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 10 monclient: shutdown 2024-09-15T21:21:07.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 msgr2=0x7f274c09db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.626+0000 7f2751c5f640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 0x7f274c09db20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f273c00b9e0 tx=0x7f273c00bea0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 shutdown_connections 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f273803b9a0 0x7f273803de60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1d4450 0x7f274c1d6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f274c09e060 0x7f274c09e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 --2- 172.21.15.119:0/2943027383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c09d740 0x7f274c09db20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 >> 172.21.15.119:0/2943027383 conn(0x7f274c0a2ae0 msgr2=0x7f274c1349b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 shutdown_connections 2024-09-15T21:21:07.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.630+0000 7f2751c5f640 1 -- 172.21.15.119:0/2943027383 wait complete. 2024-09-15T21:21:07.648 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.9 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient: get_monmap_and_config 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient: build_initial_monmap 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient: monmap: 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:07.757783+0000 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:07.757783+0000 2024-09-15T21:21:07.759 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:07.760 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 Processor -- start 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 -- start start 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient: init 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faca8135ca0 0x7faca8136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): picked mon.noname-a con 0x7faca8135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faca80a0160 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faca806ec30 0x7faca80a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): picked mon.noname-c con 0x7faca806ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca81365c0 con 0x7faca80a0160 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faca809cf30 con 0x7faca8135ca0 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faca80a1ca0 con 0x7faca806ec30 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facaea5f640 10 monclient(hunting): authenticate will time out at 1186.554541s 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca77fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faca80a0160 0x7faca80a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42226/0 (socket says 172.21.15.119:42226) 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca77fe640 1 -- 172.21.15.119:0/1057532421 learned_addr learned my addr 172.21.15.119:0/1057532421 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca7fff640 1 -- 172.21.15.119:0/1057532421 >> v2:172.21.15.17:3300/0 conn(0x7faca8135ca0 msgr2=0x7faca8136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca7fff640 1 --2- 172.21.15.119:0/1057532421 >> v2:172.21.15.17:3300/0 conn(0x7faca8135ca0 0x7faca8136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:07.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 1 --2- 172.21.15.119:0/1057532421 >> v2:172.21.15.119:3300/0 conn(0x7faca806ec30 0x7faca80a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): get_auth_request con 0x7faca806ec30 auth_method 0 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca6ffd640 1 -- 172.21.15.119:0/1057532421 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324991964 0 0) 0x7faca81365c0 con 0x7faca80a0160 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca6ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca6ffd640 10 monclient(hunting): my global_id is 34696 2024-09-15T21:21:07.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7faca6ffd640 1 -- 172.21.15.119:0/1057532421 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac90003530 con 0x7faca80a0160 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient(hunting): handle_auth_done global_id 34693 payload 274 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 1 -- 172.21.15.119:0/1057532421 >> v2:172.21.15.17:3300/0 conn(0x7faca8135ca0 msgr2=0x7faca8136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 1 --2- 172.21.15.119:0/1057532421 >> v2:172.21.15.17:3300/0 conn(0x7faca8135ca0 0x7faca8136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 1 -- 172.21.15.119:0/1057532421 >> v1:172.21.15.119:6789/0 conn(0x7faca80a0160 legacy=0x7faca80a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 1 monclient: found mon.noname-c 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.754+0000 7facacffc640 1 -- 172.21.15.119:0/1057532421 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca80a1f80 con 0x7faca806ec30 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facacffc640 1 --2- 172.21.15.119:0/1057532421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca806ec30 0x7faca80a10f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faca80a29c0 tx=0x7faca001c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff97fe0ba8f36bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 1 -- 172.21.15.119:0/1057532421 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca00a66e0 con 0x7faca806ec30 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: dump: 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:07.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 1 -- 172.21.15.119:0/1057532421 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca00a6880 con 0x7faca806ec30 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 1 -- 172.21.15.119:0/1057532421 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca00a6c50 con 0x7faca806ec30 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 5 monclient: authenticate success, global_id 34693 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 4 set_mon_vals no callback set 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca6ffd640 10 monclient: dump: 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.773 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: shutdown 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- 172.21.15.119:0/1057532421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca806ec30 msgr2=0x7faca80a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 --2- 172.21.15.119:0/1057532421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca806ec30 0x7faca80a10f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faca80a29c0 tx=0x7faca001c810 comp rx=0 tx=0).stop 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- 172.21.15.119:0/1057532421 shutdown_connections 2024-09-15T21:21:07.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 --2- 172.21.15.119:0/1057532421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca806ec30 0x7faca80a10f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 --2- 172.21.15.119:0/1057532421 >> v2:172.21.15.17:3300/0 conn(0x7faca8135ca0 0x7faca8136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- 172.21.15.119:0/1057532421 >> 172.21.15.119:0/1057532421 conn(0x7faca80a2ae0 msgr2=0x7faca80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- 172.21.15.119:0/1057532421 shutdown_connections 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- 172.21.15.119:0/1057532421 wait complete. 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: build_initial_monmap 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: monmap: 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:07.775 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 Processor -- start 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- start start 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: init 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca806ec30 0x7faca81384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): picked mon.noname-a con 0x7faca806ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 0x7faca8138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): picked mon.noname-b con 0x7faca80a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faca8135ca0 0x7faca8138f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): picked mon.noname-c con 0x7faca8135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faca8140240 con 0x7faca806ec30 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faca80a1840 con 0x7faca80a0160 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faca81403c0 con 0x7faca8135ca0 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7facaea5f640 10 monclient(hunting): authenticate will time out at 1186.559453s 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 0x7faca8138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca7fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca806ec30 msgr2=0x7faca81384e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca806ec30 0x7faca81384e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 0x7faca8138a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46880/0 (socket says 172.21.15.119:46880) 2024-09-15T21:21:07.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 -- 172.21.15.119:0/2507759657 learned_addr learned my addr 172.21.15.119:0/2507759657 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): get_auth_request con 0x7faca80a0160 auth_method 0 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient(hunting): handle_auth_done global_id 34699 payload 824 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 -- 172.21.15.119:0/2507759657 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faca8135ca0 msgr2=0x7faca8138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faca8135ca0 0x7faca8138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 -- 172.21.15.119:0/2507759657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca806ec30 msgr2=0x7faca81384e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca806ec30 0x7faca81384e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:07.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 -- 172.21.15.119:0/2507759657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca813c300 con 0x7faca80a0160 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.758+0000 7faca77fe640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 0x7faca8138a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac98001ec0 tx=0x7fac9800c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98017070 con 0x7faca80a0160 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: dump: 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.779 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98005cf0 con 0x7faca80a0160 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 5 monclient: authenticate success, global_id 34699 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 monclient: _renew_subs 2024-09-15T21:21:07.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca80a56d0 con 0x7faca80a0160 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98007290 con 0x7faca80a0160 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 10 monclient: dump: 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:07.781 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 monclient: _renew_subs 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca67fc640 4 set_mon_vals no callback set 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca813c020 con 0x7faca80a0160 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca8136080 con 0x7faca80a0160 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fac9801a070 con 0x7faca80a0160 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac9403b9a0 0x7fac9403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:07.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6100+0+0 (secure 0 0 0) 0x7fac9804f2d0 con 0x7faca80a0160 2024-09-15T21:21:07.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca7fff640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac9403b9a0 0x7fac9403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:07.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca7fff640 10 monclient: get_auth_request con 0x7fac9403b9a0 auth_method 0 2024-09-15T21:21:07.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.762+0000 7faca7fff640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac9403b9a0 0x7fac9403de60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac9c003940 tx=0x7fac9c006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:07.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.766+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fac9804d860 con 0x7faca80a0160 2024-09-15T21:21:07.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.766+0000 7faca4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:07.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.766+0000 7faca4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:07.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7facaea5f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T21:21:07.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7facaea5f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T21:21:07.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7facaea5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:07.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) -- 0x7faca809f8f0 con 0x7faca80a0160 2024-09-15T21:21:07.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v97) ==== 124+0+0 (secure 0 0 0) 0x7fac9801ad80 con 0x7faca80a0160 2024-09-15T21:21:07.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7faca4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T21:21:07.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:07.926+0000 7faca4ff9640 10 monclient: _finish_command 2 = system:0 device 'osd.9' does not appear in the crush map 2024-09-15T21:21:08.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7facaea5f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T21:21:08.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7facaea5f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T21:21:08.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7facaea5f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) -- 0x7faca813ab00 con 0x7faca80a0160 2024-09-15T21:21:08.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7faca4ff9640 1 -- 172.21.15.119:0/2507759657 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v97) ==== 124+0+0 (secure 0 0 0) 0x7fac98007610 con 0x7faca80a0160 2024-09-15T21:21:08.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7faca4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T21:21:08.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.022+0000 7faca4ff9640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-09-15T21:21:08.030 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.9' does not appear in the crush map 2024-09-15T21:21:08.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac9403b9a0 msgr2=0x7fac9403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac9403b9a0 0x7fac9403de60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac9c003940 tx=0x7fac9c006d90 comp rx=0 tx=0).stop 2024-09-15T21:21:08.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 10 monclient: shutdown 2024-09-15T21:21:08.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:08.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 msgr2=0x7faca8138a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 0x7faca8138a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac98001ec0 tx=0x7fac9800c960 comp rx=0 tx=0).stop 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 shutdown_connections 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac9403b9a0 0x7fac9403de60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faca8135ca0 0x7faca8138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca80a0160 0x7faca8138a20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 --2- 172.21.15.119:0/2507759657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca806ec30 0x7faca81384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 >> 172.21.15.119:0/2507759657 conn(0x7faca80a2ae0 msgr2=0x7faca80a4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 shutdown_connections 2024-09-15T21:21:08.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.026+0000 7facaea5f640 1 -- 172.21.15.119:0/2507759657 wait complete. 2024-09-15T21:21:08.045 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm foo 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient: get_monmap_and_config 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient: build_initial_monmap 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient: monmap: 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:08.158605+0000 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:08.158605+0000 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:08.160 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:08.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:08.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:08.161 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:08.161 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:08.161 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 1 Processor -- start 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 1 -- start start 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient: init 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f85a80a4e60 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85a80a5950 0x7f85a813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.154+0000 7f85adf7f640 10 monclient(hunting): picked mon.noname-c con 0x7f85a80a5950 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a813d480 0x7f85a813f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient(hunting): picked mon.noname-b con 0x7f85a813d480 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a80a1150 con 0x7f85a80a4e60 2024-09-15T21:21:08.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f85a80a3c80 con 0x7f85a80a5950 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f85a80a0ef0 con 0x7f85a813d480 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient(hunting): authenticate will time out at 1186.956078s 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85a80a5950 0x7f85a813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a77fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f85a80a4e60 0x7f85a80a5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42238/0 (socket says 172.21.15.119:42238) 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a77fe640 1 -- 172.21.15.119:0/3530764027 learned_addr learned my addr 172.21.15.119:0/3530764027 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): get_auth_request con 0x7f85a80a5950 auth_method 0 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:08.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a7fff640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a813d480 0x7f85a813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a7fff640 10 monclient(hunting): get_auth_request con 0x7f85a813d480 auth_method 0 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_done global_id 34702 payload 274 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 -- 172.21.15.119:0/3530764027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a813d480 msgr2=0x7f85a813f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a813d480 0x7f85a813f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 -- 172.21.15.119:0/3530764027 >> v1:172.21.15.119:6789/0 conn(0x7f85a80a4e60 legacy=0x7f85a80a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 1 -- 172.21.15.119:0/3530764027 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424662262 0 0) 0x7f85a80a1150 con 0x7f85a80a4e60 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 monclient: found mon.noname-c 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 -- 172.21.15.119:0/3530764027 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a80a6cf0 con 0x7f85a80a5950 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a7fff640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a813d480 0x7f85a813f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:21:08.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a6ffd640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a5950 0x7f85a813cf40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097c0 tx=0x7f859c01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840b0cffd2dd63e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 1 -- 172.21.15.119:0/3530764027 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0b0070 con 0x7f85a80a5950 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: dump: 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.172 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 1 -- 172.21.15.119:0/3530764027 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c0ab600 con 0x7f85a80a5950 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 5 monclient: authenticate success, global_id 34702 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 1 -- 172.21.15.119:0/3530764027 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0ab9d0 con 0x7f85a80a5950 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 4 set_mon_vals no callback set 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85a67fc640 10 monclient: dump: 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 10 monclient: shutdown 2024-09-15T21:21:08.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- 172.21.15.119:0/3530764027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a5950 msgr2=0x7f85a813cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a5950 0x7f85a813cf40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097c0 tx=0x7f859c01c360 comp rx=0 tx=0).stop 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- 172.21.15.119:0/3530764027 shutdown_connections 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a813d480 0x7f85a813f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 --2- 172.21.15.119:0/3530764027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a5950 0x7f85a813cf40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- 172.21.15.119:0/3530764027 >> 172.21.15.119:0/3530764027 conn(0x7f85a80a6f60 msgr2=0x7f85a80a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- 172.21.15.119:0/3530764027 shutdown_connections 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.158+0000 7f85adf7f640 1 -- 172.21.15.119:0/3530764027 wait complete. 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: build_initial_monmap 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: monmap: 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:08.175 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 Processor -- start 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- start start 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: init 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:08.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 0x7f85a8137550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): picked mon.noname-b con 0x7f85a80a4e60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a80a5950 0x7f85a8137a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): picked mon.noname-c con 0x7f85a80a5950 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85a813d480 0x7f85a8071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): picked mon.noname-a con 0x7f85a813d480 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f85a80a2310 con 0x7f85a813d480 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f85a80a2190 con 0x7f85a80a4e60 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f85a81425d0 con 0x7f85a80a5950 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient(hunting): authenticate will time out at 1186.961130s 2024-09-15T21:21:08.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 0x7f85a8137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a7fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85a813d480 msgr2=0x7f85a8071f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85a813d480 0x7f85a8071f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 0x7f85a8137550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46902/0 (socket says 172.21.15.119:46902) 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 -- 172.21.15.119:0/1990139122 learned_addr learned my addr 172.21.15.119:0/1990139122 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): get_auth_request con 0x7f85a80a4e60 auth_method 0 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a80a5950 0x7f85a8137a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:08.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): get_auth_request con 0x7f85a80a5950 auth_method 0 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient(hunting): handle_auth_done global_id 34708 payload 824 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 -- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a80a5950 msgr2=0x7f85a8137a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a80a5950 0x7f85a8137a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 -- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85a813d480 msgr2=0x7f85a8071f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85a813d480 0x7f85a8071f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 -- 172.21.15.119:0/1990139122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a80a2eb0 con 0x7f85a80a4e60 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a77fe640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 0x7f85a8137550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8598001ec0 tx=0x7f859800c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598017070 con 0x7f85a80a4e60 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: dump: 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.180 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: _finish_auth 0 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8598005d70 con 0x7f85a80a4e60 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 5 monclient: authenticate success, global_id 34708 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _renew_subs 2024-09-15T21:21:08.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a8121c10 con 0x7f85a80a4e60 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980072b0 con 0x7f85a80a4e60 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 10 monclient: dump: 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.182 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _renew_subs 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81224b0 con 0x7f85a80a4e60 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a5ffb640 4 set_mon_vals no callback set 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8077100 con 0x7f85a80a4e60 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f859801a070 con 0x7f85a80a4e60 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f859403b970 0x7f859403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6100+0+0 (secure 0 0 0) 0x7f859804f3e0 con 0x7f85a80a4e60 2024-09-15T21:21:08.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f859403b970 0x7f859403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 10 monclient: get_auth_request con 0x7f859403b970 auth_method 0 2024-09-15T21:21:08.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.162+0000 7f85a6ffd640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f859403b970 0x7f859403de30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ad000 tx=0x7f859c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:08.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.166+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f85a80a2eb0 con 0x7f85a80a4e60 2024-09-15T21:21:08.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.166+0000 7f858ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:08.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.166+0000 7f858ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:08.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.330+0000 7f85adf7f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:08.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.330+0000 7f85adf7f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:08.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.330+0000 7f85adf7f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.330+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7f85a8121e50 con 0x7f85a80a4e60 2024-09-15T21:21:08.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.518+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v98) ==== 120+0+0 (secure 0 0 0) 0x7f85980192c0 con 0x7f85a80a4e60 2024-09-15T21:21:08.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.518+0000 7f858ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:08.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.518+0000 7f858ffff640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-09-15T21:21:08.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f85adf7f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:08.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f85adf7f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:08.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f85adf7f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7f85a8137fd0 con 0x7f85a80a4e60 2024-09-15T21:21:08.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f858ffff640 1 -- 172.21.15.119:0/1990139122 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v98) ==== 120+0+0 (secure 0 0 0) 0x7f859804d970 con 0x7f85a80a4e60 2024-09-15T21:21:08.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f858ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T21:21:08.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.630+0000 7f858ffff640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-09-15T21:21:08.637 INFO:tasks.workunit.client.0.smithi119.stderr:device 'foo' does not appear in the crush map 2024-09-15T21:21:08.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f859403b970 msgr2=0x7f859403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f859403b970 0x7f859403de30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ad000 tx=0x7f859c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:21:08.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 10 monclient: shutdown 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 msgr2=0x7f85a8137550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 0x7f85a8137550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8598001ec0 tx=0x7f859800c960 comp rx=0 tx=0).stop 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 shutdown_connections 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f859403b970 0x7f859403de30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85a813d480 0x7f85a8071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85a80a5950 0x7f85a8137a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 --2- 172.21.15.119:0/1990139122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a80a4e60 0x7f85a8137550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 >> 172.21.15.119:0/1990139122 conn(0x7f85a80a6f60 msgr2=0x7f85a813e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 shutdown_connections 2024-09-15T21:21:08.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.634+0000 7f85adf7f640 1 -- 172.21.15.119:0/1990139122 wait complete. 2024-09-15T21:21:08.655 INFO:tasks.workunit.client.0.smithi119.stderr:++ ceph osd create 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.754+0000 7f6c1947f640 10 monclient: get_monmap_and_config 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.754+0000 7f6c1947f640 10 monclient: build_initial_monmap 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.754+0000 7f6c1947f640 10 monclient: monmap: 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:08.759720+0000 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:08.759720+0000 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.764 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:08.764 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:08.764 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:08.764 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:08.764 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:08.764 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 1 Processor -- start 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 1 -- start start 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient: init 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6c14073220 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6c140a0e00 0x7f6c14073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient(hunting): picked mon.noname-c con 0x7f6c140a0e00 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6c140a2cb0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c14071450 con 0x7f6c14073220 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c14071730 con 0x7f6c140a2cb0 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6c14071a10 con 0x7f6c140a0e00 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient(hunting): authenticate will time out at 1187.557160s 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6c140a0e00 0x7f6c14073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c12ffd640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f6c14073220 legacy=0x7f6c140a25a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6c140a0e00 0x7f6c14073600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46918/0 (socket says 172.21.15.119:46918) 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 -- 172.21.15.119:0/356476238 learned_addr learned my addr 172.21.15.119:0/356476238 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): get_auth_request con 0x7f6c140a0e00 auth_method 0 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient(hunting): handle_auth_done global_id 34711 payload 274 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 -- 172.21.15.119:0/356476238 >> v1:172.21.15.119:6789/0 conn(0x7f6c140a2cb0 legacy=0x7f6c1413ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 -- 172.21.15.119:0/356476238 >> v1:172.21.15.17:6789/0 conn(0x7f6c14073220 legacy=0x7f6c140a25a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 monclient: found mon.noname-c 2024-09-15T21:21:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 -- 172.21.15.119:0/356476238 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c14071cf0 con 0x7f6c140a0e00 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c127fc640 1 --2- 172.21.15.119:0/356476238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c14073600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009870 tx=0x7f6c0801c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb12a2c4bcb0fb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 1 -- 172.21.15.119:0/356476238 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080b1070 con 0x7f6c140a0e00 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: dump: 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 1 -- 172.21.15.119:0/356476238 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c080ac620 con 0x7f6c140a0e00 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 5 monclient: authenticate success, global_id 34711 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 1 -- 172.21.15.119:0/356476238 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080ac9f0 con 0x7f6c140a0e00 2024-09-15T21:21:08.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c1947f640 4 set_mon_vals no callback set 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.758+0000 7f6c11ffb640 10 monclient: dump: 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.772 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: shutdown 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/356476238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 msgr2=0x7f6c14073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 --2- 172.21.15.119:0/356476238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c14073600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009870 tx=0x7f6c0801c4b0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/356476238 shutdown_connections 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 --2- 172.21.15.119:0/356476238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c14073600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/356476238 >> 172.21.15.119:0/356476238 conn(0x7f6c1406cdc0 msgr2=0x7f6c1406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/356476238 shutdown_connections 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/356476238 wait complete. 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: build_initial_monmap 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: monmap: 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:08.773 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 Processor -- start 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- start start 2024-09-15T21:21:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: init 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6c14073220 0x7f6c141cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): picked mon.noname-c con 0x7f6c14073220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c1413cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): picked mon.noname-b con 0x7f6c140a0e00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c140a2cb0 0x7f6c1413d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): picked mon.noname-a con 0x7f6c140a2cb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14140950 con 0x7f6c140a2cb0 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14134dc0 con 0x7f6c140a0e00 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6c14140ad0 con 0x7f6c14073220 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient(hunting): authenticate will time out at 1187.562101s 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c1413cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c1413cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46922/0 (socket says 172.21.15.119:46922) 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 -- 172.21.15.119:0/1843629225 learned_addr learned my addr 172.21.15.119:0/1843629225 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): get_auth_request con 0x7f6c140a0e00 auth_method 0 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c137fe640 1 -- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c140a2cb0 msgr2=0x7f6c1413d1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c137fe640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c140a2cb0 0x7f6c1413d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient(hunting): handle_auth_done global_id 34714 payload 824 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c12ffd640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6c14073220 0x7f6c141cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 -- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6c14073220 msgr2=0x7f6c141cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6c14073220 0x7f6c141cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 -- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c140a2cb0 msgr2=0x7f6c1413d1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c140a2cb0 0x7f6c1413d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 monclient: found mon.noname-b 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 -- 172.21.15.119:0/1843629225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c141393f0 con 0x7f6c140a0e00 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c127fc640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c1413cc80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c080af040 tx=0x7f6c0801acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080bb020 con 0x7f6c140a0e00 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: dump: 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c080b7640 con 0x7f6c140a0e00 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 5 monclient: authenticate success, global_id 34714 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _renew_subs 2024-09-15T21:21:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c140a4a90 con 0x7f6c140a0e00 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080b1040 con 0x7f6c140a0e00 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 10 monclient: dump: 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _renew_subs 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1413b750 con 0x7f6c140a0e00 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c117fa640 4 set_mon_vals no callback set 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c14074220 con 0x7f6c140a0e00 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.762+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6c080be020 con 0x7f6c140a0e00 2024-09-15T21:21:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.766+0000 7f6bfb7fe640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6c0003b9a0 0x7f6c0003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.766+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5802+0+0 (secure 0 0 0) 0x7f6c080ea420 con 0x7f6c140a0e00 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.766+0000 7f6c12ffd640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6c0003b9a0 0x7f6c0003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.766+0000 7f6c12ffd640 10 monclient: get_auth_request con 0x7f6c0003b9a0 auth_method 0 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.766+0000 7f6c12ffd640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6c0003b9a0 0x7f6c0003de60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c1413d980 tx=0x7f6c04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.770+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6c080c5030 con 0x7f6c140a0e00 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.770+0000 7f6bfb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.770+0000 7f6bfb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:08.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.938+0000 7f6c1947f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:21:08.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.938+0000 7f6c1947f640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T21:21:08.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.938+0000 7f6c1947f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:08.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:08.938+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f6c14073600 con 0x7f6c140a0e00 2024-09-15T21:21:09.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.522+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v99) ==== 58+0+3 (secure 0 0 0) 0x7f6c08016130 con 0x7f6c140a0e00 2024-09-15T21:21:09.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.522+0000 7f6bfb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T21:21:09.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.522+0000 7f6bfb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:09.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.638+0000 7f6c1947f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:21:09.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.638+0000 7f6c1947f640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T21:21:09.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.638+0000 7f6c1947f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:09.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:09.638+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f6c14139f50 con 0x7f6c140a0e00 2024-09-15T21:21:10.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.614+0000 7f6bfb7fe640 1 -- 172.21.15.119:0/1843629225 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v100) ==== 58+0+3 (secure 0 0 0) 0x7f6c080b4d30 con 0x7f6c140a0e00 2024-09-15T21:21:10.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.614+0000 7f6bfb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T21:21:10.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.614+0000 7f6bfb7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:10.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6c0003b9a0 msgr2=0x7f6c0003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:10.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6c0003b9a0 0x7f6c0003de60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c1413d980 tx=0x7f6c04008040 comp rx=0 tx=0).stop 2024-09-15T21:21:10.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 10 monclient: shutdown 2024-09-15T21:21:10.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 msgr2=0x7f6c1413cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c1413cc80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c080af040 tx=0x7f6c0801acf0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 shutdown_connections 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6c0003b9a0 0x7f6c0003de60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c140a2cb0 0x7f6c1413d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c140a0e00 0x7f6c1413cc80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 --2- 172.21.15.119:0/1843629225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6c14073220 0x7f6c141cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 >> 172.21.15.119:0/1843629225 conn(0x7f6c1406cdc0 msgr2=0x7f6c1409fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 shutdown_connections 2024-09-15T21:21:10.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.618+0000 7f6c1947f640 1 -- 172.21.15.119:0/1843629225 wait complete. 2024-09-15T21:21:10.643 INFO:tasks.workunit.client.0.smithi119.stderr:+ o3=11 2024-09-15T21:21:10.644 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 11 123 root=default 2024-09-15T21:21:10.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient: get_monmap_and_config 2024-09-15T21:21:10.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient: build_initial_monmap 2024-09-15T21:21:10.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient: monmap: 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:10.765130+0000 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:10.765130+0000 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:10.767 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:10.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 Processor -- start 2024-09-15T21:21:10.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 -- start start 2024-09-15T21:21:10.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient: init 2024-09-15T21:21:10.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:10.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): picked mon.noname-b con 0x7fb5c0135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5c0135fd0 0x7fb5c00a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): picked mon.noname-c con 0x7fb5c0135fd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb5c0136430 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c00a1d80 con 0x7fb5c0136430 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c00a3600 con 0x7fb5c0135fd0 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00a2060 con 0x7fb5c0135bf0 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5c61b2640 10 monclient(hunting): authenticate will time out at 1189.561780s 2024-09-15T21:21:10.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5c0135fd0 0x7fb5c00a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5bffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fb5c0136430 legacy=0x7fb5c00a0f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5c0135fd0 0x7fb5c00a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46936/0 (socket says 172.21.15.119:46936) 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 -- 172.21.15.119:0/2857297258 learned_addr learned my addr 172.21.15.119:0/2857297258 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): get_auth_request con 0x7fb5c0135fd0 auth_method 0 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:10.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5bf7fe640 1 --2- 172.21.15.119:0/2857297258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb5c0135bf0 auth_method 0 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient(hunting): handle_auth_done global_id 34717 payload 274 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 -- 172.21.15.119:0/2857297258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 msgr2=0x7fb5c009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 --2- 172.21.15.119:0/2857297258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 -- 172.21.15.119:0/2857297258 >> v1:172.21.15.17:6789/0 conn(0x7fb5c0136430 legacy=0x7fb5c00a0f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 monclient: found mon.noname-c 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:10.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.762+0000 7fb5beffd640 1 -- 172.21.15.119:0/2857297258 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00a2340 con 0x7fb5c0135fd0 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5beffd640 1 --2- 172.21.15.119:0/2857297258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0135fd0 0x7fb5c00a0a50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009870 tx=0x7fb5b401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be8f76807f27cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 1 -- 172.21.15.119:0/2857297258 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40b0070 con 0x7fb5c0135fd0 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: dump: 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:10.774 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 5 monclient: authenticate success, global_id 34717 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 1 -- 172.21.15.119:0/2857297258 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b40ab600 con 0x7fb5c0135fd0 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 4 set_mon_vals no callback set 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 1 -- 172.21.15.119:0/2857297258 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40ab9d0 con 0x7fb5c0135fd0 2024-09-15T21:21:10.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5be7fc640 10 monclient: dump: 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:10.776 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: shutdown 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/2857297258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0135fd0 msgr2=0x7fb5c00a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/2857297258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0135fd0 0x7fb5c00a0a50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009870 tx=0x7fb5b401c420 comp rx=0 tx=0).stop 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/2857297258 shutdown_connections 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/2857297258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0135fd0 0x7fb5c00a0a50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/2857297258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/2857297258 >> 172.21.15.119:0/2857297258 conn(0x7fb5c00a66e0 msgr2=0x7fb5c00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/2857297258 shutdown_connections 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/2857297258 wait complete. 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: build_initial_monmap 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: monmap: 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:10.782 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 Processor -- start 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- start start 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: init 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c013c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): picked mon.noname-c con 0x7fb5c0135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:10.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c0135fd0 0x7fb5c01348f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): picked mon.noname-a con 0x7fb5c0135fd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 0x7fb5c01cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): picked mon.noname-b con 0x7fb5c0136430 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00a1820 con 0x7fb5c0135fd0 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00a16a0 con 0x7fb5c0136430 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c01402b0 con 0x7fb5c0135bf0 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient(hunting): authenticate will time out at 1189.566671s 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 0x7fb5c01cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:10.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5beffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c0135fd0 msgr2=0x7fb5c01348f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 0x7fb5c01cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46938/0 (socket says 172.21.15.119:46938) 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c0135fd0 0x7fb5c01348f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 -- 172.21.15.119:0/1512779928 learned_addr learned my addr 172.21.15.119:0/1512779928 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): get_auth_request con 0x7fb5c0136430 auth_method 0 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bf7fe640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c013c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb5c0135bf0 auth_method 0 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:10.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient(hunting): handle_auth_done global_id 34720 payload 824 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 -- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 msgr2=0x7fb5c013c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c013c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 -- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c0135fd0 msgr2=0x7fb5c01348f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c0135fd0 0x7fb5c01348f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 monclient: found mon.noname-b 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 -- 172.21.15.119:0/1512779928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c013d230 con 0x7fb5c0136430 2024-09-15T21:21:10.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bffff640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 0x7fb5c01cd4f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b790 tx=0x7fb5b800bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800c960 con 0x7fb5c0136430 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: dump: 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:10.787 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8010070 con 0x7fb5c0136430 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 5 monclient: authenticate success, global_id 34720 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: _renew_subs 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c011d650 con 0x7fb5c0136430 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800cd40 con 0x7fb5c0136430 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:10.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5a7fff640 10 monclient: dump: 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: _renew_subs 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c011f280 con 0x7fb5c0136430 2024-09-15T21:21:10.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.766+0000 7fb5bd7fa640 4 set_mon_vals no callback set 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5c61b2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5c61b2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5c61b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c009fe70 con 0x7fb5c0136430 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb5b8015470 con 0x7fb5c0136430 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5a7fff640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5ac03b970 0x7fb5ac03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6032+0+0 (secure 0 0 0) 0x7fb5b804feb0 con 0x7fb5c0136430 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5bf7fe640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5ac03b970 0x7fb5ac03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5bf7fe640 10 monclient: get_auth_request con 0x7fb5ac03b970 auth_method 0 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.770+0000 7fb5bf7fe640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5ac03b970 0x7fb5ac03de30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000b9d0 tx=0x7fb5b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.774+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb5b8007d30 con 0x7fb5c0136430 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.774+0000 7fb5a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:10.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.774+0000 7fb5a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:10.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.946+0000 7fb5c61b2640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T21:21:10.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.946+0000 7fb5c61b2640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T21:21:10.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.946+0000 7fb5c61b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:10.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:10.946+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) -- 0x7fb5c011ea40 con 0x7fb5c0136430 2024-09-15T21:21:11.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.622+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map v101) ==== 193+0+0 (secure 0 0 0) 0x7fb5b801e3f0 con 0x7fb5c0136430 2024-09-15T21:21:11.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.622+0000 7fb5a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T21:21:11.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.622+0000 7fb5a7fff640 10 monclient: _finish_command 2 = system:0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map 2024-09-15T21:21:11.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5c61b2640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T21:21:11.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5c61b2640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T21:21:11.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5c61b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:11.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) -- 0x7fb5c011da80 con 0x7fb5c0136430 2024-09-15T21:21:11.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5a7fff640 1 -- 172.21.15.119:0/1512779928 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change v101) ==== 191+0+0 (secure 0 0 0) 0x7fb5b8040020 con 0x7fb5c0136430 2024-09-15T21:21:11.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5a7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T21:21:11.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.726+0000 7fb5a7fff640 10 monclient: _finish_command 3 = system:0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-09-15T21:21:11.734 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5ac03b970 msgr2=0x7fb5ac03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5ac03b970 0x7fb5ac03de30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000b9d0 tx=0x7fb5b0008040 comp rx=0 tx=0).stop 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 10 monclient: shutdown 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 msgr2=0x7fb5c01cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 0x7fb5c01cd4f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b790 tx=0x7fb5b800bc50 comp rx=0 tx=0).stop 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 shutdown_connections 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5ac03b970 0x7fb5ac03de30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0136430 0x7fb5c01cd4f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c0135fd0 0x7fb5c01348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 --2- 172.21.15.119:0/1512779928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5c0135bf0 0x7fb5c013c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.736 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 >> 172.21.15.119:0/1512779928 conn(0x7fb5c00a66e0 msgr2=0x7fb5c00a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:11.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 shutdown_connections 2024-09-15T21:21:11.737 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.730+0000 7fb5c61b2640 1 -- 172.21.15.119:0/1512779928 wait complete. 2024-09-15T21:21:11.751 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:21:11.751 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep osd.11 2024-09-15T21:21:11.751 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 123 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient: get_monmap_and_config 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient: build_initial_monmap 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient: monmap: 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:11.864971+0000 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:11.864971+0000 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:11.867 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:11.867 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 Processor -- start 2024-09-15T21:21:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 -- start start 2024-09-15T21:21:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient: init 2024-09-15T21:21:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:11.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:21:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4a80a4d80 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:11.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 0x7fc4a813cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): picked mon.noname-b con 0x7fc4a80a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4a813d510 0x7fc4a813f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): picked mon.noname-a con 0x7fc4a813d510 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a80a2250 con 0x7fc4a80a4d80 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a80a4650 con 0x7fc4a813d510 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a806dab0 con 0x7fc4a80a5870 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4ae121640 10 monclient(hunting): authenticate will time out at 1190.661724s 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a77fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fc4a80a4d80 legacy=0x7fc4a80a5160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a7fff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fc4a813d510 msgr2=0x7fc4a813f900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:11.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4a813d510 0x7fc4a813f900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 0x7fc4a813cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 0x7fc4a813cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:36144/0 (socket says 172.21.15.119:36144) 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/1944297489 learned_addr learned my addr 172.21.15.119:0/1944297489 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): get_auth_request con 0x7fc4a80a5870 auth_method 0 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:11.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_done global_id 44234 payload 274 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/1944297489 >> v2:172.21.15.17:3300/0 conn(0x7fc4a813d510 msgr2=0x7fc4a813f900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 --2- 172.21.15.119:0/1944297489 >> v2:172.21.15.17:3300/0 conn(0x7fc4a813d510 0x7fc4a813f900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/1944297489 >> v1:172.21.15.17:6789/0 conn(0x7fc4a80a4d80 legacy=0x7fc4a80a5160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 monclient: found mon.noname-b 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:11.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.862+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/1944297489 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a80a19b0 con 0x7fc4a80a5870 2024-09-15T21:21:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a6ffd640 1 --2- 172.21.15.119:0/1944297489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 0x7fc4a813cfd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc498009e50 tx=0x7fc49801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7401724dc5eab3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 1 -- 172.21.15.119:0/1944297489 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980a5660 con 0x7fc4a80a5870 2024-09-15T21:21:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:11.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: dump: 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:11.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 1 -- 172.21.15.119:0/1944297489 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4980a5800 con 0x7fc4a80a5870 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 1 -- 172.21.15.119:0/1944297489 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980a5bd0 con 0x7fc4a80a5870 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4a67fc640 10 monclient: dump: 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:11.880 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 5 monclient: authenticate success, global_id 44234 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 4 set_mon_vals no callback set 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 10 monclient: shutdown 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 -- 172.21.15.119:0/1944297489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 msgr2=0x7fc4a813cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 --2- 172.21.15.119:0/1944297489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 0x7fc4a813cfd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc498009e50 tx=0x7fc49801c910 comp rx=0 tx=0).stop 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 -- 172.21.15.119:0/1944297489 shutdown_connections 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 --2- 172.21.15.119:0/1944297489 >> v2:172.21.15.17:3300/0 conn(0x7fc4a813d510 0x7fc4a813f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 --2- 172.21.15.119:0/1944297489 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a5870 0x7fc4a813cfd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 -- 172.21.15.119:0/1944297489 >> 172.21.15.119:0/1944297489 conn(0x7fc4a80a75c0 msgr2=0x7fc4a806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 -- 172.21.15.119:0/1944297489 shutdown_connections 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 1 -- 172.21.15.119:0/1944297489 wait complete. 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 10 monclient: build_initial_monmap 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.866+0000 7fc4ae121640 10 monclient: monmap: 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:11.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 Processor -- start 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- start start 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: init 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a4d80 0x7fc4a813a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): picked mon.noname-c con 0x7fc4a80a4d80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 0x7fc4a813a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): picked mon.noname-b con 0x7fc4a80a5870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a813d510 0x7fc4a813abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): picked mon.noname-a con 0x7fc4a813d510 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:11.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a80a2de0 con 0x7fc4a813d510 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a80a2c60 con 0x7fc4a80a5870 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a8142660 con 0x7fc4a80a4d80 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient(hunting): authenticate will time out at 1190.668904s 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a7fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a813d510 msgr2=0x7fc4a813abd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 0x7fc4a813a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a813d510 0x7fc4a813abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 0x7fc4a813a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46950/0 (socket says 172.21.15.119:46950) 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/86682244 learned_addr learned my addr 172.21.15.119:0/86682244 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:11.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): get_auth_request con 0x7fc4a80a5870 auth_method 0 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a4d80 0x7fc4a813a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): get_auth_request con 0x7fc4a80a4d80 auth_method 0 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:11.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient(hunting): handle_auth_done global_id 34723 payload 824 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a4d80 msgr2=0x7fc4a813a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a4d80 0x7fc4a813a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a813d510 msgr2=0x7fc4a813abd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a813d510 0x7fc4a813abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 monclient: found mon.noname-b 2024-09-15T21:21:11.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 -- 172.21.15.119:0/86682244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a806ecb0 con 0x7fc4a80a5870 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a6ffd640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 0x7fc4a813a690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc49801d1b0 tx=0x7fc4980196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980bc020 con 0x7fc4a80a5870 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: dump: 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:11.887 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: _finish_auth 0 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4980b8670 con 0x7fc4a80a5870 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 5 monclient: authenticate success, global_id 34723 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980b8a60 con 0x7fc4a80a5870 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:11.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 10 monclient: dump: 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4a5ffb640 4 set_mon_vals no callback set 2024-09-15T21:21:11.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _renew_subs 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a8073a60 con 0x7fc4a80a5870 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _renew_subs 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a813bc10 con 0x7fc4a80a5870 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a80a5160 con 0x7fc4a80a5870 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.870+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc4980bf020 con 0x7fc4a80a5870 2024-09-15T21:21:11.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.874+0000 7fc49ffff640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc48803b9d0 0x7fc48803de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.874+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6054+0+0 (secure 0 0 0) 0x7fc4980ead90 con 0x7fc4a80a5870 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.874+0000 7fc4a77fe640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc48803b9d0 0x7fc48803de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.874+0000 7fc4a77fe640 10 monclient: get_auth_request con 0x7fc48803b9d0 auth_method 0 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.874+0000 7fc4a77fe640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc48803b9d0 0x7fc48803de90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc490009380 tx=0x7fc490009310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.878+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc4980c6070 con 0x7fc4a80a5870 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.878+0000 7fc49ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:11.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:11.878+0000 7fc49ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:12.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc4ae121640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:12.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc4ae121640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:12.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc4ae121640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fc4a806f520 con 0x7fc4a80a5870 2024-09-15T21:21:12.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v101) ==== 56+0+1173 (secure 0 0 0) 0x7fc4a806ecb0 con 0x7fc4a80a5870 2024-09-15T21:21:12.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc49ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:12.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.034+0000 7fc49ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:12.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc4ae121640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:12.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc4ae121640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:12.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc4ae121640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fc4a80008e0 con 0x7fc4a80a5870 2024-09-15T21:21:12.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc49ffff640 1 -- 172.21.15.119:0/86682244 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v101) ==== 56+0+1173 (secure 0 0 0) 0x7fc498016620 con 0x7fc4a80a5870 2024-09-15T21:21:12.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc49ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:12.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.126+0000 7fc49ffff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:12.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc48803b9d0 msgr2=0x7fc48803de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc48803b9d0 0x7fc48803de90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc490009380 tx=0x7fc490009310 comp rx=0 tx=0).stop 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 10 monclient: shutdown 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 msgr2=0x7fc4a813a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 0x7fc4a813a690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc49801d1b0 tx=0x7fc4980196b0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 shutdown_connections 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc48803b9d0 0x7fc48803de90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a813d510 0x7fc4a813abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a80a5870 0x7fc4a813a690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 --2- 172.21.15.119:0/86682244 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4a80a4d80 0x7fc4a813a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 >> 172.21.15.119:0/86682244 conn(0x7fc4a80a75c0 msgr2=0x7fc4a8137c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:12.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 shutdown_connections 2024-09-15T21:21:12.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.130+0000 7fc4ae121640 1 -- 172.21.15.119:0/86682244 wait complete. 2024-09-15T21:21:12.149 INFO:tasks.workunit.client.0.smithi119.stdout: 11 123.00000 osd.11 down 1.00000 1.00000 2024-09-15T21:21:12.149 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush reweight osd.11 113 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.258+0000 7fde97389640 10 monclient: get_monmap_and_config 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.258+0000 7fde97389640 10 monclient: build_initial_monmap 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.258+0000 7fde97389640 10 monclient: monmap: 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:12.263740+0000 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:12.263740+0000 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:12.265 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:12.266 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:12.266 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:12.266 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:12.266 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.258+0000 7fde97389640 1 Processor -- start 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.258+0000 7fde97389640 1 -- start start 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.258+0000 7fde97389640 10 monclient: init 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fde9009d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fde9009def0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde9013b0a0 0x7fde9013d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient(hunting): picked mon.noname-a con 0x7fde9013b0a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9006e240 con 0x7fde9009d400 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9009e7f0 con 0x7fde9009def0 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fde9006e000 con 0x7fde9013b0a0 2024-09-15T21:21:12.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient(hunting): authenticate will time out at 1191.060529s 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde95125640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fde9009d400 legacy=0x7fde9009d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde95926640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fde9013b0a0 msgr2=0x7fde9013d490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fde9009def0 0x7fde90139980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59320/0 (socket says 172.21.15.119:59320) 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 -- 172.21.15.119:0/746621500 learned_addr learned my addr 172.21.15.119:0/746621500 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde95926640 1 --2- 172.21.15.119:0/746621500 >> v2:172.21.15.17:3300/0 conn(0x7fde9013b0a0 0x7fde9013d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085744450 0 0) 0x7fde9009e7f0 con 0x7fde9009def0 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): _negotiate 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): my global_id is 34726 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde80003530 con 0x7fde9009def0 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 841671868 0 0) 0x7fde80003530 con 0x7fde9009def0 2024-09-15T21:21:12.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 >> v2:172.21.15.17:3300/0 conn(0x7fde9013b0a0 msgr2=0x7fde9013d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 --2- 172.21.15.119:0/746621500 >> v2:172.21.15.17:3300/0 conn(0x7fde9013b0a0 0x7fde9013d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 >> v1:172.21.15.17:6789/0 conn(0x7fde9009d400 legacy=0x7fde9009d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900a2500 con 0x7fde9009def0 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: _finish_auth 0 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 5 monclient: authenticate success, global_id 34726 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fde7c001700 con 0x7fde9009def0 2024-09-15T21:21:12.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: dump: 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:12.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 >> v1:172.21.15.119:6789/0 conn(0x7fde9009def0 legacy=0x7fde90139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde800039c0 0x7fde80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): picked mon.a con 0x7fde800039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde80004810 0x7fde80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): picked mon.b con 0x7fde80004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80005380 0x7fde8000b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): picked mon.c con 0x7fde80005380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde9006e000 con 0x7fde800039c0 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde8000c2d0 con 0x7fde80004810 2024-09-15T21:21:12.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fde8000c4d0 con 0x7fde80005380 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde80004810 0x7fde80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): get_auth_request con 0x7fde80004810 auth_method 0 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde95125640 1 -- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde800039c0 msgr2=0x7fde80003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde95125640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde800039c0 0x7fde80003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:12.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient(hunting): handle_auth_done global_id 34726 payload 293 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 -- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80005380 msgr2=0x7fde8000b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80005380 0x7fde8000b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 -- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde800039c0 msgr2=0x7fde80003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde800039c0 0x7fde80003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient: _finish_hunting 0 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 monclient: found mon.b 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 -- 172.21.15.119:0/746621500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde80003530 con 0x7fde80004810 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient: _finish_auth 0 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 10 monclient: _check_auth_tickets 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:12.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde95926640 1 -- 172.21.15.119:0/746621500 reap_dead start 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde94924640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde80004810 0x7fde80004c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde7c007fa0 tx=0x7fde7c01ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85a30f8d12511cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c007d20 con 0x7fde80004810 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: dump: 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:12.277 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7c0ab020 con 0x7fde80004810 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde97389640 4 set_mon_vals no callback set 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 1 -- 172.21.15.119:0/746621500 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c005ae0 con 0x7fde80004810 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.262+0000 7fde8ffff640 10 monclient: dump: 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:12.278 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:12.279 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: shutdown 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/746621500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde80004810 msgr2=0x7fde80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde80004810 0x7fde80004c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde7c007fa0 tx=0x7fde7c01ed90 comp rx=0 tx=0).stop 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/746621500 shutdown_connections 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 --2- 172.21.15.119:0/746621500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde80004810 0x7fde80004c90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/746621500 >> 172.21.15.119:0/746621500 conn(0x7fde9009be60 msgr2=0x7fde9009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/746621500 shutdown_connections 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/746621500 wait complete. 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: build_initial_monmap 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: monmap: 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:12.285 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 Processor -- start 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- start start 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: init 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80004810 0x7fde90133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): picked mon.noname-c con 0x7fde80004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde901341e0 0x7fde901cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): picked mon.noname-a con 0x7fde901341e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 0x7fde901cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): picked mon.noname-b con 0x7fde901cc870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde900a0360 con 0x7fde901341e0 2024-09-15T21:21:12.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde900a01e0 con 0x7fde901cc870 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fde90134ee0 con 0x7fde80004810 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient(hunting): authenticate will time out at 1191.066054s 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde94924640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde901341e0 msgr2=0x7fde901cc330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 0x7fde901cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde94924640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde901341e0 0x7fde901cc330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 0x7fde901cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41684/0 (socket says 172.21.15.119:41684) 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 -- 172.21.15.119:0/1093468143 learned_addr learned my addr 172.21.15.119:0/1093468143 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): get_auth_request con 0x7fde901cc870 auth_method 0 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:12.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95125640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80004810 0x7fde90133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient(hunting): handle_auth_done global_id 34729 payload 824 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 -- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80004810 msgr2=0x7fde90133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80004810 0x7fde90133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 -- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde901341e0 msgr2=0x7fde901cc330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde901341e0 0x7fde901cc330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient: _finish_hunting 0 2024-09-15T21:21:12.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 monclient: found mon.noname-b 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 -- 172.21.15.119:0/1093468143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde90124730 con 0x7fde901cc870 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95125640 10 monclient: get_auth_request con 0x7fde80004810 auth_method 0 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95125640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80004810 0x7fde90133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde95926640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 0x7fde901cf850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fde90069fa0 tx=0x7fde8800b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88016070 con 0x7fde901cc870 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: dump: 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.289 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde88003dc0 con 0x7fde901cc870 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88005290 con 0x7fde901cc870 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 5 monclient: authenticate success, global_id 34729 2024-09-15T21:21:12.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _renew_subs 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde90123340 con 0x7fde901cc870 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8f7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 10 monclient: dump: 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:12.291 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _renew_subs 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde901ccce0 con 0x7fde901cc870 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fde88019070 con 0x7fde901cc870 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9009e7f0 con 0x7fde901cc870 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.266+0000 7fde8dffb640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fde7403b970 0x7fde7403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.270+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6054+0+0 (secure 0 0 0) 0x7fde8804efa0 con 0x7fde901cc870 2024-09-15T21:21:12.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.270+0000 7fde95125640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fde7403b970 0x7fde7403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:12.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.270+0000 7fde95125640 10 monclient: get_auth_request con 0x7fde7403b970 auth_method 0 2024-09-15T21:21:12.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.270+0000 7fde95125640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fde7403b970 0x7fde7403de30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde84003cc0 tx=0x7fde84008330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:12.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.274+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fde88019d50 con 0x7fde901cc870 2024-09-15T21:21:12.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.274+0000 7fde8dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:12.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.274+0000 7fde8dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:12.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.438+0000 7fde97389640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T21:21:12.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.438+0000 7fde97389640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T21:21:12.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.438+0000 7fde97389640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.438+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) -- 0x7fde90124e60 con 0x7fde901cc870 2024-09-15T21:21:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.518+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v102) ==== 156+0+0 (secure 0 0 0) 0x7fde8801a8d0 con 0x7fde901cc870 2024-09-15T21:21:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.518+0000 7fde8dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T21:21:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.518+0000 7fde8dffb640 10 monclient: _finish_command 2 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-09-15T21:21:12.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.626+0000 7fde97389640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T21:21:12.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.626+0000 7fde97389640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T21:21:12.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.626+0000 7fde97389640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:12.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:12.626+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) -- 0x7fde901d0550 con 0x7fde901cc870 2024-09-15T21:21:13.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.530+0000 7fde8dffb640 1 -- 172.21.15.119:0/1093468143 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v103) ==== 156+0+0 (secure 0 0 0) 0x7fde8801a3b0 con 0x7fde901cc870 2024-09-15T21:21:13.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.530+0000 7fde8dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T21:21:13.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.530+0000 7fde8dffb640 10 monclient: _finish_command 3 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-09-15T21:21:13.534 INFO:tasks.workunit.client.0.smithi119.stderr:reweighted item id 11 name 'osd.11' to 113 in crush map 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fde7403b970 msgr2=0x7fde7403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fde7403b970 0x7fde7403de30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde84003cc0 tx=0x7fde84008330 comp rx=0 tx=0).stop 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 10 monclient: shutdown 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 msgr2=0x7fde901cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 0x7fde901cf850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fde90069fa0 tx=0x7fde8800b7a0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 shutdown_connections 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fde7403b970 0x7fde7403de30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde901cc870 0x7fde901cf850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde901341e0 0x7fde901cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 --2- 172.21.15.119:0/1093468143 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fde80004810 0x7fde90133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 >> 172.21.15.119:0/1093468143 conn(0x7fde9009be60 msgr2=0x7fde900a4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 shutdown_connections 2024-09-15T21:21:13.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.534+0000 7fde97389640 1 -- 172.21.15.119:0/1093468143 wait complete. 2024-09-15T21:21:13.561 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush reweight osd.11 123456 2024-09-15T21:21:13.561 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:21:13.561 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush reweight osd.11 123456 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.674+0000 7efdfbe4d640 10 monclient: get_monmap_and_config 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.674+0000 7efdfbe4d640 10 monclient: build_initial_monmap 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.674+0000 7efdfbe4d640 10 monclient: monmap: 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:13.680099+0000 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:13.680099+0000 2024-09-15T21:21:13.681 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:13.682 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.674+0000 7efdfbe4d640 1 Processor -- start 2024-09-15T21:21:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 -- start start 2024-09-15T21:21:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient: init 2024-09-15T21:21:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2024-09-15T21:21:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdf40a79e0 0x7efdf40a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): picked mon.noname-a con 0x7efdf40a79e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a46b0 0x7efdf41377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): picked mon.noname-b con 0x7efdf40a46b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efdf40a1100 0x7efdf4137d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): picked mon.noname-c con 0x7efdf40a1100 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7efdf40a7dc0 con 0x7efdf40a79e0 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7efdf4135180 con 0x7efdf40a1100 2024-09-15T21:21:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efdf40a6920 con 0x7efdf40a46b0 2024-09-15T21:21:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient(hunting): authenticate will time out at 1192.476718s 2024-09-15T21:21:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efdf40a1100 0x7efdf4137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf9be9640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7efdf40a79e0 msgr2=0x7efdf40a4170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf9be9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdf40a79e0 0x7efdf40a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efdf40a1100 0x7efdf4137d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41692/0 (socket says 172.21.15.119:41692) 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 -- 172.21.15.119:0/2433467357 learned_addr learned my addr 172.21.15.119:0/2433467357 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): get_auth_request con 0x7efdf40a1100 auth_method 0 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf93e8640 1 --2- 172.21.15.119:0/2433467357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a46b0 0x7efdf41377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf93e8640 10 monclient(hunting): get_auth_request con 0x7efdf40a46b0 auth_method 0 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf93e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf93e8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf93e8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient(hunting): handle_auth_done global_id 34732 payload 274 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 -- 172.21.15.119:0/2433467357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a46b0 msgr2=0x7efdf41377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 --2- 172.21.15.119:0/2433467357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a46b0 0x7efdf41377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 -- 172.21.15.119:0/2433467357 >> v2:172.21.15.17:3300/0 conn(0x7efdf40a79e0 msgr2=0x7efdf40a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 --2- 172.21.15.119:0/2433467357 >> v2:172.21.15.17:3300/0 conn(0x7efdf40a79e0 0x7efdf40a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient: _finish_hunting 0 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 monclient: found mon.noname-c 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 -- 172.21.15.119:0/2433467357 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf40a1c30 con 0x7efdf40a1100 2024-09-15T21:21:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfa3ea640 1 --2- 172.21.15.119:0/2433467357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf4137d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efdf40a6850 tx=0x7efdec01c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d806113b8c9abb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 1 -- 172.21.15.119:0/2433467357 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0b2070 con 0x7efdf40a1100 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: dump: 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:13.697 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: _finish_auth 0 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: _check_auth_tickets 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 1 -- 172.21.15.119:0/2433467357 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec0a9070 con 0x7efdf40a1100 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 5 monclient: authenticate success, global_id 34732 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdfbe4d640 4 set_mon_vals no callback set 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 1 -- 172.21.15.119:0/2433467357 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0ad880 con 0x7efdf40a1100 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.678+0000 7efdf8be7640 10 monclient: dump: 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: shutdown 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/2433467357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 msgr2=0x7efdf4137d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- 172.21.15.119:0/2433467357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf4137d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efdf40a6850 tx=0x7efdec01c490 comp rx=0 tx=0).stop 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/2433467357 shutdown_connections 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- 172.21.15.119:0/2433467357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf4137d20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- 172.21.15.119:0/2433467357 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a46b0 0x7efdf41377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- 172.21.15.119:0/2433467357 >> v2:172.21.15.17:3300/0 conn(0x7efdf40a79e0 0x7efdf40a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/2433467357 >> 172.21.15.119:0/2433467357 conn(0x7efdf40a6ef0 msgr2=0x7efdf40a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/2433467357 shutdown_connections 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/2433467357 wait complete. 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: build_initial_monmap 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: monmap: 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 Processor -- start 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- start start 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: init 2024-09-15T21:21:13.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf41cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): picked mon.noname-b con 0x7efdf40a1100 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf40a46b0 0x7efdf41cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): picked mon.noname-a con 0x7efdf40a46b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a79e0 0x7efdf4072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): picked mon.noname-c con 0x7efdf40a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4141390 con 0x7efdf40a46b0 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4134930 con 0x7efdf40a1100 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efdf4141510 con 0x7efdf40a79e0 2024-09-15T21:21:13.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient(hunting): authenticate will time out at 1192.481826s 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf41cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf41cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41702/0 (socket says 172.21.15.119:41702) 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 -- 172.21.15.119:0/1725393622 learned_addr learned my addr 172.21.15.119:0/1725393622 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): get_auth_request con 0x7efdf40a1100 auth_method 0 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf93e8640 1 -- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf40a46b0 msgr2=0x7efdf41cff90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf93e8640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf40a46b0 0x7efdf41cff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:13.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient(hunting): handle_auth_done global_id 34735 payload 824 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfa3ea640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a79e0 0x7efdf4072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 -- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a79e0 msgr2=0x7efdf4072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efdf40a79e0 0x7efdf4072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 -- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf40a46b0 msgr2=0x7efdf41cff90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf40a46b0 0x7efdf41cff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient: _finish_hunting 0 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 monclient: found mon.noname-b 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 -- 172.21.15.119:0/1725393622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf4073ee0 con 0x7efdf40a1100 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf9be9640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf40a1100 0x7efdf41cfa50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b9e0 tx=0x7efdf000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:13.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf000cba0 con 0x7efdf40a1100 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: dump: 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:13.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0010070 con 0x7efdf40a1100 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 5 monclient: authenticate success, global_id 34735 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _renew_subs 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/1725393622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf413a520 con 0x7efdf40a1100 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0015470 con 0x7efdf40a1100 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 10 monclient: dump: 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:13.706 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:13.707 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdebfff640 4 set_mon_vals no callback set 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _renew_subs 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/1725393622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf4132060 con 0x7efdf40a1100 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdfbe4d640 1 -- 172.21.15.119:0/1725393622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf40a4170 con 0x7efdf40a1100 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7efdf001e3f0 con 0x7efdf40a1100 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdd803b9a0 0x7efdd803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6083+0+0 (secure 0 0 0) 0x7efdf0050190 con 0x7efdf40a1100 2024-09-15T21:21:13.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf93e8640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdd803b9a0 0x7efdd803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:13.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.682+0000 7efdf93e8640 10 monclient: get_auth_request con 0x7efdd803b9a0 auth_method 0 2024-09-15T21:21:13.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.686+0000 7efdf93e8640 1 --2- 172.21.15.119:0/1725393622 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efdd803b9a0 0x7efdd803de60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efde4006e90 tx=0x7efde4006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:13.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.690+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7efdf004e730 con 0x7efdf40a1100 2024-09-15T21:21:13.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.690+0000 7efdea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:13.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.690+0000 7efdea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:13.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdfbe4d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-09-15T21:21:13.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdfbe4d640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-09-15T21:21:13.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdfbe4d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:13.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdfbe4d640 1 -- 172.21.15.119:0/1725393622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0} v 0) -- 0x7efdf40a15e0 con 0x7efdf40a1100 2024-09-15T21:21:13.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdea7fc640 1 -- 172.21.15.119:0/1725393622 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}]=-75 (75) Value too large for defined data type v103) ==== 146+0+0 (secure 0 0 0) 0x7efdf001d3a0 con 0x7efdf40a1100 2024-09-15T21:21:13.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-09-15T21:21:13.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.862+0000 7efdea7fc640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-09-15T21:21:13.870 INFO:tasks.workunit.client.0.smithi119.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-09-15T21:21:13.876 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:21:13.876 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:21:13.876 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 113 2024-09-15T21:21:13.877 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep osd.11 2024-09-15T21:21:13.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient: get_monmap_and_config 2024-09-15T21:21:13.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient: build_initial_monmap 2024-09-15T21:21:13.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient: monmap: 2024-09-15T21:21:13.990 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:13.989031+0000 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:13.989031+0000 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:13.991 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 1 Processor -- start 2024-09-15T21:21:14.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 1 -- start start 2024-09-15T21:21:14.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient: init 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff328132390 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 0x7ff32813a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient(hunting): picked mon.noname-b con 0x7ff328132770 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff328132b80 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32806e1a0 con 0x7ff328132b80 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff328072480 con 0x7ff328132390 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff328131fd0 con 0x7ff328132770 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 10 monclient(hunting): authenticate will time out at 1192.785918s 2024-09-15T21:21:14.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff328132390 0x7ff3280730f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59336/0 (socket says 172.21.15.119:59336) 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 1 -- 172.21.15.119:0/446633258 learned_addr learned my addr 172.21.15.119:0/446633258 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 -- 172.21.15.119:0/446633258 >> v1:172.21.15.17:6789/0 conn(0x7ff328132b80 legacy=0x7ff32813ce20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 0x7ff32813a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): get_auth_request con 0x7ff328132770 auth_method 0 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1286228204 0 0) 0x7ff328072480 con 0x7ff328132390 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): _negotiate 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): my global_id is 34738 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff314003530 con 0x7ff328132390 2024-09-15T21:21:14.002 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3801401121 0 0) 0x7ff314003530 con 0x7ff328132390 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 msgr2=0x7ff32813a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 0x7ff32813a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 >> v1:172.21.15.17:6789/0 conn(0x7ff328132b80 legacy=0x7ff32813ce20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff328069800 con 0x7ff328132390 2024-09-15T21:21:14.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff320001a80 con 0x7ff328132390 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: dump: 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.004 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 >> v1:172.21.15.119:6789/0 conn(0x7ff328132390 legacy=0x7ff3280730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 0x7ff3140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): picked mon.c con 0x7ff328132770 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff314003f20 0x7ff314004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): picked mon.a con 0x7ff314003f20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314004d50 0x7ff3140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): picked mon.b con 0x7ff314004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff31400bb90 con 0x7ff314003f20 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff31400bd40 con 0x7ff314004d50 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff31400bf40 con 0x7ff328132770 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32cca7640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 1 -- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff314003f20 msgr2=0x7ff314004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32d4a8640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff314003f20 0x7ff314004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314004d50 0x7ff3140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): get_auth_request con 0x7ff314004d50 auth_method 0 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 0x7ff3140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 10 monclient(hunting): get_auth_request con 0x7ff328132770 auth_method 0 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32dca9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:14.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_done global_id 34738 payload 293 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 -- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 msgr2=0x7ff3140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff328132770 0x7ff3140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 -- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff314003f20 msgr2=0x7ff314004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff314003f20 0x7ff314004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 monclient: found mon.b 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 1 -- 172.21.15.119:0/446633258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff314003530 con 0x7ff314004d50 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32e4aa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 5 monclient: authenticate success, global_id 34738 2024-09-15T21:21:14.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.986+0000 7ff32ff0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314004d50 0x7ff3140051d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff31c003940 tx=0x7ff31c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83cf3413a18d5ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c009520 con 0x7ff314004d50 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: dump: 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.009 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31c003bc0 con 0x7ff314004d50 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 1 -- 172.21.15.119:0/446633258 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c005070 con 0x7ff314004d50 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 4 set_mon_vals no callback set 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32cca7640 10 monclient: dump: 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.010 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: shutdown 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- 172.21.15.119:0/446633258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314004d50 msgr2=0x7ff3140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/446633258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314004d50 0x7ff3140051d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff31c003940 tx=0x7ff31c01c420 comp rx=0 tx=0).stop 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 -- 172.21.15.119:0/446633258 reap_dead start 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- 172.21.15.119:0/446633258 shutdown_connections 2024-09-15T21:21:14.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- 172.21.15.119:0/446633258 >> 172.21.15.119:0/446633258 conn(0x7ff32806cbf0 msgr2=0x7ff32806cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- 172.21.15.119:0/446633258 shutdown_connections 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- 172.21.15.119:0/446633258 wait complete. 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: monmap: 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:14.012 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 Processor -- start 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- start start 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: init 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3281cf3c0 0x7ff328131350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): picked mon.noname-c con 0x7ff3281cf3c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328131890 0x7ff32813bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): picked mon.noname-a con 0x7ff328131890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 0x7ff32813c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): picked mon.noname-b con 0x7ff32813c430 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff32813eed0 con 0x7ff328131890 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff32813ed50 con 0x7ff32813c430 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff32813f050 con 0x7ff3281cf3c0 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32ff0d640 10 monclient(hunting): authenticate will time out at 1192.791525s 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 0x7ff32813c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 0x7ff32813c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41724/0 (socket says 172.21.15.119:41724) 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 -- 172.21.15.119:0/4276151508 learned_addr learned my addr 172.21.15.119:0/4276151508 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): get_auth_request con 0x7ff32813c430 auth_method 0 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32d4a8640 1 -- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328131890 msgr2=0x7ff32813bef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32d4a8640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328131890 0x7ff32813bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32dca9640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3281cf3c0 0x7ff328131350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient(hunting): handle_auth_done global_id 34741 payload 824 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 -- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3281cf3c0 msgr2=0x7ff328131350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3281cf3c0 0x7ff328131350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 -- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328131890 msgr2=0x7ff32813bef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328131890 0x7ff32813bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 monclient: found mon.noname-b 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32e4aa640 1 -- 172.21.15.119:0/4276151508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff328131d10 con 0x7ff32813c430 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32dca9640 10 monclient: get_auth_request con 0x7ff3281cf3c0 auth_method 0 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.990+0000 7ff32dca9640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3281cf3c0 0x7ff328131350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32e4aa640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 0x7ff32813c8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff31c097c10 tx=0x7ff314005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c006de0 con 0x7ff32813c430 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: dump: 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.016 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 5 monclient: authenticate success, global_id 34741 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31c009500 con 0x7ff32813c430 2024-09-15T21:21:14.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 monclient: _renew_subs 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32811ed10 con 0x7ff32813c430 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c005730 con 0x7ff32813c430 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 10 monclient: dump: 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.018 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31bfff640 4 set_mon_vals no callback set 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 monclient: _renew_subs 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32811d9c0 con 0x7ff32813c430 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3280730f0 con 0x7ff32813c430 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff31c0ac420 con 0x7ff32813c430 2024-09-15T21:21:14.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff30c03b9a0 0x7ff30c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6083+0+0 (secure 0 0 0) 0x7ff31c0e9910 con 0x7ff32813c430 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32dca9640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff30c03b9a0 0x7ff30c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32dca9640 10 monclient: get_auth_request con 0x7ff30c03b9a0 auth_method 0 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.994+0000 7ff32dca9640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff30c03b9a0 0x7ff30c03de60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff3200014f0 tx=0x7ff320001260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.998+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff31c016130 con 0x7ff32813c430 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.998+0000 7ff31a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:13.998+0000 7ff31a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff32ff0d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff32ff0d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff32ff0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7ff32813d9b0 con 0x7ff32813c430 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v103) ==== 56+0+1173 (secure 0 0 0) 0x7ff31c0e7db0 con 0x7ff32813c430 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff31a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:14.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.154+0000 7ff31a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:14.254 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:14.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:14.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7ff32811d690 con 0x7ff32813c430 2024-09-15T21:21:14.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff31a7fc640 1 -- 172.21.15.119:0/4276151508 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v103) ==== 56+0+1173 (secure 0 0 0) 0x7ff31c0bfd60 con 0x7ff32813c430 2024-09-15T21:21:14.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff31a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:14.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff31a7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:14.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff30c03b9a0 msgr2=0x7ff30c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff30c03b9a0 0x7ff30c03de60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff3200014f0 tx=0x7ff320001260 comp rx=0 tx=0).stop 2024-09-15T21:21:14.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 10 monclient: shutdown 2024-09-15T21:21:14.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 msgr2=0x7ff32813c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 0x7ff32813c8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff31c097c10 tx=0x7ff314005840 comp rx=0 tx=0).stop 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 shutdown_connections 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff30c03b9a0 0x7ff30c03de60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff32813c430 0x7ff32813c8e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328131890 0x7ff32813bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 --2- 172.21.15.119:0/4276151508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3281cf3c0 0x7ff328131350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 >> 172.21.15.119:0/4276151508 conn(0x7ff32806cbf0 msgr2=0x7ff32809fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.250+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 shutdown_connections 2024-09-15T21:21:14.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.254+0000 7ff32ff0d640 1 -- 172.21.15.119:0/4276151508 wait complete. 2024-09-15T21:21:14.271 INFO:tasks.workunit.client.0.smithi119.stdout: 11 113.00000 osd.11 down 1.00000 1.00000 2024-09-15T21:21:14.272 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.11 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient: get_monmap_and_config 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient: monmap: 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:14.386055+0000 2024-09-15T21:21:14.387 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:14.386055+0000 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:14.388 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 Processor -- start 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 -- start start 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient: init 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe31c0a57e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe31c0a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): picked mon.noname-b con 0x7fe31c0a5bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe31c13ac00 0x7fe31c13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): picked mon.noname-a con 0x7fe31c13ac00 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31c06dc10 con 0x7fe31c0a57e0 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe31c06d940 con 0x7fe31c13ac00 2024-09-15T21:21:14.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe31c072810 con 0x7fe31c0a5bc0 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe3229a3640 10 monclient(hunting): authenticate will time out at 1193.183456s 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe31c0a57e0 0x7fe31c0736d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59340/0 (socket says 172.21.15.119:59340) 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31bfff640 1 -- 172.21.15.119:0/334168579 learned_addr learned my addr 172.21.15.119:0/334168579 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe320f40640 1 -- 172.21.15.119:0/334168579 >> v2:172.21.15.17:3300/0 conn(0x7fe31c13ac00 msgr2=0x7fe31c13cff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe320f40640 1 --2- 172.21.15.119:0/334168579 >> v2:172.21.15.17:3300/0 conn(0x7fe31c13ac00 0x7fe31c13cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31b7fe640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe31c0a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31b7fe640 10 monclient(hunting): get_auth_request con 0x7fe31c0a5bc0 auth_method 0 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909589426 0 0) 0x7fe31c06dc10 con 0x7fe31c0a57e0 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31affd640 10 monclient(hunting): _negotiate 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31affd640 10 monclient(hunting): my global_id is 34744 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.382+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe304003530 con 0x7fe31c0a57e0 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4027693107 0 0) 0x7fe304003530 con 0x7fe31c0a57e0 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 msgr2=0x7fe31c0a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe31c0a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 >> v2:172.21.15.17:3300/0 conn(0x7fe31c13ac00 msgr2=0x7fe31c13cff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 --2- 172.21.15.119:0/334168579 >> v2:172.21.15.17:3300/0 conn(0x7fe31c13ac00 0x7fe31c13cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c0a6520 con 0x7fe31c0a57e0 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fe30c001630 con 0x7fe31c0a57e0 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: dump: 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.396 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 >> v1:172.21.15.119:6789/0 conn(0x7fe31c0a57e0 legacy=0x7fe31c0736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe304003a10 0x7fe304003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): picked mon.b con 0x7fe304003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe304004810 0x7fe304004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): picked mon.a con 0x7fe304004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe30400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): picked mon.c con 0x7fe31c0a5bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c06d940 con 0x7fe304004810 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c072810 con 0x7fe304003a10 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe3040055c0 con 0x7fe31c0a5bc0 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:14.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe304003a10 0x7fe304003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31b7fe640 1 -- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe304004810 msgr2=0x7fe304004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): get_auth_request con 0x7fe304003a10 auth_method 0 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31b7fe640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe304004810 0x7fe304004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe320f40640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe30400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient(hunting): handle_auth_done global_id 34744 payload 293 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 -- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 msgr2=0x7fe30400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe30400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 -- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe304004810 msgr2=0x7fe304004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe304004810 0x7fe304004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 monclient: found mon.b 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 -- 172.21.15.119:0/334168579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe304003530 con 0x7fe304003a10 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe320f40640 10 monclient: get_auth_request con 0x7fe31c0a5bc0 auth_method 0 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 5 monclient: authenticate success, global_id 34744 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe320f40640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c0a5bc0 0x7fe30400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:14.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31bfff640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe304003a10 0x7fe304003e30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c160 tx=0x7fe30c01cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b61a623e23ed025 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c007cb0 con 0x7fe304003a10 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: dump: 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.401 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c0a9020 con 0x7fe304003a10 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 1 -- 172.21.15.119:0/334168579 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c005b20 con 0x7fe304003a10 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 4 set_mon_vals no callback set 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe31affd640 10 monclient: dump: 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.402 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.403 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 10 monclient: shutdown 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 1 -- 172.21.15.119:0/334168579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe304003a10 msgr2=0x7fe304003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 1 --2- 172.21.15.119:0/334168579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe304003a10 0x7fe304003e30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c160 tx=0x7fe30c01cae0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe320f40640 1 -- 172.21.15.119:0/334168579 reap_dead start 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 1 -- 172.21.15.119:0/334168579 shutdown_connections 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 1 -- 172.21.15.119:0/334168579 >> 172.21.15.119:0/334168579 conn(0x7fe31c0a3b80 msgr2=0x7fe31c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 1 -- 172.21.15.119:0/334168579 shutdown_connections 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.386+0000 7fe3229a3640 1 -- 172.21.15.119:0/334168579 wait complete. 2024-09-15T21:21:14.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: monmap: 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 Processor -- start 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- start start 2024-09-15T21:21:14.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: init 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c09e130 0x7fe31c09e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): picked mon.noname-a con 0x7fe31c09e130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c09ea50 0x7fe31c09eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): picked mon.noname-c con 0x7fe31c09ea50 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 0x7fe31c1d4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): picked mon.noname-b con 0x7fe31c1d2030 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c13dec0 con 0x7fe31c09e130 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c13dd40 con 0x7fe31c1d2030 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe31c13e040 con 0x7fe31c09ea50 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient(hunting): authenticate will time out at 1193.189058s 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31bfff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c09e130 msgr2=0x7fe31c09e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c09e130 0x7fe31c09e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 0x7fe31c1d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 0x7fe31c1d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41744/0 (socket says 172.21.15.119:41744) 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 -- 172.21.15.119:0/1865895056 learned_addr learned my addr 172.21.15.119:0/1865895056 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): get_auth_request con 0x7fe31c1d2030 auth_method 0 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31b7fe640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c09ea50 0x7fe31c09eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31b7fe640 10 monclient(hunting): get_auth_request con 0x7fe31c09ea50 auth_method 0 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient(hunting): handle_auth_done global_id 34747 payload 824 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 -- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c09ea50 msgr2=0x7fe31c09eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c09ea50 0x7fe31c09eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 -- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c09e130 msgr2=0x7fe31c09e510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c09e130 0x7fe31c09e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 monclient: found mon.noname-b 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 -- 172.21.15.119:0/1865895056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c0744c0 con 0x7fe31c1d2030 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe320f40640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 0x7fe31c1d4420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe31c1321d0 tx=0x7fe31400cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314014020 con 0x7fe31c1d2030 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: dump: 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.413 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe31400aba0 con 0x7fe31c1d2030 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314010480 con 0x7fe31c1d2030 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 5 monclient: authenticate success, global_id 34747 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 10 monclient: dump: 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31a7fc640 4 set_mon_vals no callback set 2024-09-15T21:21:14.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _renew_subs 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c11dc40 con 0x7fe31c1d2030 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _renew_subs 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c0a0750 con 0x7fe31c1d2030 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe314017020 con 0x7fe31c1d2030 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31c136ce0 con 0x7fe31c1d2030 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30803b970 0x7fe30803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6083+0+0 (secure 0 0 0) 0x7fe314050340 con 0x7fe31c1d2030 2024-09-15T21:21:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31bfff640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30803b970 0x7fe30803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31bfff640 10 monclient: get_auth_request con 0x7fe30803b970 auth_method 0 2024-09-15T21:21:14.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.390+0000 7fe31bfff640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30803b970 0x7fe30803de30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0aa000 tx=0x7fe30c0019a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.394+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe31401e030 con 0x7fe31c1d2030 2024-09-15T21:21:14.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.394+0000 7fe318ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.394+0000 7fe318ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:14.560 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:21:14.560 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:21:14.560 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:21:14.563 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:21:14.563 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:21:14.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:14.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.570+0000 7fe3229a3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T21:21:14.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.570+0000 7fe3229a3640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T21:21:14.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.570+0000 7fe3229a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.570+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) -- 0x7fe31c0736d0 con 0x7fe31c1d2030 2024-09-15T21:21:14.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.642+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 removed item id 11 name 'osd.11' from crush map v104) ==== 125+0+0 (secure 0 0 0) 0x7fe31c0744c0 con 0x7fe31c1d2030 2024-09-15T21:21:14.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.642+0000 7fe318ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T21:21:14.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.642+0000 7fe318ff9640 10 monclient: _finish_command 2 = system:0 removed item id 11 name 'osd.11' from crush map 2024-09-15T21:21:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient: get_monmap_and_config 2024-09-15T21:21:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient: monmap: 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:14.690903+0000 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:14.690903+0000 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:14.693 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:14.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 Processor -- start 2024-09-15T21:21:14.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 -- start start 2024-09-15T21:21:14.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient: init 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6601378c0 0x7fa660137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): picked mon.noname-c con 0x7fa6601378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 0x7fa6600a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): picked mon.noname-b con 0x7fa6601381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 0x7fa6600a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): picked mon.noname-a con 0x7fa6600a5040 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa6600a1d00 con 0x7fa6600a5040 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa6600a1fe0 con 0x7fa6601378c0 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa6600a22c0 con 0x7fa6601381e0 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa6662d3640 10 monclient(hunting): authenticate will time out at 1198.086035s 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 0x7fa6600a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 0x7fa6600a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 0x7fa6600a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59238/0 (socket says 172.21.15.17:59238) 2024-09-15T21:21:14.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 -- 172.21.15.17:0/3131390474 learned_addr learned my addr 172.21.15.17:0/3131390474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): get_auth_request con 0x7fa6601381e0 auth_method 0 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 10 monclient(hunting): get_auth_request con 0x7fa6600a5040 auth_method 0 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 1 -- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 msgr2=0x7fa6600a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 1 -- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 msgr2=0x7fa6600a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:14.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 0x7fa6600a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa664870640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 0x7fa6600a54f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65ffff640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.119:3300/0 conn(0x7fa6601378c0 0x7fa660137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65ffff640 10 monclient(hunting): get_auth_request con 0x7fa6601378c0 auth_method 0 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_done global_id 44252 payload 274 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 -- 172.21.15.17:0/3131390474 >> v2:172.21.15.119:3300/0 conn(0x7fa6601378c0 msgr2=0x7fa660137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.119:3300/0 conn(0x7fa6601378c0 0x7fa660137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 -- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 msgr2=0x7fa6600a54f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 0x7fa6600a54f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:14.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:14.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.689+0000 7fa65f7fe640 1 -- 172.21.15.17:0/3131390474 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6600a4490 con 0x7fa6601381e0 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/3131390474 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 0x7fa6600a4b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6600ab150 tx=0x7fa65401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc4764ab75608b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 1 -- 172.21.15.17:0/3131390474 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540b1070 con 0x7fa6601381e0 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: dump: 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 5 monclient: authenticate success, global_id 44252 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 1 -- 172.21.15.17:0/3131390474 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6540ac620 con 0x7fa6601381e0 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 4 set_mon_vals no callback set 2024-09-15T21:21:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 1 -- 172.21.15.17:0/3131390474 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540ac9f0 con 0x7fa6601381e0 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65effd640 10 monclient: dump: 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: shutdown 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- 172.21.15.17:0/3131390474 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 msgr2=0x7fa6600a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- 172.21.15.17:0/3131390474 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 0x7fa6600a4b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6600ab150 tx=0x7fa65401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- 172.21.15.17:0/3131390474 shutdown_connections 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.17:3300/0 conn(0x7fa6600a5040 0x7fa6600a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- 172.21.15.17:0/3131390474 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601381e0 0x7fa6600a4b00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- 172.21.15.17:0/3131390474 >> v2:172.21.15.119:3300/0 conn(0x7fa6601378c0 0x7fa660137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- 172.21.15.17:0/3131390474 >> 172.21.15.17:0/3131390474 conn(0x7fa6600aac40 msgr2=0x7fa6600a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- 172.21.15.17:0/3131390474 shutdown_connections 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- 172.21.15.17:0/3131390474 wait complete. 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: monmap: 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 Processor -- start 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- start start 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: init 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): picked mon.noname-a con 0x7fa6600a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 0x7fa66013ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): picked mon.noname-c con 0x7fa6601378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6601381e0 0x7fa66013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): picked mon.noname-b con 0x7fa6601381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa660141730 con 0x7fa6600a5040 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601415b0 con 0x7fa6601381e0 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa6601418b0 con 0x7fa6601378c0 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa6662d3640 10 monclient(hunting): authenticate will time out at 1198.091237s 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 0x7fa66013ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 0x7fa66013ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:59240/0 (socket says 172.21.15.17:59240) 2024-09-15T21:21:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 -- 172.21.15.17:0/1484228997 learned_addr learned my addr 172.21.15.17:0/1484228997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35452/0 (socket says 172.21.15.17:35452) 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): get_auth_request con 0x7fa6601378c0 auth_method 0 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 10 monclient(hunting): get_auth_request con 0x7fa6600a5040 auth_method 0 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 1 -- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 msgr2=0x7fa660136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 1 -- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 msgr2=0x7fa660136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65ffff640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa664870640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6601381e0 0x7fa66013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient(hunting): handle_auth_done global_id 44255 payload 824 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 -- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6601381e0 msgr2=0x7fa66013d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6601381e0 0x7fa66013d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 -- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 msgr2=0x7fa660136c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:14.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 -- 172.21.15.17:0/1484228997 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66006e9e0 con 0x7fa6601378c0 2024-09-15T21:21:14.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa664870640 10 monclient: get_auth_request con 0x7fa6601381e0 auth_method 0 2024-09-15T21:21:14.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa664870640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6601381e0 0x7fa66013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:14.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.693+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 0x7fa66013ce10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa65401aa90 tx=0x7fa654019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540bb020 con 0x7fa6601378c0 2024-09-15T21:21:14.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: dump: 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.719 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6540b7420 con 0x7fa6601378c0 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 5 monclient: authenticate success, global_id 44255 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 monclient: _renew_subs 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6600a12f0 con 0x7fa6601378c0 2024-09-15T21:21:14.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540b1040 con 0x7fa6601378c0 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 10 monclient: dump: 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 monclient: _renew_subs 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65e7fc640 4 set_mon_vals no callback set 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6601225f0 con 0x7fa6601378c0 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa660137ca0 con 0x7fa6601378c0 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa6540be020 con 0x7fa6601378c0 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa64c03b9c0 0x7fa64c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 6032+0+0 (secure 0 0 0) 0x7fa6540ea450 con 0x7fa6601378c0 2024-09-15T21:21:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65ffff640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa64c03b9c0 0x7fa64c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65ffff640 10 monclient: get_auth_request con 0x7fa64c03b9c0 auth_method 0 2024-09-15T21:21:14.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.697+0000 7fa65ffff640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa64c03b9c0 0x7fa64c03de80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa650006a70 tx=0x7fa650008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.701+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa6540c5030 con 0x7fa6601378c0 2024-09-15T21:21:14.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.701+0000 7fa65cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.701+0000 7fa65cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:14.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe3229a3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T21:21:14.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe3229a3640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T21:21:14.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe3229a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) -- 0x7fe31c13c6f0 con 0x7fe31c1d2030 2024-09-15T21:21:14.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe318ff9640 1 -- 172.21.15.119:0/1865895056 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 device 'osd.11' does not appear in the crush map v104) ==== 126+0+0 (secure 0 0 0) 0x7fe31400ec60 con 0x7fe31c1d2030 2024-09-15T21:21:14.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe318ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T21:21:14.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.750+0000 7fe318ff9640 10 monclient: _finish_command 3 = system:0 device 'osd.11' does not appear in the crush map 2024-09-15T21:21:14.758 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.11' does not appear in the crush map 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30803b970 msgr2=0x7fe30803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30803b970 0x7fe30803de30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0aa000 tx=0x7fe30c0019a0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 10 monclient: shutdown 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 msgr2=0x7fe31c1d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 0x7fe31c1d4420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe31c1321d0 tx=0x7fe31400cd10 comp rx=0 tx=0).stop 2024-09-15T21:21:14.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 shutdown_connections 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe30803b970 0x7fe30803de30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe31c1d2030 0x7fe31c1d4420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe31c09ea50 0x7fe31c09eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 --2- 172.21.15.119:0/1865895056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c09e130 0x7fe31c09e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 >> 172.21.15.119:0/1865895056 conn(0x7fe31c0a3b80 msgr2=0x7fe31c135ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 shutdown_connections 2024-09-15T21:21:14.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.754+0000 7fe3229a3640 1 -- 172.21.15.119:0/1865895056 wait complete. 2024-09-15T21:21:14.775 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm osd.11 2024-09-15T21:21:14.877 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.870+0000 7fb0f2627640 10 monclient: get_monmap_and_config 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.870+0000 7fb0f2627640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.870+0000 7fb0f2627640 10 monclient: monmap: 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:14.876611+0000 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:14.876611+0000 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.878 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:14.879 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:14.879 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:14.879 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:14.879 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:14.879 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 Processor -- start 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 -- start start 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient: init 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 0x7fb0ec09d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): picked mon.noname-b con 0x7fb0ec09d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 0x7fb0ec09de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): picked mon.noname-a con 0x7fb0ec09f960 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb0ec1347b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ec09ef80 con 0x7fb0ec1347b0 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ec06da70 con 0x7fb0ec09f960 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ec0746f0 con 0x7fb0ec09d4e0 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 10 monclient(hunting): authenticate will time out at 1193.673480s 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb0ec1347b0 0x7fb0ec134be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59342/0 (socket says 172.21.15.119:59342) 2024-09-15T21:21:14.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 1 -- 172.21.15.119:0/1888225986 learned_addr learned my addr 172.21.15.119:0/1888225986 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 0x7fb0ec09de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 0x7fb0ec09d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): get_auth_request con 0x7fb0ec09f960 auth_method 0 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request con 0x7fb0ec09d4e0 auth_method 0 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827317386 0 0) 0x7fb0ec09ef80 con 0x7fb0ec1347b0 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): _negotiate 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): my global_id is 34753 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0d4003530 con 0x7fb0ec1347b0 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 -- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 msgr2=0x7fb0ec09de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 -- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 msgr2=0x7fb0ec09de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:14.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 0x7fb0ec09de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 0x7fb0ec09de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2138882823 0 0) 0x7fb0d4003530 con 0x7fb0ec1347b0 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 msgr2=0x7fb0ec09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 0x7fb0ec09d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 msgr2=0x7fb0ec09de00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 --2- 172.21.15.119:0/1888225986 >> v2:172.21.15.17:3300/0 conn(0x7fb0ec09f960 0x7fb0ec09de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:14.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec09e6e0 con 0x7fb0ec1347b0 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb0e4001a80 con 0x7fb0ec1347b0 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: dump: 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 >> v1:172.21.15.119:6789/0 conn(0x7fb0ec1347b0 legacy=0x7fb0ec134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d4003a60 0x7fb0d4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): picked mon.a con 0x7fb0d4003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): picked mon.b con 0x7fb0d4004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 0x7fb0d4005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): picked mon.c con 0x7fb0ec09d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec0746f0 con 0x7fb0d4003a60 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d400dda0 con 0x7fb0d4004810 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d400dfa0 con 0x7fb0ec09d4e0 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eaffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): get_auth_request con 0x7fb0d4004810 auth_method 0 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d4003a60 0x7fb0d4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 0x7fb0d4005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request con 0x7fb0d4003a60 auth_method 0 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0ebfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 10 monclient(hunting): get_auth_request con 0x7fb0ec09d4e0 auth_method 0 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f0bc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient(hunting): handle_auth_done global_id 34753 payload 293 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 -- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 msgr2=0x7fb0d4005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec09d4e0 0x7fb0d4005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 -- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d4003a60 msgr2=0x7fb0d4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d4003a60 0x7fb0d4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 monclient: found mon.b 2024-09-15T21:21:14.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 1 -- 172.21.15.119:0/1888225986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d4003530 con 0x7fb0d4004810 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 5 monclient: authenticate success, global_id 34753 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.874+0000 7fb0f2627640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c820 tx=0x7fb0e001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ba66ca6c7ecf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00b0070 con 0x7fb0d4004810 2024-09-15T21:21:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: dump: 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e00ab620 con 0x7fb0d4004810 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 1 -- 172.21.15.119:0/1888225986 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000f040 con 0x7fb0d4004810 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 4 set_mon_vals no callback set 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eaffd640 10 monclient: dump: 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.904 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: shutdown 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/1888225986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 msgr2=0x7fb0d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 --2- 172.21.15.119:0/1888225986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c820 tx=0x7fb0e001c510 comp rx=0 tx=0).stop 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f0bc4640 1 -- 172.21.15.119:0/1888225986 reap_dead start 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/1888225986 shutdown_connections 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/1888225986 >> 172.21.15.119:0/1888225986 conn(0x7fb0ec06cdc0 msgr2=0x7fb0ec06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/1888225986 shutdown_connections 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/1888225986 wait complete. 2024-09-15T21:21:14.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: build_initial_monmap 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: monmap: 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.906 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 Processor -- start 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- start start 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: init 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:14.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0ec1cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): picked mon.noname-b con 0x7fb0d4004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0ec1cc190 0x7fb0ec1211b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): picked mon.noname-a con 0x7fb0ec1cc190 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec1cc610 0x7fb0ec1216f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): picked mon.noname-c con 0x7fb0ec1cc610 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec0a2760 con 0x7fb0ec1cc190 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec0a25e0 con 0x7fb0d4004810 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:14.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ec0a1a80 con 0x7fb0ec1cc610 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient(hunting): authenticate will time out at 1193.679123s 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0ec1cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0ec1cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41752/0 (socket says 172.21.15.119:41752) 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 -- 172.21.15.119:0/513950917 learned_addr learned my addr 172.21.15.119:0/513950917 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request con 0x7fb0d4004810 auth_method 0 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:14.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0ec1cc190 0x7fb0ec1211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eb7fe640 10 monclient(hunting): get_auth_request con 0x7fb0ec1cc190 auth_method 0 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f0bc4640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec1cc610 0x7fb0ec1216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_done global_id 34756 payload 824 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 -- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec1cc610 msgr2=0x7fb0ec1216f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec1cc610 0x7fb0ec1216f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 -- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0ec1cc190 msgr2=0x7fb0ec1211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0ec1cc190 0x7fb0ec1211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:14.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 monclient: found mon.noname-b 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 -- 172.21.15.119:0/513950917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec124fd0 con 0x7fb0d4004810 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f0bc4640 10 monclient: get_auth_request con 0x7fb0ec1cc610 auth_method 0 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f0bc4640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec1cc610 0x7fb0ec1216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ebfff640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0ec1cbc50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0115e0 tx=0x7fb0dc011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00f8f0 con 0x7fb0d4004810 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: dump: 2024-09-15T21:21:14.912 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:14.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0dc00fa90 con 0x7fb0d4004810 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 5 monclient: authenticate success, global_id 34756 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: _renew_subs 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec06d3d0 con 0x7fb0d4004810 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00fe40 con 0x7fb0d4004810 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0e8ff9640 10 monclient: dump: 2024-09-15T21:21:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: _renew_subs 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec134590 con 0x7fb0d4004810 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.878+0000 7fb0ea7fc640 4 set_mon_vals no callback set 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0f2627640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0f2627640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0f2627640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ec0a1e70 con 0x7fb0d4004810 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb0dc015460 con 0x7fb0d4004810 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0e8ff9640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0d803b9c0 0x7fb0d803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:14.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6032+0+0 (secure 0 0 0) 0x7fb0dc050840 con 0x7fb0d4004810 2024-09-15T21:21:14.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0d803b9c0 0x7fb0d803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:14.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0eb7fe640 10 monclient: get_auth_request con 0x7fb0d803b9c0 auth_method 0 2024-09-15T21:21:14.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.882+0000 7fb0eb7fe640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0d803b9c0 0x7fb0d803de80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00ad000 tx=0x7fb0e00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:14.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.886+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb0dc019030 con 0x7fb0d4004810 2024-09-15T21:21:14.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.886+0000 7fb0e8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:14.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:14.886+0000 7fb0e8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.933+0000 7fa6662d3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.933+0000 7fa6662d3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.933+0000 7fa6662d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.933+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa660122110 con 0x7fa6601378c0 2024-09-15T21:21:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa65cff9640 1 -- 172.21.15.17:0/1484228997 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fa6540160f0 con 0x7fa6601378c0 2024-09-15T21:21:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa65cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:14.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa65cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:14.941 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa64c03b9c0 msgr2=0x7fa64c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa64c03b9c0 0x7fa64c03de80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa650006a70 tx=0x7fa650008040 comp rx=0 tx=0).stop 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 10 monclient: shutdown 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 msgr2=0x7fa66013ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 0x7fa66013ce10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa65401aa90 tx=0x7fa654019180 comp rx=0 tx=0).stop 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 shutdown_connections 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa64c03b9c0 0x7fa64c03de80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6601381e0 0x7fa66013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6601378c0 0x7fa66013ce10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 --2- 172.21.15.17:0/1484228997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6600a5040 0x7fa660136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:14.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 >> 172.21.15.17:0/1484228997 conn(0x7fa6600aac40 msgr2=0x7fa6600a8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:14.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 shutdown_connections 2024-09-15T21:21:14.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:14.937+0000 7fa6662d3640 1 -- 172.21.15.17:0/1484228997 wait complete. 2024-09-15T21:21:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.050+0000 7fb0f2627640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T21:21:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.050+0000 7fb0f2627640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T21:21:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.050+0000 7fb0f2627640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.050+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) -- 0x7fb0ec1237d0 con 0x7fb0d4004810 2024-09-15T21:21:15.057 INFO:tasks.ceph.mon.b.smithi119.stderr:2024-09-15T21:21:15.050+0000 7fb9064e7640 -1 mon.b@1(leader).osd e104 definitely_dead 0 2024-09-15T21:21:15.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.650+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 removed osd.11 v105) ==== 87+0+0 (secure 0 0 0) 0x7fb0dc024c60 con 0x7fb0d4004810 2024-09-15T21:21:15.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.650+0000 7fb0e8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T21:21:15.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.650+0000 7fb0e8ff9640 10 monclient: _finish_command 2 = system:0 removed osd.11 2024-09-15T21:21:15.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.762+0000 7fb0f2627640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T21:21:15.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.762+0000 7fb0f2627640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T21:21:15.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.762+0000 7fb0f2627640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.762+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) -- 0x7fb0ec0008e0 con 0x7fb0d4004810 2024-09-15T21:21:15.771 INFO:tasks.ceph.mon.b.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb9064e7640 -1 mon.b@1(leader).osd e105 definitely_dead 0 2024-09-15T21:21:15.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0e8ff9640 1 -- 172.21.15.119:0/513950917 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 osd.11 does not exist. v105) ==== 96+0+0 (secure 0 0 0) 0x7fb0dc01bcd0 con 0x7fb0d4004810 2024-09-15T21:21:15.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0e8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T21:21:15.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0e8ff9640 10 monclient: _finish_command 3 = system:0 osd.11 does not exist. 2024-09-15T21:21:15.771 INFO:tasks.workunit.client.0.smithi119.stderr:osd.11 does not exist. 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0d803b9c0 msgr2=0x7fb0d803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0d803b9c0 0x7fb0d803de80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00ad000 tx=0x7fb0e00046d0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 10 monclient: shutdown 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 msgr2=0x7fb0ec1cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0ec1cbc50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0115e0 tx=0x7fb0dc011aa0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 shutdown_connections 2024-09-15T21:21:15.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0d803b9c0 0x7fb0d803de80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0ec1cc610 0x7fb0ec1216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0ec1cc190 0x7fb0ec1211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 --2- 172.21.15.119:0/513950917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d4004810 0x7fb0ec1cbc50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 >> 172.21.15.119:0/513950917 conn(0x7fb0ec06cdc0 msgr2=0x7fb0ec0a5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:15.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 shutdown_connections 2024-09-15T21:21:15.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.766+0000 7fb0f2627640 1 -- 172.21.15.119:0/513950917 wait complete. 2024-09-15T21:21:15.787 INFO:tasks.workunit.client.0.smithi119.stderr:++ ceph osd create 2024-09-15T21:21:15.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.894+0000 7f4d95be6640 10 monclient: get_monmap_and_config 2024-09-15T21:21:15.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.894+0000 7f4d95be6640 10 monclient: build_initial_monmap 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.894+0000 7f4d95be6640 10 monclient: monmap: 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:15.899414+0000 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:15.899414+0000 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:15.901 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:15.902 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.894+0000 7f4d95be6640 1 Processor -- start 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 1 -- start start 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient: init 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d900a4d80 0x7f4d900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): picked mon.noname-c con 0x7f4d900a4d80 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d900a56a0 0x7f4d90137b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): picked mon.noname-a con 0x7f4d900a56a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4d90138080 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:15.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d900a26e0 con 0x7f4d90138080 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4d900a6330 con 0x7f4d900a56a0 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4d900a30a0 con 0x7f4d900a4d80 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient(hunting): authenticate will time out at 1194.696829s 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d900a4d80 0x7f4d900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d900a4d80 0x7f4d900a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41754/0 (socket says 172.21.15.119:41754) 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 -- 172.21.15.119:0/2216987967 learned_addr learned my addr 172.21.15.119:0/2216987967 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): get_auth_request con 0x7f4d900a4d80 auth_method 0 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:15.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8effd640 1 --2- 172.21.15.119:0/2216987967 >> v2:172.21.15.17:3300/0 conn(0x7f4d900a56a0 0x7f4d90137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8effd640 10 monclient(hunting): get_auth_request con 0x7f4d900a56a0 auth_method 0 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient(hunting): handle_auth_done global_id 34759 payload 274 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 -- 172.21.15.119:0/2216987967 >> v2:172.21.15.17:3300/0 conn(0x7f4d900a56a0 msgr2=0x7f4d90137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 --2- 172.21.15.119:0/2216987967 >> v2:172.21.15.17:3300/0 conn(0x7f4d900a56a0 0x7f4d90137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 -- 172.21.15.119:0/2216987967 >> v1:172.21.15.17:6789/0 conn(0x7f4d90138080 legacy=0x7f4d9013efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:15.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:15.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:21:15.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:15.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 -- 172.21.15.119:0/2216987967 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d900a6a30 con 0x7f4d900a4d80 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8f7fe640 1 --2- 172.21.15.119:0/2216987967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a4d80 0x7f4d900a5160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4d900a1440 tx=0x7f4d8001c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9cdfc24dd954199 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 1 -- 172.21.15.119:0/2216987967 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800b0070 con 0x7f4d900a4d80 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: dump: 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 1 -- 172.21.15.119:0/2216987967 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d800ab600 con 0x7f4d900a4d80 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 5 monclient: authenticate success, global_id 34759 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:15.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 1 -- 172.21.15.119:0/2216987967 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800ab9d0 con 0x7f4d900a4d80 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d95be6640 4 set_mon_vals no callback set 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.898+0000 7f4d8e7fc640 10 monclient: dump: 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: shutdown 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/2216987967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a4d80 msgr2=0x7f4d900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 --2- 172.21.15.119:0/2216987967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a4d80 0x7f4d900a5160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4d900a1440 tx=0x7f4d8001c3b0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/2216987967 shutdown_connections 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 --2- 172.21.15.119:0/2216987967 >> v2:172.21.15.17:3300/0 conn(0x7f4d900a56a0 0x7f4d90137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 --2- 172.21.15.119:0/2216987967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a4d80 0x7f4d900a5160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/2216987967 >> 172.21.15.119:0/2216987967 conn(0x7f4d900a7a00 msgr2=0x7f4d9006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/2216987967 shutdown_connections 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/2216987967 wait complete. 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: build_initial_monmap 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: monmap: 2024-09-15T21:21:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:15.916 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 Processor -- start 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- start start 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: init 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900a4d80 0x7f4d90133d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): picked mon.noname-a con 0x7f4d900a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 0x7f4d901342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): picked mon.noname-b con 0x7f4d900a56a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4d90138080 0x7f4d90139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): picked mon.noname-c con 0x7f4d90138080 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:15.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4d900a5ab0 con 0x7f4d900a4d80 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d900a1940 con 0x7f4d900a56a0 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9006d560 con 0x7f4d90138080 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient(hunting): authenticate will time out at 1194.702276s 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 0x7f4d901342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 0x7f4d901342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41760/0 (socket says 172.21.15.119:41760) 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 -- 172.21.15.119:0/1805455495 learned_addr learned my addr 172.21.15.119:0/1805455495 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): get_auth_request con 0x7f4d900a56a0 auth_method 0 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:15.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8ffff640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4d90138080 0x7f4d90139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8f7fe640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900a4d80 0x7f4d90133d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient(hunting): handle_auth_done global_id 34762 payload 824 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 -- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4d90138080 msgr2=0x7f4d90139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4d90138080 0x7f4d90139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 -- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900a4d80 msgr2=0x7f4d90133d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900a4d80 0x7f4d90133d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:15.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:15.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 monclient: found mon.noname-b 2024-09-15T21:21:15.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:15.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 -- 172.21.15.119:0/1805455495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d901401e0 con 0x7f4d900a56a0 2024-09-15T21:21:15.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8effd640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 0x7f4d901342a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400d830 tx=0x7f4d8400dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:15.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8400b6c0 con 0x7f4d900a56a0 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: dump: 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:15.921 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 5 monclient: authenticate success, global_id 34762 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: _renew_subs 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84019070 con 0x7f4d900a56a0 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d900a1570 con 0x7f4d900a56a0 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:15.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8400bae0 con 0x7f4d900a56a0 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d77fff640 10 monclient: dump: 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:15.923 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: _renew_subs 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9013da40 con 0x7f4d900a56a0 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.902+0000 7f4d8dffb640 4 set_mon_vals no callback set 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d95be6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d95be6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d95be6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d900a5f60 con 0x7f4d900a56a0 2024-09-15T21:21:15.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4d8401e460 con 0x7f4d900a56a0 2024-09-15T21:21:15.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d77fff640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4d7c03b9c0 0x7f4d7c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:15.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6080+0+0 (secure 0 0 0) 0x7f4d84050730 con 0x7f4d900a56a0 2024-09-15T21:21:15.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d8f7fe640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4d7c03b9c0 0x7f4d7c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:15.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d8f7fe640 10 monclient: get_auth_request con 0x7f4d7c03b9c0 auth_method 0 2024-09-15T21:21:15.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.906+0000 7f4d8f7fe640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4d7c03b9c0 0x7f4d7c03de80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d900a1440 tx=0x7f4d80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:15.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.910+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4d8401da00 con 0x7f4d900a56a0 2024-09-15T21:21:15.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.910+0000 7f4d77fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:15.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:15.910+0000 7f4d77fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.078+0000 7f4d95be6640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:21:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.078+0000 7f4d95be6640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T21:21:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.078+0000 7f4d95be6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.078+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f4d9013ff30 con 0x7f4d900a56a0 2024-09-15T21:21:16.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.658+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v106) ==== 58+0+3 (secure 0 0 0) 0x7f4d84014430 con 0x7f4d900a56a0 2024-09-15T21:21:16.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.658+0000 7f4d77fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T21:21:16.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.658+0000 7f4d77fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:16.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.766+0000 7f4d95be6640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:21:16.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.766+0000 7f4d95be6640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T21:21:16.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.766+0000 7f4d95be6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:16.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:16.766+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f4d9013c240 con 0x7f4d900a56a0 2024-09-15T21:21:17.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.522+0000 7f4d77fff640 1 -- 172.21.15.119:0/1805455495 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v107) ==== 58+0+3 (secure 0 0 0) 0x7f4d8401dcc0 con 0x7f4d900a56a0 2024-09-15T21:21:17.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.522+0000 7f4d77fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T21:21:17.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.522+0000 7f4d77fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:17.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4d7c03b9c0 msgr2=0x7f4d7c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:17.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4d7c03b9c0 0x7f4d7c03de80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d900a1440 tx=0x7f4d80008040 comp rx=0 tx=0).stop 2024-09-15T21:21:17.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 10 monclient: shutdown 2024-09-15T21:21:17.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:17.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 msgr2=0x7f4d901342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:17.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 0x7f4d901342a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400d830 tx=0x7f4d8400dcf0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 shutdown_connections 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4d7c03b9c0 0x7f4d7c03de80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4d90138080 0x7f4d90139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d900a56a0 0x7f4d901342a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 --2- 172.21.15.119:0/1805455495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900a4d80 0x7f4d90133d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 >> 172.21.15.119:0/1805455495 conn(0x7f4d900a7a00 msgr2=0x7f4d9006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 shutdown_connections 2024-09-15T21:21:17.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.526+0000 7f4d95be6640 1 -- 172.21.15.119:0/1805455495 wait complete. 2024-09-15T21:21:17.551 INFO:tasks.workunit.client.0.smithi119.stderr:+ o4=12 2024-09-15T21:21:17.551 INFO:tasks.workunit.client.0.smithi119.stderr:++ ceph osd create 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.658+0000 7fefe74be640 10 monclient: get_monmap_and_config 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.658+0000 7fefe74be640 10 monclient: build_initial_monmap 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.658+0000 7fefe74be640 10 monclient: monmap: 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:17.663759+0000 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:17.663759+0000 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:17.665 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:17.666 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:17.666 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:17.666 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:17.666 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:17.666 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 Processor -- start 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 -- start start 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient: init 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fefe00695a0 0x7fefe01349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:17.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): picked mon.noname-c con 0x7fefe00695a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe0072e70 0x7fefe00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): picked mon.noname-b con 0x7fefe0072e70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fefe00a56d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe0070260 con 0x7fefe00a56d0 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fefe0069980 con 0x7fefe00695a0 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefe00707e0 con 0x7fefe0072e70 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe74be640 10 monclient(hunting): authenticate will time out at 1196.461280s 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fefe00695a0 0x7fefe01349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fefe00695a0 0x7fefe01349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41762/0 (socket says 172.21.15.119:41762) 2024-09-15T21:21:17.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 -- 172.21.15.119:0/1940347781 learned_addr learned my addr 172.21.15.119:0/1940347781 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): get_auth_request con 0x7fefe00695a0 auth_method 0 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe4a59640 1 --2- 172.21.15.119:0/1940347781 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe0072e70 0x7fefe00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe4a59640 10 monclient(hunting): get_auth_request con 0x7fefe0072e70 auth_method 0 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe4a59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe4a59640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe4a59640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient(hunting): handle_auth_done global_id 34765 payload 274 2024-09-15T21:21:17.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 -- 172.21.15.119:0/1940347781 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe0072e70 msgr2=0x7fefe00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 --2- 172.21.15.119:0/1940347781 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe0072e70 0x7fefe00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 -- 172.21.15.119:0/1940347781 >> v1:172.21.15.119:6789/0 conn(0x7fefe00a56d0 legacy=0x7fefe00a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 monclient: found mon.noname-c 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:17.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 -- 172.21.15.119:0/1940347781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe006dfa0 con 0x7fefe00695a0 2024-09-15T21:21:17.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefe525a640 1 --2- 172.21.15.119:0/1940347781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe01349d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009870 tx=0x7fefdc01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475167c445a29a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefd7fff640 1 -- 172.21.15.119:0/1940347781 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0b1070 con 0x7fefe00695a0 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefd7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.662+0000 7fefd7fff640 10 monclient: dump: 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:17.677 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 1 -- 172.21.15.119:0/1940347781 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefdc0ac640 con 0x7fefe00695a0 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 1 -- 172.21.15.119:0/1940347781 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0aca10 con 0x7fefe00695a0 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 5 monclient: authenticate success, global_id 34765 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 4 set_mon_vals no callback set 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:17.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd7fff640 10 monclient: dump: 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: shutdown 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/1940347781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 msgr2=0x7fefe01349d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:17.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 --2- 172.21.15.119:0/1940347781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe01349d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009870 tx=0x7fefdc01c420 comp rx=0 tx=0).stop 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/1940347781 shutdown_connections 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 --2- 172.21.15.119:0/1940347781 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe0072e70 0x7fefe00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 --2- 172.21.15.119:0/1940347781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe01349d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/1940347781 >> 172.21.15.119:0/1940347781 conn(0x7fefe006cbf0 msgr2=0x7fefe006cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/1940347781 shutdown_connections 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/1940347781 wait complete. 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: build_initial_monmap 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: monmap: 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:17.680 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 Processor -- start 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- start start 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: init 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:17.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe013f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): picked mon.noname-b con 0x7fefe00695a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0072e70 0x7fefe013f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): picked mon.noname-a con 0x7fefe0072e70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe00a56d0 0x7fefe013faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): picked mon.noname-c con 0x7fefe00a56d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006f090 con 0x7fefe0072e70 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006ef10 con 0x7fefe00695a0 2024-09-15T21:21:17.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefe006f210 con 0x7fefe00a56d0 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient(hunting): authenticate will time out at 1196.466600s 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe013f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe013f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41764/0 (socket says 172.21.15.119:41764) 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 -- 172.21.15.119:0/3552900596 learned_addr learned my addr 172.21.15.119:0/3552900596 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): get_auth_request con 0x7fefe00695a0 auth_method 0 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:17.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe5a5b640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe00a56d0 0x7fefe013faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe4a59640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0072e70 0x7fefe013f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe5a5b640 10 monclient(hunting): get_auth_request con 0x7fefe00a56d0 auth_method 0 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe5a5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe5a5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe5a5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient(hunting): handle_auth_done global_id 34768 payload 824 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 -- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe00a56d0 msgr2=0x7fefe013faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe00a56d0 0x7fefe013faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 -- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0072e70 msgr2=0x7fefe013f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0072e70 0x7fefe013f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:17.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 monclient: found mon.noname-b 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 -- 172.21.15.119:0/3552900596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe0073e10 con 0x7fefe00695a0 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe4a59640 10 monclient: get_auth_request con 0x7fefe0072e70 auth_method 0 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe4a59640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0072e70 0x7fefe013f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe525a640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe013f070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fefdc01aad0 tx=0x7fefdc0191c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0b1040 con 0x7fefe00695a0 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: dump: 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefdc0ac440 con 0x7fefe00695a0 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0b5680 con 0x7fefe00695a0 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd5ffb640 10 monclient: dump: 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 5 monclient: authenticate success, global_id 34768 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: _renew_subs 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefd77fe640 4 set_mon_vals no callback set 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe013b440 con 0x7fefe00695a0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: _renew_subs 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.666+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe013b160 con 0x7fefe00695a0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe74be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe74be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe74be640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe01349d0 con 0x7fefe00695a0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fefdc0bc020 con 0x7fefe00695a0 2024-09-15T21:21:17.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefd5ffb640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:17.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6199+0+0 (secure 0 0 0) 0x7fefdc0e9590 con 0x7fefe00695a0 2024-09-15T21:21:17.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe4a59640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:17.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe4a59640 10 monclient: get_auth_request con 0x7fefc403b9c0 auth_method 0 2024-09-15T21:21:17.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.670+0000 7fefe4a59640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fefe006fe10 tx=0x7fefd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:17.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.674+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fefdc0c3020 con 0x7fefe00695a0 2024-09-15T21:21:17.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.674+0000 7fefd5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:17.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.674+0000 7fefd5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:17.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.854+0000 7fefe74be640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:21:17.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.854+0000 7fefe74be640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T21:21:17.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.854+0000 7fefe74be640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:17.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:17.854+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7fefe01342e0 con 0x7fefe00695a0 2024-09-15T21:21:17.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:18.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient: get_monmap_and_config 2024-09-15T21:21:18.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient: build_initial_monmap 2024-09-15T21:21:18.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient: monmap: 2024-09-15T21:21:18.093 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:18.091676+0000 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:18.091676+0000 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:18.094 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 Processor -- start 2024-09-15T21:21:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 -- start start 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient: init 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f158c0a1a20 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c0a0360 0x7f158c13cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): picked mon.noname-b con 0x7f158c0a0360 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f158c06e580 0x7f158c13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): picked mon.noname-c con 0x7f158c06e580 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158c06f5a0 con 0x7f158c0a1a20 2024-09-15T21:21:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f158c06ea00 con 0x7f158c06e580 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f158c09f980 con 0x7f158c0a0360 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159329d640 10 monclient(hunting): authenticate will time out at 1201.487358s 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c0a0360 0x7f158c13cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1591039640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f158c0a1a20 0x7f158c0a1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44250/0 (socket says 172.21.15.17:44250) 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1591039640 1 -- 172.21.15.17:0/3576073761 learned_addr learned my addr 172.21.15.17:0/3576073761 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): get_auth_request con 0x7f158c0a0360 auth_method 0 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:18.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f159183a640 1 --2- 172.21.15.17:0/3576073761 >> v2:172.21.15.119:3300/0 conn(0x7f158c06e580 0x7f158c13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 10 monclient(hunting): handle_auth_done global_id 44267 payload 274 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 1 -- 172.21.15.17:0/3576073761 >> v2:172.21.15.119:3300/0 conn(0x7f158c06e580 msgr2=0x7f158c13f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 1 --2- 172.21.15.17:0/3576073761 >> v2:172.21.15.119:3300/0 conn(0x7f158c06e580 0x7f158c13f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.089+0000 7f1590838640 1 -- 172.21.15.17:0/3576073761 >> v1:172.21.15.17:6789/0 conn(0x7f158c0a1a20 legacy=0x7f158c0a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1590838640 10 monclient: _finish_hunting 0 2024-09-15T21:21:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1590838640 1 monclient: found mon.noname-b 2024-09-15T21:21:18.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1590838640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:18.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1590838640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:18.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1590838640 1 -- 172.21.15.17:0/3576073761 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c0a2510 con 0x7f158c0a0360 2024-09-15T21:21:18.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159183a640 10 monclient: get_auth_request con 0x7f158c06e580 auth_method 0 2024-09-15T21:21:18.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159183a640 1 --2- 172.21.15.17:0/3576073761 >> v2:172.21.15.119:3300/0 conn(0x7f158c06e580 0x7f158c13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1590838640 1 --2- 172.21.15.17:0/3576073761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c0a0360 0x7f158c13cc50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f157c0097c0 tx=0x7f157c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3230807353c0f2ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 1 -- 172.21.15.17:0/3576073761 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0b0070 con 0x7f158c0a0360 2024-09-15T21:21:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: dump: 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 5 monclient: authenticate success, global_id 44267 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 1 -- 172.21.15.17:0/3576073761 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157c0ab600 con 0x7f158c0a0360 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 4 set_mon_vals no callback set 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 1 -- 172.21.15.17:0/3576073761 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0ab9d0 con 0x7f158c0a0360 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f1583fff640 10 monclient: dump: 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 10 monclient: shutdown 2024-09-15T21:21:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 -- 172.21.15.17:0/3576073761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c0a0360 msgr2=0x7f158c13cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 --2- 172.21.15.17:0/3576073761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c0a0360 0x7f158c13cc50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f157c0097c0 tx=0x7f157c01c3f0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 -- 172.21.15.17:0/3576073761 shutdown_connections 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 --2- 172.21.15.17:0/3576073761 >> v2:172.21.15.119:3300/0 conn(0x7f158c06e580 0x7f158c13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 --2- 172.21.15.17:0/3576073761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c0a0360 0x7f158c13cc50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 -- 172.21.15.17:0/3576073761 >> 172.21.15.17:0/3576073761 conn(0x7f158c0a8980 msgr2=0x7f158c0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 -- 172.21.15.17:0/3576073761 shutdown_connections 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 1 -- 172.21.15.17:0/3576073761 wait complete. 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 10 monclient: build_initial_monmap 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.093+0000 7f159329d640 10 monclient: monmap: 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 Processor -- start 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- start start 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: init 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): picked mon.noname-a con 0x7f158c06e580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c0a1a20 0x7f158c127900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): picked mon.noname-b con 0x7f158c0a1a20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c13e770 0x7f158c1cf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): picked mon.noname-c con 0x7f158c13e770 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f158c142290 con 0x7f158c06e580 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f158c142110 con 0x7f158c0a1a20 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f158c142410 con 0x7f158c13e770 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient(hunting): authenticate will time out at 1201.492693s 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44960/0 (socket says 172.21.15.17:44960) 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c13e770 0x7f158c1cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 -- 172.21.15.17:0/2901654050 learned_addr learned my addr 172.21.15.17:0/2901654050 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 10 monclient(hunting): get_auth_request con 0x7f158c06e580 auth_method 0 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): get_auth_request con 0x7f158c13e770 auth_method 0 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 -- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 msgr2=0x7f158c1257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:21:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 -- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 msgr2=0x7f158c1257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1590838640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c0a1a20 0x7f158c127900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient(hunting): handle_auth_done global_id 44270 payload 824 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 -- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c0a1a20 msgr2=0x7f158c127900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c0a1a20 0x7f158c127900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 -- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 msgr2=0x7f158c1257c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 monclient: found mon.noname-c 2024-09-15T21:21:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 -- 172.21.15.17:0/2901654050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c13f410 con 0x7f158c13e770 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159183a640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c13e770 0x7f158c1cf570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f158400cdf0 tx=0x7f158400a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158401a070 con 0x7f158c13e770 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: dump: 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158400d840 con 0x7f158c13e770 2024-09-15T21:21:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400dc30 con 0x7f158c13e770 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 5 monclient: authenticate success, global_id 44270 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _renew_subs 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c09f1f0 con 0x7f158c13e770 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1582ffd640 4 set_mon_vals no callback set 2024-09-15T21:21:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 10 monclient: dump: 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _renew_subs 2024-09-15T21:21:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c128860 con 0x7f158c13e770 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c0a1e00 con 0x7f158c13e770 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f158401d070 con 0x7f158c13e770 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f157003b9c0 0x7f157003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(107..107 src has 1..107) ==== 6199+0+0 (secure 0 0 0) 0x7f1584050830 con 0x7f158c13e770 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f157003b9c0 0x7f157003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.097+0000 7f1591039640 10 monclient: get_auth_request con 0x7f157003b9c0 auth_method 0 2024-09-15T21:21:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.101+0000 7f1591039640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f157003b9c0 0x7f157003de80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1588000f70 tx=0x7f1588009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.101+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f15840143a0 con 0x7f158c13e770 2024-09-15T21:21:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.101+0000 7f1581ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.101+0000 7f1581ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:18.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f159329d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:18.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f159329d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:18.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f159329d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:18.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f158c129060 con 0x7f158c13e770 2024-09-15T21:21:18.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f1581ffb640 1 -- 172.21.15.17:0/2901654050 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f158400ddd0 con 0x7f158c13e770 2024-09-15T21:21:18.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f1581ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:18.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.337+0000 7f1581ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:18.343 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:18.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f157003b9c0 msgr2=0x7f157003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:18.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f157003b9c0 0x7f157003de80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1588000f70 tx=0x7f1588009040 comp rx=0 tx=0).stop 2024-09-15T21:21:18.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 10 monclient: shutdown 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c13e770 msgr2=0x7f158c1cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c13e770 0x7f158c1cf570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f158400cdf0 tx=0x7f158400a710 comp rx=0 tx=0).stop 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 shutdown_connections 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f157003b9c0 0x7f157003de80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f158c13e770 0x7f158c1cf570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c0a1a20 0x7f158c127900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 --2- 172.21.15.17:0/2901654050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f158c06e580 0x7f158c1257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 >> 172.21.15.17:0/2901654050 conn(0x7f158c0a8980 msgr2=0x7f158c0a4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 shutdown_connections 2024-09-15T21:21:18.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:18.341+0000 7f159329d640 1 -- 172.21.15.17:0/2901654050 wait complete. 2024-09-15T21:21:18.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.674+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v108) ==== 58+0+3 (secure 0 0 0) 0x7fefe0073e10 con 0x7fefe00695a0 2024-09-15T21:21:18.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.674+0000 7fefd5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T21:21:18.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.674+0000 7fefd5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:18.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.786+0000 7fefe74be640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T21:21:18.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.786+0000 7fefe74be640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T21:21:18.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.786+0000 7fefe74be640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:18.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:18.786+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7fefe013a890 con 0x7fefe00695a0 2024-09-15T21:21:19.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.678+0000 7fefd5ffb640 1 -- 172.21.15.119:0/3552900596 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v109) ==== 58+0+3 (secure 0 0 0) 0x7fefdc016130 con 0x7fefe00695a0 2024-09-15T21:21:19.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.678+0000 7fefd5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T21:21:19.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.678+0000 7fefd5ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:19.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 msgr2=0x7fefc403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:19.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fefe006fe10 tx=0x7fefd0008040 comp rx=0 tx=0).stop 2024-09-15T21:21:19.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 10 monclient: shutdown 2024-09-15T21:21:19.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:19.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 msgr2=0x7fefe013f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe013f070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fefdc01aad0 tx=0x7fefdc0191c0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 shutdown_connections 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe00a56d0 0x7fefe013faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0072e70 0x7fefe013f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 --2- 172.21.15.119:0/3552900596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe00695a0 0x7fefe013f070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 >> 172.21.15.119:0/3552900596 conn(0x7fefe006cbf0 msgr2=0x7fefe0131500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 shutdown_connections 2024-09-15T21:21:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.682+0000 7fefe74be640 1 -- 172.21.15.119:0/3552900596 wait complete. 2024-09-15T21:21:19.708 INFO:tasks.workunit.client.0.smithi119.stderr:+ o5=14 2024-09-15T21:21:19.708 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 12 123 root=default host=foobaz 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.826+0000 7f486b862640 10 monclient: get_monmap_and_config 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.826+0000 7f486b862640 10 monclient: build_initial_monmap 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.826+0000 7f486b862640 10 monclient: monmap: 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:19.831755+0000 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:19.831755+0000 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:19.833 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:19.834 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:19.834 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:19.834 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:19.834 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:19.834 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:19.834 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:19.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.826+0000 7f486b862640 1 Processor -- start 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.826+0000 7f486b862640 1 -- start start 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.826+0000 7f486b862640 10 monclient: init 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4864135bf0 0x7f486409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): picked mon.noname-a con 0x7f4864135bf0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4864135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4864136460 0x7f48640a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): picked mon.noname-c con 0x7f4864136460 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48640a1d80 con 0x7f4864135fd0 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f48640a3600 con 0x7f4864135bf0 2024-09-15T21:21:19.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f48640a2060 con 0x7f4864136460 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient(hunting): authenticate will time out at 1198.628499s 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4864136460 0x7f48640a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4864136460 0x7f48640a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41780/0 (socket says 172.21.15.119:41780) 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 -- 172.21.15.119:0/2697515075 learned_addr learned my addr 172.21.15.119:0/2697515075 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): get_auth_request con 0x7f4864136460 auth_method 0 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:19.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f48695fe640 1 --2- 172.21.15.119:0/2697515075 >> v2:172.21.15.17:3300/0 conn(0x7f4864135bf0 0x7f486409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f48695fe640 10 monclient(hunting): get_auth_request con 0x7f4864135bf0 auth_method 0 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f48695fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f48695fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f48695fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient(hunting): handle_auth_done global_id 34771 payload 274 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 -- 172.21.15.119:0/2697515075 >> v2:172.21.15.17:3300/0 conn(0x7f4864135bf0 msgr2=0x7f486409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 --2- 172.21.15.119:0/2697515075 >> v2:172.21.15.17:3300/0 conn(0x7f4864135bf0 0x7f486409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 -- 172.21.15.119:0/2697515075 >> v1:172.21.15.17:6789/0 conn(0x7f4864135fd0 legacy=0x7f48640a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 monclient: found mon.noname-c 2024-09-15T21:21:19.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 -- 172.21.15.119:0/2697515075 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48640a2340 con 0x7f4864136460 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f4869dff640 1 --2- 172.21.15.119:0/2697515075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864136460 0x7f48640a1160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f485c009e50 tx=0x7f485c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc70ecd0e1d7d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 1 -- 172.21.15.119:0/2697515075 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a5750 con 0x7f4864136460 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: dump: 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:19.842 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: _finish_auth 0 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 1 -- 172.21.15.119:0/2697515075 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c0a58f0 con 0x7f4864136460 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 1 -- 172.21.15.119:0/2697515075 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a5cc0 con 0x7f4864136460 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 5 monclient: authenticate success, global_id 34771 2024-09-15T21:21:19.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f486b862640 4 set_mon_vals no callback set 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.830+0000 7f485bfff640 10 monclient: dump: 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.844 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: shutdown 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/2697515075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864136460 msgr2=0x7f48640a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 --2- 172.21.15.119:0/2697515075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864136460 0x7f48640a1160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f485c009e50 tx=0x7f485c01c8b0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/2697515075 shutdown_connections 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 --2- 172.21.15.119:0/2697515075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864136460 0x7f48640a1160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 --2- 172.21.15.119:0/2697515075 >> v2:172.21.15.17:3300/0 conn(0x7f4864135bf0 0x7f486409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/2697515075 >> 172.21.15.119:0/2697515075 conn(0x7f48640a66e0 msgr2=0x7f48640a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/2697515075 shutdown_connections 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/2697515075 wait complete. 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: build_initial_monmap 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: monmap: 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:19.846 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:19.847 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:19.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 Processor -- start 2024-09-15T21:21:19.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- start start 2024-09-15T21:21:19.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: init 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4864135bf0 0x7f4864134540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): picked mon.noname-c con 0x7f4864135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 0x7f4864134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): picked mon.noname-b con 0x7f4864135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4864136460 0x7f48641cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): picked mon.noname-a con 0x7f4864136460 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48640a1880 con 0x7f4864136460 2024-09-15T21:21:19.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f48640a1700 con 0x7f4864135fd0 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f48641402b0 con 0x7f4864135bf0 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient(hunting): authenticate will time out at 1198.633695s 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 0x7f4864134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 0x7f4864134a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41790/0 (socket says 172.21.15.119:41790) 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 -- 172.21.15.119:0/1690346126 learned_addr learned my addr 172.21.15.119:0/1690346126 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): get_auth_request con 0x7f4864135fd0 auth_method 0 2024-09-15T21:21:19.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient(hunting): handle_auth_done global_id 34774 payload 824 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4869dff640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4864136460 0x7f48641cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 -- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4864135bf0 msgr2=0x7f4864134540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4864135bf0 0x7f4864134540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 -- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4864136460 msgr2=0x7f48641cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4864136460 0x7f48641cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:19.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 monclient: found mon.noname-b 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 -- 172.21.15.119:0/1690346126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486409d240 con 0x7f4864135fd0 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4868dfd640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 0x7f4864134a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f485400c450 tx=0x7f485400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854012830 con 0x7f4864135fd0 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: dump: 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:19.854 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854017070 con 0x7f4864135fd0 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:19.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854012c30 con 0x7f4864135fd0 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 10 monclient: dump: 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:19.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f485b7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 5 monclient: authenticate success, global_id 34774 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _renew_subs 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486411ecc0 con 0x7f4864135fd0 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _renew_subs 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48640a5740 con 0x7f4864135fd0 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4864072990 con 0x7f4864135fd0 2024-09-15T21:21:19.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.834+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f485401c460 con 0x7f4864135fd0 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.838+0000 7f4859ffb640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f484803b9f0 0x7f484803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.838+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6437+0+0 (secure 0 0 0) 0x7f485404ffe0 con 0x7f4864135fd0 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.838+0000 7f48695fe640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f484803b9f0 0x7f484803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.838+0000 7f48695fe640 10 monclient: get_auth_request con 0x7f484803b9f0 auth_method 0 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.838+0000 7f48695fe640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f484803b9f0 0x7f484803deb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f486411eed0 tx=0x7f4860008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.842+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f486409d240 con 0x7f4864135fd0 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.842+0000 7f4859ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:19.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:19.842+0000 7f4859ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:20.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.014+0000 7f486b862640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:20.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.014+0000 7f486b862640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:20.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.014+0000 7f486b862640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.014+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7f486409fe70 con 0x7f4864135fd0 2024-09-15T21:21:20.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.686+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map v110) ==== 220+0+0 (secure 0 0 0) 0x7f485404e380 con 0x7f4864135fd0 2024-09-15T21:21:20.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.686+0000 7f4859ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:20.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.686+0000 7f4859ffb640 10 monclient: _finish_command 2 = system:0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map 2024-09-15T21:21:20.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f486b862640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:20.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f486b862640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:20.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f486b862640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7f486406a830 con 0x7f4864135fd0 2024-09-15T21:21:20.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f4859ffb640 1 -- 172.21.15.119:0/1690346126 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change v110) ==== 218+0+0 (secure 0 0 0) 0x7f4854012dd0 con 0x7f4864135fd0 2024-09-15T21:21:20.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f4859ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:20.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.790+0000 7f4859ffb640 10 monclient: _finish_command 3 = system:0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T21:21:20.797 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f484803b9f0 msgr2=0x7f484803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f484803b9f0 0x7f484803deb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f486411eed0 tx=0x7f4860008040 comp rx=0 tx=0).stop 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 10 monclient: shutdown 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 msgr2=0x7f4864134a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 0x7f4864134a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f485400c450 tx=0x7f485400c910 comp rx=0 tx=0).stop 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 shutdown_connections 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f484803b9f0 0x7f484803deb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4864136460 0x7f48641cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4864135fd0 0x7f4864134a80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 --2- 172.21.15.119:0/1690346126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4864135bf0 0x7f4864134540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 >> 172.21.15.119:0/1690346126 conn(0x7f48640a66e0 msgr2=0x7f4864136840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:20.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 shutdown_connections 2024-09-15T21:21:20.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.794+0000 7f486b862640 1 -- 172.21.15.119:0/1690346126 wait complete. 2024-09-15T21:21:20.814 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add 14 123 root=default host=foobaz 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient: get_monmap_and_config 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient: build_initial_monmap 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient: monmap: 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:20.934807+0000 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:20.934807+0000 2024-09-15T21:21:20.936 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:20.937 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 Processor -- start 2024-09-15T21:21:20.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 -- start start 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient: init 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffb74135ba0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 0x7ffb7413a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): picked mon.noname-c con 0x7ffb74073ae0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb7413d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): picked mon.noname-b con 0x7ffb7413ae80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb74072b20 con 0x7ffb74135ba0 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7406fc30 con 0x7ffb74073ae0 2024-09-15T21:21:20.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb74072880 con 0x7ffb7413ae80 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb78bff640 10 monclient(hunting): authenticate will time out at 1199.731787s 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb7259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffb74135ba0 0x7ffb74135f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59360/0 (socket says 172.21.15.119:59360) 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb7259c640 1 -- 172.21.15.119:0/1242951743 learned_addr learned my addr 172.21.15.119:0/1242951743 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb71d9b640 1 --2- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 0x7ffb7413a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb71d9b640 10 monclient(hunting): get_auth_request con 0x7ffb74073ae0 auth_method 0 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb71d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb71d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.930+0000 7ffb71d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb71d9b640 1 -- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 msgr2=0x7ffb7413a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb71d9b640 1 -- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 msgr2=0x7ffb7413a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb71d9b640 1 --2- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 0x7ffb7413a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:20.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb71d9b640 1 --2- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 0x7ffb7413a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb7413d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 10 monclient(hunting): get_auth_request con 0x7ffb7413ae80 auth_method 0 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 1 -- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 msgr2=0x7ffb7413d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 1 -- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 msgr2=0x7ffb7413d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb7413d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:20.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.934+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb7413d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:20.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.958+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804774384 0 0) 0x7ffb74072b20 con 0x7ffb74135ba0 2024-09-15T21:21:20.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.958+0000 7ffb7159a640 10 monclient(hunting): _negotiate 2024-09-15T21:21:20.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.958+0000 7ffb7159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:20.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.958+0000 7ffb7159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:20.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.958+0000 7ffb7159a640 10 monclient(hunting): my global_id is 34777 2024-09-15T21:21:20.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.958+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb5c003530 con 0x7ffb74135ba0 2024-09-15T21:21:20.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305965272 0 0) 0x7ffb5c003530 con 0x7ffb74135ba0 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 msgr2=0x7ffb7413d270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb7413d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 msgr2=0x7ffb7413a940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 --2- 172.21.15.119:0/1242951743 >> v2:172.21.15.119:3300/0 conn(0x7ffb74073ae0 0x7ffb7413a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb74135600 con 0x7ffb74135ba0 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: _finish_auth 0 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: _check_auth_tickets 2024-09-15T21:21:20.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ffb64001a80 con 0x7ffb74135ba0 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb78bff640 5 monclient: authenticate success, global_id 34777 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: dump: 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:20.971 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 >> v1:172.21.15.119:6789/0 conn(0x7ffb74135ba0 legacy=0x7ffb74135f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb5c003a60 0x7ffb5c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): picked mon.b con 0x7ffb5c003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c004810 0x7ffb5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): picked mon.a con 0x7ffb5c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb5c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): picked mon.c con 0x7ffb7413ae80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:20.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74072880 con 0x7ffb5c004810 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c005530 con 0x7ffb5c003a60 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5c00bf10 con 0x7ffb7413ae80 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7159a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb78bff640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb5c003a60 0x7ffb5c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): get_auth_request con 0x7ffb5c003a60 auth_method 0 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:20.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb5c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient(hunting): handle_auth_done global_id 34777 payload 293 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb71d9b640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c004810 0x7ffb5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 -- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 msgr2=0x7ffb5c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb7413ae80 0x7ffb5c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 -- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c004810 msgr2=0x7ffb5c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c004810 0x7ffb5c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 monclient: found mon.b 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:20.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 1 -- 172.21.15.119:0/1242951743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c003530 con 0x7ffb5c003a60 2024-09-15T21:21:20.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient: _finish_auth 0 2024-09-15T21:21:20.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 10 monclient: _check_auth_tickets 2024-09-15T21:21:20.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.962+0000 7ffb7259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:20.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7259c640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb5c003a60 0x7ffb5c003e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400a7f0 tx=0x7ffb6401d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6491b40f214368bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:20.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400a410 con 0x7ffb5c003a60 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: dump: 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb640ad650 con 0x7ffb5c003a60 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 1 -- 172.21.15.119:0/1242951743 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640ada40 con 0x7ffb5c003a60 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 4 set_mon_vals no callback set 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7159a640 10 monclient: dump: 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:20.982 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: shutdown 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- 172.21.15.119:0/1242951743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb5c003a60 msgr2=0x7ffb5c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 --2- 172.21.15.119:0/1242951743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb5c003a60 0x7ffb5c003e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400a7f0 tx=0x7ffb6401d260 comp rx=0 tx=0).stop 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 -- 172.21.15.119:0/1242951743 reap_dead start 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- 172.21.15.119:0/1242951743 shutdown_connections 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- 172.21.15.119:0/1242951743 >> 172.21.15.119:0/1242951743 conn(0x7ffb7406cbf0 msgr2=0x7ffb7406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- 172.21.15.119:0/1242951743 shutdown_connections 2024-09-15T21:21:20.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- 172.21.15.119:0/1242951743 wait complete. 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: build_initial_monmap 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: monmap: 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:20.984 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 Processor -- start 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- start start 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: init 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb5c003a60 0x7ffb741cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): picked mon.noname-c con 0x7ffb5c003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb741304a0 0x7ffb7409da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): picked mon.noname-a con 0x7ffb741304a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 0x7ffb7409df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): picked mon.noname-b con 0x7ffb74130920 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7413dc00 con 0x7ffb741304a0 2024-09-15T21:21:20.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7413da80 con 0x7ffb74130920 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7413dd80 con 0x7ffb5c003a60 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb78bff640 10 monclient(hunting): authenticate will time out at 1199.767298s 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 0x7ffb7409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 0x7ffb7409df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41814/0 (socket says 172.21.15.119:41814) 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 -- 172.21.15.119:0/3096553816 learned_addr learned my addr 172.21.15.119:0/3096553816 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): get_auth_request con 0x7ffb74130920 auth_method 0 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb71d9b640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb741304a0 0x7ffb7409da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7259c640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb5c003a60 0x7ffb741cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb71d9b640 10 monclient(hunting): get_auth_request con 0x7ffb741304a0 auth_method 0 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb71d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb71d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb71d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient(hunting): handle_auth_done global_id 34780 payload 824 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 -- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb5c003a60 msgr2=0x7ffb741cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb5c003a60 0x7ffb741cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 -- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb741304a0 msgr2=0x7ffb7409da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:20.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb741304a0 0x7ffb7409da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 monclient: found mon.noname-b 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 -- 172.21.15.119:0/3096553816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7411d100 con 0x7ffb74130920 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7259c640 10 monclient: get_auth_request con 0x7ffb5c003a60 auth_method 0 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb7259c640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb5c003a60 0x7ffb741cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.966+0000 7ffb72d9d640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 0x7ffb7409df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb740a4980 tx=0x7ffb6c00e770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c017020 con 0x7ffb74130920 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: dump: 2024-09-15T21:21:20.988 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6c013610 con 0x7ffb74130920 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 5 monclient: authenticate success, global_id 34780 2024-09-15T21:21:20.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 monclient: _renew_subs 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb740a5a30 con 0x7ffb74130920 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00f040 con 0x7ffb74130920 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb70d99640 4 set_mon_vals no callback set 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 10 monclient: dump: 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:20.990 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 monclient: _renew_subs 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7409f450 con 0x7ffb74130920 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffb6c01a020 con 0x7ffb74130920 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb74135f80 con 0x7ffb74130920 2024-09-15T21:21:20.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5403b9c0 0x7ffb5403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6621+0+0 (secure 0 0 0) 0x7ffb6c04fb80 con 0x7ffb74130920 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb7259c640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5403b9c0 0x7ffb5403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb7259c640 10 monclient: get_auth_request con 0x7ffb5403b9c0 auth_method 0 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.970+0000 7ffb7259c640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5403b9c0 0x7ffb5403de80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffb640ad000 tx=0x7ffb74073590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.974+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffb740a5a30 con 0x7ffb74130920 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.974+0000 7ffb62ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:20.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:20.974+0000 7ffb62ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:21.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:21.146+0000 7ffb78bff640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:21.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:21.146+0000 7ffb78bff640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:21.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:21.146+0000 7ffb78bff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:21.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:21.146+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7ffb741d02a0 con 0x7ffb74130920 2024-09-15T21:21:21.363 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:21.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.489+0000 7f7d6e78f640 10 monclient: get_monmap_and_config 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.489+0000 7f7d6e78f640 10 monclient: build_initial_monmap 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.489+0000 7f7d6e78f640 10 monclient: monmap: 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:21.493853+0000 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:21.493853+0000 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:21.496 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:21.497 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:21.497 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 Processor -- start 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 -- start start 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient: init 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d680a0890 0x7f7d680a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): picked mon.noname-a con 0x7f7d680a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d6809e8c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d681410a0 0x7f7d68143490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): picked mon.noname-b con 0x7f7d681410a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d680a31c0 con 0x7f7d6809e8c0 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7d680a1af0 con 0x7f7d680a0890 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6809d640 con 0x7f7d681410a0 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6e78f640 10 monclient(hunting): authenticate will time out at 1204.889258s 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d681410a0 0x7f7d68143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d680a0890 0x7f7d680a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d680a0890 0x7f7d680a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44974/0 (socket says 172.21.15.17:44974) 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d677fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7d6809e8c0 0x7f7d680a11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44258/0 (socket says 172.21.15.17:44258) 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 -- 172.21.15.17:0/3780891686 learned_addr learned my addr 172.21.15.17:0/3780891686 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): get_auth_request con 0x7f7d680a0890 auth_method 0 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 1 --2- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d681410a0 0x7f7d68143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34584/0 (socket says 172.21.15.17:34584) 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): get_auth_request con 0x7f7d681410a0 auth_method 0 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d6cd2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient(hunting): handle_auth_done global_id 54102 payload 274 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 -- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d681410a0 msgr2=0x7f7d68143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 --2- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d681410a0 0x7f7d68143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 -- 172.21.15.17:0/3780891686 >> v1:172.21.15.17:6789/0 conn(0x7f7d6809e8c0 legacy=0x7f7d680a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 monclient: found mon.noname-a 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d66ffd640 1 -- 172.21.15.17:0/3780891686 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061853492 0 0) 0x7f7d680a31c0 con 0x7f7d6809e8c0 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:21.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d67fff640 1 -- 172.21.15.17:0/3780891686 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6809d920 con 0x7f7d680a0890 2024-09-15T21:21:21.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.493+0000 7f7d66ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:21.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 --2- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d680a0890 0x7f7d680a0c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098a0 tx=0x7f7d5c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9c7e70c3f0d01a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 1 -- 172.21.15.17:0/3780891686 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0b0070 con 0x7f7d680a0890 2024-09-15T21:21:21.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:21.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: dump: 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:21.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 5 monclient: authenticate success, global_id 54102 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 1 -- 172.21.15.17:0/3780891686 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c0ab600 con 0x7f7d680a0890 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 1 -- 172.21.15.17:0/3780891686 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0ab9d0 con 0x7f7d680a0890 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 4 set_mon_vals no callback set 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d66ffd640 10 monclient: dump: 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.510 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: shutdown 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d680a0890 msgr2=0x7f7d680a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d680a0890 0x7f7d680a0c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098a0 tx=0x7f7d5c01c400 comp rx=0 tx=0).stop 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3780891686 shutdown_connections 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d681410a0 0x7f7d68143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3780891686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d680a0890 0x7f7d680a0c70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3780891686 >> 172.21.15.17:0/3780891686 conn(0x7f7d680a66a0 msgr2=0x7f7d680a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3780891686 shutdown_connections 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3780891686 wait complete. 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: build_initial_monmap 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: monmap: 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 Processor -- start 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- start start 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: init 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 0x7f7d68125290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): picked mon.noname-c con 0x7f7d680a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d681410a0 0x7f7d680728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): picked mon.noname-b con 0x7f7d681410a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6813c0e0 0x7f7d681d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): picked mon.noname-a con 0x7f7d6813c0e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68146580 con 0x7f7d6813c0e0 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68146400 con 0x7f7d681410a0 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7d68146700 con 0x7f7d680a0890 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient(hunting): authenticate will time out at 1204.894432s 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 0x7f7d68125290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6813c0e0 0x7f7d681d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 0x7f7d68125290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34590/0 (socket says 172.21.15.17:34590) 2024-09-15T21:21:21.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 -- 172.21.15.17:0/3306501671 learned_addr learned my addr 172.21.15.17:0/3306501671 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): get_auth_request con 0x7f7d680a0890 auth_method 0 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): get_auth_request con 0x7f7d6813c0e0 auth_method 0 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient(hunting): handle_auth_done global_id 44276 payload 824 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d677fe640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d681410a0 0x7f7d680728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 -- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d681410a0 msgr2=0x7f7d680728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d681410a0 0x7f7d680728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 -- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6813c0e0 msgr2=0x7f7d681d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6813c0e0 0x7f7d681d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 monclient: found mon.noname-c 2024-09-15T21:21:21.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 -- 172.21.15.17:0/3306501671 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d681261f0 con 0x7f7d680a0890 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d677fe640 10 monclient: get_auth_request con 0x7f7d681410a0 auth_method 0 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6cd2c640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6813c0e0 0x7f7d681d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d677fe640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d681410a0 0x7f7d680728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d67fff640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 0x7f7d68125290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c01aab0 tx=0x7f7d5c01ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0b0040 con 0x7f7d680a0890 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: dump: 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c004030 con 0x7f7d680a0890 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 5 monclient: authenticate success, global_id 44276 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: _renew_subs 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6809d0f0 con 0x7f7d680a0890 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004400 con 0x7f7d680a0890 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d64ff9640 10 monclient: dump: 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: _renew_subs 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d68126a90 con 0x7f7d680a0890 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.497+0000 7f7d667fc640 4 set_mon_vals no callback set 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d6e78f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d6e78f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d6e78f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d68126500 con 0x7f7d680a0890 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7d5c0b3070 con 0x7f7d680a0890 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d64ff9640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d5403ba20 0x7f7d5403dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(110..110 src has 1..110) ==== 6621+0+0 (secure 0 0 0) 0x7f7d5c0ea560 con 0x7f7d680a0890 2024-09-15T21:21:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d677fe640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d5403ba20 0x7f7d5403dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d677fe640 10 monclient: get_auth_request con 0x7f7d5403ba20 auth_method 0 2024-09-15T21:21:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.501+0000 7f7d677fe640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d5403ba20 0x7f7d5403dee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7d68074390 tx=0x7f7d58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.505+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7d681261f0 con 0x7f7d680a0890 2024-09-15T21:21:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.505+0000 7f7d64ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.505+0000 7f7d64ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7d680a0c70 con 0x7f7d680a0890 2024-09-15T21:21:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d64ff9640 1 -- 172.21.15.17:0/3306501671 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f7d5c0ba020 con 0x7f7d680a0890 2024-09-15T21:21:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d64ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d64ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:21.749 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d5403ba20 msgr2=0x7f7d5403dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d5403ba20 0x7f7d5403dee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7d68074390 tx=0x7f7d58009040 comp rx=0 tx=0).stop 2024-09-15T21:21:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 10 monclient: shutdown 2024-09-15T21:21:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 msgr2=0x7f7d68125290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 0x7f7d68125290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c01aab0 tx=0x7f7d5c01ac40 comp rx=0 tx=0).stop 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 shutdown_connections 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d5403ba20 0x7f7d5403dee0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6813c0e0 0x7f7d681d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d681410a0 0x7f7d680728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 --2- 172.21.15.17:0/3306501671 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d680a0890 0x7f7d68125290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 >> 172.21.15.17:0/3306501671 conn(0x7f7d680a66a0 msgr2=0x7f7d68142350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 shutdown_connections 2024-09-15T21:21:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.745+0000 7f7d6e78f640 1 -- 172.21.15.17:0/3306501671 wait complete. 2024-09-15T21:21:21.769 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:21:21.769 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:21:21.769 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:21:21.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.885+0000 7ff2a50cb640 10 monclient: get_monmap_and_config 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.885+0000 7ff2a50cb640 10 monclient: build_initial_monmap 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.885+0000 7ff2a50cb640 10 monclient: monmap: 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:21.889111+0000 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:21.889111+0000 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:21.892 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 1 Processor -- start 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 1 -- start start 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient: init 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2a01378c0 0x7ff2a0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient(hunting): picked mon.noname-a con 0x7ff2a01378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2a01381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2a00a5210 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a00a1a20 con 0x7ff2a00a5210 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a00a1d00 con 0x7ff2a01381e0 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a00a1fe0 con 0x7ff2a01378c0 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff2a50cb640 10 monclient(hunting): authenticate will time out at 1205.285194s 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2a01378c0 0x7ff2a0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29f59e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2a00a5210 0x7ff2a00a56a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44266/0 (socket says 172.21.15.17:44266) 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29f59e640 1 -- 172.21.15.17:0/3368352431 learned_addr learned my addr 172.21.15.17:0/3368352431 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): get_auth_request con 0x7ff2a01378c0 auth_method 0 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_done global_id 54111 payload 274 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 1 -- 172.21.15.17:0/3368352431 >> v1:172.21.15.119:6789/0 conn(0x7ff2a01381e0 legacy=0x7ff2a00a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 1 -- 172.21.15.17:0/3368352431 >> v1:172.21.15.17:6789/0 conn(0x7ff2a00a5210 legacy=0x7ff2a00a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 1 monclient: found mon.noname-a 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29dd9b640 1 -- 172.21.15.17:0/3368352431 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244298891 0 0) 0x7ff2980014a0 con 0x7ff2a00a5210 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 1 -- 172.21.15.17:0/3368352431 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a00a22c0 con 0x7ff2a01378c0 2024-09-15T21:21:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29dd9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.889+0000 7ff29ed9d640 1 --2- 172.21.15.17:0/3368352431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0137ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2900097f0 tx=0x7ff29001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d418cf6c01b493 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 1 -- 172.21.15.17:0/3368352431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900b1070 con 0x7ff2a01378c0 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: dump: 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 5 monclient: authenticate success, global_id 54111 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 1 -- 172.21.15.17:0/3368352431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900a47f0 con 0x7ff2a01378c0 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 4 set_mon_vals no callback set 2024-09-15T21:21:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 1 -- 172.21.15.17:0/3368352431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900a4bc0 con 0x7ff2a01378c0 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29dd9b640 10 monclient: dump: 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.905 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: shutdown 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3368352431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 msgr2=0x7ff2a0137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 --2- 172.21.15.17:0/3368352431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0137ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2900097f0 tx=0x7ff29001c040 comp rx=0 tx=0).stop 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3368352431 shutdown_connections 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 --2- 172.21.15.17:0/3368352431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0137ca0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3368352431 >> 172.21.15.17:0/3368352431 conn(0x7ff2a00aac40 msgr2=0x7ff2a00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3368352431 shutdown_connections 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3368352431 wait complete. 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: build_initial_monmap 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: monmap: 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:21.906 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 Processor -- start 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- start start 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: init 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a00a5210 0x7ff2a01d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): picked mon.noname-b con 0x7ff2a00a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0136940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): picked mon.noname-a con 0x7ff2a01378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2a01381e0 0x7ff2a013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): picked mon.noname-c con 0x7ff2a01381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01441b0 con 0x7ff2a01378c0 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0144030 con 0x7ff2a00a5210 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a0144330 con 0x7ff2a01381e0 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient(hunting): authenticate will time out at 1205.290157s 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2a01381e0 0x7ff2a013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0136940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2a01381e0 0x7ff2a013cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34598/0 (socket says 172.21.15.17:34598) 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 1 -- 172.21.15.17:0/3490439319 learned_addr learned my addr 172.21.15.17:0/3490439319 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): get_auth_request con 0x7ff2a01381e0 auth_method 0 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0136940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45004/0 (socket says 172.21.15.17:45004) 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a00a5210 0x7ff2a01d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): get_auth_request con 0x7ff2a01378c0 auth_method 0 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): get_auth_request con 0x7ff2a00a5210 auth_method 0 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient(hunting): handle_auth_done global_id 54117 payload 824 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2a01381e0 msgr2=0x7ff2a013cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2a01381e0 0x7ff2a013cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a00a5210 msgr2=0x7ff2a01d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a00a5210 0x7ff2a01d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 monclient: found mon.noname-a 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a01d14d0 con 0x7ff2a01378c0 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29e59c640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0136940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c3e0 tx=0x7ff29400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294011790 con 0x7ff2a01378c0 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: dump: 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 5 monclient: authenticate success, global_id 54117 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: _renew_subs 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294017070 con 0x7ff2a01378c0 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a01408f0 con 0x7ff2a01378c0 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294011bb0 con 0x7ff2a01378c0 2024-09-15T21:21:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2877fe640 10 monclient: dump: 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff29d59a640 4 set_mon_vals no callback set 2024-09-15T21:21:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: _renew_subs 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a013f530 con 0x7ff2a01378c0 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.893+0000 7ff2a50cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2a50cb640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 0x7ff2a013d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2a50cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a013d9d0 con 0x7ff2a013d0b0 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 0x7ff2a013d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient: get_auth_request con 0x7ff2a013d0b0 auth_method 0 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff29401c460 con 0x7ff2a01378c0 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_done global_id 54120 payload 824 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2a0137ca0 con 0x7ff2a013d0b0 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28c03b9c0 0x7ff28c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6621+0+0 (secure 0 0 0) 0x7ff29404ff50 con 0x7ff2a01378c0 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29ed9d640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 0x7ff2a013d490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00a6150 tx=0x7ff29001a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900bc070 con 0x7ff2a013d0b0 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 10 monclient: dump: 2024-09-15T21:21:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29f59e640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28c03b9c0 0x7ff28c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29f59e640 10 monclient: get_auth_request con 0x7ff28c03b9c0 auth_method 0 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff2900b7640 con 0x7ff2a013d0b0 2024-09-15T21:21:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 msgr2=0x7ff2a013d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 0x7ff2a013d490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00a6150 tx=0x7ff29001a880 comp rx=0 tx=0).stop 2024-09-15T21:21:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 msgr2=0x7ff2a013d490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff2877fe640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a013d0b0 0x7ff2a013d490 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00a6150 tx=0x7ff29001a880 comp rx=0 tx=0).stop 2024-09-15T21:21:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.897+0000 7ff29f59e640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28c03b9c0 0x7ff28c03de80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff298004fd0 tx=0x7ff298001260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff2a50cb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff2a50cb640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 0x7ff2a0139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff2a50cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0140100 con 0x7ff2a0138d60 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 0x7ff2a0139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient: get_auth_request con 0x7ff2a0138d60 auth_method 0 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 10 monclient(hunting): handle_auth_done global_id 54123 payload 824 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.901+0000 7ff29ed9d640 1 -- 172.21.15.17:0/3490439319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff2a0069270 con 0x7ff2a0138d60 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff29ed9d640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 0x7ff2a0139170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff290019e80 tx=0x7ff290019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900bc040 con 0x7ff2a0138d60 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 10 monclient: dump: 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7ff2900ac8f0 con 0x7ff2a0138d60 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 msgr2=0x7ff2a0139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 0x7ff2a0139170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff290019e80 tx=0x7ff290019100 comp rx=0 tx=0).stop 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 msgr2=0x7ff2a0139170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2877fe640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a0138d60 0x7ff2a0139170 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff290019e80 tx=0x7ff290019100 comp rx=0 tx=0).stop 2024-09-15T21:21:21.922 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7267, 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:21:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:21.935 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28c03b9c0 msgr2=0x7ff28c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff28c03b9c0 0x7ff28c03de80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff298004fd0 tx=0x7ff298001260 comp rx=0 tx=0).stop 2024-09-15T21:21:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 10 monclient: shutdown 2024-09-15T21:21:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 msgr2=0x7ff2a0136940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 --2- 172.21.15.17:0/3490439319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2a01378c0 0x7ff2a0136940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c3e0 tx=0x7ff29400c8a0 comp rx=0 tx=0).stop 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff29f59e640 1 -- 172.21.15.17:0/3490439319 reap_dead start 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff29f59e640 1 -- 172.21.15.17:0/3490439319 reap_dead start 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 shutdown_connections 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 >> 172.21.15.17:0/3490439319 conn(0x7ff2a00aac40 msgr2=0x7ff2a00a8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 shutdown_connections 2024-09-15T21:21:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:21.905+0000 7ff2a50cb640 1 -- 172.21.15.17:0/3490439319 wait complete. 2024-09-15T21:21:21.937 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:21:21.937 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:21:21.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:21:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.002+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map v111) ==== 220+0+0 (secure 0 0 0) 0x7ffb7411d100 con 0x7ffb74130920 2024-09-15T21:21:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.002+0000 7ffb62ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:22.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.002+0000 7ffb62ffd640 10 monclient: _finish_command 2 = system:0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map 2024-09-15T21:21:22.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient: get_monmap_and_config 2024-09-15T21:21:22.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient: monmap: 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:22.054773+0000 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:22.054773+0000 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:22.058 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:22.058 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:22.058 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:22.058 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 Processor -- start 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 -- start start 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient: init 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56840a0890 0x7f56840a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): picked mon.noname-b con 0x7f56840a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f568409ffc0 0x7f56840a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): picked mon.noname-a con 0x7f568409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5684135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56840a3130 con 0x7f5684135670 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f56840a16f0 con 0x7f568409ffc0 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5684073df0 con 0x7f56840a0890 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568c402640 10 monclient(hunting): authenticate will time out at 1205.450199s 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56840a0890 0x7f56840a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f568409ffc0 0x7f56840a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56840a0890 0x7f56840a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34604/0 (socket says 172.21.15.17:34604) 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f568409ffc0 0x7f56840a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45034/0 (socket says 172.21.15.17:45034) 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 1 -- 172.21.15.17:0/1989026284 learned_addr learned my addr 172.21.15.17:0/1989026284 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): get_auth_request con 0x7f56840a0890 auth_method 0 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): get_auth_request con 0x7f568409ffc0 auth_method 0 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient(hunting): handle_auth_done global_id 54126 payload 274 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 -- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56840a0890 msgr2=0x7f56840a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 --2- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56840a0890 0x7f56840a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 -- 172.21.15.17:0/1989026284 >> v1:172.21.15.119:6789/0 conn(0x7f5684135670 legacy=0x7f5684135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 monclient: found mon.noname-a 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.053+0000 7f568999d640 1 -- 172.21.15.17:0/1989026284 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56840a1bf0 con 0x7f568409ffc0 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568999d640 1 --2- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f568409ffc0 0x7f56840a11b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56840a1870 tx=0x7f568001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64d4539dcc5cb04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 1 -- 172.21.15.17:0/1989026284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800b2070 con 0x7f568409ffc0 2024-09-15T21:21:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: dump: 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.067 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 1 -- 172.21.15.17:0/1989026284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56800a57e0 con 0x7f568409ffc0 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 1 -- 172.21.15.17:0/1989026284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800a5bb0 con 0x7f568409ffc0 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 5 monclient: authenticate success, global_id 54126 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 4 set_mon_vals no callback set 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568919c640 10 monclient: dump: 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: shutdown 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 -- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f568409ffc0 msgr2=0x7f56840a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 --2- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f568409ffc0 0x7f56840a11b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56840a1870 tx=0x7f568001c810 comp rx=0 tx=0).stop 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 -- 172.21.15.17:0/1989026284 shutdown_connections 2024-09-15T21:21:22.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 --2- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f568409ffc0 0x7f56840a11b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 --2- 172.21.15.17:0/1989026284 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56840a0890 0x7f56840a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 -- 172.21.15.17:0/1989026284 >> 172.21.15.17:0/1989026284 conn(0x7f56840a66a0 msgr2=0x7f56840a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 -- 172.21.15.17:0/1989026284 shutdown_connections 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 -- 172.21.15.17:0/1989026284 wait complete. 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: monmap: 2024-09-15T21:21:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:22.072 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 Processor -- start 2024-09-15T21:21:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 -- start start 2024-09-15T21:21:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: init 2024-09-15T21:21:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.057+0000 7f568c402640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568409ffc0 0x7f5684134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): picked mon.noname-c con 0x7f568409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56840a0890 0x7f56841345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): picked mon.noname-a con 0x7f56840a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5684135670 0x7f568411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): picked mon.noname-b con 0x7f5684135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5684140690 con 0x7f56840a0890 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5684140510 con 0x7f5684135670 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5684140810 con 0x7f568409ffc0 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): authenticate will time out at 1205.456180s 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568409ffc0 0x7f5684134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56840a0890 0x7f56841345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56840a0890 0x7f56841345d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45046/0 (socket says 172.21.15.17:45046) 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 -- 172.21.15.17:0/3431110120 learned_addr learned my addr 172.21.15.17:0/3431110120 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568409ffc0 0x7f5684134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34614/0 (socket says 172.21.15.17:34614) 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): get_auth_request con 0x7f56840a0890 auth_method 0 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): get_auth_request con 0x7f568409ffc0 auth_method 0 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a99f640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5684135670 0x7f568411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient(hunting): handle_auth_done global_id 54129 payload 824 2024-09-15T21:21:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568409ffc0 msgr2=0x7f5684134090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568409ffc0 0x7f5684134090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5684135670 msgr2=0x7f568411c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5684135670 0x7f568411c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 monclient: found mon.noname-a 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568411e5e0 con 0x7f56840a0890 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a99f640 10 monclient: get_auth_request con 0x7f5684135670 auth_method 0 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a99f640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5684135670 0x7f568411c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568999d640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56840a0890 0x7f56841345d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56800b0040 tx=0x7f56800195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800b2050 con 0x7f56840a0890 2024-09-15T21:21:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: dump: 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56800be070 con 0x7f56840a0890 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 5 monclient: authenticate success, global_id 54129 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient: _renew_subs 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568411e8c0 con 0x7f56840a0890 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680009450 con 0x7f56840a0890 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: dump: 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568899b640 4 set_mon_vals no callback set 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient: _renew_subs 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568409d0f0 con 0x7f56840a0890 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 0x7f568411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5684072840 con 0x7f568411eb00 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f56800adca0 con 0x7f56840a0890 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 0x7f568411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient: get_auth_request con 0x7f568411eb00 auth_method 0 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f566803b9c0 0x7f566803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7f56800eb4a0 con 0x7f56840a0890 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 10 monclient(hunting): handle_auth_done global_id 44288 payload 824 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5684135aa0 con 0x7f568411eb00 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a19e640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 0x7f568411ca20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5678011900 tx=0x7f5678011dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a99f640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f566803b9c0 0x7f566803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a99f640 10 monclient: get_auth_request con 0x7f566803b9c0 auth_method 0 2024-09-15T21:21:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567800fb60 con 0x7f568411eb00 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: dump: 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f567800fb60 con 0x7f568411eb00 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 msgr2=0x7f568411ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 0x7f568411ca20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5678011900 tx=0x7f5678011dc0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 msgr2=0x7f568411ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f5676ffd640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411eb00 0x7f568411ca20 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5678011900 tx=0x7f5678011dc0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.061+0000 7f568a99f640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f566803b9c0 0x7f566803de80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f70 tx=0x7f567c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 0x7f5684137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f568411d370 con 0x7f568411cf60 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 0x7f5684137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient: get_auth_request con 0x7f568411cf60 auth_method 0 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 10 monclient(hunting): handle_auth_done global_id 44291 payload 824 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 1 -- 172.21.15.17:0/3431110120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f568406d020 con 0x7f568411cf60 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a19e640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 0x7f5684137ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5678012010 tx=0x7f56780046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567800fea0 con 0x7f568411cf60 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 10 monclient: dump: 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.099 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f56780133a0 con 0x7f568411cf60 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:22.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 msgr2=0x7f5684137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 0x7f5684137ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5678012010 tx=0x7f56780046d0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 msgr2=0x7f5684137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:22.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f5676ffd640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f568411cf60 0x7f5684137ab0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5678012010 tx=0x7f56780046d0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.102 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:22.102 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:22.102 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:22.102 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:21:22.102 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-15T21:21:22.103 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 50667, 2024-09-15T21:21:22.103 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:22.103 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:22.103 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:21:22.103 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:22.104 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.104 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:21:22.104 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:22.104 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:22.104 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:22.105 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:22.105 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:22.105 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:22.105 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:22.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:22.106 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:22.106 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:22.106 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:22.106 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:22.106 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:22.107 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:22.107 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.107 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:22.107 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:22.107 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:22.107 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:22.108 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:22.108 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:22.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:22.108 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:22.108 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:22.108 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:22.109 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:22.110 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:22.112 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:22.112 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:22.112 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:22.112 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:22.112 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:22.112 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:22.113 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:22.113 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:22.113 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:22.113 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:22.113 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:22.113 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:22.114 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:22.114 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.114 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:22.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:22.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.115 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:22.115 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:22.115 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:22.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:22.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:22.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:22.116 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:22.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:22.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.117 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:22.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:22.118 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:22.118 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:22.118 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:22.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:22.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.120 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.122 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:22.122 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:22.122 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:22.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:22.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:22.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:22.123 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:22.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:22.124 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:22.125 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.125 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.125 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:22.125 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:22.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.126 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:22.127 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f566803b9c0 msgr2=0x7f566803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f566803b9c0 0x7f566803de80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f70 tx=0x7f567c009040 comp rx=0 tx=0).stop 2024-09-15T21:21:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 10 monclient: shutdown 2024-09-15T21:21:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56840a0890 msgr2=0x7f56841345d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 --2- 172.21.15.17:0/3431110120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56840a0890 0x7f56841345d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56800b0040 tx=0x7f56800195b0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a99f640 1 -- 172.21.15.17:0/3431110120 reap_dead start 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568a99f640 1 -- 172.21.15.17:0/3431110120 reap_dead start 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 shutdown_connections 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 >> 172.21.15.17:0/3431110120 conn(0x7f56840a66a0 msgr2=0x7f5684072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 shutdown_connections 2024-09-15T21:21:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.069+0000 7f568c402640 1 -- 172.21.15.17:0/3431110120 wait complete. 2024-09-15T21:21:22.130 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:21:22.130 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:21:22.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:21:22.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb78bff640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:22.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb78bff640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:22.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb78bff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7ffb7406a880 con 0x7ffb74130920 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb62ffd640 1 -- 172.21.15.119:0/3096553816 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change v111) ==== 218+0+0 (secure 0 0 0) 0x7ffb6c0232f0 con 0x7ffb74130920 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb62ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.110+0000 7ffb62ffd640 10 monclient: _finish_command 3 = system:0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5403b9c0 msgr2=0x7ffb5403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5403b9c0 0x7ffb5403de80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffb640ad000 tx=0x7ffb74073590 comp rx=0 tx=0).stop 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 10 monclient: shutdown 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 msgr2=0x7ffb7409df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 0x7ffb7409df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb740a4980 tx=0x7ffb6c00e770 comp rx=0 tx=0).stop 2024-09-15T21:21:22.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 shutdown_connections 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5403b9c0 0x7ffb5403de80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb74130920 0x7ffb7409df70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb741304a0 0x7ffb7409da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 --2- 172.21.15.119:0/3096553816 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb5c003a60 0x7ffb741cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 >> 172.21.15.119:0/3096553816 conn(0x7ffb7406cbf0 msgr2=0x7ffb74134d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 shutdown_connections 2024-09-15T21:21:22.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.114+0000 7ffb78bff640 1 -- 172.21.15.119:0/3096553816 wait complete. 2024-09-15T21:21:22.134 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:21:22.134 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep osd.12 2024-09-15T21:21:22.134 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 123 2024-09-15T21:21:22.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient: get_monmap_and_config 2024-09-15T21:21:22.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient: monmap: 2024-09-15T21:21:22.248 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:22.248 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:22.247163+0000 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:22.247163+0000 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:22.249 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 Processor -- start 2024-09-15T21:21:22.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 -- start start 2024-09-15T21:21:22.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient: init 2024-09-15T21:21:22.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:21:22.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0514135d10 0x7f051406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): picked mon.noname-b con 0x7f0514135d10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05141360f0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:22.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05140a10f0 0x7f051413db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): picked mon.noname-c con 0x7f05140a10f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:22.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05140a1b90 con 0x7f05141360f0 2024-09-15T21:21:22.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f051409f6f0 con 0x7f05140a10f0 2024-09-15T21:21:22.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f05140a1e70 con 0x7f0514135d10 2024-09-15T21:21:22.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f051a740640 10 monclient(hunting): authenticate will time out at 1201.043823s 2024-09-15T21:21:22.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.242+0000 7f0518cdd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05140a10f0 0x7f051413db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05140a10f0 0x7f051413db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49056/0 (socket says 172.21.15.119:49056) 2024-09-15T21:21:22.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 -- 172.21.15.119:0/3861048484 learned_addr learned my addr 172.21.15.119:0/3861048484 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:22.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): get_auth_request con 0x7f05140a10f0 auth_method 0 2024-09-15T21:21:22.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0513fff640 1 --2- 172.21.15.119:0/3861048484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0514135d10 0x7f051406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0513fff640 10 monclient(hunting): get_auth_request con 0x7f0514135d10 auth_method 0 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0513fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0513fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0513fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient(hunting): handle_auth_done global_id 34789 payload 274 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 -- 172.21.15.119:0/3861048484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0514135d10 msgr2=0x7f051406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 --2- 172.21.15.119:0/3861048484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0514135d10 0x7f051406ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 -- 172.21.15.119:0/3861048484 >> v1:172.21.15.17:6789/0 conn(0x7f05141360f0 legacy=0x7f05140a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 monclient: found mon.noname-c 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 -- 172.21.15.119:0/3861048484 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05140a2150 con 0x7f05140a10f0 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0518cdd640 1 --2- 172.21.15.119:0/3861048484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05140a10f0 0x7f051413db00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f050c009e50 tx=0x7f050c01c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6101b49b69eb318d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 1 -- 172.21.15.119:0/3861048484 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c0a56c0 con 0x7f05140a10f0 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: dump: 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.275 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 1 -- 172.21.15.119:0/3861048484 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050c0a5860 con 0x7f05140a10f0 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 5 monclient: authenticate success, global_id 34789 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:22.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 4 set_mon_vals no callback set 2024-09-15T21:21:22.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 1 -- 172.21.15.119:0/3861048484 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c0a5c30 con 0x7f05140a10f0 2024-09-15T21:21:22.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f0512ffd640 10 monclient: dump: 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.278 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 10 monclient: shutdown 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 -- 172.21.15.119:0/3861048484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05140a10f0 msgr2=0x7f051413db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 --2- 172.21.15.119:0/3861048484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05140a10f0 0x7f051413db00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f050c009e50 tx=0x7f050c01c8f0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 -- 172.21.15.119:0/3861048484 shutdown_connections 2024-09-15T21:21:22.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 --2- 172.21.15.119:0/3861048484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05140a10f0 0x7f051413db00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 --2- 172.21.15.119:0/3861048484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0514135d10 0x7f051406ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 -- 172.21.15.119:0/3861048484 >> 172.21.15.119:0/3861048484 conn(0x7f051406cdc0 msgr2=0x7f051406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 -- 172.21.15.119:0/3861048484 shutdown_connections 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.246+0000 7f051a740640 1 -- 172.21.15.119:0/3861048484 wait complete. 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: monmap: 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:22.281 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 Processor -- start 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- start start 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: init 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05140a10f0 0x7f0514132060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): picked mon.noname-c con 0x7f05140a10f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 0x7f05141325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): picked mon.noname-b con 0x7f0514135d10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05141360f0 0x7f051413a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): picked mon.noname-a con 0x7f05141360f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0514136680 con 0x7f05141360f0 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0514136500 con 0x7f0514135d10 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0514140860 con 0x7f05140a10f0 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient(hunting): authenticate will time out at 1201.049073s 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 0x7f05141325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 0x7f05141325a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49068/0 (socket says 172.21.15.119:49068) 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 -- 172.21.15.119:0/2726422859 learned_addr learned my addr 172.21.15.119:0/2726422859 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): get_auth_request con 0x7f0514135d10 auth_method 0 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05140a10f0 0x7f0514132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05141360f0 0x7f051413a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): get_auth_request con 0x7f05141360f0 auth_method 0 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 10 monclient(hunting): get_auth_request con 0x7f05140a10f0 auth_method 0 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0518cdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient(hunting): handle_auth_done global_id 34792 payload 824 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 -- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05140a10f0 msgr2=0x7f0514132060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05140a10f0 0x7f0514132060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 -- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05141360f0 msgr2=0x7f051413a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05141360f0 0x7f051413a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 -- 172.21.15.119:0/2726422859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051411cd90 con 0x7f0514135d10 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05137fe640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 0x7f05141325a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f050800c400 tx=0x7f050800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508017070 con 0x7f0514135d10 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:22.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: dump: 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.289 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05080052f0 con 0x7f0514135d10 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508003840 con 0x7f0514135d10 2024-09-15T21:21:22.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 10 monclient: dump: 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.291 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 5 monclient: authenticate success, global_id 34792 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _renew_subs 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f05127fc640 4 set_mon_vals no callback set 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051413d8c0 con 0x7f0514135d10 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _renew_subs 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051411cab0 con 0x7f0514135d10 2024-09-15T21:21:22.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051406ec30 con 0x7f0514135d10 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f050801a070 con 0x7f0514135d10 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f050003b9c0 0x7f050003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7f050804f590 con 0x7f0514135d10 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f050003b9c0 0x7f050003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 10 monclient: get_auth_request con 0x7f050003b9c0 auth_method 0 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.250+0000 7f0513fff640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f050003b9c0 0x7f050003de80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f050400b9d0 tx=0x7f0504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.254+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f05080115e0 con 0x7f0514135d10 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.254+0000 7f0510ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.254+0000 7f0510ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:22.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient: get_monmap_and_config 2024-09-15T21:21:22.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient: monmap: 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:22.255158+0000 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:22.255158+0000 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:22.303 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 Processor -- start 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 -- start start 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient: init 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f194c0a0890 0x7f194c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): picked mon.noname-a con 0x7f194c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f194c09ffc0 0x7f194c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): picked mon.noname-c con 0x7f194c09ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f194c135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194c0a3130 con 0x7f194c135670 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f194c0a16f0 con 0x7f194c0a0890 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f194c073df0 con 0x7f194c09ffc0 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f19518c1640 10 monclient(hunting): authenticate will time out at 1205.650908s 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f194c0a0890 0x7f194c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f194c135670 0x7f194c135aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44270/0 (socket says 172.21.15.17:44270) 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f194c0a0890 0x7f194c0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45060/0 (socket says 172.21.15.17:45060) 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194b7fe640 1 -- 172.21.15.17:0/3796183930 learned_addr learned my addr 172.21.15.17:0/3796183930 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): get_auth_request con 0x7f194c0a0890 auth_method 0 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 1 --2- 172.21.15.17:0/3796183930 >> v2:172.21.15.119:3300/0 conn(0x7f194c09ffc0 0x7f194c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): get_auth_request con 0x7f194c09ffc0 auth_method 0 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient(hunting): handle_auth_done global_id 54135 payload 274 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 -- 172.21.15.17:0/3796183930 >> v2:172.21.15.119:3300/0 conn(0x7f194c09ffc0 msgr2=0x7f194c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 --2- 172.21.15.17:0/3796183930 >> v2:172.21.15.119:3300/0 conn(0x7f194c09ffc0 0x7f194c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 -- 172.21.15.17:0/3796183930 >> v1:172.21.15.17:6789/0 conn(0x7f194c135670 legacy=0x7f194c135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 monclient: found mon.noname-a 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f194affd640 1 -- 172.21.15.17:0/3796183930 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c0a1bf0 con 0x7f194c0a0890 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f1949ffb640 1 -- 172.21.15.17:0/3796183930 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081408675 0 0) 0x7f194c0a3130 con 0x7f194c135670 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.253+0000 7f1949ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f194affd640 1 --2- 172.21.15.17:0/3796183930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c0a0890 0x7f194c0a0c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f193c0097c0 tx=0x7f193c01c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700bbc1de4a975d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 1 -- 172.21.15.17:0/3796183930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b0070 con 0x7f194c0a0890 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: dump: 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.308 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 1 -- 172.21.15.17:0/3796183930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c0ab600 con 0x7f194c0a0890 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 5 monclient: authenticate success, global_id 54135 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 4 set_mon_vals no callback set 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 1 -- 172.21.15.17:0/3796183930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0ab9d0 con 0x7f194c0a0890 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f1949ffb640 10 monclient: dump: 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: shutdown 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- 172.21.15.17:0/3796183930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c0a0890 msgr2=0x7f194c0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 --2- 172.21.15.17:0/3796183930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c0a0890 0x7f194c0a0c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f193c0097c0 tx=0x7f193c01c320 comp rx=0 tx=0).stop 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- 172.21.15.17:0/3796183930 shutdown_connections 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 --2- 172.21.15.17:0/3796183930 >> v2:172.21.15.119:3300/0 conn(0x7f194c09ffc0 0x7f194c0a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 --2- 172.21.15.17:0/3796183930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c0a0890 0x7f194c0a0c70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- 172.21.15.17:0/3796183930 >> 172.21.15.17:0/3796183930 conn(0x7f194c0a66a0 msgr2=0x7f194c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- 172.21.15.17:0/3796183930 shutdown_connections 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- 172.21.15.17:0/3796183930 wait complete. 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: monmap: 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 Processor -- start 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- start start 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: init 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c09ffc0 0x7f194c133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): picked mon.noname-a con 0x7f194c09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c0a0890 0x7f194c134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): picked mon.noname-b con 0x7f194c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194c135670 0x7f194c134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): picked mon.noname-c con 0x7f194c135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f194c140690 con 0x7f194c09ffc0 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f194c140510 con 0x7f194c0a0890 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f194c140810 con 0x7f194c135670 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.257+0000 7f19518c1640 10 monclient(hunting): authenticate will time out at 1205.655931s 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c09ffc0 0x7f194c133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c09ffc0 0x7f194c133d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45072/0 (socket says 172.21.15.17:45072) 2024-09-15T21:21:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 -- 172.21.15.17:0/2588099263 learned_addr learned my addr 172.21.15.17:0/2588099263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): get_auth_request con 0x7f194c09ffc0 auth_method 0 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194c135670 0x7f194c134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): get_auth_request con 0x7f194c135670 auth_method 0 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c0a0890 0x7f194c134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): get_auth_request con 0x7f194c0a0890 auth_method 0 2024-09-15T21:21:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient(hunting): handle_auth_done global_id 54141 payload 824 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194c135670 msgr2=0x7f194c134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194c135670 0x7f194c134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c0a0890 msgr2=0x7f194c134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c0a0890 0x7f194c134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 monclient: found mon.noname-a 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c13d140 con 0x7f194c09ffc0 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c0a0890 0x7f194c134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194affd640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c09ffc0 0x7f194c133d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f193c0ae040 tx=0x7f193c0190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b0040 con 0x7f194c09ffc0 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: dump: 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c0b9070 con 0x7f194c09ffc0 2024-09-15T21:21:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b4850 con 0x7f194c09ffc0 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 5 monclient: authenticate success, global_id 54141 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 10 monclient: _renew_subs 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194c11eeb0 con 0x7f194c09ffc0 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19497fa640 4 set_mon_vals no callback set 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: dump: 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 10 monclient: _renew_subs 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194c11d8c0 con 0x7f194c09ffc0 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 0x7f194c063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f194c09d0f0 con 0x7f194c13c950 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f193c0b49f0 con 0x7f194c09ffc0 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 0x7f194c063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient: get_auth_request con 0x7f194c13c950 auth_method 0 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f193803b9c0 0x7f193803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7f193c0ea1f0 con 0x7f194c09ffc0 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 10 monclient(hunting): handle_auth_done global_id 34801 payload 824 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f194c135aa0 con 0x7f194c13c950 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f193803b9c0 0x7f193803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 10 monclient: get_auth_request con 0x7f193803b9c0 auth_method 0 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194a7fc640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 0x7f194c063e00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1940011800 tx=0x7f1940011cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194000fa30 con 0x7f194c13c950 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: dump: 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f194b7fe640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f193803b9c0 0x7f193803de80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f19440012f0 tx=0x7f1944001680 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f194000fa30 con 0x7f194c13c950 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:22.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 msgr2=0x7f194c063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 0x7f194c063e00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1940011800 tx=0x7f1940011cc0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 msgr2=0x7f194c063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.261+0000 7f1933fff640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c13c950 0x7f194c063e00 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1940011800 tx=0x7f1940011cc0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 0x7f194c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f194c1391c0 con 0x7f194c064340 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 0x7f194c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient: get_auth_request con 0x7f194c064340 auth_method 0 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 10 monclient(hunting): handle_auth_done global_id 34804 payload 824 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 1 -- 172.21.15.17:0/2588099263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f194c06d020 con 0x7f194c064340 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194a7fc640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 0x7f194c02d450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1940011800 tx=0x7f19400046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940022570 con 0x7f194c064340 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 10 monclient: dump: 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f194002c450 con 0x7f194c064340 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 msgr2=0x7f194c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 0x7f194c02d450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1940011800 tx=0x7f19400046d0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 msgr2=0x7f194c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:22.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f1933fff640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c064340 0x7f194c02d450 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1940011800 tx=0x7f19400046d0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 94766, 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:21:22.327 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:22.328 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:22.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:22.330 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:22.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:22.332 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.333 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:22.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:22.335 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:22.337 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:21:22.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:21:22.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:22.340 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:22.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f193803b9c0 msgr2=0x7f193803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f193803b9c0 0x7f193803de80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f19440012f0 tx=0x7f1944001680 comp rx=0 tx=0).stop 2024-09-15T21:21:22.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 10 monclient: shutdown 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c09ffc0 msgr2=0x7f194c133d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 --2- 172.21.15.17:0/2588099263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194c09ffc0 0x7f194c133d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f193c0ae040 tx=0x7f193c0190c0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194b7fe640 1 -- 172.21.15.17:0/2588099263 reap_dead start 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f194b7fe640 1 -- 172.21.15.17:0/2588099263 reap_dead start 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 shutdown_connections 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 >> 172.21.15.17:0/2588099263 conn(0x7f194c0a66a0 msgr2=0x7f194c0a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 shutdown_connections 2024-09-15T21:21:22.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.269+0000 7f19518c1640 1 -- 172.21.15.17:0/2588099263 wait complete. 2024-09-15T21:21:22.344 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:21:22.344 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:21:22.344 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:21:22.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f051a740640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:22.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f051a740640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f051a740640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f051406c440 con 0x7f0514135d10 2024-09-15T21:21:22.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v111) ==== 56+0+1449 (secure 0 0 0) 0x7f050801acb0 con 0x7f0514135d10 2024-09-15T21:21:22.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f0510ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.426+0000 7f0510ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.461+0000 7f7f65dcc640 10 monclient: get_monmap_and_config 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.461+0000 7f7f65dcc640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.461+0000 7f7f65dcc640 10 monclient: monmap: 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:22.465414+0000 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:22.465414+0000 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.467 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:22.468 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:22.468 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:22.468 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:22.468 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:22.468 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.461+0000 7f7f65dcc640 1 Processor -- start 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 1 -- start start 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient: init 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f600a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): picked mon.noname-b con 0x7f7f6009d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7f60133fb0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7f60139f60 0x7f7f6013c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): picked mon.noname-c con 0x7f7f60139f60 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:22.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6009e1b0 con 0x7f7f60133fb0 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6009d520 con 0x7f7f60139f60 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6009e490 con 0x7f7f6009d6c0 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient(hunting): authenticate will time out at 1205.860760s 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f600a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f600a30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34656/0 (socket says 172.21.15.17:34656) 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 -- 172.21.15.17:0/4168284136 learned_addr learned my addr 172.21.15.17:0/4168284136 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): get_auth_request con 0x7f7f6009d6c0 auth_method 0 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 1 --2- 172.21.15.17:0/4168284136 >> v2:172.21.15.119:3300/0 conn(0x7f7f60139f60 0x7f7f6013c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): get_auth_request con 0x7f7f60139f60 auth_method 0 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient(hunting): handle_auth_done global_id 44303 payload 274 2024-09-15T21:21:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 -- 172.21.15.17:0/4168284136 >> v2:172.21.15.119:3300/0 conn(0x7f7f60139f60 msgr2=0x7f7f6013c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 --2- 172.21.15.17:0/4168284136 >> v2:172.21.15.119:3300/0 conn(0x7f7f60139f60 0x7f7f6013c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 -- 172.21.15.17:0/4168284136 >> v1:172.21.15.119:6789/0 conn(0x7f7f60133fb0 legacy=0x7f7f601343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 -- 172.21.15.17:0/4168284136 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6009e770 con 0x7f7f6009d6c0 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5f7fe640 1 --2- 172.21.15.17:0/4168284136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f600a30a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009870 tx=0x7f7f5001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe5c422abc9b206 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 1 -- 172.21.15.17:0/4168284136 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500ab640 con 0x7f7f6009d6c0 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: dump: 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 1 -- 172.21.15.17:0/4168284136 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f500b0070 con 0x7f7f6009d6c0 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 5 monclient: authenticate success, global_id 44303 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 1 -- 172.21.15.17:0/4168284136 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500aba00 con 0x7f7f6009d6c0 2024-09-15T21:21:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f65dcc640 4 set_mon_vals no callback set 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.465+0000 7f7f5e7fc640 10 monclient: dump: 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: shutdown 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/4168284136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 msgr2=0x7f7f600a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/4168284136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f600a30a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009870 tx=0x7f7f5001c420 comp rx=0 tx=0).stop 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/4168284136 shutdown_connections 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/4168284136 >> v2:172.21.15.119:3300/0 conn(0x7f7f60139f60 0x7f7f6013c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/4168284136 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f600a30a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/4168284136 >> 172.21.15.17:0/4168284136 conn(0x7f7f600a66a0 msgr2=0x7f7f600a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/4168284136 shutdown_connections 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/4168284136 wait complete. 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: monmap: 2024-09-15T21:21:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 Processor -- start 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- start start 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: init 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f601cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): picked mon.noname-c con 0x7f7f6009d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 0x7f7f6012fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): picked mon.noname-a con 0x7f7f60133fb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f60139f60 0x7f7f60130140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): picked mon.noname-b con 0x7f7f60139f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6013f0b0 con 0x7f7f60133fb0 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f600a27a0 con 0x7f7f60139f60 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6013f230 con 0x7f7f6009d6c0 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient(hunting): authenticate will time out at 1205.865734s 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 0x7f7f6012fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 0x7f7f6012fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45086/0 (socket says 172.21.15.17:45086) 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 -- 172.21.15.17:0/3486552997 learned_addr learned my addr 172.21.15.17:0/3486552997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): get_auth_request con 0x7f7f60133fb0 auth_method 0 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5f7fe640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f601cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5f7fe640 10 monclient(hunting): get_auth_request con 0x7f7f6009d6c0 auth_method 0 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5ffff640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f60139f60 0x7f7f60130140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5ffff640 10 monclient(hunting): get_auth_request con 0x7f7f60139f60 auth_method 0 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient(hunting): handle_auth_done global_id 54144 payload 824 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 -- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 msgr2=0x7f7f601cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f601cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 -- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f60139f60 msgr2=0x7f7f60130140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f60139f60 0x7f7f60130140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 monclient: found mon.noname-a 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 -- 172.21.15.17:0/3486552997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6011d5c0 con 0x7f7f60133fb0 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5effd640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 0x7f7f6012fc00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c7c0 tx=0x7f7f5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54003b80 con 0x7f7f60133fb0 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: dump: 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f54003d20 con 0x7f7f60133fb0 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400aa90 con 0x7f7f60133fb0 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 10 monclient: dump: 2024-09-15T21:21:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f5dffb640 4 set_mon_vals no callback set 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 5 monclient: authenticate success, global_id 54144 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _renew_subs 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6009d0f0 con 0x7f7f60133fb0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _renew_subs 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6011cd20 con 0x7f7f60133fb0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7f5400ac30 con 0x7f7f60133fb0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.469+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f600a30a0 con 0x7f7f60133fb0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.473+0000 7f7f47fff640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f4c03b9f0 0x7f7f4c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.473+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7f7f54050220 con 0x7f7f60133fb0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.473+0000 7f7f5f7fe640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f4c03b9f0 0x7f7f4c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.473+0000 7f7f5f7fe640 10 monclient: get_auth_request con 0x7f7f4c03b9f0 auth_method 0 2024-09-15T21:21:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.473+0000 7f7f5f7fe640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f4c03b9f0 0x7f7f4c03deb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f500af000 tx=0x7f7f500a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.477+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7f540052b0 con 0x7f7f60133fb0 2024-09-15T21:21:22.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.477+0000 7f7f47fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.477+0000 7f7f47fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:22.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:22.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f051413c830 con 0x7f0514135d10 2024-09-15T21:21:22.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f0510ff9640 1 -- 172.21.15.119:0/2726422859 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v111) ==== 56+0+1449 (secure 0 0 0) 0x7f050804f110 con 0x7f0514135d10 2024-09-15T21:21:22.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f0510ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f0510ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f050003b9c0 msgr2=0x7f050003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f050003b9c0 0x7f050003de80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f050400b9d0 tx=0x7f0504008040 comp rx=0 tx=0).stop 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 10 monclient: shutdown 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 msgr2=0x7f05141325a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.530+0000 7f051a740640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 0x7f05141325a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f050800c400 tx=0x7f050800c8c0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 shutdown_connections 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f050003b9c0 0x7f050003de80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05141360f0 0x7f051413a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0514135d10 0x7f05141325a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 --2- 172.21.15.119:0/2726422859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05140a10f0 0x7f0514132060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 >> 172.21.15.119:0/2726422859 conn(0x7f051406cdc0 msgr2=0x7f05140a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 shutdown_connections 2024-09-15T21:21:22.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.534+0000 7f051a740640 1 -- 172.21.15.119:0/2726422859 wait complete. 2024-09-15T21:21:22.553 INFO:tasks.workunit.client.0.smithi119.stdout: 12 123.00000 osd.12 down 1.00000 1.00000 2024-09-15T21:21:22.553 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:21:22.553 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep osd.14 2024-09-15T21:21:22.554 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 123 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient: get_monmap_and_config 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient: monmap: 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:22.662926+0000 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:22.662926+0000 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.664 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:22.665 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:22.665 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:22.665 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:22.665 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:22.665 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 Processor -- start 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 -- start start 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient: init 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f889c09f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): picked mon.noname-b con 0x7f889c09d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f889c09d8c0 0x7f889c09dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): picked mon.noname-a con 0x7f889c09d8c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:22.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f889c1347b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889c0a09e0 con 0x7f889c1347b0 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f889c074760 con 0x7f889c09d8c0 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f889c0a0f30 con 0x7f889c09d4e0 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f88a0fe7640 10 monclient(hunting): authenticate will time out at 1201.459536s 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889ad9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f889c1347b0 0x7f889c134c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59668/0 (socket says 172.21.15.119:59668) 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889ad9d640 1 -- 172.21.15.119:0/1240202040 learned_addr learned my addr 172.21.15.119:0/1240202040 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f8899d9b640 1 --2- 172.21.15.119:0/1240202040 >> v2:172.21.15.17:3300/0 conn(0x7f889c09d8c0 0x7f889c09dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f8899d9b640 10 monclient(hunting): get_auth_request con 0x7f889c09d8c0 auth_method 0 2024-09-15T21:21:22.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f8899d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f8899d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f8899d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889a59c640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f889c09f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915376466 0 0) 0x7f889c0a09e0 con 0x7f889c1347b0 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889959a640 10 monclient(hunting): _negotiate 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.658+0000 7f889959a640 10 monclient(hunting): my global_id is 34816 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8884003530 con 0x7f889c1347b0 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): get_auth_request con 0x7f889c09d4e0 auth_method 0 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4284133321 0 0) 0x7f8884003530 con 0x7f889c1347b0 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 msgr2=0x7f889c09f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f889c09f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 >> v2:172.21.15.17:3300/0 conn(0x7f889c09d8c0 msgr2=0x7f889c09dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 --2- 172.21.15.119:0/1240202040 >> v2:172.21.15.17:3300/0 conn(0x7f889c09d8c0 0x7f889c09dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:22.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c09e6e0 con 0x7f889c1347b0 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 5 monclient: authenticate success, global_id 34816 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f8894001a80 con 0x7f889c1347b0 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: dump: 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.678 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 >> v1:172.21.15.119:6789/0 conn(0x7f889c1347b0 legacy=0x7f889c134c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8884003a60 0x7f8884003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): picked mon.b con 0x7f8884003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8884004810 0x7f8884004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): picked mon.a con 0x7f8884004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f88840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): picked mon.c con 0x7f889c09d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f889c0a0f30 con 0x7f8884004810 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f888400bad0 con 0x7f8884003a60 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f888400bcd0 con 0x7f889c09d4e0 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8884003a60 0x7f8884003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:22.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): get_auth_request con 0x7f8884003a60 auth_method 0 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8884004810 0x7f8884004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): get_auth_request con 0x7f8884004810 auth_method 0 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f8899d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889ad9d640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f88840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient(hunting): handle_auth_done global_id 34816 payload 293 2024-09-15T21:21:22.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 -- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 msgr2=0x7f88840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f88840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 -- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8884004810 msgr2=0x7f8884004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8884004810 0x7f8884004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 monclient: found mon.b 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 -- 172.21.15.119:0/1240202040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8884003530 con 0x7f8884003a60 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889ad9d640 10 monclient: get_auth_request con 0x7f889c09d4e0 auth_method 0 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889ad9d640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f889c09d4e0 0x7f88840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889a59c640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8884003a60 0x7f8884003e80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f888c009ac0 tx=0x7f888c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1262f37f7334f5de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c009690 con 0x7f8884003a60 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: dump: 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c0a9430 con 0x7f8884003a60 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 4 set_mon_vals no callback set 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 1 -- 172.21.15.119:0/1240202040 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c0a9800 con 0x7f8884003a60 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889959a640 10 monclient: dump: 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.684 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 10 monclient: shutdown 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1240202040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8884003a60 msgr2=0x7f8884003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1240202040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8884003a60 0x7f8884003e80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f888c009ac0 tx=0x7f888c01ca30 comp rx=0 tx=0).stop 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f889ad9d640 1 -- 172.21.15.119:0/1240202040 reap_dead start 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1240202040 shutdown_connections 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1240202040 >> 172.21.15.119:0/1240202040 conn(0x7f889c06cdc0 msgr2=0x7f889c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1240202040 shutdown_connections 2024-09-15T21:21:22.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.662+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1240202040 wait complete. 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: build_initial_monmap 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: monmap: 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 Processor -- start 2024-09-15T21:21:22.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- start start 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: init 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8884003a60 0x7f889c13dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): picked mon.noname-c con 0x7f8884003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f889c13e250 0x7f889c13e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): picked mon.noname-a con 0x7f889c13e250 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 0x7f889c1d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): picked mon.noname-b con 0x7f889c1cfce0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f889c13ce80 con 0x7f889c13e250 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f889c13cd00 con 0x7f889c1cfce0 2024-09-15T21:21:22.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f889c13d000 con 0x7f8884003a60 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient(hunting): authenticate will time out at 1201.465446s 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 0x7f889c1d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 0x7f889c1d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49084/0 (socket says 172.21.15.119:49084) 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 -- 172.21.15.119:0/1095059568 learned_addr learned my addr 172.21.15.119:0/1095059568 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): get_auth_request con 0x7f889c1cfce0 auth_method 0 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:22.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f8899d9b640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f889c13e250 0x7f889c13e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f8899d9b640 10 monclient(hunting): get_auth_request con 0x7f889c13e250 auth_method 0 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f8899d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f8899d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f8899d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8884003a60 0x7f889c13dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 10 monclient(hunting): get_auth_request con 0x7f8884003a60 auth_method 0 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient(hunting): handle_auth_done global_id 34819 payload 824 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 -- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8884003a60 msgr2=0x7f889c13dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8884003a60 0x7f889c13dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 -- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f889c13e250 msgr2=0x7f889c13e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f889c13e250 0x7f889c13e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 -- 172.21.15.119:0/1095059568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c13a390 con 0x7f889c1cfce0 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f8899d9b640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f889c13e250 0x7f889c13e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889ad9d640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 0x7f889c1d0140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f889c134b90 tx=0x7f889400b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894004d70 con 0x7f889c1cfce0 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: dump: 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.690 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889400bb80 con 0x7f889c1cfce0 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 5 monclient: authenticate success, global_id 34819 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _renew_subs 2024-09-15T21:21:22.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c073a80 con 0x7f889c1cfce0 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894007110 con 0x7f889c1cfce0 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 10 monclient: dump: 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:22.692 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _renew_subs 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888bfff640 4 set_mon_vals no callback set 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c139af0 con 0x7f889c1cfce0 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f889401b020 con 0x7f889c1cfce0 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c09dcc0 con 0x7f889c1cfce0 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f887c03b9c0 0x7f887c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7f88940504c0 con 0x7f889c1cfce0 2024-09-15T21:21:22.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f887c03b9c0 0x7f887c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:22.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 10 monclient: get_auth_request con 0x7f887c03b9c0 auth_method 0 2024-09-15T21:21:22.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.666+0000 7f889a59c640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f887c03b9c0 0x7f887c03de80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f888c0ae000 tx=0x7f888c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:22.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.674+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8894014430 con 0x7f889c1cfce0 2024-09-15T21:21:22.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.674+0000 7f888affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:22.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.674+0000 7f888affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:22.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.721+0000 7f7f65dcc640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:21:22.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.721+0000 7f7f65dcc640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:21:22.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.721+0000 7f7f65dcc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:22.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.721+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f7f6011d0f0 con 0x7f7f60133fb0 2024-09-15T21:21:22.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f47fff640 1 -- 172.21.15.17:0/3486552997 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f7f54005080 con 0x7f7f60133fb0 2024-09-15T21:21:22.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f47fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:21:22.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f47fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:22.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=1765913581,config=3541143443,health=1361790972,kv=3079713637,logm=395983443}) 2024-09-15T21:21:22.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=1510096639,config=3541143443,health=1361790972,kv=3059833255,logm=835274763}) 2024-09-15T21:21:22.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=1765913581,config=3541143443,health=1361790972,kv=3079713637,logm=395983443}) 2024-09-15T21:21:22.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=11,kv=12,logm=63} crc {auth=1510096639,config=3541143444,health=3819404006,kv=3079713637,logm=236193893}) 2024-09-15T21:21:22.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f4c03b9f0 msgr2=0x7f7f4c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f4c03b9f0 0x7f7f4c03deb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f500af000 tx=0x7f7f500a7000 comp rx=0 tx=0).stop 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 10 monclient: shutdown 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 msgr2=0x7f7f6012fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 0x7f7f6012fc00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c7c0 tx=0x7f7f5400cc80 comp rx=0 tx=0).stop 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 shutdown_connections 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7f4c03b9f0 0x7f7f4c03deb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f60139f60 0x7f7f60130140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f60133fb0 0x7f7f6012fc00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 --2- 172.21.15.17:0/3486552997 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7f6009d6c0 0x7f7f601cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 >> 172.21.15.17:0/3486552997 conn(0x7f7f600a66a0 msgr2=0x7f7f6009f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 shutdown_connections 2024-09-15T21:21:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f7f65dcc640 1 -- 172.21.15.17:0/3486552997 wait complete. 2024-09-15T21:21:22.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3522653988}) 2024-09-15T21:21:22.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2183295615}) 2024-09-15T21:21:22.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3522653988}) 2024-09-15T21:21:22.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.725+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3466569510}) 2024-09-15T21:21:22.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1361855776}) 2024-09-15T21:21:22.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1071273805}) 2024-09-15T21:21:22.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1361855776}) 2024-09-15T21:21:22.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=340490837}) 2024-09-15T21:21:22.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2686261870}) 2024-09-15T21:21:22.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=11863999}) 2024-09-15T21:21:22.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2686261870}) 2024-09-15T21:21:22.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.729+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1374782538}) 2024-09-15T21:21:22.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568333367}) 2024-09-15T21:21:22.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=453128507}) 2024-09-15T21:21:22.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568333367}) 2024-09-15T21:21:22.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1025652811}) 2024-09-15T21:21:22.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400436139}) 2024-09-15T21:21:22.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=89683326}) 2024-09-15T21:21:22.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400436139}) 2024-09-15T21:21:22.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.733+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=92401917}) 2024-09-15T21:21:22.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3612557615}) 2024-09-15T21:21:22.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1171760097}) 2024-09-15T21:21:22.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3612557615}) 2024-09-15T21:21:22.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1676602016}) 2024-09-15T21:21:22.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2716670495}) 2024-09-15T21:21:22.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358312404}) 2024-09-15T21:21:22.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2716670495}) 2024-09-15T21:21:22.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1157882582}) 2024-09-15T21:21:22.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1057195157}) 2024-09-15T21:21:22.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2434482619}) 2024-09-15T21:21:22.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.744 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1057195157}) 2024-09-15T21:21:22.744 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.737+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2629666645}) 2024-09-15T21:21:22.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454497639}) 2024-09-15T21:21:22.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=359383159}) 2024-09-15T21:21:22.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454497639}) 2024-09-15T21:21:22.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=712514466}) 2024-09-15T21:21:22.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3528070186}) 2024-09-15T21:21:22.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3917553348}) 2024-09-15T21:21:22.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3528070186}) 2024-09-15T21:21:22.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.741+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3027013824}) 2024-09-15T21:21:22.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110875582}) 2024-09-15T21:21:22.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081579470}) 2024-09-15T21:21:22.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110875582}) 2024-09-15T21:21:22.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1386199218}) 2024-09-15T21:21:22.750 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:21:22.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3252226787}) 2024-09-15T21:21:22.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2780435593}) 2024-09-15T21:21:22.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3252226787}) 2024-09-15T21:21:22.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.745+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2339354246}) 2024-09-15T21:21:22.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2632979720}) 2024-09-15T21:21:22.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3144105402}) 2024-09-15T21:21:22.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2632979720}) 2024-09-15T21:21:22.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1218765901}) 2024-09-15T21:21:22.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385710525}) 2024-09-15T21:21:22.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2667655080}) 2024-09-15T21:21:22.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385710525}) 2024-09-15T21:21:22.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4044913452}) 2024-09-15T21:21:22.755 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.755 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676583667}) 2024-09-15T21:21:22.755 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3890943602}) 2024-09-15T21:21:22.755 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676583667}) 2024-09-15T21:21:22.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.749+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3815874993}) 2024-09-15T21:21:22.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855261503}) 2024-09-15T21:21:22.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=806457248}) 2024-09-15T21:21:22.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855261503}) 2024-09-15T21:21:22.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048303295}) 2024-09-15T21:21:22.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=67,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9} crc {logm=3120084288,mdsmap=173658045,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=508054519,mon_config_key=1582060230}) 2024-09-15T21:21:22.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=67,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=7} crc {logm=3657298894,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1000481764,mon_config_key=1793405092}) 2024-09-15T21:21:22.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=67,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9} crc {logm=3120084288,mdsmap=173658045,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=508054519,mon_config_key=1582060230}) 2024-09-15T21:21:22.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.753+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=69,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5} crc {logm=1484109473,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1000481764,mon_config_key=354525599}) 2024-09-15T21:21:22.761 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.757+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.757+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=81} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=3068027012,osd_pg_creating=3990483567,osd_snap=841190635,osdmap=2451282639}) 2024-09-15T21:21:22.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.757+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=80} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3990483567,osd_snap=841190635,osdmap=4072188079}) 2024-09-15T21:21:22.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.757+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.757+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=81} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=3068027012,osd_pg_creating=3990483567,osd_snap=841190635,osdmap=2451282639}) 2024-09-15T21:21:22.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.757+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=81} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=166226148,osd_pg_creating=3990483567,osd_snap=841190635,osdmap=327856610}) 2024-09-15T21:21:22.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.765+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.765+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=95} crc {osdmap=1389910540}) 2024-09-15T21:21:22.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.765+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=95} crc {osdmap=292830648}) 2024-09-15T21:21:22.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.765+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:21:22.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.765+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=95} crc {osdmap=1389910540}) 2024-09-15T21:21:22.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:22.765+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=95} crc {osdmap=3332172770}) 2024-09-15T21:21:22.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f88a0fe7640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:22.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f88a0fe7640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f88a0fe7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f889c06c440 con 0x7f889c1cfce0 2024-09-15T21:21:22.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v111) ==== 56+0+1449 (secure 0 0 0) 0x7f8894011a20 con 0x7f889c1cfce0 2024-09-15T21:21:22.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f888affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.834+0000 7f888affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:22.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f88a0fe7640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:22.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f88a0fe7640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f88a0fe7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:22.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f889c13f620 con 0x7f889c1cfce0 2024-09-15T21:21:22.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f888affd640 1 -- 172.21.15.119:0/1095059568 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v111) ==== 56+0+1449 (secure 0 0 0) 0x7f88940500c0 con 0x7f889c1cfce0 2024-09-15T21:21:22.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f888affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:22.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.930+0000 7f888affd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f887c03b9c0 msgr2=0x7f887c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f887c03b9c0 0x7f887c03de80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f888c0ae000 tx=0x7f888c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 10 monclient: shutdown 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 msgr2=0x7f889c1d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 0x7f889c1d0140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f889c134b90 tx=0x7f889400b4a0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 shutdown_connections 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f887c03b9c0 0x7f887c03de80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c1cfce0 0x7f889c1d0140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f889c13e250 0x7f889c13e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 --2- 172.21.15.119:0/1095059568 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8884003a60 0x7f889c13dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:22.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 >> 172.21.15.119:0/1095059568 conn(0x7f889c06cdc0 msgr2=0x7f889c0a5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:22.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 shutdown_connections 2024-09-15T21:21:22.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:22.934+0000 7f88a0fe7640 1 -- 172.21.15.119:0/1095059568 wait complete. 2024-09-15T21:21:22.952 INFO:tasks.workunit.client.0.smithi119.stdout: 14 123.00000 osd.14 down 1.00000 1.00000 2024-09-15T21:21:22.952 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush reweight-subtree foobaz 155 2024-09-15T21:21:23.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.054+0000 7fd70774b640 10 monclient: get_monmap_and_config 2024-09-15T21:21:23.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.054+0000 7fd70774b640 10 monclient: build_initial_monmap 2024-09-15T21:21:23.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.054+0000 7fd70774b640 10 monclient: monmap: 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:23.060482+0000 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:23.060482+0000 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:23.062 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 Processor -- start 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 -- start start 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient: init 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 0x7fd70009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): picked mon.noname-b con 0x7fd70009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:23.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd70009dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd700134de0 0x7fd70013c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): picked mon.noname-a con 0x7fd700134de0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd70006e240 con 0x7fd70009dd20 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd70006e000 con 0x7fd700134de0 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd70009e7f0 con 0x7fd70009d400 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 10 monclient(hunting): authenticate will time out at 1201.857363s 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd70009dd20 0x7fd7001346d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59684/0 (socket says 172.21.15.119:59684) 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 -- 172.21.15.119:0/1540580394 learned_addr learned my addr 172.21.15.119:0/1540580394 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 0x7fd70009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800802361 0 0) 0x7fd70006e240 con 0x7fd70009dd20 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): _negotiate 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): my global_id is 34822 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 1 --2- 172.21.15.119:0/1540580394 >> v2:172.21.15.17:3300/0 conn(0x7fd700134de0 0x7fd70013c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6ec003530 con 0x7fd70009dd20 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): get_auth_request con 0x7fd70009d400 auth_method 0 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): get_auth_request con 0x7fd700134de0 auth_method 0 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2617776908 0 0) 0x7fd6ec003530 con 0x7fd70009dd20 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 msgr2=0x7fd70009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 0x7fd70009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 >> v2:172.21.15.17:3300/0 conn(0x7fd700134de0 msgr2=0x7fd70013c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 --2- 172.21.15.119:0/1540580394 >> v2:172.21.15.17:3300/0 conn(0x7fd700134de0 0x7fd70013c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:23.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70009e140 con 0x7fd70009dd20 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fd6f0001700 con 0x7fd70009dd20 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 5 monclient: authenticate success, global_id 34822 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: dump: 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 >> v1:172.21.15.119:6789/0 conn(0x7fd70009dd20 legacy=0x7fd7001346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:23.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6ec003a10 0x7fd6ec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): picked mon.a con 0x7fd6ec003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec004810 0x7fd6ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): picked mon.b con 0x7fd6ec004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 0x7fd6ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): picked mon.c con 0x7fd70009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd70006e000 con 0x7fd6ec003a10 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd70009e7f0 con 0x7fd6ec004810 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec00bb00 con 0x7fd70009d400 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd6f7fff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:23.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd70774b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec004810 0x7fd6ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): get_auth_request con 0x7fd6ec004810 auth_method 0 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6ec003a10 0x7fd6ec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): get_auth_request con 0x7fd6ec003a10 auth_method 0 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd7054e7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:23.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient(hunting): handle_auth_done global_id 34822 payload 293 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 -- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 msgr2=0x7fd6ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd705ce8640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 0x7fd6ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd70009d400 0x7fd6ec0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 -- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6ec003a10 msgr2=0x7fd6ec003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6ec003a10 0x7fd6ec003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient: _finish_hunting 0 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 monclient: found mon.b 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 1 -- 172.21.15.119:0/1540580394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec003530 con 0x7fd6ec004810 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient: _finish_auth 0 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 10 monclient: _check_auth_tickets 2024-09-15T21:21:23.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.058+0000 7fd704ce6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec004810 0x7fd6ec004c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c210 tx=0x7fd6f001cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a12b2035a8104f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0007d60 con 0x7fd6ec004810 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: dump: 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.077 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f00a9070 con 0x7fd6ec004810 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 1 -- 172.21.15.119:0/1540580394 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0005d40 con 0x7fd6ec004810 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f7fff640 10 monclient: dump: 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.078 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 4 set_mon_vals no callback set 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: shutdown 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/1540580394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec004810 msgr2=0x7fd6ec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 --2- 172.21.15.119:0/1540580394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec004810 0x7fd6ec004c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c210 tx=0x7fd6f001cbc0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd705ce8640 1 -- 172.21.15.119:0/1540580394 reap_dead start 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/1540580394 shutdown_connections 2024-09-15T21:21:23.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/1540580394 >> 172.21.15.119:0/1540580394 conn(0x7fd70009be60 msgr2=0x7fd70009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/1540580394 shutdown_connections 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/1540580394 wait complete. 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: build_initial_monmap 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: monmap: 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:23.080 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 Processor -- start 2024-09-15T21:21:23.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- start start 2024-09-15T21:21:23.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: init 2024-09-15T21:21:23.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd70012fec0 0x7fd7001302a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): picked mon.noname-a con 0x7fd70012fec0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 0x7fd700138570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): picked mon.noname-b con 0x7fd7001307e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd700138ab0 0x7fd700138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): picked mon.noname-c con 0x7fd700138ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000a0360 con 0x7fd70012fec0 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000a01e0 con 0x7fd7001307e0 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd70013d6f0 con 0x7fd700138ab0 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient(hunting): authenticate will time out at 1201.863051s 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 0x7fd700138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 0x7fd700138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49106/0 (socket says 172.21.15.119:49106) 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 -- 172.21.15.119:0/853080848 learned_addr learned my addr 172.21.15.119:0/853080848 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): get_auth_request con 0x7fd7001307e0 auth_method 0 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd7054e7640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd70012fec0 0x7fd7001302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd705ce8640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd700138ab0 0x7fd700138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd7054e7640 10 monclient(hunting): get_auth_request con 0x7fd70012fec0 auth_method 0 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd7054e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd7054e7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd7054e7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient(hunting): handle_auth_done global_id 34825 payload 824 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 -- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd700138ab0 msgr2=0x7fd700138f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd700138ab0 0x7fd700138f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 -- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd70012fec0 msgr2=0x7fd7001302a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd70012fec0 0x7fd7001302a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient: _finish_hunting 0 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 monclient: found mon.noname-b 2024-09-15T21:21:23.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 -- 172.21.15.119:0/853080848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70013bd80 con 0x7fd7001307e0 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd705ce8640 10 monclient: get_auth_request con 0x7fd700138ab0 auth_method 0 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd705ce8640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd700138ab0 0x7fd700138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd704ce6640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 0x7fd700138570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0000c10 tx=0x7fd6ec005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0007d60 con 0x7fd7001307e0 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: dump: 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.088 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f0001070 con 0x7fd7001307e0 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0004ad0 con 0x7fd7001307e0 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f5ffb640 10 monclient: dump: 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 5 monclient: authenticate success, global_id 34825 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: _renew_subs 2024-09-15T21:21:23.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd6f77fe640 4 set_mon_vals no callback set 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70013c060 con 0x7fd7001307e0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: _renew_subs 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.062+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70009d0f0 con 0x7fd7001307e0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd70774b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd70774b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd70774b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7000a4ea0 con 0x7fd7001307e0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd6f0004c70 con 0x7fd7001307e0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd6f5ffb640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7fd6f00e9750 con 0x7fd7001307e0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd7054e7640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd7054e7640 10 monclient: get_auth_request con 0x7fd6e403b9f0 auth_method 0 2024-09-15T21:21:23.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.066+0000 7fd7054e7640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0098d0 tx=0x7fd6fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:23.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.070+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd6f0016970 con 0x7fd7001307e0 2024-09-15T21:21:23.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.070+0000 7fd6f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:23.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.070+0000 7fd6f5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:23.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.242+0000 7fd70774b640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T21:21:23.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.242+0000 7fd70774b640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T21:21:23.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.242+0000 7fd70774b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:23.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:23.242+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) -- 0x7fd70013db30 con 0x7fd7001307e0 2024-09-15T21:21:23.751 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:21:23.751 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient: get_monmap_and_config 2024-09-15T21:21:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient: build_initial_monmap 2024-09-15T21:21:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient: monmap: 2024-09-15T21:21:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:23.879825+0000 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:23.879825+0000 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:23.882 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 Processor -- start 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 -- start start 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient: init 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2300a0890 0x7fd2300a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): picked mon.noname-c con 0x7fd2300a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2300a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): picked mon.noname-b con 0x7fd23009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 0x7fd230135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): picked mon.noname-a con 0x7fd230135670 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300a16f0 con 0x7fd230135670 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300a3130 con 0x7fd2300a0890 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd230073df0 con 0x7fd23009ffc0 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd236f0a640 10 monclient(hunting): authenticate will time out at 1207.275243s 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2300a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 0x7fd230135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 0x7fd230135aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45098/0 (socket says 172.21.15.17:45098) 2024-09-15T21:21:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2300a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34676/0 (socket says 172.21.15.17:34676) 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 1 -- 172.21.15.17:0/3805389921 learned_addr learned my addr 172.21.15.17:0/3805389921 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): get_auth_request con 0x7fd230135670 auth_method 0 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): get_auth_request con 0x7fd23009ffc0 auth_method 0 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd234ca6640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.119:3300/0 conn(0x7fd2300a0890 0x7fd2300a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient(hunting): handle_auth_done global_id 44318 payload 274 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 -- 172.21.15.17:0/3805389921 >> v2:172.21.15.119:3300/0 conn(0x7fd2300a0890 msgr2=0x7fd2300a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.119:3300/0 conn(0x7fd2300a0890 0x7fd2300a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 -- 172.21.15.17:0/3805389921 >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 msgr2=0x7fd230135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 0x7fd230135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 monclient: found mon.noname-b 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd22ffff640 1 -- 172.21.15.17:0/3805389921 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2300a1bf0 con 0x7fd23009ffc0 2024-09-15T21:21:23.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd2354a7640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 0x7fd230135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:23.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd234ca6640 10 monclient: get_auth_request con 0x7fd2300a0890 auth_method 0 2024-09-15T21:21:23.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.877+0000 7fd234ca6640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.119:3300/0 conn(0x7fd2300a0890 0x7fd2300a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22ffff640 1 --2- 172.21.15.17:0/3805389921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2300a11b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd23009d0a0 tx=0x7fd22001c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7274b65ecf3b622 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 1 -- 172.21.15.17:0/3805389921 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b2070 con 0x7fd23009ffc0 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: dump: 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 1 -- 172.21.15.17:0/3805389921 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2200a5780 con 0x7fd23009ffc0 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 1 -- 172.21.15.17:0/3805389921 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200a5b50 con 0x7fd23009ffc0 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 5 monclient: authenticate success, global_id 44318 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:23.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 4 set_mon_vals no callback set 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd22f7fe640 10 monclient: dump: 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.896 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 10 monclient: shutdown 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 -- 172.21.15.17:0/3805389921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 msgr2=0x7fd2300a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 --2- 172.21.15.17:0/3805389921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2300a11b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd23009d0a0 tx=0x7fd22001c7f0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 -- 172.21.15.17:0/3805389921 shutdown_connections 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.17:3300/0 conn(0x7fd230135670 0x7fd230135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 --2- 172.21.15.17:0/3805389921 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2300a11b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 --2- 172.21.15.17:0/3805389921 >> v2:172.21.15.119:3300/0 conn(0x7fd2300a0890 0x7fd2300a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 -- 172.21.15.17:0/3805389921 >> 172.21.15.17:0/3805389921 conn(0x7fd2300a66a0 msgr2=0x7fd2300a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 -- 172.21.15.17:0/3805389921 shutdown_connections 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.881+0000 7fd236f0a640 1 -- 172.21.15.17:0/3805389921 wait complete. 2024-09-15T21:21:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: build_initial_monmap 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: monmap: 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 Processor -- start 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- start start 2024-09-15T21:21:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: init 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2301cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): picked mon.noname-c con 0x7fd23009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2300a0890 0x7fd230134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): picked mon.noname-a con 0x7fd2300a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd230135670 0x7fd23009d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): picked mon.noname-b con 0x7fd230135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd23013f680 con 0x7fd2300a0890 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd23013f500 con 0x7fd230135670 2024-09-15T21:21:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd23013f800 con 0x7fd23009ffc0 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient(hunting): authenticate will time out at 1207.281398s 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2301cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2301cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34690/0 (socket says 172.21.15.17:34690) 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 -- 172.21.15.17:0/2245623200 learned_addr learned my addr 172.21.15.17:0/2245623200 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): get_auth_request con 0x7fd23009ffc0 auth_method 0 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2300a0890 0x7fd230134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): get_auth_request con 0x7fd2300a0890 auth_method 0 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd2354a7640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd230135670 0x7fd23009d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient(hunting): handle_auth_done global_id 44321 payload 824 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 -- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd230135670 msgr2=0x7fd23009d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd230135670 0x7fd23009d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 -- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2300a0890 msgr2=0x7fd230134910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2300a0890 0x7fd230134910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient: _finish_hunting 0 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 monclient: found mon.noname-c 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 -- 172.21.15.17:0/2245623200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23009f3e0 con 0x7fd23009ffc0 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd2354a7640 10 monclient: get_auth_request con 0x7fd230135670 auth_method 0 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd2354a7640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd230135670 0x7fd23009d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22ffff640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2300a0890 0x7fd230134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd234ca6640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2301cfac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c7c0 tx=0x7fd22800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228007a30 con 0x7fd23009ffc0 2024-09-15T21:21:23.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: dump: 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.903 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd228007bd0 con 0x7fd23009ffc0 2024-09-15T21:21:23.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 5 monclient: authenticate success, global_id 44321 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _renew_subs 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd230073050 con 0x7fd23009ffc0 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2280113f0 con 0x7fd23009ffc0 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 10 monclient: dump: 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22effd640 4 set_mon_vals no callback set 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _renew_subs 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23009ec90 con 0x7fd23009ffc0 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd230135aa0 con 0x7fd23009ffc0 2024-09-15T21:21:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd228011590 con 0x7fd23009ffc0 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd21003b9c0 0x7fd21003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.885+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(111..111 src has 1..111) ==== 6643+0+0 (secure 0 0 0) 0x7fd228050550 con 0x7fd23009ffc0 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.889+0000 7fd22ffff640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd21003b9c0 0x7fd21003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.889+0000 7fd22ffff640 10 monclient: get_auth_request con 0x7fd21003b9c0 auth_method 0 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.889+0000 7fd22ffff640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd21003b9c0 0x7fd21003de80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2200af000 tx=0x7fd2200a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.893+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd230073050 con 0x7fd23009ffc0 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.893+0000 7fd22d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:23.893+0000 7fd22d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:24.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.018+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v112) ==== 168+0+0 (secure 0 0 0) 0x7fd6f00aece0 con 0x7fd7001307e0 2024-09-15T21:21:24.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.018+0000 7fd6f5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T21:21:24.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.018+0000 7fd6f5ffb640 10 monclient: _finish_command 2 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-09-15T21:21:24.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.118+0000 7fd70774b640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T21:21:24.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.118+0000 7fd70774b640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T21:21:24.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.118+0000 7fd70774b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:21:24.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.118+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) -- 0x7fd7000008e0 con 0x7fd7001307e0 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.129+0000 7fd236f0a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.129+0000 7fd236f0a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.129+0000 7fd236f0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.129+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd230121dc0 con 0x7fd23009ffc0 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd22d7fa640 1 -- 172.21.15.17:0/2245623200 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fd228015020 con 0x7fd23009ffc0 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd22d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd22d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:24.137 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd21003b9c0 msgr2=0x7fd21003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd21003b9c0 0x7fd21003de80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2200af000 tx=0x7fd2200a7070 comp rx=0 tx=0).stop 2024-09-15T21:21:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 10 monclient: shutdown 2024-09-15T21:21:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 msgr2=0x7fd2301cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2301cfac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c7c0 tx=0x7fd22800cc80 comp rx=0 tx=0).stop 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 shutdown_connections 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd21003b9c0 0x7fd21003de80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd230135670 0x7fd23009d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2300a0890 0x7fd230134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 --2- 172.21.15.17:0/2245623200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd23009ffc0 0x7fd2301cfac0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 >> 172.21.15.17:0/2245623200 conn(0x7fd2300a66a0 msgr2=0x7fd2300a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 shutdown_connections 2024-09-15T21:21:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.133+0000 7fd236f0a640 1 -- 172.21.15.17:0/2245623200 wait complete. 2024-09-15T21:21:24.155 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:21:24.155 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:21:24.155 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:21:24.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:21:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient: get_monmap_and_config 2024-09-15T21:21:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient: monmap: 2024-09-15T21:21:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:24.275672+0000 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:24.275672+0000 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:24.279 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 Processor -- start 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 -- start start 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient: init 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa64c0a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c09ffc0 0x7fa64c13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): picked mon.noname-b con 0x7fa64c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa64c0a1380 0x7fa64c13d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): picked mon.noname-c con 0x7fa64c0a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64c0a3130 con 0x7fa64c0a0890 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa64c073df0 con 0x7fa64c0a1380 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c0a1bf0 con 0x7fa64c09ffc0 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa65416b640 10 monclient(hunting): authenticate will time out at 1207.670683s 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651f07640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa64c0a0890 0x7fa64c0a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44280/0 (socket says 172.21.15.17:44280) 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651f07640 1 -- 172.21.15.17:0/356299923 learned_addr learned my addr 172.21.15.17:0/356299923 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 1 --2- 172.21.15.17:0/356299923 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c09ffc0 0x7fa64c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): get_auth_request con 0x7fa64c09ffc0 auth_method 0 2024-09-15T21:21:24.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa652708640 1 --2- 172.21.15.17:0/356299923 >> v2:172.21.15.119:3300/0 conn(0x7fa64c0a1380 0x7fa64c13d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa652708640 10 monclient(hunting): get_auth_request con 0x7fa64c0a1380 auth_method 0 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa652708640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa652708640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa652708640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient(hunting): handle_auth_done global_id 44324 payload 274 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 1 -- 172.21.15.17:0/356299923 >> v2:172.21.15.119:3300/0 conn(0x7fa64c0a1380 msgr2=0x7fa64c13d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 1 --2- 172.21.15.17:0/356299923 >> v2:172.21.15.119:3300/0 conn(0x7fa64c0a1380 0x7fa64c13d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 1 -- 172.21.15.17:0/356299923 >> v1:172.21.15.17:6789/0 conn(0x7fa64c0a0890 legacy=0x7fa64c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa650f05640 1 -- 172.21.15.17:0/356299923 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003813188 0 0) 0x7fa64c0a3130 con 0x7fa64c0a0890 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 1 monclient: found mon.noname-b 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa651706640 1 -- 172.21.15.17:0/356299923 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c09ccd0 con 0x7fa64c09ffc0 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.273+0000 7fa650f05640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa651706640 1 --2- 172.21.15.17:0/356299923 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c09ffc0 0x7fa64c13aac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097c0 tx=0x7fa64001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe913b8fe779bee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 1 -- 172.21.15.17:0/356299923 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400b0070 con 0x7fa64c09ffc0 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: dump: 2024-09-15T21:21:24.286 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.287 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 1 -- 172.21.15.17:0/356299923 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6400ab600 con 0x7fa64c09ffc0 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 5 monclient: authenticate success, global_id 44324 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 4 set_mon_vals no callback set 2024-09-15T21:21:24.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 1 -- 172.21.15.17:0/356299923 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400ab9d0 con 0x7fa64c09ffc0 2024-09-15T21:21:24.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa650f05640 10 monclient: dump: 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: shutdown 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- 172.21.15.17:0/356299923 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c09ffc0 msgr2=0x7fa64c13aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 --2- 172.21.15.17:0/356299923 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c09ffc0 0x7fa64c13aac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097c0 tx=0x7fa64001c360 comp rx=0 tx=0).stop 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- 172.21.15.17:0/356299923 shutdown_connections 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 --2- 172.21.15.17:0/356299923 >> v2:172.21.15.119:3300/0 conn(0x7fa64c0a1380 0x7fa64c13d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 --2- 172.21.15.17:0/356299923 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c09ffc0 0x7fa64c13aac0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- 172.21.15.17:0/356299923 >> 172.21.15.17:0/356299923 conn(0x7fa64c0a66a0 msgr2=0x7fa64c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- 172.21.15.17:0/356299923 shutdown_connections 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- 172.21.15.17:0/356299923 wait complete. 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: monmap: 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:24.292 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 Processor -- start 2024-09-15T21:21:24.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- start start 2024-09-15T21:21:24.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: init 2024-09-15T21:21:24.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09ffc0 0x7fa64c1cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): picked mon.noname-a con 0x7fa64c09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c0a0890 0x7fa64c135b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): picked mon.noname-b con 0x7fa64c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c0a1380 0x7fa64c1360d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): picked mon.noname-c con 0x7fa64c0a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c140100 con 0x7fa64c09ffc0 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c13ff80 con 0x7fa64c0a0890 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c140280 con 0x7fa64c0a1380 2024-09-15T21:21:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa65416b640 10 monclient(hunting): authenticate will time out at 1207.675813s 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c0a1380 0x7fa64c1360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa651f07640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09ffc0 0x7fa64c1cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c0a1380 0x7fa64c1360d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34704/0 (socket says 172.21.15.17:34704) 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa651f07640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09ffc0 0x7fa64c1cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45118/0 (socket says 172.21.15.17:45118) 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 1 -- 172.21.15.17:0/288001994 learned_addr learned my addr 172.21.15.17:0/288001994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 10 monclient(hunting): get_auth_request con 0x7fa64c0a1380 auth_method 0 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.277+0000 7fa652708640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): get_auth_request con 0x7fa64c09ffc0 auth_method 0 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c0a0890 0x7fa64c135b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 10 monclient(hunting): get_auth_request con 0x7fa64c0a0890 auth_method 0 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 10 monclient(hunting): handle_auth_done global_id 44327 payload 824 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c0a0890 msgr2=0x7fa64c135b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c0a0890 0x7fa64c135b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09ffc0 msgr2=0x7fa64c1cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09ffc0 0x7fa64c1cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 monclient: found mon.noname-c 2024-09-15T21:21:24.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c09eeb0 con 0x7fa64c0a1380 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09ffc0 0x7fa64c1cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa652708640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c0a1380 0x7fa64c1360d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa63c007e40 tx=0x7fa63c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c014620 con 0x7fa64c0a1380 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: dump: 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c019070 con 0x7fa64c0a1380 2024-09-15T21:21:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c014a20 con 0x7fa64c0a1380 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 5 monclient: authenticate success, global_id 44327 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 10 monclient: _renew_subs 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c09d0f0 con 0x7fa64c0a1380 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: dump: 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64bfff640 4 set_mon_vals no callback set 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 10 monclient: _renew_subs 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c120960 con 0x7fa64c0a1380 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 0x7fa64c09d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c121dd0 con 0x7fa64c09d4e0 2024-09-15T21:21:24.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 0x7fa64c09d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient: get_auth_request con 0x7fa64c09d4e0 auth_method 0 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa63c014bc0 con 0x7fa64c0a1380 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 10 monclient(hunting): handle_auth_done global_id 54171 payload 824 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa64c0a5b70 con 0x7fa64c09d4e0 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63803b9c0 0x7fa63803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(112..112 src has 1..112) ==== 6643+0+0 (secure 0 0 0) 0x7fa63c050ac0 con 0x7fa64c0a1380 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651f07640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 0x7fa64c09d8c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa644000f70 tx=0x7fa644001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64400fa70 con 0x7fa64c09d4e0 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: dump: 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63803b9c0 0x7fa63803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 10 monclient: get_auth_request con 0x7fa63803b9c0 auth_method 0 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa644018070 con 0x7fa64c09d4e0 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 msgr2=0x7fa64c09d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 0x7fa64c09d8c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa644000f70 tx=0x7fa644001cb0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 msgr2=0x7fa64c09d8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa64a7fc640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09d4e0 0x7fa64c09d8c0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa644000f70 tx=0x7fa644001cb0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.281+0000 7fa651706640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63803b9c0 0x7fa63803de80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6400ad000 tx=0x7fa640008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 0x7fa64c137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0a0c70 con 0x7fa64c09de00 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 0x7fa64c137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient: get_auth_request con 0x7fa64c09de00 auth_method 0 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 10 monclient(hunting): handle_auth_done global_id 54174 payload 824 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 1 -- 172.21.15.17:0/288001994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa64c06d020 con 0x7fa64c09de00 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa651f07640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 0x7fa64c137ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa644000f70 tx=0x7fa644001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644013600 con 0x7fa64c09de00 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 10 monclient: dump: 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7fa644013600 con 0x7fa64c09de00 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 msgr2=0x7fa64c137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 0x7fa64c137ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa644000f70 tx=0x7fa644001090 comp rx=0 tx=0).stop 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 msgr2=0x7fa64c137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa64a7fc640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa64c09de00 0x7fa64c137ab0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:21:24.317 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9651, 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.318 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:24.319 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.320 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:24.321 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:24.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:24.325 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.327 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.330 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63803b9c0 msgr2=0x7fa63803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63803b9c0 0x7fa63803de80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6400ad000 tx=0x7fa640008040 comp rx=0 tx=0).stop 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 10 monclient: shutdown 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c0a1380 msgr2=0x7fa64c1360d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 --2- 172.21.15.17:0/288001994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa64c0a1380 0x7fa64c1360d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa63c007e40 tx=0x7fa63c00e4b0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa652708640 1 -- 172.21.15.17:0/288001994 reap_dead start 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa652708640 1 -- 172.21.15.17:0/288001994 reap_dead start 2024-09-15T21:21:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 shutdown_connections 2024-09-15T21:21:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 >> 172.21.15.17:0/288001994 conn(0x7fa64c0a66a0 msgr2=0x7fa64c06fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 shutdown_connections 2024-09-15T21:21:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.289+0000 7fa65416b640 1 -- 172.21.15.17:0/288001994 wait complete. 2024-09-15T21:21:24.334 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:21:24.334 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:21:24.335 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient: get_monmap_and_config 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient: monmap: 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:24.454416+0000 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:24.454416+0000 2024-09-15T21:21:24.456 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:24.457 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 1 Processor -- start 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 1 -- start start 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient: init 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f95a00a4a90 0x7f95a00a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient(hunting): picked mon.noname-c con 0x7f95a00a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f95a00a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f95a00a5800 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a00a2c30 con 0x7f95a00a5800 2024-09-15T21:21:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a0138670 con 0x7f95a00a53b0 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f95a00a0f00 con 0x7f95a00a4a90 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 10 monclient(hunting): authenticate will time out at 1207.849822s 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f95a00a5800 0x7f95a0137f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44290/0 (socket says 172.21.15.17:44290) 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 1 -- 172.21.15.17:0/2200778514 learned_addr learned my addr 172.21.15.17:0/2200778514 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 --2- 172.21.15.17:0/2200778514 >> v2:172.21.15.119:3300/0 conn(0x7f95a00a4a90 0x7f95a00a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): get_auth_request con 0x7f95a00a4a90 auth_method 0 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349409860 0 0) 0x7f9590001120 con 0x7f95a00a5800 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): my global_id is 54177 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f958c003530 con 0x7f95a00a5800 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1065557785 0 0) 0x7f958c003530 con 0x7f95a00a5800 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 >> v2:172.21.15.119:3300/0 conn(0x7f95a00a4a90 msgr2=0x7f95a00a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 --2- 172.21.15.17:0/2200778514 >> v2:172.21.15.119:3300/0 conn(0x7f95a00a4a90 0x7f95a00a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 >> v1:172.21.15.119:6789/0 conn(0x7f95a00a53b0 legacy=0x7f95a0137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a00a19b0 con 0x7f95a00a5800 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f95900018d0 con 0x7f95a00a5800 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 5 monclient: authenticate success, global_id 54177 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: dump: 2024-09-15T21:21:24.466 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 >> v1:172.21.15.17:6789/0 conn(0x7f95a00a5800 legacy=0x7f95a0137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f958c003a60 0x7f958c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): picked mon.c con 0x7f958c003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f958c004810 0x7f958c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): picked mon.b con 0x7f958c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0052f0 0x7f958c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): picked mon.a con 0x7f958c0052f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00a0f00 con 0x7f958c0052f0 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f958c00c1c0 con 0x7f958c004810 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f958c00c3c0 con 0x7f958c003a60 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f95a5bb1640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f958c003a60 0x7f958c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): get_auth_request con 0x7f958c003a60 auth_method 0 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0052f0 0x7f958c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): get_auth_request con 0x7f958c0052f0 auth_method 0 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient(hunting): handle_auth_done global_id 54177 payload 293 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 -- 172.21.15.17:0/2200778514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f958c004810 msgr2=0x7f958c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959effd640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f958c004810 0x7f958c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f958c004810 0x7f958c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 -- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0052f0 msgr2=0x7f958c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0052f0 0x7f958c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 monclient: found mon.c 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 1 -- 172.21.15.17:0/2200778514 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c003530 con 0x7f958c003a60 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959effd640 10 monclient: get_auth_request con 0x7f958c004810 auth_method 0 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959effd640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f958c004810 0x7f958c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0052f0 0x7f958c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.453+0000 7f959ffff640 1 -- 172.21.15.17:0/2200778514 reap_dead start 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959f7fe640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f958c003a60 0x7f958c003e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9598009ab0 tx=0x7f959801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3900fde97c389fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980096b0 con 0x7f958c003a60 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: dump: 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.471 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95980a6410 con 0x7f958c003a60 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:24.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 4 set_mon_vals no callback set 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 1 -- 172.21.15.17:0/2200778514 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980a6800 con 0x7f958c003a60 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f959e7fc640 10 monclient: dump: 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.473 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: shutdown 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f958c003a60 msgr2=0x7f958c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f958c003a60 0x7f958c003e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9598009ab0 tx=0x7f959801ca30 comp rx=0 tx=0).stop 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- 172.21.15.17:0/2200778514 shutdown_connections 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 --2- 172.21.15.17:0/2200778514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f958c003a60 0x7f958c003e80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- 172.21.15.17:0/2200778514 >> 172.21.15.17:0/2200778514 conn(0x7f95a00aabb0 msgr2=0x7f95a00a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- 172.21.15.17:0/2200778514 shutdown_connections 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- 172.21.15.17:0/2200778514 wait complete. 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: monmap: 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:24.474 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 Processor -- start 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- start start 2024-09-15T21:21:24.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: init 2024-09-15T21:21:24.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:24.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c003a60 0x7f95a01cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): picked mon.noname-a con 0x7f958c003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01cd6e0 0x7f95a0124380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): picked mon.noname-c con 0x7f95a01cd6e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95a01248c0 0x7f95a0124d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): picked mon.noname-b con 0x7f95a01248c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00a4420 con 0x7f958c003a60 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00a1f70 con 0x7f95a01248c0 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f95a0128b50 con 0x7f95a01cd6e0 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.457+0000 7f95a5bb1640 10 monclient(hunting): authenticate will time out at 1207.855970s 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01cd6e0 0x7f95a0124380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c003a60 0x7f95a01cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01cd6e0 0x7f95a0124380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34724/0 (socket says 172.21.15.17:34724) 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c003a60 0x7f95a01cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45156/0 (socket says 172.21.15.17:45156) 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 -- 172.21.15.17:0/3473006150 learned_addr learned my addr 172.21.15.17:0/3473006150 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): get_auth_request con 0x7f95a01cd6e0 auth_method 0 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): get_auth_request con 0x7f958c003a60 auth_method 0 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95a01248c0 0x7f95a0124d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 10 monclient(hunting): get_auth_request con 0x7f95a01248c0 auth_method 0 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient(hunting): handle_auth_done global_id 54180 payload 824 2024-09-15T21:21:24.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01cd6e0 msgr2=0x7f95a0124380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01cd6e0 0x7f95a0124380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95a01248c0 msgr2=0x7f95a0124d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95a01248c0 0x7f95a0124d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a00a9790 con 0x7f958c003a60 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01cd6e0 0x7f95a0124380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959f7fe640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c003a60 0x7f95a01cd1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9598008000 tx=0x7f95980197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980be070 con 0x7f958c003a60 2024-09-15T21:21:24.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: dump: 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.486 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95980b9410 con 0x7f958c003a60 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980096b0 con 0x7f958c003a60 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 5 monclient: authenticate success, global_id 54180 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 10 monclient: _renew_subs 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a00a1280 con 0x7f958c003a60 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959dffb640 4 set_mon_vals no callback set 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: dump: 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 10 monclient: _renew_subs 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a0128150 con 0x7f958c003a60 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f95980c1070 con 0x7f958c003a60 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 0x7f95a01206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f95a0127d00 con 0x7f95a01252b0 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 0x7f95a01206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f958003b9c0 0x7f958003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient: get_auth_request con 0x7f95a01252b0 auth_method 0 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6643+0+0 (secure 0 0 0) 0x7f95980e9e80 con 0x7f958c003a60 2024-09-15T21:21:24.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 10 monclient(hunting): handle_auth_done global_id 44333 payload 824 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95a00a4e70 con 0x7f95a01252b0 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f958003b9c0 0x7f958003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 10 monclient: get_auth_request con 0x7f958003b9c0 auth_method 0 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959effd640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 0x7f95a01206e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f958800e3f0 tx=0x7f958800e8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588003b80 con 0x7f95a01252b0 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: dump: 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.490 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9588019070 con 0x7f95a01252b0 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 msgr2=0x7f95a01206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 0x7f95a01206e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f958800e3f0 tx=0x7f958800e8b0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 msgr2=0x7f95a01206e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f9597fff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a01252b0 0x7f95a01206e0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f958800e3f0 tx=0x7f958800e8b0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.461+0000 7f959ffff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f958003b9c0 0x7f958003de80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f95a00a15f0 tx=0x7f9590007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 0x7f95a0121030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f95a00a44d0 con 0x7f95a0120c20 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 0x7f95a0121030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient: get_auth_request con 0x7f95a0120c20 auth_method 0 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 10 monclient(hunting): handle_auth_done global_id 44336 payload 824 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 1 -- 172.21.15.17:0/3473006150 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f95a0069200 con 0x7f95a0120c20 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959effd640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 0x7f95a0121030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958800e3f0 tx=0x7f9588007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880056a0 con 0x7f95a0120c20 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 10 monclient: dump: 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f95880056a0 con 0x7f95a0120c20 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 msgr2=0x7f95a0121030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 0x7f95a0121030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958800e3f0 tx=0x7f9588007f90 comp rx=0 tx=0).stop 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 msgr2=0x7f95a0121030 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f9597fff640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95a0120c20 0x7f95a0121030 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958800e3f0 tx=0x7f9588007f90 comp rx=0 tx=0).stop 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:24.494 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 53067, 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:24.495 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:24.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.497 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:24.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:24.500 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:24.501 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.504 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.505 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.506 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.507 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:24.507 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:24.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f958003b9c0 msgr2=0x7f958003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f958003b9c0 0x7f958003de80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f95a00a15f0 tx=0x7f9590007ab0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 10 monclient: shutdown 2024-09-15T21:21:24.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c003a60 msgr2=0x7f95a01cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 --2- 172.21.15.17:0/3473006150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c003a60 0x7f95a01cd1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9598008000 tx=0x7f95980197d0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959ffff640 1 -- 172.21.15.17:0/3473006150 reap_dead start 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f959ffff640 1 -- 172.21.15.17:0/3473006150 reap_dead start 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 shutdown_connections 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 >> 172.21.15.17:0/3473006150 conn(0x7f95a00aabb0 msgr2=0x7f95a00a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 shutdown_connections 2024-09-15T21:21:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.469+0000 7f95a5bb1640 1 -- 172.21.15.17:0/3473006150 wait complete. 2024-09-15T21:21:24.509 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:21:24.509 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:21:24.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient: get_monmap_and_config 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient: monmap: 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:24.624176+0000 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:24.624176+0000 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:24.627 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:24.627 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:24.627 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 1 Processor -- start 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 1 -- start start 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient: init 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1d9c0a0890 0x7f1d9c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient(hunting): picked mon.noname-a con 0x7f1d9c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1d9c09ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1d9c0a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d9c136490 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d9c0a3130 con 0x7f1d9c09ffc0 2024-09-15T21:21:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1d9c073df0 con 0x7f1d9c0a0890 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da422e640 10 monclient(hunting): authenticate will time out at 1208.019888s 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.621+0000 7f1da1fca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1d9c0a0890 0x7f1d9c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da27cb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1d9c0a11b0 0x7f1d9c135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44296/0 (socket says 172.21.15.17:44296) 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da27cb640 1 -- 172.21.15.17:0/362169139 learned_addr learned my addr 172.21.15.17:0/362169139 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): get_auth_request con 0x7f1d9c0a0890 auth_method 0 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient(hunting): handle_auth_done global_id 54183 payload 274 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 1 -- 172.21.15.17:0/362169139 >> v1:172.21.15.119:6789/0 conn(0x7f1d9c09ffc0 legacy=0x7f1d9c135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 1 -- 172.21.15.17:0/362169139 >> v1:172.21.15.17:6789/0 conn(0x7f1d9c0a11b0 legacy=0x7f1d9c135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 1 monclient: found mon.noname-a 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 1 -- 172.21.15.17:0/362169139 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c0a1bf0 con 0x7f1d9c0a0890 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 1 -- 172.21.15.17:0/362169139 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2059558918 0 0) 0x7f1d9c136490 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da1fca640 1 --2- 172.21.15.17:0/362169139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a0890 0x7f1d9c0a0c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009dc0 tx=0x7f1d9401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89beb022658075f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 1 -- 172.21.15.17:0/362169139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b2070 con 0x7f1d9c0a0890 2024-09-15T21:21:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: dump: 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 5 monclient: authenticate success, global_id 54183 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 1 -- 172.21.15.17:0/362169139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d940a57e0 con 0x7f1d9c0a0890 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 4 set_mon_vals no callback set 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 1 -- 172.21.15.17:0/362169139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940a5bb0 con 0x7f1d9c0a0890 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da0fc8640 10 monclient: dump: 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 10 monclient: shutdown 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 -- 172.21.15.17:0/362169139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a0890 msgr2=0x7f1d9c0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 --2- 172.21.15.17:0/362169139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a0890 0x7f1d9c0a0c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009dc0 tx=0x7f1d9401c810 comp rx=0 tx=0).stop 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 -- 172.21.15.17:0/362169139 shutdown_connections 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 --2- 172.21.15.17:0/362169139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a0890 0x7f1d9c0a0c70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 -- 172.21.15.17:0/362169139 >> 172.21.15.17:0/362169139 conn(0x7f1d9c0a66a0 msgr2=0x7f1d9c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 -- 172.21.15.17:0/362169139 shutdown_connections 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.625+0000 7f1da422e640 1 -- 172.21.15.17:0/362169139 wait complete. 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: monmap: 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 Processor -- start 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- start start 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: init 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1d9c09ffc0 0x7f1d9c1cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): picked mon.noname-c con 0x7f1d9c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c0a0890 0x7f1d9c1d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): picked mon.noname-b con 0x7f1d9c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a11b0 0x7f1d9c1d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): picked mon.noname-a con 0x7f1d9c0a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c141ab0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c141930 con 0x7f1d9c0a0890 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c141c30 con 0x7f1d9c09ffc0 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): authenticate will time out at 1208.025065s 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1d9c09ffc0 0x7f1d9c1cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a11b0 0x7f1d9c1d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1d9c09ffc0 0x7f1d9c1cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34746/0 (socket says 172.21.15.17:34746) 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 1 -- 172.21.15.17:0/349491965 learned_addr learned my addr 172.21.15.17:0/349491965 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): get_auth_request con 0x7f1d9c09ffc0 auth_method 0 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): get_auth_request con 0x7f1d9c0a11b0 auth_method 0 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c0a0890 0x7f1d9c1d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 10 monclient(hunting): get_auth_request con 0x7f1d9c0a0890 auth_method 0 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient(hunting): handle_auth_done global_id 54189 payload 824 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1d9c09ffc0 msgr2=0x7f1d9c1cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1d9c09ffc0 0x7f1d9c1cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c0a0890 msgr2=0x7f1d9c1d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c0a0890 0x7f1d9c1d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 monclient: found mon.noname-a 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c09d0f0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1d9c09ffc0 0x7f1d9c1cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da27cb640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a11b0 0x7f1d9c1d07b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c3a0 tx=0x7f1d8c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0074e0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: dump: 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8c0047f0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c004be0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 10 monclient: dump: 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 5 monclient: authenticate success, global_id 54189 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: _renew_subs 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9bfff640 4 set_mon_vals no callback set 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c120db0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: _renew_subs 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c072630 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 0x7f1d9c134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c06fb00 con 0x7f1d9c09e5a0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1d8c011070 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 0x7f1d9c134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient: get_auth_request con 0x7f1d9c09e5a0 auth_method 0 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1d8803b9f0 0x7f1d8803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6643+0+0 (secure 0 0 0) 0x7f1d8c04fbd0 con 0x7f1d9c0a11b0 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 10 monclient(hunting): handle_auth_done global_id 34846 payload 824 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1d8803b9f0 0x7f1d8803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da1fca640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d9c0a0c70 con 0x7f1d9c09e5a0 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.629+0000 7f1da17c9640 10 monclient: get_auth_request con 0x7f1d8803b9f0 auth_method 0 2024-09-15T21:21:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1da1fca640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 0x7f1d9c134f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d9401aec0 tx=0x7f1d940195b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940bf070 con 0x7f1d9c09e5a0 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 10 monclient: dump: 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.651 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1da17c9640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1d8803b9f0 0x7f1d8803deb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c09f5e0 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1d940ad820 con 0x7f1d9c09e5a0 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 msgr2=0x7f1d9c134f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 0x7f1d9c134f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d9401aec0 tx=0x7f1d940195b0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 msgr2=0x7f1d9c134f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.633+0000 7f1d9a7fc640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c09e5a0 0x7f1d9c134f80 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d9401aec0 tx=0x7f1d940195b0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da422e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da422e640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 0x7f1d9c1358d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da422e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c135e10 con 0x7f1d9c1354c0 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 0x7f1d9c1358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient: get_auth_request con 0x7f1d9c1354c0 auth_method 0 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.637+0000 7f1da1fca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da1fca640 10 monclient(hunting): handle_auth_done global_id 34849 payload 824 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da1fca640 1 -- 172.21.15.17:0/349491965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1d9c06d020 con 0x7f1d9c1354c0 2024-09-15T21:21:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da1fca640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 0x7f1d9c1358d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d9401c850 tx=0x7f1d940a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b2040 con 0x7f1d9c1354c0 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 10 monclient: dump: 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f1d94004310 con 0x7f1d9c1354c0 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 msgr2=0x7f1d9c1358d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 0x7f1d9c1358d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d9401c850 tx=0x7f1d940a7000 comp rx=0 tx=0).stop 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 msgr2=0x7f1d9c1358d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1d9a7fc640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c1354c0 0x7f1d9c1358d0 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d9401c850 tx=0x7f1d940a7000 comp rx=0 tx=0).stop 2024-09-15T21:21:24.656 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 97138, 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:24.657 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.658 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:24.660 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:24.661 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:24.662 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:24.665 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:24.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:21:24.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:24.669 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1d8803b9f0 msgr2=0x7f1d8803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1d8803b9f0 0x7f1d8803deb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c09f5e0 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 10 monclient: shutdown 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a11b0 msgr2=0x7f1d9c1d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 --2- 172.21.15.17:0/349491965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a11b0 0x7f1d9c1d07b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c3a0 tx=0x7f1d8c00c860 comp rx=0 tx=0).stop 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da27cb640 1 -- 172.21.15.17:0/349491965 reap_dead start 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da27cb640 1 -- 172.21.15.17:0/349491965 reap_dead start 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 shutdown_connections 2024-09-15T21:21:24.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 >> 172.21.15.17:0/349491965 conn(0x7f1d9c0a66a0 msgr2=0x7f1d9c072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 shutdown_connections 2024-09-15T21:21:24.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.641+0000 7f1da422e640 1 -- 172.21.15.17:0/349491965 wait complete. 2024-09-15T21:21:24.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:21:24.672 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:21:24.672 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:21:24.672 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:21:24.672 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:21:24.673 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:21:24.673 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.650+0000 7fd6f5ffb640 1 -- 172.21.15.119:0/853080848 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v113) ==== 168+0+0 (secure 0 0 0) 0x7fd6f00a5700 con 0x7fd7001307e0 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.650+0000 7fd6f5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.650+0000 7fd6f5ffb640 10 monclient: _finish_command 3 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 msgr2=0x7fd6e403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0098d0 tx=0x7fd6fc008040 comp rx=0 tx=0).stop 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 10 monclient: shutdown 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 msgr2=0x7fd700138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 0x7fd700138570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0000c10 tx=0x7fd6ec005840 comp rx=0 tx=0).stop 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 shutdown_connections 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd700138ab0 0x7fd700138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7001307e0 0x7fd700138570 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 --2- 172.21.15.119:0/853080848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd70012fec0 0x7fd7001302a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 >> 172.21.15.119:0/853080848 conn(0x7fd70009be60 msgr2=0x7fd700131710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 shutdown_connections 2024-09-15T21:21:24.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.654+0000 7fd70774b640 1 -- 172.21.15.119:0/853080848 wait complete. 2024-09-15T21:21:24.683 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush reweight-subtree foobaz 123456 2024-09-15T21:21:24.683 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:21:24.683 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush reweight-subtree foobaz 123456 2024-09-15T21:21:24.692 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:21:24.692 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:21:24.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:24.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient: get_monmap_and_config 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient: monmap: 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:24.798675+0000 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:24.798675+0000 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:24.800 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:24.801 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:24.801 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 Processor -- start 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 -- start start 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient: init 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f039c137fe0 0x7f039c1383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): picked mon.noname-c con 0x7f039c137fe0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f039c0a2320 0x7f039c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): picked mon.noname-a con 0x7f039c0a2320 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:24.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c0a53a0 0x7f039c0a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): picked mon.noname-b con 0x7f039c0a53a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0a1400 con 0x7f039c0a2320 2024-09-15T21:21:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0a3e10 con 0x7f039c137fe0 2024-09-15T21:21:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f039c0a1960 con 0x7f039c0a53a0 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f03a1465640 10 monclient(hunting): authenticate will time out at 1203.595264s 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039affd640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f039c137fe0 msgr2=0x7f039c1383c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f039c137fe0 0x7f039c1383c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f039c0a2320 0x7f039c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f039c0a2320 0x7f039c0a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:53444/0 (socket says 172.21.15.119:53444) 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 1 -- 172.21.15.119:0/2080344482 learned_addr learned my addr 172.21.15.119:0/2080344482 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): get_auth_request con 0x7f039c0a2320 auth_method 0 2024-09-15T21:21:24.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039b7fe640 1 --2- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c0a53a0 0x7f039c0a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039b7fe640 10 monclient(hunting): get_auth_request con 0x7f039c0a53a0 auth_method 0 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.794+0000 7f039a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 10 monclient(hunting): handle_auth_done global_id 54192 payload 274 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 -- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c0a53a0 msgr2=0x7f039c0a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 --2- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c0a53a0 0x7f039c0a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 -- 172.21.15.119:0/2080344482 >> v2:172.21.15.119:3300/0 conn(0x7f039c137fe0 msgr2=0x7f039c1383c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:24.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 --2- 172.21.15.119:0/2080344482 >> v2:172.21.15.119:3300/0 conn(0x7f039c137fe0 0x7f039c1383c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 monclient: found mon.noname-a 2024-09-15T21:21:24.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 -- 172.21.15.119:0/2080344482 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0a6bf0 con 0x7f039c0a2320 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient: get_monmap_and_config 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient: monmap: 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:24.803906+0000 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:24.803906+0000 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 1 Processor -- start 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 1 -- start start 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient: init 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe8dc0a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe8dc0a5580 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8dc0a59d0 0x7fe8dc140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient(hunting): picked mon.noname-a con 0x7fe8dc0a59d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc0a2c30 con 0x7fe8dc0a5580 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc0a19b0 con 0x7fe8dc0a4a90 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe8dc0a0f40 con 0x7fe8dc0a59d0 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e30ca640 10 monclient(hunting): authenticate will time out at 1208.199311s 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e0e66640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fe8dc0a4a90 legacy=0x7fe8dc0a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8dbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8dc0a5580 0x7fe8dc13cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44298/0 (socket says 172.21.15.17:44298) 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8dbfff640 1 -- 172.21.15.17:0/1574273472 learned_addr learned my addr 172.21.15.17:0/1574273472 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 1 --2- 172.21.15.17:0/1574273472 >> v2:172.21.15.17:3300/0 conn(0x7fe8dc0a59d0 0x7fe8dc140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): get_auth_request con 0x7fe8dc0a59d0 auth_method 0 2024-09-15T21:21:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8e1667640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8db7fe640 1 -- 172.21.15.17:0/1574273472 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930670960 0 0) 0x7fe8c8001120 con 0x7fe8dc0a5580 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8db7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8db7fe640 10 monclient(hunting): my global_id is 54198 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.801+0000 7fe8db7fe640 1 -- 172.21.15.17:0/1574273472 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8cc003530 con 0x7fe8dc0a5580 2024-09-15T21:21:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 10 monclient(hunting): handle_auth_done global_id 54195 payload 274 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 1 -- 172.21.15.17:0/1574273472 >> v1:172.21.15.119:6789/0 conn(0x7fe8dc0a4a90 legacy=0x7fe8dc0a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 1 -- 172.21.15.17:0/1574273472 >> v1:172.21.15.17:6789/0 conn(0x7fe8dc0a5580 legacy=0x7fe8dc13cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 1 monclient: found mon.noname-a 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 1 -- 172.21.15.17:0/1574273472 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc0a1ce0 con 0x7fe8dc0a59d0 2024-09-15T21:21:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.805+0000 7fe8e1667640 1 --2- 172.21.15.17:0/1574273472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc140440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009dc0 tx=0x7fe8d401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe09d446767b4698 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f039a7fc640 1 --2- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a2320 0x7f039c0a4e60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f039c0a8020 tx=0x7f039001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ff5d54b693dd12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 1 -- 172.21.15.119:0/2080344482 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900b0070 con 0x7f039c0a2320 2024-09-15T21:21:24.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:24.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: dump: 2024-09-15T21:21:24.824 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.825 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 1 -- 172.21.15.119:0/2080344482 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03900ab600 con 0x7f039c0a2320 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 1 -- 172.21.15.119:0/2080344482 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900ab9d0 con 0x7f039c0a2320 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f03a1465640 5 monclient: authenticate success, global_id 54192 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f03a1465640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f03a1465640 4 set_mon_vals no callback set 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.798+0000 7f0399ffb640 10 monclient: dump: 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: shutdown 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a2320 msgr2=0x7f039c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a2320 0x7f039c0a4e60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f039c0a8020 tx=0x7f039001c400 comp rx=0 tx=0).stop 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2080344482 shutdown_connections 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c0a53a0 0x7f039c0a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- 172.21.15.119:0/2080344482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a2320 0x7f039c0a4e60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- 172.21.15.119:0/2080344482 >> v2:172.21.15.119:3300/0 conn(0x7f039c137fe0 0x7f039c1383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2080344482 >> 172.21.15.119:0/2080344482 conn(0x7f039c0a6f60 msgr2=0x7f039c0a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2080344482 shutdown_connections 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2080344482 wait complete. 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: monmap: 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:24.828 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 Processor -- start 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- start start 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: init 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:24.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f039c0a2320 0x7f039c13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): picked mon.noname-b con 0x7f039c0a2320 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a53a0 0x7f039c134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): picked mon.noname-a con 0x7f039c0a53a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c137fe0 0x7f039c1cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): picked mon.noname-c con 0x7f039c137fe0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0a64d0 con 0x7f039c0a53a0 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0a6350 con 0x7f039c0a2320 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f039c141690 con 0x7f039c137fe0 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient(hunting): authenticate will time out at 1203.601711s 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039affd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f039c0a2320 msgr2=0x7f039c13b190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f039c0a2320 0x7f039c13b190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a53a0 0x7f039c134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c137fe0 0x7f039c1cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a53a0 0x7f039c134dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:53456/0 (socket says 172.21.15.119:53456) 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c137fe0 0x7f039c1cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:48678/0 (socket says 172.21.15.119:48678) 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 -- 172.21.15.119:0/2334441123 learned_addr learned my addr 172.21.15.119:0/2334441123 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): get_auth_request con 0x7f039c0a53a0 auth_method 0 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): get_auth_request con 0x7f039c137fe0 auth_method 0 2024-09-15T21:21:24.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient(hunting): handle_auth_done global_id 54201 payload 824 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 -- 172.21.15.119:0/2334441123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c137fe0 msgr2=0x7f039c1cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 --2- 172.21.15.119:0/2334441123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f039c137fe0 0x7f039c1cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 -- 172.21.15.119:0/2334441123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f039c0a2320 msgr2=0x7f039c13b190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 --2- 172.21.15.119:0/2334441123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f039c0a2320 0x7f039c13b190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 monclient: found mon.noname-a 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 -- 172.21.15.119:0/2334441123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0a1650 con 0x7f039c0a53a0 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f039a7fc640 1 --2- 172.21.15.119:0/2334441123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0a53a0 0x7f039c134dc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f03900ae040 tx=0x7f039001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900ba020 con 0x7f039c0a53a0 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: dump: 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:24.833 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03900b6420 con 0x7f039c0a53a0 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900b0040 con 0x7f039c0a53a0 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 10 monclient: dump: 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.835 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 5 monclient: authenticate success, global_id 54201 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _renew_subs 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2334441123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c13c790 con 0x7f039c0a53a0 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f0398ff9640 4 set_mon_vals no callback set 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _renew_subs 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2334441123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c13b6d0 con 0x7f039c0a53a0 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03a1465640 1 -- 172.21.15.119:0/2334441123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c13dec0 con 0x7f039c0a53a0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.802+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f03900bd020 con 0x7f039c0a53a0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.806+0000 7f03837fe640 1 --2- 172.21.15.119:0/2334441123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f038803b970 0x7f038803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.806+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7f03900ea230 con 0x7f039c0a53a0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.806+0000 7f039affd640 1 --2- 172.21.15.119:0/2334441123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f038803b970 0x7f038803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.806+0000 7f039affd640 10 monclient: get_auth_request con 0x7f038803b970 auth_method 0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.806+0000 7f039affd640 1 --2- 172.21.15.119:0/2334441123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f038803b970 0x7f038803de30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.810+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f03900c4070 con 0x7f039c0a53a0 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.810+0000 7f03837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.810+0000 7f03837fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:24.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 1 -- 172.21.15.17:0/1574273472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40b2070 con 0x7fe8dc0a59d0 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: dump: 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 1 -- 172.21.15.17:0/1574273472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d40a5850 con 0x7fe8dc0a59d0 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 1 -- 172.21.15.17:0/1574273472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40a5c20 con 0x7fe8dc0a59d0 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.809+0000 7fe8db7fe640 10 monclient: dump: 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 5 monclient: authenticate success, global_id 54195 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 4 set_mon_vals no callback set 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: shutdown 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:24.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- 172.21.15.17:0/1574273472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a59d0 msgr2=0x7fe8dc140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/1574273472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc140440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009dc0 tx=0x7fe8d401c870 comp rx=0 tx=0).stop 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- 172.21.15.17:0/1574273472 shutdown_connections 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/1574273472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc140440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- 172.21.15.17:0/1574273472 >> 172.21.15.17:0/1574273472 conn(0x7fe8dc0aabb0 msgr2=0x7fe8dc0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- 172.21.15.17:0/1574273472 shutdown_connections 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- 172.21.15.17:0/1574273472 wait complete. 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: build_initial_monmap 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: monmap: 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 Processor -- start 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- start start 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: init 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a4a90 0x7fe8dc133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): picked mon.noname-a con 0x7fe8dc0a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 0x7fe8dc138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): picked mon.noname-c con 0x7fe8dc0a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): picked mon.noname-b con 0x7fe8dc0a59d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc1437c0 con 0x7fe8dc0a4a90 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc06f2f0 con 0x7fe8dc0a59d0 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc143940 con 0x7fe8dc0a5580 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient(hunting): authenticate will time out at 1208.210995s 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 0x7fe8dc138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e1667640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8dc0a59d0 msgr2=0x7fe8dc139230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e1667640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc139230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a4a90 0x7fe8dc133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 0x7fe8dc138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:34762/0 (socket says 172.21.15.17:34762) 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 -- 172.21.15.17:0/3124973344 learned_addr learned my addr 172.21.15.17:0/3124973344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8dc0a5580 auth_method 0 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): get_auth_request con 0x7fe8dc0a4a90 auth_method 0 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e0e66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_done global_id 44348 payload 824 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 -- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8dc0a59d0 msgr2=0x7fe8dc139230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc139230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 -- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a4a90 msgr2=0x7fe8dc133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a4a90 0x7fe8dc133e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 monclient: found mon.noname-c 2024-09-15T21:21:24.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 -- 172.21.15.17:0/3124973344 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc13bdc0 con 0x7fe8dc0a5580 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 0x7fe8dc138cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c730 tx=0x7fe8c800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8017070 con 0x7fe8dc0a5580 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: dump: 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c8005ac0 con 0x7fe8dc0a5580 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007070 con 0x7fe8dc0a5580 2024-09-15T21:21:24.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 5 monclient: authenticate success, global_id 44348 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: _renew_subs 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc13fb60 con 0x7fe8dc0a5580 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8d97fa640 10 monclient: dump: 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:24.850 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.813+0000 7fe8daffd640 4 set_mon_vals no callback set 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 10 monclient: _renew_subs 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc13a370 con 0x7fe8dc0a5580 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe8c801a070 con 0x7fe8dc0a5580 2024-09-15T21:21:24.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc0a4e70 con 0x7fe8dc0a5580 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8d97fa640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8c003b970 0x7fe8c003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7fe8c804f500 con 0x7fe8dc0a5580 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e0e66640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8c003b970 0x7fe8c003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e0e66640 10 monclient: get_auth_request con 0x7fe8c003b970 auth_method 0 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.817+0000 7fe8e0e66640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8c003b970 0x7fe8c003de30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009380 tx=0x7fe8d0009310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.821+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe8dc13bdc0 con 0x7fe8dc0a5580 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.821+0000 7fe8d97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:24.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:24.821+0000 7fe8d97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:24.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03a1465640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-09-15T21:21:24.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03a1465640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-09-15T21:21:24.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03a1465640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:24.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03a1465640 1 -- 172.21.15.119:0/2334441123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0} v 0) -- 0x7f039c0a10a0 con 0x7f039c0a53a0 2024-09-15T21:21:24.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03837fe640 1 -- 172.21.15.119:0/2334441123 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}]=-75 (75) Value too large for defined data type v113) ==== 154+0+0 (secure 0 0 0) 0x7f0390016110 con 0x7f039c0a53a0 2024-09-15T21:21:24.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-09-15T21:21:24.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:24.986+0000 7f03837fe640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-09-15T21:21:24.993 INFO:tasks.workunit.client.0.smithi119.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-09-15T21:21:24.998 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:21:24.999 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:21:24.999 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep osd.12 2024-09-15T21:21:24.999 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 155 2024-09-15T21:21:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe8dc13ae60 con 0x7fe8dc0a5580 2024-09-15T21:21:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8d97fa640 1 -- 172.21.15.17:0/3124973344 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fe8dc0a2c30 con 0x7fe8dc0a5580 2024-09-15T21:21:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8d97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8d97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:25.061 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8c003b970 msgr2=0x7fe8c003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8c003b970 0x7fe8c003de30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009380 tx=0x7fe8d0009310 comp rx=0 tx=0).stop 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 10 monclient: shutdown 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 msgr2=0x7fe8dc138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 0x7fe8dc138cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c730 tx=0x7fe8c800cbf0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 shutdown_connections 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8c003b970 0x7fe8c003de30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8dc0a59d0 0x7fe8dc139230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8dc0a5580 0x7fe8dc138cf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 --2- 172.21.15.17:0/3124973344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0a4a90 0x7fe8dc133e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 >> 172.21.15.17:0/3124973344 conn(0x7fe8dc0aabb0 msgr2=0x7fe8dc0a8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 shutdown_connections 2024-09-15T21:21:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:25.057+0000 7fe8e30ca640 1 -- 172.21.15.17:0/3124973344 wait complete. 2024-09-15T21:21:25.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.106+0000 7fcd5380e640 10 monclient: get_monmap_and_config 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.106+0000 7fcd5380e640 10 monclient: build_initial_monmap 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.106+0000 7fcd5380e640 10 monclient: monmap: 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:25.112622+0000 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:25.112622+0000 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:25.114 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:25.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:25.115 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:25.115 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:25.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 1 Processor -- start 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 1 -- start start 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient: init 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcd4c0a4d80 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:25.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcd4c0a5870 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient(hunting): picked mon.noname-b con 0x7fcd4c13e6f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4c0a2250 con 0x7fcd4c0a5870 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4c06dab0 con 0x7fcd4c0a4d80 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4c0a19b0 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd5380e640 10 monclient(hunting): authenticate will time out at 1203.909692s 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd515aa640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fcd4c0a4d80 legacy=0x7fcd4c0a5160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd50da9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcd4c0a5870 0x7fcd4c13cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:49172/0 (socket says 172.21.15.119:49172) 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd50da9640 1 -- 172.21.15.119:0/485704083 learned_addr learned my addr 172.21.15.119:0/485704083 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:25.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): get_auth_request con 0x7fcd4c13e6f0 auth_method 0 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient(hunting): handle_auth_done global_id 44351 payload 274 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 1 -- 172.21.15.119:0/485704083 >> v1:172.21.15.119:6789/0 conn(0x7fcd4c0a4d80 legacy=0x7fcd4c0a5160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 1 -- 172.21.15.119:0/485704083 >> v1:172.21.15.17:6789/0 conn(0x7fcd4c0a5870 legacy=0x7fcd4c13cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient: _finish_hunting 0 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 1 monclient: found mon.noname-b 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:25.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.110+0000 7fcd51dab640 1 -- 172.21.15.119:0/485704083 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c0a1c90 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 --2- 172.21.15.119:0/485704083 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c140ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c06dc80 tx=0x7fcd4401c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10e0022d821bc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:25.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 1 -- 172.21.15.119:0/485704083 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440b2070 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: dump: 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.126 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 1 -- 172.21.15.119:0/485704083 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd440a57c0 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 5 monclient: authenticate success, global_id 44351 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:25.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 1 -- 172.21.15.119:0/485704083 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440a5b90 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 4 set_mon_vals no callback set 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd43fff640 10 monclient: dump: 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:25.128 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: shutdown 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- 172.21.15.119:0/485704083 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 msgr2=0x7fcd4c140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 --2- 172.21.15.119:0/485704083 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c140ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c06dc80 tx=0x7fcd4401c830 comp rx=0 tx=0).stop 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- 172.21.15.119:0/485704083 shutdown_connections 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 --2- 172.21.15.119:0/485704083 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c140ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- 172.21.15.119:0/485704083 >> 172.21.15.119:0/485704083 conn(0x7fcd4c0a75c0 msgr2=0x7fcd4c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- 172.21.15.119:0/485704083 shutdown_connections 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- 172.21.15.119:0/485704083 wait complete. 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: build_initial_monmap 2024-09-15T21:21:25.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: monmap: 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 Processor -- start 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- start start 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: init 2024-09-15T21:21:25.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd4c0a4d80 0x7fcd4c1cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): picked mon.noname-a con 0x7fcd4c0a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd4c0a5870 0x7fcd4c1d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): picked mon.noname-b con 0x7fcd4c0a5870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c1d0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): picked mon.noname-c con 0x7fcd4c13e6f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:25.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c0a2e40 con 0x7fcd4c0a4d80 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c0a2cc0 con 0x7fcd4c0a5870 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4c143840 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd5380e640 10 monclient(hunting): authenticate will time out at 1203.915002s 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd50da9640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd4c0a5870 msgr2=0x7fcd4c1d00d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd50da9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd4c0a5870 0x7fcd4c1d00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c1d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd4c0a4d80 0x7fcd4c1cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:25.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c1d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:48692/0 (socket says 172.21.15.119:48692) 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 -- 172.21.15.119:0/3264001828 learned_addr learned my addr 172.21.15.119:0/3264001828 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): get_auth_request con 0x7fcd4c13e6f0 auth_method 0 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): get_auth_request con 0x7fcd4c0a4d80 auth_method 0 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:25.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd515aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient(hunting): handle_auth_done global_id 44354 payload 824 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 -- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd4c0a5870 msgr2=0x7fcd4c1d00d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd4c0a5870 0x7fcd4c1d00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 -- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd4c0a4d80 msgr2=0x7fcd4c1cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd4c0a4d80 0x7fcd4c1cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient: _finish_hunting 0 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 monclient: found mon.noname-c 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.114+0000 7fcd51dab640 1 -- 172.21.15.119:0/3264001828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c13dfc0 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd51dab640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c1d0610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd4401d0d0 tx=0x7fcd440195d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440bd020 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: dump: 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:25.135 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd440b8670 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440b2040 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 5 monclient: authenticate success, global_id 44354 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 10 monclient: dump: 2024-09-15T21:21:25.136 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd42ffd640 4 set_mon_vals no callback set 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 monclient: _renew_subs 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c13c4d0 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 monclient: _renew_subs 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c13ca90 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4c0a5160 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcd440c0020 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd3003b970 0x7fcd3003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7fcd440eb290 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd515aa640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd3003b970 0x7fcd3003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:25.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd515aa640 10 monclient: get_auth_request con 0x7fcd3003b970 auth_method 0 2024-09-15T21:21:25.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.118+0000 7fcd515aa640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd3003b970 0x7fcd3003de30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd480093d0 tx=0x7fcd48009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:25.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.122+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcd44016540 con 0x7fcd4c13e6f0 2024-09-15T21:21:25.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.122+0000 7fcd41ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:25.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.122+0000 7fcd41ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:25.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.298+0000 7fcd5380e640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:25.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.298+0000 7fcd5380e640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:25.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.298+0000 7fcd5380e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:25.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:25.298+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fcd4c0a4850 con 0x7fcd4c13e6f0 2024-09-15T21:21:28.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:28.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:28.118+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:28.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:28.118+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:28.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:28.118+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:28.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:28.118+0000 7fcd40ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:28.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient: get_monmap_and_config 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient: build_initial_monmap 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient: monmap: 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:28.208148+0000 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:28.208148+0000 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:28.210 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:28.211 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:28.211 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 1 Processor -- start 2024-09-15T21:21:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 1 -- start start 2024-09-15T21:21:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient: init 2024-09-15T21:21:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3a200a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3a2009ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a200a1380 0x7f3a2013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient(hunting): picked mon.noname-c con 0x7f3a200a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a200a3130 con 0x7f3a2009ffc0 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a20073df0 con 0x7f3a200a0890 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3a200a1bf0 con 0x7f3a200a1380 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a271e1640 10 monclient(hunting): authenticate will time out at 1211.603509s 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a24f7d640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f3a200a0890 legacy=0x7f3a200a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a2577e640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f3a200a1380 msgr2=0x7f3a2013e1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a2577e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a200a1380 0x7f3a2013e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a1ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3a2009ffc0 0x7f3a2013aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36764/0 (socket says 172.21.15.17:36764) 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.205+0000 7f3a1ffff640 1 -- 172.21.15.17:0/3492906300 learned_addr learned my addr 172.21.15.17:0/3492906300 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566427562 0 0) 0x7f3a200a3130 con 0x7f3a2009ffc0 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): my global_id is 54213 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a0c003530 con 0x7f3a2009ffc0 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2812941229 0 0) 0x7f3a0c003530 con 0x7f3a2009ffc0 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 >> v2:172.21.15.119:3300/0 conn(0x7f3a200a1380 msgr2=0x7f3a2013e1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 --2- 172.21.15.17:0/3492906300 >> v2:172.21.15.119:3300/0 conn(0x7f3a200a1380 0x7f3a2013e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 >> v1:172.21.15.119:6789/0 conn(0x7f3a200a0890 legacy=0x7f3a200a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2009ccd0 con 0x7f3a2009ffc0 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f3a10001700 con 0x7f3a2009ffc0 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: dump: 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 >> v1:172.21.15.17:6789/0 conn(0x7f3a2009ffc0 legacy=0x7f3a2013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c0039c0 0x7f3a0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): picked mon.a con 0x7f3a0c0039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c004810 0x7f3a0c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): picked mon.c con 0x7f3a0c004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0c005380 0x7f3a0c00b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): picked mon.b con 0x7f3a0c005380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200a1bf0 con 0x7f3a0c0039c0 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c00c2d0 con 0x7f3a0c005380 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0c00c4d0 con 0x7f3a0c004810 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c0039c0 0x7f3a0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:28.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): get_auth_request con 0x7f3a0c0039c0 auth_method 0 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a2577e640 1 -- 172.21.15.17:0/3492906300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0c005380 msgr2=0x7f3a0c00b9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c004810 0x7f3a0c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a2577e640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0c005380 0x7f3a0c00b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): get_auth_request con 0x7f3a0c004810 auth_method 0 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_done global_id 54213 payload 293 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 -- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c004810 msgr2=0x7f3a0c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c004810 0x7f3a0c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 -- 172.21.15.17:0/3492906300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0c005380 msgr2=0x7f3a0c00b9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0c005380 0x7f3a0c00b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 monclient: found mon.a 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a2577e640 1 -- 172.21.15.17:0/3492906300 reap_dead start 2024-09-15T21:21:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 -- 172.21.15.17:0/3492906300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c003530 con 0x7f3a0c0039c0 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient: _finish_auth 0 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 10 monclient: _check_auth_tickets 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 5 monclient: authenticate success, global_id 54213 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c0039c0 0x7f3a0c003dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a08003940 tx=0x7f3a0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c901b05bf7bf6d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08009520 con 0x7f3a0c0039c0 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: dump: 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a08003bc0 con 0x7f3a0c0039c0 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 4 set_mon_vals no callback set 2024-09-15T21:21:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 1 -- 172.21.15.17:0/3492906300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08005070 con 0x7f3a0c0039c0 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a1f7fe640 10 monclient: dump: 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 10 monclient: shutdown 2024-09-15T21:21:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 -- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c0039c0 msgr2=0x7f3a0c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c0039c0 0x7f3a0c003dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a08003940 tx=0x7f3a0801c420 comp rx=0 tx=0).stop 2024-09-15T21:21:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 -- 172.21.15.17:0/3492906300 shutdown_connections 2024-09-15T21:21:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 --2- 172.21.15.17:0/3492906300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c0039c0 0x7f3a0c003dc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 -- 172.21.15.17:0/3492906300 >> 172.21.15.17:0/3492906300 conn(0x7f3a200a66a0 msgr2=0x7f3a200a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 -- 172.21.15.17:0/3492906300 shutdown_connections 2024-09-15T21:21:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.209+0000 7f3a271e1640 1 -- 172.21.15.17:0/3492906300 wait complete. 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: build_initial_monmap 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: monmap: 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:28.233 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 Processor -- start 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- start start 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: init 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 0x7f3a2009d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): picked mon.noname-c con 0x7f3a0c0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a2009dbd0 0x7f3a201cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): picked mon.noname-b con 0x7f3a2009dbd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a2009e030 0x7f3a201cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): picked mon.noname-a con 0x7f3a2009e030 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20136210 con 0x7f3a2009e030 2024-09-15T21:21:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20136090 con 0x7f3a2009dbd0 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3a20136390 con 0x7f3a0c0039c0 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient(hunting): authenticate will time out at 1211.609416s 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 0x7f3a2009d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a2009e030 0x7f3a201cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 0x7f3a2009d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55060/0 (socket says 172.21.15.17:55060) 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 -- 172.21.15.17:0/4073567848 learned_addr learned my addr 172.21.15.17:0/4073567848 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): get_auth_request con 0x7f3a0c0039c0 auth_method 0 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): get_auth_request con 0x7f3a2009e030 auth_method 0 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1ffff640 1 -- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a2009dbd0 msgr2=0x7f3a201cfa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1ffff640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a2009dbd0 0x7f3a201cfa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient(hunting): handle_auth_done global_id 44357 payload 824 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 -- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a2009dbd0 msgr2=0x7f3a201cfa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a2009dbd0 0x7f3a201cfa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 -- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a2009e030 msgr2=0x7f3a201cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a2009e030 0x7f3a201cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 monclient: found mon.noname-c 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 -- 172.21.15.17:0/4073567848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a20128f50 con 0x7f3a0c0039c0 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a2577e640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a2009e030 0x7f3a201cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a24f7d640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 0x7f3a2009d690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a0801c250 tx=0x7f3a0c005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08009500 con 0x7f3a0c0039c0 2024-09-15T21:21:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: dump: 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a080039c0 con 0x7f3a0c0039c0 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a080b7640 con 0x7f3a0c0039c0 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 10 monclient: dump: 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1effd640 4 set_mon_vals no callback set 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 5 monclient: authenticate success, global_id 44357 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _renew_subs 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a20129230 con 0x7f3a0c0039c0 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _renew_subs 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a20074580 con 0x7f3a0c0039c0 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2009d0f0 con 0x7f3a0c0039c0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.213+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3a080c0440 con 0x7f3a0c0039c0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.217+0000 7f3a1d7fa640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a0003b9a0 0x7f3a0003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.217+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7f3a080ea000 con 0x7f3a0c0039c0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.217+0000 7f3a1ffff640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a0003b9a0 0x7f3a0003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.217+0000 7f3a1ffff640 10 monclient: get_auth_request con 0x7f3a0003b9a0 auth_method 0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.217+0000 7f3a1ffff640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a0003b9a0 0x7f3a0003de60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a10000c10 tx=0x7f3a10007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.221+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3a08016130 con 0x7f3a0c0039c0 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.221+0000 7f3a1d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.221+0000 7f3a1d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:28.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.453+0000 7f3a271e1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:28.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.453+0000 7f3a271e1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.453+0000 7f3a271e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.453+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3a2011fc00 con 0x7f3a0c0039c0 2024-09-15T21:21:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a1d7fa640 1 -- 172.21.15.17:0/4073567848 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f3a08003b70 con 0x7f3a0c0039c0 2024-09-15T21:21:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a1d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a1d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:28.460 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a0003b9a0 msgr2=0x7f3a0003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a0003b9a0 0x7f3a0003de60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a10000c10 tx=0x7f3a10007900 comp rx=0 tx=0).stop 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 10 monclient: shutdown 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 msgr2=0x7f3a2009d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 0x7f3a2009d690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a0801c250 tx=0x7f3a0c005840 comp rx=0 tx=0).stop 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 shutdown_connections 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a0003b9a0 0x7f3a0003de60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a2009e030 0x7f3a201cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a2009dbd0 0x7f3a201cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 --2- 172.21.15.17:0/4073567848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a0c0039c0 0x7f3a2009d690 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 >> 172.21.15.17:0/4073567848 conn(0x7f3a200a66a0 msgr2=0x7f3a200700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 shutdown_connections 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:28.457+0000 7f3a271e1640 1 -- 172.21.15.17:0/4073567848 wait complete. 2024-09-15T21:21:29.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:29.113+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:29.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:29.113+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:29.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:29.113+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:30.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:30.113+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:30.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:30.113+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:30.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:30.113+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:31.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:31.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:31.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:31.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:31.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:31.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient: get_monmap_and_config 2024-09-15T21:21:31.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient: build_initial_monmap 2024-09-15T21:21:31.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient: monmap: 2024-09-15T21:21:31.614 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:31.614 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:31.615 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:31.611908+0000 2024-09-15T21:21:31.615 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:31.611908+0000 2024-09-15T21:21:31.615 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:31.615 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:31.615 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:31.615 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:31.616 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:31.616 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:31.616 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:31.616 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 Processor -- start 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 -- start start 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient: init 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 0x7fefc00a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): picked mon.noname-b con 0x7fefc00a28b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fefc00a31d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fefc013c570 0x7fefc0144340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): picked mon.noname-c con 0x7fefc013c570 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:31.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc006cea0 con 0x7fefc00a31d0 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fefc006d550 con 0x7fefc013c570 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefc009ba60 con 0x7fefc00a28b0 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc6622640 10 monclient(hunting): authenticate will time out at 1215.007488s 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbf7fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fefc00a31d0 legacy=0x7fefc013be60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc4bbf640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fefc013c570 msgr2=0x7fefc0144340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefc4bbf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fefc013c570 0x7fefc0144340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 0x7fefc00a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 0x7fefc00a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55076/0 (socket says 172.21.15.17:55076) 2024-09-15T21:21:31.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 1 -- 172.21.15.17:0/2356219017 learned_addr learned my addr 172.21.15.17:0/2356219017 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:31.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): get_auth_request con 0x7fefc00a28b0 auth_method 0 2024-09-15T21:21:31.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:31.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:31.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:31.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:31.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:31.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.609+0000 7fefbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:31.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 10 monclient(hunting): handle_auth_done global_id 44360 payload 274 2024-09-15T21:21:31.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 1 -- 172.21.15.17:0/2356219017 >> v2:172.21.15.119:3300/0 conn(0x7fefc013c570 msgr2=0x7fefc0144340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:31.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 1 --2- 172.21.15.17:0/2356219017 >> v2:172.21.15.119:3300/0 conn(0x7fefc013c570 0x7fefc0144340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 1 -- 172.21.15.17:0/2356219017 >> v1:172.21.15.119:6789/0 conn(0x7fefc00a31d0 legacy=0x7fefc013be60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:31.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:31.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 1 monclient: found mon.noname-b 2024-09-15T21:21:31.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:31.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:31.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 1 -- 172.21.15.17:0/2356219017 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc009cc50 con 0x7fefc00a28b0 2024-09-15T21:21:31.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbffff640 1 --2- 172.21.15.17:0/2356219017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 0x7fefc00a2c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009e50 tx=0x7fefb801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c4012daf45632f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:31.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 1 -- 172.21.15.17:0/2356219017 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80a5550 con 0x7fefc00a28b0 2024-09-15T21:21:31.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:31.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:31.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: dump: 2024-09-15T21:21:31.630 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:31.631 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:31.632 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:31.632 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:31.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:31.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:31.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:31.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 1 -- 172.21.15.17:0/2356219017 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb80a56f0 con 0x7fefc00a28b0 2024-09-15T21:21:31.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:31.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 1 -- 172.21.15.17:0/2356219017 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80a5ac0 con 0x7fefc00a28b0 2024-09-15T21:21:31.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 5 monclient: authenticate success, global_id 44360 2024-09-15T21:21:31.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:31.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 4 set_mon_vals no callback set 2024-09-15T21:21:31.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefbeffd640 10 monclient: dump: 2024-09-15T21:21:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 10 monclient: shutdown 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 -- 172.21.15.17:0/2356219017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 msgr2=0x7fefc00a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 --2- 172.21.15.17:0/2356219017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 0x7fefc00a2c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009e50 tx=0x7fefb801c910 comp rx=0 tx=0).stop 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 -- 172.21.15.17:0/2356219017 shutdown_connections 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 --2- 172.21.15.17:0/2356219017 >> v2:172.21.15.119:3300/0 conn(0x7fefc013c570 0x7fefc0144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 --2- 172.21.15.17:0/2356219017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a28b0 0x7fefc00a2c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 -- 172.21.15.17:0/2356219017 >> 172.21.15.17:0/2356219017 conn(0x7fefc00a74a0 msgr2=0x7fefc00a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 -- 172.21.15.17:0/2356219017 shutdown_connections 2024-09-15T21:21:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 -- 172.21.15.17:0/2356219017 wait complete. 2024-09-15T21:21:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 10 monclient: build_initial_monmap 2024-09-15T21:21:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 10 monclient: monmap: 2024-09-15T21:21:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:31.639 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 Processor -- start 2024-09-15T21:21:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.613+0000 7fefc6622640 1 -- start start 2024-09-15T21:21:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: init 2024-09-15T21:21:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:31.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:31.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc00a28b0 0x7fefc0135050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:31.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): picked mon.noname-b con 0x7fefc00a28b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:31.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 0x7fefc0135590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:31.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): picked mon.noname-c con 0x7fefc00a31d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:31.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 0x7fefc0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): picked mon.noname-a con 0x7fefc013c570 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fefc009d7b0 con 0x7fefc013c570 2024-09-15T21:21:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc009d630 con 0x7fefc00a28b0 2024-09-15T21:21:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefc01470a0 con 0x7fefc00a31d0 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient(hunting): authenticate will time out at 1215.012542s 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 0x7fefc0135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc00a28b0 msgr2=0x7fefc0135050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc00a28b0 0x7fefc0135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 0x7fefc0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 0x7fefc0135590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55090/0 (socket says 172.21.15.17:55090) 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 1 -- 172.21.15.17:0/4179639386 learned_addr learned my addr 172.21.15.17:0/4179639386 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 0x7fefc0071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35134/0 (socket says 172.21.15.17:35134) 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): get_auth_request con 0x7fefc00a31d0 auth_method 0 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): get_auth_request con 0x7fefc013c570 auth_method 0 2024-09-15T21:21:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient(hunting): handle_auth_done global_id 54219 payload 824 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 -- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 msgr2=0x7fefc0135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 0x7fefc0135590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 -- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc00a28b0 msgr2=0x7fefc0135050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc00a28b0 0x7fefc0135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient: _finish_hunting 0 2024-09-15T21:21:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 monclient: found mon.noname-a 2024-09-15T21:21:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 -- 172.21.15.17:0/4179639386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc013daa0 con 0x7fefc013c570 2024-09-15T21:21:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbf7fe640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 0x7fefc0135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc4bbf640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 0x7fefc0071180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefb00038c0 tx=0x7fefb000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0016070 con 0x7fefc013c570 2024-09-15T21:21:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: dump: 2024-09-15T21:21:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:31.649 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:31.649 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:31.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:31.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:31.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:31.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb0003d20 con 0x7fefc013c570 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00052b0 con 0x7fefc013c570 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 5 monclient: authenticate success, global_id 54219 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _renew_subs 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc013dd80 con 0x7fefc013c570 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbe7fc640 4 set_mon_vals no callback set 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 10 monclient: dump: 2024-09-15T21:21:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _renew_subs 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc013d7c0 con 0x7fefc013c570 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fefb0019070 con 0x7fefc013c570 2024-09-15T21:21:31.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc00a2c90 con 0x7fefc013c570 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefac03b9d0 0x7fefac03de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7fefb004f390 con 0x7fefc013c570 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbffff640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefac03b9d0 0x7fefac03de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.617+0000 7fefbffff640 10 monclient: get_auth_request con 0x7fefac03b9d0 auth_method 0 2024-09-15T21:21:31.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.621+0000 7fefbffff640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefac03b9d0 0x7fefac03de90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefb80ae000 tx=0x7fefb80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:31.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.621+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fefb000f300 con 0x7fefc013c570 2024-09-15T21:21:31.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.621+0000 7fefbcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:31.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.621+0000 7fefbcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefc6622640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefc6622640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefc6622640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:31.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fefc006e030 con 0x7fefc013c570 2024-09-15T21:21:31.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefbcff9640 1 -- 172.21.15.17:0/4179639386 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fefb000fe20 con 0x7fefc013c570 2024-09-15T21:21:31.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefbcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:31.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.865+0000 7fefbcff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:31.870 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:31.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefac03b9d0 msgr2=0x7fefac03de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:31.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefac03b9d0 0x7fefac03de90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefb80ae000 tx=0x7fefb80a7070 comp rx=0 tx=0).stop 2024-09-15T21:21:31.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 10 monclient: shutdown 2024-09-15T21:21:31.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:31.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 msgr2=0x7fefc0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 0x7fefc0071180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefb00038c0 tx=0x7fefb000b870 comp rx=0 tx=0).stop 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 shutdown_connections 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefac03b9d0 0x7fefac03de90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc013c570 0x7fefc0071180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc00a31d0 0x7fefc0135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 --2- 172.21.15.17:0/4179639386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc00a28b0 0x7fefc0135050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 >> 172.21.15.17:0/4179639386 conn(0x7fefc00a74a0 msgr2=0x7fefc009f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 shutdown_connections 2024-09-15T21:21:31.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:31.869+0000 7fefc6622640 1 -- 172.21.15.17:0/4179639386 wait complete. 2024-09-15T21:21:32.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:32.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:32.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:32.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:32.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:32.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:33.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:33.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:33.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:33.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:33.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:33.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:34.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:34.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:34.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:34.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:21:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient: get_monmap_and_config 2024-09-15T21:21:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient: build_initial_monmap 2024-09-15T21:21:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient: monmap: 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:35.019787+0000 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:35.019787+0000 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:35.022 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 1 Processor -- start 2024-09-15T21:21:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 1 -- start start 2024-09-15T21:21:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient: init 2024-09-15T21:21:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient(hunting): picked mon.noname-b con 0x7ff2e80a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2e809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2e80a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e8136490 con 0x7ff2e80a11b0 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e80a3130 con 0x7ff2e809ffc0 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e8073df0 con 0x7ff2e80a0890 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ee8da640 10 monclient(hunting): authenticate will time out at 1218.415351s 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e77fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7ff2e809ffc0 legacy=0x7ff2e8135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ece77640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2e80a11b0 0x7ff2e8135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36772/0 (socket says 172.21.15.17:36772) 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2ece77640 1 -- 172.21.15.17:0/1655816904 learned_addr learned my addr 172.21.15.17:0/1655816904 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request con 0x7ff2e80a0890 auth_method 0 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.017+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:35.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:35.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:35.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request con 0x7ff2e80a0890 auth_method 0 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:21:35.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.221+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.119:6789/0 conn(0x7ff2e809ffc0 legacy=0x7ff2e8135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:35.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:35.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request con 0x7ff2e80a0890 auth_method 0 2024-09-15T21:21:35.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:35.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:35.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:35.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.119:6789/0 conn(0x7ff2e809ffc0 legacy=0x7ff2e8135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:35.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:21:35.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:35.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:35.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:35.625+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:21:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:36.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:36.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:36.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:36.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.119:6789/0 conn(0x7ff2e809ffc0 legacy=0x7ff2e8135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request con 0x7ff2e80a0890 auth_method 0 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:36.429+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:21:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:37.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:37.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:37.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:38.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): tick 2024-09-15T21:21:38.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:21:38.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 monclient(hunting): continuing hunt 2024-09-15T21:21:38.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:21:38.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 msgr2=0x7ff2e80a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:38.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e80a0890 0x7ff2e80a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.119:6789/0 conn(0x7ff2e809ffc0 legacy=0x7ff2e8135670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.17:6789/0 conn(0x7ff2e80a11b0 legacy=0x7ff2e8135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient: _add_conns ranks=[4,3] 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): picked mon.noname-c con 0x7ff2d4003700 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): picked mon.noname-a con 0x7ff2d4006250 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e8073df0 con 0x7ff2d4006250 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d4008dd0 con 0x7ff2d4003700 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e5ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:38.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.017+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request con 0x7ff2d4006250 auth_method 0 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.021+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:38.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:38.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:38.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:38.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:38.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:38.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:38.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:38.117+0000 7fcd40ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:38.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.221+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:38.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.221+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request con 0x7ff2d4006250 auth_method 0 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:21:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:38.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:38.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.225+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:21:38.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.625+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:38.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.625+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:21:38.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:38.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request con 0x7ff2d4006250 auth_method 0 2024-09-15T21:21:38.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:38.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:38.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:38.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:21:38.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:38.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:38.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:38.629+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:21:39.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.117+0000 7fcd40ff9640 10 monclient: tick 2024-09-15T21:21:39.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.117+0000 7fcd40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:39.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.117+0000 7fcd40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.429+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.429+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:21:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request con 0x7ff2d4006250 auth_method 0 2024-09-15T21:21:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:39.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:39.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:21:39.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:21:39.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:39.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:39.433+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:21:39.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.713+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v113) ==== 56+0+1449 (secure 0 0 0) 0x7fcd440b73d0 con 0x7fcd4c13e6f0 2024-09-15T21:21:39.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.713+0000 7fcd41ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:39.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.713+0000 7fcd41ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:39.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.813+0000 7fcd5380e640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:39.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.813+0000 7fcd5380e640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:39.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.813+0000 7fcd5380e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:39.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.813+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fcd4c0a4850 con 0x7fcd4c13e6f0 2024-09-15T21:21:39.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd41ffb640 1 -- 172.21.15.119:0/3264001828 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v113) ==== 56+0+1449 (secure 0 0 0) 0x7fcd440ad7e0 con 0x7fcd4c13e6f0 2024-09-15T21:21:39.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd41ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:39.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd41ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:39.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd3003b970 msgr2=0x7fcd3003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:39.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd3003b970 0x7fcd3003de30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd480093d0 tx=0x7fcd48009360 comp rx=0 tx=0).stop 2024-09-15T21:21:39.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 10 monclient: shutdown 2024-09-15T21:21:39.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 msgr2=0x7fcd4c1d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c1d0610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd4401d0d0 tx=0x7fcd440195d0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 shutdown_connections 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcd3003b970 0x7fcd3003de30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcd4c13e6f0 0x7fcd4c1d0610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd4c0a5870 0x7fcd4c1d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 --2- 172.21.15.119:0/3264001828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd4c0a4d80 0x7fcd4c1cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 >> 172.21.15.119:0/3264001828 conn(0x7fcd4c0a75c0 msgr2=0x7fcd4c137c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 shutdown_connections 2024-09-15T21:21:39.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.817+0000 7fcd5380e640 1 -- 172.21.15.119:0/3264001828 wait complete. 2024-09-15T21:21:39.840 INFO:tasks.workunit.client.0.smithi119.stdout: 12 155.00000 osd.12 down 1.00000 1.00000 2024-09-15T21:21:39.840 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd tree 2024-09-15T21:21:39.840 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep osd.14 2024-09-15T21:21:39.840 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 155 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient: get_monmap_and_config 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient: build_initial_monmap 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient: monmap: 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:39.952940+0000 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:39.952940+0000 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:39.954 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:39.955 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:39.955 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:39.955 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:39.955 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:39.955 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:39.955 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:39.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 Processor -- start 2024-09-15T21:21:39.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 -- start start 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient: init 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f080a4d80 0x7f8f080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): picked mon.noname-a con 0x7f8f080a4d80 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8f080a56a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f08138430 0x7f8f0813fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): picked mon.noname-c con 0x7f8f08138430 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f080a2250 con 0x7f8f080a56a0 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8f0806dab0 con 0x7f8f080a4d80 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8f080a19b0 con 0x7f8f08138430 2024-09-15T21:21:39.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f0e250640 10 monclient(hunting): authenticate will time out at 1218.749840s 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f07fff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f8f08138430 msgr2=0x7f8f0813fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f07fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f08138430 0x7f8f0813fff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f080a4d80 0x7f8f080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f080a4d80 0x7f8f080a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:35894/0 (socket says 172.21.15.119:35894) 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 -- 172.21.15.119:0/2824988511 learned_addr learned my addr 172.21.15.119:0/2824988511 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f06ffd640 1 --1- 172.21.15.119:0/2824988511 >> v1:172.21.15.17:6789/0 conn(0x7f8f080a56a0 0x7f8f08137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:52008/0 (socket says 172.21.15.119:52008) 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): get_auth_request con 0x7f8f080a4d80 auth_method 0 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:39.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient(hunting): handle_auth_done global_id 54222 payload 274 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 -- 172.21.15.119:0/2824988511 >> v2:172.21.15.119:3300/0 conn(0x7f8f08138430 msgr2=0x7f8f0813fff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 --2- 172.21.15.119:0/2824988511 >> v2:172.21.15.119:3300/0 conn(0x7f8f08138430 0x7f8f0813fff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 -- 172.21.15.119:0/2824988511 >> v1:172.21.15.17:6789/0 conn(0x7f8f080a56a0 legacy=0x7f8f08137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 monclient: found mon.noname-a 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:39.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.949+0000 7f8f077fe640 1 -- 172.21.15.119:0/2824988511 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f080a1c90 con 0x7f8f080a4d80 2024-09-15T21:21:39.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f077fe640 1 --2- 172.21.15.119:0/2824988511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f080a5160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009870 tx=0x7f8ef001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6584ec3e5d8df7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 1 -- 172.21.15.119:0/2824988511 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00b2070 con 0x7f8f080a4d80 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: dump: 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:39.966 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 5 monclient: authenticate success, global_id 54222 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 1 -- 172.21.15.119:0/2824988511 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef00a9070 con 0x7f8f080a4d80 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 4 set_mon_vals no callback set 2024-09-15T21:21:39.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 1 -- 172.21.15.119:0/2824988511 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00ad880 con 0x7f8f080a4d80 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f067fc640 10 monclient: dump: 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:39.968 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: shutdown 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- 172.21.15.119:0/2824988511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 msgr2=0x7f8f080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 --2- 172.21.15.119:0/2824988511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f080a5160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009870 tx=0x7f8ef001c340 comp rx=0 tx=0).stop 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- 172.21.15.119:0/2824988511 shutdown_connections 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 --2- 172.21.15.119:0/2824988511 >> v2:172.21.15.119:3300/0 conn(0x7f8f08138430 0x7f8f0813fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 --2- 172.21.15.119:0/2824988511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f080a5160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- 172.21.15.119:0/2824988511 >> 172.21.15.119:0/2824988511 conn(0x7f8f080a75c0 msgr2=0x7f8f0806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- 172.21.15.119:0/2824988511 shutdown_connections 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- 172.21.15.119:0/2824988511 wait complete. 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: build_initial_monmap 2024-09-15T21:21:39.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: monmap: 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 Processor -- start 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- start start 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: init 2024-09-15T21:21:39.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f08131ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): picked mon.noname-a con 0x7f8f080a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 0x7f8f08132020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): picked mon.noname-c con 0x7f8f080a56a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f08138430 0x7f8f08132560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): picked mon.noname-b con 0x7f8f08138430 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080a1290 con 0x7f8f080a4d80 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080a1110 con 0x7f8f08138430 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:39.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8f080a2c60 con 0x7f8f080a56a0 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f0e250640 10 monclient(hunting): authenticate will time out at 1218.755588s 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f07fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f08138430 msgr2=0x7f8f08132560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.953+0000 7f8f07fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f08138430 0x7f8f08132560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 0x7f8f08132020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f08131ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 0x7f8f08132020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33786/0 (socket says 172.21.15.119:33786) 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 -- 172.21.15.119:0/3736113939 learned_addr learned my addr 172.21.15.119:0/3736113939 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): get_auth_request con 0x7f8f080a56a0 auth_method 0 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:39.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): get_auth_request con 0x7f8f080a4d80 auth_method 0 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient(hunting): handle_auth_done global_id 44366 payload 824 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 -- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f08138430 msgr2=0x7f8f08132560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:39.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f08138430 0x7f8f08132560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 -- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 msgr2=0x7f8f08131ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f08131ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 monclient: found mon.noname-c 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 -- 172.21.15.119:0/3736113939 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f080a1570 con 0x7f8f080a56a0 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f06ffd640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 0x7f8f08132020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c870 tx=0x7f8ef800cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8017070 con 0x7f8f080a56a0 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: dump: 2024-09-15T21:21:39.974 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: _finish_auth 0 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:39.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 5 monclient: authenticate success, global_id 44366 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 monclient: _renew_subs 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f08073540 con 0x7f8f080a56a0 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef800a5e0 con 0x7f8f080a56a0 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800a990 con 0x7f8f080a56a0 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 10 monclient: dump: 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:39.976 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f05ffb640 4 set_mon_vals no callback set 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 monclient: _renew_subs 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0813a2e0 con 0x7f8f080a56a0 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:39.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f080a5160 con 0x7f8f080a56a0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8ef801a070 con 0x7f8f080a56a0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ee803b9a0 0x7f8ee803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7f8ef804fe80 con 0x7f8f080a56a0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ee803b9a0 0x7f8ee803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 10 monclient: get_auth_request con 0x7f8ee803b9a0 auth_method 0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.957+0000 7f8f077fe640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ee803b9a0 0x7f8ee803de60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00af000 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:39.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.969+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8ef8007070 con 0x7f8f080a56a0 2024-09-15T21:21:39.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.969+0000 7f8efffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:39.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:39.969+0000 7f8efffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:40.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8f0e250640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:40.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8f0e250640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:40.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8f0e250640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f8f080a4850 con 0x7f8f080a56a0 2024-09-15T21:21:40.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v113) ==== 56+0+1449 (secure 0 0 0) 0x7f8ef801bdf0 con 0x7f8f080a56a0 2024-09-15T21:21:40.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8efffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T21:21:40.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.133+0000 7f8efffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:40.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.225+0000 7f8f0e250640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T21:21:40.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.225+0000 7f8f0e250640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:40.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.225+0000 7f8f0e250640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.225+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f8f08072cf0 con 0x7f8f080a56a0 2024-09-15T21:21:40.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8efffff640 1 -- 172.21.15.119:0/3736113939 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v113) ==== 56+0+1449 (secure 0 0 0) 0x7f8ef80323a0 con 0x7f8f080a56a0 2024-09-15T21:21:40.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8efffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T21:21:40.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8efffff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:40.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ee803b9a0 msgr2=0x7f8ee803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ee803b9a0 0x7f8ee803de60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00af000 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2024-09-15T21:21:40.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 10 monclient: shutdown 2024-09-15T21:21:40.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 msgr2=0x7f8f08132020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 0x7f8f08132020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c870 tx=0x7f8ef800cd30 comp rx=0 tx=0).stop 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 shutdown_connections 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8ee803b9a0 0x7f8ee803de60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f08138430 0x7f8f08132560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f080a56a0 0x7f8f08132020 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 --2- 172.21.15.119:0/3736113939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f080a4d80 0x7f8f08131ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 >> 172.21.15.119:0/3736113939 conn(0x7f8f080a75c0 msgr2=0x7f8f080a8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 shutdown_connections 2024-09-15T21:21:40.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.229+0000 7f8f0e250640 1 -- 172.21.15.119:0/3736113939 wait complete. 2024-09-15T21:21:40.252 INFO:tasks.workunit.client.0.smithi119.stdout: 14 155.00000 osd.14 down 1.00000 1.00000 2024-09-15T21:21:40.252 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.12 2024-09-15T21:21:40.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient: get_monmap_and_config 2024-09-15T21:21:40.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient: build_initial_monmap 2024-09-15T21:21:40.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient: monmap: 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:40.362216+0000 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:40.362216+0000 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:40.364 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 Processor -- start 2024-09-15T21:21:40.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 -- start start 2024-09-15T21:21:40.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient: init 2024-09-15T21:21:40.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f14e0137bc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14e00a09e0 0x7f14e013ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): picked mon.noname-c con 0x7f14e00a09e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f14e00a0ea0 0x7f14e013efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): picked mon.noname-a con 0x7f14e00a0ea0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e009cfa0 con 0x7f14e0137bc0 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f14e01386b0 con 0x7f14e00a0ea0 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f14e009cd20 con 0x7f14e00a09e0 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:40.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e7b6c640 10 monclient(hunting): authenticate will time out at 1219.158806s 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e5107640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f14e00a09e0 msgr2=0x7f14e013ca90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e5107640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14e00a09e0 0x7f14e013ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f14e00a0ea0 0x7f14e013efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f14e00a0ea0 0x7f14e013efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:35924/0 (socket says 172.21.15.119:35924) 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 -- 172.21.15.119:0/2795155052 learned_addr learned my addr 172.21.15.119:0/2795155052 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): get_auth_request con 0x7f14e00a0ea0 auth_method 0 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient(hunting): handle_auth_done global_id 54231 payload 274 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 -- 172.21.15.119:0/2795155052 >> v2:172.21.15.119:3300/0 conn(0x7f14e00a09e0 msgr2=0x7f14e013ca90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 --2- 172.21.15.119:0/2795155052 >> v2:172.21.15.119:3300/0 conn(0x7f14e00a09e0 0x7f14e013ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 -- 172.21.15.119:0/2795155052 >> v1:172.21.15.17:6789/0 conn(0x7f14e0137bc0 legacy=0x7f14e0137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient: _finish_hunting 0 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 monclient: found mon.noname-a 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:40.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.357+0000 7f14e6109640 1 -- 172.21.15.119:0/2795155052 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e00a1820 con 0x7f14e00a0ea0 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e6109640 1 --2- 172.21.15.119:0/2795155052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e013efe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e0069ac0 tx=0x7f14d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ca9d0448ed7356 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 1 -- 172.21.15.119:0/2795155052 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80ab680 con 0x7f14e00a0ea0 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: dump: 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.374 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: _finish_auth 0 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: _check_auth_tickets 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 1 -- 172.21.15.119:0/2795155052 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d80b0070 con 0x7f14e00a0ea0 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 5 monclient: authenticate success, global_id 54231 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:40.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 1 -- 172.21.15.119:0/2795155052 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80aba40 con 0x7f14e00a0ea0 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 4 set_mon_vals no callback set 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e4906640 10 monclient: dump: 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.376 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 10 monclient: shutdown 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2795155052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 msgr2=0x7f14e013efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2795155052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e013efe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e0069ac0 tx=0x7f14d801c460 comp rx=0 tx=0).stop 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2795155052 shutdown_connections 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2795155052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e013efe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2795155052 >> v2:172.21.15.119:3300/0 conn(0x7f14e00a09e0 0x7f14e013ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2795155052 >> 172.21.15.119:0/2795155052 conn(0x7f14e00a2ae0 msgr2=0x7f14e00a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2795155052 shutdown_connections 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2795155052 wait complete. 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 10 monclient: build_initial_monmap 2024-09-15T21:21:40.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.361+0000 7f14e7b6c640 10 monclient: monmap: 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 Processor -- start 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- start start 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: init 2024-09-15T21:21:40.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00a09e0 0x7f14e01cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): picked mon.noname-b con 0x7f14e00a09e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e01cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): picked mon.noname-a con 0x7f14e00a0ea0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 0x7f14e013ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): picked mon.noname-c con 0x7f14e0137bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0141fb0 con 0x7f14e00a0ea0 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0141e30 con 0x7f14e00a09e0 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f14e0142130 con 0x7f14e0137bc0 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient(hunting): authenticate will time out at 1219.164653s 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5908640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00a09e0 msgr2=0x7f14e01cf500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5908640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00a09e0 0x7f14e01cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e01cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 0x7f14e013ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e01cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:35930/0 (socket says 172.21.15.119:35930) 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 0x7f14e013ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33788/0 (socket says 172.21.15.119:33788) 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 1 -- 172.21.15.119:0/2448827597 learned_addr learned my addr 172.21.15.119:0/2448827597 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): get_auth_request con 0x7f14e00a0ea0 auth_method 0 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): get_auth_request con 0x7f14e0137bc0 auth_method 0 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e5107640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient(hunting): handle_auth_done global_id 44369 payload 824 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 -- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00a09e0 msgr2=0x7f14e01cf500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00a09e0 0x7f14e01cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 -- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 msgr2=0x7f14e01cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e01cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient: _finish_hunting 0 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 monclient: found mon.noname-c 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 -- 172.21.15.119:0/2448827597 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e011db30 con 0x7f14e0137bc0 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e6109640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 0x7f14e013ca90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ac000 tx=0x7f14d801aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:40.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b4800 con 0x7f14e0137bc0 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: dump: 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d80a7070 con 0x7f14e0137bc0 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b4be0 con 0x7f14e0137bc0 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 10 monclient: dump: 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.384 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 5 monclient: authenticate success, global_id 44369 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _renew_subs 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d7fff640 4 set_mon_vals no callback set 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e011de10 con 0x7f14e0137bc0 2024-09-15T21:21:40.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _renew_subs 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e013e770 con 0x7f14e0137bc0 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e0137fa0 con 0x7f14e0137bc0 2024-09-15T21:21:40.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f14d80059a0 con 0x7f14e0137bc0 2024-09-15T21:21:40.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f14c403b970 0x7f14c403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.365+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 6643+0+0 (secure 0 0 0) 0x7f14d80eb110 con 0x7f14e0137bc0 2024-09-15T21:21:40.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.369+0000 7f14e5908640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f14c403b970 0x7f14c403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.369+0000 7f14e5908640 10 monclient: get_auth_request con 0x7f14c403b970 auth_method 0 2024-09-15T21:21:40.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.369+0000 7f14e5908640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f14c403b970 0x7f14c403de30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f14dc007fd0 tx=0x7f14dc007d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:40.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.381+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f14d8016110 con 0x7f14e0137bc0 2024-09-15T21:21:40.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.381+0000 7f14d67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:40.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.381+0000 7f14d67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:40.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.545+0000 7f14e7b6c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T21:21:40.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.545+0000 7f14e7b6c640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T21:21:40.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.545+0000 7f14e7b6c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.545+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) -- 0x7f14e006c460 con 0x7f14e0137bc0 2024-09-15T21:21:40.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.733+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 removed item id 12 name 'osd.12' from crush map v114) ==== 125+0+0 (secure 0 0 0) 0x7f14d80b4d80 con 0x7f14e0137bc0 2024-09-15T21:21:40.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.733+0000 7f14d67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T21:21:40.739 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.733+0000 7f14d67fc640 10 monclient: _finish_command 2 = system:0 removed item id 12 name 'osd.12' from crush map 2024-09-15T21:21:40.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14e7b6c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T21:21:40.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14e7b6c640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T21:21:40.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14e7b6c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.839 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) -- 0x7f14e011cbd0 con 0x7f14e0137bc0 2024-09-15T21:21:40.840 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14d67fc640 1 -- 172.21.15.119:0/2448827597 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 device 'osd.12' does not appear in the crush map v114) ==== 126+0+0 (secure 0 0 0) 0x7f14d80bc900 con 0x7f14e0137bc0 2024-09-15T21:21:40.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14d67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T21:21:40.841 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.833+0000 7f14d67fc640 10 monclient: _finish_command 3 = system:0 device 'osd.12' does not appear in the crush map 2024-09-15T21:21:40.841 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.12' does not appear in the crush map 2024-09-15T21:21:40.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f14c403b970 msgr2=0x7f14c403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f14c403b970 0x7f14c403de30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f14dc007fd0 tx=0x7f14dc007d80 comp rx=0 tx=0).stop 2024-09-15T21:21:40.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 10 monclient: shutdown 2024-09-15T21:21:40.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:40.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 msgr2=0x7f14e013ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.843 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 0x7f14e013ca90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ac000 tx=0x7f14d801aca0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 shutdown_connections 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f14c403b970 0x7f14c403de30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14e0137bc0 0x7f14e013ca90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00a0ea0 0x7f14e01cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 --2- 172.21.15.119:0/2448827597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00a09e0 0x7f14e01cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 >> 172.21.15.119:0/2448827597 conn(0x7f14e00a2ae0 msgr2=0x7f14e009c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 shutdown_connections 2024-09-15T21:21:40.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.837+0000 7f14e7b6c640 1 -- 172.21.15.119:0/2448827597 wait complete. 2024-09-15T21:21:40.858 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm osd.14 2024-09-15T21:21:40.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient: get_monmap_and_config 2024-09-15T21:21:40.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient: build_initial_monmap 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient: monmap: 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:40.970251+0000 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:40.970251+0000 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:40.972 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 Processor -- start 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 -- start start 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient: init 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ff0135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 0x7f8ff013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): picked mon.noname-b con 0x7f8ff00a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ff006ec30 0x7f8ff013cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): picked mon.noname-a con 0x7f8ff006ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff009cf30 con 0x7f8ff0135ca0 2024-09-15T21:21:40.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff00a1ca0 con 0x7f8ff006ec30 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff00a1f80 con 0x7f8ff00a0160 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff8040640 10 monclient(hunting): authenticate will time out at 1219.766965s 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 0x7f8ff013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ff006ec30 0x7f8ff013cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 0x7f8ff013aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33792/0 (socket says 172.21.15.119:33792) 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 -- 172.21.15.119:0/1094036346 learned_addr learned my addr 172.21.15.119:0/1094036346 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): get_auth_request con 0x7f8ff00a0160 auth_method 0 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): get_auth_request con 0x7f8ff006ec30 auth_method 0 2024-09-15T21:21:40.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff65dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient(hunting): handle_auth_done global_id 44372 payload 274 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 -- 172.21.15.119:0/1094036346 >> v2:172.21.15.17:3300/0 conn(0x7f8ff006ec30 msgr2=0x7f8ff013cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 --2- 172.21.15.119:0/1094036346 >> v2:172.21.15.17:3300/0 conn(0x7f8ff006ec30 0x7f8ff013cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 -- 172.21.15.119:0/1094036346 >> v1:172.21.15.17:6789/0 conn(0x7f8ff0135ca0 legacy=0x7f8ff0136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient: _finish_hunting 0 2024-09-15T21:21:40.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 monclient: found mon.noname-b 2024-09-15T21:21:40.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:40.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.965+0000 7f8ff55db640 1 -- 172.21.15.119:0/1094036346 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff00a3860 con 0x7f8ff00a0160 2024-09-15T21:21:40.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff55db640 1 --2- 172.21.15.119:0/1094036346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 0x7f8ff013aa00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009870 tx=0x7f8fe001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb22904427d442fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:40.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 1 -- 172.21.15.119:0/1094036346 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b0070 con 0x7f8ff00a0160 2024-09-15T21:21:40.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:40.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: dump: 2024-09-15T21:21:40.984 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: _finish_auth 0 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: _check_auth_tickets 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 1 -- 172.21.15.119:0/1094036346 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe00ab600 con 0x7f8ff00a0160 2024-09-15T21:21:40.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 1 -- 172.21.15.119:0/1094036346 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00ab9d0 con 0x7f8ff00a0160 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff4dda640 10 monclient: dump: 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.986 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 5 monclient: authenticate success, global_id 44372 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 4 set_mon_vals no callback set 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 10 monclient: shutdown 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 -- 172.21.15.119:0/1094036346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 msgr2=0x7f8ff013aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 --2- 172.21.15.119:0/1094036346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 0x7f8ff013aa00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009870 tx=0x7f8fe001c420 comp rx=0 tx=0).stop 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 -- 172.21.15.119:0/1094036346 shutdown_connections 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 --2- 172.21.15.119:0/1094036346 >> v2:172.21.15.17:3300/0 conn(0x7f8ff006ec30 0x7f8ff013cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 --2- 172.21.15.119:0/1094036346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff00a0160 0x7f8ff013aa00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 -- 172.21.15.119:0/1094036346 >> 172.21.15.119:0/1094036346 conn(0x7f8ff00a2ae0 msgr2=0x7f8ff00a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 -- 172.21.15.119:0/1094036346 shutdown_connections 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 1 -- 172.21.15.119:0/1094036346 wait complete. 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 10 monclient: build_initial_monmap 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.969+0000 7f8ff8040640 10 monclient: monmap: 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:40.988 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 Processor -- start 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- start start 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: init 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 0x7f8ff0130cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff006ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00a0160 0x7f8ff0131200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): picked mon.noname-b con 0x7f8ff00a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff0135ca0 0x7f8ff01cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): picked mon.noname-a con 0x7f8ff0135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff013fe30 con 0x7f8ff0135ca0 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff013fcb0 con 0x7f8ff00a0160 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff013ffb0 con 0x7f8ff006ec30 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient(hunting): authenticate will time out at 1219.772761s 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff55db640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00a0160 msgr2=0x7f8ff0131200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff55db640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00a0160 0x7f8ff0131200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:40.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 0x7f8ff0130cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff0135ca0 0x7f8ff01cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 0x7f8ff0130cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33800/0 (socket says 172.21.15.119:33800) 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 -- 172.21.15.119:0/3104382801 learned_addr learned my addr 172.21.15.119:0/3104382801 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): get_auth_request con 0x7f8ff006ec30 auth_method 0 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): get_auth_request con 0x7f8ff0135ca0 auth_method 0 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff65dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient(hunting): handle_auth_done global_id 44375 payload 824 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 -- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00a0160 msgr2=0x7f8ff0131200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00a0160 0x7f8ff0131200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 -- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff0135ca0 msgr2=0x7f8ff01cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff0135ca0 0x7f8ff01cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 monclient: found mon.noname-c 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 -- 172.21.15.119:0/3104382801 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0139c40 con 0x7f8ff006ec30 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff5ddc640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 0x7f8ff0130cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c530 tx=0x7f8fec00c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec017070 con 0x7f8ff006ec30 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: dump: 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.993 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 5 monclient: authenticate success, global_id 44375 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _renew_subs 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff00a58a0 con 0x7f8ff006ec30 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec0052d0 con 0x7f8ff006ec30 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec012870 con 0x7f8ff006ec30 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:40.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 10 monclient: dump: 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe7fff640 4 set_mon_vals no callback set 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _renew_subs 2024-09-15T21:21:40.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff013bfa0 con 0x7f8ff006ec30 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff0136080 con 0x7f8ff006ec30 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8fec01a070 con 0x7f8ff006ec30 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(114..114 src has 1..114) ==== 6621+0+0 (secure 0 0 0) 0x7f8fec04f680 con 0x7f8ff006ec30 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff55db640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.973+0000 7f8ff55db640 10 monclient: get_auth_request con 0x7f8fd403b970 auth_method 0 2024-09-15T21:21:40.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.977+0000 7f8ff55db640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00ad000 tx=0x7f8fe0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:40.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.985+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8fec005480 con 0x7f8ff006ec30 2024-09-15T21:21:40.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.985+0000 7f8fe67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:40.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:40.985+0000 7f8fe67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:41.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 10 monclient(hunting): tick 2024-09-15T21:21:41.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:21:41.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 1 monclient(hunting): continuing hunt 2024-09-15T21:21:41.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:21:41.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 msgr2=0x7ff2d4008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.17:3300/0 conn(0x7ff2d4006250 0x7ff2d4008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2ece77640 1 -- 172.21.15.17:0/1655816904 reap_dead start 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.017+0000 7ff2e5ffb640 10 monclient(hunting): picked mon.noname-c con 0x7ff2d4003700 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2d4006700 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2d400a9c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e80a1bf0 con 0x7ff2d4006700 2024-09-15T21:21:41.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e80a3130 con 0x7ff2d400a9c0 2024-09-15T21:21:41.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 1 -- 172.21.15.17:0/1655816904 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e8073df0 con 0x7ff2d4003700 2024-09-15T21:21:41.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e5ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:41.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:41.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.119:6789/0 conn(0x7ff2d400a9c0 legacy=0x7ff2d400cdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058637115 0 0) 0x7ff2e80a1bf0 con 0x7ff2d4006700 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): my global_id is 54249 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2d0003530 con 0x7ff2d4006700 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4062464706 0 0) 0x7ff2d0003530 con 0x7ff2d4006700 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 msgr2=0x7ff2d4005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 --2- 172.21.15.17:0/1655816904 >> v2:172.21.15.119:3300/0 conn(0x7ff2d4003700 0x7ff2d4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.119:6789/0 conn(0x7ff2d400a9c0 legacy=0x7ff2d400cdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:21:41.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d4010690 con 0x7ff2d4006700 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff2dc009950 con 0x7ff2d4006700 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ee8da640 5 monclient: authenticate success, global_id 54249 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ee8da640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: dump: 2024-09-15T21:21:41.030 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 >> v1:172.21.15.17:6789/0 conn(0x7ff2d4006700 legacy=0x7ff2d40092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:41.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2d00039c0 0x7ff2d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): picked mon.c con 0x7ff2d00039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d0004810 0x7ff2d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): picked mon.b con 0x7ff2d0004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00052f0 0x7ff2d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): picked mon.a con 0x7ff2d00052f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8073df0 con 0x7ff2d00052f0 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d000c240 con 0x7ff2d0004810 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d000c440 con 0x7ff2d00039c0 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ee8da640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2d00039c0 0x7ff2d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request con 0x7ff2d00039c0 auth_method 0 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00052f0 0x7ff2d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e77fe640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d0004810 msgr2=0x7ff2d0004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): get_auth_request con 0x7ff2d00052f0 auth_method 0 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d0004810 0x7ff2d0004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_done global_id 54249 payload 293 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d0004810 msgr2=0x7ff2d0004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d0004810 0x7ff2d0004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00052f0 msgr2=0x7ff2d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00052f0 0x7ff2d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 monclient: found mon.c 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 1 -- 172.21.15.17:0/1655816904 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d0003530 con 0x7ff2d00039c0 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.021+0000 7ff2ece77640 1 -- 172.21.15.17:0/1655816904 reap_dead start 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2d00039c0 0x7ff2d0003dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000ee0 tx=0x7ff2d801d2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce80984cff1ed191 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80b3070 con 0x7ff2d00039c0 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: dump: 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.040 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d80ae410 con 0x7ff2d00039c0 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 4 set_mon_vals no callback set 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 1 -- 172.21.15.17:0/1655816904 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800f040 con 0x7ff2d00039c0 2024-09-15T21:21:41.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2e6ffd640 10 monclient: dump: 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 10 monclient: shutdown 2024-09-15T21:21:41.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 -- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2d00039c0 msgr2=0x7ff2d0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2d00039c0 0x7ff2d0003dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000ee0 tx=0x7ff2d801d2d0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 -- 172.21.15.17:0/1655816904 shutdown_connections 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/1655816904 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2d00039c0 0x7ff2d0003dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 -- 172.21.15.17:0/1655816904 >> 172.21.15.17:0/1655816904 conn(0x7ff2e80a66a0 msgr2=0x7ff2e80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 -- 172.21.15.17:0/1655816904 shutdown_connections 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.025+0000 7ff2ee8da640 1 -- 172.21.15.17:0/1655816904 wait complete. 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: build_initial_monmap 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: monmap: 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:41.043 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 Processor -- start 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 -- start start 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: init 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00039c0 0x7ff2e8121c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): picked mon.noname-a con 0x7ff2d00039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:41.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 0x7ff2e8122620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): picked mon.noname-c con 0x7ff2e81221a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e81d3c20 0x7ff2e81d6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): picked mon.noname-b con 0x7ff2e81d3c20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8136210 con 0x7ff2d00039c0 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8136090 con 0x7ff2e81d3c20 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e81347e0 con 0x7ff2e81221a0 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient(hunting): authenticate will time out at 1224.426300s 2024-09-15T21:21:41.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 0x7ff2e8122620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ece77640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e81d3c20 msgr2=0x7ff2e81d6010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 0x7ff2e8122620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52314/0 (socket says 172.21.15.17:52314) 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 -- 172.21.15.17:0/3670630385 learned_addr learned my addr 172.21.15.17:0/3670630385 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request con 0x7ff2e81221a0 auth_method 0 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ece77640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e81d3c20 0x7ff2e81d6010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00039c0 0x7ff2e8121c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request con 0x7ff2d00039c0 auth_method 0 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient(hunting): handle_auth_done global_id 44378 payload 824 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 -- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e81d3c20 msgr2=0x7ff2e81d6010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e81d3c20 0x7ff2e81d6010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 -- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00039c0 msgr2=0x7ff2e8121c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00039c0 0x7ff2e8121c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 monclient: found mon.noname-c 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 -- 172.21.15.17:0/3670630385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e8125ea0 con 0x7ff2e81221a0 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e77fe640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 0x7ff2e8122620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00be10 tx=0x7ff2dc00a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc00ae60 con 0x7ff2e81221a0 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: dump: 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.048 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2dc01c070 con 0x7ff2e81221a0 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc016a30 con 0x7ff2e81221a0 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 5 monclient: authenticate success, global_id 44378 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: _renew_subs 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e8125a80 con 0x7ff2e81221a0 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e67fc640 4 set_mon_vals no callback set 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2e4ff9640 10 monclient: dump: 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.050 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: _renew_subs 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.029+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e8124c10 con 0x7ff2e81221a0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff2dc016bd0 con 0x7ff2e81221a0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2ee8da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2ee8da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2ee8da640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e8138ab0 con 0x7ff2e81221a0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2e4ff9640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2d403ce60 0x7ff2d403f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(114..114 src has 1..114) ==== 6621+0+0 (secure 0 0 0) 0x7ff2dc051350 con 0x7ff2e81221a0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2d403ce60 0x7ff2d403f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2e7fff640 10 monclient: get_auth_request con 0x7ff2d403ce60 auth_method 0 2024-09-15T21:21:41.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.033+0000 7ff2e7fff640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2d403ce60 0x7ff2d403f320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80b0000 tx=0x7ff2d8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.037+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff2e8125ea0 con 0x7ff2e81221a0 2024-09-15T21:21:41.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.037+0000 7ff2e4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.037+0000 7ff2e4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.149+0000 7f8ff8040640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T21:21:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.149+0000 7f8ff8040640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T21:21:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.149+0000 7f8ff8040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.149+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) -- 0x7f8ff013d130 con 0x7f8ff006ec30 2024-09-15T21:21:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:21:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff2e81bf360 con 0x7ff2e81221a0 2024-09-15T21:21:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2e4ff9640 1 -- 172.21.15.17:0/3670630385 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7ff2dc012050 con 0x7ff2e81221a0 2024-09-15T21:21:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2e4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:21:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2e4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:41.293 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":48,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2d403ce60 msgr2=0x7ff2d403f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2d403ce60 0x7ff2d403f320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80b0000 tx=0x7ff2d8004740 comp rx=0 tx=0).stop 2024-09-15T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 10 monclient: shutdown 2024-09-15T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 msgr2=0x7ff2e8122620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 0x7ff2e8122620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00be10 tx=0x7ff2dc00a7b0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 shutdown_connections 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff2d403ce60 0x7ff2d403f320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e81d3c20 0x7ff2e81d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2e81221a0 0x7ff2e8122620 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 --2- 172.21.15.17:0/3670630385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d00039c0 0x7ff2e8121c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 >> 172.21.15.17:0/3670630385 conn(0x7ff2e80a66a0 msgr2=0x7ff2e80a53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 shutdown_connections 2024-09-15T21:21:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.289+0000 7ff2ee8da640 1 -- 172.21.15.17:0/3670630385 wait complete. 2024-09-15T21:21:41.312 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:21:41.312 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:21:41.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:21:41.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient: get_monmap_and_config 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient: build_initial_monmap 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient: monmap: 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:41.435460+0000 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:41.435460+0000 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:41.438 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:41.439 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 Processor -- start 2024-09-15T21:21:41.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 -- start start 2024-09-15T21:21:41.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient: init 2024-09-15T21:21:41.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:21:41.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f576c073640 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:41.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 0x7f576c13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): picked mon.noname-b con 0x7f576c074130 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f576c13d140 0x7f576c13f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): picked mon.noname-a con 0x7f576c13d140 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f576c0a0890 con 0x7f576c073640 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f576c0a3640 con 0x7f576c13d140 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f576c0746e0 con 0x7f576c074130 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5772da6640 10 monclient(hunting): authenticate will time out at 1224.830868s 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 0x7f576c13cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 0x7f576c13cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52324/0 (socket says 172.21.15.17:52324) 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5770b42640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f576c073640 0x7f576c073a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48156/0 (socket says 172.21.15.17:48156) 2024-09-15T21:21:41.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 1 -- 172.21.15.17:0/1080968574 learned_addr learned my addr 172.21.15.17:0/1080968574 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): get_auth_request con 0x7f576c074130 auth_method 0 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 1 --2- 172.21.15.17:0/1080968574 >> v2:172.21.15.17:3300/0 conn(0x7f576c13d140 0x7f576c13f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): get_auth_request con 0x7f576c13d140 auth_method 0 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient(hunting): handle_auth_done global_id 54255 payload 274 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 1 -- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 msgr2=0x7f576c13cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 1 --2- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 0x7f576c13cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 1 -- 172.21.15.17:0/1080968574 >> v1:172.21.15.17:6789/0 conn(0x7f576c073640 legacy=0x7f576c073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 1 monclient: found mon.noname-a 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f5771343640 1 -- 172.21.15.17:0/1080968574 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c0a1640 con 0x7f576c13d140 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.433+0000 7f576bfff640 1 --2- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 0x7f576c13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5771343640 1 --2- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c13d140 0x7f576c13f530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f575c0097c0 tx=0x7f575c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7210e7c1a1b60c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 1 -- 172.21.15.17:0/1080968574 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0b0070 con 0x7f576c13d140 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: dump: 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 1 -- 172.21.15.17:0/1080968574 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c0ab600 con 0x7f576c13d140 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 5 monclient: authenticate success, global_id 54255 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 1 -- 172.21.15.17:0/1080968574 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0ab9d0 con 0x7f576c13d140 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 4 set_mon_vals no callback set 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f576b7fe640 10 monclient: dump: 2024-09-15T21:21:41.446 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.447 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.448 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: shutdown 2024-09-15T21:21:41.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c13d140 msgr2=0x7f576c13f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 --2- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c13d140 0x7f576c13f530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f575c0097c0 tx=0x7f575c01c330 comp rx=0 tx=0).stop 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- 172.21.15.17:0/1080968574 shutdown_connections 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 --2- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c13d140 0x7f576c13f530 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 --2- 172.21.15.17:0/1080968574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c074130 0x7f576c13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- 172.21.15.17:0/1080968574 >> 172.21.15.17:0/1080968574 conn(0x7f576c0a66a0 msgr2=0x7f576c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- 172.21.15.17:0/1080968574 shutdown_connections 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- 172.21.15.17:0/1080968574 wait complete. 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: build_initial_monmap 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: monmap: 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:41.455 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 Processor -- start 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- start start 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: init 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c073640 0x7f576c120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): picked mon.noname-c con 0x7f576c073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c074130 0x7f576c11ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): picked mon.noname-a con 0x7f576c074130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f576c13d140 0x7f576c137bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): picked mon.noname-b con 0x7f576c13d140 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0a0ec0 con 0x7f576c074130 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0a0d40 con 0x7f576c13d140 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f576c142290 con 0x7f576c073640 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.437+0000 7f5772da6640 10 monclient(hunting): authenticate will time out at 1224.835913s 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c073640 0x7f576c120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c074130 0x7f576c11ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c074130 0x7f576c11ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46268/0 (socket says 172.21.15.17:46268) 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c073640 0x7f576c120d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52332/0 (socket says 172.21.15.17:52332) 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 -- 172.21.15.17:0/2327705621 learned_addr learned my addr 172.21.15.17:0/2327705621 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): get_auth_request con 0x7f576c074130 auth_method 0 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): get_auth_request con 0x7f576c073640 auth_method 0 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5771343640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f576c13d140 msgr2=0x7f576c137bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5771343640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f576c13d140 0x7f576c137bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient(hunting): handle_auth_done global_id 54261 payload 824 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c073640 msgr2=0x7f576c120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c073640 0x7f576c120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f576c13d140 msgr2=0x7f576c137bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f576c13d140 0x7f576c137bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 monclient: found mon.noname-a 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1201e0 con 0x7f576c074130 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f576c073640 0x7f576c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576bfff640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c074130 0x7f576c11ec00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f575400ea00 tx=0x7f575400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754017070 con 0x7f576c074130 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: dump: 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575400ca60 con 0x7f576c074130 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 5 monclient: authenticate success, global_id 54261 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400ce10 con 0x7f576c074130 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: dump: 2024-09-15T21:21:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f576affd640 4 set_mon_vals no callback set 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 10 monclient: _renew_subs 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c09d0f0 con 0x7f576c074130 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 10 monclient: _renew_subs 2024-09-15T21:21:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c122540 con 0x7f576c074130 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 0x7f576c1cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f576c073a20 con 0x7f576c1380f0 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 0x7f576c1cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient: get_auth_request con 0x7f576c1380f0 auth_method 0 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f575401a070 con 0x7f576c074130 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 10 monclient(hunting): handle_auth_done global_id 54264 payload 824 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f576c121d60 con 0x7f576c1380f0 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f574c03b970 0x7f574c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6621+0+0 (secure 0 0 0) 0x7f5754050130 con 0x7f576c074130 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5770b42640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 0x7f576c1cfe20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f576c121350 tx=0x7f576400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764007a30 con 0x7f576c1380f0 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: dump: 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5771343640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f574c03b970 0x7f574c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5764018070 con 0x7f576c1380f0 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 msgr2=0x7f576c1cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 0x7f576c1cfe20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f576c121350 tx=0x7f576400f5a0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 msgr2=0x7f576c1cfe20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f57697fa640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c1380f0 0x7f576c1cfe20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5771343640 10 monclient: get_auth_request con 0x7f574c03b970 auth_method 0 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.441+0000 7f5771343640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f574c03b970 0x7f574c03de30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f575c0ad000 tx=0x7f575c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 0x7f576c02d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f576c13b300 con 0x7f576c02d450 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 0x7f576c02d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient: get_auth_request con 0x7f576c02d450 auth_method 0 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 10 monclient(hunting): handle_auth_done global_id 54267 payload 824 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 1 -- 172.21.15.17:0/2327705621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f576c1d0360 con 0x7f576c02d450 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5770b42640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 0x7f576c02d860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f576c121350 tx=0x7f5764001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576400fe60 con 0x7f576c02d450 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 10 monclient: dump: 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4702 (secure 0 0 0) 0x7f576400fe60 con 0x7f576c02d450 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 msgr2=0x7f576c02d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 0x7f576c02d860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f576c121350 tx=0x7f5764001260 comp rx=0 tx=0).stop 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 msgr2=0x7f576c02d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f57697fa640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c02d450 0x7f576c02d860 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f576c121350 tx=0x7f5764001260 comp rx=0 tx=0).stop 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 48, 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 26811, 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:41.469 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:41.470 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:41.471 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:41.472 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:41.473 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:41.474 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:41.476 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:41.477 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:41.478 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:41.479 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.480 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.481 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.482 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:21:41.482 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.482 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.482 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:41.482 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:41.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f574c03b970 msgr2=0x7f574c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f574c03b970 0x7f574c03de30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f575c0ad000 tx=0x7f575c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 10 monclient: shutdown 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c074130 msgr2=0x7f576c11ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 --2- 172.21.15.17:0/2327705621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f576c074130 0x7f576c11ec00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f575400ea00 tx=0x7f575400eec0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5771343640 1 -- 172.21.15.17:0/2327705621 reap_dead start 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5771343640 1 -- 172.21.15.17:0/2327705621 reap_dead start 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 shutdown_connections 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 >> 172.21.15.17:0/2327705621 conn(0x7f576c0a66a0 msgr2=0x7f576c09e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 shutdown_connections 2024-09-15T21:21:41.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.449+0000 7f5772da6640 1 -- 172.21.15.17:0/2327705621 wait complete. 2024-09-15T21:21:41.485 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:21:41.485 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:21:41.485 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient: get_monmap_and_config 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient: build_initial_monmap 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient: monmap: 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:21:41.610281+0000 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:21:41.610281+0000 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:41.612 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:41.613 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 Processor -- start 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 -- start start 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient: init 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): picked mon.noname-b con 0x7f9ce81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ce81381e0 0x7f9ce80a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): picked mon.noname-c con 0x7f9ce81381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9ce80a5040 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce80a1d00 con 0x7f9ce80a5040 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce80a1fe0 con 0x7f9ce81381e0 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce80a22c0 con 0x7f9ce81378c0 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ceff7d640 10 monclient(hunting): authenticate will time out at 1225.005792s 2024-09-15T21:21:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52344/0 (socket says 172.21.15.17:52344) 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 -- 172.21.15.17:0/902945660 learned_addr learned my addr 172.21.15.17:0/902945660 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): get_auth_request con 0x7f9ce81378c0 auth_method 0 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ced518640 1 -- 172.21.15.17:0/902945660 >> v2:172.21.15.119:3300/0 conn(0x7f9ce81381e0 msgr2=0x7f9ce80a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9ced518640 1 --2- 172.21.15.17:0/902945660 >> v2:172.21.15.119:3300/0 conn(0x7f9ce81381e0 0x7f9ce80a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient(hunting): handle_auth_done global_id 44387 payload 274 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 -- 172.21.15.17:0/902945660 >> v2:172.21.15.119:3300/0 conn(0x7f9ce81381e0 msgr2=0x7f9ce80a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 --2- 172.21.15.17:0/902945660 >> v2:172.21.15.119:3300/0 conn(0x7f9ce81381e0 0x7f9ce80a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 -- 172.21.15.17:0/902945660 >> v1:172.21.15.17:6789/0 conn(0x7f9ce80a5040 legacy=0x7f9ce80a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 monclient: found mon.noname-b 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.609+0000 7f9cedd19640 1 -- 172.21.15.17:0/902945660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce80a5c00 con 0x7f9ce81378c0 2024-09-15T21:21:41.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 1 --2- 172.21.15.17:0/902945660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8137ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009e50 tx=0x7f9ce001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b53b5f63c5227a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 1 -- 172.21.15.17:0/902945660 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00a56e0 con 0x7f9ce81378c0 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: dump: 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 1 -- 172.21.15.17:0/902945660 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce00a5880 con 0x7f9ce81378c0 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 1 -- 172.21.15.17:0/902945660 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00a5c50 con 0x7f9ce81378c0 2024-09-15T21:21:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 5 monclient: authenticate success, global_id 44387 2024-09-15T21:21:41.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cecd17640 10 monclient: dump: 2024-09-15T21:21:41.628 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.628 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 4 set_mon_vals no callback set 2024-09-15T21:21:41.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: shutdown 2024-09-15T21:21:41.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- 172.21.15.17:0/902945660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 msgr2=0x7f9ce8137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/902945660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8137ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009e50 tx=0x7f9ce001c9a0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- 172.21.15.17:0/902945660 shutdown_connections 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/902945660 >> v2:172.21.15.119:3300/0 conn(0x7f9ce81381e0 0x7f9ce80a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/902945660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8137ca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- 172.21.15.17:0/902945660 >> 172.21.15.17:0/902945660 conn(0x7f9ce80aac40 msgr2=0x7f9ce80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- 172.21.15.17:0/902945660 shutdown_connections 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- 172.21.15.17:0/902945660 wait complete. 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: build_initial_monmap 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: monmap: 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:41.631 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 Processor -- start 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- start start 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: init 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:41.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce80a5040 0x7f9ce81cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): picked mon.noname-a con 0x7f9ce80a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8136a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): picked mon.noname-c con 0x7f9ce81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce81381e0 0x7f9ce813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): picked mon.noname-b con 0x7f9ce81381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce806f340 con 0x7f9ce80a5040 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce806f1c0 con 0x7f9ce81381e0 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce81425c0 con 0x7f9ce81378c0 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ceff7d640 10 monclient(hunting): authenticate will time out at 1225.011569s 2024-09-15T21:21:41.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce80a5040 0x7f9ce81cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8136a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce80a5040 0x7f9ce81cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46298/0 (socket says 172.21.15.17:46298) 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 1 -- 172.21.15.17:0/2183137659 learned_addr learned my addr 172.21.15.17:0/2183137659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): get_auth_request con 0x7f9ce80a5040 auth_method 0 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cee51a640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce81381e0 msgr2=0x7f9ce813cbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): get_auth_request con 0x7f9ce81378c0 auth_method 0 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cee51a640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce81381e0 0x7f9ce813cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:41.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9ced518640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.613+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 10 monclient(hunting): handle_auth_done global_id 44390 payload 824 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce81381e0 msgr2=0x7f9ce813cbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce81381e0 0x7f9ce813cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce80a5040 msgr2=0x7f9ce81cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce80a5040 0x7f9ce81cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 monclient: found mon.noname-c 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce813e6a0 con 0x7f9ce81378c0 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce80a5040 0x7f9ce81cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ced518640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8136a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800b470 tx=0x7f9cd800b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8003dd0 con 0x7f9ce81378c0 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: dump: 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd8005070 con 0x7f9ce81378c0 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 5 monclient: authenticate success, global_id 44390 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 10 monclient: _renew_subs 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce81213b0 con 0x7f9ce81378c0 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd80054a0 con 0x7f9ce81378c0 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: dump: 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.638 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce7fff640 4 set_mon_vals no callback set 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 10 monclient: _renew_subs 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81d0780 con 0x7f9ce81378c0 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 0x7f9ce802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce8137ca0 con 0x7f9ce8136f60 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 0x7f9ce802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient: get_auth_request con 0x7f9ce8136f60 auth_method 0 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9cd8005640 con 0x7f9ce81378c0 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 10 monclient(hunting): handle_auth_done global_id 44393 payload 824 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cc803b970 0x7f9cc803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ce809f210 con 0x7f9ce8136f60 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(114..114 src has 1..114) ==== 6621+0+0 (secure 0 0 0) 0x7f9cd804fd30 con 0x7f9ce81378c0 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cedd19640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 0x7f9ce802d450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ce001b050 tx=0x7f9ce0019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0009510 con 0x7f9ce8136f60 2024-09-15T21:21:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: dump: 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cee51a640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cc803b970 0x7f9cc803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cee51a640 10 monclient: get_auth_request con 0x7f9cc803b970 auth_method 0 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ce00090d0 con 0x7f9ce8136f60 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 msgr2=0x7f9ce802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 0x7f9ce802d450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ce001b050 tx=0x7f9ce0019740 comp rx=0 tx=0).stop 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 msgr2=0x7f9ce802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9ce67fc640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce8136f60 0x7f9ce802d450 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ce001b050 tx=0x7f9ce0019740 comp rx=0 tx=0).stop 2024-09-15T21:21:41.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.617+0000 7f9cee51a640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cc803b970 0x7f9cc803de30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0007fd0 tx=0x7f9cd0007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 0x7f9ce802dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce813f260 con 0x7f9ce802d990 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 0x7f9ce802dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient: get_auth_request con 0x7f9ce802d990 auth_method 0 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 10 monclient(hunting): handle_auth_done global_id 44396 payload 824 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 1 -- 172.21.15.17:0/2183137659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ce8069270 con 0x7f9ce802d990 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cedd19640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 0x7f9ce802dda0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8137e20 tx=0x7f9ce00a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00c6610 con 0x7f9ce802d990 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 10 monclient: dump: 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f9ce00c6610 con 0x7f9ce802d990 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 msgr2=0x7f9ce802dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 0x7f9ce802dda0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8137e20 tx=0x7f9ce00a7000 comp rx=0 tx=0).stop 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 msgr2=0x7f9ce802dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ce67fc640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce802d990 0x7f9ce802dda0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 48, 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 70223, 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:21:41.648 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:41.649 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:41.650 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:21:41.651 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:21:41.652 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:41.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:21:41.654 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:21:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:21:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:21:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:21:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:21:41.661 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:21:41.661 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cc803b970 msgr2=0x7f9cc803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cc803b970 0x7f9cc803de30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0007fd0 tx=0x7f9cd0007980 comp rx=0 tx=0).stop 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 10 monclient: shutdown 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 msgr2=0x7f9ce8136a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8136a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800b470 tx=0x7f9cd800b930 comp rx=0 tx=0).stop 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9cee51a640 1 -- 172.21.15.17:0/2183137659 reap_dead start 2024-09-15T21:21:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 shutdown_connections 2024-09-15T21:21:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 --2- 172.21.15.17:0/2183137659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9ce81378c0 0x7f9ce8136a20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 >> 172.21.15.17:0/2183137659 conn(0x7f9ce80aac40 msgr2=0x7f9ce80a05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 shutdown_connections 2024-09-15T21:21:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:21:41.625+0000 7f9ceff7d640 1 -- 172.21.15.17:0/2183137659 wait complete. 2024-09-15T21:21:41.663 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:21:41.663 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:21:41.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.741+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 removed item id 14 name 'osd.14' from crush map v115) ==== 125+0+0 (secure 0 0 0) 0x7f8fec021090 con 0x7f8ff006ec30 2024-09-15T21:21:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.741+0000 7f8fe67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T21:21:41.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.741+0000 7f8fe67fc640 10 monclient: _finish_command 2 = system:0 removed item id 14 name 'osd.14' from crush map 2024-09-15T21:21:41.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8ff8040640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T21:21:41.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8ff8040640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T21:21:41.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8ff8040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:41.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) -- 0x7f8ff00008e0 con 0x7f8ff006ec30 2024-09-15T21:21:41.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8fe67fc640 1 -- 172.21.15.119:0/3104382801 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 device 'osd.14' does not appear in the crush map v115) ==== 126+0+0 (secure 0 0 0) 0x7f8fec010a50 con 0x7f8ff006ec30 2024-09-15T21:21:41.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8fe67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T21:21:41.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.853+0000 7f8fe67fc640 10 monclient: _finish_command 3 = system:0 device 'osd.14' does not appear in the crush map 2024-09-15T21:21:41.860 INFO:tasks.workunit.client.0.smithi119.stderr:device 'osd.14' does not appear in the crush map 2024-09-15T21:21:41.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 msgr2=0x7f8fd403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00ad000 tx=0x7f8fe0008040 comp rx=0 tx=0).stop 2024-09-15T21:21:41.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 10 monclient: shutdown 2024-09-15T21:21:41.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 msgr2=0x7f8ff0130cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 0x7f8ff0130cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c530 tx=0x7f8fec00c9f0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 shutdown_connections 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fd403b970 0x7f8fd403de30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ff0135ca0 0x7f8ff01cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00a0160 0x7f8ff0131200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 --2- 172.21.15.119:0/3104382801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8ff006ec30 0x7f8ff0130cc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 >> 172.21.15.119:0/3104382801 conn(0x7f8ff00a2ae0 msgr2=0x7f8ff0072450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 shutdown_connections 2024-09-15T21:21:41.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.857+0000 7f8ff8040640 1 -- 172.21.15.119:0/3104382801 wait complete. 2024-09-15T21:21:41.877 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm osd.12 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient: get_monmap_and_config 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient: build_initial_monmap 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient: monmap: 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:41.985975+0000 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:41.985975+0000 2024-09-15T21:21:41.987 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:41.988 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 Processor -- start 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 -- start start 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient: init 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f41e80a4d80 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e813cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): picked mon.noname-b con 0x7f41e80a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41e813d4d0 0x7f41e813f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): picked mon.noname-a con 0x7f41e813d4d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e80a26e0 con 0x7f41e80a4d80 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f41e80a6330 con 0x7f41e813d4d0 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f41e80a30a0 con 0x7f41e80a5870 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41f0708640 10 monclient(hunting): authenticate will time out at 1220.783435s 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41ee4a4640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f41e80a4d80 legacy=0x7f41e80a5160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41edca3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e813cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41eeca5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41e813d4d0 0x7f41e813f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:41.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41edca3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e813cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33814/0 (socket says 172.21.15.119:33814) 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.981+0000 7f41edca3640 1 -- 172.21.15.119:0/620079323 learned_addr learned my addr 172.21.15.119:0/620079323 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): get_auth_request con 0x7f41e813d4d0 auth_method 0 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): get_auth_request con 0x7f41e80a5870 auth_method 0 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient(hunting): handle_auth_done global_id 54276 payload 274 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 1 -- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 msgr2=0x7f41e813cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 1 --2- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e813cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 1 -- 172.21.15.119:0/620079323 >> v1:172.21.15.119:6789/0 conn(0x7f41e80a4d80 legacy=0x7f41e80a5160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient: _finish_hunting 0 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 1 monclient: found mon.noname-a 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:41.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:41.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41eeca5640 1 -- 172.21.15.119:0/620079323 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e80a6a30 con 0x7f41e813d4d0 2024-09-15T21:21:41.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.985+0000 7f41edca3640 1 --2- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e813cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:41.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41eeca5640 1 --2- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e813d4d0 0x7f41e813f8c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009dc0 tx=0x7f41e001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4545cf53bf46f64a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:41.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 1 -- 172.21.15.119:0/620079323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00b2070 con 0x7f41e813d4d0 2024-09-15T21:21:41.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:41.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: dump: 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.997 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: _finish_auth 0 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: _check_auth_tickets 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 1 -- 172.21.15.119:0/620079323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e00a57f0 con 0x7f41e813d4d0 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 1 -- 172.21.15.119:0/620079323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00a5bc0 con 0x7f41e813d4d0 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 5 monclient: authenticate success, global_id 54276 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 4 set_mon_vals no callback set 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:41.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41ed4a2640 10 monclient: dump: 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: shutdown 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:41.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e813d4d0 msgr2=0x7f41e813f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 --2- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e813d4d0 0x7f41e813f8c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009dc0 tx=0x7f41e001c8a0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- 172.21.15.119:0/620079323 shutdown_connections 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 --2- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e813d4d0 0x7f41e813f8c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 --2- 172.21.15.119:0/620079323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e813cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- 172.21.15.119:0/620079323 >> 172.21.15.119:0/620079323 conn(0x7f41e80a7a00 msgr2=0x7f41e806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- 172.21.15.119:0/620079323 shutdown_connections 2024-09-15T21:21:42.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- 172.21.15.119:0/620079323 wait complete. 2024-09-15T21:21:42.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: build_initial_monmap 2024-09-15T21:21:42.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: monmap: 2024-09-15T21:21:42.005 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:42.005 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 Processor -- start 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- start start 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: init 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:42.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 0x7f41e81cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): picked mon.noname-a con 0x7f41e80a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e8137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): picked mon.noname-c con 0x7f41e80a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e813d4d0 0x7f41e8138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): picked mon.noname-b con 0x7f41e813d4d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f41e80a5f60 con 0x7f41e80a4d80 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:42.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41e806d660 con 0x7f41e813d4d0 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f41e80a60e0 con 0x7f41e80a5870 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41f0708640 10 monclient(hunting): authenticate will time out at 1220.791892s 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41eeca5640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e813d4d0 msgr2=0x7f41e8138430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.989+0000 7f41eeca5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e813d4d0 0x7f41e8138430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e8137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 0x7f41e81cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e8137ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33818/0 (socket says 172.21.15.119:33818) 2024-09-15T21:21:42.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 0x7f41e81cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:35966/0 (socket says 172.21.15.119:35966) 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 1 -- 172.21.15.119:0/2494288023 learned_addr learned my addr 172.21.15.119:0/2494288023 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): get_auth_request con 0x7f41e80a5870 auth_method 0 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): get_auth_request con 0x7f41e80a4d80 auth_method 0 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:42.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient(hunting): handle_auth_done global_id 54279 payload 824 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 -- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 msgr2=0x7f41e8137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e8137ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 -- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e813d4d0 msgr2=0x7f41e8138430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e813d4d0 0x7f41e8138430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient: _finish_hunting 0 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 monclient: found mon.noname-a 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 -- 172.21.15.119:0/2494288023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e80a1570 con 0x7f41e80a4d80 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e8137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:42.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ee4a4640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 0x7f41e81cf910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41e400b400 tx=0x7f41e400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4016070 con 0x7f41e80a4d80 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: dump: 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:42.011 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 5 monclient: authenticate success, global_id 54279 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 monclient: _renew_subs 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e8138970 con 0x7f41e80a4d80 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4003df0 con 0x7f41e80a4d80 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41ecca1640 4 set_mon_vals no callback set 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 monclient: _renew_subs 2024-09-15T21:21:42.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e8133990 con 0x7f41e80a4d80 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4005270 con 0x7f41e80a4d80 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 10 monclient: dump: 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:42.013 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e80a5160 con 0x7f41e80a4d80 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f41e4019070 con 0x7f41e80a4d80 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41cc03b970 0x7f41cc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6599+0+0 (secure 0 0 0) 0x7f41e404f6a0 con 0x7f41e80a4d80 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41cc03b970 0x7f41cc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 10 monclient: get_auth_request con 0x7f41cc03b970 auth_method 0 2024-09-15T21:21:42.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:41.993+0000 7f41edca3640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41cc03b970 0x7f41cc03de30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41e8132410 tx=0x7f41d8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:42.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.005+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f41e404ca90 con 0x7f41e80a4d80 2024-09-15T21:21:42.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.005+0000 7f41deffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:42.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.005+0000 7f41deffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.189+0000 7f41f0708640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T21:21:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.189+0000 7f41f0708640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T21:21:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.189+0000 7f41f0708640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.189+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) -- 0x7f41e81334d0 con 0x7f41e80a4d80 2024-09-15T21:21:42.195 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:42.189+0000 7f1c5189c640 -1 mon.a@0(leader).osd e115 definitely_dead 0 2024-09-15T21:21:42.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.737+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 removed osd.12 v116) ==== 87+0+0 (secure 0 0 0) 0x7f41e404ce10 con 0x7f41e80a4d80 2024-09-15T21:21:42.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.737+0000 7f41deffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T21:21:42.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.737+0000 7f41deffd640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-09-15T21:21:42.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.849+0000 7f41f0708640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T21:21:42.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.849+0000 7f41f0708640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T21:21:42.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.849+0000 7f41f0708640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.849+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) -- 0x7f41e80008e0 con 0x7f41e80a4d80 2024-09-15T21:21:42.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:42.853+0000 7f1c5189c640 -1 mon.a@0(leader).osd e116 definitely_dead 0 2024-09-15T21:21:42.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41deffd640 1 -- 172.21.15.119:0/2494288023 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 osd.12 does not exist. v116) ==== 96+0+0 (secure 0 0 0) 0x7f41e40114a0 con 0x7f41e80a4d80 2024-09-15T21:21:42.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41deffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T21:21:42.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41deffd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-09-15T21:21:42.859 INFO:tasks.workunit.client.0.smithi119.stderr:osd.12 does not exist. 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41cc03b970 msgr2=0x7f41cc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41cc03b970 0x7f41cc03de30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41e8132410 tx=0x7f41d8009240 comp rx=0 tx=0).stop 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 10 monclient: shutdown 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 msgr2=0x7f41e81cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 0x7f41e81cf910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41e400b400 tx=0x7f41e400b8c0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 shutdown_connections 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f41cc03b970 0x7f41cc03de30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e813d4d0 0x7f41e8138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f41e80a5870 0x7f41e8137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 --2- 172.21.15.119:0/2494288023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41e80a4d80 0x7f41e81cf910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 >> 172.21.15.119:0/2494288023 conn(0x7f41e80a7a00 msgr2=0x7f41e813e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:42.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 shutdown_connections 2024-09-15T21:21:42.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.853+0000 7f41f0708640 1 -- 172.21.15.119:0/2494288023 wait complete. 2024-09-15T21:21:42.876 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm osd.14 2024-09-15T21:21:42.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient: get_monmap_and_config 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient: build_initial_monmap 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient: monmap: 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:42.990503+0000 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:42.990503+0000 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:42.993 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:42.993 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:42.993 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:42.993 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 Processor -- start 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 -- start start 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient: init 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b0809d4e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809f960 0x7f7b08139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): picked mon.noname-b con 0x7f7b0809f960 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b0809dfd0 0x7f7b0813c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): picked mon.noname-a con 0x7f7b0809dfd0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0809ef80 con 0x7f7b0809d4e0 2024-09-15T21:21:42.995 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0806da70 con 0x7f7b0809dfd0 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b080746f0 con 0x7f7b0809f960 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0f72d640 10 monclient(hunting): authenticate will time out at 1221.787418s 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0ccc8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809f960 0x7f7b08139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b0809dfd0 0x7f7b0813c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:42.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b0809dfd0 0x7f7b0813c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:51260/0 (socket says 172.21.15.119:51260) 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1429301516 learned_addr learned my addr 172.21.15.119:0/1429301516 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0ccc8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809f960 0x7f7b08139c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32858/0 (socket says 172.21.15.119:32858) 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 10 monclient(hunting): get_auth_request con 0x7f7b0809dfd0 auth_method 0 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0dcca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0ccc8640 10 monclient(hunting): get_auth_request con 0x7f7b0809f960 auth_method 0 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0ccc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0ccc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.985+0000 7f7b0ccc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:42.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0ccc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0ccc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0ccc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_done global_id 54282 payload 274 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809f960 msgr2=0x7f7b08139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 1 --2- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809f960 0x7f7b08139c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1429301516 >> v1:172.21.15.17:6789/0 conn(0x7f7b0809d4e0 legacy=0x7f7b0809d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 10 monclient: _finish_hunting 0 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 1 monclient: found mon.noname-a 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:42.998 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1429301516 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0809e6e0 con 0x7f7b0809dfd0 2024-09-15T21:21:43.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0dcca640 1 --2- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809dfd0 0x7f7b0813c150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009e50 tx=0x7f7b0001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2be410f264cd395 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:43.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 1 -- 172.21.15.119:0/1429301516 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000a57a0 con 0x7f7b0809dfd0 2024-09-15T21:21:43.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: dump: 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:43.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: _finish_auth 0 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0f72d640 5 monclient: authenticate success, global_id 54282 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0f72d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 1 -- 172.21.15.119:0/1429301516 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b000a5940 con 0x7f7b0809dfd0 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0f72d640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 1 -- 172.21.15.119:0/1429301516 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000a5d10 con 0x7f7b0809dfd0 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7b0f72d640 4 set_mon_vals no callback set 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.989+0000 7f7afffff640 10 monclient: dump: 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:43.005 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: shutdown 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809dfd0 msgr2=0x7f7b0813c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809dfd0 0x7f7b0813c150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009e50 tx=0x7f7b0001c8c0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.006 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1429301516 shutdown_connections 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809dfd0 0x7f7b0813c150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1429301516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809f960 0x7f7b08139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1429301516 >> 172.21.15.119:0/1429301516 conn(0x7f7b0806cdc0 msgr2=0x7f7b0806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1429301516 shutdown_connections 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1429301516 wait complete. 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: build_initial_monmap 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: monmap: 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:43.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 Processor -- start 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- start start 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: init 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:43.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809d4e0 0x7f7b081cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): picked mon.noname-c con 0x7f7b0809d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b0809dfd0 0x7f7b08134b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): picked mon.noname-b con 0x7f7b0809dfd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809f960 0x7f7b0811e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): picked mon.noname-a con 0x7f7b0809f960 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0813eeb0 con 0x7f7b0809f960 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b080a2640 con 0x7f7b0809dfd0 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0813f030 con 0x7f7b0809d4e0 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:43.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient(hunting): authenticate will time out at 1221.793994s 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0ccc8640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b0809dfd0 msgr2=0x7f7b08134b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0ccc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b0809dfd0 0x7f7b08134b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809d4e0 0x7f7b081cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809f960 0x7f7b0811e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809d4e0 0x7f7b081cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32866/0 (socket says 172.21.15.119:32866) 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): get_auth_request con 0x7f7b0809f960 auth_method 0 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 1 -- 172.21.15.119:0/1574111589 learned_addr learned my addr 172.21.15.119:0/1574111589 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): get_auth_request con 0x7f7b0809d4e0 auth_method 0 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:43.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0d4c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient(hunting): handle_auth_done global_id 54288 payload 824 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809d4e0 msgr2=0x7f7b081cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809d4e0 0x7f7b081cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b0809dfd0 msgr2=0x7f7b08134b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b0809dfd0 0x7f7b08134b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient: _finish_hunting 0 2024-09-15T21:21:43.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 monclient: found mon.noname-a 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 -- 172.21.15.119:0/1574111589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b081221f0 con 0x7f7b0809f960 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0dcca640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809f960 0x7f7b0811e6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b000ab000 tx=0x7f7b00019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000bc020 con 0x7f7b0809f960 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: dump: 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.012 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 5 monclient: authenticate success, global_id 54288 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: _renew_subs 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b000b8670 con 0x7f7b0809f960 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b081d03a0 con 0x7f7b0809f960 2024-09-15T21:21:43.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000b8a40 con 0x7f7b0809f960 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7afdffb640 10 monclient: dump: 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:43.014 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7aff7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: _renew_subs 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.993+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0809d1d0 con 0x7f7b0809f960 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0f72d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0f72d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0f72d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0809d8c0 con 0x7f7b0809f960 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7b000bf020 con 0x7f7b0809f960 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7afdffb640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7aec03b9a0 0x7f7aec03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6599+0+0 (secure 0 0 0) 0x7f7b000eb3b0 con 0x7f7b0809f960 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0d4c9640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7aec03b9a0 0x7f7aec03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0d4c9640 10 monclient: get_auth_request con 0x7f7aec03b9a0 auth_method 0 2024-09-15T21:21:43.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:42.997+0000 7f7b0d4c9640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7aec03b9a0 0x7f7aec03de60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b081207e0 tx=0x7f7af8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:43.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.009+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7b000c6030 con 0x7f7b0809f960 2024-09-15T21:21:43.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.009+0000 7f7afdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:43.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.009+0000 7f7afdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:43.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.181+0000 7f7b0f72d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T21:21:43.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.181+0000 7f7b0f72d640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T21:21:43.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.181+0000 7f7b0f72d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:43.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.181+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) -- 0x7f7b08121290 con 0x7f7b0809f960 2024-09-15T21:21:43.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:43.185+0000 7f1c5189c640 -1 mon.a@0(leader).osd e116 definitely_dead 0 2024-09-15T21:21:43.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.753+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 removed osd.14 v117) ==== 87+0+0 (secure 0 0 0) 0x7f7b00016600 con 0x7f7b0809f960 2024-09-15T21:21:43.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.753+0000 7f7afdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T21:21:43.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.753+0000 7f7afdffb640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-09-15T21:21:43.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7b0f72d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T21:21:43.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7b0f72d640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T21:21:43.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7b0f72d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:43.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) -- 0x7f7b080008e0 con 0x7f7b0809f960 2024-09-15T21:21:43.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:21:43.861+0000 7f1c5189c640 -1 mon.a@0(leader).osd e117 definitely_dead 0 2024-09-15T21:21:43.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7afdffb640 1 -- 172.21.15.119:0/1574111589 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 osd.14 does not exist. v117) ==== 96+0+0 (secure 0 0 0) 0x7f7b000b5db0 con 0x7f7b0809f960 2024-09-15T21:21:43.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7afdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T21:21:43.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.857+0000 7f7afdffb640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-09-15T21:21:43.865 INFO:tasks.workunit.client.0.smithi119.stderr:osd.14 does not exist. 2024-09-15T21:21:43.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7aec03b9a0 msgr2=0x7f7aec03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7aec03b9a0 0x7f7aec03de60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b081207e0 tx=0x7f7af8009040 comp rx=0 tx=0).stop 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 10 monclient: shutdown 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809f960 msgr2=0x7f7b0811e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809f960 0x7f7b0811e6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b000ab000 tx=0x7f7b00019660 comp rx=0 tx=0).stop 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 shutdown_connections 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7aec03b9a0 0x7f7aec03de60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0809f960 0x7f7b0811e6b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b0809dfd0 0x7f7b08134b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 --2- 172.21.15.119:0/1574111589 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b0809d4e0 0x7f7b081cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 >> 172.21.15.119:0/1574111589 conn(0x7f7b0806cdc0 msgr2=0x7f7b080a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 shutdown_connections 2024-09-15T21:21:43.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.861+0000 7f7b0f72d640 1 -- 172.21.15.119:0/1574111589 wait complete. 2024-09-15T21:21:43.882 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd set-require-min-compat-client luminous 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.993+0000 7f7d96bcb640 10 monclient: get_monmap_and_config 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.993+0000 7f7d96bcb640 10 monclient: build_initial_monmap 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.993+0000 7f7d96bcb640 10 monclient: monmap: 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:44.000401+0000 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:44.000401+0000 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:44.007 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 Processor -- start 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 -- start start 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient: init 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d9009d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): picked mon.noname-b con 0x7f7d9009d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:44.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d9009f960 0x7f7d9009de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): picked mon.noname-a con 0x7f7d9009f960 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7d901347b0 0x7f7d90134be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): picked mon.noname-c con 0x7f7d901347b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7d9006da70 con 0x7f7d9009f960 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7d9009e340 con 0x7f7d901347b0 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9009ef80 con 0x7f7d9009d4e0 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d96bcb640 10 monclient(hunting): authenticate will time out at 1222.797151s 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d95168640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f7d901347b0 msgr2=0x7f7d90134be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:44.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d95168640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7d901347b0 0x7f7d90134be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d9009d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d9009f960 0x7f7d9009de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d9009d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32874/0 (socket says 172.21.15.119:32874) 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 -- 172.21.15.119:0/3518819689 learned_addr learned my addr 172.21.15.119:0/3518819689 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): get_auth_request con 0x7f7d9009d4e0 auth_method 0 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): get_auth_request con 0x7f7d9009f960 auth_method 0 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:44.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient(hunting): handle_auth_done global_id 44411 payload 274 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 -- 172.21.15.119:0/3518819689 >> v2:172.21.15.119:3300/0 conn(0x7f7d901347b0 msgr2=0x7f7d90134be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 --2- 172.21.15.119:0/3518819689 >> v2:172.21.15.119:3300/0 conn(0x7f7d901347b0 0x7f7d90134be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 -- 172.21.15.119:0/3518819689 >> v2:172.21.15.17:3300/0 conn(0x7f7d9009f960 msgr2=0x7f7d9009de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 --2- 172.21.15.119:0/3518819689 >> v2:172.21.15.17:3300/0 conn(0x7f7d9009f960 0x7f7d9009de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient: _finish_hunting 0 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 monclient: found mon.noname-b 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:44.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:43.997+0000 7f7d94967640 1 -- 172.21.15.119:0/3518819689 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d900746f0 con 0x7f7d9009d4e0 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 1 --2- 172.21.15.119:0/3518819689 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d9009d8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d90076a60 tx=0x7f7d8401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df11066e3139dcbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 1 -- 172.21.15.119:0/3518819689 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840b2070 con 0x7f7d9009d4e0 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: dump: 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:44.012 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 5 monclient: authenticate success, global_id 44411 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 1 -- 172.21.15.119:0/3518819689 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840ad640 con 0x7f7d9009d4e0 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 4 set_mon_vals no callback set 2024-09-15T21:21:44.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 1 -- 172.21.15.119:0/3518819689 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840ada10 con 0x7f7d9009d4e0 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8f7fe640 10 monclient: dump: 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:44.014 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: shutdown 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- 172.21.15.119:0/3518819689 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 msgr2=0x7f7d9009d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/3518819689 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d9009d8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d90076a60 tx=0x7f7d8401c3d0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- 172.21.15.119:0/3518819689 shutdown_connections 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/3518819689 >> v2:172.21.15.119:3300/0 conn(0x7f7d901347b0 0x7f7d90134be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/3518819689 >> v2:172.21.15.17:3300/0 conn(0x7f7d9009f960 0x7f7d9009de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/3518819689 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d9009d8c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- 172.21.15.119:0/3518819689 >> 172.21.15.119:0/3518819689 conn(0x7f7d9006cdc0 msgr2=0x7f7d9006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- 172.21.15.119:0/3518819689 shutdown_connections 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- 172.21.15.119:0/3518819689 wait complete. 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: build_initial_monmap 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: monmap: 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:44.020 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:44.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:44.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:44.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 Processor -- start 2024-09-15T21:21:44.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- start start 2024-09-15T21:21:44.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: init 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d90139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): picked mon.noname-c con 0x7f7d9009d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d9009f960 0x7f7d90139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): picked mon.noname-b con 0x7f7d9009f960 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d901347b0 0x7f7d9013a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): picked mon.noname-a con 0x7f7d901347b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006de30 con 0x7f7d901347b0 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006dcb0 con 0x7f7d9009f960 2024-09-15T21:21:44.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7d900a25e0 con 0x7f7d9009d4e0 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d96bcb640 10 monclient(hunting): authenticate will time out at 1222.803331s 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8ffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d9009f960 msgr2=0x7f7d90139b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d8ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d9009f960 0x7f7d90139b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d90139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d901347b0 0x7f7d9013a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d90139640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32884/0 (socket says 172.21.15.119:32884) 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 1 -- 172.21.15.119:0/1639359951 learned_addr learned my addr 172.21.15.119:0/1639359951 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): get_auth_request con 0x7f7d9009d4e0 auth_method 0 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): get_auth_request con 0x7f7d901347b0 auth_method 0 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d94967640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.001+0000 7f7d95168640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 10 monclient(hunting): handle_auth_done global_id 44414 payload 824 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 -- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d9009f960 msgr2=0x7f7d90139b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d9009f960 0x7f7d90139b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 -- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d901347b0 msgr2=0x7f7d9013a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d901347b0 0x7f7d9013a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 10 monclient: _finish_hunting 0 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 monclient: found mon.noname-c 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 -- 172.21.15.119:0/1639359951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d90070b20 con 0x7f7d9009d4e0 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d94967640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d90139640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7d840b0040 tx=0x7f7d84019170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840bd070 con 0x7f7d9009d4e0 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: dump: 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:44.025 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 5 monclient: authenticate success, global_id 44414 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 monclient: _renew_subs 2024-09-15T21:21:44.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9006ed00 con 0x7f7d9009d4e0 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840b6550 con 0x7f7d9009d4e0 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840b2070 con 0x7f7d9009d4e0 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 10 monclient: dump: 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:44.027 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8effd640 4 set_mon_vals no callback set 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 monclient: _renew_subs 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d900a49f0 con 0x7f7d9009d4e0 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d900a4550 con 0x7f7d9009d4e0 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7d840b6940 con 0x7f7d9009d4e0 2024-09-15T21:21:44.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d7403b970 0x7f7d7403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 6599+0+0 (secure 0 0 0) 0x7f7d840ea360 con 0x7f7d9009d4e0 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8ffff640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d7403b970 0x7f7d7403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8ffff640 10 monclient: get_auth_request con 0x7f7d7403b970 auth_method 0 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.005+0000 7f7d8ffff640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d7403b970 0x7f7d7403de30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d90070dc0 tx=0x7f7d7c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.017+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7d840c6020 con 0x7f7d9009d4e0 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.017+0000 7f7d8d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:44.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.017+0000 7f7d8d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.193+0000 7f7d96bcb640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T21:21:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.193+0000 7f7d96bcb640 10 monclient: _send_command 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T21:21:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.193+0000 7f7d96bcb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.193+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) -- 0x7f7d90134be0 con 0x7f7d9009d4e0 2024-09-15T21:21:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.757+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v118) ==== 145+0+0 (secure 0 0 0) 0x7f7d84016110 con 0x7f7d9009d4e0 2024-09-15T21:21:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.757+0000 7f7d8d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T21:21:44.763 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.757+0000 7f7d8d7fa640 10 monclient: _finish_command 2 = system:0 set require_min_compat_client to luminous 2024-09-15T21:21:44.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.865+0000 7f7d96bcb640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T21:21:44.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.865+0000 7f7d96bcb640 10 monclient: _send_command 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T21:21:44.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.865+0000 7f7d96bcb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:44.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:44.865+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) -- 0x7f7d900008e0 con 0x7f7d9009d4e0 2024-09-15T21:21:45.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.773+0000 7f7d8d7fa640 1 -- 172.21.15.119:0/1639359951 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v119) ==== 145+0+0 (secure 0 0 0) 0x7f7d840ea0d0 con 0x7f7d9009d4e0 2024-09-15T21:21:45.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.773+0000 7f7d8d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T21:21:45.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.773+0000 7f7d8d7fa640 10 monclient: _finish_command 3 = system:0 set require_min_compat_client to luminous 2024-09-15T21:21:45.780 INFO:tasks.workunit.client.0.smithi119.stderr:set require_min_compat_client to luminous 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d7403b970 msgr2=0x7f7d7403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d7403b970 0x7f7d7403de30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d90070dc0 tx=0x7f7d7c009190 comp rx=0 tx=0).stop 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 10 monclient: shutdown 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 msgr2=0x7f7d90139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d90139640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7d840b0040 tx=0x7f7d84019170 comp rx=0 tx=0).stop 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 shutdown_connections 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d7403b970 0x7f7d7403de30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d901347b0 0x7f7d9013a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d9009f960 0x7f7d90139b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 --2- 172.21.15.119:0/1639359951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d9009d4e0 0x7f7d90139640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 >> 172.21.15.119:0/1639359951 conn(0x7f7d9006cdc0 msgr2=0x7f7d90133bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:45.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 shutdown_connections 2024-09-15T21:21:45.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.777+0000 7f7d96bcb640 1 -- 172.21.15.119:0/1639359951 wait complete. 2024-09-15T21:21:45.803 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set dump 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.913+0000 7ff51b3f8640 10 monclient: get_monmap_and_config 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.913+0000 7ff51b3f8640 10 monclient: build_initial_monmap 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.913+0000 7ff51b3f8640 10 monclient: monmap: 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:45.919944+0000 2024-09-15T21:21:45.921 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:45.919944+0000 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:45.922 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:45.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 Processor -- start 2024-09-15T21:21:45.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 -- start start 2024-09-15T21:21:45.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient: init 2024-09-15T21:21:45.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5140a47c0 0x7ff5140a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): picked mon.noname-c con 0x7ff5140a47c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 0x7ff5140a1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): picked mon.noname-b con 0x7ff51409e6d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff5140a2be0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff514138190 con 0x7ff5140a2be0 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff5140a4ba0 con 0x7ff5140a47c0 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff51409f0d0 con 0x7ff51409e6d0 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:45.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff51b3f8640 10 monclient(hunting): authenticate will time out at 1224.717079s 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff519995640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7ff5140a2be0 legacy=0x7ff5140a3050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff519194640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7ff5140a47c0 msgr2=0x7ff5140a0bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff519194640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5140a47c0 0x7ff5140a0bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 0x7ff5140a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 0x7ff5140a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32894/0 (socket says 172.21.15.119:32894) 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 -- 172.21.15.119:0/103720264 learned_addr learned my addr 172.21.15.119:0/103720264 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): get_auth_request con 0x7ff51409e6d0 auth_method 0 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:45.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient(hunting): handle_auth_done global_id 44417 payload 274 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 -- 172.21.15.119:0/103720264 >> v2:172.21.15.119:3300/0 conn(0x7ff5140a47c0 msgr2=0x7ff5140a0bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 --2- 172.21.15.119:0/103720264 >> v2:172.21.15.119:3300/0 conn(0x7ff5140a47c0 0x7ff5140a0bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 -- 172.21.15.119:0/103720264 >> v1:172.21.15.119:6789/0 conn(0x7ff5140a2be0 legacy=0x7ff5140a3050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient: _finish_hunting 0 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 monclient: found mon.noname-b 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:45.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.917+0000 7ff518993640 1 -- 172.21.15.119:0/103720264 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5140a1640 con 0x7ff51409e6d0 2024-09-15T21:21:45.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 1 --2- 172.21.15.119:0/103720264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 0x7ff5140a1100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff504009870 tx=0x7ff50401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd9e7f342b22b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 1 -- 172.21.15.119:0/103720264 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040ab620 con 0x7ff51409e6d0 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: dump: 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.934 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: _finish_auth 0 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 5 monclient: authenticate success, global_id 44417 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 1 -- 172.21.15.119:0/103720264 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5040b0070 con 0x7ff51409e6d0 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 1 -- 172.21.15.119:0/103720264 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040ab9e0 con 0x7ff51409e6d0 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:45.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff50bfff640 10 monclient: dump: 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:45.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 4 set_mon_vals no callback set 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: shutdown 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- 172.21.15.119:0/103720264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 msgr2=0x7ff5140a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/103720264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 0x7ff5140a1100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff504009870 tx=0x7ff50401c4a0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- 172.21.15.119:0/103720264 shutdown_connections 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/103720264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51409e6d0 0x7ff5140a1100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/103720264 >> v2:172.21.15.119:3300/0 conn(0x7ff5140a47c0 0x7ff5140a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- 172.21.15.119:0/103720264 >> 172.21.15.119:0/103720264 conn(0x7ff5140a3cd0 msgr2=0x7ff5140a40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- 172.21.15.119:0/103720264 shutdown_connections 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- 172.21.15.119:0/103720264 wait complete. 2024-09-15T21:21:45.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: build_initial_monmap 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: monmap: 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 Processor -- start 2024-09-15T21:21:45.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- start start 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: init 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff51409e6d0 0x7ff5141cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): picked mon.noname-b con 0x7ff51409e6d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5140a2be0 0x7ff514134770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): picked mon.noname-c con 0x7ff5140a2be0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5140a47c0 0x7ff514134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): picked mon.noname-a con 0x7ff5140a47c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff514142790 con 0x7ff5140a47c0 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:45.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff514142610 con 0x7ff51409e6d0 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff514142910 con 0x7ff5140a2be0 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff51b3f8640 10 monclient(hunting): authenticate will time out at 1224.722972s 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519194640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff51409e6d0 msgr2=0x7ff5141cfb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519194640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff51409e6d0 0x7ff5141cfb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5140a2be0 0x7ff514134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5140a47c0 0x7ff514134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5140a2be0 0x7ff514134770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32910/0 (socket says 172.21.15.119:32910) 2024-09-15T21:21:45.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 1 -- 172.21.15.119:0/3079399016 learned_addr learned my addr 172.21.15.119:0/3079399016 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): get_auth_request con 0x7ff5140a2be0 auth_method 0 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): get_auth_request con 0x7ff5140a47c0 auth_method 0 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:45.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff518993640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient(hunting): handle_auth_done global_id 54297 payload 824 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 -- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5140a2be0 msgr2=0x7ff514134770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5140a2be0 0x7ff514134770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 -- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff51409e6d0 msgr2=0x7ff5141cfb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:45.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff51409e6d0 0x7ff5141cfb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient: _finish_hunting 0 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 monclient: found mon.noname-a 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.921+0000 7ff519995640 1 -- 172.21.15.119:0/3079399016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51409e3c0 con 0x7ff5140a47c0 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff519995640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5140a47c0 0x7ff514134cb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00bdd0 tx=0x7ff50c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c00cdf0 con 0x7ff5140a47c0 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: dump: 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:45.943 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 5 monclient: authenticate success, global_id 54297 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff50c010030 con 0x7ff5140a47c0 2024-09-15T21:21:45.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 monclient: _renew_subs 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff514130230 con 0x7ff5140a47c0 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c016670 con 0x7ff5140a47c0 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 10 monclient: dump: 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:45.945 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 monclient: _renew_subs 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff50b7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51412ff50 con 0x7ff5140a47c0 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:45.946 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5140a3050 con 0x7ff5140a47c0 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff50c006150 con 0x7ff5140a47c0 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4f803b970 0x7ff4f803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6599+0+0 (secure 0 0 0) 0x7ff50c050b20 con 0x7ff5140a47c0 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff519194640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4f803b970 0x7ff4f803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff519194640 10 monclient: get_auth_request con 0x7ff4f803b970 auth_method 0 2024-09-15T21:21:45.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.925+0000 7ff519194640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4f803b970 0x7ff4f803de30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:45.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.937+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff51409e3c0 con 0x7ff5140a47c0 2024-09-15T21:21:45.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.937+0000 7ff509ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:45.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:45.937+0000 7ff509ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:46.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff51b3f8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:46.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff51b3f8640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:46.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff51b3f8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7ff51412fa70 con 0x7ff5140a47c0 2024-09-15T21:21:46.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v119) ==== 73+0+3 (secure 0 0 0) 0x7ff50c013cf0 con 0x7ff5140a47c0 2024-09-15T21:21:46.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff509ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:46.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.113+0000 7ff509ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:46.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff51b3f8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:46.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff51b3f8640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:46.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff51b3f8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7ff51412ea30 con 0x7ff5140a47c0 2024-09-15T21:21:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff509ffb640 1 -- 172.21.15.119:0/3079399016 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v119) ==== 73+0+3 (secure 0 0 0) 0x7ff50c04df10 con 0x7ff5140a47c0 2024-09-15T21:21:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff509ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.213+0000 7ff509ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:46.220 INFO:tasks.workunit.client.0.smithi119.stdout:{} 2024-09-15T21:21:46.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4f803b970 msgr2=0x7ff4f803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4f803b970 0x7ff4f803de30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510006d40 comp rx=0 tx=0).stop 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 10 monclient: shutdown 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5140a47c0 msgr2=0x7ff514134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5140a47c0 0x7ff514134cb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00bdd0 tx=0x7ff50c009ed0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 shutdown_connections 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4f803b970 0x7ff4f803de30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5140a47c0 0x7ff514134cb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5140a2be0 0x7ff514134770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 --2- 172.21.15.119:0/3079399016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff51409e6d0 0x7ff5141cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 >> 172.21.15.119:0/3079399016 conn(0x7ff5140a3cd0 msgr2=0x7ff5140a5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:46.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 shutdown_connections 2024-09-15T21:21:46.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.217+0000 7ff51b3f8640 1 -- 172.21.15.119:0/3079399016 wait complete. 2024-09-15T21:21:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient: get_monmap_and_config 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient: build_initial_monmap 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient: monmap: 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:46.346485+0000 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:46.346485+0000 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:46.349 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 Processor -- start 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 -- start start 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient: init 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a6809f640 0x7f2a6809d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): picked mon.noname-b con 0x7f2a6809f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2a6809fa20 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a68139c00 0x7f2a6813bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): picked mon.noname-a con 0x7f2a68139c00 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a6809e880 con 0x7f2a6809fa20 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2a6809eb60 con 0x7f2a68139c00 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6809ee40 con 0x7f2a6809f640 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6ffad640 10 monclient(hunting): authenticate will time out at 1225.143413s 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6d548640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f2a6809fa20 legacy=0x7f2a6809fe80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a6809f640 0x7f2a6809d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6e54a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a68139c00 0x7f2a6813bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a6809f640 0x7f2a6809d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32922/0 (socket says 172.21.15.119:32922) 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 1 -- 172.21.15.119:0/2136868298 learned_addr learned my addr 172.21.15.119:0/2136868298 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 10 monclient(hunting): get_auth_request con 0x7f2a6809f640 auth_method 0 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6dd49640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6e54a640 10 monclient(hunting): get_auth_request con 0x7f2a68139c00 auth_method 0 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6e54a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6e54a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.341+0000 7f2a6e54a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_done global_id 54300 payload 274 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 1 -- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a6809f640 msgr2=0x7f2a6809d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 1 --2- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a6809f640 0x7f2a6809d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 1 -- 172.21.15.119:0/2136868298 >> v1:172.21.15.119:6789/0 conn(0x7f2a6809fa20 legacy=0x7f2a6809fe80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 1 monclient: found mon.noname-a 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 1 -- 172.21.15.119:0/2136868298 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a680a2840 con 0x7f2a68139c00 2024-09-15T21:21:46.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6e54a640 1 --2- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a68139c00 0x7f2a6813bff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a600097c0 tx=0x7f2a6001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262acb9aee91e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:46.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 1 -- 172.21.15.119:0/2136868298 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600b0070 con 0x7f2a68139c00 2024-09-15T21:21:46.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: dump: 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: _finish_auth 0 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: _check_auth_tickets 2024-09-15T21:21:46.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 1 -- 172.21.15.119:0/2136868298 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a600ab400 con 0x7f2a68139c00 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 5 monclient: authenticate success, global_id 54300 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 1 -- 172.21.15.119:0/2136868298 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600ab7d0 con 0x7f2a68139c00 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 4 set_mon_vals no callback set 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6cd47640 10 monclient: dump: 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.362 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 10 monclient: shutdown 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 -- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a68139c00 msgr2=0x7f2a6813bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a68139c00 0x7f2a6813bff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a600097c0 tx=0x7f2a6001c3f0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 -- 172.21.15.119:0/2136868298 shutdown_connections 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a68139c00 0x7f2a6813bff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/2136868298 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a6809f640 0x7f2a6809d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 -- 172.21.15.119:0/2136868298 >> 172.21.15.119:0/2136868298 conn(0x7f2a680a2ae0 msgr2=0x7f2a680a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 -- 172.21.15.119:0/2136868298 shutdown_connections 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.345+0000 7f2a6ffad640 1 -- 172.21.15.119:0/2136868298 wait complete. 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: build_initial_monmap 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: monmap: 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:46.364 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 Processor -- start 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- start start 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: init 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a6809f640 0x7f2a68131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): picked mon.noname-a con 0x7f2a6809f640 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6809fa20 0x7f2a68132220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): picked mon.noname-b con 0x7f2a6809fa20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:46.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a68139c00 0x7f2a68132760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): picked mon.noname-c con 0x7f2a68139c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680a13f0 con 0x7f2a6809f640 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680a1270 con 0x7f2a6809fa20 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2a680a1570 con 0x7f2a68139c00 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient(hunting): authenticate will time out at 1225.149089s 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6d548640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6809fa20 msgr2=0x7f2a68132220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6d548640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6809fa20 0x7f2a68132220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:46.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a68139c00 0x7f2a68132760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a6809f640 0x7f2a68131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a68139c00 0x7f2a68132760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32926/0 (socket says 172.21.15.119:32926) 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 1 -- 172.21.15.119:0/3527243067 learned_addr learned my addr 172.21.15.119:0/3527243067 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): get_auth_request con 0x7f2a68139c00 auth_method 0 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): get_auth_request con 0x7f2a6809f640 auth_method 0 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6e54a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient(hunting): handle_auth_done global_id 54303 payload 824 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 -- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a68139c00 msgr2=0x7f2a68132760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a68139c00 0x7f2a68132760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 -- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6809fa20 msgr2=0x7f2a68132220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6809fa20 0x7f2a68132220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient: _finish_hunting 0 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 monclient: found mon.noname-a 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 -- 172.21.15.119:0/3527243067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68073a60 con 0x7f2a6809f640 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6dd49640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a6809f640 0x7f2a68131ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400efc0 tx=0x7f2a6400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400cda0 con 0x7f2a6809f640 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: dump: 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.370 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:46.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 5 monclient: authenticate success, global_id 54303 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _renew_subs 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a64019070 con 0x7f2a6809f640 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a680a5300 con 0x7f2a6809f640 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _renew_subs 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5ffff640 4 set_mon_vals no callback set 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6809d1d0 con 0x7f2a6809f640 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64014680 con 0x7f2a6809f640 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a5e7fc640 10 monclient: dump: 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.373 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.349+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6809d9a0 con 0x7f2a6809f640 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.353+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2a64012050 con 0x7f2a6809f640 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.353+0000 7f2a5e7fc640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a4c03b970 0x7f2a4c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.353+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6599+0+0 (secure 0 0 0) 0x7f2a640513a0 con 0x7f2a6809f640 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.353+0000 7f2a6d548640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a4c03b970 0x7f2a4c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.353+0000 7f2a6d548640 10 monclient: get_auth_request con 0x7f2a4c03b970 auth_method 0 2024-09-15T21:21:46.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.353+0000 7f2a6d548640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a4c03b970 0x7f2a4c03de30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a68073de0 tx=0x7f2a58006d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:46.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.365+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2a680a5300 con 0x7f2a6809f640 2024-09-15T21:21:46.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.365+0000 7f2a5e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:46.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.365+0000 7f2a5e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:46.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.525+0000 7f2a6ffad640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:46.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.529+0000 7f2a6ffad640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:46.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.529+0000 7f2a6ffad640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.529+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f2a6806c460 con 0x7f2a6809f640 2024-09-15T21:21:46.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.529+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v119) ==== 71+0+0 (secure 0 0 0) 0x7f2a64010040 con 0x7f2a6809f640 2024-09-15T21:21:46.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.529+0000 7f2a5e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:46.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.529+0000 7f2a5e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:46.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.625+0000 7f2a6ffad640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:46.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.625+0000 7f2a6ffad640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:46.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.625+0000 7f2a6ffad640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.625+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f2a68134a70 con 0x7f2a6809f640 2024-09-15T21:21:46.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/3527243067 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v119) ==== 71+0+0 (secure 0 0 0) 0x7f2a64051080 con 0x7f2a6809f640 2024-09-15T21:21:46.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a5e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:46.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a5e7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:46.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a4c03b970 msgr2=0x7f2a4c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a4c03b970 0x7f2a4c03de30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a68073de0 tx=0x7f2a58006d10 comp rx=0 tx=0).stop 2024-09-15T21:21:46.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 10 monclient: shutdown 2024-09-15T21:21:46.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:46.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a6809f640 msgr2=0x7f2a68131ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a6809f640 0x7f2a68131ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400efc0 tx=0x7f2a6400c490 comp rx=0 tx=0).stop 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 shutdown_connections 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2a4c03b970 0x7f2a4c03de30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2a68139c00 0x7f2a68132760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6809fa20 0x7f2a68132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 --2- 172.21.15.119:0/3527243067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a6809f640 0x7f2a68131ce0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 >> 172.21.15.119:0/3527243067 conn(0x7f2a680a2ae0 msgr2=0x7f2a680a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 shutdown_connections 2024-09-15T21:21:46.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.629+0000 7f2a6ffad640 1 -- 172.21.15.119:0/3527243067 wait complete. 2024-09-15T21:21:46.651 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T21:21:46.651 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:21:46.651 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T21:21:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient: get_monmap_and_config 2024-09-15T21:21:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient: build_initial_monmap 2024-09-15T21:21:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient: monmap: 2024-09-15T21:21:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:46.770211+0000 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:46.770211+0000 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:46.772 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 1 Processor -- start 2024-09-15T21:21:46.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 1 -- start start 2024-09-15T21:21:46.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient: init 2024-09-15T21:21:46.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:46.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f354409d550 0x7f354409d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient(hunting): picked mon.noname-c con 0x7f354409d550 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f354409fae0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f354409de70 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3544135640 con 0x7f354409fae0 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35440a3600 con 0x7f354409de70 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f354409f1b0 con 0x7f354409d550 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3549878640 10 monclient(hunting): authenticate will time out at 1225.566996s 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f35437fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f354409de70 legacy=0x7f3544134f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3542ffd640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f354409d550 msgr2=0x7f354409d930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f3542ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f354409d550 0x7f354409d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f35427fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f354409fae0 0x7f3544134820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48764/0 (socket says 172.21.15.119:48764) 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.765+0000 7f35427fc640 1 -- 172.21.15.119:0/784849857 learned_addr learned my addr 172.21.15.119:0/784849857 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069526753 0 0) 0x7f3544135640 con 0x7f354409fae0 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): my global_id is 54306 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f352c003530 con 0x7f354409fae0 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993127523 0 0) 0x7f352c003530 con 0x7f354409fae0 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 >> v2:172.21.15.119:3300/0 conn(0x7f354409d550 msgr2=0x7f354409d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 --2- 172.21.15.119:0/784849857 >> v2:172.21.15.119:3300/0 conn(0x7f354409d550 0x7f354409d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 >> v1:172.21.15.119:6789/0 conn(0x7f354409de70 legacy=0x7f3544134f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35440a1d60 con 0x7f354409fae0 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f3538001700 con 0x7f354409fae0 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3549878640 5 monclient: authenticate success, global_id 54306 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3549878640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:21:46.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: dump: 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:46.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 >> v1:172.21.15.17:6789/0 conn(0x7f354409fae0 legacy=0x7f3544134820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f352c0039c0 0x7f352c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): picked mon.a con 0x7f352c0039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f352c004810 0x7f352c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): picked mon.c con 0x7f352c004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c005380 0x7f352c00b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): picked mon.b con 0x7f352c005380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f354409f1b0 con 0x7f352c0039c0 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f352c00c2d0 con 0x7f352c005380 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f352c00c4d0 con 0x7f352c004810 2024-09-15T21:21:46.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3541ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3549878640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35437fe640 1 -- 172.21.15.119:0/784849857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c005380 msgr2=0x7f352c00b9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35437fe640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c005380 0x7f352c00b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f352c0039c0 0x7f352c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f352c004810 0x7f352c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): get_auth_request con 0x7f352c004810 auth_method 0 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): get_auth_request con 0x7f352c0039c0 auth_method 0 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f3542ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient(hunting): handle_auth_done global_id 54306 payload 293 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 -- 172.21.15.119:0/784849857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c005380 msgr2=0x7f352c00b9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c005380 0x7f352c00b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 -- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f352c0039c0 msgr2=0x7f352c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f352c0039c0 0x7f352c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient: _finish_hunting 0 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 monclient: found mon.c 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35437fe640 1 -- 172.21.15.119:0/784849857 reap_dead start 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 -- 172.21.15.119:0/784849857 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c003530 con 0x7f352c004810 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:46.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.769+0000 7f35427fc640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f352c004810 0x7f352c004c90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f353800c1c0 tx=0x7f353801cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac15a82cd35461b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:46.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538007d00 con 0x7f352c004810 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: dump: 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.791 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35380a9020 con 0x7f352c004810 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3549878640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3549878640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 1 -- 172.21.15.119:0/784849857 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538005ae0 con 0x7f352c004810 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3549878640 4 set_mon_vals no callback set 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.773+0000 7f3541ffb640 10 monclient: dump: 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.792 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: shutdown 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f352c004810 msgr2=0x7f352c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f352c004810 0x7f352c004c90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f353800c1c0 tx=0x7f353801cb60 comp rx=0 tx=0).stop 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/784849857 shutdown_connections 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 --2- 172.21.15.119:0/784849857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f352c004810 0x7f352c004c90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/784849857 >> 172.21.15.119:0/784849857 conn(0x7f35440a66e0 msgr2=0x7f35440a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/784849857 shutdown_connections 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/784849857 wait complete. 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: build_initial_monmap 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: monmap: 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:46.794 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 Processor -- start 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- start start 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: init 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c004810 0x7f354411fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): picked mon.noname-b con 0x7f352c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3544133620 0x7f3544120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): picked mon.noname-c con 0x7f3544133620 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3544120940 0x7f35441d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): picked mon.noname-a con 0x7f3544120940 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3542ffd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c004810 msgr2=0x7f354411fec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:46.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f354409ccd0 con 0x7f3544120940 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35440a2920 con 0x7f352c004810 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c004810 0x7f354411fec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f354409ce50 con 0x7f3544133620 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient(hunting): authenticate will time out at 1225.578060s 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3544133620 0x7f3544120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3544120940 0x7f35441d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3544133620 0x7f3544120400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32942/0 (socket says 172.21.15.119:32942) 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 1 -- 172.21.15.119:0/3518619194 learned_addr learned my addr 172.21.15.119:0/3518619194 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:46.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): get_auth_request con 0x7f3544133620 auth_method 0 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): get_auth_request con 0x7f3544120940 auth_method 0 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:46.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient(hunting): handle_auth_done global_id 54309 payload 824 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 -- 172.21.15.119:0/3518619194 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3544133620 msgr2=0x7f3544120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3544133620 0x7f3544120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 -- 172.21.15.119:0/3518619194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c004810 msgr2=0x7f354411fec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c004810 0x7f354411fec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 monclient: found mon.noname-a 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 -- 172.21.15.119:0/3518619194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3544122e70 con 0x7f3544120940 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35427fc640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3544133620 0x7f3544120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35437fe640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3544120940 0x7f35441d1a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f354406a360 tx=0x7f353c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:46.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c015670 con 0x7f3544120940 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: dump: 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.799 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: _finish_auth 0 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:46.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 5 monclient: authenticate success, global_id 54309 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: _renew_subs 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/3518619194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35441241b0 con 0x7f3544120940 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c010070 con 0x7f3544120940 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f35417fa640 4 set_mon_vals no callback set 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c01a020 con 0x7f3544120940 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f352bfff640 10 monclient: dump: 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.801 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: _renew_subs 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.777+0000 7f3549878640 1 -- 172.21.15.119:0/3518619194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3544123ce0 con 0x7f3544120940 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3549878640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3549878640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3549878640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3549878640 1 -- 172.21.15.119:0/3518619194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3544135150 con 0x7f3544120940 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f353c015a30 con 0x7f3544120940 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f352bfff640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f353003b910 0x7f353003ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6599+0+0 (secure 0 0 0) 0x7f353c050c20 con 0x7f3544120940 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3542ffd640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f353003b910 0x7f353003ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3542ffd640 10 monclient: get_auth_request con 0x7f353003b910 auth_method 0 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.781+0000 7f3542ffd640 1 --2- 172.21.15.119:0/3518619194 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f353003b910 0x7f353003ddd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3544052890 tx=0x7f35340090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.793+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f353c050e20 con 0x7f3544120940 2024-09-15T21:21:46.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.793+0000 7f352bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:46.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.793+0000 7f352bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:46.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f3549878640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:46.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f3549878640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:46.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f3549878640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:46.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f3549878640 1 -- 172.21.15.119:0/3518619194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7f3544072b90 con 0x7f3544120940 2024-09-15T21:21:46.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f352bfff640 1 -- 172.21.15.119:0/3518619194 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 pool 'fooset' not found v119) ==== 152+0+0 (secure 0 0 0) 0x7f353c024080 con 0x7f3544120940 2024-09-15T21:21:46.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f352bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:46.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:46.969+0000 7f352bfff640 10 monclient: _finish_command 2 = mon:2 pool 'fooset' not found 2024-09-15T21:21:46.977 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOENT: pool 'fooset' not found 2024-09-15T21:21:46.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:21:46.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create fooset 8 2024-09-15T21:21:47.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.093+0000 7f34bf416640 10 monclient: get_monmap_and_config 2024-09-15T21:21:47.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.093+0000 7f34bf416640 10 monclient: build_initial_monmap 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.093+0000 7f34bf416640 10 monclient: monmap: 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:47.100310+0000 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:47.100310+0000 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:47.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:47.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 Processor -- start 2024-09-15T21:21:47.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 -- start start 2024-09-15T21:21:47.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient: init 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f34b8137870 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 0x7f34b80a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): picked mon.noname-b con 0x7f34b8138360 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34b813cdc0 0x7f34b813f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): picked mon.noname-c con 0x7f34b813cdc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b80a1b60 con 0x7f34b8137870 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f34b80a27f0 con 0x7f34b813cdc0 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:47.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f34b80a1e40 con 0x7f34b8138360 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bf416640 10 monclient(hunting): authenticate will time out at 1225.896958s 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bd9b3640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f34b813cdc0 msgr2=0x7f34b813f1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bd9b3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34b813cdc0 0x7f34b813f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 0x7f34b80a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 0x7f34b80a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32944/0 (socket says 172.21.15.119:32944) 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 -- 172.21.15.119:0/881281691 learned_addr learned my addr 172.21.15.119:0/881281691 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): get_auth_request con 0x7f34b8138360 auth_method 0 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bd1b2640 1 --1- 172.21.15.119:0/881281691 >> v1:172.21.15.17:6789/0 conn(0x7f34b8137870 0x7f34b8137c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48770/0 (socket says 172.21.15.119:48770) 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:47.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_done global_id 44432 payload 274 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 -- 172.21.15.119:0/881281691 >> v2:172.21.15.119:3300/0 conn(0x7f34b813cdc0 msgr2=0x7f34b813f1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 --2- 172.21.15.119:0/881281691 >> v2:172.21.15.119:3300/0 conn(0x7f34b813cdc0 0x7f34b813f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34affff640 1 -- 172.21.15.119:0/881281691 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017132092 0 0) 0x7f34b4001120 con 0x7f34b8137870 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 -- 172.21.15.119:0/881281691 >> v1:172.21.15.17:6789/0 conn(0x7f34b8137870 legacy=0x7f34b8137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient: _finish_hunting 0 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 monclient: found mon.noname-b 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34bc9b1640 1 -- 172.21.15.119:0/881281691 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b80a2120 con 0x7f34b8138360 2024-09-15T21:21:47.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.097+0000 7f34affff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 1 --2- 172.21.15.119:0/881281691 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 0x7f34b80a0e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009870 tx=0x7f34a801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734c13d0c2236bed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 1 -- 172.21.15.119:0/881281691 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80b0070 con 0x7f34b8138360 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: dump: 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.113 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: _finish_auth 0 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 5 monclient: authenticate success, global_id 44432 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 1 -- 172.21.15.119:0/881281691 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a8004230 con 0x7f34b8138360 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 1 -- 172.21.15.119:0/881281691 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8003620 con 0x7f34b8138360 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:47.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 4 set_mon_vals no callback set 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34affff640 10 monclient: dump: 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:47.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: shutdown 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- 172.21.15.119:0/881281691 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 msgr2=0x7f34b80a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 --2- 172.21.15.119:0/881281691 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 0x7f34b80a0e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009870 tx=0x7f34a801c440 comp rx=0 tx=0).stop 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- 172.21.15.119:0/881281691 shutdown_connections 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 --2- 172.21.15.119:0/881281691 >> v2:172.21.15.119:3300/0 conn(0x7f34b813cdc0 0x7f34b813f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 --2- 172.21.15.119:0/881281691 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b8138360 0x7f34b80a0e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- 172.21.15.119:0/881281691 >> 172.21.15.119:0/881281691 conn(0x7f34b809be60 msgr2=0x7f34b809c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- 172.21.15.119:0/881281691 shutdown_connections 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- 172.21.15.119:0/881281691 wait complete. 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: build_initial_monmap 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: monmap: 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:47.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 Processor -- start 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- start start 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: init 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34b8137870 0x7f34b811fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:47.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): picked mon.noname-b con 0x7f34b8137870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b8138360 0x7f34b811daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): picked mon.noname-a con 0x7f34b8138360 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 0x7f34b8136ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): picked mon.noname-c con 0x7f34b813cdc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8142130 con 0x7f34b8138360 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8141fb0 con 0x7f34b8137870 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f34b81422b0 con 0x7f34b813cdc0 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bf416640 10 monclient(hunting): authenticate will time out at 1225.903335s 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd1b2640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34b8137870 msgr2=0x7f34b811fc50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:47.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd1b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34b8137870 0x7f34b811fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 0x7f34b8136ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b8138360 0x7f34b811daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 0x7f34b8136ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32954/0 (socket says 172.21.15.119:32954) 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 1 -- 172.21.15.119:0/1205657610 learned_addr learned my addr 172.21.15.119:0/1205657610 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): get_auth_request con 0x7f34b813cdc0 auth_method 0 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 10 monclient(hunting): get_auth_request con 0x7f34b8138360 auth_method 0 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:47.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bd9b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.101+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bc9b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 10 monclient(hunting): handle_auth_done global_id 44435 payload 824 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 -- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34b8137870 msgr2=0x7f34b811fc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34b8137870 0x7f34b811fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 -- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b8138360 msgr2=0x7f34b811daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b8138360 0x7f34b811daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 10 monclient: _finish_hunting 0 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 monclient: found mon.noname-c 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:47.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 -- 172.21.15.119:0/1205657610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b811f0f0 con 0x7f34b813cdc0 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bc9b1640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b8138360 0x7f34b811daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd9b3640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 0x7f34b8136ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f34b000be60 tx=0x7f34b000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b000cb50 con 0x7f34b813cdc0 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: dump: 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:47.121 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 5 monclient: authenticate success, global_id 44435 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 monclient: _renew_subs 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b80a4dd0 con 0x7f34b813cdc0 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b0010030 con 0x7f34b813cdc0 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0016430 con 0x7f34b813cdc0 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:47.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 10 monclient: dump: 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34af7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 monclient: _renew_subs 2024-09-15T21:21:47.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b8120bb0 con 0x7f34b813cdc0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b80a0e10 con 0x7f34b813cdc0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f34b0008070 con 0x7f34b813cdc0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f349c03b970 0x7f349c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(119..119 src has 1..119) ==== 6599+0+0 (secure 0 0 0) 0x7f34b0050560 con 0x7f34b813cdc0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd1b2640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f349c03b970 0x7f349c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd1b2640 10 monclient: get_auth_request con 0x7f349c03b970 auth_method 0 2024-09-15T21:21:47.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.105+0000 7f34bd1b2640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f349c03b970 0x7f349c03de30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f34b4001090 tx=0x7f34b4007820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:47.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.117+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f34b004d950 con 0x7f34b813cdc0 2024-09-15T21:21:47.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.117+0000 7f34adffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:47.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.117+0000 7f34adffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:47.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.289+0000 7f34bf416640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T21:21:47.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.289+0000 7f34bf416640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T21:21:47.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.289+0000 7f34bf416640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.289+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) -- 0x7f34b806a810 con 0x7f34b813cdc0 2024-09-15T21:21:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.785+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' created v120) ==== 115+0+0 (secure 0 0 0) 0x7f34b004dcd0 con 0x7f34b813cdc0 2024-09-15T21:21:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.785+0000 7f34adffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T21:21:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.785+0000 7f34adffb640 10 monclient: _finish_command 2 = system:0 pool 'fooset' created 2024-09-15T21:21:47.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.889+0000 7f34bf416640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T21:21:47.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.889+0000 7f34bf416640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T21:21:47.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.889+0000 7f34bf416640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:47.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.889+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) -- 0x7f34b8137120 con 0x7f34b813cdc0 2024-09-15T21:21:47.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34adffb640 1 -- 172.21.15.119:0/1205657610 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' already exists v120) ==== 122+0+0 (secure 0 0 0) 0x7f34b0015be0 con 0x7f34b813cdc0 2024-09-15T21:21:47.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34adffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T21:21:47.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34adffb640 10 monclient: _finish_command 3 = system:0 pool 'fooset' already exists 2024-09-15T21:21:47.899 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'fooset' already exists 2024-09-15T21:21:47.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f349c03b970 msgr2=0x7f349c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:47.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f349c03b970 0x7f349c03de30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f34b4001090 tx=0x7f34b4007820 comp rx=0 tx=0).stop 2024-09-15T21:21:47.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 10 monclient: shutdown 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 msgr2=0x7f34b8136ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 0x7f34b8136ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f34b000be60 tx=0x7f34b000bf50 comp rx=0 tx=0).stop 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 shutdown_connections 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f349c03b970 0x7f349c03de30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34b813cdc0 0x7f34b8136ac0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b8138360 0x7f34b811daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 --2- 172.21.15.119:0/1205657610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34b8137870 0x7f34b811fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 >> 172.21.15.119:0/1205657610 conn(0x7f34b809be60 msgr2=0x7f34b813bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 shutdown_connections 2024-09-15T21:21:47.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:47.893+0000 7f34bf416640 1 -- 172.21.15.119:0/1205657610 wait complete. 2024-09-15T21:21:47.917 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create barset 8 2024-09-15T21:21:48.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient: get_monmap_and_config 2024-09-15T21:21:48.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient: build_initial_monmap 2024-09-15T21:21:48.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient: monmap: 2024-09-15T21:21:48.031 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:48.031 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:48.032 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:48.029044+0000 2024-09-15T21:21:48.032 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:48.029044+0000 2024-09-15T21:21:48.032 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:48.032 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:48.032 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:48.033 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:48.033 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:48.033 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:48.033 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:48.034 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 Processor -- start 2024-09-15T21:21:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 -- start start 2024-09-15T21:21:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient: init 2024-09-15T21:21:48.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff81c0a2b00 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff81c0a35f0 0x7ff81c072f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): picked mon.noname-a con 0x7ff81c0a35f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 0x7ff81c13d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): picked mon.noname-b con 0x7ff81c13ad10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81c074bd0 con 0x7ff81c0a2b00 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff81c0728c0 con 0x7ff81c0a35f0 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c072b30 con 0x7ff81c13ad10 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff820ed1640 10 monclient(hunting): authenticate will time out at 1226.825650s 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 0x7ff81c13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff81c0a35f0 0x7ff81c072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 0x7ff81c13d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32960/0 (socket says 172.21.15.119:32960) 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 1 -- 172.21.15.119:0/2185735698 learned_addr learned my addr 172.21.15.119:0/2185735698 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): get_auth_request con 0x7ff81c13ad10 auth_method 0 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81a59c640 1 --1- 172.21.15.119:0/2185735698 >> v1:172.21.15.17:6789/0 conn(0x7ff81c0a2b00 0x7ff81c0a2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48774/0 (socket says 172.21.15.119:48774) 2024-09-15T21:21:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): get_auth_request con 0x7ff81c0a35f0 auth_method 0 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient(hunting): handle_auth_done global_id 54318 payload 274 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 1 -- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 msgr2=0x7ff81c13d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 1 --2- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 0x7ff81c13d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 1 -- 172.21.15.119:0/2185735698 >> v1:172.21.15.17:6789/0 conn(0x7ff81c0a2b00 legacy=0x7ff81c0a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 1 monclient: found mon.noname-a 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff819d9b640 1 -- 172.21.15.119:0/2185735698 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c0a45f0 con 0x7ff81c0a35f0 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.025+0000 7ff81ad9d640 1 --2- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 0x7ff81c13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff819d9b640 1 --2- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c0a35f0 0x7ff81c072f50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8100097c0 tx=0x7ff81001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3296c15dc55b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 1 -- 172.21.15.119:0/2185735698 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100b0070 con 0x7ff81c0a35f0 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: dump: 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: _finish_auth 0 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: _check_auth_tickets 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 1 -- 172.21.15.119:0/2185735698 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8100ab600 con 0x7ff81c0a35f0 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 1 -- 172.21.15.119:0/2185735698 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c071980 con 0x7ff81c0a35f0 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 5 monclient: authenticate success, global_id 54318 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 4 set_mon_vals no callback set 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff81959a640 10 monclient: dump: 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 10 monclient: shutdown 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 1 -- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c0a35f0 msgr2=0x7ff81c072f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.029+0000 7ff820ed1640 1 --2- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c0a35f0 0x7ff81c072f50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8100097c0 tx=0x7ff81001c360 comp rx=0 tx=0).stop 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- 172.21.15.119:0/2185735698 shutdown_connections 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 --2- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c13ad10 0x7ff81c13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 --2- 172.21.15.119:0/2185735698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c0a35f0 0x7ff81c072f50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- 172.21.15.119:0/2185735698 >> 172.21.15.119:0/2185735698 conn(0x7ff81c0a8100 msgr2=0x7ff81c0a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- 172.21.15.119:0/2185735698 shutdown_connections 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- 172.21.15.119:0/2185735698 wait complete. 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: build_initial_monmap 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: monmap: 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:48.052 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 Processor -- start 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- start start 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: init 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c0a2b00 0x7ff81c0a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): picked mon.noname-c con 0x7ff81c0a2b00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c0a35f0 0x7ff81c135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): picked mon.noname-b con 0x7ff81c0a35f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c13ad10 0x7ff81c1cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): picked mon.noname-a con 0x7ff81c13ad10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c06ee60 con 0x7ff81c13ad10 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c06ece0 con 0x7ff81c0a35f0 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c06efe0 con 0x7ff81c0a2b00 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient(hunting): authenticate will time out at 1226.833601s 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff819d9b640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c0a35f0 msgr2=0x7ff81c135f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff819d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c0a35f0 0x7ff81c135f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c0a2b00 0x7ff81c0a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c13ad10 0x7ff81c1cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c0a2b00 0x7ff81c0a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32966/0 (socket says 172.21.15.119:32966) 2024-09-15T21:21:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 1 -- 172.21.15.119:0/730178765 learned_addr learned my addr 172.21.15.119:0/730178765 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): get_auth_request con 0x7ff81c13ad10 auth_method 0 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): get_auth_request con 0x7ff81c0a2b00 auth_method 0 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient(hunting): handle_auth_done global_id 54324 payload 824 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 -- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c0a2b00 msgr2=0x7ff81c0a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c0a2b00 0x7ff81c0a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 -- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c0a35f0 msgr2=0x7ff81c135f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c0a35f0 0x7ff81c135f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 monclient: found mon.noname-a 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 -- 172.21.15.119:0/730178765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c070a20 con 0x7ff81c13ad10 2024-09-15T21:21:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff81ad9d640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c13ad10 0x7ff81c1cf460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff81400efc0 tx=0x7ff81400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400cac0 con 0x7ff81c13ad10 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: dump: 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:48.058 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 5 monclient: authenticate success, global_id 54324 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: _renew_subs 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c0704b0 con 0x7ff81c13ad10 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81400cc80 con 0x7ff81c13ad10 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff818d99640 4 set_mon_vals no callback set 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: _renew_subs 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c0a0ca0 con 0x7ff81c13ad10 2024-09-15T21:21:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814003ba0 con 0x7ff81c13ad10 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.033+0000 7ff80affd640 10 monclient: dump: 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff820ed1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff820ed1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff820ed1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c072f50 con 0x7ff81c13ad10 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff81401b030 con 0x7ff81c13ad10 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff80affd640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7fc03b970 0x7ff7fc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6944+0+0 (secure 0 0 0) 0x7ff814051530 con 0x7ff81c13ad10 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff81a59c640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7fc03b970 0x7ff7fc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff81a59c640 10 monclient: get_auth_request con 0x7ff7fc03b970 auth_method 0 2024-09-15T21:21:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.037+0000 7ff81a59c640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7fc03b970 0x7ff7fc03de30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff80c000f70 tx=0x7ff80c007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:48.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.049+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff814022020 con 0x7ff81c13ad10 2024-09-15T21:21:48.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.049+0000 7ff80affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:48.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.049+0000 7ff80affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.209+0000 7ff820ed1640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T21:21:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.209+0000 7ff820ed1640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T21:21:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.209+0000 7ff820ed1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.209+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) -- 0x7ff81c06a810 con 0x7ff81c13ad10 2024-09-15T21:21:48.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.789+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' created v121) ==== 115+0+0 (secure 0 0 0) 0x7ff81c0704b0 con 0x7ff81c13ad10 2024-09-15T21:21:48.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.789+0000 7ff80affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T21:21:48.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.789+0000 7ff80affd640 10 monclient: _finish_command 2 = system:0 pool 'barset' created 2024-09-15T21:21:48.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff820ed1640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T21:21:48.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff820ed1640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T21:21:48.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff820ed1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:48.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) -- 0x7ff81c09f460 con 0x7ff81c13ad10 2024-09-15T21:21:48.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff80affd640 1 -- 172.21.15.119:0/730178765 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' already exists v121) ==== 122+0+0 (secure 0 0 0) 0x7ff81c070a20 con 0x7ff81c13ad10 2024-09-15T21:21:48.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff80affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T21:21:48.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.893+0000 7ff80affd640 10 monclient: _finish_command 3 = system:0 pool 'barset' already exists 2024-09-15T21:21:48.900 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'barset' already exists 2024-09-15T21:21:48.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7fc03b970 msgr2=0x7ff7fc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:48.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7fc03b970 0x7ff7fc03de30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff80c000f70 tx=0x7ff80c007ef0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 10 monclient: shutdown 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c13ad10 msgr2=0x7ff81c1cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c13ad10 0x7ff81c1cf460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff81400efc0 tx=0x7ff81400c490 comp rx=0 tx=0).stop 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 shutdown_connections 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7fc03b970 0x7ff7fc03de30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81c13ad10 0x7ff81c1cf460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c0a35f0 0x7ff81c135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 --2- 172.21.15.119:0/730178765 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff81c0a2b00 0x7ff81c0a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 >> 172.21.15.119:0/730178765 conn(0x7ff81c0a8100 msgr2=0x7ff81c13cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 shutdown_connections 2024-09-15T21:21:48.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:48.897+0000 7ff820ed1640 1 -- 172.21.15.119:0/730178765 wait complete. 2024-09-15T21:21:48.917 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set barset size 3 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient: get_monmap_and_config 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient: build_initial_monmap 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient: monmap: 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:49.024832+0000 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:49.024832+0000 2024-09-15T21:21:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:49.027 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 Processor -- start 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 -- start start 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient: init 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feb2009d550 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:49.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb20139c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): picked mon.noname-b con 0x7feb2009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb2009e040 0x7feb2013c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): picked mon.noname-c con 0x7feb2009e040 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb200a3600 con 0x7feb2009d550 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7feb2009f1b0 con 0x7feb2009e040 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb200a2920 con 0x7feb2009fae0 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb26297640 10 monclient(hunting): authenticate will time out at 1227.821361s 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb24834640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7feb2009e040 msgr2=0x7feb2013c1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb24834640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb2009e040 0x7feb2013c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb20139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:49.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7feb2009d550 0x7feb2009d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48784/0 (socket says 172.21.15.119:48784) 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1ffff640 1 -- 172.21.15.119:0/3404154341 learned_addr learned my addr 172.21.15.119:0/3404154341 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): get_auth_request con 0x7feb2009fae0 auth_method 0 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_done global_id 44444 payload 274 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 1 -- 172.21.15.119:0/3404154341 >> v2:172.21.15.119:3300/0 conn(0x7feb2009e040 msgr2=0x7feb2013c1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 1 --2- 172.21.15.119:0/3404154341 >> v2:172.21.15.119:3300/0 conn(0x7feb2009e040 0x7feb2013c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 1 -- 172.21.15.119:0/3404154341 >> v1:172.21.15.17:6789/0 conn(0x7feb2009d550 legacy=0x7feb2009d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:49.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:49.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:49.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:49.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.021+0000 7feb1f7fe640 1 -- 172.21.15.119:0/3404154341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb200a1d60 con 0x7feb2009fae0 2024-09-15T21:21:49.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 1 --2- 172.21.15.119:0/3404154341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb20139c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb080097c0 tx=0x7feb0801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b827ab69884304e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:49.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 1 -- 172.21.15.119:0/3404154341 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080b0070 con 0x7feb2009fae0 2024-09-15T21:21:49.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:49.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: dump: 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:49.038 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: _finish_auth 0 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 5 monclient: authenticate success, global_id 44444 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 1 -- 172.21.15.119:0/3404154341 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb080ab600 con 0x7feb2009fae0 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 1 -- 172.21.15.119:0/3404154341 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080ab9d0 con 0x7feb2009fae0 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 4 set_mon_vals no callback set 2024-09-15T21:21:49.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1effd640 10 monclient: dump: 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: shutdown 2024-09-15T21:21:49.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- 172.21.15.119:0/3404154341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 msgr2=0x7feb20139c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 --2- 172.21.15.119:0/3404154341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb20139c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb080097c0 tx=0x7feb0801c380 comp rx=0 tx=0).stop 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- 172.21.15.119:0/3404154341 shutdown_connections 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 --2- 172.21.15.119:0/3404154341 >> v2:172.21.15.119:3300/0 conn(0x7feb2009e040 0x7feb2013c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 --2- 172.21.15.119:0/3404154341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb20139c70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- 172.21.15.119:0/3404154341 >> 172.21.15.119:0/3404154341 conn(0x7feb200a66e0 msgr2=0x7feb200a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- 172.21.15.119:0/3404154341 shutdown_connections 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- 172.21.15.119:0/3404154341 wait complete. 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: build_initial_monmap 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: monmap: 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:49.041 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 Processor -- start 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- start start 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: init 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb2009d550 0x7feb2011ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:49.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): picked mon.noname-b con 0x7feb2009d550 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb2009e040 0x7feb20134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): picked mon.noname-a con 0x7feb2009e040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb201cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): picked mon.noname-c con 0x7feb2009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb2013d090 con 0x7feb2009e040 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb2013cf10 con 0x7feb2009d550 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb2013d210 con 0x7feb2009fae0 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb26297640 10 monclient(hunting): authenticate will time out at 1227.826785s 2024-09-15T21:21:49.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1ffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb2009d550 msgr2=0x7feb2011ece0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb2009d550 0x7feb2011ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb201cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb2009e040 0x7feb20134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb201cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32982/0 (socket says 172.21.15.119:32982) 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 -- 172.21.15.119:0/2312566715 learned_addr learned my addr 172.21.15.119:0/2312566715 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): get_auth_request con 0x7feb2009fae0 auth_method 0 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): get_auth_request con 0x7feb2009e040 auth_method 0 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:49.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient(hunting): handle_auth_done global_id 44447 payload 824 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 -- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb2009d550 msgr2=0x7feb2011ece0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb2009d550 0x7feb2011ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 -- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb2009e040 msgr2=0x7feb20134b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb2009e040 0x7feb20134b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient: _finish_hunting 0 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 monclient: found mon.noname-c 2024-09-15T21:21:49.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.025+0000 7feb24834640 1 -- 172.21.15.119:0/2312566715 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2011e180 con 0x7feb2009fae0 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb24834640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb201cf840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb1000be50 tx=0x7feb1000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10016430 con 0x7feb2009fae0 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: dump: 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:49.046 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 5 monclient: authenticate success, global_id 44447 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb10010030 con 0x7feb2009fae0 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 monclient: _renew_subs 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2013c310 con 0x7feb2009fae0 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:49.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10016850 con 0x7feb2009fae0 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 10 monclient: dump: 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:49.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1e7fc640 4 set_mon_vals no callback set 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 monclient: _renew_subs 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201204e0 con 0x7feb2009fae0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2009d930 con 0x7feb2009fae0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7feb100153e0 con 0x7feb2009fae0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb0003b970 0x7feb0003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(121..121 src has 1..121) ==== 7289+0+0 (secure 0 0 0) 0x7feb10050f10 con 0x7feb2009fae0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1ffff640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb0003b970 0x7feb0003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1ffff640 10 monclient: get_auth_request con 0x7feb0003b970 auth_method 0 2024-09-15T21:21:49.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.029+0000 7feb1ffff640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb0003b970 0x7feb0003de30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb201359d0 tx=0x7feb18007b10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:49.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.041+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7feb10013cf0 con 0x7feb2009fae0 2024-09-15T21:21:49.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.041+0000 7feb1cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:49.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.041+0000 7feb1cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:49.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.197+0000 7feb26297640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T21:21:49.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.197+0000 7feb26297640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T21:21:49.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.197+0000 7feb26297640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.197+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) -- 0x7feb2006a830 con 0x7feb2009fae0 2024-09-15T21:21:49.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.801+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v122) ==== 126+0+0 (secure 0 0 0) 0x7feb1004e050 con 0x7feb2009fae0 2024-09-15T21:21:49.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.801+0000 7feb1cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T21:21:49.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.801+0000 7feb1cff9640 10 monclient: _finish_command 2 = system:0 set pool 10 size to 3 2024-09-15T21:21:49.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.905+0000 7feb26297640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T21:21:49.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.905+0000 7feb26297640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T21:21:49.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.905+0000 7feb26297640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:49.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:49.905+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) -- 0x7feb2011cca0 con 0x7feb2009fae0 2024-09-15T21:21:50.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.805+0000 7feb1cff9640 1 -- 172.21.15.119:0/2312566715 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v123) ==== 126+0+0 (secure 0 0 0) 0x7feb2011cca0 con 0x7feb2009fae0 2024-09-15T21:21:50.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.805+0000 7feb1cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T21:21:50.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.805+0000 7feb1cff9640 10 monclient: _finish_command 3 = system:0 set pool 10 size to 3 2024-09-15T21:21:50.811 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 10 size to 3 2024-09-15T21:21:50.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb0003b970 msgr2=0x7feb0003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:50.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb0003b970 0x7feb0003de30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb201359d0 tx=0x7feb18007b10 comp rx=0 tx=0).stop 2024-09-15T21:21:50.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 10 monclient: shutdown 2024-09-15T21:21:50.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:50.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 msgr2=0x7feb201cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:50.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb201cf840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb1000be50 tx=0x7feb1000be80 comp rx=0 tx=0).stop 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 shutdown_connections 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb0003b970 0x7feb0003de30 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb201359d0 tx=0x7feb18007b10 comp rx=0 tx=0).stop 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb2009fae0 0x7feb201cf840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb2009e040 0x7feb20134b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 --2- 172.21.15.119:0/2312566715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb2009d550 0x7feb2011ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 >> 172.21.15.119:0/2312566715 conn(0x7feb200a66e0 msgr2=0x7feb200a4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 shutdown_connections 2024-09-15T21:21:50.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.809+0000 7feb26297640 1 -- 172.21.15.119:0/2312566715 wait complete. 2024-09-15T21:21:50.838 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T21:21:50.838 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:21:50.838 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T21:21:50.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.949+0000 7f187c507640 10 monclient: get_monmap_and_config 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.949+0000 7f187c507640 10 monclient: build_initial_monmap 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.949+0000 7f187c507640 10 monclient: monmap: 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:50.956372+0000 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:50.956372+0000 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:50.958 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:50.959 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:50.959 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 Processor -- start 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 -- start start 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient: init 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1874136b90 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 0x7f187413cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:50.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): picked mon.noname-b con 0x7f18740a10d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f18740a25b0 0x7f187413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): picked mon.noname-c con 0x7f18740a25b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18740a18f0 con 0x7f1874136b90 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f187409fc40 con 0x7f18740a25b0 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f18740a1bd0 con 0x7f18740a10d0 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187c507640 10 monclient(hunting): authenticate will time out at 1229.753065s 2024-09-15T21:21:50.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187a2a3640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f1874136b90 legacy=0x7f1874136f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187aaa4640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f18740a25b0 msgr2=0x7f187413f1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f187aaa4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f18740a25b0 0x7f187413f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 0x7f187413cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 0x7f187413cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:32996/0 (socket says 172.21.15.119:32996) 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 -- 172.21.15.119:0/1036177072 learned_addr learned my addr 172.21.15.119:0/1036177072 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): get_auth_request con 0x7f18740a10d0 auth_method 0 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:50.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:50.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:50.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient(hunting): handle_auth_done global_id 44450 payload 274 2024-09-15T21:21:50.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 -- 172.21.15.119:0/1036177072 >> v2:172.21.15.119:3300/0 conn(0x7f18740a25b0 msgr2=0x7f187413f1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:50.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 --2- 172.21.15.119:0/1036177072 >> v2:172.21.15.119:3300/0 conn(0x7f18740a25b0 0x7f187413f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 -- 172.21.15.119:0/1036177072 >> v1:172.21.15.119:6789/0 conn(0x7f1874136b90 legacy=0x7f1874136f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:50.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient: _finish_hunting 0 2024-09-15T21:21:50.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 monclient: found mon.noname-b 2024-09-15T21:21:50.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:50.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:50.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.953+0000 7f1879aa2640 1 -- 172.21.15.119:0/1036177072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18740a3660 con 0x7f18740a10d0 2024-09-15T21:21:50.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 1 --2- 172.21.15.119:0/1036177072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 0x7f187413cc50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1864009870 tx=0x7f186401c230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21608cca0e1e410 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 1 -- 172.21.15.119:0/1036177072 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640b0070 con 0x7f18740a10d0 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: dump: 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:50.977 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: _finish_auth 0 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: _check_auth_tickets 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 5 monclient: authenticate success, global_id 44450 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 1 -- 172.21.15.119:0/1036177072 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18640ab600 con 0x7f18740a10d0 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 4 set_mon_vals no callback set 2024-09-15T21:21:50.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 1 -- 172.21.15.119:0/1036177072 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640a4e60 con 0x7f18740a10d0 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f18792a1640 10 monclient: dump: 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:50.979 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: shutdown 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- 172.21.15.119:0/1036177072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 msgr2=0x7f187413cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 --2- 172.21.15.119:0/1036177072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 0x7f187413cc50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1864009870 tx=0x7f186401c230 comp rx=0 tx=0).stop 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- 172.21.15.119:0/1036177072 shutdown_connections 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 --2- 172.21.15.119:0/1036177072 >> v2:172.21.15.119:3300/0 conn(0x7f18740a25b0 0x7f187413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 --2- 172.21.15.119:0/1036177072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a10d0 0x7f187413cc50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- 172.21.15.119:0/1036177072 >> 172.21.15.119:0/1036177072 conn(0x7f18740a66e0 msgr2=0x7f18740a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- 172.21.15.119:0/1036177072 shutdown_connections 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- 172.21.15.119:0/1036177072 wait complete. 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: build_initial_monmap 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: monmap: 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:50.981 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 Processor -- start 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- start start 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: init 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18740a10d0 0x7f18741cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:50.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): picked mon.noname-b con 0x7f18740a10d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a25b0 0x7f1874132a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): picked mon.noname-c con 0x7f18740a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1874136b90 0x7f187413be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): picked mon.noname-a con 0x7f1874136b90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1874142170 con 0x7f1874136b90 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1874141ff0 con 0x7f18740a10d0 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f18741422f0 con 0x7f18740a25b0 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:50.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187c507640 10 monclient(hunting): authenticate will time out at 1229.758888s 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187a2a3640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18740a10d0 msgr2=0x7f18741cf6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187a2a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18740a10d0 0x7f18741cf6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a25b0 0x7f1874132a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1874136b90 0x7f187413be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1874136b90 0x7f187413be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:51378/0 (socket says 172.21.15.119:51378) 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 -- 172.21.15.119:0/837303229 learned_addr learned my addr 172.21.15.119:0/837303229 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a25b0 0x7f1874132a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33012/0 (socket says 172.21.15.119:33012) 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): get_auth_request con 0x7f1874136b90 auth_method 0 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:50.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): get_auth_request con 0x7f18740a25b0 auth_method 0 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f1879aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient(hunting): handle_auth_done global_id 54333 payload 824 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 -- 172.21.15.119:0/837303229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a25b0 msgr2=0x7f1874132a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:50.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 --2- 172.21.15.119:0/837303229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18740a25b0 0x7f1874132a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 -- 172.21.15.119:0/837303229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18740a10d0 msgr2=0x7f18741cf6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 --2- 172.21.15.119:0/837303229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18740a10d0 0x7f18741cf6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient: _finish_hunting 0 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 monclient: found mon.noname-a 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.957+0000 7f187aaa4640 1 -- 172.21.15.119:0/837303229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18741d05f0 con 0x7f1874136b90 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187aaa4640 1 --2- 172.21.15.119:0/837303229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1874136b90 0x7f187413be60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f186c00be60 tx=0x7f186c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c016430 con 0x7f1874136b90 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: dump: 2024-09-15T21:21:50.986 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 5 monclient: authenticate success, global_id 54333 2024-09-15T21:21:50.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 monclient: _renew_subs 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 1 -- 172.21.15.119:0/837303229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187409d240 con 0x7f1874136b90 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f186c010030 con 0x7f1874136b90 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186bfff640 4 set_mon_vals no callback set 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 monclient: _renew_subs 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 1 -- 172.21.15.119:0/837303229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187411caf0 con 0x7f1874136b90 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c01b060 con 0x7f1874136b90 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:50.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 10 monclient: dump: 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:50.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187c507640 1 -- 172.21.15.119:0/837303229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187411cf40 con 0x7f1874136b90 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f186c016810 con 0x7f1874136b90 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 1 --2- 172.21.15.119:0/837303229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f185803b970 0x7f185803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 7289+0+0 (secure 0 0 0) 0x7f186c050df0 con 0x7f1874136b90 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187a2a3640 1 --2- 172.21.15.119:0/837303229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f185803b970 0x7f185803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187a2a3640 10 monclient: get_auth_request con 0x7f185803b970 auth_method 0 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.961+0000 7f187a2a3640 1 --2- 172.21.15.119:0/837303229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f185803b970 0x7f185803de30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f187413f020 tx=0x7f1870006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.973+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f187409d240 con 0x7f1874136b90 2024-09-15T21:21:50.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.973+0000 7f186affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:50.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:50.973+0000 7f186affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:51.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f187c507640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:51.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f187c507640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:51.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f187c507640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f187c507640 1 -- 172.21.15.119:0/837303229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7f1874136f70 con 0x7f1874136b90 2024-09-15T21:21:51.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f186affd640 1 -- 172.21.15.119:0/837303229 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 no weight-set for pool 'fooset' v123) ==== 160+0+0 (secure 0 0 0) 0x7f18741d05f0 con 0x7f1874136b90 2024-09-15T21:21:51.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f186affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:51.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.153+0000 7f186affd640 10 monclient: _finish_command 2 = mon:2 no weight-set for pool 'fooset' 2024-09-15T21:21:51.159 INFO:tasks.workunit.client.0.smithi119.stderr:Error ENOENT: no weight-set for pool 'fooset' 2024-09-15T21:21:51.164 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:21:51.164 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create fooset flat 2024-09-15T21:21:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient: get_monmap_and_config 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient: build_initial_monmap 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient: monmap: 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:51.285444+0000 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:51.285444+0000 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:51.288 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:51.288 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 1 Processor -- start 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 1 -- start start 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient: init 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135bf0 0x7f5c9009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient(hunting): picked mon.noname-b con 0x7f5c90135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c90135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5c90136430 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c900a1870 con 0x7f5c90135fd0 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c900a1d80 con 0x7f5c90136430 2024-09-15T21:21:51.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c900a3600 con 0x7f5c90135bf0 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c97068640 10 monclient(hunting): authenticate will time out at 1230.082454s 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c95605640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f5c90136430 legacy=0x7f5c900a1160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135bf0 0x7f5c9009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5c90135fd0 0x7f5c900a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48786/0 (socket says 172.21.15.119:48786) 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8ffff640 1 -- 172.21.15.119:0/2381353681 learned_addr learned my addr 172.21.15.119:0/2381353681 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): get_auth_request con 0x7f5c90135bf0 auth_method 0 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:51.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/2381353681 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007514109 0 0) 0x7f5c900a1870 con 0x7f5c90135fd0 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8f7fe640 10 monclient(hunting): my global_id is 54336 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/2381353681 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c78003530 con 0x7f5c90135fd0 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient(hunting): handle_auth_done global_id 44456 payload 274 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 1 -- 172.21.15.119:0/2381353681 >> v1:172.21.15.119:6789/0 conn(0x7f5c90136430 legacy=0x7f5c900a1160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 1 -- 172.21.15.119:0/2381353681 >> v1:172.21.15.17:6789/0 conn(0x7f5c90135fd0 legacy=0x7f5c900a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient: _finish_hunting 0 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 1 monclient: found mon.noname-b 2024-09-15T21:21:51.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:51.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:51.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.281+0000 7f5c94e04640 1 -- 172.21.15.119:0/2381353681 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c900a2060 con 0x7f5c90135bf0 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c94e04640 1 --2- 172.21.15.119:0/2381353681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135bf0 0x7f5c9009fe70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009dc0 tx=0x7f5c8401c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9714e85f626501 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/2381353681 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840b2070 con 0x7f5c90135bf0 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: dump: 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:51.299 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 5 monclient: authenticate success, global_id 44456 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/2381353681 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c840a58c0 con 0x7f5c90135bf0 2024-09-15T21:21:51.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/2381353681 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840a5c70 con 0x7f5c90135bf0 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c8f7fe640 10 monclient: dump: 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:51.301 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 4 set_mon_vals no callback set 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 10 monclient: shutdown 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 -- 172.21.15.119:0/2381353681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135bf0 msgr2=0x7f5c9009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 --2- 172.21.15.119:0/2381353681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135bf0 0x7f5c9009fe70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009dc0 tx=0x7f5c8401c820 comp rx=0 tx=0).stop 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 -- 172.21.15.119:0/2381353681 shutdown_connections 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 --2- 172.21.15.119:0/2381353681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135bf0 0x7f5c9009fe70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 -- 172.21.15.119:0/2381353681 >> 172.21.15.119:0/2381353681 conn(0x7f5c900a66e0 msgr2=0x7f5c900a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 -- 172.21.15.119:0/2381353681 shutdown_connections 2024-09-15T21:21:51.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.285+0000 7f5c97068640 1 -- 172.21.15.119:0/2381353681 wait complete. 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: build_initial_monmap 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: monmap: 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 Processor -- start 2024-09-15T21:21:51.303 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- start start 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: init 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 0x7f5c9011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): picked mon.noname-a con 0x7f5c90135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135fd0 0x7f5c9013d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): picked mon.noname-c con 0x7f5c90135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90136430 0x7f5c901cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): picked mon.noname-b con 0x7f5c90136430 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:51.304 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9006ec70 con 0x7f5c90135bf0 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901416d0 con 0x7f5c90136430 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9011e270 con 0x7f5c90135fd0 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient(hunting): authenticate will time out at 1230.089487s 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c95605640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90136430 msgr2=0x7f5c901cd8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c95605640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90136430 0x7f5c901cd8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 0x7f5c9011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 0x7f5c9011ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:51386/0 (socket says 172.21.15.119:51386) 2024-09-15T21:21:51.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 -- 172.21.15.119:0/3998905784 learned_addr learned my addr 172.21.15.119:0/3998905784 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): get_auth_request con 0x7f5c90135bf0 auth_method 0 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135fd0 0x7f5c9013d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): get_auth_request con 0x7f5c90135fd0 auth_method 0 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:51.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient(hunting): handle_auth_done global_id 54339 payload 824 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 -- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135fd0 msgr2=0x7f5c9013d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135fd0 0x7f5c9013d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 -- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90136430 msgr2=0x7f5c901cd8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90136430 0x7f5c901cd8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient: _finish_hunting 0 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 monclient: found mon.noname-a 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 -- 172.21.15.119:0/3998905784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9009d240 con 0x7f5c90135bf0 2024-09-15T21:21:51.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c94e04640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 0x7f5c9011ca70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000f90 tx=0x7f5c840195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840bd020 con 0x7f5c90135bf0 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: dump: 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:51.308 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 5 monclient: authenticate success, global_id 54339 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _renew_subs 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9013e310 con 0x7f5c90135bf0 2024-09-15T21:21:51.309 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c84003bc0 con 0x7f5c90135bf0 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _renew_subs 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c90134f10 con 0x7f5c90135bf0 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8effd640 4 set_mon_vals no callback set 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840b2040 con 0x7f5c90135bf0 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c8d7fa640 10 monclient: dump: 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.310 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.289+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9009fe70 con 0x7f5c90135bf0 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.293+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5c840c0020 con 0x7f5c90135bf0 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.293+0000 7f5c8d7fa640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c7c03b9a0 0x7f5c7c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:51.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.293+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 7289+0+0 (secure 0 0 0) 0x7f5c840eb4e0 con 0x7f5c90135bf0 2024-09-15T21:21:51.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.293+0000 7f5c8ffff640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c7c03b9a0 0x7f5c7c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:51.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.293+0000 7f5c8ffff640 10 monclient: get_auth_request con 0x7f5c7c03b9a0 auth_method 0 2024-09-15T21:21:51.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.293+0000 7f5c8ffff640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c7c03b9a0 0x7f5c7c03de60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5c88007fd0 tx=0x7f5c88006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:51.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.305+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5c840c7030 con 0x7f5c90135bf0 2024-09-15T21:21:51.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.305+0000 7f5c8d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:51.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.305+0000 7f5c8d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:51.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.485+0000 7f5c97068640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T21:21:51.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.485+0000 7f5c97068640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T21:21:51.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.485+0000 7f5c97068640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.485+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) -- 0x7f5c9006a830 con 0x7f5c90135bf0 2024-09-15T21:21:51.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.817+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 v124) ==== 109+0+0 (secure 0 0 0) 0x7f5c84016560 con 0x7f5c90135bf0 2024-09-15T21:21:51.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.817+0000 7f5c8d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T21:21:51.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.817+0000 7f5c8d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:51.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.929+0000 7f5c97068640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T21:21:51.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.929+0000 7f5c97068640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T21:21:51.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.929+0000 7f5c97068640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:51.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.929+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) -- 0x7f5c9011dba0 con 0x7f5c90135bf0 2024-09-15T21:21:51.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/3998905784 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 weight-set for pool 'fooset' already created v124) ==== 153+0+0 (secure 0 0 0) 0x7f5c840e9450 con 0x7f5c90135bf0 2024-09-15T21:21:51.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c8d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T21:21:51.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c8d7fa640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'fooset' already created 2024-09-15T21:21:51.938 INFO:tasks.workunit.client.0.smithi119.stderr:weight-set for pool 'fooset' already created 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c7c03b9a0 msgr2=0x7f5c7c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c7c03b9a0 0x7f5c7c03de60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5c88007fd0 tx=0x7f5c88006ab0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 10 monclient: shutdown 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 msgr2=0x7f5c9011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 0x7f5c9011ca70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000f90 tx=0x7f5c840195c0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 shutdown_connections 2024-09-15T21:21:51.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c7c03b9a0 0x7f5c7c03de60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90136430 0x7f5c901cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c90135fd0 0x7f5c9013d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 --2- 172.21.15.119:0/3998905784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c90135bf0 0x7f5c9011ca70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:51.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 >> 172.21.15.119:0/3998905784 conn(0x7f5c900a66e0 msgr2=0x7f5c90136840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:51.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 shutdown_connections 2024-09-15T21:21:51.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:51.933+0000 7f5c97068640 1 -- 172.21.15.119:0/3998905784 wait complete. 2024-09-15T21:21:51.956 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create barset positional 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient: get_monmap_and_config 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient: build_initial_monmap 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient: monmap: 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:52.076848+0000 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:52.076848+0000 2024-09-15T21:21:52.078 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:52.079 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 1 Processor -- start 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 1 -- start start 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient: init 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff2140a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2140a41b0 0x7ff2140a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient(hunting): picked mon.noname-c con 0x7ff2140a41b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff21413cb80 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2141383e0 con 0x7ff21413cb80 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2140a19d0 con 0x7ff2140a7ac0 2024-09-15T21:21:52.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2140a7ea0 con 0x7ff2140a41b0 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff218ff5640 10 monclient(hunting): authenticate will time out at 1230.873437s 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7ff2140a7ac0 legacy=0x7ff214137cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7ff2140a41b0 msgr2=0x7ff2140a45c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2140a41b0 0x7ff2140a45c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff212d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff21413cb80 0x7ff21413ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:48790/0 (socket says 172.21.15.119:48790) 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff212d9d640 1 -- 172.21.15.119:0/4232589394 learned_addr learned my addr 172.21.15.119:0/4232589394 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3680752891 0 0) 0x7ff2141383e0 con 0x7ff21413cb80 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): _negotiate 2024-09-15T21:21:52.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): my global_id is 54342 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1fc003530 con 0x7ff21413cb80 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 67125368 0 0) 0x7ff1fc003530 con 0x7ff21413cb80 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 >> v2:172.21.15.119:3300/0 conn(0x7ff2140a41b0 msgr2=0x7ff2140a45c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 --2- 172.21.15.119:0/4232589394 >> v2:172.21.15.119:3300/0 conn(0x7ff2140a41b0 0x7ff2140a45c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 >> v1:172.21.15.119:6789/0 conn(0x7ff2140a7ac0 legacy=0x7ff214137cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:21:52.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2140a6960 con 0x7ff21413cb80 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: _finish_auth 0 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: _check_auth_tickets 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff204001700 con 0x7ff21413cb80 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: dump: 2024-09-15T21:21:52.086 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 >> v1:172.21.15.17:6789/0 conn(0x7ff21413cb80 legacy=0x7ff21413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:52.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1fc0039c0 0x7ff1fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): picked mon.a con 0x7ff1fc0039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1fc004810 0x7ff1fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): picked mon.c con 0x7ff1fc004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0053d0 0x7ff1fc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): picked mon.b con 0x7ff1fc0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2140a7ea0 con 0x7ff1fc0039c0 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc00c310 con 0x7ff1fc0053d0 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc00c510 con 0x7ff1fc004810 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:52.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21159a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff212d9d640 1 -- 172.21.15.119:0/4232589394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0053d0 msgr2=0x7ff1fc00ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff212d9d640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0053d0 0x7ff1fc00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1fc0039c0 0x7ff1fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): get_auth_request con 0x7ff1fc0039c0 auth_method 0 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1fc004810 0x7ff1fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:52.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): get_auth_request con 0x7ff1fc004810 auth_method 0 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff211d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient(hunting): handle_auth_done global_id 54342 payload 293 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 -- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1fc004810 msgr2=0x7ff1fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1fc004810 0x7ff1fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 -- 172.21.15.119:0/4232589394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0053d0 msgr2=0x7ff1fc00ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:52.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0053d0 0x7ff1fc00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient: _finish_hunting 0 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 1 monclient: found mon.a 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.073+0000 7ff21259c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21259c640 1 -- 172.21.15.119:0/4232589394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc003530 con 0x7ff1fc0039c0 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21259c640 10 monclient: _finish_auth 0 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21259c640 10 monclient: _check_auth_tickets 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 5 monclient: authenticate success, global_id 54342 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:52.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff212d9d640 1 -- 172.21.15.119:0/4232589394 reap_dead start 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21259c640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1fc0039c0 0x7ff1fc003dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff20c003990 tx=0x7ff20c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3390eb161030ee8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c009520 con 0x7ff1fc0039c0 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: dump: 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:52.098 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff20c003bc0 con 0x7ff1fc0039c0 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 4 set_mon_vals no callback set 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 1 -- 172.21.15.119:0/4232589394 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c005070 con 0x7ff1fc0039c0 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff21159a640 10 monclient: dump: 2024-09-15T21:21:52.099 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 10 monclient: shutdown 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:52.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 -- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1fc0039c0 msgr2=0x7ff1fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1fc0039c0 0x7ff1fc003dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff20c003990 tx=0x7ff20c01c3f0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 -- 172.21.15.119:0/4232589394 shutdown_connections 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 --2- 172.21.15.119:0/4232589394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1fc0039c0 0x7ff1fc003dc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 -- 172.21.15.119:0/4232589394 >> 172.21.15.119:0/4232589394 conn(0x7ff2140a6fd0 msgr2=0x7ff2140a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 -- 172.21.15.119:0/4232589394 shutdown_connections 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 1 -- 172.21.15.119:0/4232589394 wait complete. 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 10 monclient: build_initial_monmap 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.077+0000 7ff218ff5640 10 monclient: monmap: 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:52.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 Processor -- start 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- start start 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: init 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0039c0 0x7ff2141cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): picked mon.noname-b con 0x7ff1fc0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 0x7ff214070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): picked mon.noname-c con 0x7ff2141d04f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2141d0910 0x7ff214071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): picked mon.noname-a con 0x7ff2141d0910 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff21259c640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0039c0 msgr2=0x7ff2141cffb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff214134b10 con 0x7ff2141d0910 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff214138140 con 0x7ff1fc0039c0 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff21259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0039c0 0x7ff2141cffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff214134c90 con 0x7ff2141d04f0 2024-09-15T21:21:52.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient(hunting): authenticate will time out at 1230.880911s 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 0x7ff214070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2141d0910 0x7ff214071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 0x7ff214070e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33042/0 (socket says 172.21.15.119:33042) 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 -- 172.21.15.119:0/36832895 learned_addr learned my addr 172.21.15.119:0/36832895 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): get_auth_request con 0x7ff2141d04f0 auth_method 0 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:52.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): get_auth_request con 0x7ff2141d0910 auth_method 0 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:52.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff212d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient(hunting): handle_auth_done global_id 44462 payload 824 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 -- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0039c0 msgr2=0x7ff2141cffb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0039c0 0x7ff2141cffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 -- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2141d0910 msgr2=0x7ff214071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2141d0910 0x7ff214071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 monclient: found mon.noname-c 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 -- 172.21.15.119:0/36832895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff214120d70 con 0x7ff2141d04f0 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff211d9b640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 0x7ff214070e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff20800ebb0 tx=0x7ff20800c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:52.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800b7b0 con 0x7ff2141d04f0 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: dump: 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 5 monclient: authenticate success, global_id 44462 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _renew_subs 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff214131890 con 0x7ff2141d04f0 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _renew_subs 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff20800b950 con 0x7ff2141d04f0 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21413c530 con 0x7ff2141d04f0 2024-09-15T21:21:52.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff210d99640 4 set_mon_vals no callback set 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800bd00 con 0x7ff2141d04f0 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 10 monclient: dump: 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:52.109 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff214135020 con 0x7ff2141d04f0 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.081+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff208017440 con 0x7ff2141d04f0 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.085+0000 7ff202ffd640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1f403b970 0x7ff1f403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.085+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(124..124 src has 1..124) ==== 7621+0+0 (secure 0 0 0) 0x7ff208051be0 con 0x7ff2141d04f0 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.085+0000 7ff21259c640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1f403b970 0x7ff1f403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.085+0000 7ff21259c640 10 monclient: get_auth_request con 0x7ff1f403b970 auth_method 0 2024-09-15T21:21:52.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.085+0000 7ff21259c640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1f403b970 0x7ff1f403de30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0aa000 tx=0x7ff20c005a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:52.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.097+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff20801c060 con 0x7ff2141d04f0 2024-09-15T21:21:52.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.097+0000 7ff202ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:52.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.097+0000 7ff202ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:52.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.273+0000 7ff218ff5640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T21:21:52.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.273+0000 7ff218ff5640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T21:21:52.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.273+0000 7ff218ff5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:52.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.273+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) -- 0x7ff2141210b0 con 0x7ff2141d04f0 2024-09-15T21:21:52.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.821+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 v125) ==== 115+0+0 (secure 0 0 0) 0x7ff20800ad10 con 0x7ff2141d04f0 2024-09-15T21:21:52.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.821+0000 7ff202ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T21:21:52.829 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.821+0000 7ff202ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:52.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.929+0000 7ff218ff5640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T21:21:52.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.929+0000 7ff218ff5640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T21:21:52.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.929+0000 7ff218ff5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:52.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.929+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) -- 0x7ff21406a760 con 0x7ff2141d04f0 2024-09-15T21:21:52.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff202ffd640 1 -- 172.21.15.119:0/36832895 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 weight-set for pool 'barset' already created v125) ==== 159+0+0 (secure 0 0 0) 0x7ff2080410b0 con 0x7ff2141d04f0 2024-09-15T21:21:52.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff202ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T21:21:52.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff202ffd640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'barset' already created 2024-09-15T21:21:52.939 INFO:tasks.workunit.client.0.smithi119.stderr:weight-set for pool 'barset' already created 2024-09-15T21:21:52.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1f403b970 msgr2=0x7ff1f403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:52.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1f403b970 0x7ff1f403de30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0aa000 tx=0x7ff20c005a90 comp rx=0 tx=0).stop 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 10 monclient: shutdown 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 msgr2=0x7ff214070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 0x7ff214070e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff20800ebb0 tx=0x7ff20800c750 comp rx=0 tx=0).stop 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 shutdown_connections 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1f403b970 0x7ff1f403de30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2141d0910 0x7ff214071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff2141d04f0 0x7ff214070e30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 --2- 172.21.15.119:0/36832895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1fc0039c0 0x7ff2141cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 >> 172.21.15.119:0/36832895 conn(0x7ff2140a6fd0 msgr2=0x7ff2140a2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 shutdown_connections 2024-09-15T21:21:52.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:52.933+0000 7ff218ff5640 1 -- 172.21.15.119:0/36832895 wait complete. 2024-09-15T21:21:52.957 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:21:52.957 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep fooset 2024-09-15T21:21:53.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient: get_monmap_and_config 2024-09-15T21:21:53.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient: build_initial_monmap 2024-09-15T21:21:53.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient: monmap: 2024-09-15T21:21:53.075 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:53.074110+0000 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:53.074110+0000 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:53.076 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 Processor -- start 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 -- start start 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient: init 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6d8c135ca0 0x7f6d8c136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): picked mon.noname-a con 0x7f6d8c135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6d8c0a0160 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d8c06ec30 0x7f6d8c0a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): picked mon.noname-c con 0x7f6d8c06ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8c1365c0 con 0x7f6d8c0a0160 2024-09-15T21:21:53.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8c09cf30 con 0x7f6d8c135ca0 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8c0a1ca0 con 0x7f6d8c06ec30 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d92868640 10 monclient(hunting): authenticate will time out at 1231.870771s 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8b7fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f6d8c0a0160 legacy=0x7f6d8c0a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d90e05640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f6d8c06ec30 msgr2=0x7f6d8c0a10f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d90e05640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d8c06ec30 0x7f6d8c0a10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6d8c135ca0 0x7f6d8c136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6d8c135ca0 0x7f6d8c136080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47336/0 (socket says 172.21.15.119:47336) 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1179468474 learned_addr learned my addr 172.21.15.119:0/1179468474 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:53.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): get_auth_request con 0x7f6d8c135ca0 auth_method 0 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_done global_id 54348 payload 274 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1179468474 >> v2:172.21.15.119:3300/0 conn(0x7f6d8c06ec30 msgr2=0x7f6d8c0a10f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 --2- 172.21.15.119:0/1179468474 >> v2:172.21.15.119:3300/0 conn(0x7f6d8c06ec30 0x7f6d8c0a10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1179468474 >> v1:172.21.15.119:6789/0 conn(0x7f6d8c0a0160 legacy=0x7f6d8c0a09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 monclient: found mon.noname-a 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:53.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:53.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.069+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1179468474 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c0a1f80 con 0x7f6d8c135ca0 2024-09-15T21:21:53.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8bfff640 1 --2- 172.21.15.119:0/1179468474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c136080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0a29c0 tx=0x7f6d8401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824c37ebaaa7fdc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 1 -- 172.21.15.119:0/1179468474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840b0070 con 0x7f6d8c135ca0 2024-09-15T21:21:53.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:53.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: dump: 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:53.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 1 -- 172.21.15.119:0/1179468474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840a4e50 con 0x7f6d8c135ca0 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 1 -- 172.21.15.119:0/1179468474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840ab820 con 0x7f6d8c135ca0 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d8affd640 10 monclient: dump: 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.089 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 5 monclient: authenticate success, global_id 54348 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 4 set_mon_vals no callback set 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 10 monclient: shutdown 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:53.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 -- 172.21.15.119:0/1179468474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 msgr2=0x7f6d8c136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 --2- 172.21.15.119:0/1179468474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c136080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0a29c0 tx=0x7f6d8401c400 comp rx=0 tx=0).stop 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 -- 172.21.15.119:0/1179468474 shutdown_connections 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 --2- 172.21.15.119:0/1179468474 >> v2:172.21.15.119:3300/0 conn(0x7f6d8c06ec30 0x7f6d8c0a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 --2- 172.21.15.119:0/1179468474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c136080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 -- 172.21.15.119:0/1179468474 >> 172.21.15.119:0/1179468474 conn(0x7f6d8c0a2ae0 msgr2=0x7f6d8c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 -- 172.21.15.119:0/1179468474 shutdown_connections 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.073+0000 7f6d92868640 1 -- 172.21.15.119:0/1179468474 wait complete. 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: build_initial_monmap 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: monmap: 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:53.091 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 Processor -- start 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- start start 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: init 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 0x7f6d8c133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): picked mon.noname-c con 0x7f6d8c06ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0a0160 0x7f6d8c1cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): picked mon.noname-b con 0x7f6d8c0a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c1ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): picked mon.noname-a con 0x7f6d8c135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c140240 con 0x7f6d8c135ca0 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c0a1840 con 0x7f6d8c0a0160 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8c1403c0 con 0x7f6d8c06ec30 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient(hunting): authenticate will time out at 1231.877127s 2024-09-15T21:21:53.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8b7fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0a0160 msgr2=0x7f6d8c1cdc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0a0160 0x7f6d8c1cdc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 0x7f6d8c133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c1ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 0x7f6d8c133e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54332/0 (socket says 172.21.15.119:54332) 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1679379970 learned_addr learned my addr 172.21.15.119:0/1679379970 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): get_auth_request con 0x7f6d8c06ec30 auth_method 0 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): get_auth_request con 0x7f6d8c135ca0 auth_method 0 2024-09-15T21:21:53.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient(hunting): handle_auth_done global_id 44465 payload 824 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0a0160 msgr2=0x7f6d8c1cdc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0a0160 0x7f6d8c1cdc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 msgr2=0x7f6d8c1ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c1ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 monclient: found mon.noname-c 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 -- 172.21.15.119:0/1679379970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c13d5a0 con 0x7f6d8c06ec30 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d90e05640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c1ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8bfff640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 0x7f6d8c133e80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d8401aab0 tx=0x7f6d840191a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840b9020 con 0x7f6d8c06ec30 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:53.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: dump: 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: _finish_auth 0 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:21:53.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 5 monclient: authenticate success, global_id 44465 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _renew_subs 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c134840 con 0x7f6d8c06ec30 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _renew_subs 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d84000e50 con 0x7f6d8c06ec30 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c139750 con 0x7f6d8c06ec30 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840b0050 con 0x7f6d8c06ec30 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:53.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d88ff9640 10 monclient: dump: 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d8a7fc640 4 set_mon_vals no callback set 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.077+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c136080 con 0x7f6d8c06ec30 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.081+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6d840bc020 con 0x7f6d8c06ec30 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.081+0000 7f6d88ff9640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6c03b9a0 0x7f6d6c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.081+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 8233+0+0 (secure 0 0 0) 0x7f6d840eadf0 con 0x7f6d8c06ec30 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.081+0000 7f6d8b7fe640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6c03b9a0 0x7f6d6c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.081+0000 7f6d8b7fe640 10 monclient: get_auth_request con 0x7f6d6c03b9a0 auth_method 0 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.081+0000 7f6d8b7fe640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6c03b9a0 0x7f6d6c03de60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d74006fd0 tx=0x7f6d74006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.093+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6d840c3070 con 0x7f6d8c06ec30 2024-09-15T21:21:53.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.093+0000 7f6d88ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.093+0000 7f6d88ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:53.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d92868640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d92868640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d92868640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f6d8c09f8f0 con 0x7f6d8c06ec30 2024-09-15T21:21:53.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v125) ==== 71+0+14 (secure 0 0 0) 0x7f6d84016110 con 0x7f6d8c06ec30 2024-09-15T21:21:53.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d88ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.269+0000 7f6d88ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:53.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d92868640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d92868640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d92868640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f6d8c137f50 con 0x7f6d8c06ec30 2024-09-15T21:21:53.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d88ff9640 1 -- 172.21.15.119:0/1679379970 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v125) ==== 71+0+14 (secure 0 0 0) 0x7f6d840a4e30 con 0x7f6d8c06ec30 2024-09-15T21:21:53.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d88ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.377+0000 7f6d88ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:53.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6c03b9a0 msgr2=0x7f6d6c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6c03b9a0 0x7f6d6c03de60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d74006fd0 tx=0x7f6d74006d40 comp rx=0 tx=0).stop 2024-09-15T21:21:53.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 10 monclient: shutdown 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 msgr2=0x7f6d8c133e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 0x7f6d8c133e80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d8401aab0 tx=0x7f6d840191a0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 shutdown_connections 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6c03b9a0 0x7f6d6c03de60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d8c135ca0 0x7f6d8c1ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0a0160 0x7f6d8c1cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 --2- 172.21.15.119:0/1679379970 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d8c06ec30 0x7f6d8c133e80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 >> 172.21.15.119:0/1679379970 conn(0x7f6d8c0a2ae0 msgr2=0x7f6d8c1308e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 shutdown_connections 2024-09-15T21:21:53.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.381+0000 7f6d92868640 1 -- 172.21.15.119:0/1679379970 wait complete. 2024-09-15T21:21:53.401 INFO:tasks.workunit.client.0.smithi119.stdout:fooset 2024-09-15T21:21:53.402 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:21:53.402 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep barset 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.509+0000 7f2af44ca640 10 monclient: get_monmap_and_config 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.509+0000 7f2af44ca640 10 monclient: build_initial_monmap 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.509+0000 7f2af44ca640 10 monclient: monmap: 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:53.515928+0000 2024-09-15T21:21:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:53.515928+0000 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:53.518 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.509+0000 7f2af44ca640 1 Processor -- start 2024-09-15T21:21:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.509+0000 7f2af44ca640 1 -- start start 2024-09-15T21:21:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.509+0000 7f2af44ca640 10 monclient: init 2024-09-15T21:21:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:21:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 0x7f2aec137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): picked mon.noname-b con 0x7f2aec137bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2aec0a09e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2aec0a0ea0 0x7f2aec073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): picked mon.noname-c con 0x7f2aec0a0ea0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec09cfa0 con 0x7f2aec0a09e0 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec09cd20 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0a1820 con 0x7f2aec137bc0 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af44ca640 10 monclient(hunting): authenticate will time out at 1232.312474s 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af1a65640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f2aec0a09e0 legacy=0x7f2aec073040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2a67640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f2aec0a0ea0 msgr2=0x7f2aec073750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2a67640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2aec0a0ea0 0x7f2aec073750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 0x7f2aec137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 0x7f2aec137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54338/0 (socket says 172.21.15.119:54338) 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 -- 172.21.15.119:0/64946742 learned_addr learned my addr 172.21.15.119:0/64946742 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): get_auth_request con 0x7f2aec137bc0 auth_method 0 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient(hunting): handle_auth_done global_id 44468 payload 274 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 -- 172.21.15.119:0/64946742 >> v2:172.21.15.119:3300/0 conn(0x7f2aec0a0ea0 msgr2=0x7f2aec073750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 --2- 172.21.15.119:0/64946742 >> v2:172.21.15.119:3300/0 conn(0x7f2aec0a0ea0 0x7f2aec073750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 -- 172.21.15.119:0/64946742 >> v1:172.21.15.119:6789/0 conn(0x7f2aec0a09e0 legacy=0x7f2aec073040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient: _finish_hunting 0 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 monclient: found mon.noname-b 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.513+0000 7f2af2266640 1 -- 172.21.15.119:0/64946742 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec0a1b00 con 0x7f2aec137bc0 2024-09-15T21:21:53.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2266640 1 --2- 172.21.15.119:0/64946742 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 0x7f2aec137fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009870 tx=0x7f2adc01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b3cc73fb99218e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 1 -- 172.21.15.119:0/64946742 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0b0070 con 0x7f2aec137bc0 2024-09-15T21:21:53.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: dump: 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.529 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: _finish_auth 0 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: _check_auth_tickets 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 5 monclient: authenticate success, global_id 44468 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 1 -- 172.21.15.119:0/64946742 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc0ab600 con 0x7f2aec137bc0 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 4 set_mon_vals no callback set 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 1 -- 172.21.15.119:0/64946742 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0ab9d0 con 0x7f2aec137bc0 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1264640 10 monclient: dump: 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: shutdown 2024-09-15T21:21:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- 172.21.15.119:0/64946742 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 msgr2=0x7f2aec137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 --2- 172.21.15.119:0/64946742 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 0x7f2aec137fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009870 tx=0x7f2adc01c530 comp rx=0 tx=0).stop 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- 172.21.15.119:0/64946742 shutdown_connections 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 --2- 172.21.15.119:0/64946742 >> v2:172.21.15.119:3300/0 conn(0x7f2aec0a0ea0 0x7f2aec073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 --2- 172.21.15.119:0/64946742 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec137bc0 0x7f2aec137fa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- 172.21.15.119:0/64946742 >> 172.21.15.119:0/64946742 conn(0x7f2aec0a2ae0 msgr2=0x7f2aec0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- 172.21.15.119:0/64946742 shutdown_connections 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- 172.21.15.119:0/64946742 wait complete. 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: build_initial_monmap 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: monmap: 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 Processor -- start 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- start start 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: init 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:21:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aec0a09e0 0x7f2aec11cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): picked mon.noname-b con 0x7f2aec0a09e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 0x7f2aec1338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): picked mon.noname-c con 0x7f2aec0a0ea0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aec137bc0 0x7f2aec1cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): picked mon.noname-a con 0x7f2aec137bc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec073e10 con 0x7f2aec137bc0 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec073c90 con 0x7f2aec0a09e0 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec1422d0 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af44ca640 10 monclient(hunting): authenticate will time out at 1232.319168s 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2266640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aec0a09e0 msgr2=0x7f2aec11cb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2266640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aec0a09e0 0x7f2aec11cb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 0x7f2aec1338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aec137bc0 0x7f2aec1cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 0x7f2aec1338b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54346/0 (socket says 172.21.15.119:54346) 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 1 -- 172.21.15.119:0/2392253408 learned_addr learned my addr 172.21.15.119:0/2392253408 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): get_auth_request con 0x7f2aec0a0ea0 auth_method 0 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): get_auth_request con 0x7f2aec137bc0 auth_method 0 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af1a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.517+0000 7f2af2a67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 10 monclient(hunting): handle_auth_done global_id 44471 payload 824 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 -- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aec0a09e0 msgr2=0x7f2aec11cb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aec0a09e0 0x7f2aec11cb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 -- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aec137bc0 msgr2=0x7f2aec1cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aec137bc0 0x7f2aec1cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 10 monclient: _finish_hunting 0 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 monclient: found mon.noname-c 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 -- 172.21.15.119:0/2392253408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec13d650 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af1a65640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 0x7f2aec1338b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800b580 tx=0x7f2ae800ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8013020 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: dump: 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 5 monclient: authenticate success, global_id 44471 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 monclient: _renew_subs 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec13ea40 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae8003dd0 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af0a63640 4 set_mon_vals no callback set 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 monclient: _renew_subs 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8005270 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec11e340 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 10 monclient: dump: 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec137fa0 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2ae8016020 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ad003b970 0x7f2ad003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 8233+0+0 (secure 0 0 0) 0x7f2ae8050b70 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af2266640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ad003b970 0x7f2ad003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af2266640 10 monclient: get_auth_request con 0x7f2ad003b970 auth_method 0 2024-09-15T21:21:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.521+0000 7f2af2266640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ad003b970 0x7f2ad003de30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0ad000 tx=0x7f2adc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.533+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2ae801d020 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.533+0000 7f2ae2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.533+0000 7f2ae2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2af44ca640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2af44ca640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2af44ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.723 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f2aec09d1d0 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v125) ==== 71+0+14 (secure 0 0 0) 0x7f2ae8022480 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2ae2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.724 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.717+0000 7f2ae2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:53.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2af44ca640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2af44ca640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2af44ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f2aec0008e0 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2ae2ffd640 1 -- 172.21.15.119:0/2392253408 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v125) ==== 71+0+14 (secure 0 0 0) 0x7f2ae804d7f0 con 0x7f2aec0a0ea0 2024-09-15T21:21:53.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2ae2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:53.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.821+0000 7f2ae2ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:53.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ad003b970 msgr2=0x7f2ad003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ad003b970 0x7f2ad003de30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0ad000 tx=0x7f2adc0a6000 comp rx=0 tx=0).stop 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 10 monclient: shutdown 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 msgr2=0x7f2aec1338b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 0x7f2aec1338b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800b580 tx=0x7f2ae800ba40 comp rx=0 tx=0).stop 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 shutdown_connections 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ad003b970 0x7f2ad003de30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aec137bc0 0x7f2aec1cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2aec0a0ea0 0x7f2aec1338b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 --2- 172.21.15.119:0/2392253408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2aec0a09e0 0x7f2aec11cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 >> 172.21.15.119:0/2392253408 conn(0x7f2aec0a2ae0 msgr2=0x7f2aec1369d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 shutdown_connections 2024-09-15T21:21:53.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.825+0000 7f2af44ca640 1 -- 172.21.15.119:0/2392253408 wait complete. 2024-09-15T21:21:53.844 INFO:tasks.workunit.client.0.smithi119.stdout:barset 2024-09-15T21:21:53.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set dump 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.953+0000 7f3b21a8c640 10 monclient: get_monmap_and_config 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.953+0000 7f3b21a8c640 10 monclient: build_initial_monmap 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.953+0000 7f3b21a8c640 10 monclient: monmap: 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:53.959633+0000 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:53.959633+0000 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.962 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:53.962 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:53.962 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:53.962 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:53.962 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:53.962 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.953+0000 7f3b21a8c640 1 Processor -- start 2024-09-15T21:21:53.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.953+0000 7f3b21a8c640 1 -- start start 2024-09-15T21:21:53.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.953+0000 7f3b21a8c640 10 monclient: init 2024-09-15T21:21:53.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:53.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-09-15T21:21:53.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 0x7f3b1c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): picked mon.noname-b con 0x7f3b1c0a4ed0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3b1c0a57f0 0x7f3b1c0a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): picked mon.noname-c con 0x7f3b1c0a57f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b1c137c90 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b1c0a6db0 con 0x7f3b1c137c90 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3b1c0a1210 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1c0a1b00 con 0x7f3b1c0a4ed0 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b21a8c640 10 monclient(hunting): authenticate will time out at 1232.756314s 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1affd640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f3b1c0a57f0 msgr2=0x7f3b1c0a5bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3b1c0a57f0 0x7f3b1c0a5bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 0x7f3b1c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 0x7f3b1c0a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54348/0 (socket says 172.21.15.119:54348) 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 -- 172.21.15.119:0/1722657011 learned_addr learned my addr 172.21.15.119:0/1722657011 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): get_auth_request con 0x7f3b1c0a4ed0 auth_method 0 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1a7fc640 1 -- 172.21.15.119:0/1722657011 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157086115 0 0) 0x7f3b1c0a6db0 con 0x7f3b1c137c90 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1a7fc640 10 monclient(hunting): my global_id is 54357 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1a7fc640 1 -- 172.21.15.119:0/1722657011 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b04003530 con 0x7f3b1c137c90 2024-09-15T21:21:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient(hunting): handle_auth_done global_id 44474 payload 274 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 -- 172.21.15.119:0/1722657011 >> v2:172.21.15.119:3300/0 conn(0x7f3b1c0a57f0 msgr2=0x7f3b1c0a5bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 --2- 172.21.15.119:0/1722657011 >> v2:172.21.15.119:3300/0 conn(0x7f3b1c0a57f0 0x7f3b1c0a5bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 -- 172.21.15.119:0/1722657011 >> v1:172.21.15.17:6789/0 conn(0x7f3b1c137c90 legacy=0x7f3b1c1380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:53.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.957+0000 7f3b1b7fe640 1 -- 172.21.15.119:0/1722657011 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c0a6ad0 con 0x7f3b1c0a4ed0 2024-09-15T21:21:53.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1b7fe640 1 --2- 172.21.15.119:0/1722657011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 0x7f3b1c0a52b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009870 tx=0x7f3b0c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e76c579f563da59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 1 -- 172.21.15.119:0/1722657011 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0b0070 con 0x7f3b1c0a4ed0 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: dump: 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.975 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 1 -- 172.21.15.119:0/1722657011 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0c0ab600 con 0x7f3b1c0a4ed0 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 1 -- 172.21.15.119:0/1722657011 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0ab9d0 con 0x7f3b1c0a4ed0 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 5 monclient: authenticate success, global_id 44474 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 4 set_mon_vals no callback set 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1a7fc640 10 monclient: dump: 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: shutdown 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:53.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- 172.21.15.119:0/1722657011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 msgr2=0x7f3b1c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/1722657011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 0x7f3b1c0a52b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009870 tx=0x7f3b0c01c440 comp rx=0 tx=0).stop 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- 172.21.15.119:0/1722657011 shutdown_connections 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/1722657011 >> v2:172.21.15.119:3300/0 conn(0x7f3b1c0a57f0 0x7f3b1c0a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/1722657011 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c0a4ed0 0x7f3b1c0a52b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- 172.21.15.119:0/1722657011 >> 172.21.15.119:0/1722657011 conn(0x7f3b1c0a6fd0 msgr2=0x7f3b1c0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- 172.21.15.119:0/1722657011 shutdown_connections 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- 172.21.15.119:0/1722657011 wait complete. 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: build_initial_monmap 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: monmap: 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:53.978 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 Processor -- start 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- start start 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: init 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b1c0a4ed0 0x7f3b1c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): picked mon.noname-b con 0x7f3b1c0a4ed0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:53.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b1c0a57f0 0x7f3b1c136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): picked mon.noname-a con 0x7f3b1c0a57f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 0x7f3b1c13cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): picked mon.noname-c con 0x7f3b1c137c90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c0a7ea0 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c0a7d20 con 0x7f3b1c0a4ed0 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1c142710 con 0x7f3b1c137c90 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b21a8c640 10 monclient(hunting): authenticate will time out at 1232.763055s 2024-09-15T21:21:53.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1b7fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b1c0a4ed0 msgr2=0x7f3b1c13f100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b1c0a4ed0 0x7f3b1c13f100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 0x7f3b1c13cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b1c0a57f0 0x7f3b1c136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 0x7f3b1c13cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54358/0 (socket says 172.21.15.119:54358) 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 1 -- 172.21.15.119:0/680703472 learned_addr learned my addr 172.21.15.119:0/680703472 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): get_auth_request con 0x7f3b1c137c90 auth_method 0 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): get_auth_request con 0x7f3b1c0a57f0 auth_method 0 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:53.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.961+0000 7f3b1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 10 monclient(hunting): handle_auth_done global_id 54360 payload 824 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 -- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 msgr2=0x7f3b1c13cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 0x7f3b1c13cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 -- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b1c0a4ed0 msgr2=0x7f3b1c13f100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b1c0a4ed0 0x7f3b1c13f100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 monclient: found mon.noname-a 2024-09-15T21:21:53.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 -- 172.21.15.119:0/680703472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c1d0770 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1bfff640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 0x7f3b1c13cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1affd640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b1c0a57f0 0x7f3b1c136ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b430 tx=0x7f3b1000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10016070 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: dump: 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.983 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b10003df0 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b100052b0 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 5 monclient: authenticate success, global_id 54360 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 monclient: _renew_subs 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b19ffb640 4 set_mon_vals no callback set 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c121250 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 10 monclient: dump: 2024-09-15T21:21:53.986 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 monclient: _renew_subs 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c137840 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.987 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1c0a52b0 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3b10019070 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b0803b970 0x7f3b0803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8233+0+0 (secure 0 0 0) 0x7f3b1004fdf0 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1b7fe640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b0803b970 0x7f3b0803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1b7fe640 10 monclient: get_auth_request con 0x7f3b0803b970 auth_method 0 2024-09-15T21:21:53.988 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.965+0000 7f3b1b7fe640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b0803b970 0x7f3b0803de30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0ad000 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:53.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.977+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3b1c1d0770 con 0x7f3b1c0a57f0 2024-09-15T21:21:53.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.977+0000 7f3b03fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:53.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:53.977+0000 7f3b03fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:54.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b21a8c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:54.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b21a8c640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:54.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b21a8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7f3b1c120e80 con 0x7f3b1c0a57f0 2024-09-15T21:21:54.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v125) ==== 73+0+6748 (secure 0 0 0) 0x7f3b10020090 con 0x7f3b1c0a57f0 2024-09-15T21:21:54.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b03fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:54.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.161+0000 7f3b03fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:54.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.269+0000 7f3b21a8c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:54.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.269+0000 7f3b21a8c640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:54.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.269+0000 7f3b21a8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.269+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7f3b1c1370e0 con 0x7f3b1c0a57f0 2024-09-15T21:21:54.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b03fff640 1 -- 172.21.15.119:0/680703472 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v125) ==== 73+0+6748 (secure 0 0 0) 0x7f3b1000f300 con 0x7f3b1c0a57f0 2024-09-15T21:21:54.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b03fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T21:21:54.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b03fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout:{ 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: "9": [ 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -1, 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.279 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -2, 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.280 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -3, 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.281 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -4, 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -5, 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.282 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -6, 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.283 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -7, 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.284 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -8, 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -9, 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.285 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -10, 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -11, 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.286 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -12, 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -13, 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.287 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -14, 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -15, 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.288 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: "10": [ 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -1, 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.289 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0.263671875, 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.290 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -2, 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.291 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -3, 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.292 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.293 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -4, 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -5, 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.294 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.295 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -6, 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.296 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -7, 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.297 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.298 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -8, 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.299 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -9, 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.300 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: ], 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: [ 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0.087890625, 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0, 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -10, 2024-09-15T21:21:54.301 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -11, 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.302 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -12, 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -13, 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.303 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -14, 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: }, 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: { 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: "bucket_id": -15, 2024-09-15T21:21:54.304 INFO:tasks.workunit.client.0.smithi119.stdout: "weight_set": [ 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout: [], 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout: [] 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout: } 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout: ] 2024-09-15T21:21:54.305 INFO:tasks.workunit.client.0.smithi119.stdout:} 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b0803b970 msgr2=0x7f3b0803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b0803b970 0x7f3b0803de30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0ad000 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 10 monclient: shutdown 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b1c0a57f0 msgr2=0x7f3b1c136ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b1c0a57f0 0x7f3b1c136ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b430 tx=0x7f3b1000b8f0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 shutdown_connections 2024-09-15T21:21:54.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b0803b970 0x7f3b0803de30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3b1c137c90 0x7f3b1c13cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b1c0a57f0 0x7f3b1c136ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 --2- 172.21.15.119:0/680703472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b1c0a4ed0 0x7f3b1c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 >> 172.21.15.119:0/680703472 conn(0x7f3b1c0a6fd0 msgr2=0x7f3b1c0a8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 shutdown_connections 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.273+0000 7f3b21a8c640 1 -- 172.21.15.119:0/680703472 wait complete. 2024-09-15T21:21:54.307 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T21:21:54.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient: get_monmap_and_config 2024-09-15T21:21:54.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient: build_initial_monmap 2024-09-15T21:21:54.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient: monmap: 2024-09-15T21:21:54.415 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:54.415 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:54.413975+0000 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:54.413975+0000 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:54.416 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:54.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 Processor -- start 2024-09-15T21:21:54.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 -- start start 2024-09-15T21:21:54.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient: init 2024-09-15T21:21:54.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:54.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-09-15T21:21:54.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffba4135bf0 0x7ffba409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): picked mon.noname-c con 0x7ffba4135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffba4135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 0x7ffba40a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): picked mon.noname-b con 0x7ffba4136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba40a1d80 con 0x7ffba4135fd0 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffba40a3600 con 0x7ffba4135bf0 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40a2060 con 0x7ffba4136460 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba89f9640 10 monclient(hunting): authenticate will time out at 1233.211189s 2024-09-15T21:21:54.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba259c640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7ffba4135bf0 msgr2=0x7ffba409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffba4135bf0 0x7ffba409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 0x7ffba40a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 0x7ffba40a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54372/0 (socket says 172.21.15.119:54372) 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffb9bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffba4135fd0 0x7ffba40a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:52026/0 (socket says 172.21.15.119:52026) 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 1 -- 172.21.15.119:0/841351180 learned_addr learned my addr 172.21.15.119:0/841351180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): get_auth_request con 0x7ffba4136460 auth_method 0 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:54.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.409+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_done global_id 44480 payload 274 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 1 -- 172.21.15.119:0/841351180 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399859397 0 0) 0x7ffba40a1d80 con 0x7ffba4135fd0 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 1 -- 172.21.15.119:0/841351180 >> v2:172.21.15.119:3300/0 conn(0x7ffba4135bf0 msgr2=0x7ffba409fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 1 --2- 172.21.15.119:0/841351180 >> v2:172.21.15.119:3300/0 conn(0x7ffba4135bf0 0x7ffba409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 1 -- 172.21.15.119:0/841351180 >> v1:172.21.15.17:6789/0 conn(0x7ffba4135fd0 legacy=0x7ffba40a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 1 monclient: found mon.noname-b 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:54.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:54.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 1 -- 172.21.15.119:0/841351180 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba40a2340 con 0x7ffba4136460 2024-09-15T21:21:54.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:54.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba2d9d640 1 --2- 172.21.15.119:0/841351180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 0x7ffba40a1160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009e80 tx=0x7ffb9c01c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b405819bdaed10f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:54.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 1 -- 172.21.15.119:0/841351180 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0a5640 con 0x7ffba4136460 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: dump: 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:54.430 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: _finish_auth 0 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 1 -- 172.21.15.119:0/841351180 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c0a57e0 con 0x7ffba4136460 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 1 -- 172.21.15.119:0/841351180 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0a5bb0 con 0x7ffba4136460 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 5 monclient: authenticate success, global_id 44480 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 4 set_mon_vals no callback set 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba1d9b640 10 monclient: dump: 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:54.431 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 10 monclient: shutdown 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 1 -- 172.21.15.119:0/841351180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 msgr2=0x7ffba40a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.413+0000 7ffba89f9640 1 --2- 172.21.15.119:0/841351180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 0x7ffba40a1160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009e80 tx=0x7ffb9c01c980 comp rx=0 tx=0).stop 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- 172.21.15.119:0/841351180 shutdown_connections 2024-09-15T21:21:54.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 --2- 172.21.15.119:0/841351180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4136460 0x7ffba40a1160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 --2- 172.21.15.119:0/841351180 >> v2:172.21.15.119:3300/0 conn(0x7ffba4135bf0 0x7ffba409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- 172.21.15.119:0/841351180 >> 172.21.15.119:0/841351180 conn(0x7ffba40a66e0 msgr2=0x7ffba40a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- 172.21.15.119:0/841351180 shutdown_connections 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- 172.21.15.119:0/841351180 wait complete. 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: build_initial_monmap 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: monmap: 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:54.433 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 Processor -- start 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- start start 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: init 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4135bf0 0x7ffba4134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): picked mon.noname-c con 0x7ffba4135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba4135fd0 0x7ffba413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): picked mon.noname-b con 0x7ffba4135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba4136460 0x7ffba413cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:54.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): picked mon.noname-a con 0x7ffba4136460 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40a1820 con 0x7ffba4136460 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40a16a0 con 0x7ffba4135fd0 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffba409d550 con 0x7ffba4135bf0 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient(hunting): authenticate will time out at 1233.217687s 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9bfff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba4135fd0 msgr2=0x7ffba413ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba4135fd0 0x7ffba413ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4135bf0 0x7ffba4134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba4136460 0x7ffba413cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:54.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4135bf0 0x7ffba4134780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54374/0 (socket says 172.21.15.119:54374) 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 1 -- 172.21.15.119:0/1380023463 learned_addr learned my addr 172.21.15.119:0/1380023463 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): get_auth_request con 0x7ffba4135bf0 auth_method 0 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): get_auth_request con 0x7ffba4136460 auth_method 0 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:54.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:54.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:54.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:54.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:54.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:54.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:54.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:54.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient(hunting): handle_auth_done global_id 54366 payload 824 2024-09-15T21:21:54.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 -- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4135bf0 msgr2=0x7ffba4134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4135bf0 0x7ffba4134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 -- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba4135fd0 msgr2=0x7ffba413ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba4135fd0 0x7ffba413ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 monclient: found mon.noname-a 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 -- 172.21.15.119:0/1380023463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4139540 con 0x7ffba4136460 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba2d9d640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba4136460 0x7ffba413cfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0ab000 tx=0x7ffb9c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0bc070 con 0x7ffba4136460 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: dump: 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 5 monclient: authenticate success, global_id 54366 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: _renew_subs 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba409d240 con 0x7ffba4136460 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: _renew_subs 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba4138ca0 con 0x7ffba4136460 2024-09-15T21:21:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c0b8640 con 0x7ffba4136460 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0b89f0 con 0x7ffba4136460 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffb9affd640 10 monclient: dump: 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:54.445 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.417+0000 7ffba159a640 4 set_mon_vals no callback set 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba89f9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba89f9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba89f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb88005090 con 0x7ffba4136460 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffb9c0b8b90 con 0x7ffba4136460 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffb9affd640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb8403b970 0x7ffb8403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8233+0+0 (secure 0 0 0) 0x7ffb9c0ebaf0 con 0x7ffba4136460 2024-09-15T21:21:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba259c640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb8403b970 0x7ffb8403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba259c640 10 monclient: get_auth_request con 0x7ffb8403b970 auth_method 0 2024-09-15T21:21:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.421+0000 7ffba259c640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb8403b970 0x7ffb8403de30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb94003fb0 tx=0x7ffb940091c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.433+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffb9c016600 con 0x7ffba4136460 2024-09-15T21:21:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.433+0000 7ffb9affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.433+0000 7ffb9affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:54.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.609+0000 7ffba89f9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:54.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.609+0000 7ffba89f9640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:54.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.609+0000 7ffba89f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.609+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7ffb88005380 con 0x7ffba4136460 2024-09-15T21:21:54.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.837+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v126) ==== 129+0+0 (secure 0 0 0) 0x7ffb9c0c93e0 con 0x7ffba4136460 2024-09-15T21:21:54.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.841+0000 7ffb9affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:54.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.841+0000 7ffb9affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:54.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.949+0000 7ffba89f9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:54.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.949+0000 7ffba89f9640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:54.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.949+0000 7ffba89f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:54.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:54.949+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7ffb88005ac0 con 0x7ffba4136460 2024-09-15T21:21:55.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.844+0000 7ffb9affd640 1 -- 172.21.15.119:0/1380023463 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v127) ==== 129+0+0 (secure 0 0 0) 0x7ffb9c0ca430 con 0x7ffba4136460 2024-09-15T21:21:55.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.844+0000 7ffb9affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:55.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.844+0000 7ffb9affd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb8403b970 msgr2=0x7ffb8403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb8403b970 0x7ffb8403de30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb94003fb0 tx=0x7ffb940091c0 comp rx=0 tx=0).stop 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 10 monclient: shutdown 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba4136460 msgr2=0x7ffba413cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba4136460 0x7ffba413cfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0ab000 tx=0x7ffb9c019720 comp rx=0 tx=0).stop 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 shutdown_connections 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb8403b970 0x7ffb8403de30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba4136460 0x7ffba413cfd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:55.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba4135fd0 0x7ffba413ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:55.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 --2- 172.21.15.119:0/1380023463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffba4135bf0 0x7ffba4134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:55.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 >> 172.21.15.119:0/1380023463 conn(0x7ffba40a66e0 msgr2=0x7ffba40a41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:55.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 shutdown_connections 2024-09-15T21:21:55.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.848+0000 7ffba89f9640 1 -- 172.21.15.119:0/1380023463 wait complete. 2024-09-15T21:21:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-09-15T21:21:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:21:55.879 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.996+0000 7f18b02f6640 10 monclient: get_monmap_and_config 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.996+0000 7f18b02f6640 10 monclient: build_initial_monmap 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.996+0000 7f18b02f6640 10 monclient: monmap: 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:56.003715+0000 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:56.003715+0000 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.005 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:56.006 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:56.006 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:56.006 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:56.006 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:56.006 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.996+0000 7f18b02f6640 1 Processor -- start 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.996+0000 7f18b02f6640 1 -- start start 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:55.996+0000 7f18b02f6640 10 monclient: init 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18a8135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:56.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 0x7f18a813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient(hunting): picked mon.noname-b con 0x7f18a809fe70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f18a813afb0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a80a0300 con 0x7f18a8135bf0 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a80a1d80 con 0x7f18a813afb0 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f18a80a3600 con 0x7f18a809fe70 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 10 monclient(hunting): authenticate will time out at 1234.800364s 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ae893640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f18a813afb0 legacy=0x7f18a813d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 0x7f18a813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 0x7f18a813aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54382/0 (socket says 172.21.15.119:54382) 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 -- 172.21.15.119:0/3784592219 learned_addr learned my addr 172.21.15.119:0/3784592219 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:56.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): get_auth_request con 0x7f18a809fe70 auth_method 0 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ae092640 1 --1- 172.21.15.119:0/3784592219 >> v1:172.21.15.17:6789/0 conn(0x7f18a8135bf0 0x7f18a8135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:52028/0 (socket says 172.21.15.119:52028) 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient(hunting): handle_auth_done global_id 44486 payload 274 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 -- 172.21.15.119:0/3784592219 >> v1:172.21.15.119:6789/0 conn(0x7f18a813afb0 legacy=0x7f18a813d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 -- 172.21.15.119:0/3784592219 >> v1:172.21.15.17:6789/0 conn(0x7f18a8135bf0 legacy=0x7f18a8135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 monclient: found mon.noname-b 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 1 -- 172.21.15.119:0/3784592219 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368585861 0 0) 0x7f18a4001120 con 0x7f18a8135bf0 2024-09-15T21:21:56.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 -- 172.21.15.119:0/3784592219 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a80a2060 con 0x7f18a809fe70 2024-09-15T21:21:56.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad891640 1 --2- 172.21.15.119:0/3784592219 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 0x7f18a813aa70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098a0 tx=0x7f189c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1504e27f94017d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 1 -- 172.21.15.119:0/3784592219 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0ab5f0 con 0x7f18a809fe70 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: dump: 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.018 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: _finish_auth 0 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: _check_auth_tickets 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 5 monclient: authenticate success, global_id 44486 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 1 -- 172.21.15.119:0/3784592219 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c0b0070 con 0x7f18a809fe70 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18b02f6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 1 -- 172.21.15.119:0/3784592219 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0ab9b0 con 0x7f18a809fe70 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.000+0000 7f18ad090640 10 monclient: dump: 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.020 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 4 set_mon_vals no callback set 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: shutdown 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- 172.21.15.119:0/3784592219 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 msgr2=0x7f18a813aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 --2- 172.21.15.119:0/3784592219 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 0x7f18a813aa70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098a0 tx=0x7f189c01c410 comp rx=0 tx=0).stop 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- 172.21.15.119:0/3784592219 shutdown_connections 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 --2- 172.21.15.119:0/3784592219 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a809fe70 0x7f18a813aa70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- 172.21.15.119:0/3784592219 >> 172.21.15.119:0/3784592219 conn(0x7f18a80a66e0 msgr2=0x7f18a80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- 172.21.15.119:0/3784592219 shutdown_connections 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- 172.21.15.119:0/3784592219 wait complete. 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: build_initial_monmap 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: monmap: 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.022 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 Processor -- start 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- start start 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: init 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f18a81cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): picked mon.noname-a con 0x7f18a809fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 0x7f18a8134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): picked mon.noname-b con 0x7f18a8135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:56.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): picked mon.noname-c con 0x7f18a813afb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f18a806ede0 con 0x7f18a809fe70 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18a806ec60 con 0x7f18a8135bf0 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f18a81416f0 con 0x7f18a813afb0 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18b02f6640 10 monclient(hunting): authenticate will time out at 1234.806299s 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ad891640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 msgr2=0x7f18a8134bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ad891640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 0x7f18a8134bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f18a81cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54398/0 (socket says 172.21.15.119:54398) 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 learned_addr learned my addr 172.21.15.119:0/1605232838 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): get_auth_request con 0x7f18a813afb0 auth_method 0 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): get_auth_request con 0x7f18a809fe70 auth_method 0 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient(hunting): handle_auth_done global_id 44489 payload 824 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 msgr2=0x7f18a8134bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 0x7f18a8134bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 msgr2=0x7f18a81cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f18a81cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 monclient: found mon.noname-c 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a809d240 con 0x7f18a813afb0 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae092640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f18a81cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18a000be60 tx=0x7f18a000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 msgr2=0x7f18a809d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 msgr2=0x7f18a809d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18a000be60 tx=0x7f18a000bf50 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18a000be60 tx=0x7f18a000bf50 comp rx=0 tx=0).stop 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient: ms_handle_reset current mon [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 msgr2=0x7f18a809d580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:56.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f18a813afb0 0x7f18a809d580 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188c0034e0 0x7f188c005930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): picked mon.noname-c con 0x7f188c0034e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f188c008410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): picked mon.noname-a con 0x7f18a809fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 0x7f188c00af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): picked mon.noname-b con 0x7f18a8135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f188c00b960 con 0x7f18a809fe70 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f188c00bae0 con 0x7f18a8135bf0 2024-09-15T21:21:56.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f188c00bce0 con 0x7f188c0034e0 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f189affd640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 msgr2=0x7f188c00af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.004+0000 7f18ae893640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 0x7f188c00af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188c0034e0 0x7f188c005930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): get_auth_request con 0x7f188c0034e0 auth_method 0 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f188c008410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): get_auth_request con 0x7f18a809fe70 auth_method 0 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient(hunting): handle_auth_done global_id 44489 payload 293 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 msgr2=0x7f188c00af00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a8135bf0 0x7f188c00af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 -- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 msgr2=0x7f188c008410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18a809fe70 0x7f188c008410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 monclient: found mon.noname-c 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c00bf40 con 0x7f188c0034e0 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ae092640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188c0034e0 0x7f188c005930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000f70 tx=0x7f18a0009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4001a40 con 0x7f188c0034e0 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: dump: 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.031 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: _finish_auth 0 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a4018070 con 0x7f188c0034e0 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 5 monclient: authenticate success, global_id 44489 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 monclient: _renew_subs 2024-09-15T21:21:56.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a8072810 con 0x7f188c0034e0 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4007500 con 0x7f188c0034e0 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189bfff640 4 set_mon_vals no callback set 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 10 monclient: dump: 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.033 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 monclient: _renew_subs 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a80a54b0 con 0x7f188c0034e0 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18b02f6640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a813ab60 con 0x7f188c0034e0 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f18a401d460 con 0x7f188c0034e0 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f18a813afb0 0x7f188c0460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(127..127 src has 1..127) ==== 8233+0+0 (secure 0 0 0) 0x7f18a40504f0 con 0x7f188c0034e0 2024-09-15T21:21:56.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f18a813afb0 0x7f188c0460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 10 monclient: get_auth_request con 0x7f18a813afb0 auth_method 0 2024-09-15T21:21:56.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.008+0000 7f18ad891640 1 --2- 172.21.15.119:0/1605232838 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f18a813afb0 0x7f188c0460c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f189c0af000 tx=0x7f189c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.020+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f18a80a0300 con 0x7f188c0034e0 2024-09-15T21:21:56.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.020+0000 7f189affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.020+0000 7f189affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:56.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.184+0000 7f18b02f6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-09-15T21:21:56.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.184+0000 7f18b02f6640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-09-15T21:21:56.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.184+0000 7f18b02f6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.184+0000 7f18b02f6640 1 -- 172.21.15.119:0/1605232838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]} v 0) -- 0x7f18a8135fd0 con 0x7f188c0034e0 2024-09-15T21:21:56.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.188+0000 7f189affd640 1 -- 172.21.15.119:0/1605232838 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}]=-22 must specify exact 1 weight values v127) ==== 168+0+0 (secure 0 0 0) 0x7f18a40076a0 con 0x7f188c0034e0 2024-09-15T21:21:56.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.188+0000 7f189affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-09-15T21:21:56.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.188+0000 7f189affd640 10 monclient: _finish_command 2 = mon:22 must specify exact 1 weight values 2024-09-15T21:21:56.195 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: must specify exact 1 weight values 2024-09-15T21:21:56.202 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:21:56.202 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_false ceph osd crush weight-set reweight barset osd.0 .9 2024-09-15T21:21:56.202 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -x 2024-09-15T21:21:56.202 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 2024-09-15T21:21:56.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient: get_monmap_and_config 2024-09-15T21:21:56.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient: build_initial_monmap 2024-09-15T21:21:56.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient: monmap: 2024-09-15T21:21:56.311 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:56.311 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:56.310215+0000 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:56.310215+0000 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:56.312 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 1 Processor -- start 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 1 -- start start 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient: init 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9de40a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9de40a41b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:56.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient(hunting): picked mon.noname-b con 0x7f9de40a45c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de41383e0 con 0x7f9de40a41b0 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de40a19d0 con 0x7f9de40a7ac0 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9de40a7ea0 con 0x7f9de40a45c0 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dec3ad640 10 monclient(hunting): authenticate will time out at 1235.106780s 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea149640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f9de40a7ac0 legacy=0x7f9de4137cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:56.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54412/0 (socket says 172.21.15.119:54412) 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 learned_addr learned my addr 172.21.15.119:0/3119395703 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): get_auth_request con 0x7f9de40a45c0 auth_method 0 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9de9948640 1 --1- 172.21.15.119:0/3119395703 >> v1:172.21.15.17:6789/0 conn(0x7f9de40a41b0 0x7f9de413cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:52030/0 (socket says 172.21.15.119:52030) 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient(hunting): handle_auth_done global_id 44492 payload 274 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 >> v1:172.21.15.119:6789/0 conn(0x7f9de40a7ac0 legacy=0x7f9de4137cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:56.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 >> v1:172.21.15.17:6789/0 conn(0x7f9de40a41b0 legacy=0x7f9de413cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 monclient: found mon.noname-b 2024-09-15T21:21:56.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.317 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.304+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de40a6960 con 0x7f9de40a45c0 2024-09-15T21:21:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea94a640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0098a0 tx=0x7f9ddc01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1797147d21507fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 msgr2=0x7f9de41402b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T21:21:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 msgr2=0x7f9de41402b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea94a640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0098a0 tx=0x7f9ddc01c440 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea94a640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0098a0 tx=0x7f9ddc01c440 comp rx=0 tx=0).stop 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient: ms_handle_reset current mon [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 -- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 msgr2=0x7f9de41402b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de40a45c0 0x7f9de41402b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient: _add_conns ranks=[4,3] 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 --2- 172.21.15.119:0/3119395703 >> v2:172.21.15.119:3300/0 conn(0x7f9dd0003510 0x7f9dd0005990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient(hunting): picked mon.noname-c con 0x7f9dd0003510 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 --2- 172.21.15.119:0/3119395703 >> v2:172.21.15.17:3300/0 conn(0x7f9dd00062c0 0x7f9dd0008780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient(hunting): picked mon.noname-a con 0x7f9dd00062c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 -- 172.21.15.119:0/3119395703 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd0009050 con 0x7f9dd00062c0 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 -- 172.21.15.119:0/3119395703 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd0009200 con 0x7f9dd0003510 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea149640 1 -- 172.21.15.119:0/3119395703 >> v2:172.21.15.119:3300/0 conn(0x7f9dd0003510 msgr2=0x7f9dd0005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9dea149640 1 --2- 172.21.15.119:0/3119395703 >> v2:172.21.15.119:3300/0 conn(0x7f9dd0003510 0x7f9dd0005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 1 --2- 172.21.15.119:0/3119395703 >> v2:172.21.15.17:3300/0 conn(0x7f9dd00062c0 0x7f9dd0008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): get_auth_request con 0x7f9dd00062c0 auth_method 0 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient(hunting): handle_auth_done global_id 44492 payload 293 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 1 -- 172.21.15.119:0/3119395703 >> v2:172.21.15.119:3300/0 conn(0x7f9dd0003510 msgr2=0x7f9dd0005990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 1 --2- 172.21.15.119:0/3119395703 >> v2:172.21.15.119:3300/0 conn(0x7f9dd0003510 0x7f9dd0005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 1 monclient: found mon.noname-a 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 1 -- 172.21.15.119:0/3119395703 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd0009470 con 0x7f9dd00062c0 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9948640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9dd0008780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00034a0 tx=0x7f9ddc009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcac41002f10da5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 1 -- 172.21.15.119:0/3119395703 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40013f0 con 0x7f9dd00062c0 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.308+0000 7f9de9147640 10 monclient: dump: 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 10 monclient: _finish_auth 0 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 10 monclient: _check_auth_tickets 2024-09-15T21:21:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 5 monclient: authenticate success, global_id 44492 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 1 -- 172.21.15.119:0/3119395703 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd40072f0 con 0x7f9dd00062c0 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 1 -- 172.21.15.119:0/3119395703 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40047f0 con 0x7f9dd00062c0 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 4 set_mon_vals no callback set 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9147640 10 monclient: dump: 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: shutdown 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 msgr2=0x7f9dd0008780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9dd0008780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00034a0 tx=0x7f9ddc009240 comp rx=0 tx=0).stop 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 -- 172.21.15.119:0/3119395703 reap_dead start 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/3119395703 shutdown_connections 2024-09-15T21:21:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 --2- 172.21.15.119:0/3119395703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9dd0008780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/3119395703 >> 172.21.15.119:0/3119395703 conn(0x7f9de40a6fd0 msgr2=0x7f9de40a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/3119395703 shutdown_connections 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/3119395703 wait complete. 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: build_initial_monmap 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: monmap: 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:56.331 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 Processor -- start 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- start start 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: init 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9de41312d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): picked mon.noname-a con 0x7f9dd00062c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9de4131810 0x7f9de4072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): picked mon.noname-b con 0x7f9de4131810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de4072540 0x7f9de4072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): picked mon.noname-c con 0x7f9de4072540 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41436b0 con 0x7f9dd00062c0 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9de4143530 con 0x7f9de4131810 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9de4143830 con 0x7f9de4072540 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient(hunting): authenticate will time out at 1235.114038s 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9948640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9de4131810 msgr2=0x7f9de4072000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9de9948640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9de4131810 0x7f9de4072000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9de41312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de4072540 0x7f9de4072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9de41312d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47418/0 (socket says 172.21.15.119:47418) 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 1 -- 172.21.15.119:0/2521735502 learned_addr learned my addr 172.21.15.119:0/2521735502 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:56.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): get_auth_request con 0x7f9dd00062c0 auth_method 0 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): get_auth_request con 0x7f9de4072540 auth_method 0 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient(hunting): handle_auth_done global_id 44495 payload 824 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 -- 172.21.15.119:0/2521735502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9de4131810 msgr2=0x7f9de4072000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9de4131810 0x7f9de4072000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 -- 172.21.15.119:0/2521735502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 msgr2=0x7f9de41312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9de41312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 monclient: found mon.noname-c 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 -- 172.21.15.119:0/2521735502 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de413a050 con 0x7f9de4072540 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea149640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dd00062c0 0x7f9de41312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dea94a640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9de4072540 0x7f9de4072960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0ab000 tx=0x7f9ddc0191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0b0070 con 0x7f9de4072540 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: dump: 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: _finish_auth 0 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 5 monclient: authenticate success, global_id 44495 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: _renew_subs 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc003b60 con 0x7f9de4072540 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/2521735502 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de40a16c0 con 0x7f9de4072540 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0b4410 con 0x7f9de4072540 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddaffd640 10 monclient: dump: 2024-09-15T21:21:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9ddbfff640 4 set_mon_vals no callback set 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: _renew_subs 2024-09-15T21:21:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.312+0000 7f9dec3ad640 1 -- 172.21.15.119:0/2521735502 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de4139d70 con 0x7f9de4072540 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dec3ad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dec3ad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dec3ad640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dec3ad640 1 -- 172.21.15.119:0/2521735502 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de4137cd0 con 0x7f9de4072540 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9ddc0bd020 con 0x7f9de4072540 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9ddaffd640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dc803b970 0x7f9dc803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(127..127 src has 1..127) ==== 8233+0+0 (secure 0 0 0) 0x7f9ddc0ea9c0 con 0x7f9de4072540 2024-09-15T21:21:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dea149640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dc803b970 0x7f9dc803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dea149640 10 monclient: get_auth_request con 0x7f9dc803b970 auth_method 0 2024-09-15T21:21:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.316+0000 7f9dea149640 1 --2- 172.21.15.119:0/2521735502 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dc803b970 0x7f9dc803de30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9de0007810 tx=0x7f9de0009740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.328+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9ddc0c4020 con 0x7f9de4072540 2024-09-15T21:21:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.328+0000 7f9ddaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.328+0000 7f9ddaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:56.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.504+0000 7f9dec3ad640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:56.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.504+0000 7f9dec3ad640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:56.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.504+0000 7f9dec3ad640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:56.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.504+0000 7f9dec3ad640 1 -- 172.21.15.119:0/2521735502 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7f9de4138b30 con 0x7f9de4072540 2024-09-15T21:21:56.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.508+0000 7f9ddaffd640 1 -- 172.21.15.119:0/2521735502 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}]=-22 must specify exact 3 weight values v127) ==== 163+0+0 (secure 0 0 0) 0x7f9ddc016150 con 0x7f9de4072540 2024-09-15T21:21:56.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.508+0000 7f9ddaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T21:21:56.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.508+0000 7f9ddaffd640 10 monclient: _finish_command 2 = mon:22 must specify exact 3 weight values 2024-09-15T21:21:56.515 INFO:tasks.workunit.client.0.smithi119.stderr:Error EINVAL: must specify exact 3 weight values 2024-09-15T21:21:56.521 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:21:56.522 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 .9 .9 2024-09-15T21:21:56.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient: get_monmap_and_config 2024-09-15T21:21:56.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient: build_initial_monmap 2024-09-15T21:21:56.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient: monmap: 2024-09-15T21:21:56.646 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:56.646 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.646 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:56.644931+0000 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:56.644931+0000 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:56.647 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 Processor -- start 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 -- start start 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient: init 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5730135c80 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5730071770 0x7f5730072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): picked mon.noname-c con 0x7f5730071770 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f573013ab70 0x7f573013cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): picked mon.noname-a con 0x7f573013ab70 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5730136770 con 0x7f5730135c80 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f57300a4730 con 0x7f573013ab70 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5730073500 con 0x7f5730071770 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f5736dcd640 10 monclient(hunting): authenticate will time out at 1235.441669s 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f572ffff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f5730071770 msgr2=0x7f5730072fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f572ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5730071770 0x7f5730072fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f573013ab70 0x7f573013cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f573013ab70 0x7f573013cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47434/0 (socket says 172.21.15.119:47434) 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 -- 172.21.15.119:0/2723054326 learned_addr learned my addr 172.21.15.119:0/2723054326 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): get_auth_request con 0x7f573013ab70 auth_method 0 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient(hunting): handle_auth_done global_id 54381 payload 274 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 -- 172.21.15.119:0/2723054326 >> v2:172.21.15.119:3300/0 conn(0x7f5730071770 msgr2=0x7f5730072fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:56.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 --2- 172.21.15.119:0/2723054326 >> v2:172.21.15.119:3300/0 conn(0x7f5730071770 0x7f5730072fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 -- 172.21.15.119:0/2723054326 >> v1:172.21.15.17:6789/0 conn(0x7f5730135c80 legacy=0x7f5730136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 monclient: found mon.noname-a 2024-09-15T21:21:56.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.640+0000 7f573536a640 1 -- 172.21.15.119:0/2723054326 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573013e4b0 con 0x7f573013ab70 2024-09-15T21:21:56.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f573536a640 1 --2- 172.21.15.119:0/2723054326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f573013ab70 0x7f573013cf60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5730074080 tx=0x7f572801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f796a5b3aae776 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 1 -- 172.21.15.119:0/2723054326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280a5620 con 0x7f573013ab70 2024-09-15T21:21:56.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: dump: 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.660 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 1 -- 172.21.15.119:0/2723054326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57280a57c0 con 0x7f573013ab70 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 5 monclient: authenticate success, global_id 54381 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 4 set_mon_vals no callback set 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 1 -- 172.21.15.119:0/2723054326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280a5b90 con 0x7f573013ab70 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:56.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f572f7fe640 10 monclient: dump: 2024-09-15T21:21:56.662 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.662 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.662 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.662 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.662 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.662 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: shutdown 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:56.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- 172.21.15.119:0/2723054326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f573013ab70 msgr2=0x7f573013cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2723054326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f573013ab70 0x7f573013cf60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5730074080 tx=0x7f572801c960 comp rx=0 tx=0).stop 2024-09-15T21:21:56.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- 172.21.15.119:0/2723054326 shutdown_connections 2024-09-15T21:21:56.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2723054326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f573013ab70 0x7f573013cf60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2723054326 >> v2:172.21.15.119:3300/0 conn(0x7f5730071770 0x7f5730072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- 172.21.15.119:0/2723054326 >> 172.21.15.119:0/2723054326 conn(0x7f573006d750 msgr2=0x7f57300a8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- 172.21.15.119:0/2723054326 shutdown_connections 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- 172.21.15.119:0/2723054326 wait complete. 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: build_initial_monmap 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: monmap: 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:56.665 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:56.666 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:56.666 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:56.666 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.666 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:56.666 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 Processor -- start 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- start start 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: init 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:56.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:56.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730071770 0x7f5730134f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): picked mon.noname-b con 0x7f5730071770 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:56.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 0x7f5730135460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): picked mon.noname-a con 0x7f5730135c80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:56.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f573013ab70 0x7f5730136940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): picked mon.noname-c con 0x7f573013ab70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:56.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f573013dcb0 con 0x7f5730135c80 2024-09-15T21:21:56.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57300a4900 con 0x7f5730071770 2024-09-15T21:21:56.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:56.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f573013de30 con 0x7f573013ab70 2024-09-15T21:21:56.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:56.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5736dcd640 10 monclient(hunting): authenticate will time out at 1235.447712s 2024-09-15T21:21:56.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5734b69640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730071770 msgr2=0x7f5730134f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:56.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.644+0000 7f5734b69640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730071770 0x7f5730134f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:56.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 0x7f5730135460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 0x7f5730135460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47442/0 (socket says 172.21.15.119:47442) 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 -- 172.21.15.119:0/2281855549 learned_addr learned my addr 172.21.15.119:0/2281855549 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): get_auth_request con 0x7f5730135c80 auth_method 0 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f573013ab70 0x7f5730136940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): get_auth_request con 0x7f573013ab70 auth_method 0 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:56.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f573536a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient(hunting): handle_auth_done global_id 54384 payload 824 2024-09-15T21:21:56.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 -- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f573013ab70 msgr2=0x7f5730136940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f573013ab70 0x7f5730136940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 -- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730071770 msgr2=0x7f5730134f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730071770 0x7f5730134f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 monclient: found mon.noname-a 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 -- 172.21.15.119:0/2281855549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730138c40 con 0x7f5730135c80 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572ffff640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 0x7f5730135460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f572400b560 tx=0x7f572400ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724013020 con 0x7f5730135c80 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: dump: 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 5 monclient: authenticate success, global_id 54384 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 monclient: _renew_subs 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573013d210 con 0x7f5730135c80 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724003df0 con 0x7f5730135c80 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240052b0 con 0x7f5730135c80 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 10 monclient: dump: 2024-09-15T21:21:56.675 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 monclient: _renew_subs 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5730139200 con 0x7f5730135c80 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572effd640 4 set_mon_vals no callback set 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573009ec80 con 0x7f5730135c80 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5724016020 con 0x7f5730135c80 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f571c03b9d0 0x7f571c03de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8233+0+0 (secure 0 0 0) 0x7f5724050c10 con 0x7f5730135c80 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5734b69640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f571c03b9d0 0x7f571c03de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:56.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5734b69640 10 monclient: get_auth_request con 0x7f571c03b9d0 auth_method 0 2024-09-15T21:21:56.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.648+0000 7f5734b69640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f571c03b9d0 0x7f571c03de90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5720000bd0 tx=0x7f5720007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:56.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.660+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f572401d020 con 0x7f5730135c80 2024-09-15T21:21:56.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.660+0000 7f572d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:56.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.660+0000 7f572d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:56.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.836+0000 7f5736dcd640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T21:21:56.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.836+0000 7f5736dcd640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T21:21:56.846 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.836+0000 7f5736dcd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.847 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.836+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) -- 0x7f573013e1f0 con 0x7f5730135c80 2024-09-15T21:21:56.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.848+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v128) ==== 139+0+0 (secure 0 0 0) 0x7f572400fdc0 con 0x7f5730135c80 2024-09-15T21:21:56.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.848+0000 7f572d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T21:21:56.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.848+0000 7f572d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:56.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.956+0000 7f5736dcd640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T21:21:56.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.956+0000 7f5736dcd640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T21:21:56.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.956+0000 7f5736dcd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:56.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:56.956+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) -- 0x7f573006a810 con 0x7f5730135c80 2024-09-15T21:21:57.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.864+0000 7f572d7fa640 1 -- 172.21.15.119:0/2281855549 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v129) ==== 139+0+0 (secure 0 0 0) 0x7f572404d890 con 0x7f5730135c80 2024-09-15T21:21:57.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.864+0000 7f572d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T21:21:57.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.864+0000 7f572d7fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f571c03b9d0 msgr2=0x7f571c03de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f571c03b9d0 0x7f571c03de90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5720000bd0 tx=0x7f5720007a00 comp rx=0 tx=0).stop 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 10 monclient: shutdown 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 msgr2=0x7f5730135460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 0x7f5730135460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f572400b560 tx=0x7f572400ba20 comp rx=0 tx=0).stop 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 shutdown_connections 2024-09-15T21:21:57.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f571c03b9d0 0x7f571c03de90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:57.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f573013ab70 0x7f5730136940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:57.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5730135c80 0x7f5730135460 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:57.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 --2- 172.21.15.119:0/2281855549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730071770 0x7f5730134f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:57.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 >> 172.21.15.119:0/2281855549 conn(0x7f573006d750 msgr2=0x7f573013bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:57.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 shutdown_connections 2024-09-15T21:21:57.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.868+0000 7f5736dcd640 1 -- 172.21.15.119:0/2281855549 wait complete. 2024-09-15T21:21:57.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:21:57.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c fooset 2024-09-15T21:21:57.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 1 2024-09-15T21:21:58.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.996+0000 7fad0b734640 10 monclient: get_monmap_and_config 2024-09-15T21:21:58.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.996+0000 7fad0b734640 10 monclient: build_initial_monmap 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.996+0000 7fad0b734640 10 monclient: monmap: 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:58.003281+0000 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:58.003281+0000 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:58.005 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:58.006 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.996+0000 7fad0b734640 1 Processor -- start 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.996+0000 7fad0b734640 1 -- start start 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:57.996+0000 7fad0b734640 10 monclient: init 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fad040a4ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad040a5790 0x7fad0413ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient(hunting): picked mon.noname-a con 0x7fad040a5790 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fad0413d3f0 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040a23d0 con 0x7fad040a4ca0 2024-09-15T21:21:58.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040a20f0 con 0x7fad0413d3f0 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fad040740f0 con 0x7fad040a5790 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad0b734640 10 monclient(hunting): authenticate will time out at 1236.800658s 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad09cd1640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fad0413d3f0 legacy=0x7fad0413f7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad040a5790 0x7fad0413ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad040a5790 0x7fad0413ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47444/0 (socket says 172.21.15.119:47444) 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 -- 172.21.15.119:0/1228019276 learned_addr learned my addr 172.21.15.119:0/1228019276 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): get_auth_request con 0x7fad040a5790 auth_method 0 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient(hunting): handle_auth_done global_id 54387 payload 274 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 -- 172.21.15.119:0/1228019276 >> v1:172.21.15.119:6789/0 conn(0x7fad0413d3f0 legacy=0x7fad0413f7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 -- 172.21.15.119:0/1228019276 >> v1:172.21.15.17:6789/0 conn(0x7fad040a4ca0 legacy=0x7fad040a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient: _finish_hunting 0 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 monclient: found mon.noname-a 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 -- 172.21.15.119:0/1228019276 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040a5d20 con 0x7fad040a5790 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.000+0000 7fad08ccf640 1 --2- 172.21.15.119:0/1228019276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad0413ceb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7facf4009870 tx=0x7facf401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b4d8619b00c9a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 1 -- 172.21.15.119:0/1228019276 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40ab430 con 0x7fad040a5790 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: dump: 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.016 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.017 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.018 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: _finish_auth 0 2024-09-15T21:21:58.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:58.018 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:58.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 5 monclient: authenticate success, global_id 54387 2024-09-15T21:21:58.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:58.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 1 -- 172.21.15.119:0/1228019276 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf40b0070 con 0x7fad040a5790 2024-09-15T21:21:58.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 4 set_mon_vals no callback set 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 1 -- 172.21.15.119:0/1228019276 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40ab7f0 con 0x7fad040a5790 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facfbfff640 10 monclient: dump: 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: shutdown 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- 172.21.15.119:0/1228019276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 msgr2=0x7fad0413ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 --2- 172.21.15.119:0/1228019276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad0413ceb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7facf4009870 tx=0x7facf401c520 comp rx=0 tx=0).stop 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- 172.21.15.119:0/1228019276 shutdown_connections 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 --2- 172.21.15.119:0/1228019276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad0413ceb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- 172.21.15.119:0/1228019276 >> 172.21.15.119:0/1228019276 conn(0x7fad040a7400 msgr2=0x7fad0406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- 172.21.15.119:0/1228019276 shutdown_connections 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- 172.21.15.119:0/1228019276 wait complete. 2024-09-15T21:21:58.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: build_initial_monmap 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: monmap: 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 Processor -- start 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- start start 2024-09-15T21:21:58.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: init 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040a4ca0 0x7fad041cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): picked mon.noname-b con 0x7fad040a4ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad04137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): picked mon.noname-a con 0x7fad040a5790 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad0413d3f0 0x7fad04070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): picked mon.noname-c con 0x7fad0413d3f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad0406dd20 con 0x7fad040a5790 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad040a5fa0 con 0x7fad040a4ca0 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad04143720 con 0x7fad0413d3f0 2024-09-15T21:21:58.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:58.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad0b734640 10 monclient(hunting): authenticate will time out at 1236.806184s 2024-09-15T21:21:58.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad094d0640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040a4ca0 msgr2=0x7fad041cf7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:58.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad094d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040a4ca0 0x7fad041cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:58.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad04137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad0413d3f0 0x7fad04070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad04137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47450/0 (socket says 172.21.15.119:47450) 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 -- 172.21.15.119:0/3085666572 learned_addr learned my addr 172.21.15.119:0/3085666572 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): get_auth_request con 0x7fad040a5790 auth_method 0 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): get_auth_request con 0x7fad0413d3f0 auth_method 0 2024-09-15T21:21:58.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad09cd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient(hunting): handle_auth_done global_id 54390 payload 824 2024-09-15T21:21:58.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 -- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad0413d3f0 msgr2=0x7fad04070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad0413d3f0 0x7fad04070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 -- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040a4ca0 msgr2=0x7fad041cf7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040a4ca0 0x7fad041cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient: _finish_hunting 0 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 monclient: found mon.noname-a 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:58.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 -- 172.21.15.119:0/3085666572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad041215c0 con 0x7fad040a5790 2024-09-15T21:21:58.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7fad08ccf640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad04137c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facf40ad040 tx=0x7facf40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:58.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40abe60 con 0x7fad040a5790 2024-09-15T21:21:58.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facf9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facf9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:58.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facf9ffb640 10 monclient: dump: 2024-09-15T21:21:58.029 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.029 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.029 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.029 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.029 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.029 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facf9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.004+0000 7facf9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:21:58.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:58.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf40b4600 con 0x7fad040a5790 2024-09-15T21:21:58.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:58.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40b49d0 con 0x7fad040a5790 2024-09-15T21:21:58.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 5 monclient: authenticate success, global_id 54390 2024-09-15T21:21:58.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 monclient: _renew_subs 2024-09-15T21:21:58.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad040a1500 con 0x7fad040a5790 2024-09-15T21:21:58.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facfb7fe640 4 set_mon_vals no callback set 2024-09-15T21:21:58.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:58.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 10 monclient: dump: 2024-09-15T21:21:58.033 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.033 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.033 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.033 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.033 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.033 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.034 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.034 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.034 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.034 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 monclient: _renew_subs 2024-09-15T21:21:58.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad040726b0 con 0x7fad040a5790 2024-09-15T21:21:58.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:58.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:58.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad040a5080 con 0x7fad040a5790 2024-09-15T21:21:58.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7facf40c3060 con 0x7fad040a5790 2024-09-15T21:21:58.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b9a0 0x7face803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8233+0+0 (secure 0 0 0) 0x7facf40eb340 con 0x7fad040a5790 2024-09-15T21:21:58.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad094d0640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b9a0 0x7face803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad094d0640 10 monclient: get_auth_request con 0x7face803b9a0 auth_method 0 2024-09-15T21:21:58.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.008+0000 7fad094d0640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b9a0 0x7face803de60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fad00000c10 tx=0x7fad00007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:58.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.020+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7facf4016110 con 0x7fad040a5790 2024-09-15T21:21:58.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.020+0000 7facf9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:58.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.020+0000 7facf9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:58.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.196+0000 7fad0b734640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:58.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.196+0000 7fad0b734640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:58.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.196+0000 7fad0b734640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.196+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fad0413dc60 con 0x7fad040a5790 2024-09-15T21:21:58.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.200+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v129) ==== 71+0+14 (secure 0 0 0) 0x7facf40c3350 con 0x7fad040a5790 2024-09-15T21:21:58.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.200+0000 7facf9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:58.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.200+0000 7facf9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:58.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:58.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:58.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fad04121240 con 0x7fad040a5790 2024-09-15T21:21:58.311 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7facf9ffb640 1 -- 172.21.15.119:0/3085666572 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v129) ==== 71+0+14 (secure 0 0 0) 0x7facf40b3d90 con 0x7fad040a5790 2024-09-15T21:21:58.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7facf9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:58.312 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7facf9ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b9a0 msgr2=0x7face803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b9a0 0x7face803de60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fad00000c10 tx=0x7fad00007a00 comp rx=0 tx=0).stop 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 10 monclient: shutdown 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 msgr2=0x7fad04137c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.304+0000 7fad0b734640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad04137c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facf40ad040 tx=0x7facf40192c0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 shutdown_connections 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7face803b9a0 0x7face803de60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad0413d3f0 0x7fad04070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad040a5790 0x7fad04137c40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 --2- 172.21.15.119:0/3085666572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040a4ca0 0x7fad041cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 >> 172.21.15.119:0/3085666572 conn(0x7fad040a7400 msgr2=0x7fad04140170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:58.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 shutdown_connections 2024-09-15T21:21:58.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.308+0000 7fad0b734640 1 -- 172.21.15.119:0/3085666572 wait complete. 2024-09-15T21:21:58.329 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set rm fooset 2024-09-15T21:21:58.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.440+0000 7fa251de8640 10 monclient: get_monmap_and_config 2024-09-15T21:21:58.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.440+0000 7fa251de8640 10 monclient: build_initial_monmap 2024-09-15T21:21:58.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.440+0000 7fa251de8640 10 monclient: monmap: 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:58.447186+0000 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:58.447186+0000 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:58.449 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.440+0000 7fa251de8640 1 Processor -- start 2024-09-15T21:21:58.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.440+0000 7fa251de8640 1 -- start start 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.440+0000 7fa251de8640 10 monclient: init 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa24c0a4ed0 0x7fa24c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): picked mon.noname-a con 0x7fa24c0a4ed0 addr v2:172.21.15.17:3300/0 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa24c0a57f0 addr v1:172.21.15.17:6789/0 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c140170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): picked mon.noname-b con 0x7fa24c1383a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0a6c60 con 0x7fa24c0a57f0 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c0a1a40 con 0x7fa24c0a4ed0 2024-09-15T21:21:58.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0a1f70 con 0x7fa24c1383a0 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa251de8640 10 monclient(hunting): authenticate will time out at 1237.244551s 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa24c0a4ed0 0x7fa24c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54452/0 (socket says 172.21.15.119:54452) 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 -- 172.21.15.119:0/383410437 learned_addr learned my addr 172.21.15.119:0/383410437 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): get_auth_request con 0x7fa24c1383a0 auth_method 0 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request con 0x7fa24c0a4ed0 auth_method 0 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient(hunting): handle_auth_done global_id 44504 payload 274 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 -- 172.21.15.119:0/383410437 >> v2:172.21.15.17:3300/0 conn(0x7fa24c0a4ed0 msgr2=0x7fa24c0a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 --2- 172.21.15.119:0/383410437 >> v2:172.21.15.17:3300/0 conn(0x7fa24c0a4ed0 0x7fa24c0a52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 -- 172.21.15.119:0/383410437 >> v1:172.21.15.17:6789/0 conn(0x7fa24c0a57f0 legacy=0x7fa24c137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:58.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 monclient: found mon.noname-b 2024-09-15T21:21:58.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:58.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:58.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 -- 172.21.15.119:0/383410437 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0a7d70 con 0x7fa24c1383a0 2024-09-15T21:21:58.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24b7fe640 1 --2- 172.21.15.119:0/383410437 >> v2:172.21.15.17:3300/0 conn(0x7fa24c0a4ed0 0x7fa24c0a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:58.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.444+0000 7fa24bfff640 1 --2- 172.21.15.119:0/383410437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c140170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa24401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b825956c7a1c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:58.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 1 -- 172.21.15.119:0/383410437 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440a55b0 con 0x7fa24c1383a0 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: dump: 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.461 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 5 monclient: authenticate success, global_id 44504 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 1 -- 172.21.15.119:0/383410437 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2440a5750 con 0x7fa24c1383a0 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 1 -- 172.21.15.119:0/383410437 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440a5b20 con 0x7fa24c1383a0 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 4 set_mon_vals no callback set 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24a7fc640 10 monclient: dump: 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: shutdown 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:58.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/383410437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 msgr2=0x7fa24c140170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 --2- 172.21.15.119:0/383410437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c140170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa24401c8f0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/383410437 shutdown_connections 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 --2- 172.21.15.119:0/383410437 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c140170 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 --2- 172.21.15.119:0/383410437 >> v2:172.21.15.17:3300/0 conn(0x7fa24c0a4ed0 0x7fa24c0a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/383410437 >> 172.21.15.119:0/383410437 conn(0x7fa24c0a6fd0 msgr2=0x7fa24c0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/383410437 shutdown_connections 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/383410437 wait complete. 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: build_initial_monmap 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: monmap: 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:58.464 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 Processor -- start 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- start start 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: init 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa24c0a4ed0 0x7fa24c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): picked mon.noname-b con 0x7fa24c0a4ed0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:58.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 0x7fa24c1d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): picked mon.noname-a con 0x7fa24c0a57f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c13d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): picked mon.noname-c con 0x7fa24c1383a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0a65c0 con 0x7fa24c0a57f0 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0a6440 con 0x7fa24c0a4ed0 2024-09-15T21:21:58.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0a10e0 con 0x7fa24c1383a0 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient(hunting): authenticate will time out at 1237.250235s 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24b7fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa24c0a4ed0 msgr2=0x7fa24c13fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa24c0a4ed0 0x7fa24c13fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 0x7fa24c1d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 0x7fa24c1d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47476/0 (socket says 172.21.15.119:47476) 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 -- 172.21.15.119:0/9227484 learned_addr learned my addr 172.21.15.119:0/9227484 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): get_auth_request con 0x7fa24c0a57f0 auth_method 0 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c13d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): get_auth_request con 0x7fa24c1383a0 auth_method 0 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient(hunting): handle_auth_done global_id 54396 payload 824 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 -- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 msgr2=0x7fa24c13d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c13d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 -- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa24c0a4ed0 msgr2=0x7fa24c13fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa24c0a4ed0 0x7fa24c13fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient: _finish_hunting 0 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 monclient: found mon.noname-a 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 -- 172.21.15.119:0/9227484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0a9cc0 con 0x7fa24c0a57f0 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa24affd640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 0x7fa24c1d03d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c530 tx=0x7fa24000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240017070 con 0x7fa24c0a57f0 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: dump: 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.469 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: _finish_auth 0 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: _check_auth_tickets 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2400052d0 con 0x7fa24c0a57f0 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240012870 con 0x7fa24c0a57f0 2024-09-15T21:21:58.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa233fff640 10 monclient: dump: 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 5 monclient: authenticate success, global_id 54396 2024-09-15T21:21:58.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: _renew_subs 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa249ffb640 4 set_mon_vals no callback set 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c0a9770 con 0x7fa24c0a57f0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: _renew_subs 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.448+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c1c7c70 con 0x7fa24c0a57f0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa251de8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa251de8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa251de8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c0a52b0 con 0x7fa24c0a57f0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa24001a070 con 0x7fa24c0a57f0 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa233fff640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa23803b9a0 0x7fa23803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:58.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8233+0+0 (secure 0 0 0) 0x7fa24004fc80 con 0x7fa24c0a57f0 2024-09-15T21:21:58.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa24b7fe640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa23803b9a0 0x7fa23803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:58.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa24b7fe640 10 monclient: get_auth_request con 0x7fa23803b9a0 auth_method 0 2024-09-15T21:21:58.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.452+0000 7fa24b7fe640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa23803b9a0 0x7fa23803de60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0096a0 tx=0x7fa23c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:58.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.464+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa24c0a9770 con 0x7fa24c0a57f0 2024-09-15T21:21:58.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.464+0000 7fa233fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:58.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.464+0000 7fa233fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:58.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.640+0000 7fa251de8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T21:21:58.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.640+0000 7fa251de8640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T21:21:58.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.640+0000 7fa251de8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.640+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) -- 0x7fa24c13f580 con 0x7fa24c0a57f0 2024-09-15T21:21:58.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.876+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v130) ==== 89+0+0 (secure 0 0 0) 0x7fa2400210a0 con 0x7fa24c0a57f0 2024-09-15T21:21:58.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.876+0000 7fa233fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T21:21:58.883 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.876+0000 7fa233fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:21:58.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.992+0000 7fa251de8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T21:21:58.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.992+0000 7fa251de8640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T21:21:58.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.992+0000 7fa251de8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:21:58.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:58.992+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) -- 0x7fa24c0008e0 con 0x7fa24c0a57f0 2024-09-15T21:21:59.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa233fff640 1 -- 172.21.15.119:0/9227484 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v131) ==== 89+0+0 (secure 0 0 0) 0x7fa24c0a9cc0 con 0x7fa24c0a57f0 2024-09-15T21:21:59.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa233fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T21:21:59.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa233fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:21:59.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa23803b9a0 msgr2=0x7fa23803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa23803b9a0 0x7fa23803de60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0096a0 tx=0x7fa23c009190 comp rx=0 tx=0).stop 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 10 monclient: shutdown 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 msgr2=0x7fa24c1d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 0x7fa24c1d03d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c530 tx=0x7fa24000c9f0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 shutdown_connections 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa23803b9a0 0x7fa23803de60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa24c1383a0 0x7fa24c13d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa24c0a57f0 0x7fa24c1d03d0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c530 tx=0x7fa24000c9f0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 --2- 172.21.15.119:0/9227484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa24c0a4ed0 0x7fa24c13fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 >> 172.21.15.119:0/9227484 conn(0x7fa24c0a6fd0 msgr2=0x7fa24c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 shutdown_connections 2024-09-15T21:21:59.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.656+0000 7fa251de8640 1 -- 172.21.15.119:0/9227484 wait complete. 2024-09-15T21:21:59.685 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:21:59.685 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c fooset 2024-09-15T21:21:59.685 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 0 2024-09-15T21:21:59.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient: get_monmap_and_config 2024-09-15T21:21:59.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient: build_initial_monmap 2024-09-15T21:21:59.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient: monmap: 2024-09-15T21:21:59.799 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:59.799 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:21:59.798127+0000 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:21:59.798127+0000 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:21:59.800 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:59.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 Processor -- start 2024-09-15T21:21:59.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 -- start start 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient: init 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ba00a4ed0 0x7f7ba00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): picked mon.noname-c con 0x7f7ba00a4ed0 addr v2:172.21.15.119:3300/0 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 0x7f7ba00a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): picked mon.noname-b con 0x7f7ba00a57f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7ba0137c90 addr v1:172.21.15.119:6789/0 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba00a6db0 con 0x7f7ba0137c90 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba00a1210 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba00a1b00 con 0x7f7ba00a57f0 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba70e1640 10 monclient(hunting): authenticate will time out at 1238.594832s 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba567e640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f7ba0137c90 legacy=0x7f7ba01380e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba4e7d640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f7ba00a4ed0 msgr2=0x7f7ba00a52b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7ba4e7d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ba00a4ed0 0x7f7ba00a52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 0x7f7ba00a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 0x7f7ba00a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54488/0 (socket says 172.21.15.119:54488) 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 -- 172.21.15.119:0/186445736 learned_addr learned my addr 172.21.15.119:0/186445736 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): get_auth_request con 0x7f7ba00a57f0 auth_method 0 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:59.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient(hunting): handle_auth_done global_id 44510 payload 274 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 -- 172.21.15.119:0/186445736 >> v2:172.21.15.119:3300/0 conn(0x7f7ba00a4ed0 msgr2=0x7f7ba00a52b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 --2- 172.21.15.119:0/186445736 >> v2:172.21.15.119:3300/0 conn(0x7f7ba00a4ed0 0x7f7ba00a52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 -- 172.21.15.119:0/186445736 >> v1:172.21.15.119:6789/0 conn(0x7f7ba0137c90 legacy=0x7f7ba01380e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 monclient: found mon.noname-b 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:59.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:21:59.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.792+0000 7f7b9ffff640 1 -- 172.21.15.119:0/186445736 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba00a6ad0 con 0x7f7ba00a57f0 2024-09-15T21:21:59.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9ffff640 1 --2- 172.21.15.119:0/186445736 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 0x7f7ba00a5c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009870 tx=0x7f7b9001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f00f27272570e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:59.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 1 -- 172.21.15.119:0/186445736 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900b0070 con 0x7f7ba00a57f0 2024-09-15T21:21:59.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:59.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: dump: 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:21:59.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 5 monclient: authenticate success, global_id 44510 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 1 -- 172.21.15.119:0/186445736 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b900ab600 con 0x7f7ba00a57f0 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 10 monclient: get_monmap_and_config success 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 1 -- 172.21.15.119:0/186445736 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900a4e60 con 0x7f7ba00a57f0 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 4 set_mon_vals no callback set 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7b9f7fe640 10 monclient: dump: 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:59.813 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 10 monclient: shutdown 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 -- 172.21.15.119:0/186445736 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 msgr2=0x7f7ba00a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/186445736 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 0x7f7ba00a5c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009870 tx=0x7f7b9001c440 comp rx=0 tx=0).stop 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 -- 172.21.15.119:0/186445736 shutdown_connections 2024-09-15T21:21:59.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/186445736 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a57f0 0x7f7ba00a5c00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/186445736 >> v2:172.21.15.119:3300/0 conn(0x7f7ba00a4ed0 0x7f7ba00a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 -- 172.21.15.119:0/186445736 >> 172.21.15.119:0/186445736 conn(0x7f7ba00a6fd0 msgr2=0x7f7ba00a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 -- 172.21.15.119:0/186445736 shutdown_connections 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.796+0000 7f7ba70e1640 1 -- 172.21.15.119:0/186445736 wait complete. 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: build_initial_monmap 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: monmap: 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:59.815 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 Processor -- start 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- start start 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: init 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _reopen_session rank -1 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 0x7f7ba01cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): picked mon.noname-c con 0x7f7ba00a4ed0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba00a57f0 0x7f7ba0136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): picked mon.noname-b con 0x7f7ba00a57f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba0137c90 0x7f7ba013cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:59.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): picked mon.noname-a con 0x7f7ba0137c90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00a7ea0 con 0x7f7ba0137c90 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00a7d20 con 0x7f7ba00a57f0 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba0142710 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): _renew_subs 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient(hunting): authenticate will time out at 1238.601073s 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9ffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba00a57f0 msgr2=0x7f7ba0136e80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba00a57f0 0x7f7ba0136e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 0x7f7ba01cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba0137c90 0x7f7ba013cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:59.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 0x7f7ba01cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:54498/0 (socket says 172.21.15.119:54498) 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 -- 172.21.15.119:0/1431436987 learned_addr learned my addr 172.21.15.119:0/1431436987 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): get_auth_request con 0x7f7ba00a4ed0 auth_method 0 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): get_auth_request con 0x7f7ba0137c90 auth_method 0 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:59.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient(hunting): handle_auth_done global_id 44513 payload 824 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 -- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba00a57f0 msgr2=0x7f7ba0136e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba00a57f0 0x7f7ba0136e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 -- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba0137c90 msgr2=0x7f7ba013cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba0137c90 0x7f7ba013cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient: _finish_hunting 0 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 monclient: found mon.noname-c 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:21:59.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 -- 172.21.15.119:0/1431436987 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba0121810 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba567e640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba0137c90 0x7f7ba013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba4e7d640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 0x7f7ba01cfb70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400b550 tx=0x7f7b9400ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94003b00 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: dump: 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:59.820 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b94015070 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 5 monclient: authenticate success, global_id 44513 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _renew_subs 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba013e650 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b940050f0 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9d7fa640 10 monclient: dump: 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:21:59.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _renew_subs 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba013f1b0 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7b9effd640 4 set_mon_vals no callback set 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.800+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba00a52b0 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.804+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7b94005290 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.804+0000 7f7b9d7fa640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.804+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(131..131 src has 1..131) ==== 7901+0+0 (secure 0 0 0) 0x7f7b94050d60 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.804+0000 7f7b9ffff640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.804+0000 7f7b9ffff640 10 monclient: get_auth_request con 0x7f7b8c03b970 auth_method 0 2024-09-15T21:21:59.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.804+0000 7f7b9ffff640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7b900ad000 tx=0x7f7b90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:21:59.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.816+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7b9401adc0 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.816+0000 7f7b9d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:21:59.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.816+0000 7f7b9d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:21:59.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7ba70e1640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:59.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7ba70e1640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:21:59.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7ba70e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:21:59.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f7ba006c480 con 0x7f7ba00a4ed0 2024-09-15T21:21:59.999 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) ==== 71+0+7 (secure 0 0 0) 0x7f7b9404db40 con 0x7f7ba00a4ed0 2024-09-15T21:22:00.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7b9d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.000 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:21:59.992+0000 7f7b9d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:00.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.100+0000 7f7ba70e1640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.100+0000 7f7ba70e1640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.100+0000 7f7ba70e1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.100+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f7ba01212e0 con 0x7f7ba00a4ed0 2024-09-15T21:22:00.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7b9d7fa640 1 -- 172.21.15.119:0/1431436987 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) ==== 71+0+7 (secure 0 0 0) 0x7f7b9401a980 con 0x7f7ba00a4ed0 2024-09-15T21:22:00.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7b9d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7b9d7fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:00.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 msgr2=0x7f7b8c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7b900ad000 tx=0x7f7b90008040 comp rx=0 tx=0).stop 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 10 monclient: shutdown 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 msgr2=0x7f7ba01cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 0x7f7ba01cfb70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400b550 tx=0x7f7b9400ba10 comp rx=0 tx=0).stop 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 shutdown_connections 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba0137c90 0x7f7ba013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba00a57f0 0x7f7ba0136e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 --2- 172.21.15.119:0/1431436987 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ba00a4ed0 0x7f7ba01cfb70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 >> 172.21.15.119:0/1431436987 conn(0x7f7ba00a6fd0 msgr2=0x7f7ba0071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 shutdown_connections 2024-09-15T21:22:00.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.104+0000 7f7ba70e1640 1 -- 172.21.15.119:0/1431436987 wait complete. 2024-09-15T21:22:00.128 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:22:00.128 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep barset 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.236+0000 7faf5edef640 10 monclient: get_monmap_and_config 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.236+0000 7faf5edef640 10 monclient: build_initial_monmap 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.236+0000 7faf5edef640 10 monclient: monmap: 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:00.243833+0000 2024-09-15T21:22:00.245 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:00.243833+0000 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:00.246 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 Processor -- start 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 -- start start 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient: init 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf58135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf580a0160 0x7faf5813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): picked mon.noname-c con 0x7faf580a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:00.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf5806ec30 0x7faf5813cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): picked mon.noname-a con 0x7faf5806ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf5809cf30 con 0x7faf58135ca0 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faf580a1ca0 con 0x7faf5806ec30 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faf580a1f80 con 0x7faf580a0160 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5edef640 10 monclient(hunting): authenticate will time out at 1239.040970s 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf4ffff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7faf580a0160 msgr2=0x7faf5813aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf4ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf580a0160 0x7faf5813aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf5806ec30 0x7faf5813cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf5806ec30 0x7faf5813cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47510/0 (socket says 172.21.15.119:47510) 2024-09-15T21:22:00.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 -- 172.21.15.119:0/159176597 learned_addr learned my addr 172.21.15.119:0/159176597 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): get_auth_request con 0x7faf5806ec30 auth_method 0 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient(hunting): handle_auth_done global_id 54402 payload 274 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 -- 172.21.15.119:0/159176597 >> v2:172.21.15.119:3300/0 conn(0x7faf580a0160 msgr2=0x7faf5813aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 --2- 172.21.15.119:0/159176597 >> v2:172.21.15.119:3300/0 conn(0x7faf580a0160 0x7faf5813aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 -- 172.21.15.119:0/159176597 >> v1:172.21.15.17:6789/0 conn(0x7faf58135ca0 legacy=0x7faf58136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient: _finish_hunting 0 2024-09-15T21:22:00.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 monclient: found mon.noname-a 2024-09-15T21:22:00.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:00.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.240+0000 7faf5d38c640 1 -- 172.21.15.119:0/159176597 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf580a3860 con 0x7faf5806ec30 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 1 --2- 172.21.15.119:0/159176597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf5813cf50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e50 tx=0x7faf5001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46783401e15d1bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 1 -- 172.21.15.119:0/159176597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf500a56a0 con 0x7faf5806ec30 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: dump: 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.255 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 1 -- 172.21.15.119:0/159176597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf500a5840 con 0x7faf5806ec30 2024-09-15T21:22:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 1 -- 172.21.15.119:0/159176597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf500a5c10 con 0x7faf5806ec30 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4f7fe640 10 monclient: dump: 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 5 monclient: authenticate success, global_id 54402 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 4 set_mon_vals no callback set 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: shutdown 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- 172.21.15.119:0/159176597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 msgr2=0x7faf5813cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 --2- 172.21.15.119:0/159176597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf5813cf50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e50 tx=0x7faf5001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- 172.21.15.119:0/159176597 shutdown_connections 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 --2- 172.21.15.119:0/159176597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf5813cf50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 --2- 172.21.15.119:0/159176597 >> v2:172.21.15.119:3300/0 conn(0x7faf580a0160 0x7faf5813aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- 172.21.15.119:0/159176597 >> 172.21.15.119:0/159176597 conn(0x7faf580a2ae0 msgr2=0x7faf580a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- 172.21.15.119:0/159176597 shutdown_connections 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- 172.21.15.119:0/159176597 wait complete. 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: build_initial_monmap 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: monmap: 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:00.259 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 Processor -- start 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- start start 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: init 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf58130c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): picked mon.noname-a con 0x7faf5806ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf580a0160 0x7faf58131170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): picked mon.noname-b con 0x7faf580a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf58135ca0 0x7faf581cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): picked mon.noname-c con 0x7faf58135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:00.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faf5813fe30 con 0x7faf5806ec30 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faf5813fcb0 con 0x7faf580a0160 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faf5813ffb0 con 0x7faf58135ca0 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5edef640 10 monclient(hunting): authenticate will time out at 1239.046830s 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4ffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf580a0160 msgr2=0x7faf58131170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf580a0160 0x7faf58131170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf58130c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf58135ca0 0x7faf581cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf58130c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47522/0 (socket says 172.21.15.119:47522) 2024-09-15T21:22:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 -- 172.21.15.119:0/3257062278 learned_addr learned my addr 172.21.15.119:0/3257062278 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): get_auth_request con 0x7faf5806ec30 auth_method 0 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): get_auth_request con 0x7faf58135ca0 auth_method 0 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:00.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient(hunting): handle_auth_done global_id 54408 payload 824 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 -- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf58135ca0 msgr2=0x7faf581cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf58135ca0 0x7faf581cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 -- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf580a0160 msgr2=0x7faf58131170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf580a0160 0x7faf58131170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 monclient: found mon.noname-a 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5cb8b640 1 -- 172.21.15.119:0/3257062278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5813c180 con 0x7faf5806ec30 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.244+0000 7faf5d38c640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf58135ca0 0x7faf581cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5cb8b640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf58130c30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c500 tx=0x7faf5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54017070 con 0x7faf5806ec30 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: dump: 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.264 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 5 monclient: authenticate success, global_id 54408 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 monclient: _renew_subs 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5809d1d0 con 0x7faf5806ec30 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf540052d0 con 0x7faf5806ec30 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54012870 con 0x7faf5806ec30 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 10 monclient: dump: 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 monclient: _renew_subs 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf58139e20 con 0x7faf5806ec30 2024-09-15T21:22:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4e7fc640 4 set_mon_vals no callback set 2024-09-15T21:22:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf58136080 con 0x7faf5806ec30 2024-09-15T21:22:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7faf5401a070 con 0x7faf5806ec30 2024-09-15T21:22:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf4403b970 0x7faf4403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 7901+0+0 (secure 0 0 0) 0x7faf5404fb50 con 0x7faf5806ec30 2024-09-15T21:22:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4ffff640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf4403b970 0x7faf4403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4ffff640 10 monclient: get_auth_request con 0x7faf4403b970 auth_method 0 2024-09-15T21:22:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.248+0000 7faf4ffff640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf4403b970 0x7faf4403de30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf5813b560 tx=0x7faf48006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.260+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7faf54021080 con 0x7faf5806ec30 2024-09-15T21:22:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.260+0000 7faf4d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.260+0000 7faf4d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf5edef640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf5edef640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf5edef640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7faf58134e70 con 0x7faf5806ec30 2024-09-15T21:22:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) ==== 71+0+7 (secure 0 0 0) 0x7faf54005480 con 0x7faf5806ec30 2024-09-15T21:22:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf4d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.436+0000 7faf4d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:00.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:00.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7faf5809f8f0 con 0x7faf5806ec30 2024-09-15T21:22:00.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf4d7fa640 1 -- 172.21.15.119:0/3257062278 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) ==== 71+0+7 (secure 0 0 0) 0x7faf54003d60 con 0x7faf5806ec30 2024-09-15T21:22:00.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf4d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:00.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf4d7fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:00.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf4403b970 msgr2=0x7faf4403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf4403b970 0x7faf4403de30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf5813b560 tx=0x7faf48006d90 comp rx=0 tx=0).stop 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 10 monclient: shutdown 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 msgr2=0x7faf58130c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf58130c30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c500 tx=0x7faf5400c9c0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 shutdown_connections 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf4403b970 0x7faf4403de30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf58135ca0 0x7faf581cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf580a0160 0x7faf58131170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 --2- 172.21.15.119:0/3257062278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf5806ec30 0x7faf58130c30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 >> 172.21.15.119:0/3257062278 conn(0x7faf580a2ae0 msgr2=0x7faf58072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 shutdown_connections 2024-09-15T21:22:00.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.540+0000 7faf5edef640 1 -- 172.21.15.119:0/3257062278 wait complete. 2024-09-15T21:22:00.563 INFO:tasks.workunit.client.0.smithi119.stdout:barset 2024-09-15T21:22:00.563 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set rm barset 2024-09-15T21:22:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.672+0000 7fc485e83640 10 monclient: get_monmap_and_config 2024-09-15T21:22:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.672+0000 7fc485e83640 10 monclient: build_initial_monmap 2024-09-15T21:22:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.672+0000 7fc485e83640 10 monclient: monmap: 2024-09-15T21:22:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:00.681 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:00.680090+0000 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:00.680090+0000 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:00.682 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 Processor -- start 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 -- start start 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient: init 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4800a79e0 0x7fc4800a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): picked mon.noname-c con 0x7fc4800a79e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a46b0 0x7fc4801377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): picked mon.noname-b con 0x7fc4800a46b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4800a1100 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc480135180 con 0x7fc4800a1100 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc4800a7dc0 con 0x7fc4800a79e0 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4800a6920 con 0x7fc4800a46b0 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc485e83640 10 monclient(hunting): authenticate will time out at 1239.476983s 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47f7fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fc4800a79e0 msgr2=0x7fc4800a4170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4800a79e0 0x7fc4800a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc4800a1100 0x7fc480137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:52092/0 (socket says 172.21.15.119:52092) 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47ffff640 1 -- 172.21.15.119:0/141526961 learned_addr learned my addr 172.21.15.119:0/141526961 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:00.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 1 --2- 172.21.15.119:0/141526961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a46b0 0x7fc4801377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): get_auth_request con 0x7fc4800a46b0 auth_method 0 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47e7fc640 1 -- 172.21.15.119:0/141526961 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429586393 0 0) 0x7fc480135180 con 0x7fc4800a1100 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47e7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47e7fc640 10 monclient(hunting): my global_id is 54411 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47e7fc640 1 -- 172.21.15.119:0/141526961 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc46c003530 con 0x7fc4800a1100 2024-09-15T21:22:00.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient(hunting): handle_auth_done global_id 44519 payload 274 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 1 -- 172.21.15.119:0/141526961 >> v2:172.21.15.119:3300/0 conn(0x7fc4800a79e0 msgr2=0x7fc4800a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 1 --2- 172.21.15.119:0/141526961 >> v2:172.21.15.119:3300/0 conn(0x7fc4800a79e0 0x7fc4800a4170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 1 -- 172.21.15.119:0/141526961 >> v1:172.21.15.17:6789/0 conn(0x7fc4800a1100 legacy=0x7fc480137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 1 monclient: found mon.noname-b 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.676+0000 7fc47effd640 1 -- 172.21.15.119:0/141526961 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4800a1c30 con 0x7fc4800a46b0 2024-09-15T21:22:00.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 1 --2- 172.21.15.119:0/141526961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a46b0 0x7fc4801377e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4680097c0 tx=0x7fc46801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=286e6cb5318cb817 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:00.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 1 -- 172.21.15.119:0/141526961 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480134930 con 0x7fc4800a46b0 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: dump: 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.694 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 1 -- 172.21.15.119:0/141526961 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4680b0070 con 0x7fc4800a46b0 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 5 monclient: authenticate success, global_id 44519 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 1 -- 172.21.15.119:0/141526961 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4680ab630 con 0x7fc4800a46b0 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47e7fc640 10 monclient: dump: 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.695 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 4 set_mon_vals no callback set 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: shutdown 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:00.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- 172.21.15.119:0/141526961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a46b0 msgr2=0x7fc4801377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 --2- 172.21.15.119:0/141526961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a46b0 0x7fc4801377e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4680097c0 tx=0x7fc46801c410 comp rx=0 tx=0).stop 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- 172.21.15.119:0/141526961 shutdown_connections 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 --2- 172.21.15.119:0/141526961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a46b0 0x7fc4801377e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 --2- 172.21.15.119:0/141526961 >> v2:172.21.15.119:3300/0 conn(0x7fc4800a79e0 0x7fc4800a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- 172.21.15.119:0/141526961 >> 172.21.15.119:0/141526961 conn(0x7fc4800a6ef0 msgr2=0x7fc4800a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- 172.21.15.119:0/141526961 shutdown_connections 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- 172.21.15.119:0/141526961 wait complete. 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: build_initial_monmap 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: monmap: 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:00.697 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 Processor -- start 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- start start 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: init 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a1100 0x7fc4801cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): picked mon.noname-c con 0x7fc4800a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 0x7fc4801cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): picked mon.noname-a con 0x7fc4800a46b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4800a79e0 0x7fc4800720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): picked mon.noname-b con 0x7fc4800a79e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4801385b0 con 0x7fc4800a46b0 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc480138430 con 0x7fc4800a79e0 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4801423a0 con 0x7fc4800a1100 2024-09-15T21:22:00.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc485e83640 10 monclient(hunting): authenticate will time out at 1239.482900s 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47ffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4800a79e0 msgr2=0x7fc4800720b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4800a79e0 0x7fc4800720b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 0x7fc4801cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a1100 0x7fc4801cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 0x7fc4801cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:47532/0 (socket says 172.21.15.119:47532) 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 1 -- 172.21.15.119:0/3075318928 learned_addr learned my addr 172.21.15.119:0/3075318928 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): get_auth_request con 0x7fc4800a46b0 auth_method 0 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): get_auth_request con 0x7fc4800a1100 auth_method 0 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient(hunting): handle_auth_done global_id 44522 payload 824 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 -- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4800a79e0 msgr2=0x7fc4800720b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4800a79e0 0x7fc4800720b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 -- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 msgr2=0x7fc4801cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:00.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 0x7fc4801cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47f7fe640 1 -- 172.21.15.119:0/3075318928 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48013a080 con 0x7fc4800a1100 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.680+0000 7fc47effd640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 0x7fc4801cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc47f7fe640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a1100 0x7fc4801cf760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc47000be00 tx=0x7fc470009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470015470 con 0x7fc4800a1100 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: dump: 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.702 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: _finish_auth 0 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 5 monclient: authenticate success, global_id 44522 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 monclient: _renew_subs 2024-09-15T21:22:00.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc480073f20 con 0x7fc4800a1100 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc470010070 con 0x7fc4800a1100 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 monclient: _renew_subs 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc480139da0 con 0x7fc4800a1100 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47001a020 con 0x7fc4800a1100 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc47dffb640 4 set_mon_vals no callback set 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 10 monclient: dump: 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:00.704 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:00.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc480132120 con 0x7fc4800a1100 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc470015850 con 0x7fc4800a1100 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc46003b970 0x7fc46003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(131..131 src has 1..131) ==== 7901+0+0 (secure 0 0 0) 0x7fc470050f60 con 0x7fc4800a1100 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc47effd640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc46003b970 0x7fc46003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc47effd640 10 monclient: get_auth_request con 0x7fc46003b970 auth_method 0 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.684+0000 7fc47effd640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc46003b970 0x7fc46003de30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc480138a90 tx=0x7fc4680a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:00.706 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.696+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc47007fd50 con 0x7fc4800a1100 2024-09-15T21:22:00.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.696+0000 7fc477fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:00.707 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.696+0000 7fc477fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:00.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.864+0000 7fc485e83640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T21:22:00.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.864+0000 7fc485e83640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T21:22:00.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.864+0000 7fc485e83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:00.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.864+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) -- 0x7fc48006a760 con 0x7fc4800a1100 2024-09-15T21:22:00.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.896+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v132) ==== 89+0+0 (secure 0 0 0) 0x7fc470024080 con 0x7fc4800a1100 2024-09-15T21:22:00.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.896+0000 7fc477fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T21:22:00.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:00.896+0000 7fc477fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:01.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.008+0000 7fc485e83640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T21:22:01.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.008+0000 7fc485e83640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T21:22:01.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.008+0000 7fc485e83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:01.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.008+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) -- 0x7fc4801d0300 con 0x7fc4800a1100 2024-09-15T21:22:01.664 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:22:01.664 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:22:01.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:22:01.667 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:22:01.668 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:22:01.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:22:01.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient: get_monmap_and_config 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient: build_initial_monmap 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient: monmap: 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:01.799160+0000 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:01.799160+0000 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:01.801 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:01.802 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:01.802 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:01.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 1 Processor -- start 2024-09-15T21:22:01.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 1 -- start start 2024-09-15T21:22:01.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient: init 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09d400 0x7fbdcc09d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient(hunting): picked mon.noname-b con 0x7fbdcc09d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbdcc09dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbdcc134d70 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc0a3130 con 0x7fbdcc134d70 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc09e7a0 con 0x7fbdcc09dd20 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc0a1bc0 con 0x7fbdcc09d400 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdd15ed640 10 monclient(hunting): authenticate will time out at 1245.194820s 2024-09-15T21:22:01.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbdcc134d70 0x7fbdcc13cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34720/0 (socket says 172.21.15.17:34720) 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcb7fe640 1 -- 172.21.15.17:0/2644789449 learned_addr learned my addr 172.21.15.17:0/2644789449 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 1 --2- 172.21.15.17:0/2644789449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09d400 0x7fbdcc09d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): get_auth_request con 0x7fbdcc09d400 auth_method 0 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_done global_id 44525 payload 274 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 1 -- 172.21.15.17:0/2644789449 >> v1:172.21.15.119:6789/0 conn(0x7fbdcc09dd20 legacy=0x7fbdcc134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 1 -- 172.21.15.17:0/2644789449 >> v1:172.21.15.17:6789/0 conn(0x7fbdcc134d70 legacy=0x7fbdcc13cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:01.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:01.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 1 monclient: found mon.noname-b 2024-09-15T21:22:01.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:01.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdc9ffb640 1 -- 172.21.15.17:0/2644789449 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 55989747 0 0) 0x7fbdcc0a3130 con 0x7fbdcc134d70 2024-09-15T21:22:01.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:01.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdcaffd640 1 -- 172.21.15.17:0/2644789449 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc0a0320 con 0x7fbdcc09d400 2024-09-15T21:22:01.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.797+0000 7fbdc9ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:22:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdcaffd640 1 --2- 172.21.15.17:0/2644789449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09d400 0x7fbdcc09d7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0097f0 tx=0x7fbdbc01c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceca8639e93ddf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 1 -- 172.21.15.17:0/2644789449 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0b0070 con 0x7fbdcc09d400 2024-09-15T21:22:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: dump: 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 1 -- 172.21.15.17:0/2644789449 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdbc0ab600 con 0x7fbdcc09d400 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 1 -- 172.21.15.17:0/2644789449 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0ab9d0 con 0x7fbdcc09d400 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 5 monclient: authenticate success, global_id 44525 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 4 set_mon_vals no callback set 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdc9ffb640 10 monclient: dump: 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: shutdown 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- 172.21.15.17:0/2644789449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09d400 msgr2=0x7fbdcc09d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/2644789449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09d400 0x7fbdcc09d7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0097f0 tx=0x7fbdbc01c3a0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- 172.21.15.17:0/2644789449 shutdown_connections 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/2644789449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09d400 0x7fbdcc09d7e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- 172.21.15.17:0/2644789449 >> 172.21.15.17:0/2644789449 conn(0x7fbdcc0a66a0 msgr2=0x7fbdcc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- 172.21.15.17:0/2644789449 shutdown_connections 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- 172.21.15.17:0/2644789449 wait complete. 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: build_initial_monmap 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: monmap: 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:01.815 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 Processor -- start 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- start start 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: init 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 0x7fbdcc1d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): picked mon.noname-a con 0x7fbdcc09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09dd20 0x7fbdcc1d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): picked mon.noname-c con 0x7fbdcc09dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 0x7fbdcc11cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): picked mon.noname-b con 0x7fbdcc134d70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc0a10a0 con 0x7fbdcc09d400 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc0a3490 con 0x7fbdcc134d70 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc0a1220 con 0x7fbdcc09dd20 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.801+0000 7fbdd15ed640 10 monclient(hunting): authenticate will time out at 1245.199880s 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 0x7fbdcc1d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09dd20 0x7fbdcc1d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 0x7fbdcc1d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48834/0 (socket says 172.21.15.17:48834) 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 -- 172.21.15.17:0/3184403500 learned_addr learned my addr 172.21.15.17:0/3184403500 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): get_auth_request con 0x7fbdcc09d400 auth_method 0 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): get_auth_request con 0x7fbdcc09dd20 auth_method 0 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 0x7fbdcc11cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 10 monclient(hunting): get_auth_request con 0x7fbdcc134d70 auth_method 0 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 msgr2=0x7fbdcc11cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 msgr2=0x7fbdcc11cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 0x7fbdcc11cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcb7fe640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 0x7fbdcc11cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient(hunting): handle_auth_done global_id 54420 payload 824 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09dd20 msgr2=0x7fbdcc1d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09dd20 0x7fbdcc1d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 msgr2=0x7fbdcc11cd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 0x7fbdcc11cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 monclient: found mon.noname-a 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 -- 172.21.15.17:0/3184403500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc1205a0 con 0x7fbdcc09d400 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdcaffd640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 0x7fbdcc1d0220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0ae040 tx=0x7fbdbc01abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0b0050 con 0x7fbdcc09d400 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: dump: 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 5 monclient: authenticate success, global_id 54420 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdbc0b1040 con 0x7fbdcc09d400 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 monclient: _renew_subs 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc120880 con 0x7fbdcc09d400 2024-09-15T21:22:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc004220 con 0x7fbdcc09d400 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 10 monclient: dump: 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:01.822 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc97fa640 4 set_mon_vals no callback set 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 monclient: _renew_subs 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc073ea0 con 0x7fbdcc09d400 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc0a1600 con 0x7fbdcc09d400 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbdbc0be060 con 0x7fbdcc09d400 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 7289+0+0 (secure 0 0 0) 0x7fbdbc0ea1f0 con 0x7fbdcc09d400 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 10 monclient: get_auth_request con 0x7fbdac03b9f0 auth_method 0 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 msgr2=0x7fbdac03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 msgr2=0x7fbdac03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4007fd0 tx=0x7fbdb4009040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.805+0000 7fbdca7fc640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.809+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbdbc0160b0 con 0x7fbdcc09d400 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.809+0000 7fbdc37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:01.809+0000 7fbdc37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:01.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.908+0000 7fc477fff640 1 -- 172.21.15.119:0/3075318928 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v133) ==== 89+0+0 (secure 0 0 0) 0x7fc4700143d0 con 0x7fc4800a1100 2024-09-15T21:22:01.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.908+0000 7fc477fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T21:22:01.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.908+0000 7fc477fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:01.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.912+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc46003b970 msgr2=0x7fc46003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:01.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.912+0000 7fc485e83640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc46003b970 0x7fc46003de30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc480138a90 tx=0x7fc4680a7000 comp rx=0 tx=0).stop 2024-09-15T21:22:01.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.912+0000 7fc485e83640 10 monclient: shutdown 2024-09-15T21:22:01.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.912+0000 7fc485e83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:01.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.912+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a1100 msgr2=0x7fc4801cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:01.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.912+0000 7fc485e83640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a1100 0x7fc4801cf760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc47000be00 tx=0x7fc470009f70 comp rx=0 tx=0).stop 2024-09-15T21:22:01.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 shutdown_connections 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc46003b970 0x7fc46003de30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4800a79e0 0x7fc4800720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4800a46b0 0x7fc4801cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 --2- 172.21.15.119:0/3075318928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4800a1100 0x7fc4801cf760 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 >> 172.21.15.119:0/3075318928 conn(0x7fc4800a6ef0 msgr2=0x7fc4800a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 shutdown_connections 2024-09-15T21:22:01.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:01.916+0000 7fc485e83640 1 -- 172.21.15.119:0/3075318928 wait complete. 2024-09-15T21:22:01.945 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:22:01.945 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -c barset 2024-09-15T21:22:01.945 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep -q 0 2024-09-15T21:22:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.009+0000 7fbdca7fc640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.009+0000 7fbdca7fc640 10 monclient: get_auth_request con 0x7fbdac03b9f0 auth_method 0 2024-09-15T21:22:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.009+0000 7fbdca7fc640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4007fd0 tx=0x7fbdb400a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdd15ed640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:22:02.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdd15ed640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:02.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdd15ed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:02.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbdcc11ea30 con 0x7fbdcc09d400 2024-09-15T21:22:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdc37fe640 1 -- 172.21.15.17:0/3184403500 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fbdbc0b5610 con 0x7fbdcc09d400 2024-09-15T21:22:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdc37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.057+0000 7fbdc37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:02.063 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":48,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient: get_monmap_and_config 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient: build_initial_monmap 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient: monmap: 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:02.062461+0000 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:02.062461+0000 2024-09-15T21:22:02.064 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 1 Processor -- start 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 1 -- start start 2024-09-15T21:22:02.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient: init 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 msgr2=0x7fbdac03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4007fd0 tx=0x7fbdb400a000 comp rx=0 tx=0).stop 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 10 monclient: shutdown 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 msgr2=0x7fbdcc1d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 0x7fbdcc1d0220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0ae040 tx=0x7fbdbc01abe0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 shutdown_connections 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdac03b9f0 0x7fbdac03deb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc134d70 0x7fbdcc11cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdcc09dd20 0x7fbdcc1d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 --2- 172.21.15.17:0/3184403500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc09d400 0x7fbdcc1d0220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 >> 172.21.15.17:0/3184403500 conn(0x7fbdcc0a66a0 msgr2=0x7fbdcc13d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:02.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 shutdown_connections 2024-09-15T21:22:02.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:02.061+0000 7fbdd15ed640 1 -- 172.21.15.17:0/3184403500 wait complete. 2024-09-15T21:22:02.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:02.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:22:02.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 0x7f2fb00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient(hunting): picked mon.noname-c con 0x7f2fb00a4e60 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:02.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2fb00a5780 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2fb01383a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb00a1150 con 0x7f2fb00a5780 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb00a0ef0 con 0x7f2fb01383a0 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb00a3c80 con 0x7f2fb00a4e60 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb872d640 10 monclient(hunting): authenticate will time out at 1240.859149s 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 0x7f2fb00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb6cca640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2fb01383a0 0x7f2fb013ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50454/0 (socket says 172.21.15.119:50454) 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb6cca640 1 -- 172.21.15.119:0/3165462050 learned_addr learned my addr 172.21.15.119:0/3165462050 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 10 monclient(hunting): get_auth_request con 0x7f2fb00a4e60 auth_method 0 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3165462050 >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 msgr2=0x7f2fb00a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:02.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3165462050 >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 msgr2=0x7f2fb00a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:02.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3165462050 >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 0x7f2fb00a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:02.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3165462050 >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 0x7f2fb00a5240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909630332 0 0) 0x7f2fb00a1150 con 0x7f2fb00a5780 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb54c7640 10 monclient(hunting): _negotiate 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb54c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb54c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb54c7640 10 monclient(hunting): my global_id is 54423 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.056+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f9c003530 con 0x7f2fb00a5780 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4125840483 0 0) 0x7f2f9c003530 con 0x7f2fb00a5780 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 msgr2=0x7f2fb00a5240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 --2- 172.21.15.119:0/3165462050 >> v2:172.21.15.119:3300/0 conn(0x7f2fb00a4e60 0x7f2fb00a5240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 >> v1:172.21.15.119:6789/0 conn(0x7f2fb01383a0 legacy=0x7f2fb013ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: _finish_hunting 0 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:22:02.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb00a6cf0 con 0x7f2fb00a5780 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: _finish_auth 0 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: _check_auth_tickets 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f2fa4001700 con 0x7f2fb00a5780 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: dump: 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.081 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.082 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.082 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.082 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.082 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 >> v1:172.21.15.17:6789/0 conn(0x7f2fb00a5780 legacy=0x7f2fb0137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c003a60 0x7f2f9c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): picked mon.b con 0x7f2f9c003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f9c004810 0x7f2f9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): picked mon.c con 0x7f2f9c004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c0053b0 0x7f2f9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): picked mon.a con 0x7f2f9c0053b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00c2c0 con 0x7f2f9c0053b0 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00c470 con 0x7f2f9c003a60 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9c00c670 con 0x7f2f9c004810 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:22:02.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c003a60 0x7f2f9c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): get_auth_request con 0x7f2f9c003a60 auth_method 0 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb5cc8640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f9c004810 0x7f2f9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_done global_id 54423 payload 293 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f9c004810 msgr2=0x7f2f9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f9c004810 0x7f2f9c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c0053b0 msgr2=0x7f2f9c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:02.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c0053b0 0x7f2f9c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient: _finish_hunting 0 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 monclient: found mon.b 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb6cca640 1 -- 172.21.15.119:0/3165462050 reap_dead start 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3165462050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c003530 con 0x7f2f9c003a60 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient: _finish_auth 0 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 5 monclient: authenticate success, global_id 54423 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c003a60 0x7f2f9c003e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c820 tx=0x7f2fac01c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d83ed80ead063f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00f040 con 0x7f2f9c003a60 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: dump: 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac0b0070 con 0x7f2f9c003a60 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 1 -- 172.21.15.119:0/3165462050 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0ab670 con 0x7f2f9c003a60 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 4 set_mon_vals no callback set 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb54c7640 10 monclient: dump: 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.088 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 10 monclient: shutdown 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 -- 172.21.15.119:0/3165462050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c003a60 msgr2=0x7f2f9c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c003a60 0x7f2f9c003e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c820 tx=0x7f2fac01c4f0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 -- 172.21.15.119:0/3165462050 shutdown_connections 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3165462050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c003a60 0x7f2f9c003e80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 -- 172.21.15.119:0/3165462050 >> 172.21.15.119:0/3165462050 conn(0x7f2fb00a6f60 msgr2=0x7f2fb00a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 -- 172.21.15.119:0/3165462050 shutdown_connections 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.060+0000 7f2fb872d640 1 -- 172.21.15.119:0/3165462050 wait complete. 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: build_initial_monmap 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: monmap: 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:02.090 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 Processor -- start 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- start start 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: init 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c003a60 0x7f2fb0137290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): picked mon.noname-a con 0x7f2f9c003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): picked mon.noname-b con 0x7f2fb01377d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2fb01cce10 0x7f2fb01cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): picked mon.noname-c con 0x7f2fb01cce10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00a7e30 con 0x7f2f9c003a60 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00a7cb0 con 0x7f2fb01377d0 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb00a2190 con 0x7f2fb01cce10 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient(hunting): authenticate will time out at 1240.865115s 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60468/0 (socket says 172.21.15.119:60468) 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 -- 172.21.15.119:0/3501143862 learned_addr learned my addr 172.21.15.119:0/3501143862 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 10 monclient(hunting): get_auth_request con 0x7f2fb01377d0 auth_method 0 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 msgr2=0x7f2fb01cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 msgr2=0x7f2fb01cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb5cc8640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2fb01cce10 0x7f2fb01cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c003a60 0x7f2fb0137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): get_auth_request con 0x7f2fb01cce10 auth_method 0 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): get_auth_request con 0x7f2f9c003a60 auth_method 0 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient(hunting): handle_auth_done global_id 44531 payload 824 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 msgr2=0x7f2fb01cc8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c003a60 msgr2=0x7f2fb0137290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c003a60 0x7f2fb0137290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient: _finish_hunting 0 2024-09-15T21:22:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 monclient: found mon.noname-c 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 -- 172.21.15.119:0/3501143862 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01253c0 con 0x7f2fb01cce10 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb6cca640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2fb01cce10 0x7f2fb01cfe10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fb006e4e0 tx=0x7f2fa800ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8017070 con 0x7f2fb01cce10 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: dump: 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.095 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa800a960 con 0x7f2fb01cce10 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800ad50 con 0x7f2fb01cce10 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 5 monclient: authenticate success, global_id 44531 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _renew_subs 2024-09-15T21:22:02.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb00a9490 con 0x7f2fb01cce10 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb4cc6640 4 set_mon_vals no callback set 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 10 monclient: dump: 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.097 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _renew_subs 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb0125c60 con 0x7f2fb01cce10 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb0077180 con 0x7f2fb01cce10 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2fa801a070 con 0x7f2fb01cce10 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(133..133 src has 1..133) ==== 7289+0+0 (secure 0 0 0) 0x7f2fa804fc20 con 0x7f2fb01cce10 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.064+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.068+0000 7f2fb64c9640 10 monclient: get_auth_request con 0x7f2f9403b9c0 auth_method 0 2024-09-15T21:22:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.068+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 msgr2=0x7f2f9403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:22:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.068+0000 7f2fb64c9640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 msgr2=0x7f2f9403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.068+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0af000 tx=0x7f2fac0046d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.068+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.072+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2fb01253c0 con 0x7f2fb01cce10 2024-09-15T21:22:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.072+0000 7f2fa2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:02.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.072+0000 7f2fa2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:02.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fb872d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:02.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fb872d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:02.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fb872d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f2fb0137c90 con 0x7f2fb01cce10 2024-09-15T21:22:02.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) ==== 71+0+0 (secure 0 0 0) 0x7f2fa8007a00 con 0x7f2fb01cce10 2024-09-15T21:22:02.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fa2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:02.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.248+0000 7f2fa2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:02.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.272+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.272+0000 7f2fb64c9640 10 monclient: get_auth_request con 0x7f2f9403b9c0 auth_method 0 2024-09-15T21:22:02.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.272+0000 7f2fb64c9640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2fac01c530 tx=0x7f2fac0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fb872d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:02.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fb872d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:02.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fb872d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f2fb01d0350 con 0x7f2fb01cce10 2024-09-15T21:22:02.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fa2ffd640 1 -- 172.21.15.119:0/3501143862 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) ==== 71+0+0 (secure 0 0 0) 0x7f2fa8019cd0 con 0x7f2fb01cce10 2024-09-15T21:22:02.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fa2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:02.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.356+0000 7f2fa2ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 msgr2=0x7f2f9403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2fac01c530 tx=0x7f2fac0a7000 comp rx=0 tx=0).stop 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 10 monclient: shutdown 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2fb01cce10 msgr2=0x7f2fb01cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2fb01cce10 0x7f2fb01cfe10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fb006e4e0 tx=0x7f2fa800ccd0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 shutdown_connections 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2f9403b9c0 0x7f2f9403de80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2fb01cce10 0x7f2fb01cfe10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb01377d0 0x7f2fb01cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 --2- 172.21.15.119:0/3501143862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f9c003a60 0x7f2fb0137290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 >> 172.21.15.119:0/3501143862 conn(0x7f2fb00a6f60 msgr2=0x7f2fb00a8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:02.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 shutdown_connections 2024-09-15T21:22:02.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.360+0000 7f2fb872d640 1 -- 172.21.15.119:0/3501143862 wait complete. 2024-09-15T21:22:02.381 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.488+0000 7f3bb3b75640 10 monclient: get_monmap_and_config 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.488+0000 7f3bb3b75640 10 monclient: build_initial_monmap 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.488+0000 7f3bb3b75640 10 monclient: monmap: 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:02.495900+0000 2024-09-15T21:22:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:02.495900+0000 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:02.498 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.488+0000 7f3bb3b75640 1 Processor -- start 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.488+0000 7f3bb3b75640 1 -- start start 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.488+0000 7f3bb3b75640 10 monclient: init 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3bac135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac0a0160 0x7f3bac13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): picked mon.noname-b con 0x7f3bac0a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bac06ec30 0x7f3bac13cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): picked mon.noname-a con 0x7f3bac06ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac09cf30 con 0x7f3bac135ca0 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3bac0a1ca0 con 0x7f3bac06ec30 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0a1f80 con 0x7f3bac0a0160 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb3b75640 10 monclient(hunting): authenticate will time out at 1241.292594s 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bac06ec30 0x7f3bac13cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac0a0160 0x7f3bac13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bac06ec30 0x7f3bac13cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:51880/0 (socket says 172.21.15.119:51880) 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 1 -- 172.21.15.119:0/886163496 learned_addr learned my addr 172.21.15.119:0/886163496 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): get_auth_request con 0x7f3bac06ec30 auth_method 0 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1911640 1 --1- 172.21.15.119:0/886163496 >> v1:172.21.15.17:6789/0 conn(0x7f3bac135ca0 0x7f3bac136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:49012/0 (socket says 172.21.15.119:49012) 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): get_auth_request con 0x7f3bac0a0160 auth_method 0 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:02.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb2112640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient(hunting): handle_auth_done global_id 44534 payload 274 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 -- 172.21.15.119:0/886163496 >> v2:172.21.15.17:3300/0 conn(0x7f3bac06ec30 msgr2=0x7f3bac13cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 --2- 172.21.15.119:0/886163496 >> v2:172.21.15.17:3300/0 conn(0x7f3bac06ec30 0x7f3bac13cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 -- 172.21.15.119:0/886163496 >> v1:172.21.15.17:6789/0 conn(0x7f3bac135ca0 legacy=0x7f3bac136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient: _finish_hunting 0 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 monclient: found mon.noname-b 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 -- 172.21.15.119:0/886163496 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0a3860 con 0x7f3bac0a0160 2024-09-15T21:22:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.492+0000 7f3bb1110640 1 --2- 172.21.15.119:0/886163496 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac0a0160 0x7f3bac13aa00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0097c0 tx=0x7f3b9c01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a7286744000f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 1 -- 172.21.15.119:0/886163496 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b0070 con 0x7f3bac0a0160 2024-09-15T21:22:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: dump: 2024-09-15T21:22:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: _finish_auth 0 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: _check_auth_tickets 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 1 -- 172.21.15.119:0/886163496 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9c0ab600 con 0x7f3bac0a0160 2024-09-15T21:22:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 5 monclient: authenticate success, global_id 44534 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 4 set_mon_vals no callback set 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 1 -- 172.21.15.119:0/886163496 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0ab9d0 con 0x7f3bac0a0160 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb090f640 10 monclient: dump: 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: shutdown 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- 172.21.15.119:0/886163496 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac0a0160 msgr2=0x7f3bac13aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/886163496 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac0a0160 0x7f3bac13aa00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0097c0 tx=0x7f3b9c01c360 comp rx=0 tx=0).stop 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- 172.21.15.119:0/886163496 shutdown_connections 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/886163496 >> v2:172.21.15.17:3300/0 conn(0x7f3bac06ec30 0x7f3bac13cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/886163496 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac0a0160 0x7f3bac13aa00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- 172.21.15.119:0/886163496 >> 172.21.15.119:0/886163496 conn(0x7f3bac0a2ae0 msgr2=0x7f3bac0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- 172.21.15.119:0/886163496 shutdown_connections 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- 172.21.15.119:0/886163496 wait complete. 2024-09-15T21:22:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: build_initial_monmap 2024-09-15T21:22:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: monmap: 2024-09-15T21:22:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:02.513 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 Processor -- start 2024-09-15T21:22:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- start start 2024-09-15T21:22:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: init 2024-09-15T21:22:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac06ec30 0x7f3bac1cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): picked mon.noname-c con 0x7f3bac06ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bac0a0160 0x7f3bac1cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): picked mon.noname-a con 0x7f3bac0a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): picked mon.noname-b con 0x7f3bac135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac13fe30 con 0x7f3bac0a0160 2024-09-15T21:22:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac13fcb0 con 0x7f3bac135ca0 2024-09-15T21:22:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac13ffb0 con 0x7f3bac06ec30 2024-09-15T21:22:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb3b75640 10 monclient(hunting): authenticate will time out at 1241.298762s 2024-09-15T21:22:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55032/0 (socket says 172.21.15.119:55032) 2024-09-15T21:22:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 -- 172.21.15.119:0/2340086844 learned_addr learned my addr 172.21.15.119:0/2340086844 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 10 monclient(hunting): get_auth_request con 0x7f3bac135ca0 auth_method 0 2024-09-15T21:22:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 -- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 msgr2=0x7f3bac1cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:22:02.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 -- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 msgr2=0x7f3bac1cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:02.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac06ec30 0x7f3bac1cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:02.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bac0a0160 0x7f3bac1cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): get_auth_request con 0x7f3bac06ec30 auth_method 0 2024-09-15T21:22:02.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb2112640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): get_auth_request con 0x7f3bac0a0160 auth_method 0 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:02.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient(hunting): handle_auth_done global_id 44537 payload 824 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 -- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 msgr2=0x7f3bac1cdc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 -- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bac0a0160 msgr2=0x7f3bac1cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bac0a0160 0x7f3bac1cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient: _finish_hunting 0 2024-09-15T21:22:02.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 monclient: found mon.noname-c 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 -- 172.21.15.119:0/2340086844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac09d1d0 con 0x7f3bac06ec30 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1110640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bac0a0160 0x7f3bac1cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.496+0000 7f3bb1911640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac06ec30 0x7f3bac1cd1c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c3c0 tx=0x7f3ba800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80074e0 con 0x7f3bac06ec30 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: dump: 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.523 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: _finish_auth 0 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 5 monclient: authenticate success, global_id 44537 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 monclient: _renew_subs 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac13be70 con 0x7f3bac06ec30 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba8007680 con 0x7f3bac06ec30 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8004a10 con 0x7f3bac06ec30 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 10 monclient: dump: 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:02.525 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba3fff640 4 set_mon_vals no callback set 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 monclient: _renew_subs 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac13a950 con 0x7f3bac06ec30 2024-09-15T21:22:02.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:02.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:02.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac136080 con 0x7f3bac06ec30 2024-09-15T21:22:02.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3ba801a020 con 0x7f3bac06ec30 2024-09-15T21:22:02.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b9003b9c0 0x7f3b9003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:02.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(133..133 src has 1..133) ==== 7289+0+0 (secure 0 0 0) 0x7f3ba8050460 con 0x7f3bac06ec30 2024-09-15T21:22:02.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb1110640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b9003b9c0 0x7f3b9003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:02.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb1110640 10 monclient: get_auth_request con 0x7f3b9003b9c0 auth_method 0 2024-09-15T21:22:02.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.500+0000 7f3bb1110640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b9003b9c0 0x7f3b9003de80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0ad000 tx=0x7f3b9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:02.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.512+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3ba8021070 con 0x7f3bac06ec30 2024-09-15T21:22:02.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.512+0000 7f3ba27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:02.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.512+0000 7f3ba27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:02.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.676+0000 7f3bb3b75640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:02.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.676+0000 7f3bb3b75640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:02.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.676+0000 7f3bb3b75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:02.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.676+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f3bac09f8f0 con 0x7f3bac06ec30 2024-09-15T21:22:02.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.908+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v134) ==== 82+0+0 (secure 0 0 0) 0x7f3ba80113a0 con 0x7f3bac06ec30 2024-09-15T21:22:02.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.908+0000 7f3ba27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:02.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:02.908+0000 7f3ba27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:03.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3bb3b75640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:03.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3bb3b75640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:03.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3bb3b75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:03.015 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f3bac0008e0 con 0x7f3bac06ec30 2024-09-15T21:22:03.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3ba27fc640 1 -- 172.21.15.119:0/2340086844 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v134) ==== 115+0+0 (secure 0 0 0) 0x7f3ba804d4a0 con 0x7f3bac06ec30 2024-09-15T21:22:03.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3ba27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:03.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.008+0000 7f3ba27fc640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T21:22:03.017 INFO:tasks.workunit.client.0.smithi119.stderr:compat weight-set already created 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b9003b9c0 msgr2=0x7f3b9003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b9003b9c0 0x7f3b9003de80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0ad000 tx=0x7f3b9c008040 comp rx=0 tx=0).stop 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 10 monclient: shutdown 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac06ec30 msgr2=0x7f3bac1cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac06ec30 0x7f3bac1cd1c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c3c0 tx=0x7f3ba800c880 comp rx=0 tx=0).stop 2024-09-15T21:22:03.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 shutdown_connections 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3b9003b9c0 0x7f3b9003de80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bac135ca0 0x7f3bac1cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bac0a0160 0x7f3bac1cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 --2- 172.21.15.119:0/2340086844 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3bac06ec30 0x7f3bac1cd1c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 >> 172.21.15.119:0/2340086844 conn(0x7f3bac0a2ae0 msgr2=0x7f3bac0722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 shutdown_connections 2024-09-15T21:22:03.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.012+0000 7f3bb3b75640 1 -- 172.21.15.119:0/2340086844 wait complete. 2024-09-15T21:22:03.034 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '(compat)' 2024-09-15T21:22:03.034 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set ls 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.140+0000 7ff7fd736640 10 monclient: get_monmap_and_config 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.140+0000 7ff7fd736640 10 monclient: build_initial_monmap 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.140+0000 7ff7fd736640 10 monclient: monmap: 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:03.147760+0000 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:03.147760+0000 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:03.149 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.150 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:03.150 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:03.150 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:03.150 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:03.150 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:03.150 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.140+0000 7ff7fd736640 1 Processor -- start 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.140+0000 7ff7fd736640 1 -- start start 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.140+0000 7ff7fd736640 10 monclient: init 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff7f80a4ed0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): picked mon.noname-b con 0x7ff7f80a59c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:03.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): picked mon.noname-c con 0x7ff7f80a5dd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f80a6db0 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f80a1210 con 0x7ff7f80a5dd0 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f80a1b00 con 0x7ff7f80a59c0 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7fd736640 10 monclient(hunting): authenticate will time out at 1241.944642s 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55048/0 (socket says 172.21.15.119:55048) 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 -- 172.21.15.119:0/3576939838 learned_addr learned my addr 172.21.15.119:0/3576939838 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 10 monclient(hunting): get_auth_request con 0x7ff7f80a5dd0 auth_method 0 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 -- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 msgr2=0x7ff7f813f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 -- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 msgr2=0x7ff7f813f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 --2- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f77fe640 1 --2- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 1 --2- 172.21.15.119:0/3576939838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): get_auth_request con 0x7ff7f80a59c0 auth_method 0 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f5ffb640 1 -- 172.21.15.119:0/3576939838 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3627846926 0 0) 0x7ff7f80a6db0 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f5ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f5ffb640 10 monclient(hunting): my global_id is 54438 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f5ffb640 1 -- 172.21.15.119:0/3576939838 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7e0003530 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_done global_id 44540 payload 274 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 1 -- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 msgr2=0x7ff7f813f420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 1 --2- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 1 -- 172.21.15.119:0/3576939838 >> v1:172.21.15.17:6789/0 conn(0x7ff7f80a4ed0 legacy=0x7ff7f80a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:22:03.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 1 monclient: found mon.noname-b 2024-09-15T21:22:03.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:03.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:03.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.144+0000 7ff7f67fc640 1 -- 172.21.15.119:0/3576939838 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f80a6ad0 con 0x7ff7f80a59c0 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f67fc640 1 --2- 172.21.15.119:0/3576939838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813ced0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0097c0 tx=0x7ff7ec01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b5d3a057da5f096 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 1 -- 172.21.15.119:0/3576939838 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0b0070 con 0x7ff7f80a59c0 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: dump: 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.158 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7fd736640 5 monclient: authenticate success, global_id 44540 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7fd736640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 1 -- 172.21.15.119:0/3576939838 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec0ab400 con 0x7ff7f80a59c0 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7fd736640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7fd736640 4 set_mon_vals no callback set 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 1 -- 172.21.15.119:0/3576939838 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0ab7b0 con 0x7ff7f80a59c0 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.148+0000 7ff7f5ffb640 10 monclient: dump: 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: shutdown 2024-09-15T21:22:03.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3576939838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 msgr2=0x7ff7f813ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3576939838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813ced0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0097c0 tx=0x7ff7ec01c360 comp rx=0 tx=0).stop 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3576939838 shutdown_connections 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3576939838 >> v2:172.21.15.119:3300/0 conn(0x7ff7f80a5dd0 0x7ff7f813f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3576939838 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813ced0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3576939838 >> 172.21.15.119:0/3576939838 conn(0x7ff7f80a6fd0 msgr2=0x7ff7f80a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3576939838 shutdown_connections 2024-09-15T21:22:03.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3576939838 wait complete. 2024-09-15T21:22:03.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: build_initial_monmap 2024-09-15T21:22:03.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: monmap: 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 Processor -- start 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- start start 2024-09-15T21:22:03.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: init 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7f80a4ed0 0x7ff7f8127a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): picked mon.noname-a con 0x7ff7f80a4ed0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): picked mon.noname-c con 0x7ff7f80a59c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): picked mon.noname-b con 0x7ff7f80a5dd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8142180 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f80a7e30 con 0x7ff7f80a5dd0 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f8142300 con 0x7ff7f80a59c0 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient(hunting): authenticate will time out at 1241.953672s 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55060/0 (socket says 172.21.15.119:55060) 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 -- 172.21.15.119:0/3215693533 learned_addr learned my addr 172.21.15.119:0/3215693533 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 10 monclient(hunting): get_auth_request con 0x7ff7f80a5dd0 auth_method 0 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 -- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 msgr2=0x7ff7f8125480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:22:03.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 -- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 msgr2=0x7ff7f8125480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f77fe640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7f80a4ed0 0x7ff7f8127a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): get_auth_request con 0x7ff7f80a4ed0 auth_method 0 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): get_auth_request con 0x7ff7f80a59c0 auth_method 0 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient(hunting): handle_auth_done global_id 54441 payload 824 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 -- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 msgr2=0x7ff7f813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 -- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 msgr2=0x7ff7f8125480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:03.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 monclient: found mon.noname-a 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 -- 172.21.15.119:0/3215693533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f8073840 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f6ffd640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7f80a4ed0 0x7ff7f8127a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c430 tx=0x7ff7e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8017070 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: dump: 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.172 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 5 monclient: authenticate success, global_id 54441 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _renew_subs 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81d0240 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e80052f0 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _renew_subs 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7f4ff9640 4 set_mon_vals no callback set 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f8129250 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8003800 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7df7fe640 10 monclient: dump: 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.174 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.152+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f80a52b0 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.156+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff7e801a070 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.156+0000 7ff7df7fe640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7e403b9c0 0x7ff7e403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.156+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 7621+0+0 (secure 0 0 0) 0x7ff7e804fc20 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.156+0000 7ff7f67fc640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7e403b9c0 0x7ff7e403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.156+0000 7ff7f67fc640 10 monclient: get_auth_request con 0x7ff7e403b9c0 auth_method 0 2024-09-15T21:22:03.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.156+0000 7ff7f67fc640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7e403b9c0 0x7ff7e403de80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ad000 tx=0x7ff7ec0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:03.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.168+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff7e804cc10 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.168+0000 7ff7df7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:03.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.168+0000 7ff7df7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:03.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.340+0000 7ff7fd736640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:03.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.340+0000 7ff7fd736640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:03.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.340+0000 7ff7fd736640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.340+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7ff7f806c480 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.344+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v134) ==== 71+0+9 (secure 0 0 0) 0x7ff7e8011590 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.344+0000 7ff7df7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:03.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.344+0000 7ff7df7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:03.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.440+0000 7ff7fd736640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:03.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.440+0000 7ff7fd736640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:03.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.440+0000 7ff7fd736640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.440+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7ff7f81289b0 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7df7fe640 1 -- 172.21.15.119:0/3215693533 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v134) ==== 71+0+9 (secure 0 0 0) 0x7ff7e80039a0 con 0x7ff7f80a4ed0 2024-09-15T21:22:03.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7df7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T21:22:03.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7df7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:03.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7e403b9c0 msgr2=0x7ff7e403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7e403b9c0 0x7ff7e403de80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ad000 tx=0x7ff7ec0a6000 comp rx=0 tx=0).stop 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 10 monclient: shutdown 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7f80a4ed0 msgr2=0x7ff7f8127a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7f80a4ed0 0x7ff7f8127a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c430 tx=0x7ff7e800c8f0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 shutdown_connections 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff7e403b9c0 0x7ff7e403de80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7f80a5dd0 0x7ff7f8125480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7f80a59c0 0x7ff7f813e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 --2- 172.21.15.119:0/3215693533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7f80a4ed0 0x7ff7f8127a50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 >> 172.21.15.119:0/3215693533 conn(0x7ff7f80a6fd0 msgr2=0x7ff7f8071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 shutdown_connections 2024-09-15T21:22:03.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.448+0000 7ff7fd736640 1 -- 172.21.15.119:0/3215693533 wait complete. 2024-09-15T21:22:03.467 INFO:tasks.workunit.client.0.smithi119.stdout:(compat) 2024-09-15T21:22:03.467 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T21:22:03.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: get_monmap_and_config 2024-09-15T21:22:03.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: build_initial_monmap 2024-09-15T21:22:03.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: monmap: 2024-09-15T21:22:03.579 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:03.579 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:03.578186+0000 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:03.578186+0000 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:03.580 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 Processor -- start 2024-09-15T21:22:03.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- start start 2024-09-15T21:22:03.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: init 2024-09-15T21:22:03.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:03.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:22:03.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5ad8135cf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ad806ec30 0x7f5ad81360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): picked mon.noname-a con 0x7f5ad806ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad80a0160 0x7f5ad80a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): picked mon.noname-b con 0x7f5ad80a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad809cf30 con 0x7f5ad8135cf0 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad8136610 con 0x7f5ad806ec30 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad80a1ca0 con 0x7f5ad80a0160 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient(hunting): authenticate will time out at 1242.374897s 2024-09-15T21:22:03.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad6ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5ad8135cf0 0x7f5ad80a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32774/0 (socket says 172.21.15.119:32774) 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad6ffd640 1 -- 172.21.15.119:0/1815434645 learned_addr learned my addr 172.21.15.119:0/1815434645 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 --2- 172.21.15.119:0/1815434645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad80a0160 0x7f5ad80a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 1 --2- 172.21.15.119:0/1815434645 >> v2:172.21.15.17:3300/0 conn(0x7f5ad806ec30 0x7f5ad81360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): get_auth_request con 0x7f5ad80a0160 auth_method 0 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): get_auth_request con 0x7f5ad806ec30 auth_method 0 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_done global_id 44546 payload 274 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 -- 172.21.15.119:0/1815434645 >> v2:172.21.15.17:3300/0 conn(0x7f5ad806ec30 msgr2=0x7f5ad81360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 --2- 172.21.15.119:0/1815434645 >> v2:172.21.15.17:3300/0 conn(0x7f5ad806ec30 0x7f5ad81360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 -- 172.21.15.119:0/1815434645 >> v1:172.21.15.119:6789/0 conn(0x7f5ad8135cf0 legacy=0x7f5ad80a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 monclient: found mon.noname-b 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 -- 172.21.15.119:0/1815434645 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad80a3860 con 0x7f5ad80a0160 2024-09-15T21:22:03.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5acffff640 1 --2- 172.21.15.119:0/1815434645 >> v2:172.21.15.17:3300/0 conn(0x7f5ad806ec30 0x7f5ad81360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad77fe640 1 --2- 172.21.15.119:0/1815434645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad80a0160 0x7f5ad80a10f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009d90 tx=0x7f5ad001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96260918dddbd9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 1 -- 172.21.15.119:0/1815434645 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00b2070 con 0x7f5ad80a0160 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: dump: 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.593 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: _finish_auth 0 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 1 -- 172.21.15.119:0/1815434645 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad00a5890 con 0x7f5ad80a0160 2024-09-15T21:22:03.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 1 -- 172.21.15.119:0/1815434645 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00a5c60 con 0x7f5ad80a0160 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 5 monclient: authenticate success, global_id 44546 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 4 set_mon_vals no callback set 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5ad67fc640 10 monclient: dump: 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.595 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: shutdown 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- 172.21.15.119:0/1815434645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad80a0160 msgr2=0x7f5ad80a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 --2- 172.21.15.119:0/1815434645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad80a0160 0x7f5ad80a10f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009d90 tx=0x7f5ad001c8c0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- 172.21.15.119:0/1815434645 shutdown_connections 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 --2- 172.21.15.119:0/1815434645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad80a0160 0x7f5ad80a10f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 --2- 172.21.15.119:0/1815434645 >> v2:172.21.15.17:3300/0 conn(0x7f5ad806ec30 0x7f5ad81360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- 172.21.15.119:0/1815434645 >> 172.21.15.119:0/1815434645 conn(0x7f5ad80a2ae0 msgr2=0x7f5ad80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- 172.21.15.119:0/1815434645 shutdown_connections 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 1 -- 172.21.15.119:0/1815434645 wait complete. 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: build_initial_monmap 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.576+0000 7f5add525640 10 monclient: monmap: 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:03.597 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 Processor -- start 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- start start 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: init 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): picked mon.noname-b con 0x7f5ad806ec30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ad80a0160 0x7f5ad813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): picked mon.noname-a con 0x7f5ad80a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad8135cf0 0x7f5ad81cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): picked mon.noname-c con 0x7f5ad8135cf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8140240 con 0x7f5ad80a0160 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80a1840 con 0x7f5ad806ec30 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad81403c0 con 0x7f5ad8135cf0 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient(hunting): authenticate will time out at 1242.380886s 2024-09-15T21:22:03.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55076/0 (socket says 172.21.15.119:55076) 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 -- 172.21.15.119:0/651111785 learned_addr learned my addr 172.21.15.119:0/651111785 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 10 monclient(hunting): get_auth_request con 0x7f5ad806ec30 auth_method 0 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 -- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 msgr2=0x7f5ad813cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 -- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 msgr2=0x7f5ad813cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad6ffd640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ad80a0160 0x7f5ad813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad8135cf0 0x7f5ad81cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): get_auth_request con 0x7f5ad80a0160 auth_method 0 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): get_auth_request con 0x7f5ad8135cf0 auth_method 0 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:03.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient(hunting): handle_auth_done global_id 54447 payload 824 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 -- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad8135cf0 msgr2=0x7f5ad81cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad8135cf0 0x7f5ad81cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 -- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 msgr2=0x7f5ad813cbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 monclient: found mon.noname-a 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 -- 172.21.15.119:0/651111785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81393c0 con 0x7f5ad80a0160 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad77fe640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad8135cf0 0x7f5ad81cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acffff640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ad80a0160 0x7f5ad813d110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9b0 tx=0x7f5ac000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0017070 con 0x7f5ad80a0160 2024-09-15T21:22:03.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: dump: 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.603 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 5 monclient: authenticate success, global_id 54447 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _renew_subs 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac000c9a0 con 0x7f5ad80a0160 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad8137e80 con 0x7f5ad80a0160 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000cd90 con 0x7f5ad80a0160 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:03.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 10 monclient: dump: 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5ad5ffb640 4 set_mon_vals no callback set 2024-09-15T21:22:03.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _renew_subs 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad813d650 con 0x7f5ad80a0160 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad81360d0 con 0x7f5ad80a0160 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.580+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5ac001a070 con 0x7f5ad80a0160 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.584+0000 7f5acf7fe640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ab803b9c0 0x7f5ab803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.584+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 7621+0+0 (secure 0 0 0) 0x7f5ac0050a80 con 0x7f5ad80a0160 2024-09-15T21:22:03.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.584+0000 7f5ad6ffd640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ab803b9c0 0x7f5ab803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:03.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.584+0000 7f5ad6ffd640 10 monclient: get_auth_request con 0x7f5ab803b9c0 auth_method 0 2024-09-15T21:22:03.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.584+0000 7f5ad6ffd640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ab803b9c0 0x7f5ab803de80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8000f70 tx=0x7f5ac8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:03.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.596+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5ac0011350 con 0x7f5ad80a0160 2024-09-15T21:22:03.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.596+0000 7f5acf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:03.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.596+0000 7f5acf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:03.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.768+0000 7f5add525640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:22:03.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.768+0000 7f5add525640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:22:03.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.768+0000 7f5add525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:03.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.768+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f5ad8134af0 con 0x7f5ad80a0160 2024-09-15T21:22:03.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.924+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v135) ==== 78+0+0 (secure 0 0 0) 0x7f5ac001bdd0 con 0x7f5ad80a0160 2024-09-15T21:22:03.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.924+0000 7f5acf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:22:03.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:03.924+0000 7f5acf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:04.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.020+0000 7f5add525640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:22:04.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.020+0000 7f5add525640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:22:04.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.020+0000 7f5add525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:04.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.020+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f5ad81319e0 con 0x7f5ad80a0160 2024-09-15T21:22:04.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.928+0000 7f5acf7fe640 1 -- 172.21.15.119:0/651111785 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v136) ==== 78+0+0 (secure 0 0 0) 0x7f5ac0009300 con 0x7f5ad80a0160 2024-09-15T21:22:04.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.928+0000 7f5acf7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:22:04.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.928+0000 7f5acf7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:04.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ab803b9c0 msgr2=0x7f5ab803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:04.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ab803b9c0 0x7f5ab803de80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8000f70 tx=0x7f5ac8009040 comp rx=0 tx=0).stop 2024-09-15T21:22:04.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 10 monclient: shutdown 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ad80a0160 msgr2=0x7f5ad813d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ad80a0160 0x7f5ad813d110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9b0 tx=0x7f5ac000ee70 comp rx=0 tx=0).stop 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 shutdown_connections 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ab803b9c0 0x7f5ab803de80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ad8135cf0 0x7f5ad81cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ad80a0160 0x7f5ad813d110 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 --2- 172.21.15.119:0/651111785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad806ec30 0x7f5ad813cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 >> 172.21.15.119:0/651111785 conn(0x7f5ad80a2ae0 msgr2=0x7f5ad8130b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 shutdown_connections 2024-09-15T21:22:04.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:04.932+0000 7f5add525640 1 -- 172.21.15.119:0/651111785 wait complete. 2024-09-15T21:22:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create cool 2 2024-09-15T21:22:05.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.068+0000 7f7d1cae8640 10 monclient: get_monmap_and_config 2024-09-15T21:22:05.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.068+0000 7f7d1cae8640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.068+0000 7f7d1cae8640 10 monclient: monmap: 2024-09-15T21:22:05.073 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:05.073 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.074 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:05.071573+0000 2024-09-15T21:22:05.074 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:05.071573+0000 2024-09-15T21:22:05.074 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.074 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.074 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:05.075 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:05.075 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:05.075 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:05.075 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:05.075 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.068+0000 7f7d1cae8640 1 Processor -- start 2024-09-15T21:22:05.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.068+0000 7f7d1cae8640 1 -- start start 2024-09-15T21:22:05.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.068+0000 7f7d1cae8640 10 monclient: init 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7d180a79e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d180a1100 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 0x7f7d18140670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient(hunting): picked mon.noname-c con 0x7f7d1813e280 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d18135180 con 0x7f7d180a1100 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d180a6920 con 0x7f7d180a79e0 2024-09-15T21:22:05.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7d180a7dc0 con 0x7f7d1813e280 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 10 monclient(hunting): authenticate will time out at 1243.868239s 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7d180a79e0 0x7f7d180a4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32778/0 (socket says 172.21.15.119:32778) 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 1 -- 172.21.15.119:0/1305048123 learned_addr learned my addr 172.21.15.119:0/1305048123 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 0x7f7d18140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 10 monclient(hunting): get_auth_request con 0x7f7d1813e280 auth_method 0 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 -- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 msgr2=0x7f7d18140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 -- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 msgr2=0x7f7d18140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 0x7f7d18140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:05.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 0x7f7d18140670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008079978 0 0) 0x7f7d18135180 con 0x7f7d180a1100 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): my global_id is 54450 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d04003530 con 0x7f7d180a1100 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1796833862 0 0) 0x7f7d04003530 con 0x7f7d180a1100 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 msgr2=0x7f7d18140670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 --2- 172.21.15.119:0/1305048123 >> v2:172.21.15.119:3300/0 conn(0x7f7d1813e280 0x7f7d18140670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 >> v1:172.21.15.119:6789/0 conn(0x7f7d180a79e0 legacy=0x7f7d180a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d180a1c30 con 0x7f7d180a1100 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7d00001700 con 0x7f7d180a1100 2024-09-15T21:22:05.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: dump: 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.093 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 >> v1:172.21.15.17:6789/0 conn(0x7f7d180a1100 legacy=0x7f7d1813cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:05.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04003a60 0x7f7d04003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): picked mon.c con 0x7f7d04003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d04004810 0x7f7d04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): picked mon.b con 0x7f7d04004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d040053d0 0x7f7d0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): picked mon.a con 0x7f7d040053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7d180a7dc0 con 0x7f7d040053d0 2024-09-15T21:22:05.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0400c260 con 0x7f7d04004810 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0400c460 con 0x7f7d04003a60 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d15d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d04004810 0x7f7d04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): get_auth_request con 0x7f7d04004810 auth_method 0 2024-09-15T21:22:05.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04003a60 0x7f7d04003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d040053d0 0x7f7d0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 10 monclient(hunting): get_auth_request con 0x7f7d04003a60 auth_method 0 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient(hunting): handle_auth_done global_id 54450 payload 293 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 -- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04003a60 msgr2=0x7f7d04003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04003a60 0x7f7d04003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 -- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d040053d0 msgr2=0x7f7d0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d040053d0 0x7f7d0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 monclient: found mon.b 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 1 -- 172.21.15.119:0/1305048123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04003530 con 0x7f7d04004810 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 5 monclient: authenticate success, global_id 54450 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d1cae8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 10 monclient: get_auth_request con 0x7f7d040053d0 auth_method 0 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d040053d0 0x7f7d0400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.072+0000 7f7d16d9d640 1 -- 172.21.15.119:0/1305048123 reap_dead start 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d04004810 0x7f7d04004c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c190 tx=0x7f7d0001f2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf94ac7d1541388c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d000b5070 con 0x7f7d04004810 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: dump: 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d000b0440 con 0x7f7d04004810 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 1 -- 172.21.15.119:0/1305048123 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00007c30 con 0x7f7d04004810 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 4 set_mon_vals no callback set 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d15d9b640 10 monclient: dump: 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.103 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.104 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.104 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.104 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.104 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.104 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.104 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: shutdown 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1305048123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d04004810 msgr2=0x7f7d04004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d04004810 0x7f7d04004c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c190 tx=0x7f7d0001f2b0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1305048123 shutdown_connections 2024-09-15T21:22:05.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1305048123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d04004810 0x7f7d04004c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1305048123 >> 172.21.15.119:0/1305048123 conn(0x7f7d180a6ef0 msgr2=0x7f7d180a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1305048123 shutdown_connections 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1305048123 wait complete. 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: monmap: 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:05.106 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 Processor -- start 2024-09-15T21:22:05.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- start start 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: init 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04004810 0x7f7d181cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): picked mon.noname-c con 0x7f7d04004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): picked mon.noname-b con 0x7f7d1813d590 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:05.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d181362c0 0x7f7d18136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): picked mon.noname-a con 0x7f7d181362c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18134720 con 0x7f7d181362c0 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7d180a6c00 con 0x7f7d1813d590 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7d181348a0 con 0x7f7d04004810 2024-09-15T21:22:05.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1cae8640 10 monclient(hunting): authenticate will time out at 1243.874452s 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55098/0 (socket says 172.21.15.119:55098) 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 -- 172.21.15.119:0/1511469938 learned_addr learned my addr 172.21.15.119:0/1511469938 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 10 monclient(hunting): get_auth_request con 0x7f7d1813d590 auth_method 0 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 -- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 msgr2=0x7f7d1813da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 -- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 msgr2=0x7f7d1813da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d181362c0 0x7f7d18136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04004810 0x7f7d181cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): get_auth_request con 0x7f7d181362c0 auth_method 0 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): get_auth_request con 0x7f7d04004810 auth_method 0 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient(hunting): handle_auth_done global_id 44552 payload 824 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 -- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 msgr2=0x7f7d1813da10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 -- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d181362c0 msgr2=0x7f7d18136770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d181362c0 0x7f7d18136770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 monclient: found mon.noname-c 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 -- 172.21.15.119:0/1511469938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1813df50 con 0x7f7d04004810 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d16d9d640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d181362c0 0x7f7d18136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.076+0000 7f7d1659c640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04004810 0x7f7d181cc680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7d080012a0 tx=0x7f7d08001760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08019070 con 0x7f7d04004810 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: dump: 2024-09-15T21:22:05.113 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 5 monclient: authenticate success, global_id 44552 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 monclient: _renew_subs 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d180a15e0 con 0x7f7d04004810 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 monclient: _renew_subs 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181ccbc0 con 0x7f7d04004810 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08010510 con 0x7f7d04004810 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080108c0 con 0x7f7d04004810 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1559a640 4 set_mon_vals no callback set 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 10 monclient: dump: 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.116 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc005090 con 0x7f7d04004810 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7d0801c070 con 0x7f7d04004810 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7cf803b9c0 0x7f7cf803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(136..136 src has 1..136) ==== 7289+0+0 (secure 0 0 0) 0x7f7d08050180 con 0x7f7d04004810 2024-09-15T21:22:05.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7cf803b9c0 0x7f7cf803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0ffff640 10 monclient: get_auth_request con 0x7f7cf803b9c0 auth_method 0 2024-09-15T21:22:05.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.080+0000 7f7d0ffff640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7cf803b9c0 0x7f7cf803de80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d0001d9b0 tx=0x7f7d00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.118 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:22:05.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.092+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7d180a15e0 con 0x7f7d04004810 2024-09-15T21:22:05.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.092+0000 7f7d0effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.092+0000 7f7d0effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:05.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.248+0000 7f7d1cae8640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T21:22:05.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.248+0000 7f7d1cae8640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T21:22:05.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.248+0000 7f7d1cae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:05.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.248+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) -- 0x7f7cfc005650 con 0x7f7d04004810 2024-09-15T21:22:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient: get_monmap_and_config 2024-09-15T21:22:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient: monmap: 2024-09-15T21:22:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:05.251417+0000 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:05.251417+0000 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:05.254 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 1 Processor -- start 2024-09-15T21:22:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 1 -- start start 2024-09-15T21:22:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient: init 2024-09-15T21:22:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7e600736a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 0x7f7e6013a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient(hunting): picked mon.noname-b con 0x7f7e600a21e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7e6013aed0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e60074190 con 0x7f7e600736a0 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e600a1520 con 0x7f7e6013aed0 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7e600a1800 con 0x7f7e600a21e0 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e64de7640 10 monclient(hunting): authenticate will time out at 1248.647450s 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 0x7f7e6013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 0x7f7e6013a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48734/0 (socket says 172.21.15.17:48734) 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/39825682 learned_addr learned my addr 172.21.15.17:0/39825682 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): get_auth_request con 0x7f7e600a21e0 auth_method 0 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.249+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_done global_id 44555 payload 274 2024-09-15T21:22:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/39825682 >> v1:172.21.15.119:6789/0 conn(0x7f7e6013aed0 legacy=0x7f7e6013d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/39825682 >> v1:172.21.15.17:6789/0 conn(0x7f7e600736a0 legacy=0x7f7e60073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 1 monclient: found mon.noname-b 2024-09-15T21:22:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/39825682 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e600a1ae0 con 0x7f7e600a21e0 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5dd9b640 1 --2- 172.21.15.17:0/39825682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 0x7f7e6013a990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e480098d0 tx=0x7f7e4801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8967cc9bd5de8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 1 -- 172.21.15.17:0/39825682 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480b0070 con 0x7f7e600a21e0 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: dump: 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.266 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 5 monclient: authenticate success, global_id 44555 2024-09-15T21:22:05.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 1 -- 172.21.15.17:0/39825682 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480ab600 con 0x7f7e600a21e0 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 4 set_mon_vals no callback set 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 1 -- 172.21.15.17:0/39825682 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480ab9d0 con 0x7f7e600a21e0 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e5d59a640 10 monclient: dump: 2024-09-15T21:22:05.268 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 10 monclient: shutdown 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 -- 172.21.15.17:0/39825682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 msgr2=0x7f7e6013a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 --2- 172.21.15.17:0/39825682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 0x7f7e6013a990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e480098d0 tx=0x7f7e4801c440 comp rx=0 tx=0).stop 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 -- 172.21.15.17:0/39825682 shutdown_connections 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 --2- 172.21.15.17:0/39825682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e600a21e0 0x7f7e6013a990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 -- 172.21.15.17:0/39825682 >> 172.21.15.17:0/39825682 conn(0x7f7e600a4e90 msgr2=0x7f7e600a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 -- 172.21.15.17:0/39825682 shutdown_connections 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 1 -- 172.21.15.17:0/39825682 wait complete. 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.253+0000 7f7e64de7640 10 monclient: monmap: 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 Processor -- start 2024-09-15T21:22:05.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- start start 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: init 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 0x7f7e60135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): picked mon.noname-b con 0x7f7e600736a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 0x7f7e601379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): picked mon.noname-a con 0x7f7e600a21e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e6013aed0 0x7f7e60137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): picked mon.noname-c con 0x7f7e6013aed0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7e601419a0 con 0x7f7e600a21e0 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60141820 con 0x7f7e600736a0 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7e60141b20 con 0x7f7e6013aed0 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient(hunting): authenticate will time out at 1248.652850s 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 0x7f7e601379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 0x7f7e601379c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48838/0 (socket says 172.21.15.17:48838) 2024-09-15T21:22:05.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/4108907967 learned_addr learned my addr 172.21.15.17:0/4108907967 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): get_auth_request con 0x7f7e600a21e0 auth_method 0 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e6013aed0 0x7f7e60137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 0x7f7e60135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): get_auth_request con 0x7f7e6013aed0 auth_method 0 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 10 monclient(hunting): get_auth_request con 0x7f7e600736a0 auth_method 0 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 1 -- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 msgr2=0x7f7e60135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 1 -- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 msgr2=0x7f7e60135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 0x7f7e60135430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 0x7f7e60135430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient(hunting): handle_auth_done global_id 54456 payload 824 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e6013aed0 msgr2=0x7f7e60137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e6013aed0 0x7f7e60137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 msgr2=0x7f7e60135430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 0x7f7e60135430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 -- 172.21.15.17:0/4108907967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6013a420 con 0x7f7e600a21e0 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5ed9d640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e6013aed0 0x7f7e60137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5dd9b640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 0x7f7e601379c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e480ae040 tx=0x7f7e480191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480b0040 con 0x7f7e600a21e0 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:05.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: dump: 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 5 monclient: authenticate success, global_id 54456 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480b9070 con 0x7f7e600a21e0 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _renew_subs 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6011ec40 con 0x7f7e600a21e0 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480b4880 con 0x7f7e600a21e0 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 10 monclient: dump: 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5cd99640 4 set_mon_vals no callback set 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _renew_subs 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6009f180 con 0x7f7e600a21e0 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e60073a80 con 0x7f7e600a21e0 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7e480ab4c0 con 0x7f7e600a21e0 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e4403b9c0 0x7f7e4403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 7289+0+0 (secure 0 0 0) 0x7f7e480ea2b0 con 0x7f7e600a21e0 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e4403b9c0 0x7f7e4403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.257+0000 7f7e5e59c640 10 monclient: get_auth_request con 0x7f7e4403b9c0 auth_method 0 2024-09-15T21:22:05.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.261+0000 7f7e5e59c640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e4403b9c0 0x7f7e4403de80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e580012f0 tx=0x7f7e58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.265+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7e6013a420 con 0x7f7e600a21e0 2024-09-15T21:22:05.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.265+0000 7f7e56ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.265+0000 7f7e56ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:22:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7e600a04a0 con 0x7f7e600a21e0 2024-09-15T21:22:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e56ffd640 1 -- 172.21.15.17:0/4108907967 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f7e48016150 con 0x7f7e600a21e0 2024-09-15T21:22:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e56ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e56ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:05.517 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:22:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e4403b9c0 msgr2=0x7f7e4403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e4403b9c0 0x7f7e4403de80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e580012f0 tx=0x7f7e58009040 comp rx=0 tx=0).stop 2024-09-15T21:22:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 10 monclient: shutdown 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 msgr2=0x7f7e601379c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 0x7f7e601379c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e480ae040 tx=0x7f7e480191e0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 shutdown_connections 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7e4403b9c0 0x7f7e4403de80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7e6013aed0 0x7f7e60137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e600a21e0 0x7f7e601379c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 --2- 172.21.15.17:0/4108907967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e600736a0 0x7f7e60135430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 >> 172.21.15.17:0/4108907967 conn(0x7f7e600a4e90 msgr2=0x7f7e600a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.513+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 shutdown_connections 2024-09-15T21:22:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.517+0000 7f7e64de7640 1 -- 172.21.15.17:0/4108907967 wait complete. 2024-09-15T21:22:05.536 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:22:05.536 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:22:05.536 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:22:05.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient: get_monmap_and_config 2024-09-15T21:22:05.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient: monmap: 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:05.662237+0000 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:05.662237+0000 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:05.664 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 1 Processor -- start 2024-09-15T21:22:05.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 1 -- start start 2024-09-15T21:22:05.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient: init 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4a881378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a881383b0 0x7f4a8813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient(hunting): picked mon.noname-a con 0x7f4a881383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4a8813d2c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a880a1a20 con 0x7f4a8813d2c0 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a880a1d00 con 0x7f4a881378c0 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4a880a1fe0 con 0x7f4a881383b0 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8ca08640 10 monclient(hunting): authenticate will time out at 1249.057745s 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a881383b0 0x7f4a8813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a86d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4a8813d2c0 0x7f4a8813f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34738/0 (socket says 172.21.15.17:34738) 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a86d9d640 1 -- 172.21.15.17:0/1937861828 learned_addr learned my addr 172.21.15.17:0/1937861828 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): get_auth_request con 0x7f4a881383b0 auth_method 0 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_done global_id 54459 payload 274 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 1 -- 172.21.15.17:0/1937861828 >> v1:172.21.15.119:6789/0 conn(0x7f4a881378c0 legacy=0x7f4a88137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 1 -- 172.21.15.17:0/1937861828 >> v1:172.21.15.17:6789/0 conn(0x7f4a8813d2c0 legacy=0x7f4a8813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 1 monclient: found mon.noname-a 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8559a640 1 -- 172.21.15.17:0/1937861828 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827729900 0 0) 0x7f4a80001120 con 0x7f4a8813d2c0 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a85d9b640 1 -- 172.21.15.17:0/1937861828 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880a22c0 con 0x7f4a881383b0 2024-09-15T21:22:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.661+0000 7f4a8559a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:22:05.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a85d9b640 1 --2- 172.21.15.17:0/1937861828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881383b0 0x7f4a8813cd80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009dc0 tx=0x7f4a7801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f196b7bff59e765 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 1 -- 172.21.15.17:0/1937861828 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b1070 con 0x7f4a881383b0 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: dump: 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 1 -- 172.21.15.17:0/1937861828 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a780a4db0 con 0x7f4a881383b0 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 5 monclient: authenticate success, global_id 54459 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 4 set_mon_vals no callback set 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 1 -- 172.21.15.17:0/1937861828 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780ac660 con 0x7f4a881383b0 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8559a640 10 monclient: dump: 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.673 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 10 monclient: shutdown 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 -- 172.21.15.17:0/1937861828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881383b0 msgr2=0x7f4a8813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/1937861828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881383b0 0x7f4a8813cd80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009dc0 tx=0x7f4a7801c4f0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 -- 172.21.15.17:0/1937861828 shutdown_connections 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/1937861828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881383b0 0x7f4a8813cd80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 -- 172.21.15.17:0/1937861828 >> 172.21.15.17:0/1937861828 conn(0x7f4a880aac40 msgr2=0x7f4a880a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 -- 172.21.15.17:0/1937861828 shutdown_connections 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 -- 172.21.15.17:0/1937861828 wait complete. 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 10 monclient: monmap: 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:05.675 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 Processor -- start 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 1 -- start start 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.665+0000 7f4a8ca08640 10 monclient: init 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 0x7f4a8809ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): picked mon.noname-a con 0x7f4a881378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a881383b0 0x7f4a88136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): picked mon.noname-c con 0x7f4a881383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a8813d2c0 0x7f4a881d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): picked mon.noname-b con 0x7f4a8813d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88144280 con 0x7f4a881378c0 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88144100 con 0x7f4a8813d2c0 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a88144400 con 0x7f4a881383b0 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): authenticate will time out at 1249.064389s 2024-09-15T21:22:05.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 0x7f4a8809ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a881383b0 0x7f4a88136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 0x7f4a8809ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48860/0 (socket says 172.21.15.17:48860) 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a881383b0 0x7f4a88136920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48762/0 (socket says 172.21.15.17:48762) 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 -- 172.21.15.17:0/3395850749 learned_addr learned my addr 172.21.15.17:0/3395850749 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): get_auth_request con 0x7f4a881378c0 auth_method 0 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): get_auth_request con 0x7f4a881383b0 auth_method 0 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a86d9d640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a8813d2c0 0x7f4a881d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient(hunting): handle_auth_done global_id 54465 payload 824 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a881383b0 msgr2=0x7f4a88136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a881383b0 0x7f4a88136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a8813d2c0 msgr2=0x7f4a881d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a8813d2c0 0x7f4a881d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 monclient: found mon.noname-a 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880a0790 con 0x7f4a881378c0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a86d9d640 10 monclient: get_auth_request con 0x7f4a8813d2c0 auth_method 0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a86d9d640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a8813d2c0 0x7f4a881d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8659c640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 0x7f4a8809ef90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c450 tx=0x7f4a7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70016020 con 0x7f4a881378c0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:05.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: dump: 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a700052d0 con 0x7f4a881378c0 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70007260 con 0x7f4a881378c0 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: dump: 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.686 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a84d99640 4 set_mon_vals no callback set 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 5 monclient: authenticate success, global_id 54465 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient: _renew_subs 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a880a12f0 con 0x7f4a881378c0 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient: _renew_subs 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8809e430 con 0x7f4a881378c0 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 0x7f4a8802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8813a470 con 0x7f4a88136e60 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 0x7f4a8802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient: get_auth_request con 0x7f4a88136e60 auth_method 0 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4a70007400 con 0x7f4a881378c0 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_done global_id 54468 payload 824 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a88137ca0 con 0x7f4a88136e60 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a6803b9f0 0x7f4a6803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 7289+0+0 (secure 0 0 0) 0x7f4a7004fdb0 con 0x7f4a881378c0 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a85d9b640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 0x7f4a8802d450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a880a6150 tx=0x7f4a780195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b1040 con 0x7f4a88136e60 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: dump: 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.690 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a86d9d640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a6803b9f0 0x7f4a6803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a86d9d640 10 monclient: get_auth_request con 0x7f4a6803b9f0 auth_method 0 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4a780b2040 con 0x7f4a88136e60 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:05.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 msgr2=0x7f4a8802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 0x7f4a8802d450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a880a6150 tx=0x7f4a780195b0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 msgr2=0x7f4a8802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:05.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.669+0000 7f4a7effd640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a88136e60 0x7f4a8802d450 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a880a6150 tx=0x7f4a780195b0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.673+0000 7f4a86d9d640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a6803b9f0 0x7f4a6803deb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4a881375d0 tx=0x7f4a80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 0x7f4a8802dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88069270 con 0x7f4a8802d990 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 0x7f4a8802dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient: get_auth_request con 0x7f4a8802d990 auth_method 0 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 10 monclient(hunting): handle_auth_done global_id 54471 payload 824 2024-09-15T21:22:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 1 -- 172.21.15.17:0/3395850749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a881d1420 con 0x7f4a8802d990 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a85d9b640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 0x7f4a8802dda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a78019420 tx=0x7f4a780a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780c6070 con 0x7f4a8802d990 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 10 monclient: dump: 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f4a780c2cc0 con 0x7f4a8802d990 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 msgr2=0x7f4a8802dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 0x7f4a8802dda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a78019420 tx=0x7f4a780a6000 comp rx=0 tx=0).stop 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 msgr2=0x7f4a8802dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a7effd640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a8802d990 0x7f4a8802dda0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 51039, 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:05.713 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a6803b9f0 msgr2=0x7f4a6803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a6803b9f0 0x7f4a6803deb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4a881375d0 tx=0x7f4a80009040 comp rx=0 tx=0).stop 2024-09-15T21:22:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 10 monclient: shutdown 2024-09-15T21:22:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 msgr2=0x7f4a8809ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 0x7f4a8809ef90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c450 tx=0x7f4a7000c910 comp rx=0 tx=0).stop 2024-09-15T21:22:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a86d9d640 1 -- 172.21.15.17:0/3395850749 reap_dead start 2024-09-15T21:22:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 shutdown_connections 2024-09-15T21:22:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 --2- 172.21.15.17:0/3395850749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a881378c0 0x7f4a8809ef90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 >> 172.21.15.17:0/3395850749 conn(0x7f4a880aac40 msgr2=0x7f4a8813f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 shutdown_connections 2024-09-15T21:22:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.677+0000 7f4a8ca08640 1 -- 172.21.15.17:0/3395850749 wait complete. 2024-09-15T21:22:05.715 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:22:05.715 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:22:05.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:22:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.829+0000 7fc7bb09e640 10 monclient: get_monmap_and_config 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.829+0000 7fc7bb09e640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.829+0000 7fc7bb09e640 10 monclient: monmap: 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:05.834152+0000 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:05.834152+0000 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.836 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.837 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:05.837 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:05.837 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:05.837 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:05.837 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:05.837 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 Processor -- start 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 -- start start 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient: init 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc7b40a0890 0x7fc7b40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): picked mon.noname-a con 0x7fc7b40a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b40a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): picked mon.noname-b con 0x7fc7b409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc7b4135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b40a3130 con 0x7fc7b4135670 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b40a16f0 con 0x7fc7b40a0890 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b4073df0 con 0x7fc7b409ffc0 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7bb09e640 10 monclient(hunting): authenticate will time out at 1249.229520s 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b40a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b40a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48766/0 (socket says 172.21.15.17:48766) 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 1 -- 172.21.15.17:0/1810481079 learned_addr learned my addr 172.21.15.17:0/1810481079 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): get_auth_request con 0x7fc7b409ffc0 auth_method 0 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/1810481079 >> v2:172.21.15.17:3300/0 conn(0x7fc7b40a0890 0x7fc7b40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): get_auth_request con 0x7fc7b40a0890 auth_method 0 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_done global_id 54474 payload 274 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 1 -- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 msgr2=0x7fc7b40a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b40a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 1 -- 172.21.15.17:0/1810481079 >> v1:172.21.15.119:6789/0 conn(0x7fc7b4135670 legacy=0x7fc7b4135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 1 monclient: found mon.noname-a 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b8e3a640 1 -- 172.21.15.17:0/1810481079 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b40a1bf0 con 0x7fc7b40a0890 2024-09-15T21:22:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.833+0000 7fc7b3fff640 1 --2- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b40a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b40a0890 0x7fc7b40a0c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40a1870 tx=0x7fc7a401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d184e60395dd6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 1 -- 172.21.15.17:0/1810481079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40b0070 con 0x7fc7b40a0890 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: dump: 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.845 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 1 -- 172.21.15.17:0/1810481079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a40ab600 con 0x7fc7b40a0890 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 5 monclient: authenticate success, global_id 54474 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 4 set_mon_vals no callback set 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 1 -- 172.21.15.17:0/1810481079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40ab9d0 con 0x7fc7b40a0890 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b37fe640 10 monclient: dump: 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: shutdown 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b40a0890 msgr2=0x7fc7b40a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b40a0890 0x7fc7b40a0c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40a1870 tx=0x7fc7a401c360 comp rx=0 tx=0).stop 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/1810481079 shutdown_connections 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b40a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/1810481079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b40a0890 0x7fc7b40a0c70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/1810481079 >> 172.21.15.17:0/1810481079 conn(0x7fc7b40a66a0 msgr2=0x7fc7b40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/1810481079 shutdown_connections 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/1810481079 wait complete. 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: build_initial_monmap 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: monmap: 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:05.848 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:05.849 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 Processor -- start 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- start start 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: init 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b413c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): picked mon.noname-c con 0x7fc7b409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7b40a0890 0x7fc7b41346e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): picked mon.noname-b con 0x7fc7b40a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:05.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 0x7fc7b4134c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): picked mon.noname-a con 0x7fc7b4135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4140690 con 0x7fc7b4135670 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4140510 con 0x7fc7b40a0890 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b4140810 con 0x7fc7b409ffc0 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient(hunting): authenticate will time out at 1249.234734s 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 0x7fc7b4134c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b413c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 0x7fc7b4134c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48900/0 (socket says 172.21.15.17:48900) 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b413c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48770/0 (socket says 172.21.15.17:48770) 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 -- 172.21.15.17:0/2397585265 learned_addr learned my addr 172.21.15.17:0/2397585265 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): get_auth_request con 0x7fc7b4135670 auth_method 0 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): get_auth_request con 0x7fc7b409ffc0 auth_method 0 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b3fff640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7b40a0890 0x7fc7b41346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b3fff640 10 monclient(hunting): get_auth_request con 0x7fc7b40a0890 auth_method 0 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient(hunting): handle_auth_done global_id 54477 payload 824 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 msgr2=0x7fc7b413c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b413c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7b40a0890 msgr2=0x7fc7b41346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7b40a0890 0x7fc7b41346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 monclient: found mon.noname-a 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b409d8e0 con 0x7fc7b4135670 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b409ffc0 0x7fc7b413c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b963b640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 0x7fc7b4134c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c7c0 tx=0x7fc7ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac003b80 con 0x7fc7b4135670 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: dump: 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: _finish_auth 0 2024-09-15T21:22:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 5 monclient: authenticate success, global_id 54477 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ac003d20 con 0x7fc7b4135670 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00aa30 con 0x7fc7b4135670 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: _renew_subs 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b4072750 con 0x7fc7b4135670 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b2ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7b17fa640 10 monclient: dump: 2024-09-15T21:22:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: _renew_subs 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.837+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b4121090 con 0x7fc7b4135670 2024-09-15T21:22:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7bb09e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 0x7fc7b41cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b4135aa0 con 0x7fc7b41cf820 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 0x7fc7b41cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient: get_auth_request con 0x7fc7b41cf820 auth_method 0 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc7ac00abd0 con 0x7fc7b4135670 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_done global_id 44570 payload 824 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7b40008e0 con 0x7fc7b41cf820 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc79403b9c0 0x7fc79403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 7289+0+0 (secure 0 0 0) 0x7fc7ac050340 con 0x7fc7b4135670 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 0x7fc7b41cfc00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7b406f1c0 tx=0x7fc7a401aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40bb030 con 0x7fc7b41cf820 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 10 monclient: dump: 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b3fff640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc79403b9c0 0x7fc79403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b3fff640 10 monclient: get_auth_request con 0x7fc79403b9c0 auth_method 0 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc7a40b6530 con 0x7fc7b41cf820 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 msgr2=0x7fc7b41cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 0x7fc7b41cfc00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7b406f1c0 tx=0x7fc7a401aba0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 msgr2=0x7fc7b41cfc00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b17fa640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41cf820 0x7fc7b41cfc00 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7b406f1c0 tx=0x7fc7a401aba0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.841+0000 7fc7b3fff640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc79403b9c0 0x7fc79403de80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4074330 tx=0x7fc79c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.845+0000 7fc7bb09e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.845+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 0x7fc7b4138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.845+0000 7fc7bb09e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.845+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b41d0520 con 0x7fc7b41d0140 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 0x7fc7b4138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient: get_auth_request con 0x7fc7b41d0140 auth_method 0 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 10 monclient(hunting): handle_auth_done global_id 44573 payload 824 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 1 -- 172.21.15.17:0/2397585265 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc7b41d06d0 con 0x7fc7b41d0140 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b8e3a640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 0x7fc7b4138ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401c3a0 tx=0x7fc7a4019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40bb040 con 0x7fc7b41d0140 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 10 monclient: dump: 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7fc7a40c1810 con 0x7fc7b41d0140 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 msgr2=0x7fc7b4138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 0x7fc7b4138ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401c3a0 tx=0x7fc7a4019100 comp rx=0 tx=0).stop 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 msgr2=0x7fc7b4138ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b17fa640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7b41d0140 0x7fc7b4138ab0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401c3a0 tx=0x7fc7a4019100 comp rx=0 tx=0).stop 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 94447, 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:22:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:05.882 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc79403b9c0 msgr2=0x7fc79403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc79403b9c0 0x7fc79403de80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4074330 tx=0x7fc79c008040 comp rx=0 tx=0).stop 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 10 monclient: shutdown 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 msgr2=0x7fc7b4134c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 0x7fc7b4134c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c7c0 tx=0x7fc7ac00cc80 comp rx=0 tx=0).stop 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7b963b640 1 -- 172.21.15.17:0/2397585265 reap_dead start 2024-09-15T21:22:05.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 shutdown_connections 2024-09-15T21:22:05.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 --2- 172.21.15.17:0/2397585265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7b4135670 0x7fc7b4134c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:05.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 >> 172.21.15.17:0/2397585265 conn(0x7fc7b40a66a0 msgr2=0x7fc7b4072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:05.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 shutdown_connections 2024-09-15T21:22:05.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:05.849+0000 7fc7bb09e640 1 -- 172.21.15.17:0/2397585265 wait complete. 2024-09-15T21:22:05.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:22:05.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:22:05.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:22:05.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.948+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' created v137) ==== 111+0+0 (secure 0 0 0) 0x7f7d08011cf0 con 0x7f7d04004810 2024-09-15T21:22:05.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.948+0000 7f7d0effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T21:22:05.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:05.948+0000 7f7d0effd640 10 monclient: _finish_command 2 = system:0 pool 'cool' created 2024-09-15T21:22:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient: get_monmap_and_config 2024-09-15T21:22:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient: build_initial_monmap 2024-09-15T21:22:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient: monmap: 2024-09-15T21:22:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:06.007804+0000 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:06.007804+0000 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:06.010 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 Processor -- start 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 -- start start 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient: init 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2a01378c0 0x7fd2a0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): picked mon.noname-c con 0x7fd2a01378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01381e0 0x7fd2a00a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): picked mon.noname-b con 0x7fd2a01381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 0x7fd2a00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): picked mon.noname-a con 0x7fd2a00a5040 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a00a1d00 con 0x7fd2a00a5040 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a00a1fe0 con 0x7fd2a01378c0 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a00a22c0 con 0x7fd2a01381e0 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd2a4e46640 10 monclient(hunting): authenticate will time out at 1249.403779s 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.005+0000 7fd29ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 0x7fd2a00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 0x7fd2a00a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48904/0 (socket says 172.21.15.17:48904) 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 1 -- 172.21.15.17:0/2877585804 learned_addr learned my addr 172.21.15.17:0/2877585804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): get_auth_request con 0x7fd2a00a5040 auth_method 0 2024-09-15T21:22:06.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2877585804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01381e0 0x7fd2a00a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): get_auth_request con 0x7fd2a01381e0 auth_method 0 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.119:3300/0 conn(0x7fd2a01378c0 0x7fd2a0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29e59c640 10 monclient(hunting): get_auth_request con 0x7fd2a01378c0 auth_method 0 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_done global_id 44576 payload 274 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2877585804 >> v2:172.21.15.119:3300/0 conn(0x7fd2a01378c0 msgr2=0x7fd2a0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.119:3300/0 conn(0x7fd2a01378c0 0x7fd2a0137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2877585804 >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 msgr2=0x7fd2a00a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 0x7fd2a00a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 monclient: found mon.noname-b 2024-09-15T21:22:06.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:06.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2877585804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a00a4490 con 0x7fd2a01381e0 2024-09-15T21:22:06.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29ed9d640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 0x7fd2a00a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:06.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.009+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.119:3300/0 conn(0x7fd2a01378c0 0x7fd2a0137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:22:06.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2877585804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01381e0 0x7fd2a00a4b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00ab150 tx=0x7fd29001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41c52e40e9e6e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 1 -- 172.21.15.17:0/2877585804 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900b0070 con 0x7fd2a01381e0 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: dump: 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.025 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: _finish_auth 0 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 1 -- 172.21.15.17:0/2877585804 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2900ab600 con 0x7fd2a01381e0 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 1 -- 172.21.15.17:0/2877585804 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900ab9d0 con 0x7fd2a01381e0 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 5 monclient: authenticate success, global_id 44576 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 4 set_mon_vals no callback set 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29d59a640 10 monclient: dump: 2024-09-15T21:22:06.026 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: shutdown 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2877585804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01381e0 msgr2=0x7fd2a00a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2877585804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01381e0 0x7fd2a00a4b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00ab150 tx=0x7fd29001c340 comp rx=0 tx=0).stop 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2877585804 shutdown_connections 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.17:3300/0 conn(0x7fd2a00a5040 0x7fd2a00a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2877585804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01381e0 0x7fd2a00a4b00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2877585804 >> v2:172.21.15.119:3300/0 conn(0x7fd2a01378c0 0x7fd2a0137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2877585804 >> 172.21.15.17:0/2877585804 conn(0x7fd2a00aac40 msgr2=0x7fd2a00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2877585804 shutdown_connections 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2877585804 wait complete. 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: build_initial_monmap 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: monmap: 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:06.028 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 Processor -- start 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- start start 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: init 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a00a5040 0x7fd2a01cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): picked mon.noname-b con 0x7fd2a00a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01378c0 0x7fd2a0136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): picked mon.noname-c con 0x7fd2a01378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a01381e0 0x7fd2a013cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): picked mon.noname-a con 0x7fd2a01381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0141730 con 0x7fd2a01381e0 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01415b0 con 0x7fd2a00a5040 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a01418b0 con 0x7fd2a01378c0 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd2a4e46640 10 monclient(hunting): authenticate will time out at 1249.411111s 2024-09-15T21:22:06.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01378c0 0x7fd2a0136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a01381e0 0x7fd2a013cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01378c0 0x7fd2a0136d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48820/0 (socket says 172.21.15.17:48820) 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2236908740 learned_addr learned my addr 172.21.15.17:0/2236908740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): get_auth_request con 0x7fd2a01378c0 auth_method 0 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): get_auth_request con 0x7fd2a01381e0 auth_method 0 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a00a5040 0x7fd2a01cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29e59c640 10 monclient(hunting): get_auth_request con 0x7fd2a00a5040 auth_method 0 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient(hunting): handle_auth_done global_id 44579 payload 824 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a00a5040 msgr2=0x7fd2a01cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a00a5040 0x7fd2a01cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a01381e0 msgr2=0x7fd2a013cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a01381e0 0x7fd2a013cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 monclient: found mon.noname-c 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a01215a0 con 0x7fd2a01378c0 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd29dd9b640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01378c0 0x7fd2a0136d20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd29001cbe0 tx=0x7fd2900190e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.013+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900ba070 con 0x7fd2a01378c0 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: dump: 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.033 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: _finish_auth 0 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2900b5460 con 0x7fd2a01378c0 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900b0040 con 0x7fd2a01378c0 2024-09-15T21:22:06.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: dump: 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29cd99640 4 set_mon_vals no callback set 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 5 monclient: authenticate success, global_id 44579 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 10 monclient: _renew_subs 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a0121e20 con 0x7fd2a01378c0 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 10 monclient: _renew_subs 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a01226c0 con 0x7fd2a01378c0 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 0x7fd2a013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0137ca0 con 0x7fd2a0120ec0 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd2900bd070 con 0x7fd2a01378c0 2024-09-15T21:22:06.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 0x7fd2a013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd28003b9f0 0x7fd28003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient: get_auth_request con 0x7fd2a0120ec0 auth_method 0 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(137..137 src has 1..137) ==== 7632+0+0 (secure 0 0 0) 0x7fd2900eaa30 con 0x7fd2a01378c0 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29ed9d640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd28003b9f0 0x7fd28003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29ed9d640 10 monclient: get_auth_request con 0x7fd28003b9f0 auth_method 0 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 10 monclient(hunting): handle_auth_done global_id 64108 payload 824 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2a00a1270 con 0x7fd2a0120ec0 2024-09-15T21:22:06.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 0x7fd2a013d520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2940107c0 tx=0x7fd294010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd29ed9d640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd28003b9f0 0x7fd28003deb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd298009c90 tx=0x7fd29800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29401a030 con 0x7fd2a0120ec0 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: dump: 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd294004020 con 0x7fd2a0120ec0 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 msgr2=0x7fd2a013d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 0x7fd2a013d520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2940107c0 tx=0x7fd294010c80 comp rx=0 tx=0).stop 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 msgr2=0x7fd2a013d520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.017+0000 7fd28effd640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0120ec0 0x7fd2a013d520 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2940107c0 tx=0x7fd294010c80 comp rx=0 tx=0).stop 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd2a4e46640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 0x7fd2a013da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd2a4e46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0069270 con 0x7fd2a0137260 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 0x7fd2a013da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient: get_auth_request con 0x7fd2a0137260 auth_method 0 2024-09-15T21:22:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 10 monclient(hunting): handle_auth_done global_id 64111 payload 824 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 1 -- 172.21.15.17:0/2236908740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd2a006cba0 con 0x7fd2a0137260 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd29e59c640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 0x7fd2a013da60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd294008010 tx=0x7fd2940046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29400f040 con 0x7fd2a0137260 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 10 monclient: dump: 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7fd294003620 con 0x7fd2a0137260 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 msgr2=0x7fd2a013da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 0x7fd2a013da60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd294008010 tx=0x7fd2940046d0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 msgr2=0x7fd2a013da60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.025+0000 7fd28effd640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2a0137260 0x7fd2a013da60 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd294008010 tx=0x7fd2940046d0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-15T21:22:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 4287, 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:06.057 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd28003b9f0 msgr2=0x7fd28003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd28003b9f0 0x7fd28003deb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd298009c90 tx=0x7fd29800b040 comp rx=0 tx=0).stop 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 10 monclient: shutdown 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01378c0 msgr2=0x7fd2a0136d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 --2- 172.21.15.17:0/2236908740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2a01378c0 0x7fd2a0136d20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd29001cbe0 tx=0x7fd2900190e0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd29ed9d640 1 -- 172.21.15.17:0/2236908740 reap_dead start 2024-09-15T21:22:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd29ed9d640 1 -- 172.21.15.17:0/2236908740 reap_dead start 2024-09-15T21:22:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 shutdown_connections 2024-09-15T21:22:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 >> 172.21.15.17:0/2236908740 conn(0x7fd2a00aac40 msgr2=0x7fd2a006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 shutdown_connections 2024-09-15T21:22:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.029+0000 7fd2a4e46640 1 -- 172.21.15.17:0/2236908740 wait complete. 2024-09-15T21:22:06.060 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:22:06.060 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:22:06.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:22:06.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d1cae8640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T21:22:06.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d1cae8640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T21:22:06.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d1cae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) -- 0x7f7cfc005b40 con 0x7f7d04004810 2024-09-15T21:22:06.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d0effd640 1 -- 172.21.15.119:0/1511469938 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' already exists v137) ==== 118+0+0 (secure 0 0 0) 0x7f7d0804d1c0 con 0x7f7d04004810 2024-09-15T21:22:06.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d0effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T21:22:06.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.056+0000 7f7d0effd640 10 monclient: _finish_command 3 = system:0 pool 'cool' already exists 2024-09-15T21:22:06.062 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'cool' already exists 2024-09-15T21:22:06.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7cf803b9c0 msgr2=0x7f7cf803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7cf803b9c0 0x7f7cf803de80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d0001d9b0 tx=0x7f7d00009040 comp rx=0 tx=0).stop 2024-09-15T21:22:06.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 10 monclient: shutdown 2024-09-15T21:22:06.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04004810 msgr2=0x7f7d181cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04004810 0x7f7d181cc680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7d080012a0 tx=0x7f7d08001760 comp rx=0 tx=0).stop 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 shutdown_connections 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7cf803b9c0 0x7f7cf803de80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d181362c0 0x7f7d18136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d1813d590 0x7f7d1813da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 --2- 172.21.15.119:0/1511469938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7d04004810 0x7f7d181cc680 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 >> 172.21.15.119:0/1511469938 conn(0x7f7d180a6ef0 msgr2=0x7f7d18133020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 shutdown_connections 2024-09-15T21:22:06.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.060+0000 7f7d1cae8640 1 -- 172.21.15.119:0/1511469938 wait complete. 2024-09-15T21:22:06.079 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool create cold 2 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.169+0000 7fa2fed6b640 10 monclient: get_monmap_and_config 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.169+0000 7fa2fed6b640 10 monclient: build_initial_monmap 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.169+0000 7fa2fed6b640 10 monclient: monmap: 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:06.173573+0000 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:06.173573+0000 2024-09-15T21:22:06.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:06.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 Processor -- start 2024-09-15T21:22:06.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 -- start start 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient: init 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 0x7fa2f8137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): picked mon.noname-b con 0x7fa2f81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa2f81381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2f80a5210 0x7fa2f80a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): picked mon.noname-c con 0x7fa2f80a5210 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f80a1d00 con 0x7fa2f81381e0 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f80a1fe0 con 0x7fa2f80a5210 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f80a22c0 con 0x7fa2f81378c0 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fed6b640 10 monclient(hunting): authenticate will time out at 1249.569013s 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 0x7fa2f8137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 0x7fa2f8137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48826/0 (socket says 172.21.15.17:48826) 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 -- 172.21.15.17:0/3188038152 learned_addr learned my addr 172.21.15.17:0/3188038152 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): get_auth_request con 0x7fa2f81378c0 auth_method 0 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fd308640 1 --2- 172.21.15.17:0/3188038152 >> v2:172.21.15.119:3300/0 conn(0x7fa2f80a5210 0x7fa2f80a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fd308640 10 monclient(hunting): get_auth_request con 0x7fa2f80a5210 auth_method 0 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fd308640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fd308640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fd308640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_done global_id 44582 payload 274 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 -- 172.21.15.17:0/3188038152 >> v2:172.21.15.119:3300/0 conn(0x7fa2f80a5210 msgr2=0x7fa2f80a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 --2- 172.21.15.17:0/3188038152 >> v2:172.21.15.119:3300/0 conn(0x7fa2f80a5210 0x7fa2f80a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 -- 172.21.15.17:0/3188038152 >> v1:172.21.15.119:6789/0 conn(0x7fa2f81381e0 legacy=0x7fa2f80a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient: _finish_hunting 0 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 monclient: found mon.noname-b 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.173+0000 7fa2fcb07640 1 -- 172.21.15.17:0/3188038152 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f80a5c00 con 0x7fa2f81378c0 2024-09-15T21:22:06.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 1 --2- 172.21.15.17:0/3188038152 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 0x7fa2f8137ca0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009870 tx=0x7fa2ec01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6842d9bdd8c09c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 1 -- 172.21.15.17:0/3188038152 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0b0070 con 0x7fa2f81378c0 2024-09-15T21:22:06.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: dump: 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 5 monclient: authenticate success, global_id 44582 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 1 -- 172.21.15.17:0/3188038152 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec0ab600 con 0x7fa2f81378c0 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 1 -- 172.21.15.17:0/3188038152 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0ab9d0 con 0x7fa2f81378c0 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 4 set_mon_vals no callback set 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f77fe640 10 monclient: dump: 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: shutdown 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- 172.21.15.17:0/3188038152 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 msgr2=0x7fa2f8137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/3188038152 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 0x7fa2f8137ca0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009870 tx=0x7fa2ec01c420 comp rx=0 tx=0).stop 2024-09-15T21:22:06.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- 172.21.15.17:0/3188038152 shutdown_connections 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/3188038152 >> v2:172.21.15.119:3300/0 conn(0x7fa2f80a5210 0x7fa2f80a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/3188038152 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81378c0 0x7fa2f8137ca0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- 172.21.15.17:0/3188038152 >> 172.21.15.17:0/3188038152 conn(0x7fa2f80aac40 msgr2=0x7fa2f80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- 172.21.15.17:0/3188038152 shutdown_connections 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- 172.21.15.17:0/3188038152 wait complete. 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: build_initial_monmap 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: monmap: 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 Processor -- start 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- start start 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: init 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2f80a5210 0x7fa2f81cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): picked mon.noname-a con 0x7fa2f80a5210 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f81378c0 0x7fa2f8136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): picked mon.noname-b con 0x7fa2f81378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 0x7fa2f813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): picked mon.noname-c con 0x7fa2f81381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f806f340 con 0x7fa2f80a5210 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f806f1c0 con 0x7fa2f81378c0 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f81425c0 con 0x7fa2f81381e0 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fed6b640 10 monclient(hunting): authenticate will time out at 1249.575181s 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 0x7fa2f813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2f80a5210 0x7fa2f81cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 0x7fa2f813cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48828/0 (socket says 172.21.15.17:48828) 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 -- 172.21.15.17:0/1455861809 learned_addr learned my addr 172.21.15.17:0/1455861809 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f81378c0 0x7fa2f8136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): get_auth_request con 0x7fa2f80a5210 auth_method 0 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): get_auth_request con 0x7fa2f81381e0 auth_method 0 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): get_auth_request con 0x7fa2f81378c0 auth_method 0 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fcb07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient(hunting): handle_auth_done global_id 44585 payload 824 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 -- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f81378c0 msgr2=0x7fa2f8136b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f81378c0 0x7fa2f8136b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 -- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2f80a5210 msgr2=0x7fa2f81cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2f80a5210 0x7fa2f81cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient: _finish_hunting 0 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 monclient: found mon.noname-c 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 -- 172.21.15.17:0/1455861809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f8120750 con 0x7fa2f81381e0 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.177+0000 7fa2fd308640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 0x7fa2f813cd00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0007e70 tx=0x7fa2f000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0014620 con 0x7fa2f81381e0 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: dump: 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: _finish_auth 0 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f0019070 con 0x7fa2f81381e0 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0014a20 con 0x7fa2f81381e0 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 5 monclient: authenticate success, global_id 44585 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 monclient: _renew_subs 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f80a8bc0 con 0x7fa2f81381e0 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f6ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 10 monclient: dump: 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 monclient: _renew_subs 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81214b0 con 0x7fa2f81381e0 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f80a5680 con 0x7fa2f81381e0 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa2f0014bc0 con 0x7fa2f81381e0 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2dc03b9c0 0x7fa2dc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(137..137 src has 1..137) ==== 7632+0+0 (secure 0 0 0) 0x7fa2f0050e00 con 0x7fa2f81381e0 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fcb07640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2dc03b9c0 0x7fa2dc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fcb07640 10 monclient: get_auth_request con 0x7fa2dc03b9c0 auth_method 0 2024-09-15T21:22:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.181+0000 7fa2fcb07640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2dc03b9c0 0x7fa2dc03de80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0ad000 tx=0x7fa2ec0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.185+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa2f80a8bc0 con 0x7fa2f81381e0 2024-09-15T21:22:06.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.185+0000 7fa2f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.185+0000 7fa2f57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:06.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.180+0000 7f0a03825640 10 monclient: get_monmap_and_config 2024-09-15T21:22:06.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.180+0000 7f0a03825640 10 monclient: build_initial_monmap 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.180+0000 7f0a03825640 10 monclient: monmap: 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:06.183986+0000 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:06.183986+0000 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:06.209 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.180+0000 7f0a03825640 1 Processor -- start 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.180+0000 7f0a03825640 1 -- start start 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.180+0000 7f0a03825640 10 monclient: init 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09fc137fe0 0x7f09fc1383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): picked mon.noname-c con 0x7f09fc137fe0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f09fc0a2320 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09fc0a5570 0x7f09fc13fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): picked mon.noname-a con 0x7f09fc0a5570 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09fc0a3e10 con 0x7f09fc0a2320 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f09fc0a1400 con 0x7f09fc0a5570 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f09fc0a1960 con 0x7f09fc137fe0 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient(hunting): authenticate will time out at 1244.980436s 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09fc137fe0 0x7f09fc1383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09fc137fe0 0x7f09fc1383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55116/0 (socket says 172.21.15.119:55116) 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 -- 172.21.15.119:0/2168985583 learned_addr learned my addr 172.21.15.119:0/2168985583 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): get_auth_request con 0x7f09fc137fe0 auth_method 0 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 1 --2- 172.21.15.119:0/2168985583 >> v2:172.21.15.17:3300/0 conn(0x7f09fc0a5570 0x7f09fc13fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): get_auth_request con 0x7f09fc0a5570 auth_method 0 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient(hunting): handle_auth_done global_id 64120 payload 274 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 -- 172.21.15.119:0/2168985583 >> v2:172.21.15.17:3300/0 conn(0x7f09fc0a5570 msgr2=0x7f09fc13fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 --2- 172.21.15.119:0/2168985583 >> v2:172.21.15.17:3300/0 conn(0x7f09fc0a5570 0x7f09fc13fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 -- 172.21.15.119:0/2168985583 >> v1:172.21.15.17:6789/0 conn(0x7f09fc0a2320 legacy=0x7f09fc0a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient: _finish_hunting 0 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 monclient: found mon.noname-c 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 -- 172.21.15.119:0/2168985583 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc0a6bf0 con 0x7f09fc137fe0 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a015c1640 1 --2- 172.21.15.119:0/2168985583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc1383c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0a8020 tx=0x7f09f801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fe601d5cba5c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 1 -- 172.21.15.119:0/2168985583 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b0070 con 0x7f09fc137fe0 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: dump: 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.213 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: _finish_auth 0 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 5 monclient: authenticate success, global_id 64120 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:06.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 1 -- 172.21.15.119:0/2168985583 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f80ab600 con 0x7f09fc137fe0 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 4 set_mon_vals no callback set 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 1 -- 172.21.15.119:0/2168985583 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80ab9d0 con 0x7f09fc137fe0 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f09effff640 10 monclient: dump: 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.215 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 10 monclient: shutdown 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- 172.21.15.119:0/2168985583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 msgr2=0x7f09fc1383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 --2- 172.21.15.119:0/2168985583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc1383c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0a8020 tx=0x7f09f801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- 172.21.15.119:0/2168985583 shutdown_connections 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 --2- 172.21.15.119:0/2168985583 >> v2:172.21.15.17:3300/0 conn(0x7f09fc0a5570 0x7f09fc13fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 --2- 172.21.15.119:0/2168985583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc1383c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- 172.21.15.119:0/2168985583 >> 172.21.15.119:0/2168985583 conn(0x7f09fc0a6f60 msgr2=0x7f09fc0a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- 172.21.15.119:0/2168985583 shutdown_connections 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.184+0000 7f0a03825640 1 -- 172.21.15.119:0/2168985583 wait complete. 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: build_initial_monmap 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: monmap: 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:06.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 Processor -- start 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- start start 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: init 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09fc0a2320 0x7f09fc1367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): picked mon.noname-c con 0x7f09fc0a2320 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09fc0a5570 0x7f09fc136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): picked mon.noname-a con 0x7f09fc0a5570 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc13cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): picked mon.noname-b con 0x7f09fc137fe0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc0a64d0 con 0x7f09fc0a5570 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc0a6350 con 0x7f09fc137fe0 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc142be0 con 0x7f09fc0a2320 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient(hunting): authenticate will time out at 1244.985127s 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc13cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc13cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55120/0 (socket says 172.21.15.119:55120) 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 -- 172.21.15.119:0/3399449330 learned_addr learned my addr 172.21.15.119:0/3399449330 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): get_auth_request con 0x7f09fc137fe0 auth_method 0 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09fc0a2320 0x7f09fc1367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09fc0a5570 0x7f09fc136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): get_auth_request con 0x7f09fc0a5570 auth_method 0 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): get_auth_request con 0x7f09fc0a2320 auth_method 0 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:06.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a00dc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient(hunting): handle_auth_done global_id 64123 payload 824 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 -- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09fc0a2320 msgr2=0x7f09fc1367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09fc0a2320 0x7f09fc1367c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 -- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09fc0a5570 msgr2=0x7f09fc136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09fc0a5570 0x7f09fc136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient: _finish_hunting 0 2024-09-15T21:22:06.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 monclient: found mon.noname-b 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 -- 172.21.15.119:0/3399449330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc1215a0 con 0x7f09fc137fe0 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a01dc2640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc13cbd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09f400efc0 tx=0x7f09f400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4018020 con 0x7f09fc137fe0 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: dump: 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.222 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 5 monclient: authenticate success, global_id 64123 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f4014480 con 0x7f09fc137fe0 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _renew_subs 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:06.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc0a1650 con 0x7f09fc137fe0 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4014870 con 0x7f09fc137fe0 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 10 monclient: dump: 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:06.224 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _renew_subs 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09ef7fe640 4 set_mon_vals no callback set 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc13fda0 con 0x7f09fc137fe0 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc1383c0 con 0x7f09fc137fe0 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f09f401b020 con 0x7f09fc137fe0 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f09e003b9f0 0x7f09e003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:06.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 7632+0+0 (secure 0 0 0) 0x7f09f40511a0 con 0x7f09fc137fe0 2024-09-15T21:22:06.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f09e003b9f0 0x7f09e003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:06.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 10 monclient: get_auth_request con 0x7f09e003b9f0 auth_method 0 2024-09-15T21:22:06.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.188+0000 7f0a015c1640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f09e003b9f0 0x7f09e003deb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ad000 tx=0x7f09f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:06.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.192+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f09f4022070 con 0x7f09fc137fe0 2024-09-15T21:22:06.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.192+0000 7f09edffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:06.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.192+0000 7f09edffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:06.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.352+0000 7f0a03825640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T21:22:06.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.352+0000 7f0a03825640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T21:22:06.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.352+0000 7f0a03825640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:06.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.352+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) -- 0x7f09fc0a10a0 con 0x7f09fc137fe0 2024-09-15T21:22:06.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.421+0000 7fa2fed6b640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:22:06.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.421+0000 7fa2fed6b640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:22:06.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.421+0000 7fa2fed6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:06.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.421+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa2f8121130 con 0x7fa2f81381e0 2024-09-15T21:22:06.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.425+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.425+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=557650287,config=3541143443,health=1258394751,kv=3079713637,logm=145565528}) 2024-09-15T21:22:06.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.425+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=1251264863,config=3541143443,health=1258394751,kv=1618701482,logm=1142480423}) 2024-09-15T21:22:06.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.425+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.425+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=557650287,config=3541143443,health=1258394751,kv=3079713637,logm=145565528}) 2024-09-15T21:22:06.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.425+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=1,health=12,kv=11,logm=63} crc {auth=557650287,config=1228700967,health=1258394751,kv=2290493906,logm=1835939535}) 2024-09-15T21:22:06.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.425+0000 7fa2f57fa640 1 -- 172.21.15.17:0/1455861809 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa2f8120750 con 0x7fa2f81381e0 2024-09-15T21:22:06.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.425+0000 7fa2f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:22:06.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.425+0000 7fa2f57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:06.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2dc03b9c0 msgr2=0x7fa2dc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2dc03b9c0 0x7fa2dc03de80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0ad000 tx=0x7fa2ec0a6000 comp rx=0 tx=0).stop 2024-09-15T21:22:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 10 monclient: shutdown 2024-09-15T21:22:06.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2344856988}) 2024-09-15T21:22:06.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3167011237}) 2024-09-15T21:22:06.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 msgr2=0x7fa2f813cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 0x7fa2f813cd00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0007e70 tx=0x7fa2f000e4b0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 shutdown_connections 2024-09-15T21:22:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2dc03b9c0 0x7fa2dc03de80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f81381e0 0x7fa2f813cd00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2344856988}) 2024-09-15T21:22:06.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=243678928}) 2024-09-15T21:22:06.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f81378c0 0x7fa2f8136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 --2- 172.21.15.17:0/1455861809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2f80a5210 0x7fa2f81cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:06.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 >> 172.21.15.17:0/1455861809 conn(0x7fa2f80aac40 msgr2=0x7fa2f806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:06.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 shutdown_connections 2024-09-15T21:22:06.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:06.429+0000 7fa2fed6b640 1 -- 172.21.15.17:0/1455861809 wait complete. 2024-09-15T21:22:06.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=768919968}) 2024-09-15T21:22:06.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=223390559}) 2024-09-15T21:22:06.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=768919968}) 2024-09-15T21:22:06.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.429+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1668172618}) 2024-09-15T21:22:06.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3842374158}) 2024-09-15T21:22:06.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=969837561}) 2024-09-15T21:22:06.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3842374158}) 2024-09-15T21:22:06.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2568505321}) 2024-09-15T21:22:06.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520215267}) 2024-09-15T21:22:06.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=749033001}) 2024-09-15T21:22:06.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520215267}) 2024-09-15T21:22:06.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2798102902}) 2024-09-15T21:22:06.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3974382891}) 2024-09-15T21:22:06.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3627016320}) 2024-09-15T21:22:06.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3974382891}) 2024-09-15T21:22:06.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.433+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1747660509}) 2024-09-15T21:22:06.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=544300076}) 2024-09-15T21:22:06.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2955341174}) 2024-09-15T21:22:06.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=544300076}) 2024-09-15T21:22:06.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2698372923}) 2024-09-15T21:22:06.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3125034133}) 2024-09-15T21:22:06.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3370135471}) 2024-09-15T21:22:06.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3125034133}) 2024-09-15T21:22:06.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.437+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3363580359}) 2024-09-15T21:22:06.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1137964006}) 2024-09-15T21:22:06.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1260850272}) 2024-09-15T21:22:06.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1137964006}) 2024-09-15T21:22:06.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1113304672}) 2024-09-15T21:22:06.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=398398601}) 2024-09-15T21:22:06.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1826458606}) 2024-09-15T21:22:06.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=398398601}) 2024-09-15T21:22:06.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.441+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3758845113}) 2024-09-15T21:22:06.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1167521397}) 2024-09-15T21:22:06.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2246750108}) 2024-09-15T21:22:06.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1167521397}) 2024-09-15T21:22:06.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2695227396}) 2024-09-15T21:22:06.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483606500}) 2024-09-15T21:22:06.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902232512}) 2024-09-15T21:22:06.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483606500}) 2024-09-15T21:22:06.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.445+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1798317049}) 2024-09-15T21:22:06.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:22:06.451 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3400123701}) 2024-09-15T21:22:06.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1205164970}) 2024-09-15T21:22:06.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3400123701}) 2024-09-15T21:22:06.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2748291414}) 2024-09-15T21:22:06.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=638340680}) 2024-09-15T21:22:06.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3812550700}) 2024-09-15T21:22:06.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=638340680}) 2024-09-15T21:22:06.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1227083301}) 2024-09-15T21:22:06.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603523109}) 2024-09-15T21:22:06.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2815409364}) 2024-09-15T21:22:06.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.449+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603523109}) 2024-09-15T21:22:06.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3159733968}) 2024-09-15T21:22:06.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2890296772}) 2024-09-15T21:22:06.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=414123493}) 2024-09-15T21:22:06.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2890296772}) 2024-09-15T21:22:06.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.453+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3172597118}) 2024-09-15T21:22:06.459 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610748601}) 2024-09-15T21:22:06.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2531571}) 2024-09-15T21:22:06.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610748601}) 2024-09-15T21:22:06.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3495132930}) 2024-09-15T21:22:06.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=761466036}) 2024-09-15T21:22:06.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4037286388}) 2024-09-15T21:22:06.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=761466036}) 2024-09-15T21:22:06.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.457+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4094042727}) 2024-09-15T21:22:06.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=682266586}) 2024-09-15T21:22:06.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=574227029}) 2024-09-15T21:22:06.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=682266586}) 2024-09-15T21:22:06.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=539907977}) 2024-09-15T21:22:06.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3683603290}) 2024-09-15T21:22:06.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4008428192}) 2024-09-15T21:22:06.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3683603290}) 2024-09-15T21:22:06.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.461+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1680580282}) 2024-09-15T21:22:06.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.465+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.465+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698831622}) 2024-09-15T21:22:06.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.465+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3136102841}) 2024-09-15T21:22:06.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.465+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.465+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698831622}) 2024-09-15T21:22:06.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.465+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3080724194}) 2024-09-15T21:22:06.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1255564321}) 2024-09-15T21:22:06.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1108180715}) 2024-09-15T21:22:06.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1255564321}) 2024-09-15T21:22:06.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2537768493}) 2024-09-15T21:22:06.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1621614930}) 2024-09-15T21:22:06.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3221317694}) 2024-09-15T21:22:06.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1621614930}) 2024-09-15T21:22:06.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.469+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3441148118}) 2024-09-15T21:22:06.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=11,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=37} crc {logm=1188160335,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3893084702,mon_config_key=2252869222,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=929694421,osd_snap=841190635,osdmap=4214124007}) 2024-09-15T21:22:06.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=11,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=38} crc {logm=1188160335,mdsmap=3009587245,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3893084702,mon_config_key=2203944935,monmap=186996059,nvmeofgw=3517310312,osd_metadata=1154780841,osd_pg_creating=929694421,osd_snap=841190635,osdmap=2462530081}) 2024-09-15T21:22:06.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=11,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=37} crc {logm=1188160335,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3893084702,mon_config_key=2252869222,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=929694421,osd_snap=841190635,osdmap=4214124007}) 2024-09-15T21:22:06.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=39} crc {logm=2180586719,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3893084702,mon_config_key=1866194834,monmap=3140514009,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=929694421,osd_snap=841190635,osdmap=4254114808}) 2024-09-15T21:22:06.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=497470879}) 2024-09-15T21:22:06.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3552324199}) 2024-09-15T21:22:06.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=497470879}) 2024-09-15T21:22:06.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.473+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4240918797}) 2024-09-15T21:22:06.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.485+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.485+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=68} crc {osdmap=1521008400}) 2024-09-15T21:22:06.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.485+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=69} crc {osdmap=1159564385}) 2024-09-15T21:22:06.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.485+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:22:06.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.485+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=68} crc {osdmap=1521008400}) 2024-09-15T21:22:06.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:22:06.485+0000 7f1c5189c640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=70} crc {osdmap=3703040065}) 2024-09-15T21:22:06.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.972+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' created v138) ==== 111+0+0 (secure 0 0 0) 0x7f09f4010040 con 0x7f09fc137fe0 2024-09-15T21:22:06.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.972+0000 7f09edffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T21:22:06.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:06.972+0000 7f09edffb640 10 monclient: _finish_command 2 = system:0 pool 'cold' created 2024-09-15T21:22:07.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f0a03825640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T21:22:07.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f0a03825640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T21:22:07.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f0a03825640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) -- 0x7f09fc0008e0 con 0x7f09fc137fe0 2024-09-15T21:22:07.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f09edffb640 1 -- 172.21.15.119:0/3399449330 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' already exists v138) ==== 118+0+0 (secure 0 0 0) 0x7f09f401cda0 con 0x7f09fc137fe0 2024-09-15T21:22:07.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f09edffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T21:22:07.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.076+0000 7f09edffb640 10 monclient: _finish_command 3 = system:0 pool 'cold' already exists 2024-09-15T21:22:07.081 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'cold' already exists 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f09e003b9f0 msgr2=0x7f09e003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f09e003b9f0 0x7f09e003deb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ad000 tx=0x7f09f8008040 comp rx=0 tx=0).stop 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 10 monclient: shutdown 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 msgr2=0x7f09fc13cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc13cbd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09f400efc0 tx=0x7f09f400c490 comp rx=0 tx=0).stop 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 shutdown_connections 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f09e003b9f0 0x7f09e003deb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09fc137fe0 0x7f09fc13cbd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09fc0a5570 0x7f09fc136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 --2- 172.21.15.119:0/3399449330 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09fc0a2320 0x7f09fc1367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 >> 172.21.15.119:0/3399449330 conn(0x7f09fc0a6f60 msgr2=0x7f09fc0a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:07.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 shutdown_connections 2024-09-15T21:22:07.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.080+0000 7f0a03825640 1 -- 172.21.15.119:0/3399449330 wait complete. 2024-09-15T21:22:07.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool set cold size 2 2024-09-15T21:22:07.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient: get_monmap_and_config 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient: build_initial_monmap 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient: monmap: 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:07.214403+0000 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:07.214403+0000 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:07.216 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:07.217 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:07.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 1 Processor -- start 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 1 -- start start 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient: init 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd350135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:07.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd35009fe70 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd35013c190 0x7fd35013e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient(hunting): picked mon.noname-a con 0x7fd35013c190 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3500a0300 con 0x7fd350135bf0 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3500a1d80 con 0x7fd35009fe70 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd3500a3600 con 0x7fd35013c190 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd3575cd640 10 monclient(hunting): authenticate will time out at 1246.011064s 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd354b68640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd35009fe70 0x7fd35013aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32780/0 (socket says 172.21.15.119:32780) 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd354b68640 1 -- 172.21.15.119:0/2833191080 learned_addr learned my addr 172.21.15.119:0/2833191080 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 1 --2- 172.21.15.119:0/2833191080 >> v2:172.21.15.17:3300/0 conn(0x7fd35013c190 0x7fd35013e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): get_auth_request con 0x7fd35013c190 auth_method 0 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390876000 0 0) 0x7fd3500a1d80 con 0x7fd35009fe70 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 10 monclient(hunting): _negotiate 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 10 monclient(hunting): my global_id is 64126 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd33c003530 con 0x7fd35009fe70 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1426606738 0 0) 0x7fd33c003530 con 0x7fd35009fe70 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 >> v2:172.21.15.17:3300/0 conn(0x7fd35013c190 msgr2=0x7fd35013e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.212+0000 7fd347fff640 1 --2- 172.21.15.119:0/2833191080 >> v2:172.21.15.17:3300/0 conn(0x7fd35013c190 0x7fd35013e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 >> v1:172.21.15.17:6789/0 conn(0x7fd350135bf0 legacy=0x7fd350135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3500a2060 con 0x7fd35009fe70 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 5 monclient: authenticate success, global_id 64126 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fd340001700 con 0x7fd35009fe70 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:22:07.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: dump: 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 >> v1:172.21.15.119:6789/0 conn(0x7fd35009fe70 legacy=0x7fd35013aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd33c003a60 0x7fd33c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): picked mon.a con 0x7fd33c003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd33c004810 0x7fd33c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): picked mon.c con 0x7fd33c004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd33c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): picked mon.b con 0x7fd33c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd3500a3600 con 0x7fd33c003a60 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c00c240 con 0x7fd33c0053d0 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c00c440 con 0x7fd33c004810 2024-09-15T21:22:07.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd33c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): get_auth_request con 0x7fd33c0053d0 auth_method 0 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355369640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd33c003a60 0x7fd33c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:07.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355369640 10 monclient(hunting): get_auth_request con 0x7fd33c003a60 auth_method 0 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355369640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355369640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355369640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd354b68640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd33c004810 0x7fd33c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient(hunting): handle_auth_done global_id 64126 payload 293 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 -- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd33c004810 msgr2=0x7fd33c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd33c004810 0x7fd33c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 -- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd33c003a60 msgr2=0x7fd33c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd33c003a60 0x7fd33c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 monclient: found mon.b 2024-09-15T21:22:07.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 -- 172.21.15.119:0/2833191080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c003530 con 0x7fd33c0053d0 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 -- 172.21.15.119:0/2833191080 reap_dead start 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd355b6a640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd33c00b9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd348009fc0 tx=0x7fd34801cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde6569bc2eab3ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348009c10 con 0x7fd33c0053d0 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: dump: 2024-09-15T21:22:07.236 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3480a5c80 con 0x7fd33c0053d0 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:07.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 4 set_mon_vals no callback set 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 1 -- 172.21.15.119:0/2833191080 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348003be0 con 0x7fd33c0053d0 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd347fff640 10 monclient: dump: 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.238 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 10 monclient: shutdown 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 -- 172.21.15.119:0/2833191080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 msgr2=0x7fd33c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd33c00b9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd348009fc0 tx=0x7fd34801cc70 comp rx=0 tx=0).stop 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 -- 172.21.15.119:0/2833191080 shutdown_connections 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 --2- 172.21.15.119:0/2833191080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd33c00b9d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 -- 172.21.15.119:0/2833191080 >> 172.21.15.119:0/2833191080 conn(0x7fd3500a66e0 msgr2=0x7fd3500a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 -- 172.21.15.119:0/2833191080 shutdown_connections 2024-09-15T21:22:07.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.216+0000 7fd3575cd640 1 -- 172.21.15.119:0/2833191080 wait complete. 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: build_initial_monmap 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: monmap: 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:07.240 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 Processor -- start 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- start start 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: init 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:07.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): picked mon.noname-b con 0x7fd33c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd3501396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): picked mon.noname-a con 0x7fd3501cddb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd350139c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): picked mon.noname-c con 0x7fd3501ce230 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd35006edb0 con 0x7fd3501cddb0 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd35006ec30 con 0x7fd33c0053d0 2024-09-15T21:22:07.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd350135fd0 con 0x7fd3501ce230 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient(hunting): authenticate will time out at 1246.017340s 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55128/0 (socket says 172.21.15.119:55128) 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 learned_addr learned my addr 172.21.15.119:0/665424071 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): get_auth_request con 0x7fd33c0053d0 auth_method 0 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient(hunting): handle_auth_done global_id 64129 payload 824 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd354b68640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd3501396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd350139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 msgr2=0x7fd350139c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd350139c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 msgr2=0x7fd3501396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd3501396e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 monclient: found mon.noname-b 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35011e040 con 0x7fd33c0053d0 2024-09-15T21:22:07.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd354b68640 10 monclient: get_auth_request con 0x7fd3501cddb0 auth_method 0 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd354b68640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd3501396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00e500 tx=0x7fd34c00e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0052e0 con 0x7fd33c0053d0 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: dump: 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.245 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c018070 con 0x7fd33c0053d0 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 5 monclient: authenticate success, global_id 64129 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _renew_subs 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35011e320 con 0x7fd33c0053d0 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c003840 con 0x7fd33c0053d0 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 10 monclient: dump: 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3477fe640 4 set_mon_vals no callback set 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _renew_subs 2024-09-15T21:22:07.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35013bde0 con 0x7fd33c0053d0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd350136360 con 0x7fd33c0053d0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd34c0039e0 con 0x7fd33c0053d0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd33403b9f0 0x7fd33403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 7975+0+0 (secure 0 0 0) 0x7fd34c0503c0 con 0x7fd33c0053d0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd354b68640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd33403b9f0 0x7fd33403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.220+0000 7fd354b68640 10 monclient: get_auth_request con 0x7fd33403b9f0 auth_method 0 2024-09-15T21:22:07.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.224+0000 7fd354b68640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd33403b9f0 0x7fd33403deb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd340001870 tx=0x7fd340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.228+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd3500a0300 con 0x7fd33c0053d0 2024-09-15T21:22:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.228+0000 7fd345ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.228+0000 7fd345ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:07.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.396+0000 7fd3575cd640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:07.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.396+0000 7fd3575cd640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:07.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.396+0000 7fd3575cd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:07.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.396+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) -- 0x7fd35011cef0 con 0x7fd33c0053d0 2024-09-15T21:22:07.452 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:22:07.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:22:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient: get_monmap_and_config 2024-09-15T21:22:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient: build_initial_monmap 2024-09-15T21:22:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient: monmap: 2024-09-15T21:22:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:07.582797+0000 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:07.582797+0000 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:07.585 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 Processor -- start 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 -- start start 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient: init 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb3e009d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e009def0 0x7fb3e01398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): picked mon.noname-b con 0x7fb3e009def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3e0139de0 0x7fb3e013c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): picked mon.noname-c con 0x7fb3e0139de0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e00a3490 con 0x7fb3e009d400 2024-09-15T21:22:07.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e009f0b0 con 0x7fb3e0139de0 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e00a0320 con 0x7fb3e009def0 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e75cd640 10 monclient(hunting): authenticate will time out at 1250.978361s 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5369640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb3e009d400 0x7fb3e009d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34748/0 (socket says 172.21.15.17:34748) 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e009def0 0x7fb3e01398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5369640 1 -- 172.21.15.17:0/222053986 learned_addr learned my addr 172.21.15.17:0/222053986 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): get_auth_request con 0x7fb3e009def0 auth_method 0 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5b6a640 1 --2- 172.21.15.17:0/222053986 >> v2:172.21.15.119:3300/0 conn(0x7fb3e0139de0 0x7fb3e013c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5b6a640 10 monclient(hunting): get_auth_request con 0x7fb3e0139de0 auth_method 0 2024-09-15T21:22:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5b6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5b6a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e5b6a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_done global_id 44591 payload 274 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 1 -- 172.21.15.17:0/222053986 >> v2:172.21.15.119:3300/0 conn(0x7fb3e0139de0 msgr2=0x7fb3e013c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 1 --2- 172.21.15.17:0/222053986 >> v2:172.21.15.119:3300/0 conn(0x7fb3e0139de0 0x7fb3e013c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 1 -- 172.21.15.17:0/222053986 >> v1:172.21.15.17:6789/0 conn(0x7fb3e009d400 legacy=0x7fb3e009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3d7fff640 1 -- 172.21.15.17:0/222053986 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585086270 0 0) 0x7fb3e00a3490 con 0x7fb3e009d400 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 1 monclient: found mon.noname-b 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3e4b68640 1 -- 172.21.15.17:0/222053986 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e009e7a0 con 0x7fb3e009def0 2024-09-15T21:22:07.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.581+0000 7fb3d7fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:22:07.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 --2- 172.21.15.17:0/222053986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e009def0 0x7fb3e01398a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00097c0 tx=0x7fb3d001efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49e0d740e925ce2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 1 -- 172.21.15.17:0/222053986 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00a9070 con 0x7fb3e009def0 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: dump: 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 5 monclient: authenticate success, global_id 44591 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 1 -- 172.21.15.17:0/222053986 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d00b2070 con 0x7fb3e009def0 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 1 -- 172.21.15.17:0/222053986 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00ad890 con 0x7fb3e009def0 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 4 set_mon_vals no callback set 2024-09-15T21:22:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3d7fff640 10 monclient: dump: 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: shutdown 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- 172.21.15.17:0/222053986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e009def0 msgr2=0x7fb3e01398a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/222053986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e009def0 0x7fb3e01398a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00097c0 tx=0x7fb3d001efa0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- 172.21.15.17:0/222053986 shutdown_connections 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/222053986 >> v2:172.21.15.119:3300/0 conn(0x7fb3e0139de0 0x7fb3e013c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/222053986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e009def0 0x7fb3e01398a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- 172.21.15.17:0/222053986 >> 172.21.15.17:0/222053986 conn(0x7fb3e00a66a0 msgr2=0x7fb3e00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- 172.21.15.17:0/222053986 shutdown_connections 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- 172.21.15.17:0/222053986 wait complete. 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: build_initial_monmap 2024-09-15T21:22:07.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: monmap: 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:07.603 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 Processor -- start 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- start start 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: init 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e009d400 0x7fb3e011f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): picked mon.noname-b con 0x7fb3e009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 0x7fb3e011f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): picked mon.noname-a con 0x7fb3e009def0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e0139de0 0x7fb3e011fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): picked mon.noname-c con 0x7fb3e0139de0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:07.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e009ce50 con 0x7fb3e009def0 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e009ccd0 con 0x7fb3e009d400 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e00a10a0 con 0x7fb3e0139de0 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e75cd640 10 monclient(hunting): authenticate will time out at 1250.983269s 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 0x7fb3e011f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 0x7fb3e011f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48920/0 (socket says 172.21.15.17:48920) 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 -- 172.21.15.17:0/1891108631 learned_addr learned my addr 172.21.15.17:0/1891108631 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): get_auth_request con 0x7fb3e009def0 auth_method 0 2024-09-15T21:22:07.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e0139de0 0x7fb3e011fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): get_auth_request con 0x7fb3e0139de0 auth_method 0 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5369640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e009d400 0x7fb3e011f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient(hunting): handle_auth_done global_id 54507 payload 824 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 -- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e0139de0 msgr2=0x7fb3e011fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e0139de0 0x7fb3e011fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 -- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e009d400 msgr2=0x7fb3e011f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e009d400 0x7fb3e011f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 monclient: found mon.noname-a 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e4b68640 1 -- 172.21.15.17:0/1891108631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e0123570 con 0x7fb3e009def0 2024-09-15T21:22:07.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5369640 10 monclient: get_auth_request con 0x7fb3e009d400 auth_method 0 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5369640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e009d400 0x7fb3e011f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.585+0000 7fb3e5b6a640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e0139de0 0x7fb3e011fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e4b68640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 0x7fb3e011f870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00af040 tx=0x7fb3d001cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0004220 con 0x7fb3e009def0 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: dump: 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.608 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d00043c0 con 0x7fb3e009def0 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 5 monclient: authenticate success, global_id 54507 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0003820 con 0x7fb3e009def0 2024-09-15T21:22:07.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 10 monclient: dump: 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d6ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 monclient: _renew_subs 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e0123150 con 0x7fb3e009def0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 monclient: _renew_subs 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e0120df0 con 0x7fb3e009def0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e00a14b0 con 0x7fb3e009def0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb3d00039c0 con 0x7fb3e009def0 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3c403b9c0 0x7fb3c403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 7975+0+0 (secure 0 0 0) 0x7fb3d00ed020 con 0x7fb3e009def0 2024-09-15T21:22:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e5369640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3c403b9c0 0x7fb3c403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e5369640 10 monclient: get_auth_request con 0x7fb3c403b9c0 auth_method 0 2024-09-15T21:22:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.589+0000 7fb3e5369640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3c403b9c0 0x7fb3c403de80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc000f70 tx=0x7fb3dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.593+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb3e0123570 con 0x7fb3e009def0 2024-09-15T21:22:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.593+0000 7fb3d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.593+0000 7fb3d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:07.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:22:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb3e0139b00 con 0x7fb3e009def0 2024-09-15T21:22:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3d5ffb640 1 -- 172.21.15.17:0/1891108631 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fb3d0018280 con 0x7fb3e009def0 2024-09-15T21:22:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3d5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:07.833 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:22:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3c403b9c0 msgr2=0x7fb3c403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3c403b9c0 0x7fb3c403de80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc000f70 tx=0x7fb3dc009040 comp rx=0 tx=0).stop 2024-09-15T21:22:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 10 monclient: shutdown 2024-09-15T21:22:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 msgr2=0x7fb3e011f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 0x7fb3e011f870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00af040 tx=0x7fb3d001cd80 comp rx=0 tx=0).stop 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 shutdown_connections 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3c403b9c0 0x7fb3c403de80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e0139de0 0x7fb3e011fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e009def0 0x7fb3e011f870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 --2- 172.21.15.17:0/1891108631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e009d400 0x7fb3e011f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 >> 172.21.15.17:0/1891108631 conn(0x7fb3e00a66a0 msgr2=0x7fb3e00a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 shutdown_connections 2024-09-15T21:22:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.829+0000 7fb3e75cd640 1 -- 172.21.15.17:0/1891108631 wait complete. 2024-09-15T21:22:07.850 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:22:07.850 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:22:07.851 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:22:07.851 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient: get_monmap_and_config 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient: build_initial_monmap 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient: monmap: 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:07.966491+0000 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:07.966491+0000 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:07.969 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:07.969 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:07.969 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:07.969 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 Processor -- start 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 -- start start 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient: init 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f17740a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): picked mon.noname-b con 0x7f17740a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17740a53b0 0x7f17740a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): picked mon.noname-c con 0x7f17740a53b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1774137850 0x7f1774137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): picked mon.noname-a con 0x7f1774137850 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f17740a2c30 con 0x7f1774137850 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f17740a0f00 con 0x7f17740a53b0 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f17740a19b0 con 0x7f17740a4a90 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f177939d640 10 monclient(hunting): authenticate will time out at 1251.362098s 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1774137850 0x7f1774137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f17740a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1774137850 0x7f1774137ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40324/0 (socket says 172.21.15.17:40324) 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 1 -- 172.21.15.17:0/2520485513 learned_addr learned my addr 172.21.15.17:0/2520485513 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): get_auth_request con 0x7f1774137850 auth_method 0 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): get_auth_request con 0x7f17740a4a90 auth_method 0 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17727fc640 1 --2- 172.21.15.17:0/2520485513 >> v2:172.21.15.119:3300/0 conn(0x7f17740a53b0 0x7f17740a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17727fc640 10 monclient(hunting): get_auth_request con 0x7f17740a53b0 auth_method 0 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17727fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f17727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient(hunting): handle_auth_done global_id 44597 payload 274 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 -- 172.21.15.17:0/2520485513 >> v2:172.21.15.119:3300/0 conn(0x7f17740a53b0 msgr2=0x7f17740a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 --2- 172.21.15.17:0/2520485513 >> v2:172.21.15.119:3300/0 conn(0x7f17740a53b0 0x7f17740a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 -- 172.21.15.17:0/2520485513 >> v2:172.21.15.17:3300/0 conn(0x7f1774137850 msgr2=0x7f1774137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 --2- 172.21.15.17:0/2520485513 >> v2:172.21.15.17:3300/0 conn(0x7f1774137850 0x7f1774137ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 monclient: found mon.noname-b 2024-09-15T21:22:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.965+0000 7f1772ffd640 1 -- 172.21.15.17:0/2520485513 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17740a1c90 con 0x7f17740a4a90 2024-09-15T21:22:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1772ffd640 1 --2- 172.21.15.17:0/2520485513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f17740a4e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17740a7a70 tx=0x7f176401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f98dd15fb52de7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 1 -- 172.21.15.17:0/2520485513 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640b0070 con 0x7f17740a4a90 2024-09-15T21:22:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: dump: 2024-09-15T21:22:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.980 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 1 -- 172.21.15.17:0/2520485513 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17640ab600 con 0x7f17740a4a90 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 1 -- 172.21.15.17:0/2520485513 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640ab9d0 con 0x7f17740a4a90 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f1771ffb640 10 monclient: dump: 2024-09-15T21:22:07.981 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 5 monclient: authenticate success, global_id 44597 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 4 set_mon_vals no callback set 2024-09-15T21:22:07.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 10 monclient: shutdown 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 -- 172.21.15.17:0/2520485513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 msgr2=0x7f17740a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 --2- 172.21.15.17:0/2520485513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f17740a4e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17740a7a70 tx=0x7f176401c3f0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 -- 172.21.15.17:0/2520485513 shutdown_connections 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 --2- 172.21.15.17:0/2520485513 >> v2:172.21.15.119:3300/0 conn(0x7f17740a53b0 0x7f17740a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 --2- 172.21.15.17:0/2520485513 >> v2:172.21.15.17:3300/0 conn(0x7f1774137850 0x7f1774137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 --2- 172.21.15.17:0/2520485513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f17740a4e70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 -- 172.21.15.17:0/2520485513 >> 172.21.15.17:0/2520485513 conn(0x7f17740aabb0 msgr2=0x7f17740a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:07.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 -- 172.21.15.17:0/2520485513 shutdown_connections 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 1 -- 172.21.15.17:0/2520485513 wait complete. 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 10 monclient: build_initial_monmap 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.969+0000 7f177939d640 10 monclient: monmap: 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 Processor -- start 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 -- start start 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient: init 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f1774133b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): picked mon.noname-c con 0x7f17740a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17740a53b0 0x7f17741340a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f1774133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): picked mon.noname-a con 0x7f17740a53b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f1774133b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36588/0 (socket says 172.21.15.17:36588) 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1774137850 0x7f17741cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17740a53b0 0x7f17741340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): picked mon.noname-b con 0x7f1774137850 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 1 -- 172.21.15.17:0/1727003725 learned_addr learned my addr 172.21.15.17:0/1727003725 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f177406f2d0 con 0x7f17740a53b0 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f177406f150 con 0x7f1774137850 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1774141540 con 0x7f17740a4a90 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f177939d640 10 monclient(hunting): authenticate will time out at 1251.369239s 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): get_auth_request con 0x7f17740a4a90 auth_method 0 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): get_auth_request con 0x7f17740a53b0 auth_method 0 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient(hunting): handle_auth_done global_id 54513 payload 824 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 msgr2=0x7f1774133b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17740a4a90 0x7f1774133b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1774137850 msgr2=0x7f17741cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1774137850 0x7f17741cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient: _finish_hunting 0 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 monclient: found mon.noname-a 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177413ad40 con 0x7f17740a53b0 2024-09-15T21:22:07.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.973+0000 7f17727fc640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17740a53b0 0x7f17741340a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f175c007e90 tx=0x7f175c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.980+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v139) ==== 124+0+0 (secure 0 0 0) 0x7fd34c01ddc0 con 0x7fd33c0053d0 2024-09-15T21:22:07.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.980+0000 7fd345ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:07.989 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:07.980+0000 7fd345ffb640 10 monclient: _finish_command 2 = system:0 set pool 12 size to 2 2024-09-15T21:22:07.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c014620 con 0x7f17740a53b0 2024-09-15T21:22:07.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:07.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: dump: 2024-09-15T21:22:07.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175c019070 con 0x7f17740a53b0 2024-09-15T21:22:07.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c0149e0 con 0x7f17740a53b0 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: dump: 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.995 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 5 monclient: authenticate success, global_id 54513 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 10 monclient: _renew_subs 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f17717fa640 4 set_mon_vals no callback set 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17740a3ac0 con 0x7f17740a53b0 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 10 monclient: _renew_subs 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177413ce40 con 0x7f17740a53b0 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 0x7f17741d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f177413cca0 con 0x7f17741cfe70 2024-09-15T21:22:07.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 0x7f17741d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient: get_auth_request con 0x7f17741cfe70 auth_method 0 2024-09-15T21:22:07.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:07.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:07.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:07.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f175c014b80 con 0x7f17740a53b0 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 10 monclient(hunting): handle_auth_done global_id 54516 payload 824 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17741d0790 con 0x7f17741cfe70 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f175403b9c0 0x7f175403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 7975+0+0 (secure 0 0 0) 0x7f175c050f20 con 0x7f17740a53b0 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f1772ffd640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 0x7f17741d0250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f176401cc90 tx=0x7f176401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640b0040 con 0x7f17741cfe70 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: dump: 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:07.998 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f17737fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f175403b9c0 0x7f175403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f17737fe640 10 monclient: get_auth_request con 0x7f175403b9c0 auth_method 0 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f17640093e0 con 0x7f17741cfe70 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:07.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 msgr2=0x7f17741d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 0x7f17741d0250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f176401cc90 tx=0x7f176401ac30 comp rx=0 tx=0).stop 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 msgr2=0x7f17741d0250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f176b7fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17741cfe70 0x7f17741d0250 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.977+0000 7f17737fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f175403b9c0 0x7f175403de80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f176c006aa0 tx=0x7f176c006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 0x7f1774135fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f17740a57d0 con 0x7f1774135bd0 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 0x7f1774135fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient: get_auth_request con 0x7f1774135bd0 auth_method 0 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 10 monclient(hunting): handle_auth_done global_id 54519 payload 824 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 1 -- 172.21.15.17:0/1727003725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f177413b640 con 0x7f1774135bd0 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f1772ffd640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 0x7f1774135fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1764000c10 tx=0x7f1764019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640b0040 con 0x7f1774135bd0 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 10 monclient: dump: 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.001 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f17640c5070 con 0x7f1774135bd0 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 msgr2=0x7f1774135fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 0x7f1774135fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1764000c10 tx=0x7f1764019190 comp rx=0 tx=0).stop 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 msgr2=0x7f1774135fe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f176b7fe640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1774135bd0 0x7f1774135fe0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1764000c10 tx=0x7f1764019190 comp rx=0 tx=0).stop 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 53347, 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:08.003 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.004 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.005 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:08.006 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.007 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:08.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:08.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:08.012 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.013 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.014 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:08.015 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f175403b9c0 msgr2=0x7f175403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f175403b9c0 0x7f175403de80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f176c006aa0 tx=0x7f176c006a30 comp rx=0 tx=0).stop 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 10 monclient: shutdown 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17740a53b0 msgr2=0x7f17741340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 --2- 172.21.15.17:0/1727003725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17740a53b0 0x7f17741340a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f175c007e90 tx=0x7f175c00e4b0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f17737fe640 1 -- 172.21.15.17:0/1727003725 reap_dead start 2024-09-15T21:22:08.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f17737fe640 1 -- 172.21.15.17:0/1727003725 reap_dead start 2024-09-15T21:22:08.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 shutdown_connections 2024-09-15T21:22:08.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 >> 172.21.15.17:0/1727003725 conn(0x7f17740aabb0 msgr2=0x7f17740a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:08.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 shutdown_connections 2024-09-15T21:22:08.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:07.985+0000 7f177939d640 1 -- 172.21.15.17:0/1727003725 wait complete. 2024-09-15T21:22:08.018 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:22:08.018 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:22:08.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:22:08.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:08.080+0000 7fd3575cd640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:08.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:08.080+0000 7fd3575cd640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:08.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:08.080+0000 7fd3575cd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:08.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:08.080+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) -- 0x7fd350134fc0 con 0x7fd33c0053d0 2024-09-15T21:22:08.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient: get_monmap_and_config 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient: build_initial_monmap 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient: monmap: 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:08.126257+0000 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:08.126257+0000 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:08.129 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 Processor -- start 2024-09-15T21:22:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 -- start start 2024-09-15T21:22:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient: init 2024-09-15T21:22:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:22:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf00749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): picked mon.noname-b con 0x7fcbf00745d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbf00a4530 0x7fcbf006dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): picked mon.noname-c con 0x7fcbf00a4530 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcbf006e1f0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf009f630 con 0x7fcbf006e1f0 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf009f910 con 0x7fcbf00a4530 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf00a04d0 con 0x7fcbf00745d0 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbf643f640 10 monclient(hunting): authenticate will time out at 1251.521777s 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf00749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf00749b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36602/0 (socket says 172.21.15.17:36602) 2024-09-15T21:22:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 -- 172.21.15.17:0/1756592855 learned_addr learned my addr 172.21.15.17:0/1756592855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): get_auth_request con 0x7fcbf00745d0 auth_method 0 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbef7fe640 1 --2- 172.21.15.17:0/1756592855 >> v2:172.21.15.119:3300/0 conn(0x7fcbf00a4530 0x7fcbf006dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbef7fe640 10 monclient(hunting): get_auth_request con 0x7fcbf00a4530 auth_method 0 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient(hunting): handle_auth_done global_id 44603 payload 274 2024-09-15T21:22:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 -- 172.21.15.17:0/1756592855 >> v2:172.21.15.119:3300/0 conn(0x7fcbf00a4530 msgr2=0x7fcbf006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 --2- 172.21.15.17:0/1756592855 >> v2:172.21.15.119:3300/0 conn(0x7fcbf00a4530 0x7fcbf006dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 -- 172.21.15.17:0/1756592855 >> v1:172.21.15.119:6789/0 conn(0x7fcbf006e1f0 legacy=0x7fcbf006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 monclient: found mon.noname-b 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.125+0000 7fcbeffff640 1 -- 172.21.15.17:0/1756592855 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf0074ef0 con 0x7fcbf00745d0 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 1 --2- 172.21.15.17:0/1756592855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf00749b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663437a24391abe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 1 -- 172.21.15.17:0/1756592855 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80a5590 con 0x7fcbf00745d0 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: dump: 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.142 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: _finish_auth 0 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 1 -- 172.21.15.17:0/1756592855 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe80a5730 con 0x7fcbf00745d0 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 1 -- 172.21.15.17:0/1756592855 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80a5b00 con 0x7fcbf00745d0 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 5 monclient: authenticate success, global_id 44603 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 4 set_mon_vals no callback set 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:08.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeeffd640 10 monclient: dump: 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: shutdown 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:08.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- 172.21.15.17:0/1756592855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 msgr2=0x7fcbf00749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 --2- 172.21.15.17:0/1756592855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf00749b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe801c950 comp rx=0 tx=0).stop 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- 172.21.15.17:0/1756592855 shutdown_connections 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 --2- 172.21.15.17:0/1756592855 >> v2:172.21.15.119:3300/0 conn(0x7fcbf00a4530 0x7fcbf006dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 --2- 172.21.15.17:0/1756592855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf00749b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- 172.21.15.17:0/1756592855 >> 172.21.15.17:0/1756592855 conn(0x7fcbf00a4e90 msgr2=0x7fcbf00a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- 172.21.15.17:0/1756592855 shutdown_connections 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- 172.21.15.17:0/1756592855 wait complete. 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: build_initial_monmap 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: monmap: 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:08.145 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 Processor -- start 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- start start 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: init 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf006e1f0 0x7fcbf0124e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): picked mon.noname-b con 0x7fcbf006e1f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf013bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): picked mon.noname-c con 0x7fcbf00745d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:08.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 0x7fcbf01cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): picked mon.noname-a con 0x7fcbf00a4530 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01423b0 con 0x7fcbf00a4530 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf00a24e0 con 0x7fcbf006e1f0 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf0142530 con 0x7fcbf00745d0 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf643f640 10 monclient(hunting): authenticate will time out at 1251.527408s 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf013bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 0x7fcbf01cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 0x7fcbf01cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40362/0 (socket says 172.21.15.17:40362) 2024-09-15T21:22:08.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 -- 172.21.15.17:0/3937815975 learned_addr learned my addr 172.21.15.17:0/3937815975 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): get_auth_request con 0x7fcbf00a4530 auth_method 0 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): get_auth_request con 0x7fcbf00745d0 auth_method 0 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf006e1f0 0x7fcbf0124e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 10 monclient(hunting): get_auth_request con 0x7fcbf006e1f0 auth_method 0 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient(hunting): handle_auth_done global_id 54522 payload 824 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 msgr2=0x7fcbf013bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf013bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf006e1f0 msgr2=0x7fcbf0124e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf006e1f0 0x7fcbf0124e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient: _finish_hunting 0 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 monclient: found mon.noname-a 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:08.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbf49dc640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf0126640 con 0x7fcbf00a4530 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbeffff640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf006e1f0 0x7fcbf0124e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.129+0000 7fcbef7fe640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf00745d0 0x7fcbf013bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf49dc640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 0x7fcbf01cf840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c3f0 tx=0x7fcbe000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0011790 con 0x7fcbf00a4530 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: dump: 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.150 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: _finish_auth 0 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe0017070 con 0x7fcbf00a4530 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 5 monclient: authenticate success, global_id 54522 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 10 monclient: _renew_subs 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf009f180 con 0x7fcbf00a4530 2024-09-15T21:22:08.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0011b70 con 0x7fcbf00a4530 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: dump: 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.152 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 10 monclient: _renew_subs 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbee7fc640 4 set_mon_vals no callback set 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf0126360 con 0x7fcbf00a4530 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 0x7fcbf01cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf00749b0 con 0x7fcbf0122e40 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 0x7fcbf01cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient: get_auth_request con 0x7fcbf0122e40 auth_method 0 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcbe0011d10 con 0x7fcbf00a4530 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 10 monclient(hunting): handle_auth_done global_id 44609 payload 824 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbf009ef60 con 0x7fcbf0122e40 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbd003b9c0 0x7fcbd003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 7975+0+0 (secure 0 0 0) 0x7fcbe00504d0 con 0x7fcbf00a4530 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbeffff640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 0x7fcbf01cfd80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00a4940 tx=0x7fcbe80196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8009510 con 0x7fcbf0122e40 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: dump: 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbef7fe640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbd003b9c0 0x7fcbd003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbef7fe640 10 monclient: get_auth_request con 0x7fcbd003b9c0 auth_method 0 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcbe80b6420 con 0x7fcbf0122e40 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 msgr2=0x7fcbf01cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 0x7fcbf01cfd80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00a4940 tx=0x7fcbe80196f0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 msgr2=0x7fcbf01cfd80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbecff9640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf0122e40 0x7fcbf01cfd80 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00a4940 tx=0x7fcbe80196f0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.133+0000 7fcbef7fe640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbd003b9c0 0x7fcbd003de80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8007e50 tx=0x7fcbd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 0x7fcbf01d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf0123d30 con 0x7fcbf013c1f0 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 0x7fcbf01d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient: get_auth_request con 0x7fcbf013c1f0 auth_method 0 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 10 monclient(hunting): handle_auth_done global_id 44612 payload 824 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 1 -- 172.21.15.17:0/3937815975 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcbf01dab90 con 0x7fcbf013c1f0 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbeffff640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 0x7fcbf01d02c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe80a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80c2230 con 0x7fcbf013c1f0 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 10 monclient: dump: 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7fcbe80c2230 con 0x7fcbf013c1f0 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 msgr2=0x7fcbf01d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 0x7fcbf01d02c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe80a7000 comp rx=0 tx=0).stop 2024-09-15T21:22:08.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 msgr2=0x7fcbf01d02c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbecff9640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbf013c1f0 0x7fcbf01d02c0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe80a7000 comp rx=0 tx=0).stop 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-15T21:22:08.161 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 96739, 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.162 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:08.165 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:08.166 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:08.167 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.168 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.170 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:08.173 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.174 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.176 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:08.176 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbd003b9c0 msgr2=0x7fcbd003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbd003b9c0 0x7fcbd003de80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8007e50 tx=0x7fcbd8008040 comp rx=0 tx=0).stop 2024-09-15T21:22:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 10 monclient: shutdown 2024-09-15T21:22:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 msgr2=0x7fcbf01cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 0x7fcbf01cf840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c3f0 tx=0x7fcbe000c8b0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf49dc640 1 -- 172.21.15.17:0/3937815975 reap_dead start 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 shutdown_connections 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 --2- 172.21.15.17:0/3937815975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00a4530 0x7fcbf01cf840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 >> 172.21.15.17:0/3937815975 conn(0x7fcbf00a4e90 msgr2=0x7fcbf013edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 shutdown_connections 2024-09-15T21:22:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.141+0000 7fcbf643f640 1 -- 172.21.15.17:0/3937815975 wait complete. 2024-09-15T21:22:08.178 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:22:08.178 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:22:08.179 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:22:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient: get_monmap_and_config 2024-09-15T21:22:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient: build_initial_monmap 2024-09-15T21:22:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient: monmap: 2024-09-15T21:22:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:08.290896+0000 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:08.290896+0000 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:08.293 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 1 Processor -- start 2024-09-15T21:22:08.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 1 -- start start 2024-09-15T21:22:08.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient: init 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faad80a4c00 0x7faad809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient(hunting): picked mon.noname-a con 0x7faad80a4c00 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faad80a4fe0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faad813ce10 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad809c700 con 0x7faad813ce10 2024-09-15T21:22:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad80a5d10 con 0x7faad80a4fe0 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faad806f820 con 0x7faad80a4c00 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faadde80640 10 monclient(hunting): authenticate will time out at 1251.687097s 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faad813ce10 0x7faad813f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41464/0 (socket says 172.21.15.17:41464) 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad7fff640 1 -- 172.21.15.17:0/272504455 learned_addr learned my addr 172.21.15.17:0/272504455 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 1 --2- 172.21.15.17:0/272504455 >> v2:172.21.15.17:3300/0 conn(0x7faad80a4c00 0x7faad809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): get_auth_request con 0x7faad80a4c00 auth_method 0 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad6ffd640 1 -- 172.21.15.17:0/272504455 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911345694 0 0) 0x7faad809c700 con 0x7faad813ce10 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad6ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad6ffd640 10 monclient(hunting): my global_id is 54528 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad6ffd640 1 -- 172.21.15.17:0/272504455 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faac4003530 con 0x7faad813ce10 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient(hunting): handle_auth_done global_id 54525 payload 274 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 1 -- 172.21.15.17:0/272504455 >> v1:172.21.15.119:6789/0 conn(0x7faad80a4fe0 legacy=0x7faad80a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 1 -- 172.21.15.17:0/272504455 >> v1:172.21.15.17:6789/0 conn(0x7faad813ce10 legacy=0x7faad813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:08.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 1 monclient: found mon.noname-a 2024-09-15T21:22:08.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:08.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.289+0000 7faad77fe640 1 -- 172.21.15.17:0/272504455 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad806f200 con 0x7faad80a4c00 2024-09-15T21:22:08.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad77fe640 1 --2- 172.21.15.17:0/272504455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4c00 0x7faad809cb20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faac00097c0 tx=0x7faac001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d96cc33d81820b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 1 -- 172.21.15.17:0/272504455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00b0070 con 0x7faad80a4c00 2024-09-15T21:22:08.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: dump: 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 5 monclient: authenticate success, global_id 54525 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 1 -- 172.21.15.17:0/272504455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac00ab600 con 0x7faad80a4c00 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 1 -- 172.21.15.17:0/272504455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00ab9b0 con 0x7faad80a4c00 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faad6ffd640 10 monclient: dump: 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.309 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 4 set_mon_vals no callback set 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 10 monclient: shutdown 2024-09-15T21:22:08.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 -- 172.21.15.17:0/272504455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4c00 msgr2=0x7faad809cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 --2- 172.21.15.17:0/272504455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4c00 0x7faad809cb20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faac00097c0 tx=0x7faac001c3f0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 -- 172.21.15.17:0/272504455 shutdown_connections 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 --2- 172.21.15.17:0/272504455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4c00 0x7faad809cb20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 -- 172.21.15.17:0/272504455 >> 172.21.15.17:0/272504455 conn(0x7faad80aaeb0 msgr2=0x7faad80a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 -- 172.21.15.17:0/272504455 shutdown_connections 2024-09-15T21:22:08.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 -- 172.21.15.17:0/272504455 wait complete. 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 10 monclient: build_initial_monmap 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 10 monclient: monmap: 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 Processor -- start 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 1 -- start start 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.293+0000 7faadde80640 10 monclient: init 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80a4c00 0x7faad81373b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): picked mon.noname-b con 0x7faad80a4c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4fe0 0x7faad81378f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): picked mon.noname-a con 0x7faad80a4fe0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faad813ce10 0x7faad8139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): picked mon.noname-c con 0x7faad813ce10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faad8143760 con 0x7faad80a4fe0 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faad81438e0 con 0x7faad80a4c00 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faad8143a60 con 0x7faad813ce10 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): authenticate will time out at 1251.692318s 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4fe0 0x7faad81378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4fe0 0x7faad81378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40374/0 (socket says 172.21.15.17:40374) 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 -- 172.21.15.17:0/2021603373 learned_addr learned my addr 172.21.15.17:0/2021603373 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): get_auth_request con 0x7faad80a4fe0 auth_method 0 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faad813ce10 0x7faad8139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): get_auth_request con 0x7faad813ce10 auth_method 0 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80a4c00 0x7faad81373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): get_auth_request con 0x7faad80a4c00 auth_method 0 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient(hunting): handle_auth_done global_id 54531 payload 824 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faad813ce10 msgr2=0x7faad8139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faad813ce10 0x7faad8139c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80a4c00 msgr2=0x7faad81373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80a4c00 0x7faad81373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 monclient: found mon.noname-a 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad81213b0 con 0x7faad80a4fe0 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faacffff640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4fe0 0x7faad81378f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faac800c530 tx=0x7faac800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8017070 con 0x7faad80a4fe0 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: dump: 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac80052d0 con 0x7faad80a4fe0 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 5 monclient: authenticate success, global_id 54531 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient: _renew_subs 2024-09-15T21:22:08.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad81402b0 con 0x7faad80a4fe0 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8012870 con 0x7faad80a4fe0 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: dump: 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad67fc640 4 set_mon_vals no callback set 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient: _renew_subs 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad813be80 con 0x7faad80a4fe0 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 0x7faad813a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faad80a5fc0 con 0x7faad813a150 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7faac801a070 con 0x7faad80a4fe0 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 0x7faad813a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient: get_auth_request con 0x7faad813a150 auth_method 0 2024-09-15T21:22:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faab803b9f0 0x7faab803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 7975+0+0 (secure 0 0 0) 0x7faac804fb00 con 0x7faad80a4fe0 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faab803b9f0 0x7faab803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 10 monclient(hunting): handle_auth_done global_id 64150 payload 824 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faad809cb20 con 0x7faad813a150 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 10 monclient: get_auth_request con 0x7faab803b9f0 auth_method 0 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad77fe640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 0x7faad813a530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faad809ba60 tx=0x7faac001ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00ba070 con 0x7faad813a150 2024-09-15T21:22:08.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad4ff9640 10 monclient: dump: 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.297+0000 7faad7fff640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faab803b9f0 0x7faab803deb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faad0005c10 tx=0x7faad0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faac00093e0 con 0x7faad813a150 2024-09-15T21:22:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 msgr2=0x7faad813a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 0x7faad813a530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faad809ba60 tx=0x7faac001ac30 comp rx=0 tx=0).stop 2024-09-15T21:22:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 msgr2=0x7faad813a530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.301+0000 7faad4ff9640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813a150 0x7faad813a530 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faad809ba60 tx=0x7faac001ac30 comp rx=0 tx=0).stop 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faadde80640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faadde80640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 0x7faad809df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faadde80640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faad806a080 con 0x7faad813aa70 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 0x7faad809df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient: get_auth_request con 0x7faad813aa70 auth_method 0 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 10 monclient(hunting): handle_auth_done global_id 64153 payload 824 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.305+0000 7faad77fe640 1 -- 172.21.15.17:0/2021603373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faad8120d50 con 0x7faad813aa70 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad77fe640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 0x7faad809df60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faac0019000 tx=0x7faac0019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00ba040 con 0x7faad813aa70 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 10 monclient: dump: 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7faac00ab700 con 0x7faad813aa70 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 msgr2=0x7faad809df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 0x7faad809df60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faac0019000 tx=0x7faac0019190 comp rx=0 tx=0).stop 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 msgr2=0x7faad809df60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad4ff9640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad813aa70 0x7faad809df60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 6567, 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:08.328 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.329 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.330 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:08.331 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:08.332 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:08.333 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:08.335 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:08.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:08.338 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:08.340 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:22:08.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:08.340 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:08.340 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:08.340 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:08.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:08.341 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:08.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:08.341 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:08.341 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faab803b9f0 msgr2=0x7faab803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faab803b9f0 0x7faab803deb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faad0005c10 tx=0x7faad0009040 comp rx=0 tx=0).stop 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 10 monclient: shutdown 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4fe0 msgr2=0x7faad81378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 --2- 172.21.15.17:0/2021603373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faad80a4fe0 0x7faad81378f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faac800c530 tx=0x7faac800c9f0 comp rx=0 tx=0).stop 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad7fff640 1 -- 172.21.15.17:0/2021603373 reap_dead start 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faad7fff640 1 -- 172.21.15.17:0/2021603373 reap_dead start 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 shutdown_connections 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 >> 172.21.15.17:0/2021603373 conn(0x7faad80aaeb0 msgr2=0x7faad80a2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 shutdown_connections 2024-09-15T21:22:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:08.309+0000 7faadde80640 1 -- 172.21.15.17:0/2021603373 wait complete. 2024-09-15T21:22:08.344 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:22:08.344 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:22:08.344 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-15T21:22:08.344 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:22:08.344 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:22:08.344 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:22:08.345 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:22:08.356 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:22:08.356 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T21:22:08.356 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T21:22:08.356 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:22:10.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:10.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:10.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:10.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:10.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:10.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:10.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:10.220+0000 7fd344ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:11.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:11.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:11.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:11.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:11.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:11.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:12.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:12.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:12.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:12.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:12.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:12.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:13.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:13.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:13.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:13.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:13.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:13.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:14.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:14.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:14.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:14.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:15.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:15.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:15.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:15.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:16.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:16.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:16.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:16.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:16.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:16.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:17.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:17.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:17.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:17.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:17.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:17.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:18.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:18.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:18.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:18.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:18.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:18.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:19.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:19.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:19.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:19.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:19.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:19.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:20.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:20.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:20.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:20.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:20.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:20.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:20.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:20.220+0000 7fd344ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:21.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:21.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:21.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:21.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:21.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:21.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:22.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:22.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:22.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:22.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:22.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:22.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:23.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:23.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:23.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:23.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:23.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:23.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:24.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:24.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:24.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:24.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:24.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:24.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:25.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:25.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:25.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:25.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:25.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:25.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:26.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:26.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:26.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:26.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:26.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:26.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:27.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:27.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:27.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:27.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:27.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:27.220+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:28.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:28.220+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:28.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:28.220+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:28.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:28.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:28.357 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T21:22:28.357 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T21:22:28.357 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:22:28.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient: get_monmap_and_config 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient: build_initial_monmap 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient: monmap: 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:28.487629+0000 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:28.487629+0000 2024-09-15T21:22:28.489 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:28.490 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 Processor -- start 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 -- start start 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient: init 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 0x7f4fd40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): picked mon.noname-a con 0x7f4fd40a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:28.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4fd409ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): picked mon.noname-b con 0x7f4fd40a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd40a3130 con 0x7f4fd409ffc0 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fd40a0890 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd40a1bf0 con 0x7f4fd40a11b0 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fdac19640 10 monclient(hunting): authenticate will time out at 1271.883046s 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd89b5640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 msgr2=0x7f4fd40a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd89b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 0x7f4fd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:28.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48410/0 (socket says 172.21.15.17:48410) 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 learned_addr learned my addr 172.21.15.17:0/4037047651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request con 0x7f4fd40a11b0 auth_method 0 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:28.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.484+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:28.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 msgr2=0x7f4fd40a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:28.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 0x7f4fd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request con 0x7f4fd40a11b0 auth_method 0 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:28.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:28.688+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:29.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 msgr2=0x7f4fd40a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:29.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 0x7f4fd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:29.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request con 0x7f4fd40a11b0 auth_method 0 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:29.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.092+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:29.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:29.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:29.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:29.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:29.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:29.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 msgr2=0x7f4fd40a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request con 0x7f4fd40a11b0 auth_method 0 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 0x7f4fd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:29.896+0000 7f4fd91b6640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:30.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:30.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:30.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:30.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:30.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:30.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:30.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:30.224+0000 7fd344ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:31.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:31.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:31.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:31.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:31.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:31.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:31.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): tick 2024-09-15T21:22:31.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 msgr2=0x7f4fd4135d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fd40a11b0 0x7f4fd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 msgr2=0x7f4fd40a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fd40a0890 0x7f4fd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fd409ffc0 legacy=0x7f4fd4135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient: _add_conns ranks=[4,0] 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc0003700 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4fc0006270 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd40a15e0 con 0x7f4fc0006270 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fc0003700 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0006270 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0003700 auth_method 0 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.484+0000 7f4fd89b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.688+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0006270 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0003700 auth_method 0 2024-09-15T21:22:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:31.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:32.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.092+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0006270 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0003700 auth_method 0 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:32.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:32.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:32.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:32.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:32.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:32.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:32.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.896+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0006270 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0003700 auth_method 0 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:32.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:33.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:33.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:33.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:33.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:33.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:33.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:34.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:34.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:34.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:34.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:34.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:34.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient(hunting): tick 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc0003700 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0006270 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:34.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 reap_dead start 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4fd409ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc000a690 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4fc000ab90 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.484+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc000afc0 con 0x7f4fc000ab90 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd40a15e0 con 0x7f4fd409ffc0 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fc000a690 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc000ab90 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000a690 auth_method 0 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:34.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc000ab90 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000a690 auth_method 0 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:34.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:35.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:35.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000a690 auth_method 0 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc000ab90 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:35.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:35.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:35.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:35.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:35.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:35.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:35.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc000ab90 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000a690 auth_method 0 2024-09-15T21:22:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:22:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:35.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:36.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:36.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:36.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:36.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:36.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:36.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:37.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:37.224+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:37.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:37.224+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:37.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:37.224+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:37.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): tick 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fd409ffc0 legacy=0x7f4fc0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc000ab90 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient: _add_conns ranks=[2,3] 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-b con 0x7f4fc0009450 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 0x7f4fc00137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-a con 0x7f4fc00113f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fc00113f0 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd40a1bf0 con 0x7f4fc0009450 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 msgr2=0x7f4fc00137e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 0x7f4fc00137e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0009450 auth_method 0 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 msgr2=0x7f4fc00137e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 0x7f4fc00137e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0009450 auth_method 0 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:37.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 msgr2=0x7f4fc00137e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 0x7f4fc00137e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0009450 auth_method 0 2024-09-15T21:22:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:38.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:38.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:38.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:38.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:38.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:38.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:38.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:38.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:38.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 msgr2=0x7fd3501cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:22:38.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 msgr2=0x7fd3501cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:38.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00e500 tx=0x7fd34c00e9c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:38.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00e500 tx=0x7fd34c00e9c0 comp rx=0 tx=0).stop 2024-09-15T21:22:38.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:38.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:38.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 msgr2=0x7fd3501cd5e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:38.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd3501cd5e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:38.362 INFO:tasks.ceph.mon.b.smithi119.stderr:2024-09-15T21:22:38.355+0000 7f9826265640 -1 mon.b@1(probing) e1 get_health_metrics reporting 10 slow ops, oldest is log(1 entries from seq 34 at 2024-09-15T21:22:07.982796+0000) 2024-09-15T21:22:38.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:38.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:38.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): picked mon.c con 0x7fd3501ce230 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:38.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 0x7fd334049c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:38.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): picked mon.b con 0x7fd334047790 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:38.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:38.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): picked mon.a con 0x7fd3501cddb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd33404cf10 con 0x7fd3501cddb0 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd33404d090 con 0x7fd334047790 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd33404d210 con 0x7fd3501ce230 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd345ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 0x7fd334049c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 msgr2=0x7fd33404c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:38.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 10 monclient(hunting): get_auth_request con 0x7fd3501ce230 auth_method 0 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.355+0000 7fd355369640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): get_auth_request con 0x7fd334047790 auth_method 0 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:38.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient(hunting): handle_auth_done global_id 64129 payload 293 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 msgr2=0x7fd334046f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 msgr2=0x7fd33404c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:38.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient: _finish_hunting 0 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 monclient: found mon.b 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7fd33404d410 con 0x7fd334047790 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) -- 0x7fd33c003530 con 0x7fd334047790 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient: _finish_auth 0 2024-09-15T21:22:38.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 10 monclient: _check_auth_tickets 2024-09-15T21:22:38.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:38.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 0x7fd334049c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd34801c6e0 tx=0x7fd348019a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:38.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480bf020 con 0x7fd334047790 2024-09-15T21:22:38.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd345ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:38.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd345ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd345ffb640 10 monclient: dump: 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:38.372 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:38.373 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:38.373 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:38.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348003ba0 con 0x7fd334047790 2024-09-15T21:22:38.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd345ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:38.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.359+0000 7fd346ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:38.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:38.363+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd348004030 con 0x7fd334047790 2024-09-15T21:22:38.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:38.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0009450 auth_method 0 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 msgr2=0x7f4fc00137e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 0x7f4fc00137e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:38.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:39.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:39.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:39.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:39.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:40.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:40.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:40.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:40.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:40.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:40.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:40.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:40.223+0000 7fd344ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:40.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): tick 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 msgr2=0x7f4fc0009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009450 0x7f4fc0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 msgr2=0x7f4fc00137e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc00113f0 0x7f4fc00137e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 reap_dead start 2024-09-15T21:22:40.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4fc0009450 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-a con 0x7f4fc000a690 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4fc0009830 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc0009130 con 0x7f4fc0009450 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc000afc0 con 0x7f4fc0009830 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fc000a690 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0009450 legacy=0x7f4fc0009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:40.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0009450 legacy=0x7f4fc0009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:40.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.692+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:40.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:40.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:41.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:41.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0009450 legacy=0x7f4fc0009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:41.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:41.096+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:41.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:41.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:41.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:41.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:41.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:41.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:41.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:41.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:41.428 INFO:tasks.ceph.mon.c.smithi017.stderr:2024-09-15T21:22:41.424+0000 7f1418784640 -1 mon.c@2(electing) e1 get_health_metrics reporting 2 slow ops, oldest is log(1 entries from seq 187 at 2024-09-15T21:22:08.142945+0000) 2024-09-15T21:22:41.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:41.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0009450 legacy=0x7f4fc0009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:41.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:41.900+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:41.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:41.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:42.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:42.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:42.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:42.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:42.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:42.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:43.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:43.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:43.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:43.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 msgr2=0x7fd334049c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:22:43.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 msgr2=0x7fd334049c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:43.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 0x7fd334049c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd34801c6e0 tx=0x7fd348019a10 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:43.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 0x7fd334049c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd34801c6e0 tx=0x7fd348019a10 comp rx=0 tx=0).stop 2024-09-15T21:22:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.356+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fc0009830 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-09-15T21:22:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.356+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fc0009830 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:22:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.356+0000 7f4fd91b6640 1 --1- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fc0009830 0x7f4fc0008730 :-1 s=OPENED pgs=59 cs=1 l=1).handle_message read tag failed 2024-09-15T21:22:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.356+0000 7f4fd91b6640 1 --1- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fc0009830 0x7f4fc0008730 :-1 s=OPENED pgs=59 cs=1 l=1).fault on lossy channel, failing 2024-09-15T21:22:43.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.356+0000 7f4fd37fe640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.119:6789/0 2024-09-15T21:22:43.362 INFO:tasks.ceph.mon.b.smithi119.stderr:2024-09-15T21:22:43.355+0000 7f9826265640 -1 mon.b@1(electing) e1 get_health_metrics reporting 10 slow ops, oldest is log(1 entries from seq 34 at 2024-09-15T21:22:07.982796+0000) 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 msgr2=0x7fd334049c20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd334047790 0x7fd334049c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): picked mon.c con 0x7fd3501ce230 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): picked mon.a con 0x7fd3501cddb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd334053ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:43.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): picked mon.b con 0x7fd33c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd3340548d0 con 0x7fd3501cddb0 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd334054a50 con 0x7fd33c0053d0 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd334054bd0 con 0x7fd3501ce230 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd345ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355b6a640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 msgr2=0x7fd33404c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355b6a640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355369640 10 monclient(hunting): get_auth_request con 0x7fd3501ce230 auth_method 0 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355369640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:43.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355369640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.355+0000 7fd355369640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd334053ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): get_auth_request con 0x7fd33c0053d0 auth_method 0 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient(hunting): handle_auth_done global_id 64129 payload 293 2024-09-15T21:22:43.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 msgr2=0x7fd334053ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd334053ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 msgr2=0x7fd33404c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient: _finish_hunting 0 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 monclient: found mon.c 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7fd33404dc00 con 0x7fd3501ce230 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 -- 172.21.15.119:0/665424071 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) -- 0x7fd350134fc0 con 0x7fd3501ce230 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient: _finish_auth 0 2024-09-15T21:22:43.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 10 monclient: _check_auth_tickets 2024-09-15T21:22:43.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:43.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd355369640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd34c04e120 tx=0x7fd34c044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:43.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c044070 con 0x7fd3501ce230 2024-09-15T21:22:43.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd345ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:43.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd345ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd345ffb640 10 monclient: dump: 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c01d460 con 0x7fd3501ce230 2024-09-15T21:22:43.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd345ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:43.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.359+0000 7fd3477fe640 4 set_mon_vals no callback set 2024-09-15T21:22:43.370 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:43.363+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd34c052990 con 0x7fd3501ce230 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): tick 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 msgr2=0x7f4fc0005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.17:3300/0 conn(0x7f4fc000a690 0x7f4fc0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fc0009830 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.17:6789/0 conn(0x7f4fc0009450 legacy=0x7f4fc0009f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:43.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient: _add_conns ranks=[2,4] 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-b con 0x7f4fc000d3c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc000fcf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fc000fcf0 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd40a1bf0 con 0x7f4fc000d3c0 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc000d3c0 auth_method 0 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:43.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000fcf0 auth_method 0 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:43.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:43.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc000d3c0 auth_method 0 2024-09-15T21:22:43.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000fcf0 auth_method 0 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:43.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:43.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:43.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:43.692+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc000d3c0 auth_method 0 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000fcf0 auth_method 0 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.096+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:22:44.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:44.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:44.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:44.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:44.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:44.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc000d3c0 auth_method 0 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000fcf0 auth_method 0 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:44.900+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:22:45.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:45.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:45.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:45.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:45.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:45.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.223+0000 7fd344ff9640 10 monclient: tick 2024-09-15T21:22:46.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.223+0000 7fd344ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.223+0000 7fd344ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.271+0000 7fd345ffb640 1 -- 172.21.15.119:0/665424071 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v140) ==== 124+0+0 (secure 0 0 0) 0x7fd34c018d00 con 0x7fd3501ce230 2024-09-15T21:22:46.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.271+0000 7fd345ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T21:22:46.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.271+0000 7fd345ffb640 10 monclient: _finish_command 3 = system:0 set pool 12 size to 2 2024-09-15T21:22:46.276 INFO:tasks.workunit.client.0.smithi119.stderr:set pool 12 size to 2 2024-09-15T21:22:46.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd33403b9f0 msgr2=0x7fd33403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd33403b9f0 0x7fd33403deb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd340001870 tx=0x7fd340009040 comp rx=0 tx=0).stop 2024-09-15T21:22:46.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 10 monclient: shutdown 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 msgr2=0x7fd334046f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd34c04e120 tx=0x7fd34c044000 comp rx=0 tx=0).stop 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 shutdown_connections 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c0053d0 0x7fd334053ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3501cddb0 0x7fd33404c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd3501ce230 0x7fd334046f90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 --2- 172.21.15.119:0/665424071 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd33403b9f0 0x7fd33403deb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 >> 172.21.15.119:0/665424071 conn(0x7fd3500a66e0 msgr2=0x7fd35013d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 shutdown_connections 2024-09-15T21:22:46.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.275+0000 7fd3575cd640 1 -- 172.21.15.119:0/665424071 wait complete. 2024-09-15T21:22:46.301 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T21:22:46.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient: get_monmap_and_config 2024-09-15T21:22:46.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient: build_initial_monmap 2024-09-15T21:22:46.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient: monmap: 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:46.418185+0000 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:46.418185+0000 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.420 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:46.421 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:46.421 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:46.421 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:46.421 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:46.421 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 1 Processor -- start 2024-09-15T21:22:46.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 1 -- start start 2024-09-15T21:22:46.432 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient: init 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff3b409f640 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff3b40a0130 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 0x7ff3b413d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient(hunting): picked mon.noname-b con 0x7ff3b409d9a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3b409e880 con 0x7ff3b409f640 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3b409eb60 con 0x7ff3b40a0130 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b409de10 con 0x7ff3b409d9a0 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3baa71640 10 monclient(hunting): authenticate will time out at 1285.215470s 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b880d640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7ff3b409f640 legacy=0x7ff3b409fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff3b40a0130 0x7ff3b4139c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40304/0 (socket says 172.21.15.119:40304) 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b3fff640 1 -- 172.21.15.119:0/2257239620 learned_addr learned my addr 172.21.15.119:0/2257239620 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b900e640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 0x7ff3b413d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919262420 0 0) 0x7ff3b409eb60 con 0x7ff3b40a0130 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b900e640 10 monclient(hunting): get_auth_request con 0x7ff3b409d9a0 auth_method 0 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b900e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.434 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b900e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b900e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.415+0000 7ff3b37fe640 10 monclient(hunting): my global_id is 64156 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3a4003530 con 0x7ff3b40a0130 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4275778088 0 0) 0x7ff3a4003530 con 0x7ff3b40a0130 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 msgr2=0x7ff3b413d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 0x7ff3b413d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 >> v1:172.21.15.17:6789/0 conn(0x7ff3b409f640 legacy=0x7ff3b409fa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:22:46.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b409ee40 con 0x7ff3b40a0130 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff3a0001700 con 0x7ff3b40a0130 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 5 monclient: authenticate success, global_id 64156 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: dump: 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.438 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 >> v1:172.21.15.119:6789/0 conn(0x7ff3b40a0130 legacy=0x7ff3b4139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 0x7ff3a40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): picked mon.c con 0x7ff3b409d9a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a4003f40 0x7ff3a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): picked mon.a con 0x7ff3a4003f40 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): picked mon.b con 0x7ff3a4004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b409de10 con 0x7ff3a4003f40 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a400bb60 con 0x7ff3a4004d50 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a400df40 con 0x7ff3b409d9a0 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b3fff640 1 -- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a4003f40 msgr2=0x7ff3a4004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:46.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b3fff640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a4003f40 0x7ff3a4004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): get_auth_request con 0x7ff3a4004d50 auth_method 0 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b880d640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 0x7ff3a40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b880d640 10 monclient(hunting): get_auth_request con 0x7ff3b409d9a0 auth_method 0 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b880d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b880d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b880d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient(hunting): handle_auth_done global_id 64156 payload 293 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 -- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 msgr2=0x7ff3a40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b409d9a0 0x7ff3a40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 -- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a4003f40 msgr2=0x7ff3a4004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a4003f40 0x7ff3a4004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 monclient: found mon.b 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 -- 172.21.15.119:0/2257239620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a4003530 con 0x7ff3a4004d50 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009fc0 tx=0x7ff3ac021070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1459e6d3e64fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0b6070 con 0x7ff3a4004d50 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: dump: 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.444 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac0b1440 con 0x7ff3a4004d50 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 4 set_mon_vals no callback set 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 1 -- 172.21.15.119:0/2257239620 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac009c10 con 0x7ff3a4004d50 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b37fe640 10 monclient: dump: 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:46.445 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.446 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 10 monclient: shutdown 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 1 -- 172.21.15.119:0/2257239620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 msgr2=0x7ff3a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 1 --2- 172.21.15.119:0/2257239620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009fc0 tx=0x7ff3ac021070 comp rx=0 tx=0).stop 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3b900e640 1 -- 172.21.15.119:0/2257239620 reap_dead start 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 1 -- 172.21.15.119:0/2257239620 shutdown_connections 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 1 -- 172.21.15.119:0/2257239620 >> 172.21.15.119:0/2257239620 conn(0x7ff3b40a2ae0 msgr2=0x7ff3b40a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 1 -- 172.21.15.119:0/2257239620 shutdown_connections 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.419+0000 7ff3baa71640 1 -- 172.21.15.119:0/2257239620 wait complete. 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: build_initial_monmap 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: monmap: 2024-09-15T21:22:46.447 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 Processor -- start 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- start start 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: init 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3b411e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): picked mon.noname-b con 0x7ff3a4004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b41cb6b0 0x7ff3b41cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): picked mon.noname-c con 0x7ff3b41cb6b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3b411c630 0x7ff3b411cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): picked mon.noname-a con 0x7ff3b411c630 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b40a1be0 con 0x7ff3b411c630 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b40a1a60 con 0x7ff3a4004d50 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b40a1d60 con 0x7ff3b41cb6b0 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient(hunting): authenticate will time out at 1285.221362s 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3b411e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b900e640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3b411c630 msgr2=0x7ff3b411cac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3b411e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36734/0 (socket says 172.21.15.119:36734) 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b900e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3b411c630 0x7ff3b411cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 -- 172.21.15.119:0/3882355029 learned_addr learned my addr 172.21.15.119:0/3882355029 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): get_auth_request con 0x7ff3a4004d50 auth_method 0 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b41cb6b0 0x7ff3b41cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 10 monclient(hunting): get_auth_request con 0x7ff3b41cb6b0 auth_method 0 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient(hunting): handle_auth_done global_id 64159 payload 824 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 -- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b41cb6b0 msgr2=0x7ff3b41cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b41cb6b0 0x7ff3b41cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 -- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3b411c630 msgr2=0x7ff3b411cac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3b411c630 0x7ff3b411cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 monclient: found mon.noname-b 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 -- 172.21.15.119:0/3882355029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b411fde0 con 0x7ff3a4004d50 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b880d640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3b411e8c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d5b0 tx=0x7ff3a800da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8016020 con 0x7ff3a4004d50 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: dump: 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.453 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a8003bd0 con 0x7ff3a4004d50 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 5 monclient: authenticate success, global_id 64159 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _renew_subs 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b411dd80 con 0x7ff3a4004d50 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80050d0 con 0x7ff3a4004d50 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 10 monclient: dump: 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.456 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b2ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _renew_subs 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b406ed80 con 0x7ff3a4004d50 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b409fa20 con 0x7ff3a4004d50 2024-09-15T21:22:46.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff3a8019020 con 0x7ff3a4004d50 2024-09-15T21:22:46.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff39803b970 0x7ff39803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 7975+0+0 (secure 0 0 0) 0x7ff3a8050220 con 0x7ff3a4004d50 2024-09-15T21:22:46.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff39803b970 0x7ff39803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.423+0000 7ff3b3fff640 10 monclient: get_auth_request con 0x7ff39803b970 auth_method 0 2024-09-15T21:22:46.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.427+0000 7ff3b3fff640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff39803b970 0x7ff39803de30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3b411fff0 tx=0x7ff3a0001a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.431+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff3a8020030 con 0x7ff3a4004d50 2024-09-15T21:22:46.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.431+0000 7ff3b17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.431+0000 7ff3b17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:46.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): tick 2024-09-15T21:22:46.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:22:46.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:22:46.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:22:46.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:46.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:46.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd91b6640 1 -- 172.21.15.17:0/4037047651 reap_dead start 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc000fcf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-b con 0x7f4fc000d3c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4fc0009830 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc0013890 con 0x7f4fc0009830 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4073df0 con 0x7f4fc000fcf0 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 1 -- 172.21.15.17:0/4037047651 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd40a1bf0 con 0x7f4fc000d3c0 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> v2:172.21.15.119:3300/0 conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000d3c0 auth_method 0 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc000fcf0 auth_method 0 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_done global_id 64162 payload 274 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fc000f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 >> v1:172.21.15.119:6789/0 conn(0x7f4fc0009830 legacy=0x7f4fc0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 1 monclient: found mon.noname-c 2024-09-15T21:22:46.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd89b5640 1 -- 172.21.15.17:0/4037047651 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc0009130 con 0x7f4fc000fcf0 2024-09-15T21:22:46.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.488+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000fcf0 0x7f4fc0010100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005010 tx=0x7f4fc401d0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c2e4bc436288765 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 1 -- 172.21.15.17:0/4037047651 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40b4070 con 0x7f4fc000fcf0 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: dump: 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 1 -- 172.21.15.17:0/4037047651 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc40a7c50 con 0x7f4fc000fcf0 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 1 -- 172.21.15.17:0/4037047651 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40af660 con 0x7f4fc000fcf0 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 5 monclient: authenticate success, global_id 64162 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 4 set_mon_vals no callback set 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fd37fe640 10 monclient: dump: 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.503 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 10 monclient: shutdown 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 -- 172.21.15.17:0/4037047651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000fcf0 msgr2=0x7f4fc0010100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000fcf0 0x7f4fc0010100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005010 tx=0x7f4fc401d0b0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 -- 172.21.15.17:0/4037047651 shutdown_connections 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000fcf0 0x7f4fc0010100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 --2- 172.21.15.17:0/4037047651 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc000d3c0 0x7f4fc000f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 -- 172.21.15.17:0/4037047651 >> 172.21.15.17:0/4037047651 conn(0x7f4fd40a66a0 msgr2=0x7f4fd40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 -- 172.21.15.17:0/4037047651 shutdown_connections 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.496+0000 7f4fdac19640 1 -- 172.21.15.17:0/4037047651 wait complete. 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: build_initial_monmap 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: monmap: 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 Processor -- start 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 -- start start 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: init 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 0x7f4fd41cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc0009830 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000d3c0 0x7f4fd41ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): picked mon.noname-b con 0x7f4fc000d3c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc000fcf0 0x7f4fd4122fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): picked mon.noname-a con 0x7f4fc000fcf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4136210 con 0x7f4fc000fcf0 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4136090 con 0x7f4fc000d3c0 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd4136390 con 0x7f4fc0009830 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient(hunting): authenticate will time out at 1289.897973s 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd91b6640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc000fcf0 msgr2=0x7f4fd4122fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 0x7f4fd41cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd91b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc000fcf0 0x7f4fd4122fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 0x7f4fd41cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:41312/0 (socket says 172.21.15.17:41312) 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 -- 172.21.15.17:0/3686106769 learned_addr learned my addr 172.21.15.17:0/3686106769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request con 0x7f4fc0009830 auth_method 0 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000d3c0 0x7f4fd41ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request con 0x7f4fc000d3c0 auth_method 0 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient(hunting): handle_auth_done global_id 44627 payload 824 2024-09-15T21:22:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 -- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000d3c0 msgr2=0x7f4fd41ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000d3c0 0x7f4fd41ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 -- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc000fcf0 msgr2=0x7f4fd4122fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc000fcf0 0x7f4fd4122fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 monclient: found mon.noname-c 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 -- 172.21.15.17:0/3686106769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd409d0f0 con 0x7f4fc0009830 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd89b5640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 0x7f4fd41cdd30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005010 tx=0x7f4fc4005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40bc070 con 0x7f4fc0009830 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: dump: 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc40a7a50 con 0x7f4fc0009830 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 5 monclient: authenticate success, global_id 44627 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: _renew_subs 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd4126a30 con 0x7f4fc0009830 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40b4040 con 0x7f4fc0009830 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd17fa640 10 monclient: dump: 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fd2ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: _renew_subs 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.500+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41352c0 con 0x7f4fc0009830 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4fc40bf070 con 0x7f4fc0009830 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fd17fa640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fc003e4c0 0x7f4fc0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fdac19640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(140..140 src has 1..140) ==== 7975+0+0 (secure 0 0 0) 0x7f4fc40ec880 con 0x7f4fc0009830 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fdac19640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fdac19640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd4138ab0 con 0x7f4fc0009830 2024-09-15T21:22:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fc003e4c0 0x7f4fc0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fd3fff640 10 monclient: get_auth_request con 0x7f4fc003e4c0 auth_method 0 2024-09-15T21:22:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.504+0000 7f4fd3fff640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fc003e4c0 0x7f4fc0040980 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4072740 tx=0x7f4fc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.508+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4fd409d0f0 con 0x7f4fc0009830 2024-09-15T21:22:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.508+0000 7f4fd17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.508+0000 7f4fd17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:46.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.607+0000 7ff3baa71640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:46.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.607+0000 7ff3baa71640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:46.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.607+0000 7ff3baa71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:46.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:46.607+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7ff3b406c460 con 0x7ff3a4004d50 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.744+0000 7f4fdac19640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.744+0000 7f4fdac19640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.744+0000 7f4fdac19640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.744+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4fd41249b0 con 0x7f4fc0009830 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fd17fa640 1 -- 172.21.15.17:0/3686106769 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7f4fc4012040 con 0x7f4fc0009830 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fd17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:22:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fd17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:46.752 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":52,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fc003e4c0 msgr2=0x7f4fc0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fc003e4c0 0x7f4fc0040980 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4072740 tx=0x7f4fc8009040 comp rx=0 tx=0).stop 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 10 monclient: shutdown 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 msgr2=0x7f4fd41cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 0x7f4fd41cdd30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005010 tx=0x7f4fc4005040 comp rx=0 tx=0).stop 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 shutdown_connections 2024-09-15T21:22:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fc003e4c0 0x7f4fc0040980 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc000fcf0 0x7f4fd4122fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc000d3c0 0x7f4fd41ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 --2- 172.21.15.17:0/3686106769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fc0009830 0x7f4fd41cdd30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 >> 172.21.15.17:0/3686106769 conn(0x7f4fd40a66a0 msgr2=0x7f4fd4070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 shutdown_connections 2024-09-15T21:22:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.748+0000 7f4fdac19640 1 -- 172.21.15.17:0/3686106769 wait complete. 2024-09-15T21:22:46.772 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:22:46.772 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:22:46.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 10 monclient: get_monmap_and_config 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 10 monclient: build_initial_monmap 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 10 monclient: monmap: 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:46.900723+0000 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:46.900723+0000 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:46.903 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:46.903 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:46.903 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:46.903 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:46.903 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 1 Processor -- start 2024-09-15T21:22:46.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 1 -- start start 2024-09-15T21:22:46.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 10 monclient: init 2024-09-15T21:22:46.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.896+0000 7f07af824640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:46.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f07a80a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient(hunting): picked mon.noname-b con 0x7f07a809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f07a80a1380 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a80a3130 con 0x7f07a80a1380 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a8073df0 con 0x7f07a80a0890 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07a80a1bf0 con 0x7f07a809ffc0 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07af824640 10 monclient(hunting): authenticate will time out at 1290.296123s 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07addc1640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f07a80a1380 legacy=0x7f07a813d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a813aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:41324/0 (socket says 172.21.15.17:41324) 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 -- 172.21.15.17:0/3450216673 learned_addr learned my addr 172.21.15.17:0/3450216673 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): get_auth_request con 0x7f07a809ffc0 auth_method 0 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient(hunting): handle_auth_done global_id 44630 payload 274 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 -- 172.21.15.17:0/3450216673 >> v1:172.21.15.119:6789/0 conn(0x7f07a80a0890 legacy=0x7f07a80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 -- 172.21.15.17:0/3450216673 >> v1:172.21.15.17:6789/0 conn(0x7f07a80a1380 legacy=0x7f07a813d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:46.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 monclient: found mon.noname-b 2024-09-15T21:22:46.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.900+0000 7f07acdbf640 1 -- 172.21.15.17:0/3450216673 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a809ccd0 con 0x7f07a809ffc0 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07acdbf640 1 --2- 172.21.15.17:0/3450216673 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a813aac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07980098a0 tx=0x7f079801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7641c7bed83c3bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 1 -- 172.21.15.17:0/3450216673 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980b1070 con 0x7f07a809ffc0 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: dump: 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 5 monclient: authenticate success, global_id 44630 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 1 -- 172.21.15.17:0/3450216673 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07980ac640 con 0x7f07a809ffc0 2024-09-15T21:22:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 4 set_mon_vals no callback set 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 1 -- 172.21.15.17:0/3450216673 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980aca10 con 0x7f07a809ffc0 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079ffff640 10 monclient: dump: 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: shutdown 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/3450216673 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 msgr2=0x7f07a813aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 --2- 172.21.15.17:0/3450216673 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a813aac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07980098a0 tx=0x7f079801c440 comp rx=0 tx=0).stop 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/3450216673 shutdown_connections 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 --2- 172.21.15.17:0/3450216673 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a813aac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/3450216673 >> 172.21.15.17:0/3450216673 conn(0x7f07a80a66a0 msgr2=0x7f07a80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/3450216673 shutdown_connections 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/3450216673 wait complete. 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: build_initial_monmap 2024-09-15T21:22:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: monmap: 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 Processor -- start 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- start start 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: init 2024-09-15T21:22:46.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a81353a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): picked mon.noname-c con 0x7f07a809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07a80a0890 0x7f07a81358e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): picked mon.noname-b con 0x7f07a80a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a80a1380 0x7f07a811c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): picked mon.noname-a con 0x7f07a80a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81412e0 con 0x7f07a80a1380 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8141160 con 0x7f07a80a0890 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07a8141460 con 0x7f07a809ffc0 2024-09-15T21:22:46.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:46.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient(hunting): authenticate will time out at 1290.302958s 2024-09-15T21:22:46.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07addc1640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a80a1380 msgr2=0x7f07a811c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:46.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07addc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a80a1380 0x7f07a811c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:46.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a81353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a81353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:41338/0 (socket says 172.21.15.17:41338) 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 -- 172.21.15.17:0/1669833326 learned_addr learned my addr 172.21.15.17:0/1669833326 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): get_auth_request con 0x7f07a809ffc0 auth_method 0 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07acdbf640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07a80a0890 0x7f07a81358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07acdbf640 10 monclient(hunting): get_auth_request con 0x7f07a80a0890 auth_method 0 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07acdbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07acdbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07acdbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient(hunting): handle_auth_done global_id 44633 payload 824 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07a80a0890 msgr2=0x7f07a81358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07a80a0890 0x7f07a81358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a80a1380 msgr2=0x7f07a811c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a80a1380 0x7f07a811c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient: _finish_hunting 0 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 monclient: found mon.noname-c 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a811dc00 con 0x7f07a809ffc0 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07ad5c0640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a81353a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c530 tx=0x7f07a400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4017070 con 0x7f07a809ffc0 2024-09-15T21:22:46.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: dump: 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a40052d0 con 0x7f07a809ffc0 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4012870 con 0x7f07a809ffc0 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079dffb640 10 monclient: dump: 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 5 monclient: authenticate success, global_id 44633 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: _renew_subs 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f079f7fe640 4 set_mon_vals no callback set 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a8135e20 con 0x7f07a809ffc0 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: _renew_subs 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.904+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a811f0a0 con 0x7f07a809ffc0 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07af824640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07af824640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 0x7f07a811ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07af824640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07a811d340 con 0x7f07a811ca20 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 0x7f07a811ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient: get_auth_request con 0x7f07a811ca20 auth_method 0 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f07a401a070 con 0x7f07a809ffc0 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 10 monclient(hunting): handle_auth_done global_id 44636 payload 824 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07a809d0f0 con 0x7f07a811ca20 2024-09-15T21:22:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f078c03b9a0 0x7f078c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(140..140 src has 1..140) ==== 7975+0+0 (secure 0 0 0) 0x7f07a404f980 con 0x7f07a809ffc0 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07acdbf640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 0x7f07a811ce00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07a806f1c0 tx=0x7f079801ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980bb070 con 0x7f07a811ca20 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 10 monclient: dump: 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07addc1640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f078c03b9a0 0x7f078c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07addc1640 10 monclient: get_auth_request con 0x7f078c03b9a0 auth_method 0 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0798009490 con 0x7f07a811ca20 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 msgr2=0x7f07a811ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 0x7f07a811ce00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07a806f1c0 tx=0x7f079801ac80 comp rx=0 tx=0).stop 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 msgr2=0x7f07a811ce00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f079dffb640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a811ca20 0x7f07a811ce00 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07a806f1c0 tx=0x7f079801ac80 comp rx=0 tx=0).stop 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.908+0000 7f07addc1640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f078c03b9a0 0x7f078c03de60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07a811d650 tx=0x7f07a0009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.912+0000 7f07af824640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.912+0000 7f07af824640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 0x7f07a8138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.912+0000 7f07af824640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.912+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07a813dc10 con 0x7f07a8138ab0 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 0x7f07a8138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient: get_auth_request con 0x7f07a8138ab0 auth_method 0 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 10 monclient(hunting): handle_auth_done global_id 44639 payload 824 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 1 -- 172.21.15.17:0/1669833326 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f07a80a0c70 con 0x7f07a8138ab0 2024-09-15T21:22:46.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07acdbf640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 0x7f07a8138e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07980097f0 tx=0x7f07980191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980bb040 con 0x7f07a8138ab0 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 10 monclient: dump: 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:46.936 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4868 (secure 0 0 0) 0x7f07980c5070 con 0x7f07a8138ab0 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 msgr2=0x7f07a8138e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 0x7f07a8138e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07980097f0 tx=0x7f07980191e0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 msgr2=0x7f07a8138e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f079dffb640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a8138ab0 0x7f07a8138e90 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07980097f0 tx=0x7f07980191e0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.937 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 52, 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 135514, 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:46.938 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:46.939 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:46.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:46.941 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:46.942 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:46.943 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:46.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:46.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:46.946 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.947 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:46.948 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:46.949 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:46.950 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f078c03b9a0 msgr2=0x7f078c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f078c03b9a0 0x7f078c03de60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07a811d650 tx=0x7f07a0009270 comp rx=0 tx=0).stop 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 10 monclient: shutdown 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 msgr2=0x7f07a81353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 --2- 172.21.15.17:0/1669833326 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07a809ffc0 0x7f07a81353a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c530 tx=0x7f07a400c9f0 comp rx=0 tx=0).stop 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07addc1640 1 -- 172.21.15.17:0/1669833326 reap_dead start 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07addc1640 1 -- 172.21.15.17:0/1669833326 reap_dead start 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 shutdown_connections 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 >> 172.21.15.17:0/1669833326 conn(0x7f07a80a66a0 msgr2=0x7f07a80a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:46.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 shutdown_connections 2024-09-15T21:22:46.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:46.916+0000 7f07af824640 1 -- 172.21.15.17:0/1669833326 wait complete. 2024-09-15T21:22:46.953 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:22:46.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:22:46.953 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:22:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient: get_monmap_and_config 2024-09-15T21:22:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient: build_initial_monmap 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient: monmap: 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:22:47.071357+0000 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:22:47.071357+0000 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 1 Processor -- start 2024-09-15T21:22:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 1 -- start start 2024-09-15T21:22:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient: init 2024-09-15T21:22:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:22:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 0x7f87fc0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient(hunting): picked mon.noname-b con 0x7f87fc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f87fc09ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f87fc0a11b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87fc136490 con 0x7f87fc09ffc0 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87fc0a3130 con 0x7f87fc0a11b0 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f87fc073df0 con 0x7f87fc0a0890 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8803693640 10 monclient(hunting): authenticate will time out at 1290.467116s 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f8800c2e640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f87fc09ffc0 legacy=0x7f87fc135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 0x7f87fc0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 0x7f87fc0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:41364/0 (socket says 172.21.15.17:41364) 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 -- 172.21.15.17:0/121924476 learned_addr learned my addr 172.21.15.17:0/121924476 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): get_auth_request con 0x7f87fc0a0890 auth_method 0 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient(hunting): handle_auth_done global_id 44642 payload 274 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 -- 172.21.15.17:0/121924476 >> v1:172.21.15.119:6789/0 conn(0x7f87fc0a11b0 legacy=0x7f87fc135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 -- 172.21.15.17:0/121924476 >> v1:172.21.15.17:6789/0 conn(0x7f87fc09ffc0 legacy=0x7f87fc135670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient: _finish_hunting 0 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 monclient: found mon.noname-b 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.068+0000 7f880142f640 1 -- 172.21.15.17:0/121924476 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc0a1bf0 con 0x7f87fc0a0890 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 1 --2- 172.21.15.17:0/121924476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 0x7f87fc0a0c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009e80 tx=0x7f87f801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ca946c5e832b71 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 1 -- 172.21.15.17:0/121924476 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f80a56a0 con 0x7f87fc0a0890 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: dump: 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: _finish_auth 0 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 1 -- 172.21.15.17:0/121924476 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f80a5840 con 0x7f87fc0a0890 2024-09-15T21:22:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 5 monclient: authenticate success, global_id 44642 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 1 -- 172.21.15.17:0/121924476 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f80a5c10 con 0x7f87fc0a0890 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 4 set_mon_vals no callback set 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f87f3fff640 10 monclient: dump: 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.083 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: shutdown 2024-09-15T21:22:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- 172.21.15.17:0/121924476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 msgr2=0x7f87fc0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 --2- 172.21.15.17:0/121924476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 0x7f87fc0a0c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009e80 tx=0x7f87f801c910 comp rx=0 tx=0).stop 2024-09-15T21:22:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- 172.21.15.17:0/121924476 shutdown_connections 2024-09-15T21:22:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 --2- 172.21.15.17:0/121924476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a0890 0x7f87fc0a0c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- 172.21.15.17:0/121924476 >> 172.21.15.17:0/121924476 conn(0x7f87fc0a66a0 msgr2=0x7f87fc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- 172.21.15.17:0/121924476 shutdown_connections 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- 172.21.15.17:0/121924476 wait complete. 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: build_initial_monmap 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: monmap: 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 Processor -- start 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- start start 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: init 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc09ffc0 0x7f87fc135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): picked mon.noname-b con 0x7f87fc09ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc0a0890 0x7f87fc1358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): picked mon.noname-a con 0x7f87fc0a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a11b0 0x7f87fc13c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): picked mon.noname-c con 0x7f87fc0a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc141ab0 con 0x7f87fc0a0890 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc141930 con 0x7f87fc09ffc0 2024-09-15T21:22:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f87fc141c30 con 0x7f87fc0a11b0 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8803693640 10 monclient(hunting): authenticate will time out at 1290.475352s 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8800c2e640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc0a0890 msgr2=0x7f87fc1358c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8800c2e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc0a0890 0x7f87fc1358c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a11b0 0x7f87fc13c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a11b0 0x7f87fc13c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:41374/0 (socket says 172.21.15.17:41374) 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 -- 172.21.15.17:0/2610986379 learned_addr learned my addr 172.21.15.17:0/2610986379 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): get_auth_request con 0x7f87fc0a11b0 auth_method 0 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc09ffc0 0x7f87fc135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): get_auth_request con 0x7f87fc09ffc0 auth_method 0 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient(hunting): handle_auth_done global_id 44645 payload 824 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc09ffc0 msgr2=0x7f87fc135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc09ffc0 0x7f87fc135380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc0a0890 msgr2=0x7f87fc1358c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc0a0890 0x7f87fc1358c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient: _finish_hunting 0 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 1 monclient: found mon.noname-c 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.076+0000 7f8801c30640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8801c30640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc13db60 con 0x7f87fc0a11b0 2024-09-15T21:22:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8801c30640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a11b0 0x7f87fc13c460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87f4001ec0 tx=0x7f87f400c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4017070 con 0x7f87fc0a11b0 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: dump: 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f40052d0 con 0x7f87fc0a11b0 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4003800 con 0x7f87fc0a11b0 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: dump: 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f2ffd640 4 set_mon_vals no callback set 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 5 monclient: authenticate success, global_id 44645 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 10 monclient: _renew_subs 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc09d0f0 con 0x7f87fc0a11b0 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 10 monclient: _renew_subs 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc09de70 con 0x7f87fc0a11b0 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 0x7f87fc138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc06f870 con 0x7f87fc138ab0 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f87f401a070 con 0x7f87fc0a11b0 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 0x7f87fc138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient: get_auth_request con 0x7f87fc138ab0 auth_method 0 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f87e003b970 0x7f87e003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(140..140 src has 1..140) ==== 7975+0+0 (secure 0 0 0) 0x7f87f404f9b0 con 0x7f87fc0a11b0 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8800c2e640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f87e003b970 0x7f87e003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 10 monclient(hunting): handle_auth_done global_id 64183 payload 824 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87fc0a0c70 con 0x7f87fc138ab0 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8800c2e640 10 monclient: get_auth_request con 0x7f87e003b970 auth_method 0 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f880142f640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 0x7f87fc138e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06e7b0 tx=0x7f87f80196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8009510 con 0x7f87fc138ab0 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f87f1ffb640 10 monclient: dump: 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.080+0000 7f8800c2e640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f87e003b970 0x7f87e003de30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06f620 tx=0x7f87ec006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f87f80c0690 con 0x7f87fc138ab0 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 msgr2=0x7f87fc138e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 0x7f87fc138e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06e7b0 tx=0x7f87f80196b0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 msgr2=0x7f87fc138e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.084+0000 7f87f1ffb640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc138ab0 0x7f87fc138e90 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06e7b0 tx=0x7f87f80196b0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f8803693640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:22:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f8803693640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 0x7f87fc13c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f8803693640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc09e120 con 0x7f87fc1393d0 2024-09-15T21:22:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 0x7f87fc13c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient: get_auth_request con 0x7f87fc1393d0 auth_method 0 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 10 monclient(hunting): handle_auth_done global_id 64186 payload 824 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 1 -- 172.21.15.17:0/2610986379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f87fc06d020 con 0x7f87fc1393d0 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f880142f640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 0x7f87fc13c9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87f801c950 tx=0x7f87f80a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f80c1da0 con 0x7f87fc1393d0 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f87f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f87f1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.088+0000 7f87f1ffb640 10 monclient: dump: 2024-09-15T21:22:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4702 (secure 0 0 0) 0x7f87f8003fa0 con 0x7f87fc1393d0 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 msgr2=0x7f87fc13c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 0x7f87fc13c9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87f801c950 tx=0x7f87f80a7000 comp rx=0 tx=0).stop 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 msgr2=0x7f87fc13c9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f87f1ffb640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87fc1393d0 0x7f87fc13c9a0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87f801c950 tx=0x7f87f80a7000 comp rx=0 tx=0).stop 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 52, 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 45350, 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:22:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:22:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:22:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:22:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:22:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:22:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:22:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:22:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:22:47.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:22:47.128 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:22:47.128 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:22:47.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f87e003b970 msgr2=0x7f87e003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f87e003b970 0x7f87e003de30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06f620 tx=0x7f87ec006d90 comp rx=0 tx=0).stop 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 10 monclient: shutdown 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a11b0 msgr2=0x7f87fc13c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 --2- 172.21.15.17:0/2610986379 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f87fc0a11b0 0x7f87fc13c460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87f4001ec0 tx=0x7f87f400c890 comp rx=0 tx=0).stop 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8801c30640 1 -- 172.21.15.17:0/2610986379 reap_dead start 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8801c30640 1 -- 172.21.15.17:0/2610986379 reap_dead start 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 shutdown_connections 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 >> 172.21.15.17:0/2610986379 conn(0x7f87fc0a66a0 msgr2=0x7f87fc0a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:47.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 shutdown_connections 2024-09-15T21:22:47.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:22:47.092+0000 7f8803693640 1 -- 172.21.15.17:0/2610986379 wait complete. 2024-09-15T21:22:47.131 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:22:47.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:22:47.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.271+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v141) ==== 82+0+0 (secure 0 0 0) 0x7ff3a801b780 con 0x7ff3a4004d50 2024-09-15T21:22:47.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.271+0000 7ff3b17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:47.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.271+0000 7ff3b17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:47.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3baa71640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:47.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3baa71640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:47.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3baa71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7ff3b4134d40 con 0x7ff3a4004d50 2024-09-15T21:22:47.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3b17fa640 1 -- 172.21.15.119:0/3882355029 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v141) ==== 115+0+0 (secure 0 0 0) 0x7ff3a8009360 con 0x7ff3a4004d50 2024-09-15T21:22:47.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3b17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:22:47.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.383+0000 7ff3b17fa640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T21:22:47.390 INFO:tasks.workunit.client.0.smithi119.stderr:compat weight-set already created 2024-09-15T21:22:47.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff39803b970 msgr2=0x7ff39803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff39803b970 0x7ff39803de30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3b411fff0 tx=0x7ff3a0001a30 comp rx=0 tx=0).stop 2024-09-15T21:22:47.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 10 monclient: shutdown 2024-09-15T21:22:47.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 msgr2=0x7ff3b411e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3b411e8c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d5b0 tx=0x7ff3a800da70 comp rx=0 tx=0).stop 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 shutdown_connections 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff39803b970 0x7ff39803de30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3b411c630 0x7ff3b411cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff3b41cb6b0 0x7ff3b41cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 --2- 172.21.15.119:0/3882355029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a4004d50 0x7ff3b411e8c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 >> 172.21.15.119:0/3882355029 conn(0x7ff3b40a2ae0 msgr2=0x7ff3b4071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 shutdown_connections 2024-09-15T21:22:47.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.387+0000 7ff3baa71640 1 -- 172.21.15.119:0/3882355029 wait complete. 2024-09-15T21:22:47.407 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create cool flat 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.523+0000 7f47fdf94640 10 monclient: get_monmap_and_config 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.523+0000 7f47fdf94640 10 monclient: build_initial_monmap 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.523+0000 7f47fdf94640 10 monclient: monmap: 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:47.527053+0000 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:47.527053+0000 2024-09-15T21:22:47.528 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:47.529 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.523+0000 7f47fdf94640 1 Processor -- start 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.523+0000 7f47fdf94640 1 -- start start 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.523+0000 7f47fdf94640 10 monclient: init 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f47f80a29b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47f8073a30 0x7f47f813ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): picked mon.noname-c con 0x7f47f8073a30 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47f813b0a0 0x7f47f813d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): picked mon.noname-a con 0x7f47f813b0a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f80727d0 con 0x7f47f80a29b0 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f47f806f900 con 0x7f47f813b0a0 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f47f8073070 con 0x7f47f8073a30 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient(hunting): authenticate will time out at 1286.324282s 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f77fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f47f80a29b0 legacy=0x7f47f80a2d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f7fff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f47f813b0a0 msgr2=0x7f47f813d490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47f8073a30 0x7f47f813ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47f813b0a0 0x7f47f813d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47f8073a30 0x7f47f813ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36736/0 (socket says 172.21.15.119:36736) 2024-09-15T21:22:47.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 -- 172.21.15.119:0/2507962437 learned_addr learned my addr 172.21.15.119:0/2507962437 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): get_auth_request con 0x7f47f8073a30 auth_method 0 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient(hunting): handle_auth_done global_id 64189 payload 274 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 -- 172.21.15.119:0/2507962437 >> v2:172.21.15.17:3300/0 conn(0x7f47f813b0a0 msgr2=0x7f47f813d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 --2- 172.21.15.119:0/2507962437 >> v2:172.21.15.17:3300/0 conn(0x7f47f813b0a0 0x7f47f813d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 -- 172.21.15.119:0/2507962437 >> v1:172.21.15.17:6789/0 conn(0x7f47f80a29b0 legacy=0x7f47f80a2d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 monclient: found mon.noname-c 2024-09-15T21:22:47.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:47.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 -- 172.21.15.119:0/2507962437 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f8072ab0 con 0x7f47f8073a30 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47effff640 1 --2- 172.21.15.119:0/2507962437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f813ab60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47f80763c0 tx=0x7f47e001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23d7a22db47a914 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 1 -- 172.21.15.119:0/2507962437 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00b2070 con 0x7f47f8073a30 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: dump: 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.539 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 1 -- 172.21.15.119:0/2507962437 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e00a9070 con 0x7f47f8073a30 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:47.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 1 -- 172.21.15.119:0/2507962437 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00ad880 con 0x7f47f8073a30 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 5 monclient: authenticate success, global_id 64189 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 4 set_mon_vals no callback set 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47f6ffd640 10 monclient: dump: 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:47.541 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 10 monclient: shutdown 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- 172.21.15.119:0/2507962437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 msgr2=0x7f47f813ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 --2- 172.21.15.119:0/2507962437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f813ab60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47f80763c0 tx=0x7f47e001c490 comp rx=0 tx=0).stop 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- 172.21.15.119:0/2507962437 shutdown_connections 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 --2- 172.21.15.119:0/2507962437 >> v2:172.21.15.17:3300/0 conn(0x7f47f813b0a0 0x7f47f813d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 --2- 172.21.15.119:0/2507962437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f813ab60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- 172.21.15.119:0/2507962437 >> 172.21.15.119:0/2507962437 conn(0x7f47f80a3d00 msgr2=0x7f47f806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- 172.21.15.119:0/2507962437 shutdown_connections 2024-09-15T21:22:47.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.527+0000 7f47fdf94640 1 -- 172.21.15.119:0/2507962437 wait complete. 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: build_initial_monmap 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: monmap: 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:47.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 Processor -- start 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- start start 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: init 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): picked mon.noname-b con 0x7f47f8073a30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47f80a29b0 0x7f47f81312b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): picked mon.noname-c con 0x7f47f80a29b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47f813b0a0 0x7f47f81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): picked mon.noname-a con 0x7f47f813b0a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81401f0 con 0x7f47f813b0a0 2024-09-15T21:22:47.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47f8069a90 con 0x7f47f8073a30 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f47f8140370 con 0x7f47f80a29b0 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient(hunting): authenticate will time out at 1286.329264s 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f8130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36746/0 (socket says 172.21.15.119:36746) 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 -- 172.21.15.119:0/3760705745 learned_addr learned my addr 172.21.15.119:0/3760705745 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): get_auth_request con 0x7f47f8073a30 auth_method 0 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:47.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f7fff640 1 -- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47f813b0a0 msgr2=0x7f47f81317f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f7fff640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47f813b0a0 0x7f47f81317f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient(hunting): handle_auth_done global_id 64192 payload 824 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 -- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47f80a29b0 msgr2=0x7f47f81312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47effff640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47f80a29b0 0x7f47f81312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47f80a29b0 0x7f47f81312b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 -- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47f813b0a0 msgr2=0x7f47f81317f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47f813b0a0 0x7f47f81317f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:47.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 monclient: found mon.noname-b 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 -- 172.21.15.119:0/3760705745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f811db80 con 0x7f47f8073a30 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f77fe640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f8130d70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47e800b530 tx=0x7f47e800b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8013020 con 0x7f47f8073a30 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: dump: 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.547 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.548 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.548 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:47.548 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.548 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.548 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.548 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e8003df0 con 0x7f47f8073a30 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80052b0 con 0x7f47f8073a30 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 10 monclient: dump: 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.549 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f67fc640 4 set_mon_vals no callback set 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 5 monclient: authenticate success, global_id 64192 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _renew_subs 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f80a5e70 con 0x7f47f8073a30 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _renew_subs 2024-09-15T21:22:47.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f8074600 con 0x7f47f8073a30 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f80a2d90 con 0x7f47f8073a30 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f47e8016020 con 0x7f47f8073a30 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47d803b9a0 0x7f47d803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 8307+0+0 (secure 0 0 0) 0x7f47e8050610 con 0x7f47f8073a30 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.531+0000 7f47effff640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47d803b9a0 0x7f47d803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.535+0000 7f47effff640 10 monclient: get_auth_request con 0x7f47d803b9a0 auth_method 0 2024-09-15T21:22:47.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.535+0000 7f47effff640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47d803b9a0 0x7f47d803de60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f47e00af000 tx=0x7f47e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:47.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.539+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f47e801d020 con 0x7f47f8073a30 2024-09-15T21:22:47.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.539+0000 7f47f4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:47.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.539+0000 7f47f4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:47.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.715+0000 7f47fdf94640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T21:22:47.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.715+0000 7f47fdf94640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T21:22:47.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.715+0000 7f47fdf94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:47.721 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:47.715+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) -- 0x7f47f8074bd0 con 0x7f47f8073a30 2024-09-15T21:22:48.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.283+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 v142) ==== 107+0+0 (secure 0 0 0) 0x7f47e800fe20 con 0x7f47f8073a30 2024-09-15T21:22:48.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.287+0000 7f47f4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T21:22:48.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.287+0000 7f47f4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:48.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T21:22:48.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T21:22:48.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) -- 0x7f47f811df40 con 0x7f47f8073a30 2024-09-15T21:22:48.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47f4ff9640 1 -- 172.21.15.119:0/3760705745 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 weight-set for pool 'cool' already created v143) ==== 149+0+0 (secure 0 0 0) 0x7f47e800d380 con 0x7f47f8073a30 2024-09-15T21:22:48.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47f4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T21:22:48.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47f4ff9640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cool' already created 2024-09-15T21:22:48.408 INFO:tasks.workunit.client.0.smithi119.stderr:weight-set for pool 'cool' already created 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47d803b9a0 msgr2=0x7f47d803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47d803b9a0 0x7f47d803de60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f47e00af000 tx=0x7f47e0008040 comp rx=0 tx=0).stop 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 10 monclient: shutdown 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 msgr2=0x7f47f8130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f8130d70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47e800b530 tx=0x7f47e800b9f0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 shutdown_connections 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47d803b9a0 0x7f47d803de60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47f813b0a0 0x7f47f81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47f80a29b0 0x7f47f81312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 --2- 172.21.15.119:0/3760705745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f8073a30 0x7f47f8130d70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.403+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 >> 172.21.15.119:0/3760705745 conn(0x7f47f80a3d00 msgr2=0x7f47f80a5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.407+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 shutdown_connections 2024-09-15T21:22:48.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.407+0000 7f47fdf94640 1 -- 172.21.15.119:0/3760705745 wait complete. 2024-09-15T21:22:48.425 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create cold positional 2024-09-15T21:22:48.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient: get_monmap_and_config 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient: build_initial_monmap 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient: monmap: 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:48.545103+0000 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:48.545103+0000 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 Processor -- start 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 -- start start 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient: init 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2b3c0a2cf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b3c06d090 0x7f2b3c13af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): picked mon.noname-c con 0x7f2b3c06d090 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b3c0a37e0 0x7f2b3c13d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): picked mon.noname-a con 0x7f2b3c0a37e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3c0695a0 con 0x7f2b3c0a2cf0 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3c06cbf0 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3c09edd0 con 0x7f2b3c06d090 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b41340640 10 monclient(hunting): authenticate will time out at 1287.341748s 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3affd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2b3c0a2cf0 0x7f2b3c0a30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40306/0 (socket says 172.21.15.119:40306) 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3affd640 1 -- 172.21.15.119:0/793631736 learned_addr learned my addr 172.21.15.119:0/793631736 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3b7fe640 1 -- 172.21.15.119:0/793631736 >> v2:172.21.15.17:3300/0 conn(0x7f2b3c0a37e0 msgr2=0x7f2b3c13d450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3b7fe640 1 --2- 172.21.15.119:0/793631736 >> v2:172.21.15.17:3300/0 conn(0x7f2b3c0a37e0 0x7f2b3c13d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 1 --2- 172.21.15.119:0/793631736 >> v2:172.21.15.119:3300/0 conn(0x7f2b3c06d090 0x7f2b3c13af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): get_auth_request con 0x7f2b3c06d090 auth_method 0 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 1 -- 172.21.15.119:0/793631736 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46610573 0 0) 0x7f2b3c0695a0 con 0x7f2b3c0a2cf0 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 10 monclient(hunting): my global_id is 64195 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 1 -- 172.21.15.119:0/793631736 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b24003530 con 0x7f2b3c0a2cf0 2024-09-15T21:22:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient(hunting): handle_auth_done global_id 64198 payload 274 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 1 -- 172.21.15.119:0/793631736 >> v2:172.21.15.17:3300/0 conn(0x7f2b3c0a37e0 msgr2=0x7f2b3c13d450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 1 --2- 172.21.15.119:0/793631736 >> v2:172.21.15.17:3300/0 conn(0x7f2b3c0a37e0 0x7f2b3c13d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 1 -- 172.21.15.119:0/793631736 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3136413730 0 0) 0x7f2b24003530 con 0x7f2b3c0a2cf0 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 1 -- 172.21.15.119:0/793631736 >> v1:172.21.15.119:6789/0 conn(0x7f2b3c0a2cf0 legacy=0x7f2b3c0a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 1 monclient: found mon.noname-c 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b3a7fc640 1 -- 172.21.15.119:0/793631736 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c09e250 con 0x7f2b3c06d090 2024-09-15T21:22:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.543+0000 7f2b39ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b3a7fc640 1 --2- 172.21.15.119:0/793631736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c06d090 0x7f2b3c13af00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0745a0 tx=0x7f2b3001c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f590cffe1668cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 1 -- 172.21.15.119:0/793631736 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b300b0070 con 0x7f2b3c06d090 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: dump: 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.561 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 5 monclient: authenticate success, global_id 64198 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 1 -- 172.21.15.119:0/793631736 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b300ab400 con 0x7f2b3c06d090 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:48.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 4 set_mon_vals no callback set 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 1 -- 172.21.15.119:0/793631736 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c001700 con 0x7f2b3c06d090 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b39ffb640 10 monclient: dump: 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:48.563 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 10 monclient: shutdown 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 -- 172.21.15.119:0/793631736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c06d090 msgr2=0x7f2b3c13af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 --2- 172.21.15.119:0/793631736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c06d090 0x7f2b3c13af00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0745a0 tx=0x7f2b3001c280 comp rx=0 tx=0).stop 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 -- 172.21.15.119:0/793631736 shutdown_connections 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 --2- 172.21.15.119:0/793631736 >> v2:172.21.15.17:3300/0 conn(0x7f2b3c0a37e0 0x7f2b3c13d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 --2- 172.21.15.119:0/793631736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c06d090 0x7f2b3c13af00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 -- 172.21.15.119:0/793631736 >> 172.21.15.119:0/793631736 conn(0x7f2b3c0a3de0 msgr2=0x7f2b3c0a41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 -- 172.21.15.119:0/793631736 shutdown_connections 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 1 -- 172.21.15.119:0/793631736 wait complete. 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 10 monclient: build_initial_monmap 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.547+0000 7f2b41340640 10 monclient: monmap: 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:48.565 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 Processor -- start 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- start start 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: init 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c06d090 0x7f2b3c135180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): picked mon.noname-a con 0x7f2b3c06d090 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b3c0a2cf0 0x7f2b3c1356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): picked mon.noname-c con 0x7f2b3c0a2cf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 0x7f2b3c135c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): picked mon.noname-b con 0x7f2b3c0a37e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:48.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c140330 con 0x7f2b3c06d090 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1401b0 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c1404b0 con 0x7f2b3c0a2cf0 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient(hunting): authenticate will time out at 1287.348802s 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 0x7f2b3c135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3affd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c06d090 msgr2=0x7f2b3c135180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c06d090 0x7f2b3c135180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 0x7f2b3c135c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36756/0 (socket says 172.21.15.119:36756) 2024-09-15T21:22:48.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 -- 172.21.15.119:0/2160387972 learned_addr learned my addr 172.21.15.119:0/2160387972 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): get_auth_request con 0x7f2b3c0a37e0 auth_method 0 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3a7fc640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b3c0a2cf0 0x7f2b3c1356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient(hunting): handle_auth_done global_id 64201 payload 824 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 -- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b3c0a2cf0 msgr2=0x7f2b3c1356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b3c0a2cf0 0x7f2b3c1356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 -- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c06d090 msgr2=0x7f2b3c135180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:48.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c06d090 0x7f2b3c135180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 -- 172.21.15.119:0/2160387972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c0a6260 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3a7fc640 10 monclient: get_auth_request con 0x7f2b3c0a2cf0 auth_method 0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3a7fc640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b3c0a2cf0 0x7f2b3c1356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3b7fe640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 0x7f2b3c135c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400b760 tx=0x7f2b3400bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400c960 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: dump: 2024-09-15T21:22:48.569 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:48.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b34010070 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 5 monclient: authenticate success, global_id 64201 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _renew_subs 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c120460 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400cdc0 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 10 monclient: dump: 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:48.571 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _renew_subs 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c120180 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b397fa640 4 set_mon_vals no callback set 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:48.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c0a30d0 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2b34015470 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b2803b9a0 0x7f2b2803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 8639+0+0 (secure 0 0 0) 0x7f2b340509d0 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3affd640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b2803b9a0 0x7f2b2803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3affd640 10 monclient: get_auth_request con 0x7f2b2803b9a0 auth_method 0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.551+0000 7f2b3affd640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b2803b9a0 0x7f2b2803de60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c001390 tx=0x7f2b2c001080 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.555+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2b3c0a6260 con 0x7f2b3c0a37e0 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.555+0000 7f2b237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:48.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.555+0000 7f2b237fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:48.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.731+0000 7f2b41340640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T21:22:48.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.731+0000 7f2b41340640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T21:22:48.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.731+0000 7f2b41340640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:48.738 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:48.731+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) -- 0x7f2b3c06a880 con 0x7f2b3c0a37e0 2024-09-15T21:22:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.371+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 v144) ==== 113+0+0 (secure 0 0 0) 0x7f2b3404e4d0 con 0x7f2b3c0a37e0 2024-09-15T21:22:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.371+0000 7f2b237fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T21:22:49.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.371+0000 7f2b237fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:49.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.483+0000 7f2b41340640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T21:22:49.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.483+0000 7f2b41340640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T21:22:49.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.483+0000 7f2b41340640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:49.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.483+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) -- 0x7f2b3c13c000 con 0x7f2b3c0a37e0 2024-09-15T21:22:49.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b237fe640 1 -- 172.21.15.119:0/2160387972 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 weight-set for pool 'cold' already created v144) ==== 155+0+0 (secure 0 0 0) 0x7f2b34014a30 con 0x7f2b3c0a37e0 2024-09-15T21:22:49.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b237fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T21:22:49.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b237fe640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cold' already created 2024-09-15T21:22:49.490 INFO:tasks.workunit.client.0.smithi119.stderr:weight-set for pool 'cold' already created 2024-09-15T21:22:49.492 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b2803b9a0 msgr2=0x7f2b2803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b2803b9a0 0x7f2b2803de60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c001390 tx=0x7f2b2c001080 comp rx=0 tx=0).stop 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 10 monclient: shutdown 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 msgr2=0x7f2b3c135c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 0x7f2b3c135c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400b760 tx=0x7f2b3400bc20 comp rx=0 tx=0).stop 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 shutdown_connections 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b2803b9a0 0x7f2b2803de60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b3c0a37e0 0x7f2b3c135c00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b3c0a2cf0 0x7f2b3c1356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 --2- 172.21.15.119:0/2160387972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c06d090 0x7f2b3c135180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 >> 172.21.15.119:0/2160387972 conn(0x7f2b3c0a3de0 msgr2=0x7f2b3c0734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:49.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 shutdown_connections 2024-09-15T21:22:49.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.487+0000 7f2b41340640 1 -- 172.21.15.119:0/2160387972 wait complete. 2024-09-15T21:22:49.508 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm-device-class osd.0 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.619+0000 7fc2f0d10640 10 monclient: get_monmap_and_config 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.619+0000 7fc2f0d10640 10 monclient: build_initial_monmap 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.619+0000 7fc2f0d10640 10 monclient: monmap: 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:49.623906+0000 2024-09-15T21:22:49.625 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:49.623906+0000 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:49.626 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:49.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 Processor -- start 2024-09-15T21:22:49.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 -- start start 2024-09-15T21:22:49.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient: init 2024-09-15T21:22:49.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc2ec0a4e60 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc2ec0a5950 0x7fc2ec13cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): picked mon.noname-a con 0x7fc2ec0a5950 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 0x7fc2ec13f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): picked mon.noname-b con 0x7fc2ec13d480 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ec0a1150 con 0x7fc2ec0a4e60 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ec0a3c80 con 0x7fc2ec0a5950 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ec0a0ef0 con 0x7fc2ec13d480 2024-09-15T21:22:49.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2f0d10640 10 monclient(hunting): authenticate will time out at 1288.421082s 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ea59c640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fc2ec0a4e60 legacy=0x7fc2ec0a5240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2e1d9b640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fc2ec0a5950 msgr2=0x7fc2ec13cf40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2e1d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc2ec0a5950 0x7fc2ec13cf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 0x7fc2ec13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 0x7fc2ec13f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:34766/0 (socket says 172.21.15.119:34766) 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 -- 172.21.15.119:0/8622432 learned_addr learned my addr 172.21.15.119:0/8622432 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): get_auth_request con 0x7fc2ec13d480 auth_method 0 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:49.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_done global_id 44648 payload 274 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 -- 172.21.15.119:0/8622432 >> v2:172.21.15.17:3300/0 conn(0x7fc2ec0a5950 msgr2=0x7fc2ec13cf40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 --2- 172.21.15.119:0/8622432 >> v2:172.21.15.17:3300/0 conn(0x7fc2ec0a5950 0x7fc2ec13cf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 -- 172.21.15.119:0/8622432 >> v1:172.21.15.17:6789/0 conn(0x7fc2ec0a4e60 legacy=0x7fc2ec0a5240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient: _finish_hunting 0 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 monclient: found mon.noname-b 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:22:49.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.623+0000 7fc2ead9d640 1 -- 172.21.15.119:0/8622432 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec0a6cf0 con 0x7fc2ec13d480 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 --2- 172.21.15.119:0/8622432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 0x7fc2ec13f870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0a1c70 tx=0x7fc2e401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f1875f1c19843a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 1 -- 172.21.15.119:0/8622432 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40a5530 con 0x7fc2ec13d480 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: dump: 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:49.636 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: _finish_auth 0 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 1 -- 172.21.15.119:0/8622432 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e40a56d0 con 0x7fc2ec13d480 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 1 -- 172.21.15.119:0/8622432 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40a5aa0 con 0x7fc2ec13d480 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 5 monclient: authenticate success, global_id 44648 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:49.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 4 set_mon_vals no callback set 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e9d9b640 10 monclient: dump: 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:49.638 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: shutdown 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/8622432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 msgr2=0x7fc2ec13f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/8622432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 0x7fc2ec13f870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0a1c70 tx=0x7fc2e401c8f0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/8622432 shutdown_connections 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/8622432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec13d480 0x7fc2ec13f870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/8622432 >> v2:172.21.15.17:3300/0 conn(0x7fc2ec0a5950 0x7fc2ec13cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/8622432 >> 172.21.15.119:0/8622432 conn(0x7fc2ec0a6f60 msgr2=0x7fc2ec0a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/8622432 shutdown_connections 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/8622432 wait complete. 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: build_initial_monmap 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: monmap: 2024-09-15T21:22:49.639 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 Processor -- start 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- start start 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: init 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:49.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec0a4e60 0x7fc2ec135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): picked mon.noname-c con 0x7fc2ec0a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2ec0a5950 0x7fc2ec135680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): picked mon.noname-a con 0x7fc2ec0a5950 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 0x7fc2ec1cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): picked mon.noname-b con 0x7fc2ec13d480 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec0a2310 con 0x7fc2ec0a5950 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec0a2190 con 0x7fc2ec13d480 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ec1425d0 con 0x7fc2ec0a4e60 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:49.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient(hunting): authenticate will time out at 1288.427026s 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e1d9b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2ec0a5950 msgr2=0x7fc2ec135680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 0x7fc2ec1cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2ec0a5950 0x7fc2ec135680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 0x7fc2ec1cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36764/0 (socket says 172.21.15.119:36764) 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 -- 172.21.15.119:0/2095585062 learned_addr learned my addr 172.21.15.119:0/2095585062 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): get_auth_request con 0x7fc2ec13d480 auth_method 0 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:49.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ea59c640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec0a4e60 0x7fc2ec135140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ea59c640 10 monclient(hunting): get_auth_request con 0x7fc2ec0a4e60 auth_method 0 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient(hunting): handle_auth_done global_id 64204 payload 824 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 -- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec0a4e60 msgr2=0x7fc2ec135140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec0a4e60 0x7fc2ec135140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 -- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2ec0a5950 msgr2=0x7fc2ec135680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2ec0a5950 0x7fc2ec135680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient: _finish_hunting 0 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 monclient: found mon.noname-b 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:49.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 -- 172.21.15.119:0/2095585062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec070ce0 con 0x7fc2ec13d480 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2ead9d640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 0x7fc2ec1cf6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2e401cdd0 tx=0x7fc2e4019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40bc020 con 0x7fc2ec13d480 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: dump: 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:49.644 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e40b8470 con 0x7fc2ec13d480 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 5 monclient: authenticate success, global_id 64204 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: _renew_subs 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec0a3100 con 0x7fc2ec13d480 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:49.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40b8860 con 0x7fc2ec13d480 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e37fe640 10 monclient: dump: 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:49.646 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2e959a640 4 set_mon_vals no callback set 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: _renew_subs 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.627+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec0727a0 con 0x7fc2ec13d480 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2f0d10640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2f0d10640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2f0d10640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc2e40bf020 con 0x7fc2ec13d480 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec077180 con 0x7fc2ec13d480 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2e37fe640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc2cc03b970 0x7fc2cc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9111+0+0 (secure 0 0 0) 0x7fc2e40ebf00 con 0x7fc2ec13d480 2024-09-15T21:22:49.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2ea59c640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc2cc03b970 0x7fc2cc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:49.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2ea59c640 10 monclient: get_auth_request con 0x7fc2cc03b970 auth_method 0 2024-09-15T21:22:49.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.631+0000 7fc2ea59c640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc2cc03b970 0x7fc2cc03de30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc006fd0 tx=0x7fc2dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:49.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.635+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc2e4016600 con 0x7fc2ec13d480 2024-09-15T21:22:49.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.635+0000 7fc2e37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:49.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.635+0000 7fc2e37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:49.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.811+0000 7fc2f0d10640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:22:49.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.811+0000 7fc2f0d10640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:22:49.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.811+0000 7fc2f0d10640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:49.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:49.811+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7fc2ec071840 con 0x7fc2ec13d480 2024-09-15T21:22:50.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.379+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v145) ==== 123+0+0 (secure 0 0 0) 0x7fc2e40b6cc0 con 0x7fc2ec13d480 2024-09-15T21:22:50.384 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.379+0000 7fc2e37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:22:50.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.379+0000 7fc2e37fe640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-09-15T21:22:50.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.491+0000 7fc2f0d10640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:22:50.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.491+0000 7fc2f0d10640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:22:50.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.491+0000 7fc2f0d10640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:50.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:50.491+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7fc2ec06c390 con 0x7fc2ec13d480 2024-09-15T21:22:51.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.383+0000 7fc2e37fe640 1 -- 172.21.15.119:0/2095585062 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v146) ==== 149+0+0 (secure 0 0 0) 0x7fc2e40c1410 con 0x7fc2ec13d480 2024-09-15T21:22:51.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.383+0000 7fc2e37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:22:51.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.383+0000 7fc2e37fe640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:22:51.388 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc2cc03b970 msgr2=0x7fc2cc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc2cc03b970 0x7fc2cc03de30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc006fd0 tx=0x7fc2dc008040 comp rx=0 tx=0).stop 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 10 monclient: shutdown 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 msgr2=0x7fc2ec1cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 0x7fc2ec1cf6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2e401cdd0 tx=0x7fc2e4019690 comp rx=0 tx=0).stop 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 shutdown_connections 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc2cc03b970 0x7fc2cc03de30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2ec13d480 0x7fc2ec1cf6e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2ec0a5950 0x7fc2ec135680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 --2- 172.21.15.119:0/2095585062 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2ec0a4e60 0x7fc2ec135140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 >> 172.21.15.119:0/2095585062 conn(0x7fc2ec0a6f60 msgr2=0x7fc2ec137e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:51.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 shutdown_connections 2024-09-15T21:22:51.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.387+0000 7fc2f0d10640 1 -- 172.21.15.119:0/2095585062 wait complete. 2024-09-15T21:22:51.413 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight-compat osd.0 10.5 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient: get_monmap_and_config 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient: build_initial_monmap 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient: monmap: 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:51.534541+0000 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:51.534541+0000 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:51.536 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:51.537 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:51.537 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:51.537 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:51.537 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:51.537 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:51.537 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:51.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 1 Processor -- start 2024-09-15T21:22:51.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 1 -- start start 2024-09-15T21:22:51.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient: init 2024-09-15T21:22:51.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d60137bb0 0x7f3d60137f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient(hunting): picked mon.noname-c con 0x7f3d60137bb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3d601384d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3d600a5500 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d600a3e60 con 0x7f3d600a5500 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d600a1c70 con 0x7f3d601384d0 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3d600a6c40 con 0x7f3d60137bb0 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d664a9640 10 monclient(hunting): authenticate will time out at 1290.331131s 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d60137bb0 0x7f3d60137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:51.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d60137bb0 0x7f3d60137f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36770/0 (socket says 172.21.15.119:36770) 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5f7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3d601384d0 0x7f3d600a4df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40310/0 (socket says 172.21.15.119:40310) 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 -- 172.21.15.119:0/1075351536 learned_addr learned my addr 172.21.15.119:0/1075351536 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): get_auth_request con 0x7f3d60137bb0 auth_method 0 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d64a46640 1 -- 172.21.15.119:0/1075351536 >> v1:172.21.15.17:6789/0 conn(0x7f3d600a5500 legacy=0x7f3d600a5970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:51.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient(hunting): handle_auth_done global_id 64207 payload 274 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 -- 172.21.15.119:0/1075351536 >> v1:172.21.15.119:6789/0 conn(0x7f3d601384d0 legacy=0x7f3d600a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 -- 172.21.15.119:0/1075351536 >> v1:172.21.15.17:6789/0 conn(0x7f3d600a5500 legacy=0x7f3d600a5970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 monclient: found mon.noname-c 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.531+0000 7f3d5ffff640 1 -- 172.21.15.119:0/1075351536 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d600a1990 con 0x7f3d60137bb0 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5ffff640 1 --2- 172.21.15.119:0/1075351536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d60137bb0 0x7f3d60137f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3d600a6dc0 tx=0x7f3d5001c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e4dedfde387697 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 1 -- 172.21.15.119:0/1075351536 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500a56a0 con 0x7f3d60137bb0 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:51.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: dump: 2024-09-15T21:22:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: _finish_auth 0 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 1 -- 172.21.15.119:0/1075351536 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d500a5840 con 0x7f3d60137bb0 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 5 monclient: authenticate success, global_id 64207 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:51.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 1 -- 172.21.15.119:0/1075351536 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500a5c10 con 0x7f3d60137bb0 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 4 set_mon_vals no callback set 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d5effd640 10 monclient: dump: 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:51.546 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: shutdown 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 -- 172.21.15.119:0/1075351536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d60137bb0 msgr2=0x7f3d60137f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1075351536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d60137bb0 0x7f3d60137f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3d600a6dc0 tx=0x7f3d5001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 -- 172.21.15.119:0/1075351536 shutdown_connections 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1075351536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d60137bb0 0x7f3d60137f90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 -- 172.21.15.119:0/1075351536 >> 172.21.15.119:0/1075351536 conn(0x7f3d600a6ef0 msgr2=0x7f3d600a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 -- 172.21.15.119:0/1075351536 shutdown_connections 2024-09-15T21:22:51.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 -- 172.21.15.119:0/1075351536 wait complete. 2024-09-15T21:22:51.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: build_initial_monmap 2024-09-15T21:22:51.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: monmap: 2024-09-15T21:22:51.551 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:51.551 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 Processor -- start 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 -- start start 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: init 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:51.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d600a5500 0x7f3d6013d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 10 monclient(hunting): picked mon.noname-a con 0x7f3d600a5500 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.535+0000 7f3d664a9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d60137bb0 0x7f3d6013f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): picked mon.noname-c con 0x7f3d60137bb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 0x7f3d60136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): picked mon.noname-b con 0x7f3d601384d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600a62c0 con 0x7f3d600a5500 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600a1170 con 0x7f3d601384d0 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3d600a6440 con 0x7f3d60137bb0 2024-09-15T21:22:51.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient(hunting): authenticate will time out at 1290.336054s 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5ffff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d600a5500 msgr2=0x7f3d6013d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 0x7f3d60136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d600a5500 0x7f3d6013d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 0x7f3d60136d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36782/0 (socket says 172.21.15.119:36782) 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 -- 172.21.15.119:0/1178722858 learned_addr learned my addr 172.21.15.119:0/1178722858 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): get_auth_request con 0x7f3d601384d0 auth_method 0 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:51.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5f7fe640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d60137bb0 0x7f3d6013f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient(hunting): handle_auth_done global_id 64213 payload 824 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 -- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d60137bb0 msgr2=0x7f3d6013f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d60137bb0 0x7f3d6013f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 -- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d600a5500 msgr2=0x7f3d6013d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d600a5500 0x7f3d6013d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient: _finish_hunting 0 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 monclient: found mon.noname-b 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:51.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 -- 172.21.15.119:0/1178722858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6013fec0 con 0x7f3d601384d0 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d64a46640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 0x7f3d60136d50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800b430 tx=0x7f3d5800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58016070 con 0x7f3d601384d0 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: dump: 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:51.556 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: _finish_auth 0 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58003da0 con 0x7f3d601384d0 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580052b0 con 0x7f3d601384d0 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 5 monclient: authenticate success, global_id 64213 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:51.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 10 monclient: dump: 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5e7fc640 4 set_mon_vals no callback set 2024-09-15T21:22:51.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient: _renew_subs 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6013e9d0 con 0x7f3d601384d0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient: _renew_subs 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d60140d30 con 0x7f3d601384d0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3d58019070 con 0x7f3d601384d0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d60137f90 con 0x7f3d601384d0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d4003b970 0x7f3d4003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9079+0+0 (secure 0 0 0) 0x7f3d5804fef0 con 0x7f3d601384d0 2024-09-15T21:22:51.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5ffff640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d4003b970 0x7f3d4003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5ffff640 10 monclient: get_auth_request con 0x7f3d4003b970 auth_method 0 2024-09-15T21:22:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.539+0000 7f3d5ffff640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d4003b970 0x7f3d4003de30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3d500ae000 tx=0x7f3d500a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.543+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3d5800f300 con 0x7f3d601384d0 2024-09-15T21:22:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.543+0000 7f3d5cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:51.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.543+0000 7f3d5cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.719+0000 7f3d664a9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T21:22:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.719+0000 7f3d664a9640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T21:22:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.719+0000 7f3d664a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:51.719+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) -- 0x7f3d6006a880 con 0x7f3d601384d0 2024-09-15T21:22:52.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.399+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v147) ==== 119+0+0 (secure 0 0 0) 0x7f3d5800fe20 con 0x7f3d601384d0 2024-09-15T21:22:52.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.399+0000 7f3d5cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T21:22:52.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.399+0000 7f3d5cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:52.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.519+0000 7f3d664a9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T21:22:52.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.519+0000 7f3d664a9640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T21:22:52.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.519+0000 7f3d664a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:52.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:52.519+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) -- 0x7f3d60137290 con 0x7f3d601384d0 2024-09-15T21:22:53.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.363+0000 7f3d5cff9640 1 -- 172.21.15.119:0/1178722858 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v148) ==== 119+0+0 (secure 0 0 0) 0x7f3d5804d830 con 0x7f3d601384d0 2024-09-15T21:22:53.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.363+0000 7f3d5cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T21:22:53.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.363+0000 7f3d5cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:53.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d4003b970 msgr2=0x7f3d4003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:53.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d4003b970 0x7f3d4003de30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3d500ae000 tx=0x7f3d500a7070 comp rx=0 tx=0).stop 2024-09-15T21:22:53.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 10 monclient: shutdown 2024-09-15T21:22:53.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:53.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 msgr2=0x7f3d60136d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:53.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 0x7f3d60136d50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800b430 tx=0x7f3d5800b8f0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 shutdown_connections 2024-09-15T21:22:53.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d4003b970 0x7f3d4003de30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d601384d0 0x7f3d60136d50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d60137bb0 0x7f3d6013f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 --2- 172.21.15.119:0/1178722858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d600a5500 0x7f3d6013d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 >> 172.21.15.119:0/1178722858 conn(0x7f3d600a6ef0 msgr2=0x7f3d60071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:53.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 shutdown_connections 2024-09-15T21:22:53.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.367+0000 7f3d664a9640 1 -- 172.21.15.119:0/1178722858 wait complete. 2024-09-15T21:22:53.395 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight cool osd.0 11.5 2024-09-15T21:22:53.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.511+0000 7f4c534a9640 10 monclient: get_monmap_and_config 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.511+0000 7f4c534a9640 10 monclient: build_initial_monmap 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.511+0000 7f4c534a9640 10 monclient: monmap: 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:53.515422+0000 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:53.515422+0000 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:53.517 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.511+0000 7f4c534a9640 1 Processor -- start 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.511+0000 7f4c534a9640 1 -- start start 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.511+0000 7f4c534a9640 10 monclient: init 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4c4c135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c4c09fe70 0x7f4c4c13aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): picked mon.noname-a con 0x7f4c4c09fe70 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c4c13afb0 0x7f4c4c13d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:53.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): picked mon.noname-c con 0x7f4c4c13afb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4c0a0300 con 0x7f4c4c135bf0 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4c4c0a3600 con 0x7f4c4c09fe70 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4c4c0a1d80 con 0x7f4c4c13afb0 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient(hunting): authenticate will time out at 1292.312408s 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51245640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f4c4c135bf0 legacy=0x7f4c4c135fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c50a44640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f4c4c09fe70 msgr2=0x7f4c4c13aa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c50a44640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c4c09fe70 0x7f4c4c13aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c4c13afb0 0x7f4c4c13d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c4c13afb0 0x7f4c4c13d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49168/0 (socket says 172.21.15.119:49168) 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 -- 172.21.15.119:0/1928155879 learned_addr learned my addr 172.21.15.119:0/1928155879 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:53.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): get_auth_request con 0x7f4c4c13afb0 auth_method 0 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient(hunting): handle_auth_done global_id 64216 payload 274 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 -- 172.21.15.119:0/1928155879 >> v2:172.21.15.17:3300/0 conn(0x7f4c4c09fe70 msgr2=0x7f4c4c13aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 --2- 172.21.15.119:0/1928155879 >> v2:172.21.15.17:3300/0 conn(0x7f4c4c09fe70 0x7f4c4c13aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 -- 172.21.15.119:0/1928155879 >> v1:172.21.15.17:6789/0 conn(0x7f4c4c135bf0 legacy=0x7f4c4c135fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient: _finish_hunting 0 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 monclient: found mon.noname-c 2024-09-15T21:22:53.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:53.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 -- 172.21.15.119:0/1928155879 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c0a2060 con 0x7f4c4c13afb0 2024-09-15T21:22:53.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c51a46640 1 --2- 172.21.15.119:0/1928155879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c13d3a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009e50 tx=0x7f4c4401c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82715793b4d324d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 1 -- 172.21.15.119:0/1928155879 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440a5560 con 0x7f4c4c13afb0 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: dump: 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:53.529 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: _finish_auth 0 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 5 monclient: authenticate success, global_id 64216 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 1 -- 172.21.15.119:0/1928155879 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c440a5700 con 0x7f4c4c13afb0 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 4 set_mon_vals no callback set 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 1 -- 172.21.15.119:0/1928155879 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440a5ad0 con 0x7f4c4c13afb0 2024-09-15T21:22:53.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c43fff640 10 monclient: dump: 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:53.531 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 10 monclient: shutdown 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- 172.21.15.119:0/1928155879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 msgr2=0x7f4c4c13d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 --2- 172.21.15.119:0/1928155879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c13d3a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009e50 tx=0x7f4c4401c6c0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- 172.21.15.119:0/1928155879 shutdown_connections 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 --2- 172.21.15.119:0/1928155879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c13d3a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 --2- 172.21.15.119:0/1928155879 >> v2:172.21.15.17:3300/0 conn(0x7f4c4c09fe70 0x7f4c4c13aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- 172.21.15.119:0/1928155879 >> 172.21.15.119:0/1928155879 conn(0x7f4c4c0a66e0 msgr2=0x7f4c4c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:53.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.515+0000 7f4c534a9640 1 -- 172.21.15.119:0/1928155879 shutdown_connections 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- 172.21.15.119:0/1928155879 wait complete. 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: build_initial_monmap 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: monmap: 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:53.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 Processor -- start 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- start start 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: init 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c4c09fe70 0x7f4c4c11da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): picked mon.noname-c con 0x7f4c4c09fe70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c4c135bf0 0x7f4c4c1348f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): picked mon.noname-a con 0x7f4c4c135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c1cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): picked mon.noname-b con 0x7f4c4c13afb0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c140100 con 0x7f4c4c135bf0 2024-09-15T21:22:53.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c06ecc0 con 0x7f4c4c13afb0 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4c140280 con 0x7f4c4c09fe70 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient(hunting): authenticate will time out at 1292.317553s 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c1cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c50a44640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c4c135bf0 msgr2=0x7f4c4c1348f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c1cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49172/0 (socket says 172.21.15.119:49172) 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 -- 172.21.15.119:0/3469405855 learned_addr learned my addr 172.21.15.119:0/3469405855 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): get_auth_request con 0x7f4c4c13afb0 auth_method 0 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:53.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c50a44640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c4c135bf0 0x7f4c4c1348f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51245640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c4c09fe70 0x7f4c4c11da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51245640 10 monclient(hunting): get_auth_request con 0x7f4c4c09fe70 auth_method 0 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51245640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51245640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51245640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient(hunting): handle_auth_done global_id 64219 payload 824 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 -- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c4c09fe70 msgr2=0x7f4c4c11da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c4c09fe70 0x7f4c4c11da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 -- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c4c135bf0 msgr2=0x7f4c4c1348f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c4c135bf0 0x7f4c4c1348f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient: _finish_hunting 0 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 monclient: found mon.noname-b 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 -- 172.21.15.119:0/3469405855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c13abf0 con 0x7f4c4c13afb0 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c51a46640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c1cd3b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c440af040 tx=0x7f4c44019460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440bc020 con 0x7f4c4c13afb0 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: dump: 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:53.537 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: _finish_auth 0 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c440b8670 con 0x7f4c4c13afb0 2024-09-15T21:22:53.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 5 monclient: authenticate success, global_id 64219 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _renew_subs 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c11d620 con 0x7f4c4c13afb0 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440b8a20 con 0x7f4c4c13afb0 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 10 monclient: dump: 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:53.539 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c437fe640 4 set_mon_vals no callback set 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _renew_subs 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c11e9e0 con 0x7f4c4c13afb0 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:53.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c135fd0 con 0x7f4c4c13afb0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4c440bf020 con 0x7f4c4c13afb0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c3003b970 0x7f4c3003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.519+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9236+0+0 (secure 0 0 0) 0x7f4c440eba80 con 0x7f4c4c13afb0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.523+0000 7f4c51245640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c3003b970 0x7f4c3003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.523+0000 7f4c51245640 10 monclient: get_auth_request con 0x7f4c3003b970 auth_method 0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.523+0000 7f4c51245640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c3003b970 0x7f4c3003de30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006fd0 tx=0x7f4c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.527+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4c440ede20 con 0x7f4c4c13afb0 2024-09-15T21:22:53.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.527+0000 7f4c41ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:53.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.527+0000 7f4c41ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:53.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.703+0000 7f4c534a9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T21:22:53.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.703+0000 7f4c534a9640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T21:22:53.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.703+0000 7f4c534a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:53.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:53.703+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) -- 0x7f4c4c06a830 con 0x7f4c4c13afb0 2024-09-15T21:22:54.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.367+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v149) ==== 128+0+0 (secure 0 0 0) 0x7f4c440c6070 con 0x7f4c4c13afb0 2024-09-15T21:22:54.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.367+0000 7f4c41ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T21:22:54.372 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.367+0000 7f4c41ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:54.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.483+0000 7f4c534a9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T21:22:54.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.483+0000 7f4c534a9640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T21:22:54.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.483+0000 7f4c534a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:54.489 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:54.483+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) -- 0x7f4c4c135590 con 0x7f4c4c13afb0 2024-09-15T21:22:55.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.371+0000 7f4c41ffb640 1 -- 172.21.15.119:0/3469405855 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v150) ==== 128+0+0 (secure 0 0 0) 0x7f4c4c0a0300 con 0x7f4c4c13afb0 2024-09-15T21:22:55.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.371+0000 7f4c41ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T21:22:55.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.371+0000 7f4c41ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:55.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c3003b970 msgr2=0x7f4c3003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:55.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c3003b970 0x7f4c3003de30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006fd0 tx=0x7f4c48008040 comp rx=0 tx=0).stop 2024-09-15T21:22:55.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 10 monclient: shutdown 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 msgr2=0x7f4c4c1cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c1cd3b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c440af040 tx=0x7f4c44019460 comp rx=0 tx=0).stop 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 shutdown_connections 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4c3003b970 0x7f4c3003de30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c4c13afb0 0x7f4c4c1cd3b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c4c135bf0 0x7f4c4c1348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 --2- 172.21.15.119:0/3469405855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c4c09fe70 0x7f4c4c11da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 >> 172.21.15.119:0/3469405855 conn(0x7f4c4c0a66e0 msgr2=0x7f4c4c0a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 shutdown_connections 2024-09-15T21:22:55.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.375+0000 7f4c534a9640 1 -- 172.21.15.119:0/3469405855 wait complete. 2024-09-15T21:22:55.395 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight cold osd.0 12.5 12.4 2024-09-15T21:22:55.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.507+0000 7ff4c1ff7640 10 monclient: get_monmap_and_config 2024-09-15T21:22:55.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.507+0000 7ff4c1ff7640 10 monclient: build_initial_monmap 2024-09-15T21:22:55.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.507+0000 7ff4c1ff7640 10 monclient: monmap: 2024-09-15T21:22:55.513 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:55.512194+0000 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:55.512194+0000 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:55.514 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 1 Processor -- start 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 1 -- start start 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient: init 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4bc0a3c90 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4bc09fe90 addr v1:172.21.15.17:6789/0 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 0x7ff4bc13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient(hunting): picked mon.noname-b con 0x7ff4bc0a02e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:55.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc0a2730 con 0x7ff4bc09fe90 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc06dc80 con 0x7ff4bc0a3c90 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc0a4700 con 0x7ff4bc0a02e0 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 10 monclient(hunting): authenticate will time out at 1294.308765s 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff4bc0a3c90 0x7ff4bc09f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51146/0 (socket says 172.21.15.119:51146) 2024-09-15T21:22:55.526 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 -- 172.21.15.119:0/3215356079 learned_addr learned my addr 172.21.15.119:0/3215356079 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4baffd640 1 -- 172.21.15.119:0/3215356079 >> v1:172.21.15.17:6789/0 conn(0x7ff4bc09fe90 legacy=0x7ff4bc13ae90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 0x7ff4bc13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179254975 0 0) 0x7ff4bc06dc80 con 0x7ff4bc0a3c90 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): my global_id is 64222 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4a4003530 con 0x7ff4bc0a3c90 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): get_auth_request con 0x7ff4bc0a02e0 auth_method 0 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:55.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2353392152 0 0) 0x7ff4a4003530 con 0x7ff4bc0a3c90 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 msgr2=0x7ff4bc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 0x7ff4bc13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 >> v1:172.21.15.17:6789/0 conn(0x7ff4bc09fe90 legacy=0x7ff4bc13ae90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc0a13f0 con 0x7ff4bc0a3c90 2024-09-15T21:22:55.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: _finish_auth 0 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff4ac001700 con 0x7ff4bc0a3c90 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 5 monclient: authenticate success, global_id 64222 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: dump: 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.530 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 >> v1:172.21.15.119:6789/0 conn(0x7ff4bc0a3c90 legacy=0x7ff4bc09f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4a40039f0 0x7ff4a4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): picked mon.b con 0x7ff4a40039f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a4004810 0x7ff4a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): picked mon.a con 0x7ff4a4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:55.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 0x7ff4a4005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): picked mon.c con 0x7ff4bc0a02e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a400bb60 con 0x7ff4a4004810 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a400bd10 con 0x7ff4a40039f0 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a400bf10 con 0x7ff4bc0a02e0 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4ba7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4c1ff7640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4a40039f0 0x7ff4a4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): get_auth_request con 0x7ff4a40039f0 auth_method 0 2024-09-15T21:22:55.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4baffd640 1 -- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a4004810 msgr2=0x7ff4a4004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4baffd640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a4004810 0x7ff4a4004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_done global_id 64222 payload 293 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bbfff640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 0x7ff4a4005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 -- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 msgr2=0x7ff4a4005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc0a02e0 0x7ff4a4005280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 -- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a4004810 msgr2=0x7ff4a4004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:55.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a4004810 0x7ff4a4004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 monclient: found mon.b 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 1 -- 172.21.15.119:0/3215356079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a4003530 con 0x7ff4a40039f0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.511+0000 7ff4bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bb7fe640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4a40039f0 0x7ff4a4003df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c320 tx=0x7ff4ac01cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f3be8106f536e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac007da0 con 0x7ff4a40039f0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: dump: 2024-09-15T21:22:55.534 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:55.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ac0a9020 con 0x7ff4a40039f0 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 1 -- 172.21.15.119:0/3215356079 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac005b20 con 0x7ff4a40039f0 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 4 set_mon_vals no callback set 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4ba7fc640 10 monclient: dump: 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:55.536 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: shutdown 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/3215356079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4a40039f0 msgr2=0x7ff4a4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/3215356079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4a40039f0 0x7ff4a4003df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c320 tx=0x7ff4ac01cbc0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 -- 172.21.15.119:0/3215356079 reap_dead start 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/3215356079 shutdown_connections 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/3215356079 >> 172.21.15.119:0/3215356079 conn(0x7ff4bc06cbf0 msgr2=0x7ff4bc06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/3215356079 shutdown_connections 2024-09-15T21:22:55.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/3215356079 wait complete. 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: build_initial_monmap 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: monmap: 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 Processor -- start 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- start start 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: init 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a40039f0 0x7ff4bc1358f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): picked mon.noname-a con 0x7ff4a40039f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc11fd40 0x7ff4bc135e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): picked mon.noname-c con 0x7ff4bc11fd40 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 0x7ff4bc11ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): picked mon.noname-b con 0x7ff4bc11e8d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc13edd0 con 0x7ff4a40039f0 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc074410 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc13ef50 con 0x7ff4bc11fd40 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient(hunting): authenticate will time out at 1294.314662s 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bb7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a40039f0 msgr2=0x7ff4bc1358f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a40039f0 0x7ff4bc1358f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 0x7ff4bc11ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 0x7ff4bc11ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49176/0 (socket says 172.21.15.119:49176) 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 -- 172.21.15.119:0/4246197614 learned_addr learned my addr 172.21.15.119:0/4246197614 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): get_auth_request con 0x7ff4bc11e8d0 auth_method 0 2024-09-15T21:22:55.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4baffd640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc11fd40 0x7ff4bc135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4baffd640 10 monclient(hunting): get_auth_request con 0x7ff4bc11fd40 auth_method 0 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4baffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient(hunting): handle_auth_done global_id 64225 payload 824 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 -- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc11fd40 msgr2=0x7ff4bc135e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:55.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc11fd40 0x7ff4bc135e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 -- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a40039f0 msgr2=0x7ff4bc1358f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a40039f0 0x7ff4bc1358f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 monclient: found mon.noname-b 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 -- 172.21.15.119:0/4246197614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc121540 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4bbfff640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 0x7ff4bc11ed80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc074fc0 tx=0x7ff4b400f4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401c070 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:55.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: dump: 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:55.543 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: _finish_auth 0 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4017400 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40177f0 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4a3fff640 10 monclient: dump: 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:55.544 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4b9ffb640 4 set_mon_vals no callback set 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 5 monclient: authenticate success, global_id 64225 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: _renew_subs 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.515+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc09f470 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 10 monclient: _renew_subs 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc136370 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc09f780 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff4b401f070 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4a3fff640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4a803b970 0x7ff4a803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9236+0+0 (secure 0 0 0) 0x7ff4b4050f40 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4bb7fe640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4a803b970 0x7ff4a803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4bb7fe640 10 monclient: get_auth_request con 0x7ff4a803b970 auth_method 0 2024-09-15T21:22:55.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.519+0000 7ff4bb7fe640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4a803b970 0x7ff4a803de30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0aa000 tx=0x7ff4ac001a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:55.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.523+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff4b4012070 con 0x7ff4bc11e8d0 2024-09-15T21:22:55.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.523+0000 7ff4a3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:55.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.523+0000 7ff4a3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:55.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.687+0000 7ff4c1ff7640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T21:22:55.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.687+0000 7ff4c1ff7640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T21:22:55.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.687+0000 7ff4c1ff7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:55.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:55.687+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) -- 0x7ff4bc06a7f0 con 0x7ff4bc11e8d0 2024-09-15T21:22:56.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.427+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v151) ==== 134+0+0 (secure 0 0 0) 0x7ff4b404e7e0 con 0x7ff4bc11e8d0 2024-09-15T21:22:56.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.427+0000 7ff4a3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T21:22:56.433 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.427+0000 7ff4a3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.543+0000 7ff4c1ff7640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T21:22:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.543+0000 7ff4c1ff7640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T21:22:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.543+0000 7ff4c1ff7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:56.543+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) -- 0x7ff4bc13af40 con 0x7ff4bc11e8d0 2024-09-15T21:22:57.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.431+0000 7ff4a3fff640 1 -- 172.21.15.119:0/4246197614 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v152) ==== 134+0+0 (secure 0 0 0) 0x7ff4b4040210 con 0x7ff4bc11e8d0 2024-09-15T21:22:57.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.431+0000 7ff4a3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T21:22:57.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.431+0000 7ff4a3fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:57.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4a803b970 msgr2=0x7ff4a803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:57.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4a803b970 0x7ff4a803de30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0aa000 tx=0x7ff4ac001a00 comp rx=0 tx=0).stop 2024-09-15T21:22:57.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 10 monclient: shutdown 2024-09-15T21:22:57.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:57.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 msgr2=0x7ff4bc11ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:57.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 0x7ff4bc11ed80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc074fc0 tx=0x7ff4b400f4a0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 shutdown_connections 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4a803b970 0x7ff4a803de30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc11e8d0 0x7ff4bc11ed80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc11fd40 0x7ff4bc135e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 --2- 172.21.15.119:0/4246197614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4a40039f0 0x7ff4bc1358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 >> 172.21.15.119:0/4246197614 conn(0x7ff4bc06cbf0 msgr2=0x7ff4bc09d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 shutdown_connections 2024-09-15T21:22:57.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.435+0000 7ff4c1ff7640 1 -- 172.21.15.119:0/4246197614 wait complete. 2024-09-15T21:22:57.466 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush set-device-class fish osd.0 2024-09-15T21:22:57.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.583+0000 7fed50ab8640 10 monclient: get_monmap_and_config 2024-09-15T21:22:57.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.583+0000 7fed50ab8640 10 monclient: build_initial_monmap 2024-09-15T21:22:57.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.583+0000 7fed50ab8640 10 monclient: monmap: 2024-09-15T21:22:57.589 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:57.589 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:57.587939+0000 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:57.587939+0000 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:57.590 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:57.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.583+0000 7fed50ab8640 1 Processor -- start 2024-09-15T21:22:57.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.583+0000 7fed50ab8640 1 -- start start 2024-09-15T21:22:57.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.583+0000 7fed50ab8640 10 monclient: init 2024-09-15T21:22:57.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fed4c137870 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed4c138360 0x7fed4c0a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): picked mon.noname-c con 0x7fed4c138360 addr v2:172.21.15.119:3300/0 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c13cdc0 0x7fed4c13f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): picked mon.noname-b con 0x7fed4c13cdc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed4c0a1b60 con 0x7fed4c137870 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fed4c0a27f0 con 0x7fed4c138360 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fed4c0a1e40 con 0x7fed4c13cdc0 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:57.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient(hunting): authenticate will time out at 1296.384521s 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed4c138360 0x7fed4c0a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed4c138360 0x7fed4c0a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49178/0 (socket says 172.21.15.119:49178) 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 -- 172.21.15.119:0/1253483210 learned_addr learned my addr 172.21.15.119:0/1253483210 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4a59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fed4c137870 0x7fed4c137c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51150/0 (socket says 172.21.15.119:51150) 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): get_auth_request con 0x7fed4c138360 auth_method 0 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:57.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4ad9d640 1 --2- 172.21.15.119:0/1253483210 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c13cdc0 0x7fed4c13f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4ad9d640 10 monclient(hunting): get_auth_request con 0x7fed4c13cdc0 auth_method 0 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient(hunting): handle_auth_done global_id 64228 payload 274 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 -- 172.21.15.119:0/1253483210 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c13cdc0 msgr2=0x7fed4c13f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 --2- 172.21.15.119:0/1253483210 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c13cdc0 0x7fed4c13f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 -- 172.21.15.119:0/1253483210 >> v1:172.21.15.119:6789/0 conn(0x7fed4c137870 legacy=0x7fed4c137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 monclient: found mon.noname-c 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 1 -- 172.21.15.119:0/1253483210 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 227817382 0 0) 0x7fed40001120 con 0x7fed4c137870 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:22:57.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 -- 172.21.15.119:0/1253483210 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c0a2120 con 0x7fed4c138360 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed49d9b640 1 --2- 172.21.15.119:0/1253483210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c138360 0x7fed4c0a0e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009870 tx=0x7fed3c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51fc83886a4e39a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 1 -- 172.21.15.119:0/1253483210 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0b0070 con 0x7fed4c138360 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: dump: 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:57.597 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: _finish_auth 0 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: _check_auth_tickets 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 1 -- 172.21.15.119:0/1253483210 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c0ab600 con 0x7fed4c138360 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 5 monclient: authenticate success, global_id 64228 2024-09-15T21:22:57.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed50ab8640 4 set_mon_vals no callback set 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 1 -- 172.21.15.119:0/1253483210 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0ab9d0 con 0x7fed4c138360 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.587+0000 7fed4959a640 10 monclient: dump: 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:57.599 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:57.600 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:57.600 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:57.600 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:57.600 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:57.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: shutdown 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/1253483210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c138360 msgr2=0x7fed4c0a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 --2- 172.21.15.119:0/1253483210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c138360 0x7fed4c0a0e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009870 tx=0x7fed3c01c420 comp rx=0 tx=0).stop 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/1253483210 shutdown_connections 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 --2- 172.21.15.119:0/1253483210 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c13cdc0 0x7fed4c13f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 --2- 172.21.15.119:0/1253483210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c138360 0x7fed4c0a0e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/1253483210 >> 172.21.15.119:0/1253483210 conn(0x7fed4c09be60 msgr2=0x7fed4c09c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/1253483210 shutdown_connections 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/1253483210 wait complete. 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: build_initial_monmap 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: monmap: 2024-09-15T21:22:57.605 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 Processor -- start 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- start start 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: init 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:57.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed4c137870 0x7fed4c11f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): picked mon.noname-a con 0x7fed4c137870 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c138360 0x7fed4c1cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): picked mon.noname-c con 0x7fed4c138360 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 0x7fed4c11d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): picked mon.noname-b con 0x7fed4c13cdc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1421a0 con 0x7fed4c137870 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c142020 con 0x7fed4c13cdc0 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fed4c142320 con 0x7fed4c138360 2024-09-15T21:22:57.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient(hunting): authenticate will time out at 1296.389893s 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4a59c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed4c137870 msgr2=0x7fed4c11f7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 0x7fed4c11d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed4c137870 0x7fed4c11f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 0x7fed4c11d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49180/0 (socket says 172.21.15.119:49180) 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 -- 172.21.15.119:0/482072645 learned_addr learned my addr 172.21.15.119:0/482072645 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): get_auth_request con 0x7fed4c13cdc0 auth_method 0 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:57.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed49d9b640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c138360 0x7fed4c1cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient(hunting): handle_auth_done global_id 64234 payload 824 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 -- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c138360 msgr2=0x7fed4c1cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c138360 0x7fed4c1cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 -- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed4c137870 msgr2=0x7fed4c11f7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed4c137870 0x7fed4c11f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 -- 172.21.15.119:0/482072645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c120cf0 con 0x7fed4c13cdc0 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed49d9b640 10 monclient: get_auth_request con 0x7fed4c138360 auth_method 0 2024-09-15T21:22:57.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed49d9b640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c138360 0x7fed4c1cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed4ad9d640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 0x7fed4c11d240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed44007e60 tx=0x7fed4400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44014620 con 0x7fed4c13cdc0 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: dump: 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:57.610 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: _finish_auth 0 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed44019070 con 0x7fed4c13cdc0 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 5 monclient: authenticate success, global_id 64234 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _renew_subs 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4c06d6e0 con 0x7fed4c13cdc0 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44014a00 con 0x7fed4c13cdc0 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:57.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3bfff640 4 set_mon_vals no callback set 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 10 monclient: dump: 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _renew_subs 2024-09-15T21:22:57.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4c11ec70 con 0x7fed4c13cdc0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fed44014ba0 con 0x7fed4c13cdc0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4c0a0e10 con 0x7fed4c13cdc0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed2c03b9a0 0x7fed2c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.591+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9236+0+0 (secure 0 0 0) 0x7fed44051600 con 0x7fed4c13cdc0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.595+0000 7fed4a59c640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed2c03b9a0 0x7fed2c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.595+0000 7fed4a59c640 10 monclient: get_auth_request con 0x7fed2c03b9a0 auth_method 0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.595+0000 7fed4a59c640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed2c03b9a0 0x7fed2c03de60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fed40001090 tx=0x7fed40007820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.599+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fed4404eea0 con 0x7fed4c13cdc0 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.599+0000 7fed3affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:57.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.599+0000 7fed3affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:57.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.771+0000 7fed50ab8640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T21:22:57.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.771+0000 7fed50ab8640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T21:22:57.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.771+0000 7fed50ab8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:57.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:57.771+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) -- 0x7fed4c136c60 con 0x7fed4c13cdc0 2024-09-15T21:22:58.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.367+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'fish' v153) ==== 137+0+0 (secure 0 0 0) 0x7fed4401ec10 con 0x7fed4c13cdc0 2024-09-15T21:22:58.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.367+0000 7fed3affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T21:22:58.371 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.367+0000 7fed3affd640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'fish' 2024-09-15T21:22:58.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.483+0000 7fed50ab8640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T21:22:58.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.483+0000 7fed50ab8640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T21:22:58.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.483+0000 7fed50ab8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:58.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:58.483+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) -- 0x7fed4c1c7490 con 0x7fed4c13cdc0 2024-09-15T21:22:59.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.451+0000 7fed3affd640 1 -- 172.21.15.119:0/482072645 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' v154) ==== 241+0+0 (secure 0 0 0) 0x7fed4401c360 con 0x7fed4c13cdc0 2024-09-15T21:22:59.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.451+0000 7fed3affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T21:22:59.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.451+0000 7fed3affd640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-09-15T21:22:59.458 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-09-15T21:22:59.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed2c03b9a0 msgr2=0x7fed2c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed2c03b9a0 0x7fed2c03de60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fed40001090 tx=0x7fed40007820 comp rx=0 tx=0).stop 2024-09-15T21:22:59.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 10 monclient: shutdown 2024-09-15T21:22:59.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:59.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 msgr2=0x7fed4c11d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 0x7fed4c11d240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed44007e60 tx=0x7fed4400e4b0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 shutdown_connections 2024-09-15T21:22:59.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fed2c03b9a0 0x7fed2c03de60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c13cdc0 0x7fed4c11d240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fed4c138360 0x7fed4c1cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 --2- 172.21.15.119:0/482072645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed4c137870 0x7fed4c11f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 >> 172.21.15.119:0/482072645 conn(0x7fed4c09be60 msgr2=0x7fed4c09dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:59.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 shutdown_connections 2024-09-15T21:22:59.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.459+0000 7fed50ab8640 1 -- 172.21.15.119:0/482072645 wait complete. 2024-09-15T21:22:59.481 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:22:59.482 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:22:59.482 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep fish 2024-09-15T21:22:59.482 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '10\.' 2024-09-15T21:22:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.583+0000 7f07ddad9640 10 monclient: get_monmap_and_config 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.583+0000 7f07ddad9640 10 monclient: build_initial_monmap 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.583+0000 7f07ddad9640 10 monclient: monmap: 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:22:59.587450+0000 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:22:59.587450+0000 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:22:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:22:59.590 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.583+0000 7f07ddad9640 1 Processor -- start 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.583+0000 7f07ddad9640 1 -- start start 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.583+0000 7f07ddad9640 10 monclient: init 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f07d80a4ed0 addr v1:172.21.15.119:6789/0 2024-09-15T21:22:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 0x7f07d813ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): picked mon.noname-b con 0x7f07d80a59c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07d80a5dd0 0x7f07d813f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): picked mon.noname-a con 0x7f07d80a5dd0 addr v2:172.21.15.17:3300/0 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d80a6db0 con 0x7f07d80a4ed0 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f07d80a1210 con 0x7f07d80a5dd0 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07d80a1b00 con 0x7f07d80a59c0 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient(hunting): authenticate will time out at 1298.384483s 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d77fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f07d80a4ed0 0x7f07d80a52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51164/0 (socket says 172.21.15.119:51164) 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d77fe640 1 -- 172.21.15.119:0/1981096302 learned_addr learned my addr 172.21.15.119:0/1981096302 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 1 -- 172.21.15.119:0/1981096302 >> v2:172.21.15.17:3300/0 conn(0x7f07d80a5dd0 msgr2=0x7f07d813f420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 1 --2- 172.21.15.119:0/1981096302 >> v2:172.21.15.17:3300/0 conn(0x7f07d80a5dd0 0x7f07d813f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 0x7f07d813ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080916445 0 0) 0x7f07d80a6db0 con 0x7f07d80a4ed0 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): _negotiate 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): my global_id is 64237 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07c0003530 con 0x7f07d80a4ed0 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): get_auth_request con 0x7f07d80a59c0 auth_method 0 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3349629365 0 0) 0x7f07c0003530 con 0x7f07d80a4ed0 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 msgr2=0x7f07d813ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 0x7f07d813ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 >> v2:172.21.15.17:3300/0 conn(0x7f07d80a5dd0 msgr2=0x7f07d813f420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 --2- 172.21.15.119:0/1981096302 >> v2:172.21.15.17:3300/0 conn(0x7f07d80a5dd0 0x7f07d813f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d80a6ad0 con 0x7f07d80a4ed0 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: _finish_auth 0 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:22:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 5 monclient: authenticate success, global_id 64237 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f07d0001a80 con 0x7f07d80a4ed0 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: dump: 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 >> v1:172.21.15.119:6789/0 conn(0x7f07d80a4ed0 legacy=0x7f07d80a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07c00039c0 0x7f07c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): picked mon.a con 0x7f07c00039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0004810 0x7f07c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): picked mon.b con 0x7f07c0004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 0x7f07c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): picked mon.c con 0x7f07d80a59c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f07d80a1b00 con 0x7f07c00039c0 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0005560 con 0x7f07c0004810 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07c000bf10 con 0x7f07d80a59c0 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d77fe640 1 -- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07c00039c0 msgr2=0x7f07c0003de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d77fe640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07c00039c0 0x7f07c0003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0004810 0x7f07c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): get_auth_request con 0x7f07c0004810 auth_method 0 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 0x7f07c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 10 monclient(hunting): get_auth_request con 0x7f07d80a59c0 auth_method 0 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_done global_id 64237 payload 293 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 -- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 msgr2=0x7f07c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d80a59c0 0x7f07c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 -- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07c00039c0 msgr2=0x7f07c0003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:22:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07c00039c0 0x7f07c0003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 monclient: found mon.b 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 -- 172.21.15.119:0/1981096302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c0003530 con 0x7f07c0004810 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0004810 0x7f07c0004c90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07cc01ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f05e174da13627a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0096b0 con 0x7f07c0004810 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: dump: 2024-09-15T21:22:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc0a6440 con 0x7f07c0004810 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 10 monclient: get_monmap_and_config success 2024-09-15T21:22:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07ddad9640 4 set_mon_vals no callback set 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 1 -- 172.21.15.119:0/1981096302 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0a6830 con 0x7f07c0004810 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.587+0000 7f07d67fc640 10 monclient: dump: 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:59.609 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: shutdown 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/1981096302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0004810 msgr2=0x7f07c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 --2- 172.21.15.119:0/1981096302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0004810 0x7f07c0004c90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07cc01ca50 comp rx=0 tx=0).stop 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d7fff640 1 -- 172.21.15.119:0/1981096302 reap_dead start 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/1981096302 shutdown_connections 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/1981096302 >> 172.21.15.119:0/1981096302 conn(0x7f07d80a6fd0 msgr2=0x7f07d80a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/1981096302 shutdown_connections 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/1981096302 wait complete. 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: build_initial_monmap 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: monmap: 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:22:59.610 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 Processor -- start 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- start start 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: init 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _reopen_session rank -1 2024-09-15T21:22:59.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d81235b0 0x7f07d81cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): picked mon.noname-c con 0x7f07d81235b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 0x7f07d81cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): picked mon.noname-b con 0x7f07d8138400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07d8138880 0x7f07d81cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): picked mon.noname-a con 0x7f07d8138880 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f07d80a7ea0 con 0x7f07d8138880 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07d80a7d20 con 0x7f07d8138400 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f07d8140170 con 0x7f07d81235b0 2024-09-15T21:22:59.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): _renew_subs 2024-09-15T21:22:59.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient(hunting): authenticate will time out at 1298.389953s 2024-09-15T21:22:59.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 0x7f07d81cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:59.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d7fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07d8138880 msgr2=0x7f07d81cc9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:22:59.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 0x7f07d81cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49204/0 (socket says 172.21.15.119:49204) 2024-09-15T21:22:59.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 -- 172.21.15.119:0/2230090730 learned_addr learned my addr 172.21.15.119:0/2230090730 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:22:59.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): get_auth_request con 0x7f07d8138400 auth_method 0 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d7fff640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07d8138880 0x7f07d81cc9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d77fe640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d81235b0 0x7f07d81cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d77fe640 10 monclient(hunting): get_auth_request con 0x7f07d81235b0 auth_method 0 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:22:59.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient(hunting): handle_auth_done global_id 64240 payload 824 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 -- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d81235b0 msgr2=0x7f07d81cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d81235b0 0x7f07d81cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 -- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07d8138880 msgr2=0x7f07d81cc9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07d8138880 0x7f07d81cc9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 monclient: found mon.noname-b 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 -- 172.21.15.119:0/2230090730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d80a9780 con 0x7f07d8138400 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d6ffd640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 0x7f07d81cc460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07cc0197f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0be070 con 0x7f07d8138400 2024-09-15T21:22:59.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: dump: 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.619 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: _finish_auth 0 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: _check_auth_tickets 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc0a65f0 con 0x7f07d8138400 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:22:59.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0a69c0 con 0x7f07d8138400 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 5 monclient: authenticate success, global_id 64240 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _renew_subs 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d80a16c0 con 0x7f07d8138400 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 10 monclient: dump: 2024-09-15T21:22:59.621 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d5ffb640 4 set_mon_vals no callback set 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _renew_subs 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d8122400 con 0x7f07d8138400 2024-09-15T21:22:59.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d80a52b0 con 0x7f07d8138400 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f07cc0c1070 con 0x7f07d8138400 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07c403b970 0x7f07c403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9901+0+0 (secure 0 0 0) 0x7f07cc0ea600 con 0x7f07d8138400 2024-09-15T21:22:59.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.591+0000 7f07d77fe640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07c403b970 0x7f07c403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:22:59.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.595+0000 7f07d77fe640 10 monclient: get_auth_request con 0x7f07c403b970 auth_method 0 2024-09-15T21:22:59.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.595+0000 7f07d77fe640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07c403b970 0x7f07c403de30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f07d0001c30 tx=0x7f07d0001ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:22:59.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.599+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f07cc0ac020 con 0x7f07d8138400 2024-09-15T21:22:59.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.599+0000 7f07bffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:22:59.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.599+0000 7f07bffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:22:59.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07ddad9640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:22:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07ddad9640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:22:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07ddad9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f07d8124d10 con 0x7f07d8138400 2024-09-15T21:22:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v154) ==== 83+0+2175 (secure 0 0 0) 0x7f07cc016760 con 0x7f07d8138400 2024-09-15T21:22:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07bffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:22:59.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.771+0000 7f07bffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:22:59.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07ddad9640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:22:59.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07ddad9640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:22:59.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07ddad9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:22:59.875 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f07d80008e0 con 0x7f07d8138400 2024-09-15T21:22:59.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07bffff640 1 -- 172.21.15.119:0/2230090730 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v154) ==== 83+0+2175 (secure 0 0 0) 0x7f07cc0b0e10 con 0x7f07d8138400 2024-09-15T21:22:59.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07bffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:22:59.876 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.871+0000 7f07bffff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:22:59.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07c403b970 msgr2=0x7f07c403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07c403b970 0x7f07c403de30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f07d0001c30 tx=0x7f07d0001ac0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.878 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 10 monclient: shutdown 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 msgr2=0x7f07d81cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 0x7f07d81cc460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07cc0197f0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 shutdown_connections 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f07c403b970 0x7f07c403de30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07d8138880 0x7f07d81cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d8138400 0x7f07d81cc460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 --2- 172.21.15.119:0/2230090730 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07d81235b0 0x7f07d81cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 >> 172.21.15.119:0/2230090730 conn(0x7f07d80a6fd0 msgr2=0x7f07d813a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 shutdown_connections 2024-09-15T21:22:59.879 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:22:59.875+0000 7f07ddad9640 1 -- 172.21.15.119:0/2230090730 wait complete. 2024-09-15T21:22:59.892 INFO:tasks.workunit.client.0.smithi119.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:22:59.892 INFO:tasks.workunit.client.0.smithi119.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:22:59.892 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:22:59.893 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:22:59.893 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '11\.' 2024-09-15T21:22:59.893 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep fish 2024-09-15T21:23:00.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient: get_monmap_and_config 2024-09-15T21:23:00.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient: build_initial_monmap 2024-09-15T21:23:00.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient: monmap: 2024-09-15T21:23:00.007 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:00.007 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:00.006086+0000 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:00.006086+0000 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:00.008 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 Processor -- start 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 -- start start 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient: init 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4980a57e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4980a5bc0 0x7ff4980a6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): picked mon.noname-c con 0x7ff4980a5bc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff49813ac00 0x7ff49813cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): picked mon.noname-a con 0x7ff49813ac00 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49806dc10 con 0x7ff4980a57e0 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff49806d940 con 0x7ff49813ac00 2024-09-15T21:23:00.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff498072810 con 0x7ff4980a5bc0 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49efaf640 10 monclient(hunting): authenticate will time out at 1298.803498s 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49cd4b640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7ff4980a57e0 legacy=0x7ff4980736d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4980a5bc0 0x7ff4980a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49d54c640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7ff49813ac00 msgr2=0x7ff49813cff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff49d54c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff49813ac00 0x7ff49813cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4980a5bc0 0x7ff4980a6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49216/0 (socket says 172.21.15.119:49216) 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 1 -- 172.21.15.119:0/2120767311 learned_addr learned my addr 172.21.15.119:0/2120767311 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): get_auth_request con 0x7ff4980a5bc0 auth_method 0 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.012 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 10 monclient(hunting): handle_auth_done global_id 64243 payload 274 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 1 -- 172.21.15.119:0/2120767311 >> v2:172.21.15.17:3300/0 conn(0x7ff49813ac00 msgr2=0x7ff49813cff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 1 --2- 172.21.15.119:0/2120767311 >> v2:172.21.15.17:3300/0 conn(0x7ff49813ac00 0x7ff49813cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.003+0000 7ff497fff640 1 -- 172.21.15.119:0/2120767311 >> v1:172.21.15.17:6789/0 conn(0x7ff4980a57e0 legacy=0x7ff4980736d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff497fff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff497fff640 1 monclient: found mon.noname-c 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff497fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff497fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff497fff640 1 -- 172.21.15.119:0/2120767311 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4980a6550 con 0x7ff4980a5bc0 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff497fff640 1 --2- 172.21.15.119:0/2120767311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4980a5bc0 0x7ff4980a6010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff49806d720 tx=0x7ff48c01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7fb9cd390360fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 1 -- 172.21.15.119:0/2120767311 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0b0070 con 0x7ff4980a5bc0 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: dump: 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.019 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 1 -- 172.21.15.119:0/2120767311 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c0ab600 con 0x7ff4980a5bc0 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 5 monclient: authenticate success, global_id 64243 2024-09-15T21:23:00.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 1 -- 172.21.15.119:0/2120767311 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0ab9d0 con 0x7ff4980a5bc0 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff4977fe640 10 monclient: dump: 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.021 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 4 set_mon_vals no callback set 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 10 monclient: shutdown 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 -- 172.21.15.119:0/2120767311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4980a5bc0 msgr2=0x7ff4980a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 --2- 172.21.15.119:0/2120767311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4980a5bc0 0x7ff4980a6010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff49806d720 tx=0x7ff48c01c400 comp rx=0 tx=0).stop 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 -- 172.21.15.119:0/2120767311 shutdown_connections 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 --2- 172.21.15.119:0/2120767311 >> v2:172.21.15.17:3300/0 conn(0x7ff49813ac00 0x7ff49813cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 --2- 172.21.15.119:0/2120767311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4980a5bc0 0x7ff4980a6010 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 -- 172.21.15.119:0/2120767311 >> 172.21.15.119:0/2120767311 conn(0x7ff4980a3b80 msgr2=0x7ff49806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 -- 172.21.15.119:0/2120767311 shutdown_connections 2024-09-15T21:23:00.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 -- 172.21.15.119:0/2120767311 wait complete. 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 10 monclient: build_initial_monmap 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 10 monclient: monmap: 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 Processor -- start 2024-09-15T21:23:00.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.007+0000 7ff49efaf640 1 -- start start 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: init 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4980a57e0 0x7ff4981313b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): picked mon.noname-c con 0x7ff4980a57e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4980a5bc0 0x7ff4981318f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): picked mon.noname-a con 0x7ff4980a5bc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 0x7ff4981cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): picked mon.noname-b con 0x7ff49813ac00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff49813fed0 con 0x7ff4980a5bc0 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff49813fd50 con 0x7ff49813ac00 2024-09-15T21:23:00.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff498140050 con 0x7ff4980a57e0 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient(hunting): authenticate will time out at 1298.808499s 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 0x7ff4981cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff497fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4980a5bc0 msgr2=0x7ff4981318f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff497fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4980a5bc0 0x7ff4981318f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 0x7ff4981cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49230/0 (socket says 172.21.15.119:49230) 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 -- 172.21.15.119:0/3903349164 learned_addr learned my addr 172.21.15.119:0/3903349164 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): get_auth_request con 0x7ff49813ac00 auth_method 0 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:00.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:00.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:00.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4980a57e0 0x7ff4981313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 10 monclient(hunting): get_auth_request con 0x7ff4980a57e0 auth_method 0 2024-09-15T21:23:00.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient(hunting): handle_auth_done global_id 64246 payload 824 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 -- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4980a57e0 msgr2=0x7ff4981313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4980a57e0 0x7ff4981313b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 -- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4980a5bc0 msgr2=0x7ff4981318f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4980a5bc0 0x7ff4981318f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient: _finish_hunting 0 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 monclient: found mon.noname-b 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 -- 172.21.15.119:0/3903349164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49813d000 con 0x7ff49813ac00 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49d54c640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 0x7ff4981cd260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff49000be60 tx=0x7ff49000bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490015470 con 0x7ff49813ac00 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: dump: 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.029 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 5 monclient: authenticate success, global_id 64246 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff490010070 con 0x7ff49813ac00 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490015890 con 0x7ff49813ac00 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _renew_subs 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.030 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4980744c0 con 0x7ff49813ac00 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 10 monclient: dump: 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.031 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _renew_subs 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49809fa70 con 0x7ff49813ac00 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff496ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff49001e3f0 con 0x7ff49813ac00 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4980a6010 con 0x7ff49813ac00 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff48403b970 0x7ff48403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9901+0+0 (secure 0 0 0) 0x7ff4900515e0 con 0x7ff49813ac00 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff48403b970 0x7ff48403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 10 monclient: get_auth_request con 0x7ff48403b970 auth_method 0 2024-09-15T21:23:00.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.011+0000 7ff49cd4b640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff48403b970 0x7ff48403de30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff488003940 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.015+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff4980744c0 con 0x7ff49813ac00 2024-09-15T21:23:00.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.015+0000 7ff4957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.015+0000 7ff4957fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:00.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff49efaf640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff49efaf640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff49efaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7ff49809f310 con 0x7ff49813ac00 2024-09-15T21:23:00.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v154) ==== 83+0+2175 (secure 0 0 0) 0x7ff49813d000 con 0x7ff49813ac00 2024-09-15T21:23:00.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff4957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.179+0000 7ff4957fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:00.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff49efaf640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff49efaf640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff49efaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7ff4980008e0 con 0x7ff49813ac00 2024-09-15T21:23:00.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff4957fa640 1 -- 172.21.15.119:0/3903349164 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v154) ==== 83+0+2175 (secure 0 0 0) 0x7ff49004ebf0 con 0x7ff49813ac00 2024-09-15T21:23:00.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff4957fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.279+0000 7ff4957fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff48403b970 msgr2=0x7ff48403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff48403b970 0x7ff48403de30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff488003940 tx=0x7ff488008040 comp rx=0 tx=0).stop 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 10 monclient: shutdown 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 msgr2=0x7ff4981cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 0x7ff4981cd260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff49000be60 tx=0x7ff49000bf50 comp rx=0 tx=0).stop 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 shutdown_connections 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff48403b970 0x7ff48403de30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49813ac00 0x7ff4981cd260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4980a5bc0 0x7ff4981318f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 --2- 172.21.15.119:0/3903349164 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4980a57e0 0x7ff4981313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 >> 172.21.15.119:0/3903349164 conn(0x7ff4980a3b80 msgr2=0x7ff4980a1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:00.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 shutdown_connections 2024-09-15T21:23:00.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.283+0000 7ff49efaf640 1 -- 172.21.15.119:0/3903349164 wait complete. 2024-09-15T21:23:00.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:00.301 INFO:tasks.workunit.client.0.smithi119.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:00.302 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:00.302 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:00.302 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '12\.' 2024-09-15T21:23:00.302 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep fish 2024-09-15T21:23:00.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.411+0000 7efc4f817640 10 monclient: get_monmap_and_config 2024-09-15T21:23:00.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.411+0000 7efc4f817640 10 monclient: build_initial_monmap 2024-09-15T21:23:00.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.411+0000 7efc4f817640 10 monclient: monmap: 2024-09-15T21:23:00.418 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:00.418 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.418 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:00.415853+0000 2024-09-15T21:23:00.419 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:00.415853+0000 2024-09-15T21:23:00.419 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:00.419 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.419 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:00.419 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:00.420 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:00.420 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:00.420 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:00.420 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.411+0000 7efc4f817640 1 Processor -- start 2024-09-15T21:23:00.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.411+0000 7efc4f817640 1 -- start start 2024-09-15T21:23:00.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.411+0000 7efc4f817640 10 monclient: init 2024-09-15T21:23:00.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:00.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2024-09-15T21:23:00.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc48135ca0 0x7efc48136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): picked mon.noname-c con 0x7efc48135ca0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:00.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc480a0160 0x7efc480a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): picked mon.noname-b con 0x7efc480a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:00.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efc4806ec30 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:00.436 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc481365c0 con 0x7efc4806ec30 2024-09-15T21:23:00.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7efc4809cf30 con 0x7efc48135ca0 2024-09-15T21:23:00.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efc480a1ca0 con 0x7efc480a0160 2024-09-15T21:23:00.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:00.437 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient(hunting): authenticate will time out at 1299.212464s 2024-09-15T21:23:00.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc48135ca0 0x7efc48136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc48135ca0 0x7efc48136080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49238/0 (socket says 172.21.15.119:49238) 2024-09-15T21:23:00.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4ddb4640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7efc4806ec30 legacy=0x7efc480a0f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 -- 172.21.15.119:0/884470506 learned_addr learned my addr 172.21.15.119:0/884470506 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): get_auth_request con 0x7efc48135ca0 auth_method 0 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:00.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:00.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:00.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4cdb2640 1 --2- 172.21.15.119:0/884470506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc480a0160 0x7efc480a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4cdb2640 10 monclient(hunting): get_auth_request con 0x7efc480a0160 auth_method 0 2024-09-15T21:23:00.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4cdb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4cdb2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4cdb2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient(hunting): handle_auth_done global_id 64249 payload 274 2024-09-15T21:23:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 -- 172.21.15.119:0/884470506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc480a0160 msgr2=0x7efc480a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 --2- 172.21.15.119:0/884470506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc480a0160 0x7efc480a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 -- 172.21.15.119:0/884470506 >> v1:172.21.15.17:6789/0 conn(0x7efc4806ec30 legacy=0x7efc480a0f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient: _finish_hunting 0 2024-09-15T21:23:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 monclient: found mon.noname-c 2024-09-15T21:23:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 -- 172.21.15.119:0/884470506 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc480a1f80 con 0x7efc48135ca0 2024-09-15T21:23:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4d5b3640 1 --2- 172.21.15.119:0/884470506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48136080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc44009870 tx=0x7efc4401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3ce70c9fd98f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 1 -- 172.21.15.119:0/884470506 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440b0070 con 0x7efc48135ca0 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: dump: 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.444 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: _finish_auth 0 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 5 monclient: authenticate success, global_id 64249 2024-09-15T21:23:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 1 -- 172.21.15.119:0/884470506 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc440ab600 con 0x7efc48135ca0 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 4 set_mon_vals no callback set 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 1 -- 172.21.15.119:0/884470506 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440ab9d0 con 0x7efc48135ca0 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc3ffff640 10 monclient: dump: 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 10 monclient: shutdown 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 -- 172.21.15.119:0/884470506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 msgr2=0x7efc48136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.415+0000 7efc4f817640 1 --2- 172.21.15.119:0/884470506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48136080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc44009870 tx=0x7efc4401c420 comp rx=0 tx=0).stop 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/884470506 shutdown_connections 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 --2- 172.21.15.119:0/884470506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc480a0160 0x7efc480a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 --2- 172.21.15.119:0/884470506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48136080 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/884470506 >> 172.21.15.119:0/884470506 conn(0x7efc480a2ae0 msgr2=0x7efc480a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/884470506 shutdown_connections 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/884470506 wait complete. 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: build_initial_monmap 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: monmap: 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.448 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 Processor -- start 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- start start 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: init 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc4806ec30 0x7efc481cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): picked mon.noname-c con 0x7efc4806ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc480a0160 0x7efc48133be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): picked mon.noname-a con 0x7efc480a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:00.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): picked mon.noname-b con 0x7efc48135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efc481403c0 con 0x7efc480a0160 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efc48140240 con 0x7efc48135ca0 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efc48140540 con 0x7efc4806ec30 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient(hunting): authenticate will time out at 1299.217594s 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4cdb2640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc480a0160 msgr2=0x7efc48133be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4cdb2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc480a0160 0x7efc48133be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:00.450 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48134120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49252/0 (socket says 172.21.15.119:49252) 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 -- 172.21.15.119:0/4037500029 learned_addr learned my addr 172.21.15.119:0/4037500029 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): get_auth_request con 0x7efc48135ca0 auth_method 0 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:00.451 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc4806ec30 0x7efc481cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 10 monclient(hunting): get_auth_request con 0x7efc4806ec30 auth_method 0 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient(hunting): handle_auth_done global_id 64252 payload 824 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 -- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc4806ec30 msgr2=0x7efc481cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc4806ec30 0x7efc481cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 -- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc480a0160 msgr2=0x7efc48133be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc480a0160 0x7efc48133be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient: _finish_hunting 0 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 monclient: found mon.noname-b 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 -- 172.21.15.119:0/4037500029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4813d420 con 0x7efc48135ca0 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc4806ec30 0x7efc481cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4ddb4640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48134120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc4000be20 tx=0x7efc40009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4000ce10 con 0x7efc48135ca0 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: dump: 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.453 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc40010030 con 0x7efc48135ca0 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:00.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40016670 con 0x7efc48135ca0 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 5 monclient: authenticate success, global_id 64252 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _renew_subs 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc481d08a0 con 0x7efc48135ca0 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 10 monclient: dump: 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.455 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3f7fe640 4 set_mon_vals no callback set 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _renew_subs 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4809d4e0 con 0x7efc48135ca0 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7efc400153e0 con 0x7efc48135ca0 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc48136080 con 0x7efc48135ca0 2024-09-15T21:23:00.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc2c03b970 0x7efc2c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9901+0+0 (secure 0 0 0) 0x7efc40051420 con 0x7efc48135ca0 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc2c03b970 0x7efc2c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.419+0000 7efc4d5b3640 10 monclient: get_auth_request con 0x7efc2c03b970 auth_method 0 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.423+0000 7efc4d5b3640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc2c03b970 0x7efc2c03de30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc440ad000 tx=0x7efc44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.427+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7efc4004ea30 con 0x7efc48135ca0 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.427+0000 7efc3dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.427+0000 7efc3dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:00.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc4f817640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc4f817640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc4f817640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7efc4809f8f0 con 0x7efc48135ca0 2024-09-15T21:23:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v154) ==== 83+0+2175 (secure 0 0 0) 0x7efc40014d50 con 0x7efc48135ca0 2024-09-15T21:23:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc3dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.607+0000 7efc3dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:00.713 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.707+0000 7efc4f817640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.707+0000 7efc4f817640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.707+0000 7efc4f817640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.707+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7efc480008e0 con 0x7efc48135ca0 2024-09-15T21:23:00.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc3dffb640 1 -- 172.21.15.119:0/4037500029 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v154) ==== 83+0+2175 (secure 0 0 0) 0x7efc400060d0 con 0x7efc48135ca0 2024-09-15T21:23:00.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc3dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:00.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc3dffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc2c03b970 msgr2=0x7efc2c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc2c03b970 0x7efc2c03de30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc440ad000 tx=0x7efc44008040 comp rx=0 tx=0).stop 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 10 monclient: shutdown 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 msgr2=0x7efc48134120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48134120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc4000be20 tx=0x7efc40009f10 comp rx=0 tx=0).stop 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 shutdown_connections 2024-09-15T21:23:00.717 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efc2c03b970 0x7efc2c03de30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc48135ca0 0x7efc48134120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc480a0160 0x7efc48133be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 --2- 172.21.15.119:0/4037500029 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efc4806ec30 0x7efc481cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 >> 172.21.15.119:0/4037500029 conn(0x7efc480a2ae0 msgr2=0x7efc480721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:00.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 shutdown_connections 2024-09-15T21:23:00.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.711+0000 7efc4f817640 1 -- 172.21.15.119:0/4037500029 wait complete. 2024-09-15T21:23:00.731 INFO:tasks.workunit.client.0.smithi119.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:00.731 INFO:tasks.workunit.client.0.smithi119.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:00.731 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm-device-class osd.0 2024-09-15T21:23:00.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient: get_monmap_and_config 2024-09-15T21:23:00.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient: build_initial_monmap 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient: monmap: 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:00.843302+0000 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:00.843302+0000 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:00.845 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:00.846 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 1 Processor -- start 2024-09-15T21:23:00.852 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 1 -- start start 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient: init 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0260135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a0160 0x7f026013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.839+0000 7f02668f4640 10 monclient(hunting): picked mon.noname-b con 0x7f02600a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f026006ec30 0x7f026013cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient(hunting): picked mon.noname-c con 0x7f026006ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026009cf30 con 0x7f0260135ca0 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f02600a1ca0 con 0x7f026006ec30 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f02600a1f80 con 0x7f02600a0160 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient(hunting): authenticate will time out at 1299.640092s 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025ffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f0260135ca0 legacy=0x7f0260136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f026006ec30 0x7f026013cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f026006ec30 0x7f026013cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49260/0 (socket says 172.21.15.119:49260) 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 -- 172.21.15.119:0/3791363938 learned_addr learned my addr 172.21.15.119:0/3791363938 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): get_auth_request con 0x7f026006ec30 auth_method 0 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:00.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0257fff640 1 --2- 172.21.15.119:0/3791363938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a0160 0x7f026013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0257fff640 10 monclient(hunting): get_auth_request con 0x7f02600a0160 auth_method 0 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0257fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0257fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0257fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient(hunting): handle_auth_done global_id 64255 payload 274 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 -- 172.21.15.119:0/3791363938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a0160 msgr2=0x7f026013aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 --2- 172.21.15.119:0/3791363938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a0160 0x7f026013aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 -- 172.21.15.119:0/3791363938 >> v1:172.21.15.17:6789/0 conn(0x7f0260135ca0 legacy=0x7f0260136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient: _finish_hunting 0 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 monclient: found mon.noname-c 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.855 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 -- 172.21.15.119:0/3791363938 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600a3860 con 0x7f026006ec30 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f0264e91640 1 --2- 172.21.15.119:0/3791363938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026006ec30 0x7f026013cf50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0258009e50 tx=0x7f025801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f9fb3e23d3f6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 1 -- 172.21.15.119:0/3791363938 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02580a5640 con 0x7f026006ec30 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: dump: 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.856 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 5 monclient: authenticate success, global_id 64255 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 1 -- 172.21.15.119:0/3791363938 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02580a57e0 con 0x7f026006ec30 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 4 set_mon_vals no callback set 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 1 -- 172.21.15.119:0/3791363938 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02580a5bb0 con 0x7f026006ec30 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f025f7fe640 10 monclient: dump: 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient: shutdown 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:00.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- 172.21.15.119:0/3791363938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026006ec30 msgr2=0x7f026013cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 --2- 172.21.15.119:0/3791363938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026006ec30 0x7f026013cf50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0258009e50 tx=0x7f025801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- 172.21.15.119:0/3791363938 shutdown_connections 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 --2- 172.21.15.119:0/3791363938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026006ec30 0x7f026013cf50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 --2- 172.21.15.119:0/3791363938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a0160 0x7f026013aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- 172.21.15.119:0/3791363938 >> 172.21.15.119:0/3791363938 conn(0x7f02600a2ae0 msgr2=0x7f02600a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- 172.21.15.119:0/3791363938 shutdown_connections 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 1 -- 172.21.15.119:0/3791363938 wait complete. 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient: build_initial_monmap 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.843+0000 7f02668f4640 10 monclient: monmap: 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:00.859 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 Processor -- start 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- start start 2024-09-15T21:23:00.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: init 2024-09-15T21:23:00.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:00.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:00.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f026006ec30 0x7f026009d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): picked mon.noname-c con 0x7f026006ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 0x7f026013b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): picked mon.noname-b con 0x7f02600a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0260135ca0 0x7f0260133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): picked mon.noname-a con 0x7f0260135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f026013fe30 con 0x7f0260135ca0 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f026013fcb0 con 0x7f02600a0160 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f026013ffb0 con 0x7f026006ec30 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:00.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient(hunting): authenticate will time out at 1299.644778s 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 0x7f026013b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 0x7f026013b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49268/0 (socket says 172.21.15.119:49268) 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0264e91640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0260135ca0 msgr2=0x7f0260133b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 -- 172.21.15.119:0/3032124965 learned_addr learned my addr 172.21.15.119:0/3032124965 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): get_auth_request con 0x7f02600a0160 auth_method 0 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0264e91640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0260135ca0 0x7f0260133b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:00.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025ffff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f026006ec30 0x7f026009d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient(hunting): handle_auth_done global_id 64258 payload 824 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 -- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f026006ec30 msgr2=0x7f026009d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f026006ec30 0x7f026009d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 -- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0260135ca0 msgr2=0x7f0260133b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0260135ca0 0x7f0260133b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 monclient: found mon.noname-b 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 -- 172.21.15.119:0/3032124965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600a57d0 con 0x7f02600a0160 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025ffff640 10 monclient: get_auth_request con 0x7f026006ec30 auth_method 0 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025ffff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f026006ec30 0x7f026009d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f0257fff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 0x7f026013b800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f024800d800 tx=0x7f024800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800b6c0 con 0x7f02600a0160 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: dump: 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.867 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0248019070 con 0x7f02600a0160 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800bae0 con 0x7f02600a0160 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 10 monclient: dump: 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.868 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 5 monclient: authenticate success, global_id 64258 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _renew_subs 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025effd640 4 set_mon_vals no callback set 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026013ce00 con 0x7f02600a0160 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _renew_subs 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026009f1f0 con 0x7f02600a0160 2024-09-15T21:23:00.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f024800bc80 con 0x7f02600a0160 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026009f8f0 con 0x7f02600a0160 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024403b9a0 0x7f024403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9901+0+0 (secure 0 0 0) 0x7f0248051450 con 0x7f02600a0160 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025ffff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024403b9a0 0x7f024403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025ffff640 10 monclient: get_auth_request con 0x7f024403b9a0 auth_method 0 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.847+0000 7f025ffff640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024403b9a0 0x7f024403de60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0250006fd0 tx=0x7f0250008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.851+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f024804ea60 con 0x7f02600a0160 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.851+0000 7f025d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:00.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:00.851+0000 7f025d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:01.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.011+0000 7f02668f4640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:01.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.011+0000 7f02668f4640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:01.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.011+0000 7f02668f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:01.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.011+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7f02601340d0 con 0x7f02600a0160 2024-09-15T21:23:01.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.463+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v155) ==== 123+0+0 (secure 0 0 0) 0x7f0248011d50 con 0x7f02600a0160 2024-09-15T21:23:01.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.463+0000 7f025d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:01.469 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.463+0000 7f025d7fa640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-09-15T21:23:01.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.567+0000 7f02668f4640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:01.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.567+0000 7f02668f4640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:01.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.567+0000 7f02668f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:01.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:01.567+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7f02601343c0 con 0x7f02600a0160 2024-09-15T21:23:02.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.467+0000 7f025d7fa640 1 -- 172.21.15.119:0/3032124965 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v156) ==== 149+0+0 (secure 0 0 0) 0x7f0248013390 con 0x7f02600a0160 2024-09-15T21:23:02.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.467+0000 7f025d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:02.473 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.467+0000 7f025d7fa640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:23:02.473 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:23:02.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024403b9a0 msgr2=0x7f024403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:02.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024403b9a0 0x7f024403de60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0250006fd0 tx=0x7f0250008040 comp rx=0 tx=0).stop 2024-09-15T21:23:02.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 10 monclient: shutdown 2024-09-15T21:23:02.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:02.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 msgr2=0x7f026013b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:02.477 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 0x7f026013b800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f024800d800 tx=0x7f024800dcc0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 shutdown_connections 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024403b9a0 0x7f024403de60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0260135ca0 0x7f0260133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02600a0160 0x7f026013b800 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 --2- 172.21.15.119:0/3032124965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f026006ec30 0x7f026009d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 >> 172.21.15.119:0/3032124965 conn(0x7f02600a2ae0 msgr2=0x7f0260071020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 shutdown_connections 2024-09-15T21:23:02.478 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.471+0000 7f02668f4640 1 -- 172.21.15.119:0/3032124965 wait complete. 2024-09-15T21:23:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush set-device-class globster osd.0 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient: get_monmap_and_config 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient: build_initial_monmap 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient: monmap: 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:02.613657+0000 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:02.613657+0000 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:02.615 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:02.616 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:02.616 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:02.616 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:02.616 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:02.616 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:02.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 Processor -- start 2024-09-15T21:23:02.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 -- start start 2024-09-15T21:23:02.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient: init 2024-09-15T21:23:02.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37100a0900 0x7f37100a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): picked mon.noname-c con 0x7f37100a0900 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a1220 0x7f37101356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): picked mon.noname-b con 0x7f37100a1220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3710135c20 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37100a2240 con 0x7f3710135c20 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f371009fdd0 con 0x7f37100a0900 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f37100a1c80 con 0x7f37100a1220 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:02.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3717606640 10 monclient(hunting): authenticate will time out at 1301.410208s 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37100a0900 0x7f37100a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37100a0900 0x7f37100a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56836/0 (socket says 172.21.15.119:56836) 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 -- 172.21.15.119:0/4017624126 learned_addr learned my addr 172.21.15.119:0/4017624126 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): get_auth_request con 0x7f37100a0900 auth_method 0 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3714ba1640 1 --2- 172.21.15.119:0/4017624126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a1220 0x7f37101356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3714ba1640 10 monclient(hunting): get_auth_request con 0x7f37100a1220 auth_method 0 2024-09-15T21:23:02.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3714ba1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3714ba1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3714ba1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient(hunting): handle_auth_done global_id 64261 payload 274 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 -- 172.21.15.119:0/4017624126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a1220 msgr2=0x7f37101356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 --2- 172.21.15.119:0/4017624126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a1220 0x7f37101356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 -- 172.21.15.119:0/4017624126 >> v1:172.21.15.119:6789/0 conn(0x7f3710135c20 legacy=0x7f371013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient: _finish_hunting 0 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 monclient: found mon.noname-c 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f37153a2640 1 -- 172.21.15.119:0/4017624126 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37100a1f60 con 0x7f37100a0900 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3703fff640 1 -- 172.21.15.119:0/4017624126 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828111169 0 0) 0x7f37100a2240 con 0x7f3710135c20 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.611+0000 7f3703fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:23:02.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f37153a2640 1 --2- 172.21.15.119:0/4017624126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a0900 0x7f37100a0ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f370401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4d3e93035980032 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 1 -- 172.21.15.119:0/4017624126 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37040ab640 con 0x7f37100a0900 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: dump: 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.626 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:02.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 1 -- 172.21.15.119:0/4017624126 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37040b0070 con 0x7f37100a0900 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 5 monclient: authenticate success, global_id 64261 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 1 -- 172.21.15.119:0/4017624126 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37040aba00 con 0x7f37100a0900 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 4 set_mon_vals no callback set 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3703fff640 10 monclient: dump: 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:02.628 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: shutdown 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- 172.21.15.119:0/4017624126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a0900 msgr2=0x7f37100a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 --2- 172.21.15.119:0/4017624126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a0900 0x7f37100a0ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f370401c420 comp rx=0 tx=0).stop 2024-09-15T21:23:02.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- 172.21.15.119:0/4017624126 shutdown_connections 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 --2- 172.21.15.119:0/4017624126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a1220 0x7f37101356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 --2- 172.21.15.119:0/4017624126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a0900 0x7f37100a0ce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- 172.21.15.119:0/4017624126 >> 172.21.15.119:0/4017624126 conn(0x7f371009be60 msgr2=0x7f371009c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- 172.21.15.119:0/4017624126 shutdown_connections 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- 172.21.15.119:0/4017624126 wait complete. 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: build_initial_monmap 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: monmap: 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:02.630 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 Processor -- start 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- start start 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: init 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:02.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a0900 0x7f371011c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): picked mon.noname-c con 0x7f37100a0900 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 0x7f371011ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): picked mon.noname-b con 0x7f37100a1220 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3710135c20 0x7f371011cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): picked mon.noname-a con 0x7f3710135c20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f371006ed10 con 0x7f3710135c20 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f371006df10 con 0x7f37100a1220 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f371006ee90 con 0x7f37100a0900 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:02.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient(hunting): authenticate will time out at 1301.415116s 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 0x7f371011ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 0x7f371011ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56842/0 (socket says 172.21.15.119:56842) 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 -- 172.21.15.119:0/1923139288 learned_addr learned my addr 172.21.15.119:0/1923139288 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): get_auth_request con 0x7f37100a1220 auth_method 0 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3715ba3640 1 -- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3710135c20 msgr2=0x7f371011cfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3715ba3640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3710135c20 0x7f371011cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:02.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f37153a2640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a0900 0x7f371011c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient(hunting): handle_auth_done global_id 64267 payload 824 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 -- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a0900 msgr2=0x7f371011c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a0900 0x7f371011c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 -- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3710135c20 msgr2=0x7f371011cfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3710135c20 0x7f371011cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient: _finish_hunting 0 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 monclient: found mon.noname-b 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:02.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 -- 172.21.15.119:0/1923139288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371011eb70 con 0x7f37100a1220 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f37153a2640 10 monclient: get_auth_request con 0x7f37100a0900 auth_method 0 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f37153a2640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a0900 0x7f371011c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3714ba1640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 0x7f371011ca90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f370c00d800 tx=0x7f370c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c004020 con 0x7f37100a1220 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 10 monclient: dump: 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:02.635 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370c0041c0 con 0x7f37100a1220 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 5 monclient: authenticate success, global_id 64267 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: _renew_subs 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.615+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371013d480 con 0x7f37100a1220 2024-09-15T21:23:02.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c0043b0 con 0x7f37100a1220 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 10 monclient: dump: 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:02.637 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f37037fe640 4 set_mon_vals no callback set 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 10 monclient: _renew_subs 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371011f130 con 0x7f37100a1220 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f370c01b020 con 0x7f37100a1220 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37100a0170 con 0x7f37100a1220 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f36f403b970 0x7f36f403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9236+0+0 (secure 0 0 0) 0x7f370c050b70 con 0x7f37100a1220 2024-09-15T21:23:02.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f37153a2640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f36f403b970 0x7f36f403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:02.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f37153a2640 10 monclient: get_auth_request con 0x7f36f403b970 auth_method 0 2024-09-15T21:23:02.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.619+0000 7f37153a2640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f36f403b970 0x7f36f403de30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37040af000 tx=0x7f3704008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:02.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.623+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f370c04e410 con 0x7f37100a1220 2024-09-15T21:23:02.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.623+0000 7f3701ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:02.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.623+0000 7f3701ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:02.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.799+0000 7f3717606640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T21:23:02.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.799+0000 7f3717606640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T21:23:02.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.799+0000 7f3717606640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:02.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:02.799+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) -- 0x7f371011dce0 con 0x7f37100a1220 2024-09-15T21:23:03.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.371+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'globster' v157) ==== 145+0+0 (secure 0 0 0) 0x7f370c012ce0 con 0x7f37100a1220 2024-09-15T21:23:03.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.371+0000 7f3701ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T21:23:03.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.371+0000 7f3701ffb640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'globster' 2024-09-15T21:23:03.483 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.479+0000 7f3717606640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T21:23:03.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.479+0000 7f3717606640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T21:23:03.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.479+0000 7f3717606640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:03.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:03.479+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) -- 0x7f371006a810 con 0x7f37100a1220 2024-09-15T21:23:04.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.379+0000 7f3701ffb640 1 -- 172.21.15.119:0/1923139288 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' v158) ==== 257+0+0 (secure 0 0 0) 0x7f370c003620 con 0x7f37100a1220 2024-09-15T21:23:04.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.379+0000 7f3701ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T21:23:04.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.379+0000 7f3701ffb640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-09-15T21:23:04.385 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-09-15T21:23:04.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f36f403b970 msgr2=0x7f36f403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f36f403b970 0x7f36f403de30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37040af000 tx=0x7f3704008040 comp rx=0 tx=0).stop 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 10 monclient: shutdown 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 msgr2=0x7f371011ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 0x7f371011ca90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f370c00d800 tx=0x7f370c00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 shutdown_connections 2024-09-15T21:23:04.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f36f403b970 0x7f36f403de30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3710135c20 0x7f371011cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37100a1220 0x7f371011ca90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 --2- 172.21.15.119:0/1923139288 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f37100a0900 0x7f371011c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 >> 172.21.15.119:0/1923139288 conn(0x7f371009be60 msgr2=0x7f37100a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:04.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 shutdown_connections 2024-09-15T21:23:04.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.383+0000 7f3717606640 1 -- 172.21.15.119:0/1923139288 wait complete. 2024-09-15T21:23:04.409 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:04.409 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:04.409 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '10\.' 2024-09-15T21:23:04.409 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep globster 2024-09-15T21:23:04.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient: get_monmap_and_config 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient: build_initial_monmap 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient: monmap: 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:04.526408+0000 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:04.526408+0000 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:04.528 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:04.529 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:04.529 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:04.529 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:04.529 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 1 Processor -- start 2024-09-15T21:23:04.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 1 -- start start 2024-09-15T21:23:04.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient: init 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 0x7f9f4c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient(hunting): picked mon.noname-b con 0x7f9f4c0a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9f4c0a5780 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9f4c1383a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c0a1150 con 0x7f9f4c1383a0 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c0a0ef0 con 0x7f9f4c0a5780 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c0a3c80 con 0x7f9f4c0a4e60 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f54614640 10 monclient(hunting): authenticate will time out at 1303.323612s 2024-09-15T21:23:04.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f51baf640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9f4c0a5780 0x7f9f4c137c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57096/0 (socket says 172.21.15.119:57096) 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f51baf640 1 -- 172.21.15.119:0/3713479040 learned_addr learned my addr 172.21.15.119:0/3713479040 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.523+0000 7f9f52bb1640 1 -- 172.21.15.119:0/3713479040 >> v1:172.21.15.17:6789/0 conn(0x7f9f4c1383a0 legacy=0x7f9f4c13ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 0x7f9f4c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): get_auth_request con 0x7f9f4c0a4e60 auth_method 0 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420701918 0 0) 0x7f9f4c0a0ef0 con 0x7f9f4c0a5780 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): _negotiate 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): my global_id is 64270 2024-09-15T21:23:04.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f38003530 con 0x7f9f4c0a5780 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 702577761 0 0) 0x7f9f38003530 con 0x7f9f4c0a5780 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 msgr2=0x7f9f4c0a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 0x7f9f4c0a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 >> v1:172.21.15.17:6789/0 conn(0x7f9f4c1383a0 legacy=0x7f9f4c13ff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: _finish_hunting 0 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c0a6cf0 con 0x7f9f4c0a5780 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: _finish_auth 0 2024-09-15T21:23:04.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: _check_auth_tickets 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f9f3c001700 con 0x7f9f4c0a5780 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 5 monclient: authenticate success, global_id 64270 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: dump: 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.544 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 >> v1:172.21.15.119:6789/0 conn(0x7f9f4c0a5780 legacy=0x7f9f4c137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 0x7f9f380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): picked mon.c con 0x7f9f4c0a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003f20 0x7f9f38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): picked mon.b con 0x7f9f38003f20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f38004d50 0x7f9f380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): picked mon.a con 0x7f9f38004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3800bb90 con 0x7f9f38004d50 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3800bd40 con 0x7f9f38003f20 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3800bf40 con 0x7f9f4c0a4e60 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:04.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f52bb1640 1 -- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f38004d50 msgr2=0x7f9f380051d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003f20 0x7f9f38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f52bb1640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f38004d50 0x7f9f380051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): get_auth_request con 0x7f9f38003f20 auth_method 0 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:04.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 0x7f9f380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): get_auth_request con 0x7f9f4c0a4e60 auth_method 0 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f523b0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient(hunting): handle_auth_done global_id 64270 payload 293 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 -- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 msgr2=0x7f9f380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f4c0a4e60 0x7f9f380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 -- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f38004d50 msgr2=0x7f9f380051d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f38004d50 0x7f9f380051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient: _finish_hunting 0 2024-09-15T21:23:04.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 monclient: found mon.b 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 -- 172.21.15.119:0/3713479040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f38003530 con 0x7f9f38003f20 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient: _finish_auth 0 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 10 monclient: _check_auth_tickets 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f51baf640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003f20 0x7f9f38004810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c240 tx=0x7f9f3c01cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=763ddf7a57400a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c007d70 con 0x7f9f38003f20 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: dump: 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.549 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0a9020 con 0x7f9f38003f20 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 1 -- 172.21.15.119:0/3713479040 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c005b20 con 0x7f9f38003f20 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 4 set_mon_vals no callback set 2024-09-15T21:23:04.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f513ae640 10 monclient: dump: 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 10 monclient: shutdown 2024-09-15T21:23:04.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 1 -- 172.21.15.119:0/3713479040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003f20 msgr2=0x7f9f38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.527+0000 7f9f54614640 1 --2- 172.21.15.119:0/3713479040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003f20 0x7f9f38004810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c240 tx=0x7f9f3c01cbc0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 -- 172.21.15.119:0/3713479040 reap_dead start 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/3713479040 shutdown_connections 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/3713479040 >> 172.21.15.119:0/3713479040 conn(0x7f9f4c0a6f60 msgr2=0x7f9f4c0a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/3713479040 shutdown_connections 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/3713479040 wait complete. 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: build_initial_monmap 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: monmap: 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:04.552 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 Processor -- start 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- start start 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: init 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f38003f20 0x7f9f4c136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): picked mon.noname-c con 0x7f9f38003f20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:04.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f4c136cb0 0x7f9f4c13ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): picked mon.noname-a con 0x7f9f4c136cb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 0x7f9f4c13f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): picked mon.noname-b con 0x7f9f4c1370b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c0a7e30 con 0x7f9f4c136cb0 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c0a7cb0 con 0x7f9f4c1370b0 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c0a2190 con 0x7f9f38003f20 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient(hunting): authenticate will time out at 1303.329819s 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f51baf640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f4c136cb0 msgr2=0x7f9f4c13ed20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 0x7f9f4c13f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f51baf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f4c136cb0 0x7f9f4c13ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 0x7f9f4c13f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56860/0 (socket says 172.21.15.119:56860) 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 -- 172.21.15.119:0/2871799141 learned_addr learned my addr 172.21.15.119:0/2871799141 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): get_auth_request con 0x7f9f4c1370b0 auth_method 0 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f523b0640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f38003f20 0x7f9f4c136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f523b0640 10 monclient(hunting): get_auth_request con 0x7f9f38003f20 auth_method 0 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f523b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f523b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f523b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient(hunting): handle_auth_done global_id 64273 payload 824 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 -- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f38003f20 msgr2=0x7f9f4c136770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f38003f20 0x7f9f4c136770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 -- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f4c136cb0 msgr2=0x7f9f4c13ed20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f4c136cb0 0x7f9f4c13ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient: _finish_hunting 0 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 monclient: found mon.noname-b 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 -- 172.21.15.119:0/2871799141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c122280 con 0x7f9f4c1370b0 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f52bb1640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 0x7f9f4c13f260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c138780 tx=0x7f9f4400bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:04.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44015470 con 0x7f9f4c1370b0 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: dump: 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.557 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f44010070 con 0x7f9f4c1370b0 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 5 monclient: authenticate success, global_id 64273 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _renew_subs 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c0a3140 con 0x7f9f4c1370b0 2024-09-15T21:23:04.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44015870 con 0x7f9f4c1370b0 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 10 monclient: dump: 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.559 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f43fff640 4 set_mon_vals no callback set 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _renew_subs 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c121cc0 con 0x7f9f4c1370b0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c077180 con 0x7f9f4c1370b0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9f4401e3f0 con 0x7f9f4c1370b0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3003b970 0x7f9f3003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.531+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9925+0+0 (secure 0 0 0) 0x7f9f44051330 con 0x7f9f4c1370b0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.535+0000 7f9f523b0640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3003b970 0x7f9f3003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.535+0000 7f9f523b0640 10 monclient: get_auth_request con 0x7f9f3003b970 auth_method 0 2024-09-15T21:23:04.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.535+0000 7f9f523b0640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3003b970 0x7f9f3003de30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009a30 tx=0x7f9f48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:04.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.539+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9f4404e920 con 0x7f9f4c1370b0 2024-09-15T21:23:04.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.539+0000 7f9f42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:04.561 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.539+0000 7f9f42ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:04.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f54614640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:04.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f54614640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:04.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f54614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f9f4c120d60 con 0x7f9f4c1370b0 2024-09-15T21:23:04.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v158) ==== 83+0+2378 (secure 0 0 0) 0x7f9f4401d3a0 con 0x7f9f4c1370b0 2024-09-15T21:23:04.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:04.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.715+0000 7f9f42ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:04.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.811+0000 7f9f54614640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:04.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.811+0000 7f9f54614640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:04.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.811+0000 7f9f54614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.811+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f9f4c137c10 con 0x7f9f4c1370b0 2024-09-15T21:23:04.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f42ffd640 1 -- 172.21.15.119:0/2871799141 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v158) ==== 83+0+2378 (secure 0 0 0) 0x7f9f44014cf0 con 0x7f9f4c1370b0 2024-09-15T21:23:04.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f42ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:04.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f42ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:04.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3003b970 msgr2=0x7f9f3003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3003b970 0x7f9f3003de30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009a30 tx=0x7f9f48008040 comp rx=0 tx=0).stop 2024-09-15T21:23:04.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 10 monclient: shutdown 2024-09-15T21:23:04.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:04.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 msgr2=0x7f9f4c13f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 0x7f9f4c13f260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c138780 tx=0x7f9f4400bf80 comp rx=0 tx=0).stop 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 shutdown_connections 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3003b970 0x7f9f3003de30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f4c1370b0 0x7f9f4c13f260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f4c136cb0 0x7f9f4c13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 --2- 172.21.15.119:0/2871799141 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f38003f20 0x7f9f4c136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 >> 172.21.15.119:0/2871799141 conn(0x7f9f4c0a6f60 msgr2=0x7f9f4c0715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 shutdown_connections 2024-09-15T21:23:04.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.815+0000 7f9f54614640 1 -- 172.21.15.119:0/2871799141 wait complete. 2024-09-15T21:23:04.835 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:04.835 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:04.836 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:04.836 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:04.836 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep globster 2024-09-15T21:23:04.836 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '11\.' 2024-09-15T21:23:04.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient: get_monmap_and_config 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient: build_initial_monmap 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient: monmap: 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:04.950382+0000 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:04.950382+0000 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.952 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:04.953 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:04.953 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:04.953 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:04.953 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:04.953 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 Processor -- start 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 -- start start 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient: init 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa0b4135ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:04.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): picked mon.noname-b con 0x7fa0b40a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0b406ec30 0x7fa0b413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): picked mon.noname-c con 0x7fa0b406ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b409cf30 con 0x7fa0b4135ca0 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b40a1ca0 con 0x7fa0b406ec30 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b40a1f80 con 0x7fa0b40a0160 2024-09-15T21:23:04.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:04.957 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0babe9640 10 monclient(hunting): authenticate will time out at 1303.746990s 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b8985640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa0b4135ca0 0x7fa0b4136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57098/0 (socket says 172.21.15.119:57098) 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b8985640 1 -- 172.21.15.119:0/3363434041 learned_addr learned my addr 172.21.15.119:0/3363434041 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 1 --2- 172.21.15.119:0/3363434041 >> v2:172.21.15.119:3300/0 conn(0x7fa0b406ec30 0x7fa0b413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): get_auth_request con 0x7fa0b406ec30 auth_method 0 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b3fff640 1 --2- 172.21.15.119:0/3363434041 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b3fff640 10 monclient(hunting): get_auth_request con 0x7fa0b40a0160 auth_method 0 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient(hunting): handle_auth_done global_id 64276 payload 274 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 1 -- 172.21.15.119:0/3363434041 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 msgr2=0x7fa0b413aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 1 --2- 172.21.15.119:0/3363434041 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 1 -- 172.21.15.119:0/3363434041 >> v1:172.21.15.119:6789/0 conn(0x7fa0b4135ca0 legacy=0x7fa0b4136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient: _finish_hunting 0 2024-09-15T21:23:04.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 1 monclient: found mon.noname-c 2024-09-15T21:23:04.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:04.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.960 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.947+0000 7fa0b9186640 1 -- 172.21.15.119:0/3363434041 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b40a3860 con 0x7fa0b406ec30 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b9186640 1 --2- 172.21.15.119:0/3363434041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b406ec30 0x7fa0b413cf50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009d90 tx=0x7fa0ac01c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5c2280444d2b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 1 -- 172.21.15.119:0/3363434041 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0b2070 con 0x7fa0b406ec30 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: dump: 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.967 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 1 -- 172.21.15.119:0/3363434041 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac0a5800 con 0x7fa0b406ec30 2024-09-15T21:23:04.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 5 monclient: authenticate success, global_id 64276 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 4 set_mon_vals no callback set 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 1 -- 172.21.15.119:0/3363434041 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0a5bd0 con 0x7fa0b406ec30 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0b37fe640 10 monclient: dump: 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.969 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 10 monclient: shutdown 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 -- 172.21.15.119:0/3363434041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b406ec30 msgr2=0x7fa0b413cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 --2- 172.21.15.119:0/3363434041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b406ec30 0x7fa0b413cf50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009d90 tx=0x7fa0ac01c830 comp rx=0 tx=0).stop 2024-09-15T21:23:04.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 -- 172.21.15.119:0/3363434041 shutdown_connections 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 --2- 172.21.15.119:0/3363434041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b406ec30 0x7fa0b413cf50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 --2- 172.21.15.119:0/3363434041 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 -- 172.21.15.119:0/3363434041 >> 172.21.15.119:0/3363434041 conn(0x7fa0b40a2ae0 msgr2=0x7fa0b40a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 -- 172.21.15.119:0/3363434041 shutdown_connections 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 1 -- 172.21.15.119:0/3363434041 wait complete. 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 10 monclient: build_initial_monmap 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.951+0000 7fa0babe9640 10 monclient: monmap: 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:04.971 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 Processor -- start 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- start start 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: init 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0b406ec30 0x7fa0b409daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): picked mon.noname-a con 0x7fa0b406ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): picked mon.noname-c con 0x7fa0b40a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 0x7fa0b4133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): picked mon.noname-b con 0x7fa0b4135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b413fe30 con 0x7fa0b406ec30 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b413fcb0 con 0x7fa0b4135ca0 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b413ffb0 con 0x7fa0b40a0160 2024-09-15T21:23:04.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient(hunting): authenticate will time out at 1303.752661s 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b8985640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0b406ec30 msgr2=0x7fa0b409daf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 0x7fa0b4133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b8985640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0b406ec30 0x7fa0b409daf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 0x7fa0b4133b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56888/0 (socket says 172.21.15.119:56888) 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 -- 172.21.15.119:0/4219229056 learned_addr learned my addr 172.21.15.119:0/4219229056 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): get_auth_request con 0x7fa0b4135ca0 auth_method 0 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b3fff640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b3fff640 10 monclient(hunting): get_auth_request con 0x7fa0b40a0160 auth_method 0 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient(hunting): handle_auth_done global_id 64282 payload 824 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 -- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 msgr2=0x7fa0b413b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:04.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 -- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0b406ec30 msgr2=0x7fa0b409daf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0b406ec30 0x7fa0b409daf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient: _finish_hunting 0 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 monclient: found mon.noname-b 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 -- 172.21.15.119:0/4219229056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b409d1d0 con 0x7fa0b4135ca0 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b9186640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 0x7fa0b4133b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0b0040 tx=0x7fa0ac0195d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0b2050 con 0x7fa0b4135ca0 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.976 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: dump: 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.977 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac0be070 con 0x7fa0b4135ca0 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 5 monclient: authenticate success, global_id 64282 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac009450 con 0x7fa0b4135ca0 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _renew_subs 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b409ea50 con 0x7fa0b4135ca0 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b2ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:04.978 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 10 monclient: dump: 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:04.979 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _renew_subs 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b413be60 con 0x7fa0b4135ca0 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4137200 con 0x7fa0b4135ca0 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa0ac0c3460 con 0x7fa0b4135ca0 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa0a003b970 0x7fa0a003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9925+0+0 (secure 0 0 0) 0x7fa0ac0ec580 con 0x7fa0b4135ca0 2024-09-15T21:23:04.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b8985640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa0a003b970 0x7fa0a003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:04.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b8985640 10 monclient: get_auth_request con 0x7fa0a003b970 auth_method 0 2024-09-15T21:23:04.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.955+0000 7fa0b8985640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa0a003b970 0x7fa0a003de30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004820 tx=0x7fa0a4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:04.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.959+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa0b409d1d0 con 0x7fa0b4135ca0 2024-09-15T21:23:04.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.959+0000 7fa0b17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:04.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:04.959+0000 7fa0b17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:05.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0babe9640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0babe9640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0babe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fa0b409f8f0 con 0x7fa0b4135ca0 2024-09-15T21:23:05.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v158) ==== 83+0+2378 (secure 0 0 0) 0x7fa0ac016540 con 0x7fa0b4135ca0 2024-09-15T21:23:05.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0b17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.131+0000 7fa0b17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:05.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.227+0000 7fa0babe9640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.227+0000 7fa0babe9640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.227+0000 7fa0babe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.227+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fa0b40008e0 con 0x7fa0b4135ca0 2024-09-15T21:23:05.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0b17fa640 1 -- 172.21.15.119:0/4219229056 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v158) ==== 83+0+2378 (secure 0 0 0) 0x7fa0ac0e9b70 con 0x7fa0b4135ca0 2024-09-15T21:23:05.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0b17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0b17fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:05.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa0a003b970 msgr2=0x7fa0a003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa0a003b970 0x7fa0a003de30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004820 tx=0x7fa0a4007b50 comp rx=0 tx=0).stop 2024-09-15T21:23:05.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 10 monclient: shutdown 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 msgr2=0x7fa0b4133b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 0x7fa0b4133b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0b0040 tx=0x7fa0ac0195d0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 shutdown_connections 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa0a003b970 0x7fa0a003de30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0b4135ca0 0x7fa0b4133b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0b40a0160 0x7fa0b413b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 --2- 172.21.15.119:0/4219229056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0b406ec30 0x7fa0b409daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 >> 172.21.15.119:0/4219229056 conn(0x7fa0b40a2ae0 msgr2=0x7fa0b40a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 shutdown_connections 2024-09-15T21:23:05.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.231+0000 7fa0babe9640 1 -- 172.21.15.119:0/4219229056 wait complete. 2024-09-15T21:23:05.250 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:05.251 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:05.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:05.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:05.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep globster 2024-09-15T21:23:05.252 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '12\.' 2024-09-15T21:23:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.359+0000 7f0b7a408640 10 monclient: get_monmap_and_config 2024-09-15T21:23:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.359+0000 7f0b7a408640 10 monclient: build_initial_monmap 2024-09-15T21:23:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.359+0000 7f0b7a408640 10 monclient: monmap: 2024-09-15T21:23:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.365 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:05.363881+0000 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:05.363881+0000 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:05.366 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.359+0000 7f0b7a408640 1 Processor -- start 2024-09-15T21:23:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.359+0000 7f0b7a408640 1 -- start start 2024-09-15T21:23:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.359+0000 7f0b7a408640 10 monclient: init 2024-09-15T21:23:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:05.375 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0b7409f640 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b740a0130 0x7f0b7409d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): picked mon.noname-c con 0x7f0b740a0130 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b74139c00 0x7f0b7413bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): picked mon.noname-a con 0x7f0b74139c00 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7409e880 con 0x7f0b7409f640 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7409eb60 con 0x7f0b74139c00 2024-09-15T21:23:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7409ee40 con 0x7f0b740a0130 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient(hunting): authenticate will time out at 1304.160425s 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b740a0130 0x7f0b7409d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b73fff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f0b7409f640 legacy=0x7f0b7409fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b789a5640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f0b74139c00 msgr2=0x7f0b7413bff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b740a0130 0x7f0b7409d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56896/0 (socket says 172.21.15.119:56896) 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b789a5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b74139c00 0x7f0b7413bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 -- 172.21.15.119:0/2599857618 learned_addr learned my addr 172.21.15.119:0/2599857618 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): get_auth_request con 0x7f0b740a0130 auth_method 0 2024-09-15T21:23:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient(hunting): handle_auth_done global_id 64285 payload 274 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 -- 172.21.15.119:0/2599857618 >> v2:172.21.15.17:3300/0 conn(0x7f0b74139c00 msgr2=0x7f0b7413bff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 --2- 172.21.15.119:0/2599857618 >> v2:172.21.15.17:3300/0 conn(0x7f0b74139c00 0x7f0b7413bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 -- 172.21.15.119:0/2599857618 >> v1:172.21.15.17:6789/0 conn(0x7f0b7409f640 legacy=0x7f0b7409fa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 monclient: found mon.noname-c 2024-09-15T21:23:05.378 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 -- 172.21.15.119:0/2599857618 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b740a2840 con 0x7f0b740a0130 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b6bfff640 1 --2- 172.21.15.119:0/2599857618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b740a0130 0x7f0b7409d9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b7409e5a0 tx=0x7f0b5c01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd816e4e82e8192 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 1 -- 172.21.15.119:0/2599857618 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0b0070 con 0x7f0b740a0130 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: dump: 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.379 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:05.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 5 monclient: authenticate success, global_id 64285 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 1 -- 172.21.15.119:0/2599857618 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c0a4e50 con 0x7f0b740a0130 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 1 -- 172.21.15.119:0/2599857618 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0ab820 con 0x7f0b740a0130 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 4 set_mon_vals no callback set 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b737fe640 10 monclient: dump: 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.381 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 10 monclient: shutdown 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:05.382 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- 172.21.15.119:0/2599857618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b740a0130 msgr2=0x7f0b7409d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 --2- 172.21.15.119:0/2599857618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b740a0130 0x7f0b7409d9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b7409e5a0 tx=0x7f0b5c01c400 comp rx=0 tx=0).stop 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- 172.21.15.119:0/2599857618 shutdown_connections 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 --2- 172.21.15.119:0/2599857618 >> v2:172.21.15.17:3300/0 conn(0x7f0b74139c00 0x7f0b7413bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 --2- 172.21.15.119:0/2599857618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b740a0130 0x7f0b7409d9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- 172.21.15.119:0/2599857618 >> 172.21.15.119:0/2599857618 conn(0x7f0b740a2ae0 msgr2=0x7f0b740a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- 172.21.15.119:0/2599857618 shutdown_connections 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.363+0000 7f0b7a408640 1 -- 172.21.15.119:0/2599857618 wait complete. 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: build_initial_monmap 2024-09-15T21:23:05.383 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: monmap: 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:05.384 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 Processor -- start 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- start start 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: init 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b7409f640 0x7f0b7411edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): picked mon.noname-c con 0x7f0b7409f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740a0130 0x7f0b7411cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.385 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): picked mon.noname-a con 0x7f0b740a0130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 0x7f0b74134c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): picked mon.noname-b con 0x7f0b74139c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7413eed0 con 0x7f0b740a0130 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7413ed50 con 0x7f0b74139c00 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7413f050 con 0x7f0b7409f640 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient(hunting): authenticate will time out at 1304.165085s 2024-09-15T21:23:05.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 0x7f0b74134c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b6bfff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740a0130 msgr2=0x7f0b7411cdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 0x7f0b74134c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56898/0 (socket says 172.21.15.119:56898) 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 -- 172.21.15.119:0/1237959226 learned_addr learned my addr 172.21.15.119:0/1237959226 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): get_auth_request con 0x7f0b74139c00 auth_method 0 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b6bfff640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740a0130 0x7f0b7411cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:05.387 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b73fff640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b7409f640 0x7f0b7411edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient(hunting): handle_auth_done global_id 64288 payload 824 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 -- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b7409f640 msgr2=0x7f0b7411edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b7409f640 0x7f0b7411edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 -- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740a0130 msgr2=0x7f0b7411cdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740a0130 0x7f0b7411cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient: _finish_hunting 0 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 monclient: found mon.noname-b 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.388 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 -- 172.21.15.119:0/1237959226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7411e260 con 0x7f0b74139c00 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b73fff640 10 monclient: get_auth_request con 0x7f0b7409f640 auth_method 0 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b73fff640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b7409f640 0x7f0b7411edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b789a5640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 0x7f0b74134c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00be60 tx=0x7f0b6c00bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c015470 con 0x7f0b74139c00 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: dump: 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.389 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6c010070 con 0x7f0b74139c00 2024-09-15T21:23:05.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 5 monclient: authenticate success, global_id 64288 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _renew_subs 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7411fd20 con 0x7f0b74139c00 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c015890 con 0x7f0b74139c00 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 10 monclient: dump: 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.391 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b72ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _renew_subs 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b74135160 con 0x7f0b74139c00 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7409d9a0 con 0x7f0b74139c00 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0b6c01e3f0 con 0x7f0b74139c00 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b5403b970 0x7f0b5403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9925+0+0 (secure 0 0 0) 0x7f0b6c051690 con 0x7f0b74139c00 2024-09-15T21:23:05.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b73fff640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b5403b970 0x7f0b5403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b73fff640 10 monclient: get_auth_request con 0x7f0b5403b970 auth_method 0 2024-09-15T21:23:05.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.367+0000 7f0b73fff640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b5403b970 0x7f0b5403de30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b741ce340 tx=0x7f0b64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:05.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.371+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0b6c04ec80 con 0x7f0b74139c00 2024-09-15T21:23:05.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.371+0000 7f0b717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:05.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.371+0000 7f0b717fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:05.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b7a408640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b7a408640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b7a408640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f0b741200b0 con 0x7f0b74139c00 2024-09-15T21:23:05.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v158) ==== 83+0+2378 (secure 0 0 0) 0x7f0b6c005070 con 0x7f0b74139c00 2024-09-15T21:23:05.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.551+0000 7f0b717fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f0b74135420 con 0x7f0b74139c00 2024-09-15T21:23:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b717fa640 1 -- 172.21.15.119:0/1237959226 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v158) ==== 83+0+2378 (secure 0 0 0) 0x7f0b6c013c90 con 0x7f0b74139c00 2024-09-15T21:23:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:05.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b717fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:05.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b5403b970 msgr2=0x7f0b5403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b5403b970 0x7f0b5403de30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b741ce340 tx=0x7f0b64008040 comp rx=0 tx=0).stop 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 10 monclient: shutdown 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 msgr2=0x7f0b74134c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 0x7f0b74134c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00be60 tx=0x7f0b6c00bf50 comp rx=0 tx=0).stop 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 shutdown_connections 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b5403b970 0x7f0b5403de30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b74139c00 0x7f0b74134c20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740a0130 0x7f0b7411cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 --2- 172.21.15.119:0/1237959226 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b7409f640 0x7f0b7411edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 >> 172.21.15.119:0/1237959226 conn(0x7f0b740a2ae0 msgr2=0x7f0b740a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 shutdown_connections 2024-09-15T21:23:05.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.659+0000 7f0b7a408640 1 -- 172.21.15.119:0/1237959226 wait complete. 2024-09-15T21:23:05.680 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:05.680 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-09-15T21:23:05.680 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight-compat osd.0 7.5 2024-09-15T21:23:05.794 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient: get_monmap_and_config 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient: build_initial_monmap 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient: monmap: 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:05.793594+0000 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:05.793594+0000 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:05.795 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:05.796 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:05.796 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 1 Processor -- start 2024-09-15T21:23:05.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 1 -- start start 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient: init 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f64e0136b90 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f64e00a10d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64e00a25b0 0x7f64e0140380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient(hunting): picked mon.noname-a con 0x7f64e00a25b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00a18f0 con 0x7f64e0136b90 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00a3660 con 0x7f64e00a10d0 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f64e009fc40 con 0x7f64e00a25b0 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:05.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 10 monclient(hunting): authenticate will time out at 1304.590285s 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64df7fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f64e0136b90 legacy=0x7f64e0136f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f64e00a25b0 msgr2=0x7f64e0140380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f64e00a10d0 0x7f64e013cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57110/0 (socket says 172.21.15.119:57110) 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 -- 172.21.15.119:0/1856649461 learned_addr learned my addr 172.21.15.119:0/1856649461 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 1 --2- 172.21.15.119:0/1856649461 >> v2:172.21.15.17:3300/0 conn(0x7f64e00a25b0 0x7f64e0140380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192546704 0 0) 0x7f64e00a3660 con 0x7f64e00a10d0 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): my global_id is 64291 2024-09-15T21:23:05.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64c8003530 con 0x7f64e00a10d0 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1302229863 0 0) 0x7f64c8003530 con 0x7f64e00a10d0 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 >> v2:172.21.15.17:3300/0 conn(0x7f64e00a25b0 msgr2=0x7f64e0140380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 --2- 172.21.15.119:0/1856649461 >> v2:172.21.15.17:3300/0 conn(0x7f64e00a25b0 0x7f64e0140380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 >> v1:172.21.15.17:6789/0 conn(0x7f64e0136b90 legacy=0x7f64e0136f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e0137680 con 0x7f64e00a10d0 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 5 monclient: authenticate success, global_id 64291 2024-09-15T21:23:05.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f64d0001700 con 0x7f64e00a10d0 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: dump: 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.802 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 >> v1:172.21.15.119:6789/0 conn(0x7f64e00a10d0 legacy=0x7f64e013cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64c8003a10 0x7f64c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): picked mon.a con 0x7f64c8003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): picked mon.b con 0x7f64c8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64c8005420 0x7f64c800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): picked mon.c con 0x7f64c8005420 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f64c800c330 con 0x7f64c8003a10 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f64c800c4e0 con 0x7f64c8004810 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f64c800c6e0 con 0x7f64c8005420 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64de7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64e5bc8640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64df7fe640 1 -- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64c8003a10 msgr2=0x7f64c8003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64df7fe640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64c8003a10 0x7f64c8003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): get_auth_request con 0x7f64c8004810 auth_method 0 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:05.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64c8005420 0x7f64c800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): get_auth_request con 0x7f64c8005420 auth_method 0 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:05.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient(hunting): handle_auth_done global_id 64291 payload 293 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 -- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64c8005420 msgr2=0x7f64c800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64c8005420 0x7f64c800ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 -- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64c8003a10 msgr2=0x7f64c8003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64c8003a10 0x7f64c8003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 monclient: found mon.b 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 1 -- 172.21.15.119:0/1856649461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c8003530 con 0x7f64c8004810 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:05.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.791+0000 7f64dffff640 1 -- 172.21.15.119:0/1856649461 reap_dead start 2024-09-15T21:23:05.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64deffd640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64c8004c90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64d000c190 tx=0x7f64d001caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0eb6e579db0b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:05.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0007c70 con 0x7f64c8004810 2024-09-15T21:23:05.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:05.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: dump: 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d00a9020 con 0x7f64c8004810 2024-09-15T21:23:05.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 4 set_mon_vals no callback set 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 1 -- 172.21.15.119:0/1856649461 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0005ae0 con 0x7f64c8004810 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64de7fc640 10 monclient: dump: 2024-09-15T21:23:05.813 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.814 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 10 monclient: shutdown 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 -- 172.21.15.119:0/1856649461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 msgr2=0x7f64c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64c8004c90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64d000c190 tx=0x7f64d001caa0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 -- 172.21.15.119:0/1856649461 shutdown_connections 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/1856649461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64c8004c90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 -- 172.21.15.119:0/1856649461 >> 172.21.15.119:0/1856649461 conn(0x7f64e00a66e0 msgr2=0x7f64e00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 -- 172.21.15.119:0/1856649461 shutdown_connections 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 -- 172.21.15.119:0/1856649461 wait complete. 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 10 monclient: build_initial_monmap 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 10 monclient: monmap: 2024-09-15T21:23:05.815 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 Processor -- start 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 1 -- start start 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.795+0000 7f64e5bc8640 10 monclient: init 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:05.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64e01cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): picked mon.noname-b con 0x7f64c8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64e0135900 0x7f64e0135d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): picked mon.noname-a con 0x7f64e0135900 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64e011f4a0 0x7f64e011f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): picked mon.noname-c con 0x7f64e011f4a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01382c0 con 0x7f64e0135900 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0138140 con 0x7f64c8004810 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f64e0138440 con 0x7f64e011f4a0 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient(hunting): authenticate will time out at 1304.596233s 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64e01cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64e01cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56926/0 (socket says 172.21.15.119:56926) 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 -- 172.21.15.119:0/3908606530 learned_addr learned my addr 172.21.15.119:0/3908606530 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): get_auth_request con 0x7f64c8004810 auth_method 0 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:05.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64deffd640 1 -- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64e0135900 msgr2=0x7f64e0135d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64deffd640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64e0135900 0x7f64e0135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient(hunting): handle_auth_done global_id 64294 payload 824 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 -- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64e011f4a0 msgr2=0x7f64e011f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64e011f4a0 0x7f64e011f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 -- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64e0135900 msgr2=0x7f64e0135d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64e0135900 0x7f64e0135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:05.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 monclient: found mon.noname-b 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 -- 172.21.15.119:0/3908606530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e0122a20 con 0x7f64c8004810 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64df7fe640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64e01cf480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b530 tx=0x7f64d400b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003bd0 con 0x7f64c8004810 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: dump: 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.820 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d4015070 con 0x7f64c8004810 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 5 monclient: authenticate success, global_id 64294 2024-09-15T21:23:05.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4005070 con 0x7f64c8004810 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _renew_subs 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e01cf9c0 con 0x7f64c8004810 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64dd7fa640 4 set_mon_vals no callback set 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 10 monclient: dump: 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:05.822 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _renew_subs 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e0122d00 con 0x7f64c8004810 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e013ac40 con 0x7f64c8004810 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f64d4005210 con 0x7f64c8004810 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f64cc03b9a0 0x7f64cc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:05.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9925+0+0 (secure 0 0 0) 0x7f64d4050980 con 0x7f64c8004810 2024-09-15T21:23:05.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64deffd640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f64cc03b9a0 0x7f64cc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:05.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64deffd640 10 monclient: get_auth_request con 0x7f64cc03b9a0 auth_method 0 2024-09-15T21:23:05.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.799+0000 7f64deffd640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f64cc03b9a0 0x7f64cc03de60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f64d00aa000 tx=0x7f64e00a3830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:05.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.803+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f64e0122a20 con 0x7f64c8004810 2024-09-15T21:23:05.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.803+0000 7f64c7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:05.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.803+0000 7f64c7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:05.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.979+0000 7f64e5bc8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T21:23:05.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.979+0000 7f64e5bc8640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T21:23:05.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.979+0000 7f64e5bc8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:05.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:05.979+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) -- 0x7f64e006a830 con 0x7f64c8004810 2024-09-15T21:23:06.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.495+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v159) ==== 118+0+0 (secure 0 0 0) 0x7f64d400f380 con 0x7f64c8004810 2024-09-15T21:23:06.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.495+0000 7f64c7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T21:23:06.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.495+0000 7f64c7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:06.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.607+0000 7f64e5bc8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T21:23:06.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.607+0000 7f64e5bc8640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T21:23:06.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.607+0000 7f64e5bc8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:06.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:06.607+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) -- 0x7f64e011fe50 con 0x7f64c8004810 2024-09-15T21:23:07.132 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:23:07.132 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:23:07.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:23:07.135 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:23:07.135 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T21:23:07.135 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T21:23:07.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:23:08.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:08.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:08.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:08.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:08.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:08.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:08.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:08.799+0000 7f64c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:09.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:09.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:09.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:09.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:09.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:09.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:10.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:10.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:10.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:10.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:10.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:10.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:11.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:11.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:11.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:11.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:11.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:11.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:12.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:12.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:12.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:12.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:12.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:12.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:13.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:13.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:13.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:13.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:13.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:13.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:14.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:14.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:14.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:14.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:14.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:14.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:15.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:15.799+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:15.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:15.799+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:15.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:15.799+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:16.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:16.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:16.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:16.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:16.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:16.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:17.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:17.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:17.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:17.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:17.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:17.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:18.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:18.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:18.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:18.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:18.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:18.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:18.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:18.798+0000 7f64c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:19.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:19.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:19.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:19.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:19.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:19.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:20.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:20.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:20.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:20.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:20.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:20.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:21.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:21.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:21.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:21.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:21.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:21.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:22.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:22.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:22.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:22.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:22.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:22.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:23.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:23.802+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:23.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:23.802+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:23.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:23.802+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:24.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:24.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:24.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:24.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:25.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:25.802+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:25.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:25.802+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:25.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:25.802+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:26.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:26.798+0000 7f64c6ffd640 10 monclient: tick 2024-09-15T21:23:26.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:26.798+0000 7f64c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:26.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:26.798+0000 7f64c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:27.136 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T21:23:27.136 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T21:23:27.137 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:23:27.137 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:23:27.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient: get_monmap_and_config 2024-09-15T21:23:27.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient: build_initial_monmap 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient: monmap: 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:27.267886+0000 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:27.267886+0000 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:27.275 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:27.275 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:27.275 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 Processor -- start 2024-09-15T21:23:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 -- start start 2024-09-15T21:23:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient: init 2024-09-15T21:23:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): picked mon.noname-a con 0x7f5a00073640 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): picked mon.noname-c con 0x7f5a00073f60 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5a00137bd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a000a0890 con 0x7f5a00137bd0 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5a000746e0 con 0x7f5a00073640 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5a000a1640 con 0x7f5a00073f60 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f5a04c43640 10 monclient(hunting): authenticate will time out at 1330.663296s 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57296/0 (socket says 172.21.15.17:57296) 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 -- 172.21.15.17:0/3409694179 learned_addr learned my addr 172.21.15.17:0/3409694179 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 10 monclient(hunting): get_auth_request con 0x7f5a00073640 auth_method 0 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 msgr2=0x7f5a00073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 msgr2=0x7f5a00073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request con 0x7f5a00073f60 auth_method 0 2024-09-15T21:23:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.264+0000 7f59fdd9b640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 msgr2=0x7f5a00137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.268+0000 7f59fdd9b640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 msgr2=0x7f5a00137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.268+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.268+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:27.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 10 monclient(hunting): get_auth_request con 0x7f5a00073640 auth_method 0 2024-09-15T21:23:27.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:27.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 msgr2=0x7f5a00073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:27.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 msgr2=0x7f5a00073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:27.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:27.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.468+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:23:27.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request con 0x7f5a00073f60 auth_method 0 2024-09-15T21:23:27.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:27.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 msgr2=0x7f5a00137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:27.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 msgr2=0x7f5a00137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:27.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:27.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.472+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:23:27.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.604+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110921902 0 0) 0x7f5a000a0890 con 0x7f5a00137bd0 2024-09-15T21:23:27.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.604+0000 7f59fd59a640 10 monclient(hunting): _negotiate 2024-09-15T21:23:27.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.604+0000 7f59fd59a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.604+0000 7f59fd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.604+0000 7f59fd59a640 10 monclient(hunting): my global_id is 64297 2024-09-15T21:23:27.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.604+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59e8003530 con 0x7f5a00137bd0 2024-09-15T21:23:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1771869757 0 0) 0x7f59e8003530 con 0x7f5a00137bd0 2024-09-15T21:23:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 msgr2=0x7f5a00137690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.119:3300/0 conn(0x7f5a00073f60 0x7f5a00137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 msgr2=0x7f5a00073a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 --2- 172.21.15.17:0/3409694179 >> v2:172.21.15.17:3300/0 conn(0x7f5a00073640 0x7f5a00073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: _finish_hunting 0 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000a1920 con 0x7f5a00137bd0 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: _finish_auth 0 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: _check_auth_tickets 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f59f8001aa0 con 0x7f5a00137bd0 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f5a04c43640 5 monclient: authenticate success, global_id 64297 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f5a04c43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: dump: 2024-09-15T21:23:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 >> v1:172.21.15.119:6789/0 conn(0x7f5a00137bd0 legacy=0x7f5a0013ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:23:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59e8003a60 0x7f59e8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): picked mon.a con 0x7f59e8003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e8004810 0x7f59e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): picked mon.b con 0x7f59e8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f59e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): picked mon.c con 0x7f59e80052d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000746e0 con 0x7f59e8003a60 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000a1640 con 0x7f59e8004810 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f59e800c290 con 0x7f59e80052d0 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fd59a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f5a04c43640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fed9d640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f59e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59e8003a60 0x7f59e8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fe59c640 10 monclient(hunting): get_auth_request con 0x7f59e8003a60 auth_method 0 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fe59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fed9d640 10 monclient(hunting): get_auth_request con 0x7f59e80052d0 auth_method 0 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.616+0000 7f59fed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e8004810 0x7f59e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fe59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request con 0x7f59e8004810 auth_method 0 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient(hunting): handle_auth_done global_id 64297 payload 293 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 1 -- 172.21.15.17:0/3409694179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e8004810 msgr2=0x7f59e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e8004810 0x7f59e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 1 -- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59e8003a60 msgr2=0x7f59e8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59e8003a60 0x7f59e8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 1 monclient: found mon.c 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 1 -- 172.21.15.17:0/3409694179 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e8003530 con 0x7f59e80052d0 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient: _finish_auth 0 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 10 monclient: _check_auth_tickets 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fed9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.620+0000 7f59fe59c640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59e8003a60 0x7f59e8003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fed9d640 1 -- 172.21.15.17:0/3409694179 reap_dead start 2024-09-15T21:23:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fed9d640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f59e800b9d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59f8002240 tx=0x7f59f801ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f404a4ed69c10197 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f000f040 con 0x7f59e80052d0 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: dump: 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f80a5d90 con 0x7f59e80052d0 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 4 set_mon_vals no callback set 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 1 -- 172.21.15.17:0/3409694179 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f800c100 con 0x7f59e80052d0 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f59fd59a640 10 monclient: dump: 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 10 monclient: shutdown 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 -- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 msgr2=0x7f59e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f59e800b9d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59f8002240 tx=0x7f59f801ce30 comp rx=0 tx=0).stop 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 -- 172.21.15.17:0/3409694179 shutdown_connections 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 --2- 172.21.15.17:0/3409694179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f59e800b9d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 -- 172.21.15.17:0/3409694179 >> 172.21.15.17:0/3409694179 conn(0x7f5a000a66a0 msgr2=0x7f5a000a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 -- 172.21.15.17:0/3409694179 shutdown_connections 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.624+0000 7f5a04c43640 1 -- 172.21.15.17:0/3409694179 wait complete. 2024-09-15T21:23:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient: build_initial_monmap 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient: monmap: 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 Processor -- start 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 -- start start 2024-09-15T21:23:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient: init 2024-09-15T21:23:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f5a0011f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): picked mon.noname-c con 0x7f59e80052d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a00135e80 0x7f5a0011c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): picked mon.noname-b con 0x7f5a00135e80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a001362e0 0x7f5a0011cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): picked mon.noname-a con 0x7f5a001362e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0009ce50 con 0x7f5a001362e0 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0009ccd0 con 0x7f5a00135e80 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5a000a0d00 con 0x7f59e80052d0 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f5a04c43640 10 monclient(hunting): authenticate will time out at 1331.026273s 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f5a0011f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f5a0011f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55786/0 (socket says 172.21.15.17:55786) 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 -- 172.21.15.17:0/903203805 learned_addr learned my addr 172.21.15.17:0/903203805 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): get_auth_request con 0x7f59e80052d0 auth_method 0 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a00135e80 0x7f5a0011c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a001362e0 0x7f5a0011cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): get_auth_request con 0x7f5a001362e0 auth_method 0 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request con 0x7f5a00135e80 auth_method 0 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient(hunting): handle_auth_done global_id 44699 payload 824 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 -- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a00135e80 msgr2=0x7f5a0011c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a00135e80 0x7f5a0011c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 -- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a001362e0 msgr2=0x7f5a0011cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a001362e0 0x7f5a0011cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient: _finish_hunting 0 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 monclient: found mon.noname-c 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 -- 172.21.15.17:0/903203805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0011f9a0 con 0x7f59e80052d0 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.628+0000 7f59fe59c640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f5a0011f010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e590 tx=0x7f59f000ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f001b070 con 0x7f59e80052d0 2024-09-15T21:23:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: dump: 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f0012900 con 0x7f59e80052d0 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 5 monclient: authenticate success, global_id 44699 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 10 monclient: _renew_subs 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a00120810 con 0x7f59e80052d0 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0012cd0 con 0x7f59e80052d0 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59eeffd640 10 monclient: dump: 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.646 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.647 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:27.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f59fcd99640 4 set_mon_vals no callback set 2024-09-15T21:23:27.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 10 monclient: _renew_subs 2024-09-15T21:23:27.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.632+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0009d0f0 con 0x7f59e80052d0 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f59f001e070 con 0x7f59e80052d0 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f59eeffd640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59e003b9c0 0x7f59e003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f5a04c43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f5a04c43640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f5a04c43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(161..161 src has 1..161) ==== 9863+0+0 (secure 0 0 0) 0x7f59f0050f40 con 0x7f59e80052d0 2024-09-15T21:23:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a00073a20 con 0x7f59e80052d0 2024-09-15T21:23:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59e003b9c0 0x7f59e003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f59fdd9b640 10 monclient: get_auth_request con 0x7f59e003b9c0 auth_method 0 2024-09-15T21:23:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.636+0000 7f59fdd9b640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59e003b9c0 0x7f59e003de80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5a00137060 tx=0x7f59f400e1f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.640+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f59f0025030 con 0x7f59e80052d0 2024-09-15T21:23:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.640+0000 7f59eeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.640+0000 7f59eeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.658+0000 7f64c7fff640 1 -- 172.21.15.119:0/3908606530 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v162) ==== 118+0+0 (secure 0 0 0) 0x7f64d4040070 con 0x7f64c8004810 2024-09-15T21:23:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.658+0000 7f64c7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T21:23:27.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.658+0000 7f64c7fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:27.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f64cc03b9a0 msgr2=0x7f64cc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f64cc03b9a0 0x7f64cc03de60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f64d00aa000 tx=0x7f64e00a3830 comp rx=0 tx=0).stop 2024-09-15T21:23:27.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 10 monclient: shutdown 2024-09-15T21:23:27.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:27.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 msgr2=0x7f64e01cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64e01cf480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b530 tx=0x7f64d400b9f0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 shutdown_connections 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f64cc03b9a0 0x7f64cc03de60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f64e011f4a0 0x7f64e011f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64e0135900 0x7f64e0135d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 --2- 172.21.15.119:0/3908606530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64c8004810 0x7f64e01cf480 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 >> 172.21.15.119:0/3908606530 conn(0x7f64e00a66e0 msgr2=0x7f64e00738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 shutdown_connections 2024-09-15T21:23:27.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.662+0000 7f64e5bc8640 1 -- 172.21.15.119:0/3908606530 wait complete. 2024-09-15T21:23:27.684 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight cool osd.0 8.5 2024-09-15T21:23:27.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient: get_monmap_and_config 2024-09-15T21:23:27.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient: build_initial_monmap 2024-09-15T21:23:27.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient: monmap: 2024-09-15T21:23:27.804 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:27.804 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.804 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:27.802395+0000 2024-09-15T21:23:27.804 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:27.802395+0000 2024-09-15T21:23:27.804 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:27.804 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:27.805 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:27.805 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:27.805 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:27.805 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:27.805 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:27.805 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 Processor -- start 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 -- start start 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient: init 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f135c135bf0 0x7f135c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): picked mon.noname-c con 0x7f135c135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f135c135fd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f135c136460 0x7f135c0a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): picked mon.noname-a con 0x7f135c136460 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135c0a1d80 con 0x7f135c135fd0 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f135c0a3600 con 0x7f135c136460 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f135c0a2060 con 0x7f135c135bf0 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f1362a6f640 10 monclient(hunting): authenticate will time out at 1326.598984s 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f135c135bf0 0x7f135c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f135bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f135c135fd0 0x7f135c0a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49932/0 (socket says 172.21.15.119:49932) 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f135bfff640 1 -- 172.21.15.119:0/2102261667 learned_addr learned my addr 172.21.15.119:0/2102261667 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): get_auth_request con 0x7f135c135bf0 auth_method 0 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 1 --2- 172.21.15.119:0/2102261667 >> v2:172.21.15.17:3300/0 conn(0x7f135c136460 0x7f135c0a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): get_auth_request con 0x7f135c136460 auth_method 0 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136100c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient(hunting): handle_auth_done global_id 64303 payload 274 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 1 -- 172.21.15.119:0/2102261667 >> v2:172.21.15.17:3300/0 conn(0x7f135c136460 msgr2=0x7f135c0a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 1 --2- 172.21.15.119:0/2102261667 >> v2:172.21.15.17:3300/0 conn(0x7f135c136460 0x7f135c0a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 1 -- 172.21.15.119:0/2102261667 >> v1:172.21.15.119:6789/0 conn(0x7f135c135fd0 legacy=0x7f135c0a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f135b7fe640 1 -- 172.21.15.119:0/2102261667 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094024255 0 0) 0x7f135c0a1d80 con 0x7f135c135fd0 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 1 monclient: found mon.noname-c 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f136080b640 1 -- 172.21.15.119:0/2102261667 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c0a2340 con 0x7f135c135bf0 2024-09-15T21:23:27.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.798+0000 7f135b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:23:27.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f136080b640 1 --2- 172.21.15.119:0/2102261667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135bf0 0x7f135c09fe70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13500097c0 tx=0x7f135001c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f302d19cb0177824 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:27.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 1 -- 172.21.15.119:0/2102261667 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500b0070 con 0x7f135c135bf0 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: dump: 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.817 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 1 -- 172.21.15.119:0/2102261667 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13500ab600 con 0x7f135c135bf0 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 5 monclient: authenticate success, global_id 64303 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 1 -- 172.21.15.119:0/2102261667 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500ab9d0 con 0x7f135c135bf0 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 4 set_mon_vals no callback set 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:27.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135b7fe640 10 monclient: dump: 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: shutdown 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- 172.21.15.119:0/2102261667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135bf0 msgr2=0x7f135c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2102261667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135bf0 0x7f135c09fe70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13500097c0 tx=0x7f135001c320 comp rx=0 tx=0).stop 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- 172.21.15.119:0/2102261667 shutdown_connections 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2102261667 >> v2:172.21.15.17:3300/0 conn(0x7f135c136460 0x7f135c0a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2102261667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135bf0 0x7f135c09fe70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- 172.21.15.119:0/2102261667 >> 172.21.15.119:0/2102261667 conn(0x7f135c0a66e0 msgr2=0x7f135c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- 172.21.15.119:0/2102261667 shutdown_connections 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- 172.21.15.119:0/2102261667 wait complete. 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: build_initial_monmap 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: monmap: 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:27.820 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 Processor -- start 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- start start 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: init 2024-09-15T21:23:27.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135c135bf0 0x7f135c13c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): picked mon.noname-a con 0x7f135c135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 0x7f135c1cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): picked mon.noname-b con 0x7f135c135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f135c136460 0x7f135c11c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): picked mon.noname-c con 0x7f135c136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f135c0a1820 con 0x7f135c135bf0 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f135c0a16a0 con 0x7f135c135fd0 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:27.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f135c1402b0 con 0x7f135c136460 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f1362a6f640 10 monclient(hunting): authenticate will time out at 1326.607645s 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 0x7f135c1cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 0x7f135c1cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45638/0 (socket says 172.21.15.119:45638) 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 1 -- 172.21.15.119:0/2072293131 learned_addr learned my addr 172.21.15.119:0/2072293131 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): get_auth_request con 0x7f135c135fd0 auth_method 0 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f136100c640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f135c136460 0x7f135c11c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f135bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f136100c640 10 monclient(hunting): get_auth_request con 0x7f135c136460 auth_method 0 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.806+0000 7f136080b640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135c135bf0 0x7f135c13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136100c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136100c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136100c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 10 monclient(hunting): get_auth_request con 0x7f135c135bf0 auth_method 0 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136100c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:27.824 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136100c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136100c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 10 monclient(hunting): handle_auth_done global_id 64309 payload 824 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 -- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f135c136460 msgr2=0x7f135c11c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f135c136460 0x7f135c11c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 -- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135c135bf0 msgr2=0x7f135c13c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135c135bf0 0x7f135c13c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 monclient: found mon.noname-b 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:27.825 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 -- 172.21.15.119:0/2072293131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c11e2a0 con 0x7f135c135fd0 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135c135bf0 0x7f135c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135bfff640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 0x7f135c1cda40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c450 tx=0x7f134c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c018070 con 0x7f135c135fd0 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: dump: 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:27.826 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 5 monclient: authenticate success, global_id 64309 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c0074e0 con 0x7f135c135fd0 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 monclient: _renew_subs 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c11f110 con 0x7f135c135fd0 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:27.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c004a30 con 0x7f135c135fd0 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 10 monclient: dump: 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.828 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:27.829 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:27.829 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:27.829 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:27.829 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:27.829 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 monclient: _renew_subs 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c134da0 con 0x7f135c135fd0 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f135affd640 4 set_mon_vals no callback set 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:27.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135c09fe70 con 0x7f135c135fd0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f134c01b070 con 0x7f135c135fd0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f134803b9c0 0x7f134803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9921+0+0 (secure 0 0 0) 0x7f134c050200 con 0x7f135c135fd0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f134803b9c0 0x7f134803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 10 monclient: get_auth_request con 0x7f134803b9c0 auth_method 0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.810+0000 7f136080b640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f134803b9c0 0x7f134803de80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f135c0a7110 tx=0x7f13500a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:27.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.814+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f134c010300 con 0x7f135c135fd0 2024-09-15T21:23:27.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.814+0000 7f13597fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:27.832 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.814+0000 7f13597fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:27.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f5a04c43640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:23:27.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f5a04c43640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:27.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f5a04c43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:27.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5a0006d020 con 0x7f59e80052d0 2024-09-15T21:23:27.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f59eeffd640 1 -- 172.21.15.17:0/903203805 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f59f004e510 con 0x7f59e80052d0 2024-09-15T21:23:27.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f59eeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:27.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.888+0000 7f59eeffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:27.895 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:23:27.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59e003b9c0 msgr2=0x7f59e003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59e003b9c0 0x7f59e003de80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5a00137060 tx=0x7f59f400e1f0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 10 monclient: shutdown 2024-09-15T21:23:27.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:27.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 msgr2=0x7f5a0011f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:27.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f5a0011f010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e590 tx=0x7f59f000ea50 comp rx=0 tx=0).stop 2024-09-15T21:23:27.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 shutdown_connections 2024-09-15T21:23:27.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59e003b9c0 0x7f59e003de80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a001362e0 0x7f5a0011cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a00135e80 0x7f5a0011c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 --2- 172.21.15.17:0/903203805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59e80052d0 0x7f5a0011f010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:27.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 >> 172.21.15.17:0/903203805 conn(0x7f5a000a66a0 msgr2=0x7f5a000a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:27.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 shutdown_connections 2024-09-15T21:23:27.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:27.892+0000 7f5a04c43640 1 -- 172.21.15.17:0/903203805 wait complete. 2024-09-15T21:23:27.913 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:23:27.914 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:23:27.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:23:28.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.998+0000 7f1362a6f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T21:23:28.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.998+0000 7f1362a6f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T21:23:28.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.998+0000 7f1362a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:28.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:27.998+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) -- 0x7f135c06a830 con 0x7f135c135fd0 2024-09-15T21:23:28.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient: get_monmap_and_config 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient: monmap: 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:28.038684+0000 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:28.038684+0000 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:28.041 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:28.041 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 Processor -- start 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 -- start start 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient: init 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f85401378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85401383b0 0x7f854013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): picked mon.noname-c con 0x7f85401383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 0x7f854013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): picked mon.noname-b con 0x7f854013d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85400a1a20 con 0x7f85401378c0 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f85400a1d00 con 0x7f85401383b0 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f85400a1fe0 con 0x7f854013d2c0 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f8547241640 10 monclient(hunting): authenticate will time out at 1331.434015s 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 0x7f854013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 0x7f854013f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55792/0 (socket says 172.21.15.17:55792) 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 -- 172.21.15.17:0/645483302 learned_addr learned my addr 172.21.15.17:0/645483302 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): get_auth_request con 0x7f854013d2c0 auth_method 0 2024-09-15T21:23:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f853ffff640 1 --2- 172.21.15.17:0/645483302 >> v2:172.21.15.119:3300/0 conn(0x7f85401383b0 0x7f854013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f853ffff640 10 monclient(hunting): get_auth_request con 0x7f85401383b0 auth_method 0 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f853ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f853ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f853ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient(hunting): handle_auth_done global_id 44705 payload 274 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 -- 172.21.15.17:0/645483302 >> v2:172.21.15.119:3300/0 conn(0x7f85401383b0 msgr2=0x7f854013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 --2- 172.21.15.17:0/645483302 >> v2:172.21.15.119:3300/0 conn(0x7f85401383b0 0x7f854013cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 -- 172.21.15.17:0/645483302 >> v1:172.21.15.119:6789/0 conn(0x7f85401378c0 legacy=0x7f8540137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 monclient: found mon.noname-b 2024-09-15T21:23:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.036+0000 7f85457de640 1 -- 172.21.15.17:0/645483302 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85400a22c0 con 0x7f854013d2c0 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f85457de640 1 --2- 172.21.15.17:0/645483302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 0x7f854013f6b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8538009e80 tx=0x7f853801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1424d238ac73e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 1 -- 172.21.15.17:0/645483302 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380a4a10 con 0x7f854013d2c0 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: dump: 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.054 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 1 -- 172.21.15.17:0/645483302 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380a4bb0 con 0x7f854013d2c0 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 5 monclient: authenticate success, global_id 44705 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 1 -- 172.21.15.17:0/645483302 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380ac3f0 con 0x7f854013d2c0 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 4 set_mon_vals no callback set 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853f7fe640 10 monclient: dump: 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: shutdown 2024-09-15T21:23:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/645483302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 msgr2=0x7f854013f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 --2- 172.21.15.17:0/645483302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 0x7f854013f6b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8538009e80 tx=0x7f853801c3d0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/645483302 shutdown_connections 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 --2- 172.21.15.17:0/645483302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f854013d2c0 0x7f854013f6b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 --2- 172.21.15.17:0/645483302 >> v2:172.21.15.119:3300/0 conn(0x7f85401383b0 0x7f854013cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/645483302 >> 172.21.15.17:0/645483302 conn(0x7f85400aac40 msgr2=0x7f85400a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/645483302 shutdown_connections 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/645483302 wait complete. 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: monmap: 2024-09-15T21:23:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 Processor -- start 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- start start 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: init 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85401378c0 0x7f854013c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): picked mon.noname-a con 0x7f85401378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85401383b0 0x7f85401d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): picked mon.noname-c con 0x7f85401383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f854013d2c0 0x7f85401d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): picked mon.noname-b con 0x7f854013d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8540143130 con 0x7f85401378c0 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8540142fb0 con 0x7f854013d2c0 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f85401432b0 con 0x7f85401383b0 2024-09-15T21:23:28.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient(hunting): authenticate will time out at 1331.438948s 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85401383b0 0x7f85401d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85401378c0 0x7f854013c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85401383b0 0x7f85401d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55794/0 (socket says 172.21.15.17:55794) 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 -- 172.21.15.17:0/3390395725 learned_addr learned my addr 172.21.15.17:0/3390395725 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): get_auth_request con 0x7f85401383b0 auth_method 0 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): get_auth_request con 0x7f85401378c0 auth_method 0 2024-09-15T21:23:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f85457de640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f854013d2c0 0x7f85401d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f85457de640 10 monclient(hunting): get_auth_request con 0x7f854013d2c0 auth_method 0 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f85457de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f85457de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f85457de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient(hunting): handle_auth_done global_id 44708 payload 824 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f854013d2c0 msgr2=0x7f85401d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f854013d2c0 0x7f85401d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85401378c0 msgr2=0x7f854013c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85401378c0 0x7f854013c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 monclient: found mon.noname-c 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854009e770 con 0x7f85401383b0 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853ffff640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85401383b0 0x7f85401d0c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f853400d830 tx=0x7f853400dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400b6c0 con 0x7f85401383b0 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:28.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: dump: 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8534019070 con 0x7f85401383b0 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400ba60 con 0x7f85401383b0 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 5 monclient: authenticate success, global_id 44708 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: _renew_subs 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85401d16a0 con 0x7f85401383b0 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853effd640 4 set_mon_vals no callback set 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f853d7fa640 10 monclient: dump: 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: _renew_subs 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.040+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f854013f6c0 con 0x7f85401383b0 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8547241640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8547241640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 0x7f854002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8540137ca0 con 0x7f854009c220 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 0x7f854002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient: get_auth_request con 0x7f854009c220 auth_method 0 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f853401e460 con 0x7f85401383b0 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 10 monclient(hunting): handle_auth_done global_id 74109 payload 824 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f852c03b9c0 0x7f852c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f854013edb0 con 0x7f854009c220 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(162..162 src has 1..162) ==== 9921+0+0 (secure 0 0 0) 0x7f85340515f0 con 0x7f85401383b0 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f8544fdd640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 0x7f854002d450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f853000f770 tx=0x7f853000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f85457de640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f852c03b9c0 0x7f852c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f85457de640 10 monclient: get_auth_request con 0x7f852c03b9c0 auth_method 0 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530003970 con 0x7f854009c220 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 10 monclient: dump: 2024-09-15T21:23:28.068 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8530019070 con 0x7f854009c220 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:28.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 msgr2=0x7f854002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 0x7f854002d450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f853000f770 tx=0x7f853000fc30 comp rx=0 tx=0).stop 2024-09-15T21:23:28.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 msgr2=0x7f854002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:28.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f853d7fa640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c220 0x7f854002d450 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.044+0000 7f85457de640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f852c03b9c0 0x7f852c03de80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f85380ad000 tx=0x7f85380a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 0x7f854002d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8540069270 con 0x7f854009c600 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 0x7f854002d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient: get_auth_request con 0x7f854009c600 auth_method 0 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 10 monclient(hunting): handle_auth_done global_id 74112 payload 824 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 1 -- 172.21.15.17:0/3390395725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f854006cba0 con 0x7f854009c600 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8544fdd640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 0x7f854002d990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f853000f770 tx=0x7f8530003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853000dd70 con 0x7f854009c600 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 10 monclient: dump: 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.073 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f853000dd70 con 0x7f854009c600 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 msgr2=0x7f854002d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 0x7f854002d990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f853000f770 tx=0x7f8530003770 comp rx=0 tx=0).stop 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 msgr2=0x7f854002d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f853d7fa640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f854009c600 0x7f854002d990 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f853000f770 tx=0x7f8530003770 comp rx=0 tx=0).stop 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:28.074 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20835, 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:28.075 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.077 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:28.078 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.079 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:28.083 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.085 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.085 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:28.085 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.089 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:23:28.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.092 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:28.092 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f852c03b9c0 msgr2=0x7f852c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f852c03b9c0 0x7f852c03de80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f85380ad000 tx=0x7f85380a6070 comp rx=0 tx=0).stop 2024-09-15T21:23:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 10 monclient: shutdown 2024-09-15T21:23:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85401383b0 msgr2=0x7f85401d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 --2- 172.21.15.17:0/3390395725 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85401383b0 0x7f85401d0c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f853400d830 tx=0x7f853400dcf0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f85457de640 1 -- 172.21.15.17:0/3390395725 reap_dead start 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f85457de640 1 -- 172.21.15.17:0/3390395725 reap_dead start 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 shutdown_connections 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 >> 172.21.15.17:0/3390395725 conn(0x7f85400aac40 msgr2=0x7f8540136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 shutdown_connections 2024-09-15T21:23:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.052+0000 7f8547241640 1 -- 172.21.15.17:0/3390395725 wait complete. 2024-09-15T21:23:28.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:23:28.094 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:23:28.095 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:23:28.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient: get_monmap_and_config 2024-09-15T21:23:28.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient: monmap: 2024-09-15T21:23:28.211 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:28.210521+0000 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:28.210521+0000 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:28.212 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 Processor -- start 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 -- start start 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient: init 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feba80a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feba809ffc0 0x7feba813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): picked mon.noname-a con 0x7feba809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feba80a1380 0x7feba813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): picked mon.noname-c con 0x7feba80a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba80a3130 con 0x7feba80a0890 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7feba8073df0 con 0x7feba809ffc0 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7feba80a1bf0 con 0x7feba80a1380 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febae2ee640 10 monclient(hunting): authenticate will time out at 1331.605718s 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feba809ffc0 0x7feba813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feba809ffc0 0x7feba813aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34678/0 (socket says 172.21.15.17:34678) 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 -- 172.21.15.17:0/612437021 learned_addr learned my addr 172.21.15.17:0/612437021 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): get_auth_request con 0x7feba809ffc0 auth_method 0 2024-09-15T21:23:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba7fff640 1 --1- 172.21.15.17:0/612437021 >> v1:172.21.15.17:6789/0 conn(0x7feba80a0890 0x7feba80a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43536/0 (socket says 172.21.15.17:43536) 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febac88b640 1 --2- 172.21.15.17:0/612437021 >> v2:172.21.15.119:3300/0 conn(0x7feba80a1380 0x7feba813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febac88b640 10 monclient(hunting): get_auth_request con 0x7feba80a1380 auth_method 0 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febac88b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febac88b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7febac88b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient(hunting): handle_auth_done global_id 74115 payload 274 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 -- 172.21.15.17:0/612437021 >> v2:172.21.15.119:3300/0 conn(0x7feba80a1380 msgr2=0x7feba813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 --2- 172.21.15.17:0/612437021 >> v2:172.21.15.119:3300/0 conn(0x7feba80a1380 0x7feba813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 -- 172.21.15.17:0/612437021 >> v1:172.21.15.17:6789/0 conn(0x7feba80a0890 legacy=0x7feba80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 monclient: found mon.noname-a 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.208+0000 7feba77fe640 1 -- 172.21.15.17:0/612437021 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba809ccd0 con 0x7feba809ffc0 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba77fe640 1 --2- 172.21.15.17:0/612437021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba809ffc0 0x7feba813aac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb90009e50 tx=0x7feb9001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da441c4ff24282f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 1 -- 172.21.15.17:0/612437021 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb900a5570 con 0x7feba809ffc0 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: dump: 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 1 -- 172.21.15.17:0/612437021 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb900a5710 con 0x7feba809ffc0 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 5 monclient: authenticate success, global_id 74115 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 1 -- 172.21.15.17:0/612437021 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb900a5ae0 con 0x7feba809ffc0 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 4 set_mon_vals no callback set 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7feba6ffd640 10 monclient: dump: 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: shutdown 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- 172.21.15.17:0/612437021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba809ffc0 msgr2=0x7feba813aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 --2- 172.21.15.17:0/612437021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba809ffc0 0x7feba813aac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb90009e50 tx=0x7feb9001c8b0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- 172.21.15.17:0/612437021 shutdown_connections 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 --2- 172.21.15.17:0/612437021 >> v2:172.21.15.119:3300/0 conn(0x7feba80a1380 0x7feba813d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 --2- 172.21.15.17:0/612437021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba809ffc0 0x7feba813aac0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- 172.21.15.17:0/612437021 >> 172.21.15.17:0/612437021 conn(0x7feba80a66a0 msgr2=0x7feba80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- 172.21.15.17:0/612437021 shutdown_connections 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- 172.21.15.17:0/612437021 wait complete. 2024-09-15T21:23:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: monmap: 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 Processor -- start 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- start start 2024-09-15T21:23:28.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: init 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba809ffc0 0x7feba8132440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): picked mon.noname-c con 0x7feba809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba80a0890 0x7feba81396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): picked mon.noname-a con 0x7feba80a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba80a1380 0x7feba8139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): picked mon.noname-b con 0x7feba80a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feba8140100 con 0x7feba80a0890 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feba813ff80 con 0x7feba80a1380 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feba8140280 con 0x7feba809ffc0 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.212+0000 7febae2ee640 10 monclient(hunting): authenticate will time out at 1331.611970s 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba809ffc0 0x7feba8132440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba809ffc0 0x7feba8132440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55798/0 (socket says 172.21.15.17:55798) 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 -- 172.21.15.17:0/2140844001 learned_addr learned my addr 172.21.15.17:0/2140844001 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): get_auth_request con 0x7feba809ffc0 auth_method 0 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba80a0890 0x7feba81396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): get_auth_request con 0x7feba80a0890 auth_method 0 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba80a1380 0x7feba8139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 10 monclient(hunting): get_auth_request con 0x7feba80a1380 auth_method 0 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient(hunting): handle_auth_done global_id 44711 payload 824 2024-09-15T21:23:28.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba80a1380 msgr2=0x7feba8139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba80a1380 0x7feba8139c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba80a0890 msgr2=0x7feba81396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feba80a0890 0x7feba81396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 monclient: found mon.noname-c 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba813c0c0 con 0x7feba809ffc0 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba7fff640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba809ffc0 0x7feba8132440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feba000c480 tx=0x7feba000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0018070 con 0x7feba809ffc0 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: dump: 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba0007480 con 0x7feba809ffc0 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0004a10 con 0x7feba809ffc0 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 5 monclient: authenticate success, global_id 44711 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 10 monclient: _renew_subs 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba811ce20 con 0x7feba809ffc0 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba67fc640 4 set_mon_vals no callback set 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: dump: 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 10 monclient: _renew_subs 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feba811cb40 con 0x7feba809ffc0 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 0x7feba813a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feba80a0c70 con 0x7feba8132980 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 0x7feba813a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient: get_auth_request con 0x7feba8132980 auth_method 0 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7feba001b070 con 0x7feba809ffc0 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 10 monclient(hunting): handle_auth_done global_id 44714 payload 824 2024-09-15T21:23:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feba80008e0 con 0x7feba8132980 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb8c03b9c0 0x7feb8c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(162..162 src has 1..162) ==== 9921+0+0 (secure 0 0 0) 0x7feba0050250 con 0x7feba809ffc0 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba77fe640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 0x7feba813a170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feba809d0a0 tx=0x7feb90019650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90009510 con 0x7feba8132980 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: dump: 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.238 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb8c03b9c0 0x7feb8c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 10 monclient: get_auth_request con 0x7feb8c03b9c0 auth_method 0 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7feb900b6800 con 0x7feba8132980 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 msgr2=0x7feba813a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 0x7feba813a170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feba809d0a0 tx=0x7feb90019650 comp rx=0 tx=0).stop 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 msgr2=0x7feba813a170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7feba4ff9640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba8132980 0x7feba813a170 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feba809d0a0 tx=0x7feb90019650 comp rx=0 tx=0).stop 2024-09-15T21:23:28.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.216+0000 7febac88b640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb8c03b9c0 0x7feb8c03de80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feb9800b950 tx=0x7feb98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 0x7feba8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feba813aa90 con 0x7feba813a6b0 2024-09-15T21:23:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 0x7feba8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient: get_auth_request con 0x7feba813a6b0 auth_method 0 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 10 monclient(hunting): handle_auth_done global_id 44717 payload 824 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 1 -- 172.21.15.17:0/2140844001 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feba806d020 con 0x7feba813a6b0 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba77fe640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 0x7feba8071190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb9001a4e0 tx=0x7feb900a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb900c0440 con 0x7feba813a6b0 2024-09-15T21:23:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 10 monclient: dump: 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7feb900cb400 con 0x7feba813a6b0 2024-09-15T21:23:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 msgr2=0x7feba8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 0x7feba8071190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb9001a4e0 tx=0x7feb900a7000 comp rx=0 tx=0).stop 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 msgr2=0x7feba8071190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7feba4ff9640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba813a6b0 0x7feba8071190 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb9001a4e0 tx=0x7feb900a7000 comp rx=0 tx=0).stop 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 176822, 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:28.257 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb8c03b9c0 msgr2=0x7feb8c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb8c03b9c0 0x7feb8c03de80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feb9800b950 tx=0x7feb98008040 comp rx=0 tx=0).stop 2024-09-15T21:23:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 10 monclient: shutdown 2024-09-15T21:23:28.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba809ffc0 msgr2=0x7feba8132440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 --2- 172.21.15.17:0/2140844001 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feba809ffc0 0x7feba8132440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feba000c480 tx=0x7feba000c940 comp rx=0 tx=0).stop 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febac88b640 1 -- 172.21.15.17:0/2140844001 reap_dead start 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febac88b640 1 -- 172.21.15.17:0/2140844001 reap_dead start 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 shutdown_connections 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 >> 172.21.15.17:0/2140844001 conn(0x7feba80a66a0 msgr2=0x7feba806fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 shutdown_connections 2024-09-15T21:23:28.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.224+0000 7febae2ee640 1 -- 172.21.15.17:0/2140844001 wait complete. 2024-09-15T21:23:28.259 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:23:28.260 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:23:28.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:23:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient: get_monmap_and_config 2024-09-15T21:23:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient: monmap: 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:28.375566+0000 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:28.375566+0000 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:28.378 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 1 Processor -- start 2024-09-15T21:23:28.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 1 -- start start 2024-09-15T21:23:28.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient: init 2024-09-15T21:23:28.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9b64099730 0x7f9b64099b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient(hunting): picked mon.noname-c con 0x7f9b64099730 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9b6409e510 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9b6409e980 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b640a3cb0 con 0x7f9b6409e510 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b640a2ce0 con 0x7f9b6409e980 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9b6409e180 con 0x7f9b64099730 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b68f26640 10 monclient(hunting): authenticate will time out at 1331.771525s 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b5bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9b6409e510 0x7f9b640a1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43538/0 (socket says 172.21.15.17:43538) 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.372+0000 7f9b5bfff640 1 -- 172.21.15.17:0/3907815301 learned_addr learned my addr 172.21.15.17:0/3907815301 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61358711 0 0) 0x7f9b4c001120 con 0x7f9b6409e510 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): my global_id is 74121 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b50003530 con 0x7f9b6409e510 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1877512390 0 0) 0x7f9b50003530 con 0x7f9b6409e510 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 >> v2:172.21.15.119:3300/0 conn(0x7f9b64099730 msgr2=0x7f9b64099b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 --2- 172.21.15.17:0/3907815301 >> v2:172.21.15.119:3300/0 conn(0x7f9b64099730 0x7f9b64099b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 >> v1:172.21.15.119:6789/0 conn(0x7f9b6409e980 legacy=0x7f9b640a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b64068fb0 con 0x7f9b6409e510 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f9b4c0018f0 con 0x7f9b6409e510 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: dump: 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 >> v1:172.21.15.17:6789/0 conn(0x7f9b6409e510 legacy=0x7f9b640a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50003a60 0x7f9b50003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): picked mon.a con 0x7f9b50003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b50004810 0x7f9b50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): picked mon.c con 0x7f9b50004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b500099c0 0x7f9b5000bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): picked mon.b con 0x7f9b500099c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6409e180 con 0x7f9b50003a60 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5000c3a0 con 0x7f9b500099c0 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5000c5a0 con 0x7f9b50004810 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b61d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50003a60 0x7f9b50003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): get_auth_request con 0x7f9b50003a60 auth_method 0 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b50004810 0x7f9b50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): get_auth_request con 0x7f9b50004810 auth_method 0 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b62d9d640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b500099c0 0x7f9b5000bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b62d9d640 10 monclient(hunting): get_auth_request con 0x7f9b500099c0 auth_method 0 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b62d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b62d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_done global_id 74121 payload 293 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 -- 172.21.15.17:0/3907815301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b500099c0 msgr2=0x7f9b5000bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b500099c0 0x7f9b5000bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 -- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50003a60 msgr2=0x7f9b50003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50003a60 0x7f9b50003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 monclient: found mon.c 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 1 -- 172.21.15.17:0/3907815301 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b64068fb0 con 0x7f9b50004810 2024-09-15T21:23:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b5bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b62d9d640 1 -- 172.21.15.17:0/3907815301 reap_dead start 2024-09-15T21:23:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b68f26640 5 monclient: authenticate success, global_id 74121 2024-09-15T21:23:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.376+0000 7f9b68f26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50003a60 0x7f9b50003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b50004810 0x7f9b50004c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c490 tx=0x7f9b4c01d690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=227322a5e0035af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00c0e0 con 0x7f9b50004810 2024-09-15T21:23:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: dump: 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0a9020 con 0x7f9b50004810 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 1 -- 172.21.15.17:0/3907815301 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c005b00 con 0x7f9b50004810 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 4 set_mon_vals no callback set 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b61d9b640 10 monclient: dump: 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.397 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: shutdown 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b50004810 msgr2=0x7f9b50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b50004810 0x7f9b50004c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c490 tx=0x7f9b4c01d690 comp rx=0 tx=0).stop 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/3907815301 shutdown_connections 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 --2- 172.21.15.17:0/3907815301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b50004810 0x7f9b50004c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/3907815301 >> 172.21.15.17:0/3907815301 conn(0x7f9b640a2fb0 msgr2=0x7f9b640a3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/3907815301 shutdown_connections 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/3907815301 wait complete. 2024-09-15T21:23:28.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: monmap: 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 Processor -- start 2024-09-15T21:23:28.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- start start 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: init 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50004810 0x7f9b641cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): picked mon.noname-a con 0x7f9b50004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b641d0290 0x7f9b641d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): picked mon.noname-b con 0x7f9b641d0290 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b641cc930 0x7f9b641ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): picked mon.noname-c con 0x7f9b641cc930 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640a2850 con 0x7f9b50004810 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640a26d0 con 0x7f9b641d0290 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9b640a29d0 con 0x7f9b641cc930 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient(hunting): authenticate will time out at 1331.778773s 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50004810 0x7f9b641cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b641cc930 0x7f9b641ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50004810 0x7f9b641cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34696/0 (socket says 172.21.15.17:34696) 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): get_auth_request con 0x7f9b641cc930 auth_method 0 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 1 -- 172.21.15.17:0/2173921850 learned_addr learned my addr 172.21.15.17:0/2173921850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): get_auth_request con 0x7f9b50004810 auth_method 0 2024-09-15T21:23:28.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b641d0290 0x7f9b641d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5bfff640 10 monclient(hunting): get_auth_request con 0x7f9b641d0290 auth_method 0 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient(hunting): handle_auth_done global_id 44720 payload 824 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b641d0290 msgr2=0x7f9b641d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b641d0290 0x7f9b641d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50004810 msgr2=0x7f9b641cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50004810 0x7f9b641cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 monclient: found mon.noname-c 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6409d6a0 con 0x7f9b641cc930 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6259c640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b50004810 0x7f9b641cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b62d9d640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b641cc930 0x7f9b641ccd70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9b640693c0 tx=0x7f9b5c00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c003980 con 0x7f9b641cc930 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: dump: 2024-09-15T21:23:28.404 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b5c018070 con 0x7f9b641cc930 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 5 monclient: authenticate success, global_id 44720 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: _renew_subs 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b640950f0 con 0x7f9b641cc930 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c003da0 con 0x7f9b641cc930 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b5affd640 10 monclient: dump: 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.406 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b6159a640 4 set_mon_vals no callback set 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: _renew_subs 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.380+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b64095cc0 con 0x7f9b641cc930 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b68f26640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b68f26640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 0x7f9b641cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b68f26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b64095f00 con 0x7f9b6402e5d0 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9b5c0050d0 con 0x7f9b641cc930 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 0x7f9b641cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient: get_auth_request con 0x7f9b6402e5d0 auth_method 0 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9b4403b9f0 0x7f9b4403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(162..162 src has 1..162) ==== 9921+0+0 (secure 0 0 0) 0x7f9b5c050c00 con 0x7f9b641cc930 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9b4403b9f0 0x7f9b4403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5bfff640 10 monclient: get_auth_request con 0x7f9b4403b9f0 auth_method 0 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 10 monclient(hunting): handle_auth_done global_id 64327 payload 824 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b6402e9b0 con 0x7f9b6402e5d0 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b6259c640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 0x7f9b641cd2b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b5400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5bfff640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9b4403b9f0 0x7f9b4403deb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b6409ccb0 tx=0x7f9b4c0a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540093b0 con 0x7f9b6402e5d0 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 10 monclient: dump: 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.409 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9b54009390 con 0x7f9b6402e5d0 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 msgr2=0x7f9b641cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 0x7f9b641cd2b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b5400f610 comp rx=0 tx=0).stop 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 msgr2=0x7f9b641cd2b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:28.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.384+0000 7f9b5affd640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b6402e5d0 0x7f9b641cd2b0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b5400f610 comp rx=0 tx=0).stop 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 0x7f9b640641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640962b0 con 0x7f9b64063e00 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 0x7f9b640641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient: get_auth_request con 0x7f9b64063e00 auth_method 0 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 10 monclient(hunting): handle_auth_done global_id 64330 payload 824 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 1 -- 172.21.15.17:0/2173921850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9b64094b40 con 0x7f9b64063e00 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b6259c640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 0x7f9b640641e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b54004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54004220 con 0x7f9b64063e00 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 10 monclient: dump: 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.413 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f9b54004220 con 0x7f9b64063e00 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 msgr2=0x7f9b640641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 0x7f9b640641e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b54004710 comp rx=0 tx=0).stop 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 msgr2=0x7f9b640641e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b5affd640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b64063e00 0x7f9b640641e0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b54004710 comp rx=0 tx=0).stop 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 86654, 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:28.415 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.416 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.417 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:28.418 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:28.419 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.420 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.421 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:28.422 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:28.424 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:28.425 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:28.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:28.427 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:28.428 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9b4403b9f0 msgr2=0x7f9b4403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9b4403b9f0 0x7f9b4403deb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b6409ccb0 tx=0x7f9b4c0a7040 comp rx=0 tx=0).stop 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 10 monclient: shutdown 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b641cc930 msgr2=0x7f9b641ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b62d9d640 1 -- 172.21.15.17:0/2173921850 reap_dead start 2024-09-15T21:23:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b641cc930 0x7f9b641ccd70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9b640693c0 tx=0x7f9b5c00ec10 comp rx=0 tx=0).stop 2024-09-15T21:23:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.392+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 shutdown_connections 2024-09-15T21:23:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.396+0000 7f9b68f26640 1 --2- 172.21.15.17:0/2173921850 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9b641cc930 0x7f9b641ccd70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.396+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 >> 172.21.15.17:0/2173921850 conn(0x7f9b640a2fb0 msgr2=0x7f9b640721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.396+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 shutdown_connections 2024-09-15T21:23:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.396+0000 7f9b68f26640 1 -- 172.21.15.17:0/2173921850 wait complete. 2024-09-15T21:23:28.430 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:23:28.430 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:23:28.430 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.540+0000 7f4557c2c640 10 monclient: get_monmap_and_config 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.540+0000 7f4557c2c640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.540+0000 7f4557c2c640 10 monclient: monmap: 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:28.545232+0000 2024-09-15T21:23:28.546 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:28.545232+0000 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:28.547 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 Processor -- start 2024-09-15T21:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 -- start start 2024-09-15T21:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient: init 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient: _add_conns ranks=[4,0,3,2,1] 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45500a28b0 0x7f45500a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): picked mon.noname-c con 0x7f45500a28b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f45500a31d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4550135d80 0x7f455013dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): picked mon.noname-a con 0x7f4550135d80 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455009e410 con 0x7f45500a31d0 2024-09-15T21:23:28.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f455009ef80 con 0x7f4550135d80 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f45500a1330 con 0x7f45500a28b0 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f4557c2c640 10 monclient(hunting): authenticate will time out at 1331.941226s 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4550135d80 0x7f455013dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4550135d80 0x7f455013dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34704/0 (socket says 172.21.15.17:34704) 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 -- 172.21.15.17:0/1297484119 learned_addr learned my addr 172.21.15.17:0/1297484119 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): get_auth_request con 0x7f4550135d80 auth_method 0 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45559c8640 1 --2- 172.21.15.17:0/1297484119 >> v2:172.21.15.119:3300/0 conn(0x7f45500a28b0 0x7f45500a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45559c8640 10 monclient(hunting): get_auth_request con 0x7f45500a28b0 auth_method 0 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45559c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45559c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45559c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient(hunting): handle_auth_done global_id 74127 payload 274 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 -- 172.21.15.17:0/1297484119 >> v2:172.21.15.119:3300/0 conn(0x7f45500a28b0 msgr2=0x7f45500a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 --2- 172.21.15.17:0/1297484119 >> v2:172.21.15.119:3300/0 conn(0x7f45500a28b0 0x7f45500a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 -- 172.21.15.17:0/1297484119 >> v1:172.21.15.17:6789/0 conn(0x7f45500a31d0 legacy=0x7f4550135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 monclient: found mon.noname-a 2024-09-15T21:23:28.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 -- 172.21.15.17:0/1297484119 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455006d0d0 con 0x7f4550135d80 2024-09-15T21:23:28.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.544+0000 7f45561c9640 1 --2- 172.21.15.17:0/1297484119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4550135d80 0x7f455013dae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45500701f0 tx=0x7f454801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a838ad68a1215373 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 1 -- 172.21.15.17:0/1297484119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480b1070 con 0x7f4550135d80 2024-09-15T21:23:28.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:28.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: dump: 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 1 -- 172.21.15.17:0/1297484119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45480ac410 con 0x7f4550135d80 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 5 monclient: authenticate success, global_id 74127 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 4 set_mon_vals no callback set 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 1 -- 172.21.15.17:0/1297484119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480ac7e0 con 0x7f4550135d80 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45549c6640 10 monclient: dump: 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: shutdown 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1297484119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4550135d80 msgr2=0x7f455013dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1297484119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4550135d80 0x7f455013dae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45500701f0 tx=0x7f454801c3b0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1297484119 shutdown_connections 2024-09-15T21:23:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1297484119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4550135d80 0x7f455013dae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1297484119 >> v2:172.21.15.119:3300/0 conn(0x7f45500a28b0 0x7f45500a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1297484119 >> 172.21.15.17:0/1297484119 conn(0x7f45500a74a0 msgr2=0x7f45500a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1297484119 shutdown_connections 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1297484119 wait complete. 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: build_initial_monmap 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: monmap: 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 Processor -- start 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- start start 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: init 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45500a28b0 0x7f455009bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): picked mon.noname-c con 0x7f45500a28b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 0x7f455009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): picked mon.noname-a con 0x7f45500a31d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:28.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4550135d80 0x7f45501cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): picked mon.noname-b con 0x7f4550135d80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f45500a4310 con 0x7f45500a31d0 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45500a35e0 con 0x7f4550135d80 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45500a4490 con 0x7f45500a28b0 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient(hunting): authenticate will time out at 1331.946228s 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45500a28b0 0x7f455009bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 0x7f455009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45500a28b0 0x7f455009bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55850/0 (socket says 172.21.15.17:55850) 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 1 -- 172.21.15.17:0/1053762882 learned_addr learned my addr 172.21.15.17:0/1053762882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 0x7f455009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34708/0 (socket says 172.21.15.17:34708) 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): get_auth_request con 0x7f45500a28b0 auth_method 0 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): get_auth_request con 0x7f45500a31d0 auth_method 0 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45559c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45561c9640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4550135d80 0x7f45501cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient(hunting): handle_auth_done global_id 74133 payload 824 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 -- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45500a28b0 msgr2=0x7f455009bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45500a28b0 0x7f455009bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 -- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4550135d80 msgr2=0x7f45501cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4550135d80 0x7f45501cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient: _finish_hunting 0 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 monclient: found mon.noname-a 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 -- 172.21.15.17:0/1053762882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455013b8f0 con 0x7f45500a31d0 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45561c9640 10 monclient: get_auth_request con 0x7f4550135d80 auth_method 0 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45561c9640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4550135d80 0x7f45501cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f45551c7640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 0x7f455009c480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f454400c390 tx=0x7f454400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544017070 con 0x7f45500a31d0 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: dump: 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.566 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45440052d0 con 0x7f45500a31d0 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 5 monclient: authenticate success, global_id 74133 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _renew_subs 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45500a62f0 con 0x7f45500a31d0 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440054c0 con 0x7f45500a31d0 2024-09-15T21:23:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 10 monclient: dump: 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454f7fe640 4 set_mon_vals no callback set 2024-09-15T21:23:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _renew_subs 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455013b610 con 0x7f45500a31d0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f454401a070 con 0x7f45500a31d0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.548+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45500a2c90 con 0x7f45500a31d0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.552+0000 7f454e7fc640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f453c03b9c0 0x7f453c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.552+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9921+0+0 (secure 0 0 0) 0x7f4544050200 con 0x7f45500a31d0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.552+0000 7f45559c8640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f453c03b9c0 0x7f453c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.552+0000 7f45559c8640 10 monclient: get_auth_request con 0x7f453c03b9c0 auth_method 0 2024-09-15T21:23:28.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.552+0000 7f45559c8640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f453c03b9c0 0x7f453c03de80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4540003bd0 tx=0x7f4540009970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:28.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.556+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f454404d7f0 con 0x7f45500a31d0 2024-09-15T21:23:28.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.556+0000 7f454e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:28.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.556+0000 7f454e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:28.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.670+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v163) ==== 127+0+0 (secure 0 0 0) 0x7f134c010e20 con 0x7f135c135fd0 2024-09-15T21:23:28.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.670+0000 7f13597fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T21:23:28.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.670+0000 7f13597fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:28.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.786+0000 7f1362a6f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T21:23:28.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.786+0000 7f1362a6f640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T21:23:28.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.786+0000 7f1362a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:28.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:28.786+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) -- 0x7f135c11d3a0 con 0x7f135c135fd0 2024-09-15T21:23:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f4557c2c640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:23:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f4557c2c640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:23:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f4557c2c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f455006e030 con 0x7f45500a31d0 2024-09-15T21:23:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f454e7fc640 1 -- 172.21.15.17:0/1053762882 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f45440102e0 con 0x7f45500a31d0 2024-09-15T21:23:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f454e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:23:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.796+0000 7f454e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:28.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.796+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.796+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=12,kv=12,logm=62} crc {auth=145739141,config=3541143443,health=3721292393,kv=3079713637,logm=1395746468}) 2024-09-15T21:23:28.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.796+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=14,config=2,health=12,kv=12,logm=60} crc {auth=36559625,config=3541143443,health=3721292393,kv=3079713637,logm=2882872556}) 2024-09-15T21:23:28.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.796+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.803 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.796+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=12,kv=12,logm=62} crc {auth=145739141,config=3541143443,health=3721292393,kv=3079713637,logm=1395746468}) 2024-09-15T21:23:28.803 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.796+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=1,health=11,kv=12,logm=63} crc {auth=998620031,config=555582061,health=2731460370,kv=3079713637,logm=3172867902}) 2024-09-15T21:23:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f453c03b9c0 msgr2=0x7f453c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f453c03b9c0 0x7f453c03de80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4540003bd0 tx=0x7f4540009970 comp rx=0 tx=0).stop 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 10 monclient: shutdown 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 msgr2=0x7f455009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 0x7f455009c480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f454400c390 tx=0x7f454400c850 comp rx=0 tx=0).stop 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 shutdown_connections 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f453c03b9c0 0x7f453c03de80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4550135d80 0x7f45501cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45500a31d0 0x7f455009c480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 --2- 172.21.15.17:0/1053762882 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45500a28b0 0x7f455009bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:28.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 >> 172.21.15.17:0/1053762882 conn(0x7f45500a74a0 msgr2=0x7f45500a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:28.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 shutdown_connections 2024-09-15T21:23:28.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:28.800+0000 7f4557c2c640 1 -- 172.21.15.17:0/1053762882 wait complete. 2024-09-15T21:23:28.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3163161425}) 2024-09-15T21:23:28.806 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2602394887}) 2024-09-15T21:23:28.806 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.806 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3163161425}) 2024-09-15T21:23:28.806 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=753412832}) 2024-09-15T21:23:28.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.800+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1099336780}) 2024-09-15T21:23:28.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4098376273}) 2024-09-15T21:23:28.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1099336780}) 2024-09-15T21:23:28.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=902474424}) 2024-09-15T21:23:28.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=592595984}) 2024-09-15T21:23:28.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=587524133}) 2024-09-15T21:23:28.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=592595984}) 2024-09-15T21:23:28.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.804+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2748105412}) 2024-09-15T21:23:28.811 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.811 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1036756596}) 2024-09-15T21:23:28.811 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3171201884}) 2024-09-15T21:23:28.811 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.811 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1036756596}) 2024-09-15T21:23:28.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4119709285}) 2024-09-15T21:23:28.813 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.813 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2903069905}) 2024-09-15T21:23:28.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1263575186}) 2024-09-15T21:23:28.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2903069905}) 2024-09-15T21:23:28.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.808+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=553408042}) 2024-09-15T21:23:28.815 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4167302906}) 2024-09-15T21:23:28.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2781743763}) 2024-09-15T21:23:28.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4167302906}) 2024-09-15T21:23:28.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2207324127}) 2024-09-15T21:23:28.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2891783083}) 2024-09-15T21:23:28.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353342488}) 2024-09-15T21:23:28.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2891783083}) 2024-09-15T21:23:28.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.812+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1066172391}) 2024-09-15T21:23:28.819 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:23:28.820 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.820 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=126467064}) 2024-09-15T21:23:28.820 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3152063933}) 2024-09-15T21:23:28.820 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.820 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=126467064}) 2024-09-15T21:23:28.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=938587379}) 2024-09-15T21:23:28.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1047971091}) 2024-09-15T21:23:28.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3632475929}) 2024-09-15T21:23:28.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1047971091}) 2024-09-15T21:23:28.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.816+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2579201724}) 2024-09-15T21:23:28.824 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.824 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3352232359}) 2024-09-15T21:23:28.824 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2644525765}) 2024-09-15T21:23:28.824 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3352232359}) 2024-09-15T21:23:28.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2459232593}) 2024-09-15T21:23:28.826 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.826 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=989799838}) 2024-09-15T21:23:28.826 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2410229871}) 2024-09-15T21:23:28.826 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.827 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=989799838}) 2024-09-15T21:23:28.827 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.820+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2569354333}) 2024-09-15T21:23:28.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=733749372}) 2024-09-15T21:23:28.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4277242401}) 2024-09-15T21:23:28.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=733749372}) 2024-09-15T21:23:28.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510880929}) 2024-09-15T21:23:28.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1875870670}) 2024-09-15T21:23:28.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=759419436}) 2024-09-15T21:23:28.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1875870670}) 2024-09-15T21:23:28.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.824+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=612403130}) 2024-09-15T21:23:28.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.828+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.828+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=569568416}) 2024-09-15T21:23:28.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.828+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2037217167}) 2024-09-15T21:23:28.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.828+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.828+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=569568416}) 2024-09-15T21:23:28.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.828+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1031759268}) 2024-09-15T21:23:28.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423143250}) 2024-09-15T21:23:28.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=564164160}) 2024-09-15T21:23:28.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423143250}) 2024-09-15T21:23:28.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3867149818}) 2024-09-15T21:23:28.837 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4017806431}) 2024-09-15T21:23:28.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3797467650}) 2024-09-15T21:23:28.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4017806431}) 2024-09-15T21:23:28.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.832+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2508141301}) 2024-09-15T21:23:28.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1905883238}) 2024-09-15T21:23:28.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237769609}) 2024-09-15T21:23:28.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1905883238}) 2024-09-15T21:23:28.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1220500481}) 2024-09-15T21:23:28.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3591251879}) 2024-09-15T21:23:28.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2403812876}) 2024-09-15T21:23:28.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3591251879}) 2024-09-15T21:23:28.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.836+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3928963930}) 2024-09-15T21:23:28.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3175628318}) 2024-09-15T21:23:28.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4038108206}) 2024-09-15T21:23:28.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3175628318}) 2024-09-15T21:23:28.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2837015355}) 2024-09-15T21:23:28.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=298962684}) 2024-09-15T21:23:28.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3017877430}) 2024-09-15T21:23:28.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=298962684}) 2024-09-15T21:23:28.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.840+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2266112822}) 2024-09-15T21:23:28.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1419495191}) 2024-09-15T21:23:28.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=687876681}) 2024-09-15T21:23:28.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1419495191}) 2024-09-15T21:23:28.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656327912}) 2024-09-15T21:23:28.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083687602}) 2024-09-15T21:23:28.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1142334928}) 2024-09-15T21:23:28.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083687602}) 2024-09-15T21:23:28.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.844+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510457852}) 2024-09-15T21:23:28.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3587317988}) 2024-09-15T21:23:28.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2779592807}) 2024-09-15T21:23:28.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3587317988}) 2024-09-15T21:23:28.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1638896759}) 2024-09-15T21:23:28.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3441717472}) 2024-09-15T21:23:28.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1314677309}) 2024-09-15T21:23:28.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3441717472}) 2024-09-15T21:23:28.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.848+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3396293593}) 2024-09-15T21:23:28.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754413502}) 2024-09-15T21:23:28.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2692379842}) 2024-09-15T21:23:28.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754413502}) 2024-09-15T21:23:28.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1731996288}) 2024-09-15T21:23:28.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2995737311}) 2024-09-15T21:23:28.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3436512771}) 2024-09-15T21:23:28.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2995737311}) 2024-09-15T21:23:28.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.852+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739674866}) 2024-09-15T21:23:28.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400338013}) 2024-09-15T21:23:28.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=735817394}) 2024-09-15T21:23:28.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400338013}) 2024-09-15T21:23:28.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1593862712}) 2024-09-15T21:23:28.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=78,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10} crc {logm=1411112386,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1344883956}) 2024-09-15T21:23:28.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=77,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11} crc {logm=3983483488,mdsmap=173658045,mgr=3162623449,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=4048569522}) 2024-09-15T21:23:28.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=78,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10} crc {logm=1411112386,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1344883956}) 2024-09-15T21:23:28.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.856+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=78,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9} crc {logm=1031063678,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=194275390}) 2024-09-15T21:23:28.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.860+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.860+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=3,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=69} crc {mgrstat=1048895405,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=1459412894,osd_snap=3058936622,osdmap=3607002196}) 2024-09-15T21:23:28.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.860+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=3,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=69} crc {mgrstat=1048895405,mon_config_key=3562240218,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=1459412894,osd_snap=841190635,osdmap=1188883770}) 2024-09-15T21:23:28.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.860+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.860+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=3,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=69} crc {mgrstat=1048895405,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=1459412894,osd_snap=3058936622,osdmap=3607002196}) 2024-09-15T21:23:28.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.860+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=2,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=3,osd_pg_creating=1,osd_snap=2,osdmap=74} crc {mgrstat=1443862369,mon_config_key=3200474398,monmap=186996059,nvmeofgw=598337686,osd_metadata=1354849262,osd_pg_creating=1459412894,osd_snap=841190635,osdmap=2352310350}) 2024-09-15T21:23:28.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.864+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.864+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1025931408}) 2024-09-15T21:23:28.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.864+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2149924284}) 2024-09-15T21:23:28.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.864+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.864+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1025931408}) 2024-09-15T21:23:28.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.864+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3186100778}) 2024-09-15T21:23:28.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.872+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.872+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=1254546539}) 2024-09-15T21:23:28.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.872+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=79} crc {osdmap=1960265230}) 2024-09-15T21:23:28.879 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.872+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:23:28.879 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.872+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=1254546539}) 2024-09-15T21:23:28.879 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:23:28.872+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=78} crc {osdmap=879967502}) 2024-09-15T21:23:29.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f13597fa640 1 -- 172.21.15.119:0/2072293131 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v164) ==== 127+0+0 (secure 0 0 0) 0x7f134c004db0 con 0x7f135c135fd0 2024-09-15T21:23:29.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f13597fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T21:23:29.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f13597fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:29.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f134803b9c0 msgr2=0x7f134803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f134803b9c0 0x7f134803de80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f135c0a7110 tx=0x7f13500a6000 comp rx=0 tx=0).stop 2024-09-15T21:23:29.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 10 monclient: shutdown 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 msgr2=0x7f135c1cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 0x7f135c1cda40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c450 tx=0x7f134c00c910 comp rx=0 tx=0).stop 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 shutdown_connections 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f134803b9c0 0x7f134803de80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f135c136460 0x7f135c11c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f135c135fd0 0x7f135c1cda40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 --2- 172.21.15.119:0/2072293131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135c135bf0 0x7f135c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 >> 172.21.15.119:0/2072293131 conn(0x7f135c0a66e0 msgr2=0x7f135c0a47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 shutdown_connections 2024-09-15T21:23:29.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.694+0000 7f1362a6f640 1 -- 172.21.15.119:0/2072293131 wait complete. 2024-09-15T21:23:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set reweight cold osd.0 6.5 6.6 2024-09-15T21:23:29.820 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:23:29.820 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:23:29.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.838+0000 7fbb477da640 10 monclient: get_monmap_and_config 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.838+0000 7fbb477da640 10 monclient: build_initial_monmap 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.838+0000 7fbb477da640 10 monclient: monmap: 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:29.843956+0000 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:29.843956+0000 2024-09-15T21:23:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:29.846 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:29.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.838+0000 7fbb477da640 1 Processor -- start 2024-09-15T21:23:29.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.838+0000 7fbb477da640 1 -- start start 2024-09-15T21:23:29.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.838+0000 7fbb477da640 10 monclient: init 2024-09-15T21:23:29.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:29.857 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb4009d550 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb40139c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): picked mon.noname-b con 0x7fbb4009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb4009e040 0x7fbb4013c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): picked mon.noname-a con 0x7fbb4009e040 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb400a3600 con 0x7fbb4009d550 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbb4009f1b0 con 0x7fbb4009e040 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbb400a2920 con 0x7fbb4009fae0 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb477da640 10 monclient(hunting): authenticate will time out at 1328.640646s 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb4009e040 0x7fbb4013c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.858 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb40139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb40139c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:57056/0 (socket says 172.21.15.119:57056) 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 -- 172.21.15.119:0/2696608886 learned_addr learned my addr 172.21.15.119:0/2696608886 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb4009e040 0x7fbb4013c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:53642/0 (socket says 172.21.15.119:53642) 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): get_auth_request con 0x7fbb4009fae0 auth_method 0 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): get_auth_request con 0x7fbb4009e040 auth_method 0 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.859 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient(hunting): handle_auth_done global_id 44726 payload 274 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 -- 172.21.15.119:0/2696608886 >> v2:172.21.15.17:3300/0 conn(0x7fbb4009e040 msgr2=0x7fbb4013c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 --2- 172.21.15.119:0/2696608886 >> v2:172.21.15.17:3300/0 conn(0x7fbb4009e040 0x7fbb4013c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 -- 172.21.15.119:0/2696608886 >> v1:172.21.15.17:6789/0 conn(0x7fbb4009d550 legacy=0x7fbb4009d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient: _finish_hunting 0 2024-09-15T21:23:29.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 monclient: found mon.noname-b 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.842+0000 7fbb44d75640 1 -- 172.21.15.119:0/2696608886 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb400a1d60 con 0x7fbb4009fae0 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 1 --2- 172.21.15.119:0/2696608886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb40139c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000bb10 tx=0x7fbb3001eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf41f67e0a5a72a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 1 -- 172.21.15.119:0/2696608886 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300b4070 con 0x7fbb4009fae0 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: dump: 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.861 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 1 -- 172.21.15.119:0/2696608886 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb300a79d0 con 0x7fbb4009fae0 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 5 monclient: authenticate success, global_id 44726 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:29.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 1 -- 172.21.15.119:0/2696608886 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300a7da0 con 0x7fbb4009fae0 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 4 set_mon_vals no callback set 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb37fff640 10 monclient: dump: 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.863 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: shutdown 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- 172.21.15.119:0/2696608886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 msgr2=0x7fbb40139c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 --2- 172.21.15.119:0/2696608886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb40139c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000bb10 tx=0x7fbb3001eaf0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- 172.21.15.119:0/2696608886 shutdown_connections 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 --2- 172.21.15.119:0/2696608886 >> v2:172.21.15.17:3300/0 conn(0x7fbb4009e040 0x7fbb4013c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 --2- 172.21.15.119:0/2696608886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb40139c70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- 172.21.15.119:0/2696608886 >> 172.21.15.119:0/2696608886 conn(0x7fbb400a66e0 msgr2=0x7fbb400a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- 172.21.15.119:0/2696608886 shutdown_connections 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- 172.21.15.119:0/2696608886 wait complete. 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: build_initial_monmap 2024-09-15T21:23:29.864 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: monmap: 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 Processor -- start 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- start start 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: init 2024-09-15T21:23:29.865 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): picked mon.noname-b con 0x7fbb4009d550 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb40131660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): picked mon.noname-a con 0x7fbb4009e040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb401378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): picked mon.noname-c con 0x7fbb4009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbb400a13e0 con 0x7fbb4009e040 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb400a1260 con 0x7fbb4009d550 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbb400a1560 con 0x7fbb4009fae0 2024-09-15T21:23:29.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb477da640 10 monclient(hunting): authenticate will time out at 1328.647196s 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45646/0 (socket says 172.21.15.119:45646) 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 learned_addr learned my addr 172.21.15.119:0/3353392609 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): get_auth_request con 0x7fbb4009d550 auth_method 0 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb40131660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb401378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): get_auth_request con 0x7fbb4009e040 auth_method 0 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): get_auth_request con 0x7fbb4009fae0 auth_method 0 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.868 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb44d75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient(hunting): handle_auth_done global_id 64336 payload 824 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 msgr2=0x7fbb401378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb401378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 msgr2=0x7fbb40131660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb40131660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient: _finish_hunting 0 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 monclient: found mon.noname-b 2024-09-15T21:23:29.869 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.846+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4013ae00 con 0x7fbb4009d550 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c4d0 tx=0x7fbb3c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0052d0 con 0x7fbb4009d550 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: dump: 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.870 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb3c005470 con 0x7fbb4009d550 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 5 monclient: authenticate success, global_id 64336 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 monclient: _renew_subs 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4013b3c0 con 0x7fbb4009d550 2024-09-15T21:23:29.871 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c012870 con 0x7fbb4009d550 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 10 monclient: dump: 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.872 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb377fe640 4 set_mon_vals no callback set 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 monclient: _renew_subs 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb40139340 con 0x7fbb4009d550 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4009d930 con 0x7fbb4009d550 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbb3c012a10 con 0x7fbb4009d550 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbb2403b9c0 0x7fbb2403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 9921+0+0 (secure 0 0 0) 0x7fbb3c050ae0 con 0x7fbb4009d550 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb44d75640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbb2403b9c0 0x7fbb2403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb44d75640 10 monclient: get_auth_request con 0x7fbb2403b9c0 auth_method 0 2024-09-15T21:23:29.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.850+0000 7fbb44d75640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbb2403b9c0 0x7fbb2403de80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbb300b1000 tx=0x7fbb300a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:29.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.854+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbb3c04e0d0 con 0x7fbb4009d550 2024-09-15T21:23:29.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.854+0000 7fbb35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:29.874 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:29.854+0000 7fbb35ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient: get_monmap_and_config 2024-09-15T21:23:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient: build_initial_monmap 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient: monmap: 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:29.951529+0000 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:29.951529+0000 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:29.953 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 1 Processor -- start 2024-09-15T21:23:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 1 -- start start 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient: init 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9cd009d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd01398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient(hunting): picked mon.noname-b con 0x7f9cd009def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9cd0139de0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd00a3490 con 0x7f9cd009d400 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd00a0320 con 0x7f9cd0139de0 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd00a1bc0 con 0x7f9cd009def0 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cd5488640 10 monclient(hunting): authenticate will time out at 1333.347074s 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd01398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cceffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9cd009d400 0x7f9cd009d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43554/0 (socket says 172.21.15.17:43554) 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cceffd640 1 -- 172.21.15.17:0/3836788910 learned_addr learned my addr 172.21.15.17:0/3836788910 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd01398a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55864/0 (socket says 172.21.15.17:55864) 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): get_auth_request con 0x7f9cd009def0 auth_method 0 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_done global_id 44732 payload 274 2024-09-15T21:23:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3836788910 >> v1:172.21.15.119:6789/0 conn(0x7f9cd0139de0 legacy=0x7f9cd013c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3836788910 >> v1:172.21.15.17:6789/0 conn(0x7f9cd009d400 legacy=0x7f9cd009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:23:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 1 monclient: found mon.noname-b 2024-09-15T21:23:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.948+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3836788910 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd009e7a0 con 0x7f9cd009def0 2024-09-15T21:23:29.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cce7fc640 1 --2- 172.21.15.17:0/3836788910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd01398a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009dc0 tx=0x7f9cc401d280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f857a306a1c585e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:29.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 1 -- 172.21.15.17:0/3836788910 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40b2070 con 0x7f9cd009def0 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: dump: 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.964 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 1 -- 172.21.15.17:0/3836788910 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc40a5900 con 0x7f9cd009def0 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 5 monclient: authenticate success, global_id 44732 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 1 -- 172.21.15.17:0/3836788910 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a5cd0 con 0x7f9cd009def0 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 4 set_mon_vals no callback set 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9ccdffb640 10 monclient: dump: 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 10 monclient: shutdown 2024-09-15T21:23:29.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 -- 172.21.15.17:0/3836788910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 msgr2=0x7f9cd01398a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3836788910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd01398a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009dc0 tx=0x7f9cc401d280 comp rx=0 tx=0).stop 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 -- 172.21.15.17:0/3836788910 shutdown_connections 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3836788910 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd01398a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 -- 172.21.15.17:0/3836788910 >> 172.21.15.17:0/3836788910 conn(0x7f9cd00a66a0 msgr2=0x7f9cd00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 -- 172.21.15.17:0/3836788910 shutdown_connections 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.952+0000 7f9cd5488640 1 -- 172.21.15.17:0/3836788910 wait complete. 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: build_initial_monmap 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: monmap: 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 Processor -- start 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- start start 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: init 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd009d400 0x7f9cd0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): picked mon.noname-b con 0x7f9cd009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd0134a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): picked mon.noname-c con 0x7f9cd009def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cd0139de0 0x7f9cd0134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): picked mon.noname-a con 0x7f9cd0139de0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd009ce50 con 0x7f9cd0139de0 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd009ccd0 con 0x7f9cd009d400 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd0140520 con 0x7f9cd009def0 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient(hunting): authenticate will time out at 1333.353141s 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd0134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cd0139de0 0x7f9cd0134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd0134a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55868/0 (socket says 172.21.15.17:55868) 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3696122415 learned_addr learned my addr 172.21.15.17:0/3696122415 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): get_auth_request con 0x7f9cd009def0 auth_method 0 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): get_auth_request con 0x7f9cd0139de0 auth_method 0 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd009d400 0x7f9cd0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): get_auth_request con 0x7f9cd009d400 auth_method 0 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient(hunting): handle_auth_done global_id 44735 payload 824 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd009d400 msgr2=0x7f9cd0073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd009d400 0x7f9cd0073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cd0139de0 msgr2=0x7f9cd0134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cd0139de0 0x7f9cd0134f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 monclient: found mon.noname-c 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 -- 172.21.15.17:0/3696122415 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd0073bd0 con 0x7f9cd009def0 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccf7fe640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cd0139de0 0x7f9cd0134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cce7fc640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd0134a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009dc0 tx=0x7f9cc40195c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40bc070 con 0x7f9cd009def0 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: dump: 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc40ad6e0 con 0x7f9cd009def0 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 5 monclient: authenticate success, global_id 44735 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _renew_subs 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd00a41e0 con 0x7f9cd009def0 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40b2040 con 0x7f9cd009def0 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 10 monclient: dump: 2024-09-15T21:23:29.974 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _renew_subs 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9ccd7fa640 4 set_mon_vals no callback set 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd0121100 con 0x7f9cd009def0 2024-09-15T21:23:29.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd009d7e0 con 0x7f9cd009def0 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9cc40b5070 con 0x7f9cd009def0 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cbc03b9c0 0x7f9cbc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(164..164 src has 1..164) ==== 9921+0+0 (secure 0 0 0) 0x7f9cc40eb940 con 0x7f9cd009def0 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.956+0000 7f9cceffd640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cbc03b9c0 0x7f9cbc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.960+0000 7f9cceffd640 10 monclient: get_auth_request con 0x7f9cbc03b9c0 auth_method 0 2024-09-15T21:23:29.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.960+0000 7f9cceffd640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cbc03b9c0 0x7f9cbc03de80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0005890 tx=0x7f9cc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:29.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.964+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9cd00a41e0 con 0x7f9cd009def0 2024-09-15T21:23:29.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.964+0000 7f9cb77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:29.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:29.964+0000 7f9cb77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:30.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.034+0000 7fbb477da640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T21:23:30.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.034+0000 7fbb477da640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T21:23:30.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.034+0000 7fbb477da640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:23:30.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.034+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) -- 0x7fbb40139a20 con 0x7fbb4009d550 2024-09-15T21:23:30.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cd5488640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:23:30.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cd5488640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:30.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cd5488640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9cd013b690 con 0x7f9cd009def0 2024-09-15T21:23:30.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cb77fe640 1 -- 172.21.15.17:0/3696122415 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f9cc4016560 con 0x7f9cd009def0 2024-09-15T21:23:30.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cb77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:30.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.200+0000 7f9cb77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:30.206 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:23:30.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cbc03b9c0 msgr2=0x7f9cbc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cbc03b9c0 0x7f9cbc03de80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0005890 tx=0x7f9cc0009040 comp rx=0 tx=0).stop 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 10 monclient: shutdown 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 msgr2=0x7f9cd0134a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd0134a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009dc0 tx=0x7f9cc40195c0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 shutdown_connections 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9cbc03b9c0 0x7f9cbc03de80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cd0139de0 0x7f9cd0134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9cd009def0 0x7f9cd0134a10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 --2- 172.21.15.17:0/3696122415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd009d400 0x7f9cd0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 >> 172.21.15.17:0/3696122415 conn(0x7f9cd00a66a0 msgr2=0x7f9cd013c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 shutdown_connections 2024-09-15T21:23:30.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.204+0000 7f9cd5488640 1 -- 172.21.15.17:0/3696122415 wait complete. 2024-09-15T21:23:30.225 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:23:30.226 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:23:30.226 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:23:30.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient: get_monmap_and_config 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient: monmap: 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:30.342980+0000 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:30.342980+0000 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:30.344 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:30.345 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:30.345 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:30.345 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:30.345 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 1 Processor -- start 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 1 -- start start 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient: init 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffa7c1340c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffa7c09d6c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:30.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c13c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient(hunting): picked mon.noname-b con 0x7ffa7c13a570 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c09e1b0 con 0x7ffa7c1340c0 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c09e490 con 0x7ffa7c09d6c0 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c09d520 con 0x7ffa7c13a570 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa8248e640 10 monclient(hunting): authenticate will time out at 1333.738851s 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c13c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa7bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffa7c1340c0 0x7ffa7c1344a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43570/0 (socket says 172.21.15.17:43570) 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa7bfff640 1 -- 172.21.15.17:0/2910688024 learned_addr learned my addr 172.21.15.17:0/2910688024 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): get_auth_request con 0x7ffa7c13a570 auth_method 0 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_done global_id 44738 payload 274 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa7affd640 1 -- 172.21.15.17:0/2910688024 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3076208359 0 0) 0x7ffa7c09e1b0 con 0x7ffa7c1340c0 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 1 -- 172.21.15.17:0/2910688024 >> v1:172.21.15.119:6789/0 conn(0x7ffa7c09d6c0 legacy=0x7ffa7c138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 1 -- 172.21.15.17:0/2910688024 >> v1:172.21.15.17:6789/0 conn(0x7ffa7c1340c0 legacy=0x7ffa7c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 1 monclient: found mon.noname-b 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa80a2b640 1 -- 172.21.15.17:0/2910688024 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c09e770 con 0x7ffa7c13a570 2024-09-15T21:23:30.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.340+0000 7ffa7affd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:23:30.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa80a2b640 1 --2- 172.21.15.17:0/2910688024 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c13c960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400a990 tx=0x7ffa7401d770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300c60ed392e0017 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 1 -- 172.21.15.17:0/2910688024 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400b040 con 0x7ffa7c13a570 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: dump: 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 5 monclient: authenticate success, global_id 44738 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 1 -- 172.21.15.17:0/2910688024 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa740a9630 con 0x7ffa7c13a570 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 4 set_mon_vals no callback set 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 1 -- 172.21.15.17:0/2910688024 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740a9a00 con 0x7ffa7c13a570 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa7affd640 10 monclient: dump: 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 10 monclient: shutdown 2024-09-15T21:23:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 -- 172.21.15.17:0/2910688024 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 msgr2=0x7ffa7c13c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 --2- 172.21.15.17:0/2910688024 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c13c960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400a990 tx=0x7ffa7401d770 comp rx=0 tx=0).stop 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 -- 172.21.15.17:0/2910688024 shutdown_connections 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 --2- 172.21.15.17:0/2910688024 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c13c960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 -- 172.21.15.17:0/2910688024 >> 172.21.15.17:0/2910688024 conn(0x7ffa7c0a66a0 msgr2=0x7ffa7c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 -- 172.21.15.17:0/2910688024 shutdown_connections 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.344+0000 7ffa8248e640 1 -- 172.21.15.17:0/2910688024 wait complete. 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: monmap: 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:30.359 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 Processor -- start 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- start start 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: init 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa7c09d6c0 0x7ffa7c1cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): picked mon.noname-b con 0x7ffa7c09d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1340c0 0x7ffa7c1cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): picked mon.noname-a con 0x7ffa7c1340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): picked mon.noname-c con 0x7ffa7c13a570 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c13f240 con 0x7ffa7c1340c0 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c13f0c0 con 0x7ffa7c09d6c0 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c13f3c0 con 0x7ffa7c13a570 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): authenticate will time out at 1333.744930s 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1340c0 0x7ffa7c1cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1340c0 0x7ffa7c1cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34732/0 (socket says 172.21.15.17:34732) 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 1 -- 172.21.15.17:0/3591739341 learned_addr learned my addr 172.21.15.17:0/3591739341 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa7c1340c0 auth_method 0 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): get_auth_request con 0x7ffa7c13a570 auth_method 0 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa7c09d6c0 0x7ffa7c1cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request con 0x7ffa7c09d6c0 auth_method 0 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient(hunting): handle_auth_done global_id 44741 payload 824 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa7c09d6c0 msgr2=0x7ffa7c1cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa7c09d6c0 0x7ffa7c1cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1340c0 msgr2=0x7ffa7c1cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1340c0 0x7ffa7c1cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 monclient: found mon.noname-c 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c120020 con 0x7ffa7c13a570 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7b7fe640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1340c0 0x7ffa7c1cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa80a2b640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c11c4e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa740ad000 tx=0x7ffa7401a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740a9430 con 0x7ffa7c13a570 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: dump: 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.364 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa74004030 con 0x7ffa7c13a570 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74004400 con 0x7ffa7c13a570 2024-09-15T21:23:30.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 10 monclient: dump: 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7a7fc640 4 set_mon_vals no callback set 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 5 monclient: authenticate success, global_id 44741 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: _renew_subs 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c120300 con 0x7ffa7c13a570 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: _renew_subs 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c09d0f0 con 0x7ffa7c13a570 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 0x7ffa7c11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c0a3d60 con 0x7ffa7c1d0410 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 0x7ffa7c11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient: get_auth_request con 0x7ffa7c1d0410 auth_method 0 2024-09-15T21:23:30.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffa740c1060 con 0x7ffa7c13a570 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa6803b9f0 0x7ffa6803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_done global_id 74154 payload 824 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa7c1344a0 con 0x7ffa7c1d0410 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(164..164 src has 1..164) ==== 9921+0+0 (secure 0 0 0) 0x7ffa740eb130 con 0x7ffa7c13a570 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.348+0000 7ffa7bfff640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 0x7ffa7c11ca20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f70 tx=0x7ffa6c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa7b7fe640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa6803b9f0 0x7ffa6803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c019070 con 0x7ffa7c1d0410 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 10 monclient: dump: 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.369 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa7b7fe640 10 monclient: get_auth_request con 0x7ffa6803b9f0 auth_method 0 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ffa6c010510 con 0x7ffa7c1d0410 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 msgr2=0x7ffa7c11ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 0x7ffa7c11ca20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f70 tx=0x7ffa6c0019c0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 msgr2=0x7ffa7c11ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa78ff9640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c1d0410 0x7ffa7c11ca20 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f70 tx=0x7ffa6c0019c0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.352+0000 7ffa7b7fe640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa6803b9f0 0x7ffa6803deb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c073c80 tx=0x7ffa70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 0x7ffa7c1366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c11d370 con 0x7ffa7c11cf60 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 0x7ffa7c1366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient: get_auth_request con 0x7ffa7c11cf60 auth_method 0 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_done global_id 74157 payload 824 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 1 -- 172.21.15.17:0/3591739341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffa7c132f40 con 0x7ffa7c11cf60 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa7bfff640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 0x7ffa7c1366c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c016000 tx=0x7ffa6c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c019040 con 0x7ffa7c11cf60 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 10 monclient: dump: 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7ffa6c01b620 con 0x7ffa7c11cf60 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 msgr2=0x7ffa7c1366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 0x7ffa7c1366c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c016000 tx=0x7ffa6c000ee0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 msgr2=0x7ffa7c1366c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa78ff9640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa7c11cf60 0x7ffa7c1366c0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c016000 tx=0x7ffa6c000ee0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 23139, 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:30.375 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.376 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.377 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:30.378 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.379 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.380 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.385 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.386 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.387 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.388 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:30.388 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa6803b9f0 msgr2=0x7ffa6803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffa6803b9f0 0x7ffa6803deb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c073c80 tx=0x7ffa70009040 comp rx=0 tx=0).stop 2024-09-15T21:23:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 10 monclient: shutdown 2024-09-15T21:23:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 msgr2=0x7ffa7c11c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa8248e640 1 --2- 172.21.15.17:0/3591739341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffa7c13a570 0x7ffa7c11c4e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa740ad000 tx=0x7ffa7401a510 comp rx=0 tx=0).stop 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa80a2b640 1 -- 172.21.15.17:0/3591739341 reap_dead start 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.356+0000 7ffa80a2b640 1 -- 172.21.15.17:0/3591739341 reap_dead start 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.360+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 shutdown_connections 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.360+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 >> 172.21.15.17:0/3591739341 conn(0x7ffa7c0a66a0 msgr2=0x7ffa7c139910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.360+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 shutdown_connections 2024-09-15T21:23:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.360+0000 7ffa8248e640 1 -- 172.21.15.17:0/3591739341 wait complete. 2024-09-15T21:23:30.390 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:23:30.390 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:23:30.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.500+0000 7f1948040640 10 monclient: get_monmap_and_config 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.500+0000 7f1948040640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.500+0000 7f1948040640 10 monclient: monmap: 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:30.505096+0000 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:30.505096+0000 2024-09-15T21:23:30.506 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:30.507 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.500+0000 7f1948040640 1 Processor -- start 2024-09-15T21:23:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.500+0000 7f1948040640 1 -- start start 2024-09-15T21:23:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.500+0000 7f1948040640 10 monclient: init 2024-09-15T21:23:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f19401340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f194009d6c0 0x7f1940138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): picked mon.noname-a con 0x7f194009d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940139390 0x7f194013b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): picked mon.noname-b con 0x7f1940139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194009e1b0 con 0x7f19401340c0 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f194009d520 con 0x7f194009d6c0 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f194009e490 con 0x7f1940139390 2024-09-15T21:23:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f1948040640 10 monclient(hunting): authenticate will time out at 1333.900447s 2024-09-15T21:23:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f194009d6c0 0x7f1940138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f194009d6c0 0x7f1940138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34752/0 (socket says 172.21.15.17:34752) 2024-09-15T21:23:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 -- 172.21.15.17:0/2065723555 learned_addr learned my addr 172.21.15.17:0/2065723555 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): get_auth_request con 0x7f194009d6c0 auth_method 0 2024-09-15T21:23:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940139390 0x7f194013b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): get_auth_request con 0x7f1940139390 auth_method 0 2024-09-15T21:23:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient(hunting): handle_auth_done global_id 74160 payload 274 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 -- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940139390 msgr2=0x7f194013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940139390 0x7f194013b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 -- 172.21.15.17:0/2065723555 >> v1:172.21.15.119:6789/0 conn(0x7f19401340c0 legacy=0x7f19401344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 monclient: found mon.noname-a 2024-09-15T21:23:30.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19455db640 1 -- 172.21.15.17:0/2065723555 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194009e770 con 0x7f194009d6c0 2024-09-15T21:23:30.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.504+0000 7f19465dd640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940139390 0x7f194013b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f19455db640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194009d6c0 0x7f1940138e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f193001e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652403f0aa6ee502 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 1 -- 172.21.15.17:0/2065723555 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930004230 con 0x7f194009d6c0 2024-09-15T21:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: dump: 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 5 monclient: authenticate success, global_id 74160 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 1 -- 172.21.15.17:0/2065723555 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19300043d0 con 0x7f194009d6c0 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 4 set_mon_vals no callback set 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 1 -- 172.21.15.17:0/2065723555 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930003820 con 0x7f194009d6c0 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1944dda640 10 monclient: dump: 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 10 monclient: shutdown 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 -- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194009d6c0 msgr2=0x7f1940138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194009d6c0 0x7f1940138e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f193001e8b0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 -- 172.21.15.17:0/2065723555 shutdown_connections 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940139390 0x7f194013b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 --2- 172.21.15.17:0/2065723555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f194009d6c0 0x7f1940138e50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 -- 172.21.15.17:0/2065723555 >> 172.21.15.17:0/2065723555 conn(0x7f19400a66a0 msgr2=0x7f19400a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 -- 172.21.15.17:0/2065723555 shutdown_connections 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.508+0000 7f1948040640 1 -- 172.21.15.17:0/2065723555 wait complete. 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: monmap: 2024-09-15T21:23:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 Processor -- start 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- start start 2024-09-15T21:23:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: init 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194009d6c0 0x7f19401cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): picked mon.noname-c con 0x7f194009d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401340c0 0x7f1940133090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): picked mon.noname-a con 0x7f19401340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1940139390 0x7f19401335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): picked mon.noname-b con 0x7f1940139390 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f194013e060 con 0x7f19401340c0 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f194013dee0 con 0x7f1940139390 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f194013e1e0 con 0x7f194009d6c0 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): authenticate will time out at 1333.909306s 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194009d6c0 0x7f19401cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401340c0 0x7f1940133090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194009d6c0 0x7f19401cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55906/0 (socket says 172.21.15.17:55906) 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 -- 172.21.15.17:0/983529869 learned_addr learned my addr 172.21.15.17:0/983529869 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): get_auth_request con 0x7f194009d6c0 auth_method 0 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): get_auth_request con 0x7f19401340c0 auth_method 0 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19465dd640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1940139390 0x7f19401335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19465dd640 10 monclient(hunting): get_auth_request con 0x7f1940139390 auth_method 0 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19465dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19465dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19465dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient(hunting): handle_auth_done global_id 44747 payload 824 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1940139390 msgr2=0x7f19401335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1940139390 0x7f19401335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401340c0 msgr2=0x7f1940133090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401340c0 0x7f1940133090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 monclient: found mon.noname-c 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1940073100 con 0x7f194009d6c0 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401340c0 0x7f1940133090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1945ddc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194009d6c0 0x7f19401cf7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c500 tx=0x7f193c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c017070 con 0x7f194009d6c0 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: dump: 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.533 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 5 monclient: authenticate success, global_id 44747 2024-09-15T21:23:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c0052d0 con 0x7f194009d6c0 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: _renew_subs 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194009d0f0 con 0x7f194009d6c0 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c003840 con 0x7f194009d6c0 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: dump: 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1937fff640 4 set_mon_vals no callback set 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: _renew_subs 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401211c0 con 0x7f194009d6c0 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 0x7f1940071640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f19401344a0 con 0x7f1940073c60 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 0x7f1940071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient: get_auth_request con 0x7f1940073c60 auth_method 0 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f193c01a070 con 0x7f194009d6c0 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 10 monclient(hunting): handle_auth_done global_id 44750 payload 824 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19401215a0 con 0x7f1940073c60 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f192403b9c0 0x7f192403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(164..164 src has 1..164) ==== 9921+0+0 (secure 0 0 0) 0x7f193c0503d0 con 0x7f194009d6c0 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19455db640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 0x7f1940071640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f194009d0a0 tx=0x7f193001b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300039d0 con 0x7f1940073c60 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.512+0000 7f19367fc640 10 monclient: dump: 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19465dd640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f192403b9c0 0x7f192403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19465dd640 10 monclient: get_auth_request con 0x7f192403b9c0 auth_method 0 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f19300b8040 con 0x7f1940073c60 2024-09-15T21:23:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 msgr2=0x7f1940071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 0x7f1940071640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f194009d0a0 tx=0x7f193001b650 comp rx=0 tx=0).stop 2024-09-15T21:23:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 msgr2=0x7f1940071640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19367fc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940073c60 0x7f1940071640 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f194009d0a0 tx=0x7f193001b650 comp rx=0 tx=0).stop 2024-09-15T21:23:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.516+0000 7f19465dd640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f192403b9c0 0x7f192403de80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1940134bd0 tx=0x7f193800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:30.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 0x7f1940071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f19400724d0 con 0x7f1940071b80 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 0x7f1940071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient: get_auth_request con 0x7f1940071b80 auth_method 0 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 10 monclient(hunting): handle_auth_done global_id 44753 payload 824 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 1 -- 172.21.15.17:0/983529869 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f19401dab60 con 0x7f1940071b80 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19455db640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 0x7f1940071f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f19300ad000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300a8710 con 0x7f1940071b80 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 10 monclient: dump: 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f19300a8710 con 0x7f1940071b80 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 msgr2=0x7f1940071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 0x7f1940071f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f19300ad000 comp rx=0 tx=0).stop 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 msgr2=0x7f1940071f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19367fc640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1940071b80 0x7f1940071f90 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f19300ad000 comp rx=0 tx=0).stop 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:30.548 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 179118, 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.549 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:30.550 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:30.551 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:30.552 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.553 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:30.555 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:30.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.557 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:30.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:30.561 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f192403b9c0 msgr2=0x7f192403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f192403b9c0 0x7f192403de80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1940134bd0 tx=0x7f193800b040 comp rx=0 tx=0).stop 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 10 monclient: shutdown 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194009d6c0 msgr2=0x7f19401cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 --2- 172.21.15.17:0/983529869 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f194009d6c0 0x7f19401cf7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c500 tx=0x7f193c00c9c0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19465dd640 1 -- 172.21.15.17:0/983529869 reap_dead start 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f19465dd640 1 -- 172.21.15.17:0/983529869 reap_dead start 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 shutdown_connections 2024-09-15T21:23:30.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 >> 172.21.15.17:0/983529869 conn(0x7f19400a66a0 msgr2=0x7f1940139770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 shutdown_connections 2024-09-15T21:23:30.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.520+0000 7f1948040640 1 -- 172.21.15.17:0/983529869 wait complete. 2024-09-15T21:23:30.564 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:23:30.564 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:23:30.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient: get_monmap_and_config 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient: monmap: 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:30.687984+0000 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:30.687984+0000 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.689 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.690 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:30.690 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:30.690 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:30.690 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:30.690 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:30.690 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 Processor -- start 2024-09-15T21:23:30.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 -- start start 2024-09-15T21:23:30.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient: init 2024-09-15T21:23:30.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f89400a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): picked mon.noname-a con 0x7f89400a5580 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): picked mon.noname-c con 0x7f89400a59d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89400a2c30 con 0x7f89400a4a90 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f89400a0f40 con 0x7f89400a5580 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f89400a19b0 con 0x7f89400a59d0 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f8945e3c640 10 monclient(hunting): authenticate will time out at 1334.083485s 2024-09-15T21:23:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34776/0 (socket says 172.21.15.17:34776) 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 1 -- 172.21.15.17:0/3151712444 learned_addr learned my addr 172.21.15.17:0/3151712444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): get_auth_request con 0x7f89400a5580 auth_method 0 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893ffff640 1 --2- 172.21.15.17:0/3151712444 >> v2:172.21.15.119:3300/0 conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893ffff640 10 monclient(hunting): get_auth_request con 0x7f89400a59d0 auth_method 0 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.684+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 10 monclient(hunting): handle_auth_done global_id 74166 payload 274 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 1 -- 172.21.15.17:0/3151712444 >> v2:172.21.15.119:3300/0 conn(0x7f89400a59d0 msgr2=0x7f894013f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 1 --2- 172.21.15.17:0/3151712444 >> v2:172.21.15.119:3300/0 conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 1 -- 172.21.15.17:0/3151712444 >> v1:172.21.15.119:6789/0 conn(0x7f89400a4a90 legacy=0x7f89400a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 1 monclient: found mon.noname-a 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 1 -- 172.21.15.17:0/3151712444 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89400a1c90 con 0x7f89400a5580 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893effd640 1 --2- 172.21.15.17:0/3151712444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013cd10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8934009870 tx=0x7f893401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a959a9d3d9fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 1 -- 172.21.15.17:0/3151712444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340b2070 con 0x7f89400a5580 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: dump: 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 1 -- 172.21.15.17:0/3151712444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89340a9070 con 0x7f89400a5580 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 1 -- 172.21.15.17:0/3151712444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340ad880 con 0x7f89400a5580 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 5 monclient: authenticate success, global_id 74166 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 4 set_mon_vals no callback set 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f893e7fc640 10 monclient: dump: 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 10 monclient: shutdown 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 -- 172.21.15.17:0/3151712444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 msgr2=0x7f894013cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 --2- 172.21.15.17:0/3151712444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013cd10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8934009870 tx=0x7f893401c440 comp rx=0 tx=0).stop 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 -- 172.21.15.17:0/3151712444 shutdown_connections 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 --2- 172.21.15.17:0/3151712444 >> v2:172.21.15.119:3300/0 conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 --2- 172.21.15.17:0/3151712444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 -- 172.21.15.17:0/3151712444 >> 172.21.15.17:0/3151712444 conn(0x7f89400aabb0 msgr2=0x7f89400a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 -- 172.21.15.17:0/3151712444 shutdown_connections 2024-09-15T21:23:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 1 -- 172.21.15.17:0/3151712444 wait complete. 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.688+0000 7f8945e3c640 10 monclient: monmap: 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 Processor -- start 2024-09-15T21:23:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 -- start start 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient: init 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 0x7f8940134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): picked mon.noname-c con 0x7f89400a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a5580 0x7f89401351b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): picked mon.noname-b con 0x7f89400a5580 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a59d0 0x7f89401cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): picked mon.noname-a con 0x7f89400a59d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f89401421d0 con 0x7f89400a59d0 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f894006f290 con 0x7f89400a5580 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8940142350 con 0x7f89400a4a90 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f8945e3c640 10 monclient(hunting): authenticate will time out at 1334.089112s 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 0x7f8940134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a59d0 0x7f89401cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 0x7f8940134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55926/0 (socket says 172.21.15.17:55926) 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): get_auth_request con 0x7f89400a59d0 auth_method 0 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 -- 172.21.15.17:0/913707834 learned_addr learned my addr 172.21.15.17:0/913707834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): get_auth_request con 0x7f89400a4a90 auth_method 0 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893effd640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a5580 0x7f89401351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893effd640 10 monclient(hunting): get_auth_request con 0x7f89400a5580 auth_method 0 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 msgr2=0x7f8940134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 msgr2=0x7f8940134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 0x7f8940134c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893f7fe640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 0x7f8940134c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient(hunting): handle_auth_done global_id 74169 payload 824 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 msgr2=0x7f8940134c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a4a90 0x7f8940134c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a5580 msgr2=0x7f89401351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a5580 0x7f89401351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 monclient: found mon.noname-a 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894013d480 con 0x7f89400a59d0 2024-09-15T21:23:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893effd640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a5580 0x7f89401351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:23:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.692+0000 7f893ffff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a59d0 0x7f89401cf530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8938009d90 tx=0x7f893800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.690+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v165) ==== 132+0+0 (secure 0 0 0) 0x7fbb3c012d00 con 0x7fbb4009d550 2024-09-15T21:23:30.708 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.690+0000 7fbb35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T21:23:30.709 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.690+0000 7fbb35ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938018020 con 0x7f89400a59d0 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.712 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8938014480 con 0x7f89400a59d0 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938014870 con 0x7f89400a59d0 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 5 monclient: authenticate success, global_id 74169 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 10 monclient: _renew_subs 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894013f400 con 0x7f89400a59d0 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893dffb640 4 set_mon_vals no callback set 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 10 monclient: _renew_subs 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894013d1a0 con 0x7f89400a59d0 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 0x7f89401cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8940076dd0 con 0x7f894013c240 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f893801b020 con 0x7f89400a59d0 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 0x7f89401cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient: get_auth_request con 0x7f894013c240 auth_method 0 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9921+0+0 (secure 0 0 0) 0x7f8938051e60 con 0x7f89400a59d0 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893effd640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893effd640 10 monclient: get_auth_request con 0x7f892c03b9c0 auth_method 0 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 10 monclient(hunting): handle_auth_done global_id 64360 payload 824 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f894013d960 con 0x7f894013c240 2024-09-15T21:23:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893f7fe640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 0x7f89401cfa70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f893000fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.696+0000 7f893effd640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f89340af000 tx=0x7f8934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89300050d0 con 0x7f894013c240 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f89300050d0 con 0x7f894013c240 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 msgr2=0x7f89401cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 0x7f89401cfa70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f893000fc00 comp rx=0 tx=0).stop 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 msgr2=0x7f89401cfa70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.700+0000 7f8927fff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894013c240 0x7f89401cfa70 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f893000fc00 comp rx=0 tx=0).stop 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f8945e3c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f8945e3c640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 0x7f894009d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f8945e3c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89401d03c0 con 0x7f89401cffb0 2024-09-15T21:23:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f893f7fe640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 0x7f894009d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f893f7fe640 10 monclient: get_auth_request con 0x7f89401cffb0 auth_method 0 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f893f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f893f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.704+0000 7f893f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893f7fe640 10 monclient(hunting): handle_auth_done global_id 64363 payload 824 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893f7fe640 1 -- 172.21.15.17:0/913707834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f89400a4e70 con 0x7f89401cffb0 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893f7fe640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 0x7f894009d520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8930001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893000d9d0 con 0x7f89401cffb0 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:23:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.720 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f893000d9d0 con 0x7f89401cffb0 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 msgr2=0x7f894009d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 0x7f894009d520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8930001130 comp rx=0 tx=0).stop 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 msgr2=0x7f894009d520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8927fff640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89401cffb0 0x7f894009d520 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8930001130 comp rx=0 tx=0).stop 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 88966, 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:30.734 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 msgr2=0x7f892c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f89340af000 tx=0x7f8934008040 comp rx=0 tx=0).stop 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 10 monclient: shutdown 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a59d0 msgr2=0x7f89401cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 --2- 172.21.15.17:0/913707834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a59d0 0x7f89401cf530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8938009d90 tx=0x7f893800c490 comp rx=0 tx=0).stop 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893ffff640 1 -- 172.21.15.17:0/913707834 reap_dead start 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f893ffff640 1 -- 172.21.15.17:0/913707834 reap_dead start 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 shutdown_connections 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 >> 172.21.15.17:0/913707834 conn(0x7f89400aabb0 msgr2=0x7f8940137a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 shutdown_connections 2024-09-15T21:23:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.708+0000 7f8945e3c640 1 -- 172.21.15.17:0/913707834 wait complete. 2024-09-15T21:23:30.737 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:23:30.737 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:23:30.737 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:23:30.737 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:23:30.737 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:23:30.737 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:23:30.737 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:23:30.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 msgr2=0x7fbb40131120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 msgr2=0x7fbb40131120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c4d0 tx=0x7fbb3c00c990 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c4d0 tx=0x7fbb3c00c990 comp rx=0 tx=0).stop 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 msgr2=0x7fbb40131120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb4009d550 0x7fbb40131120 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24047a80 0x7fbb24049e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): picked mon.b con 0x7fbb24047a80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb2404c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): picked mon.a con 0x7fbb4009e040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb2404ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): picked mon.c con 0x7fbb4009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2404f960 con 0x7fbb4009e040 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2404fae0 con 0x7fbb24047a80 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbb24054070 con 0x7fbb4009fae0 2024-09-15T21:23:30.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24047a80 msgr2=0x7fbb24049e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24047a80 0x7fbb24049e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb2404c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb2404ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): get_auth_request con 0x7fbb4009e040 auth_method 0 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): get_auth_request con 0x7fbb4009fae0 auth_method 0 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45d77640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient(hunting): handle_auth_done global_id 64336 payload 293 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24047a80 msgr2=0x7fbb24049e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24047a80 0x7fbb24049e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 msgr2=0x7fbb2404c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb2404c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 monclient: found mon.c 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7fbb24054240 con 0x7fbb4009fae0 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb45576640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb2404ef50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c04e0a0 tx=0x7fbb3c007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c04a020 con 0x7fbb4009fae0 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: dump: 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb3c003b20 con 0x7fbb4009fae0 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb35ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.742+0000 7fbb36ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.746+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbb3c04d020 con 0x7fbb4009fae0 2024-09-15T21:23:30.760 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:23:30.760 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:23:30.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:23:30.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.810+0000 7fbb477da640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T21:23:30.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.810+0000 7fbb477da640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T21:23:30.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.810+0000 7fbb477da640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:30.810+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) -- 0x7fbb18005110 con 0x7fbb4009fae0 2024-09-15T21:23:30.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient: get_monmap_and_config 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient: monmap: 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:30.871850+0000 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:30.871850+0000 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:30.874 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:30.874 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:30.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 Processor -- start 2024-09-15T21:23:30.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 -- start start 2024-09-15T21:23:30.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient: init 2024-09-15T21:23:30.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-09-15T21:23:30.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85f40a0890 0x7f85f40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): picked mon.noname-a con 0x7f85f40a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f85f409ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a11b0 0x7f85f4135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): picked mon.noname-b con 0x7f85f40a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85f40a3130 con 0x7f85f409ffc0 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f85f4073df0 con 0x7f85f40a0890 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f85f40a1bf0 con 0x7f85f40a11b0 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f960e640 10 monclient(hunting): authenticate will time out at 1334.267165s 2024-09-15T21:23:30.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f27fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f85f409ffc0 0x7f85f4135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43572/0 (socket says 172.21.15.17:43572) 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f27fc640 1 -- 172.21.15.17:0/2079285105 learned_addr learned my addr 172.21.15.17:0/2079285105 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2079285105 >> v2:172.21.15.17:3300/0 conn(0x7f85f40a0890 0x7f85f40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 1 --2- 172.21.15.17:0/2079285105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a11b0 0x7f85f4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): get_auth_request con 0x7f85f40a0890 auth_method 0 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): get_auth_request con 0x7f85f40a11b0 auth_method 0 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2079285105 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2682445451 0 0) 0x7f85f40a3130 con 0x7f85f409ffc0 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f1ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f1ffb640 10 monclient(hunting): my global_id is 74175 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2079285105 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85dc003530 con 0x7f85f409ffc0 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.868+0000 7f85f37fe640 10 monclient(hunting): handle_auth_done global_id 44759 payload 274 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 1 -- 172.21.15.17:0/2079285105 >> v2:172.21.15.17:3300/0 conn(0x7f85f40a0890 msgr2=0x7f85f40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 1 --2- 172.21.15.17:0/2079285105 >> v2:172.21.15.17:3300/0 conn(0x7f85f40a0890 0x7f85f40a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 1 -- 172.21.15.17:0/2079285105 >> v1:172.21.15.17:6789/0 conn(0x7f85f409ffc0 legacy=0x7f85f4135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 1 monclient: found mon.noname-b 2024-09-15T21:23:30.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:30.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 1 -- 172.21.15.17:0/2079285105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f40a15e0 con 0x7f85f40a11b0 2024-09-15T21:23:30.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2079285105 >> v2:172.21.15.17:3300/0 conn(0x7f85f40a0890 0x7f85f40a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f37fe640 1 --2- 172.21.15.17:0/2079285105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a11b0 0x7f85f4135d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009dc0 tx=0x7f85ec01c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdff0292ea0bce2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2079285105 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0b2070 con 0x7f85f40a11b0 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: dump: 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.885 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2079285105 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec0a57e0 con 0x7f85f40a11b0 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 5 monclient: authenticate success, global_id 44759 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2079285105 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0a5bb0 con 0x7f85f40a11b0 2024-09-15T21:23:30.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f1ffb640 10 monclient: dump: 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:30.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 4 set_mon_vals no callback set 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 10 monclient: shutdown 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 -- 172.21.15.17:0/2079285105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a11b0 msgr2=0x7f85f4135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 --2- 172.21.15.17:0/2079285105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a11b0 0x7f85f4135d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009dc0 tx=0x7f85ec01c810 comp rx=0 tx=0).stop 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 -- 172.21.15.17:0/2079285105 shutdown_connections 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 --2- 172.21.15.17:0/2079285105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a11b0 0x7f85f4135d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 --2- 172.21.15.17:0/2079285105 >> v2:172.21.15.17:3300/0 conn(0x7f85f40a0890 0x7f85f40a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 -- 172.21.15.17:0/2079285105 >> 172.21.15.17:0/2079285105 conn(0x7f85f40a66a0 msgr2=0x7f85f40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 -- 172.21.15.17:0/2079285105 shutdown_connections 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 1 -- 172.21.15.17:0/2079285105 wait complete. 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 10 monclient: build_initial_monmap 2024-09-15T21:23:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.872+0000 7f85f960e640 10 monclient: monmap: 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 Processor -- start 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- start start 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: init 2024-09-15T21:23:30.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 0x7f85f41cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): picked mon.noname-a con 0x7f85f409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a0890 0x7f85f41352c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): picked mon.noname-c con 0x7f85f40a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85f40a11b0 0x7f85f4135800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): picked mon.noname-b con 0x7f85f40a11b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4140510 con 0x7f85f409ffc0 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f85f41364d0 con 0x7f85f40a11b0 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f85f4140690 con 0x7f85f40a0890 2024-09-15T21:23:30.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient(hunting): authenticate will time out at 1334.272702s 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 0x7f85f41cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 0x7f85f41cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34798/0 (socket says 172.21.15.17:34798) 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 -- 172.21.15.17:0/402970694 learned_addr learned my addr 172.21.15.17:0/402970694 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): get_auth_request con 0x7f85f409ffc0 auth_method 0 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f37fe640 1 -- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85f40a11b0 msgr2=0x7f85f4135800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a0890 0x7f85f41352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f37fe640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85f40a11b0 0x7f85f4135800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): get_auth_request con 0x7f85f40a0890 auth_method 0 2024-09-15T21:23:30.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient(hunting): handle_auth_done global_id 74178 payload 824 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 -- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a0890 msgr2=0x7f85f41352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a0890 0x7f85f41352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 -- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85f40a11b0 msgr2=0x7f85f4135800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85f40a11b0 0x7f85f4135800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 monclient: found mon.noname-a 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 -- 172.21.15.17:0/402970694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f406f530 con 0x7f85f409ffc0 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 0x7f85f41cf630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f85e400ebe0 tx=0x7f85e400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4017070 con 0x7f85f409ffc0 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: dump: 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.893 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e400cc30 con 0x7f85f409ffc0 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4013470 con 0x7f85f409ffc0 2024-09-15T21:23:30.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 5 monclient: authenticate success, global_id 74178 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _renew_subs 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f409d0f0 con 0x7f85f409ffc0 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 10 monclient: dump: 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:30.895 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f17fa640 4 set_mon_vals no callback set 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _renew_subs 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f4121c70 con 0x7f85f409ffc0 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f85e401a070 con 0x7f85f409ffc0 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f40a0c70 con 0x7f85f409ffc0 2024-09-15T21:23:30.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f85e003b970 0x7f85e003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9921+0+0 (secure 0 0 0) 0x7f85e4050ea0 con 0x7f85f409ffc0 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f85e003b970 0x7f85e003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.876+0000 7f85f27fc640 10 monclient: get_auth_request con 0x7f85e003b970 auth_method 0 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.880+0000 7f85f27fc640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f85e003b970 0x7f85e003de30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f85e8007fd0 tx=0x7f85e8006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.880+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f85e404e490 con 0x7f85f409ffc0 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.880+0000 7f85db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:30.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:30.880+0000 7f85db7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85f960e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:23:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85f960e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85f960e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f85f406d020 con 0x7f85f409ffc0 2024-09-15T21:23:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85db7fe640 1 -- 172.21.15.17:0/402970694 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f85e4013ae0 con 0x7f85f409ffc0 2024-09-15T21:23:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.120+0000 7f85db7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:31.126 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f85e003b970 msgr2=0x7f85e003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f85e003b970 0x7f85e003de30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f85e8007fd0 tx=0x7f85e8006ab0 comp rx=0 tx=0).stop 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 10 monclient: shutdown 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 msgr2=0x7f85f41cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 0x7f85f41cf630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f85e400ebe0 tx=0x7f85e400c520 comp rx=0 tx=0).stop 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 shutdown_connections 2024-09-15T21:23:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f85e003b970 0x7f85e003de30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85f40a11b0 0x7f85f4135800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f85f40a0890 0x7f85f41352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 --2- 172.21.15.17:0/402970694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f409ffc0 0x7f85f41cf630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 >> 172.21.15.17:0/402970694 conn(0x7f85f40a66a0 msgr2=0x7f85f40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 shutdown_connections 2024-09-15T21:23:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:31.124+0000 7f85f960e640 1 -- 172.21.15.17:0/402970694 wait complete. 2024-09-15T21:23:32.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:32.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:32.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:32.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:32.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:32.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:32.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:32.850+0000 7fbb34ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:33.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:33.846+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:33.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:33.846+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:33.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:33.846+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:34.144 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:23:34.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient: get_monmap_and_config 2024-09-15T21:23:34.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient: build_initial_monmap 2024-09-15T21:23:34.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient: monmap: 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:34.272570+0000 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:34.272570+0000 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:34.274 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 Processor -- start 2024-09-15T21:23:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 -- start start 2024-09-15T21:23:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient: init 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ed81340c0 0x7f0ed81344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): picked mon.noname-c con 0x7f0ed81340c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0ed809d6c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 0x7f0ed813be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): picked mon.noname-b con 0x7f0ed80a0fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed81349e0 con 0x7f0ed809d6c0 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed809d520 con 0x7f0ed81340c0 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed809e1b0 con 0x7f0ed80a0fa0 2024-09-15T21:23:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.268+0000 7f0ede281640 10 monclient(hunting): authenticate will time out at 1337.667993s 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed77fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f0ed809d6c0 legacy=0x7f0ed80a0890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 0x7f0ed813be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed7fff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f0ed81340c0 msgr2=0x7f0ed81344a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ed81340c0 0x7f0ed81344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 0x7f0ed813be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55954/0 (socket says 172.21.15.17:55954) 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 -- 172.21.15.17:0/2383511342 learned_addr learned my addr 172.21.15.17:0/2383511342 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): get_auth_request con 0x7f0ed80a0fa0 auth_method 0 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:34.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient(hunting): handle_auth_done global_id 44765 payload 274 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 -- 172.21.15.17:0/2383511342 >> v2:172.21.15.119:3300/0 conn(0x7f0ed81340c0 msgr2=0x7f0ed81344a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 --2- 172.21.15.17:0/2383511342 >> v2:172.21.15.119:3300/0 conn(0x7f0ed81340c0 0x7f0ed81344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 -- 172.21.15.17:0/2383511342 >> v1:172.21.15.119:6789/0 conn(0x7f0ed809d6c0 legacy=0x7f0ed80a0890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient: _finish_hunting 0 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 monclient: found mon.noname-b 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 -- 172.21.15.17:0/2383511342 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed809e490 con 0x7f0ed80a0fa0 2024-09-15T21:23:34.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0edc81e640 1 --2- 172.21.15.17:0/2383511342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 0x7f0ed813be70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009e50 tx=0x7f0ed001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33841e62ffbe29c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2383511342 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00a55e0 con 0x7f0ed80a0fa0 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: dump: 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2383511342 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed00a5780 con 0x7f0ed80a0fa0 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2383511342 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00a5b50 con 0x7f0ed80a0fa0 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ede281640 5 monclient: authenticate success, global_id 44765 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ede281640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ede281640 4 set_mon_vals no callback set 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.272+0000 7f0ed6ffd640 10 monclient: dump: 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: shutdown 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2383511342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 msgr2=0x7f0ed813be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 --2- 172.21.15.17:0/2383511342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 0x7f0ed813be70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009e50 tx=0x7f0ed001c910 comp rx=0 tx=0).stop 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2383511342 shutdown_connections 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 --2- 172.21.15.17:0/2383511342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed80a0fa0 0x7f0ed813be70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 --2- 172.21.15.17:0/2383511342 >> v2:172.21.15.119:3300/0 conn(0x7f0ed81340c0 0x7f0ed81344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2383511342 >> 172.21.15.17:0/2383511342 conn(0x7f0ed80a66a0 msgr2=0x7f0ed80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2383511342 shutdown_connections 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2383511342 wait complete. 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: build_initial_monmap 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: monmap: 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 Processor -- start 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- start start 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: init 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed809d6c0 0x7f0ed813d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): picked mon.noname-b con 0x7f0ed809d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 0x7f0ed8139ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): picked mon.noname-a con 0x7f0ed80a0fa0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 0x7f0ed813a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): picked mon.noname-c con 0x7f0ed81340c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed813c530 con 0x7f0ed80a0fa0 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed813c3b0 con 0x7f0ed809d6c0 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed813c6b0 con 0x7f0ed81340c0 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient(hunting): authenticate will time out at 1337.673724s 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed7fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed809d6c0 msgr2=0x7f0ed813d3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed809d6c0 0x7f0ed813d3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 0x7f0ed8139ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 0x7f0ed813a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 0x7f0ed8139ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34810/0 (socket says 172.21.15.17:34810) 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 0x7f0ed813a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55962/0 (socket says 172.21.15.17:55962) 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 1 -- 172.21.15.17:0/2086007238 learned_addr learned my addr 172.21.15.17:0/2086007238 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): get_auth_request con 0x7f0ed80a0fa0 auth_method 0 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): get_auth_request con 0x7f0ed81340c0 auth_method 0 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient(hunting): handle_auth_done global_id 44768 payload 824 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 -- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed809d6c0 msgr2=0x7f0ed813d3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed809d6c0 0x7f0ed813d3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 -- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 msgr2=0x7f0ed8139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 0x7f0ed8139ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient: _finish_hunting 0 2024-09-15T21:23:34.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 monclient: found mon.noname-c 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 -- 172.21.15.17:0/2086007238 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed811ce80 con 0x7f0ed81340c0 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 0x7f0ed8139ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0edc81e640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 0x7f0ed813a0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00ab000 tx=0x7f0ed00196b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00bc030 con 0x7f0ed81340c0 2024-09-15T21:23:34.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: dump: 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:23:34.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 5 monclient: authenticate success, global_id 44768 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed00b8470 con 0x7f0ed81340c0 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _renew_subs 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00b8820 con 0x7f0ed81340c0 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed811d160 con 0x7f0ed81340c0 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 10 monclient: dump: 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed67fc640 4 set_mon_vals no callback set 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _renew_subs 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed813dea0 con 0x7f0ed81340c0 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:34.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed81344a0 con 0x7f0ed81340c0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.276+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0ed00bf070 con 0x7f0ed81340c0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.280+0000 7f0ed4ff9640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0eb803b970 0x7f0eb803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.280+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(165..165 src has 1..165) ==== 9921+0+0 (secure 0 0 0) 0x7f0ed00ebdf0 con 0x7f0ed81340c0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.280+0000 7f0ed7fff640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0eb803b970 0x7f0eb803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.280+0000 7f0ed7fff640 10 monclient: get_auth_request con 0x7f0eb803b970 auth_method 0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.280+0000 7f0ed7fff640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0eb803b970 0x7f0eb803de30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0003940 tx=0x7f0ec0006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:34.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.284+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0ed0016620 con 0x7f0ed81340c0 2024-09-15T21:23:34.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.284+0000 7f0ed4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:34.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.284+0000 7f0ed4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ede281640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:23:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ede281640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ede281640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0ed811c4e0 con 0x7f0ed81340c0 2024-09-15T21:23:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ed4ff9640 1 -- 172.21.15.17:0/2086007238 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f0ed00adcf0 con 0x7f0ed81340c0 2024-09-15T21:23:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ed4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.512+0000 7f0ed4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:34.517 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:23:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0eb803b970 msgr2=0x7f0eb803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0eb803b970 0x7f0eb803de30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0003940 tx=0x7f0ec0006d90 comp rx=0 tx=0).stop 2024-09-15T21:23:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 10 monclient: shutdown 2024-09-15T21:23:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 msgr2=0x7f0ed813a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 0x7f0ed813a0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00ab000 tx=0x7f0ed00196b0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 shutdown_connections 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0eb803b970 0x7f0eb803de30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ed81340c0 0x7f0ed813a0e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed80a0fa0 0x7f0ed8139ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 --2- 172.21.15.17:0/2086007238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed809d6c0 0x7f0ed813d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 >> 172.21.15.17:0/2086007238 conn(0x7f0ed80a66a0 msgr2=0x7f0ed809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 shutdown_connections 2024-09-15T21:23:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:34.516+0000 7f0ede281640 1 -- 172.21.15.17:0/2086007238 wait complete. 2024-09-15T21:23:34.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:34.846+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:34.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:34.846+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:34.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:34.846+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:35.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:35.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:35.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:35.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:35.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:35.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:36.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:36.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:36.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:36.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:36.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:36.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:37.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:23:37.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient: get_monmap_and_config 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient: build_initial_monmap 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient: monmap: 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:37.667843+0000 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:37.667843+0000 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:37.670 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:37.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 Processor -- start 2024-09-15T21:23:37.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 -- start start 2024-09-15T21:23:37.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient: init 2024-09-15T21:23:37.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5fc41378c0 0x7f5fc4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): picked mon.noname-c con 0x7f5fc41378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fc41381e0 0x7f5fc40a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): picked mon.noname-a con 0x7f5fc41381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5fc40a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc40a1d00 con 0x7f5fc40a5040 2024-09-15T21:23:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc40a1fe0 con 0x7f5fc41381e0 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc40a22c0 con 0x7f5fc41378c0 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcc4cf640 10 monclient(hunting): authenticate will time out at 1341.063474s 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fcaa6c640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f5fc40a5040 legacy=0x7f5fc40a54d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fca26b640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f5fc41378c0 msgr2=0x7f5fc4137ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fca26b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5fc41378c0 0x7f5fc4137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fc41381e0 0x7f5fc40a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fc41381e0 0x7f5fc40a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60952/0 (socket says 172.21.15.17:60952) 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 1 -- 172.21.15.17:0/4022146305 learned_addr learned my addr 172.21.15.17:0/4022146305 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): get_auth_request con 0x7f5fc41381e0 auth_method 0 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.664+0000 7f5fc9a6a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 10 monclient(hunting): handle_auth_done global_id 74184 payload 274 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 1 -- 172.21.15.17:0/4022146305 >> v2:172.21.15.119:3300/0 conn(0x7f5fc41378c0 msgr2=0x7f5fc4137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 1 --2- 172.21.15.17:0/4022146305 >> v2:172.21.15.119:3300/0 conn(0x7f5fc41378c0 0x7f5fc4137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 1 -- 172.21.15.17:0/4022146305 >> v1:172.21.15.119:6789/0 conn(0x7f5fc40a5040 legacy=0x7f5fc40a54d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 10 monclient: _finish_hunting 0 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 1 monclient: found mon.noname-a 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 1 -- 172.21.15.17:0/4022146305 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc40a4490 con 0x7f5fc41381e0 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9a6a640 1 --2- 172.21.15.17:0/4022146305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc41381e0 0x7f5fc40a4b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009e50 tx=0x7f5fc001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec0a22db6ae358f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 1 -- 172.21.15.17:0/4022146305 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00a5770 con 0x7f5fc41381e0 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: dump: 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: _finish_auth 0 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: _check_auth_tickets 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 1 -- 172.21.15.17:0/4022146305 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc00a5910 con 0x7f5fc41381e0 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 5 monclient: authenticate success, global_id 74184 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 1 -- 172.21.15.17:0/4022146305 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00a5ce0 con 0x7f5fc41381e0 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 4 set_mon_vals no callback set 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fc9269640 10 monclient: dump: 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 10 monclient: shutdown 2024-09-15T21:23:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/4022146305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc41381e0 msgr2=0x7f5fc40a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/4022146305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc41381e0 0x7f5fc40a4b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009e50 tx=0x7f5fc001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/4022146305 shutdown_connections 2024-09-15T21:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/4022146305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc41381e0 0x7f5fc40a4b00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/4022146305 >> v2:172.21.15.119:3300/0 conn(0x7f5fc41378c0 0x7f5fc4137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/4022146305 >> 172.21.15.17:0/4022146305 conn(0x7f5fc40aac40 msgr2=0x7f5fc40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/4022146305 shutdown_connections 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/4022146305 wait complete. 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 10 monclient: build_initial_monmap 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.668+0000 7f5fcc4cf640 10 monclient: monmap: 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 Processor -- start 2024-09-15T21:23:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- start start 2024-09-15T21:23:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: init 2024-09-15T21:23:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:23:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 0x7f5fc41cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): picked mon.noname-a con 0x7f5fc40a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc41378c0 0x7f5fc41d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): picked mon.noname-b con 0x7f5fc41378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 0x7f5fc413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): picked mon.noname-c con 0x7f5fc41381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc406f340 con 0x7f5fc40a5040 2024-09-15T21:23:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc406f1c0 con 0x7f5fc41378c0 2024-09-15T21:23:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc41425c0 con 0x7f5fc41381e0 2024-09-15T21:23:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient(hunting): authenticate will time out at 1341.068730s 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 0x7f5fc413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 0x7f5fc41cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fc9a6a640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc41378c0 msgr2=0x7f5fc41d0270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 0x7f5fc413bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49830/0 (socket says 172.21.15.17:49830) 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fc9a6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc41378c0 0x7f5fc41d0270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 1 -- 172.21.15.17:0/1466014290 learned_addr learned my addr 172.21.15.17:0/1466014290 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): get_auth_request con 0x7f5fc41381e0 auth_method 0 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 0x7f5fc41cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60958/0 (socket says 172.21.15.17:60958) 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): get_auth_request con 0x7f5fc40a5040 auth_method 0 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient(hunting): handle_auth_done global_id 74187 payload 824 2024-09-15T21:23:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 -- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 msgr2=0x7f5fc413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 0x7f5fc413bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 -- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc41378c0 msgr2=0x7f5fc41d0270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc41378c0 0x7f5fc41d0270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 monclient: found mon.noname-a 2024-09-15T21:23:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 -- 172.21.15.17:0/1466014290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc41216e0 con 0x7f5fc40a5040 2024-09-15T21:23:37.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcaa6c640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 0x7f5fc413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:37.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fca26b640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 0x7f5fc41cfaa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800b430 tx=0x7f5fb800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:37.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8016070 con 0x7f5fc40a5040 2024-09-15T21:23:37.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:37.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:37.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: dump: 2024-09-15T21:23:37.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:37.699 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:37.699 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.699 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.699 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:37.700 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:37.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:37.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:37.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:37.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:37.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:37.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:37.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:37.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb8003d20 con 0x7f5fc40a5040 2024-09-15T21:23:37.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:37.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80052b0 con 0x7f5fc40a5040 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 10 monclient: dump: 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fc8a68640 4 set_mon_vals no callback set 2024-09-15T21:23:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 5 monclient: authenticate success, global_id 74187 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _renew_subs 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc4120e40 con 0x7f5fc40a5040 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _renew_subs 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc4137390 con 0x7f5fc40a5040 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc4137ca0 con 0x7f5fc40a5040 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5fb8019070 con 0x7f5fc40a5040 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fa803b970 0x7f5fa803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9921+0+0 (secure 0 0 0) 0x7f5fb80501e0 con 0x7f5fc40a5040 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fc9a6a640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fa803b970 0x7f5fa803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.672+0000 7f5fc9a6a640 10 monclient: get_auth_request con 0x7f5fa803b970 auth_method 0 2024-09-15T21:23:37.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.676+0000 7f5fc9a6a640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fa803b970 0x7f5fa803de30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00ae000 tx=0x7f5fc00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:37.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.676+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5fb800f300 con 0x7f5fc40a5040 2024-09-15T21:23:37.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.676+0000 7f5fb6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:37.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.676+0000 7f5fb6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:37.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:37.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:37.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:37.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:37.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:37.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:37.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fcc4cf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:23:37.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fcc4cf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:37.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fcc4cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:37.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5fc4121360 con 0x7f5fc40a5040 2024-09-15T21:23:37.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fb6ffd640 1 -- 172.21.15.17:0/1466014290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f5fb804d7d0 con 0x7f5fc40a5040 2024-09-15T21:23:37.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fb6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:37.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fb6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:37.921 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:23:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fa803b970 msgr2=0x7f5fa803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.916+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fa803b970 0x7f5fa803de30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00ae000 tx=0x7f5fc00a7070 comp rx=0 tx=0).stop 2024-09-15T21:23:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 10 monclient: shutdown 2024-09-15T21:23:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 msgr2=0x7f5fc41cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 0x7f5fc41cfaa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800b430 tx=0x7f5fb800b8f0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 shutdown_connections 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fa803b970 0x7f5fa803de30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5fc41381e0 0x7f5fc413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fc41378c0 0x7f5fc41d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 --2- 172.21.15.17:0/1466014290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fc40a5040 0x7f5fc41cfaa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 >> 172.21.15.17:0/1466014290 conn(0x7f5fc40aac40 msgr2=0x7f5fc413efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 shutdown_connections 2024-09-15T21:23:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:37.920+0000 7f5fcc4cf640 1 -- 172.21.15.17:0/1466014290 wait complete. 2024-09-15T21:23:38.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:38.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:38.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:38.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:38.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:38.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:39.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:39.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:39.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:39.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:39.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:39.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:40.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:40.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:40.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:40.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:40.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:40.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:40.941 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:23:41.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient: get_monmap_and_config 2024-09-15T21:23:41.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient: build_initial_monmap 2024-09-15T21:23:41.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient: monmap: 2024-09-15T21:23:41.076 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:41.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:41.075378+0000 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:41.075378+0000 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:41.077 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:41.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 1 Processor -- start 2024-09-15T21:23:41.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 1 -- start start 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient: init 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9e0c09d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient(hunting): picked mon.noname-c con 0x7f9e0c09d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9e0c09dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9e0c134d70 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c0a3490 con 0x7f9e0c134d70 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c0a0320 con 0x7f9e0c09dd20 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0c0a1bc0 con 0x7f9e0c09d400 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e12179640 10 monclient(hunting): authenticate will time out at 1344.470915s 2024-09-15T21:23:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e0affd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9e0c134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e0b7fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9e0c09d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e0b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9e0c09d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e0bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9e0c134d70 0x7f9e0c13c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45742/0 (socket says 172.21.15.17:45742) 2024-09-15T21:23:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.072+0000 7f9e0bfff640 1 -- 172.21.15.17:0/3488033100 learned_addr learned my addr 172.21.15.17:0/3488033100 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:23:41.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.276+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9e0c09d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:41.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.276+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9e0c134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:41.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.276+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9e0c09d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:23:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.680+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9e0c134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.680+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9e0c09d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:41.680+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9e0c09d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:23:41.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:41.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:41.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:41.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:41.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:41.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:42.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:42.484+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9e0c134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:42.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:42.484+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9e0c09d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:42.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:42.484+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9e0c09d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:23:42.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:42.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:42.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:42.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:42.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:42.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:42.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:42.850+0000 7fbb34ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:43.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:43.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:43.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:43.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:43.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:43.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:44.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): tick 2024-09-15T21:23:44.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9e0c09d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9e0c09d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9e0c134660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.17:6789/0 conn(0x7f9e0c134d70 legacy=0x7f9e0c13c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient: _add_conns ranks=[3,2] 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): picked mon.noname-a con 0x7f9df80058e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): picked mon.noname-b con 0x7f9df80083f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:44.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0c0a1bc0 con 0x7f9df80058e0 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9df800afe0 con 0x7f9df80083f0 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e097fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request con 0x7f9df80058e0 auth_method 0 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 10 monclient(hunting): get_auth_request con 0x7f9df80083f0 auth_method 0 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:44.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:44.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:44.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:23:44.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:44.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:44.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:44.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.072+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.076+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:44.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.076+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 10 monclient(hunting): get_auth_request con 0x7f9df80083f0 auth_method 0 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request con 0x7f9df80058e0 auth_method 0 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:44.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.280+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:23:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 10 monclient(hunting): get_auth_request con 0x7f9df80083f0 auth_method 0 2024-09-15T21:23:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request con 0x7f9df80058e0 auth_method 0 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:23:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:44.684+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:23:44.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:44.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:44.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:44.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:44.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:44.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 10 monclient(hunting): get_auth_request con 0x7f9df80083f0 auth_method 0 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request con 0x7f9df80058e0 auth_method 0 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:23:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:45.488+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:23:45.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:45.850+0000 7fbb34ff9640 10 monclient: tick 2024-09-15T21:23:45.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:45.850+0000 7fbb34ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:45.854 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:45.850+0000 7fbb34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:46.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.738+0000 7fbb35ffb640 1 -- 172.21.15.119:0/3353392609 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v167) ==== 132+0+0 (secure 0 0 0) 0x7fbb3c03bdd0 con 0x7fbb4009fae0 2024-09-15T21:23:46.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.738+0000 7fbb35ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T21:23:46.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.738+0000 7fbb35ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:46.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbb2403b9c0 msgr2=0x7fbb2403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:46.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbb2403b9c0 0x7fbb2403de80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbb300b1000 tx=0x7fbb300a9070 comp rx=0 tx=0).stop 2024-09-15T21:23:46.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 10 monclient: shutdown 2024-09-15T21:23:46.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:46.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 msgr2=0x7fbb2404ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:46.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb2404ef50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c04e0a0 tx=0x7fbb3c007ea0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 shutdown_connections 2024-09-15T21:23:46.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbb4009fae0 0x7fbb2404ef50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb4009e040 0x7fbb2404c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24047a80 0x7fbb24049e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 --2- 172.21.15.119:0/3353392609 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbb2403b9c0 0x7fbb2403de80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 >> 172.21.15.119:0/3353392609 conn(0x7fbb400a66e0 msgr2=0x7fbb40072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:46.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 shutdown_connections 2024-09-15T21:23:46.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.742+0000 7fbb477da640 1 -- 172.21.15.119:0/3353392609 wait complete. 2024-09-15T21:23:46.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:46.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep globster 2024-09-15T21:23:46.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:46.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '7\.' 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient: get_monmap_and_config 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient: build_initial_monmap 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient: monmap: 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:46.885860+0000 2024-09-15T21:23:46.886 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:46.885860+0000 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:46.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 Processor -- start 2024-09-15T21:23:46.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 -- start start 2024-09-15T21:23:46.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient: init 2024-09-15T21:23:46.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:46.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-09-15T21:23:46.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99500a57e0 0x7f99500721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:46.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): picked mon.noname-c con 0x7f99500a57e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:46.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f99500a5bc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99501392d0 0x7f995013b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): picked mon.noname-a con 0x7f99501392d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995006fe50 con 0x7f99500a5bc0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9950071790 con 0x7f99501392d0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9950070ef0 con 0x7f99500a57e0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f99579e4640 10 monclient(hunting): authenticate will time out at 1345.683343s 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955780640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f99500a57e0 msgr2=0x7f99500721c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955780640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99500a57e0 0x7f99500721c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99501392d0 0x7f995013b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:46.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99501392d0 0x7f995013b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36984/0 (socket says 172.21.15.119:36984) 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 1 -- 172.21.15.119:0/2479600706 learned_addr learned my addr 172.21.15.119:0/2479600706 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 10 monclient(hunting): get_auth_request con 0x7f99501392d0 auth_method 0 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.882+0000 7f9955f81640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 10 monclient(hunting): handle_auth_done global_id 74190 payload 274 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 1 -- 172.21.15.119:0/2479600706 >> v2:172.21.15.119:3300/0 conn(0x7f99500a57e0 msgr2=0x7f99500721c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 1 --2- 172.21.15.119:0/2479600706 >> v2:172.21.15.119:3300/0 conn(0x7f99500a57e0 0x7f99500721c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 1 -- 172.21.15.119:0/2479600706 >> v1:172.21.15.17:6789/0 conn(0x7f99500a5bc0 legacy=0x7f99500a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 10 monclient: _finish_hunting 0 2024-09-15T21:23:46.891 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 1 monclient: found mon.noname-a 2024-09-15T21:23:46.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:46.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:46.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 1 -- 172.21.15.119:0/2479600706 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995006d710 con 0x7f99501392d0 2024-09-15T21:23:46.892 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9955f81640 1 --2- 172.21.15.119:0/2479600706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995013b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9948009e50 tx=0x7f994801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5fd96ff3fcf106 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 1 -- 172.21.15.119:0/2479600706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480a5620 con 0x7f99501392d0 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: dump: 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:46.893 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 1 -- 172.21.15.119:0/2479600706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480a57c0 con 0x7f99501392d0 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 1 -- 172.21.15.119:0/2479600706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480a5b90 con 0x7f99501392d0 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f99579e4640 5 monclient: authenticate success, global_id 74190 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f99579e4640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f99579e4640 4 set_mon_vals no callback set 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.886+0000 7f9947fff640 10 monclient: dump: 2024-09-15T21:23:46.894 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:46.895 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: shutdown 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/2479600706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 msgr2=0x7f995013b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 --2- 172.21.15.119:0/2479600706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995013b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9948009e50 tx=0x7f994801c960 comp rx=0 tx=0).stop 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/2479600706 shutdown_connections 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 --2- 172.21.15.119:0/2479600706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995013b6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 --2- 172.21.15.119:0/2479600706 >> v2:172.21.15.119:3300/0 conn(0x7f99500a57e0 0x7f99500721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/2479600706 >> 172.21.15.119:0/2479600706 conn(0x7f99500a3b80 msgr2=0x7f995006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:46.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/2479600706 shutdown_connections 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/2479600706 wait complete. 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: build_initial_monmap 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: monmap: 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:46.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 Processor -- start 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- start start 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: init 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99500a57e0 0x7f99501cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): picked mon.noname-b con 0x7f99500a57e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99500a5bc0 0x7f99501344c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): picked mon.noname-c con 0x7f99500a5bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995009ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): picked mon.noname-a con 0x7f99501392d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9950069720 con 0x7f99501392d0 2024-09-15T21:23:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f99500695a0 con 0x7f99500a57e0 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f99500698a0 con 0x7f99500a5bc0 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient(hunting): authenticate will time out at 1345.689866s 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955780640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99500a57e0 msgr2=0x7f99501cf5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955780640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99500a57e0 0x7f99501cf5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995009ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995009ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36988/0 (socket says 172.21.15.119:36988) 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99500a5bc0 0x7f99501344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 -- 172.21.15.119:0/1800729308 learned_addr learned my addr 172.21.15.119:0/1800729308 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): get_auth_request con 0x7f99501392d0 auth_method 0 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): get_auth_request con 0x7f99500a5bc0 auth_method 0 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9954f7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient(hunting): handle_auth_done global_id 74196 payload 824 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 -- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99500a5bc0 msgr2=0x7f99501344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:46.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99500a5bc0 0x7f99501344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 -- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99500a57e0 msgr2=0x7f99501cf5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99500a57e0 0x7f99501cf5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient: _finish_hunting 0 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 monclient: found mon.noname-a 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 -- 172.21.15.119:0/1800729308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99500a20d0 con 0x7f99501392d0 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9955f81640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995009ea80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f99480af040 tx=0x7f9948019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480bc020 con 0x7f99501392d0 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: dump: 2024-09-15T21:23:46.905 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 5 monclient: authenticate success, global_id 74196 2024-09-15T21:23:46.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: _renew_subs 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9950122350 con 0x7f99501392d0 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480b8880 con 0x7f99501392d0 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948009510 con 0x7f99501392d0 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f9945ffb640 10 monclient: dump: 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:46.907 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99477fe640 4 set_mon_vals no callback set 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: _renew_subs 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.890+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9950121d00 con 0x7f99501392d0 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f99579e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f99579e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f99579e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99500721c0 con 0x7f99501392d0 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f99480bf020 con 0x7f99501392d0 2024-09-15T21:23:46.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f9945ffb640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f993403b970 0x7f993403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:46.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7f99480ec740 con 0x7f99501392d0 2024-09-15T21:23:46.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f9955780640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f993403b970 0x7f993403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:46.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f9955780640 10 monclient: get_auth_request con 0x7f993403b970 auth_method 0 2024-09-15T21:23:46.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.894+0000 7f9955780640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f993403b970 0x7f993403de30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f994c003990 tx=0x7f994c006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:46.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.906+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f99480c6070 con 0x7f99501392d0 2024-09-15T21:23:46.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.906+0000 7f9945ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:46.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:46.906+0000 7f9945ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): tick 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 msgr2=0x7f9df800a8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df80083f0 0x7f9df800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 msgr2=0x7f9df8007d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.17:3300/0 conn(0x7f9df80058e0 0x7f9df8007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9e0c09dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e0bfff640 1 -- 172.21.15.17:0/3488033100 reap_dead start 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9df8010b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): picked mon.noname-c con 0x7f9e0c09d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9e0c134d70 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c09e7a0 con 0x7f9e0c134d70 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c0a0320 con 0x7f9e0c09dd20 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 1 -- 172.21.15.17:0/3488033100 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0c0a1bc0 con 0x7f9e0c09d400 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e097fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9df8010b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9df800d430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.072+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9df8010b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717082830 0 0) 0x7f9e0c09e7a0 con 0x7f9e0c134d70 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): my global_id is 74199 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9df4003530 con 0x7f9e0c134d70 2024-09-15T21:23:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 33616945 0 0) 0x7f9df4003530 con 0x7f9e0c134d70 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 msgr2=0x7f9df8010b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 --2- 172.21.15.17:0/3488033100 >> v2:172.21.15.119:3300/0 conn(0x7f9e0c09d400 0x7f9df8010b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.119:6789/0 conn(0x7f9e0c09dd20 legacy=0x7f9df800d430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df8006e00 con 0x7f9e0c134d70 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f9e04001580 con 0x7f9e0c134d70 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: dump: 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 >> v1:172.21.15.17:6789/0 conn(0x7f9e0c134d70 legacy=0x7f9df8013150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:47.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9df40039c0 0x7f9df4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): picked mon.b con 0x7f9df40039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9df4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): picked mon.a con 0x7f9df4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df40052f0 0x7f9df400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): picked mon.c con 0x7f9df40052f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c0a1bc0 con 0x7f9df4004810 2024-09-15T21:23:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9df400c240 con 0x7f9df40039c0 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9df400c440 con 0x7f9df40052f0 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9df40039c0 msgr2=0x7f9df4003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9df40039c0 0x7f9df4003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9df4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): get_auth_request con 0x7f9df4004810 auth_method 0 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df40052f0 0x7f9df400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): get_auth_request con 0x7f9df40052f0 auth_method 0 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient(hunting): handle_auth_done global_id 74199 payload 293 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df40052f0 msgr2=0x7f9df400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9df40052f0 0x7f9df400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9df40039c0 msgr2=0x7f9df4003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9df40039c0 0x7f9df4003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 monclient: found mon.a 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 1 -- 172.21.15.17:0/3488033100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df4003530 con 0x7f9df4004810 2024-09-15T21:23:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e12179640 5 monclient: authenticate success, global_id 74199 2024-09-15T21:23:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e12179640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.076+0000 7f9e0bfff640 1 -- 172.21.15.17:0/3488033100 reap_dead start 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f99579e4640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f99579e4640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f99579e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f9950121690 con 0x7f99501392d0 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2320 (secure 0 0 0) 0x7f99480165e0 con 0x7f99501392d0 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f9945ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.082+0000 7f9945ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0affd640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9df4004c90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e00003940 tx=0x7f9e0001d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bd80579f758310 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00009a80 con 0x7f9df4004810 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: dump: 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e000b1070 con 0x7f9df4004810 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 4 set_mon_vals no callback set 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 1 -- 172.21.15.17:0/3488033100 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000ac660 con 0x7f9df4004810 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e0a7fc640 10 monclient: dump: 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 10 monclient: shutdown 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 -- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 msgr2=0x7f9df4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9df4004c90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e00003940 tx=0x7f9e0001d820 comp rx=0 tx=0).stop 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 -- 172.21.15.17:0/3488033100 shutdown_connections 2024-09-15T21:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 --2- 172.21.15.17:0/3488033100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9df4004c90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 -- 172.21.15.17:0/3488033100 >> 172.21.15.17:0/3488033100 conn(0x7f9e0c0a66a0 msgr2=0x7f9e0c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 -- 172.21.15.17:0/3488033100 shutdown_connections 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.080+0000 7f9e12179640 1 -- 172.21.15.17:0/3488033100 wait complete. 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient: monmap: 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 Processor -- start 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 -- start start 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient: init 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9e0c1cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): picked mon.noname-a con 0x7f9df4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9e0c133910 0x7f9e0c133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): picked mon.noname-c con 0x7f9e0c133910 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e0c125530 0x7f9e0c125990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): picked mon.noname-b con 0x7f9e0c125530 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c0a32b0 con 0x7f9df4004810 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c0a3130 con 0x7f9e0c125530 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c09ccd0 con 0x7f9e0c133910 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.084+0000 7f9e12179640 10 monclient(hunting): authenticate will time out at 1350.483934s 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0bfff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e0c125530 msgr2=0x7f9e0c125990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9e0c1cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e0c125530 0x7f9e0c125990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9e0c1cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32776/0 (socket says 172.21.15.17:32776) 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/564262920 learned_addr learned my addr 172.21.15.17:0/564262920 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request con 0x7f9df4004810 auth_method 0 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0affd640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9e0c133910 0x7f9e0c133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0affd640 10 monclient(hunting): get_auth_request con 0x7f9e0c133910 auth_method 0 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient(hunting): handle_auth_done global_id 74202 payload 824 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9e0c133910 msgr2=0x7f9e0c133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9e0c133910 0x7f9e0c133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e0c125530 msgr2=0x7f9e0c125990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e0c125530 0x7f9e0c125990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 monclient: found mon.noname-a 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 -- 172.21.15.17:0/564262920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c0a57c0 con 0x7f9df4004810 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0b7fe640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9e0c1cfc20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00fdc0 tx=0x7f9dfc00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc009e90 con 0x7f9df4004810 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: dump: 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc00d440 con 0x7f9df4004810 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 5 monclient: authenticate success, global_id 74202 2024-09-15T21:23:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 monclient: _renew_subs 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c129350 con 0x7f9df4004810 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00d650 con 0x7f9df4004810 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 10 monclient: dump: 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.112 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 monclient: _renew_subs 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e09ffb640 4 set_mon_vals no callback set 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c129070 con 0x7f9df4004810 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e0c134660 con 0x7f9df4004810 2024-09-15T21:23:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9dfc00d7f0 con 0x7f9df4004810 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9df803c8a0 0x7f9df803ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7f9dfc051fe0 con 0x7f9df4004810 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.088+0000 7f9e0affd640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9df803c8a0 0x7f9df803ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.092+0000 7f9e0affd640 10 monclient: get_auth_request con 0x7f9df803c8a0 auth_method 0 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.092+0000 7f9e0affd640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9df803c8a0 0x7f9df803ed60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e000b0000 tx=0x7f9e000b7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.096+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9e0c0a57c0 con 0x7f9df4004810 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.096+0000 7f9df3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.096+0000 7f9df3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:47.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.190+0000 7f99579e4640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.190+0000 7f99579e4640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.190+0000 7f99579e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.190+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f995006c520 con 0x7f99501392d0 2024-09-15T21:23:47.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.190+0000 7f9945ffb640 1 -- 172.21.15.119:0/1800729308 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2320 (secure 0 0 0) 0x7f99480e9d00 con 0x7f99501392d0 2024-09-15T21:23:47.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f9945ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f9945ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:47.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f993403b970 msgr2=0x7f993403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f993403b970 0x7f993403de30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f994c003990 tx=0x7f994c006d90 comp rx=0 tx=0).stop 2024-09-15T21:23:47.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 10 monclient: shutdown 2024-09-15T21:23:47.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 msgr2=0x7f995009ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995009ea80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f99480af040 tx=0x7f9948019700 comp rx=0 tx=0).stop 2024-09-15T21:23:47.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 shutdown_connections 2024-09-15T21:23:47.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f993403b970 0x7f993403de30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99501392d0 0x7f995009ea80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99500a5bc0 0x7f99501344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 --2- 172.21.15.119:0/1800729308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99500a57e0 0x7f99501cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 >> 172.21.15.119:0/1800729308 conn(0x7f99500a3b80 msgr2=0x7f99500a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 shutdown_connections 2024-09-15T21:23:47.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.194+0000 7f99579e4640 1 -- 172.21.15.119:0/1800729308 wait complete. 2024-09-15T21:23:47.215 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-09-15T21:23:47.216 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-09-15T21:23:47.216 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:47.216 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:47.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep globster 2024-09-15T21:23:47.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '8\.' 2024-09-15T21:23:47.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient: get_monmap_and_config 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient: monmap: 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:47.334207+0000 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:47.334207+0000 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:47.335 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:47.336 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:47.336 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:47.336 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 Processor -- start 2024-09-15T21:23:47.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 -- start start 2024-09-15T21:23:47.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient: init 2024-09-15T21:23:47.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2024-09-15T21:23:47.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a600a3600 0x7f5a600a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): picked mon.noname-a con 0x7f5a600a3600 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5a600a1270 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a6013c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): picked mon.noname-b con 0x7f5a60134370 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a600a1700 con 0x7f5a600a1270 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5a600a39e0 con 0x7f5a600a3600 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5a600a27a0 con 0x7f5a60134370 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a67869640 10 monclient(hunting): authenticate will time out at 1346.130940s 2024-09-15T21:23:47.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a600a3600 0x7f5a600a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a64e04640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5a600a1270 0x7f5a60133c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:44092/0 (socket says 172.21.15.119:44092) 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a6013c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a64e04640 1 -- 172.21.15.119:0/4259544835 learned_addr learned my addr 172.21.15.119:0/4259544835 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): get_auth_request con 0x7f5a600a3600 auth_method 0 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): get_auth_request con 0x7f5a60134370 auth_method 0 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65e06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient(hunting): handle_auth_done global_id 74205 payload 274 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 1 -- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 msgr2=0x7f5a6013c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 1 --2- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a6013c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 1 -- 172.21.15.119:0/4259544835 >> v1:172.21.15.17:6789/0 conn(0x7f5a600a1270 legacy=0x7f5a60133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 1 monclient: found mon.noname-a 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.330+0000 7f5a65605640 1 -- 172.21.15.119:0/4259544835 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6009df80 con 0x7f5a600a3600 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a65605640 1 --2- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a600a1b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000ba60 tx=0x7f5a5001ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48016acb8a1051c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 1 -- 172.21.15.119:0/4259544835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a500b4070 con 0x7f5a600a3600 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: dump: 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.347 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 5 monclient: authenticate success, global_id 74205 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 1 -- 172.21.15.119:0/4259544835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a500a7a50 con 0x7f5a600a3600 2024-09-15T21:23:47.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 1 -- 172.21.15.119:0/4259544835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a500a7e20 con 0x7f5a600a3600 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a57fff640 10 monclient: dump: 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.349 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 4 set_mon_vals no callback set 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 10 monclient: shutdown 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 -- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 msgr2=0x7f5a600a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 --2- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a600a1b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000ba60 tx=0x7f5a5001ea00 comp rx=0 tx=0).stop 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 -- 172.21.15.119:0/4259544835 shutdown_connections 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 --2- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a6013c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 --2- 172.21.15.119:0/4259544835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a600a1b30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 -- 172.21.15.119:0/4259544835 >> 172.21.15.119:0/4259544835 conn(0x7f5a600a2ae0 msgr2=0x7f5a600a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 -- 172.21.15.119:0/4259544835 shutdown_connections 2024-09-15T21:23:47.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 1 -- 172.21.15.119:0/4259544835 wait complete. 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.334+0000 7f5a67869640 10 monclient: monmap: 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 Processor -- start 2024-09-15T21:23:47.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- start start 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: init 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a6013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): picked mon.noname-a con 0x7f5a600a3600 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a601368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): picked mon.noname-c con 0x7f5a60134370 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a60136df0 0x7f5a60137230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): picked mon.noname-b con 0x7f5a60136df0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600a21f0 con 0x7f5a600a3600 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600a2070 con 0x7f5a60136df0 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6013e550 con 0x7f5a60134370 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient(hunting): authenticate will time out at 1346.136761s 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65e06640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a60136df0 msgr2=0x7f5a60137230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65e06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a60136df0 0x7f5a60137230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a6013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a601368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a601368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:41760/0 (socket says 172.21.15.119:41760) 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 1 -- 172.21.15.119:0/2405793317 learned_addr learned my addr 172.21.15.119:0/2405793317 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): get_auth_request con 0x7f5a60134370 auth_method 0 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): get_auth_request con 0x7f5a600a3600 auth_method 0 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient(hunting): handle_auth_done global_id 74211 payload 824 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 -- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 msgr2=0x7f5a601368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a601368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 -- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a60136df0 msgr2=0x7f5a60137230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a60136df0 0x7f5a60137230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 monclient: found mon.noname-a 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 -- 172.21.15.119:0/2405793317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a60138370 con 0x7f5a600a3600 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a64e04640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a601368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:47.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a65605640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a6013ae50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a500ae000 tx=0x7f5a5001b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a500bd020 con 0x7f5a600a3600 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: dump: 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.356 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 5 monclient: authenticate success, global_id 74211 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _renew_subs 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6013a6d0 con 0x7f5a600a3600 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a500b5040 con 0x7f5a600a3600 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a500b4070 con 0x7f5a600a3600 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 10 monclient: dump: 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.358 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _renew_subs 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a601390a0 con 0x7f5a600a3600 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a577fe640 4 set_mon_vals no callback set 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a600a1b30 con 0x7f5a600a3600 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.338+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5a500c0020 con 0x7f5a600a3600 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.342+0000 7f5a55ffb640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a4403b9a0 0x7f5a4403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.342+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7f5a500ee2d0 con 0x7f5a600a3600 2024-09-15T21:23:47.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.342+0000 7f5a64e04640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a4403b9a0 0x7f5a4403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.342+0000 7f5a64e04640 10 monclient: get_auth_request con 0x7f5a4403b9a0 auth_method 0 2024-09-15T21:23:47.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.342+0000 7f5a64e04640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a4403b9a0 0x7f5a4403de60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5a60138680 tx=0x7f5a5c007f40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.354+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5a500c7070 con 0x7f5a600a3600 2024-09-15T21:23:47.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.354+0000 7f5a55ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.354+0000 7f5a55ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9e12179640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:23:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9e12179640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9e12179640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9e0c11dbf0 con 0x7f9df4004810 2024-09-15T21:23:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9df3fff640 1 -- 172.21.15.17:0/564262920 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7f9dfc017370 con 0x7f9df4004810 2024-09-15T21:23:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9df3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:23:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.340+0000 7f9df3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:47.362 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":64,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:23:47.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9df803c8a0 msgr2=0x7f9df803ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9df803c8a0 0x7f9df803ed60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e000b0000 tx=0x7f9e000b7040 comp rx=0 tx=0).stop 2024-09-15T21:23:47.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 10 monclient: shutdown 2024-09-15T21:23:47.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 msgr2=0x7f9e0c1cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9e0c1cfc20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00fdc0 tx=0x7f9dfc00c7a0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 shutdown_connections 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9df803c8a0 0x7f9df803ed60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e0c125530 0x7f9e0c125990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9e0c133910 0x7f9e0c133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 --2- 172.21.15.17:0/564262920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9df4004810 0x7f9e0c1cfc20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 >> 172.21.15.17:0/564262920 conn(0x7f9e0c0a66a0 msgr2=0x7f9e0c0a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 shutdown_connections 2024-09-15T21:23:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.344+0000 7f9e12179640 1 -- 172.21.15.17:0/564262920 wait complete. 2024-09-15T21:23:47.365 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:23:47.365 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:23:47.365 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:23:47.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient: get_monmap_and_config 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient: monmap: 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:47.491744+0000 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:47.491744+0000 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:47.494 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:47.494 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 Processor -- start 2024-09-15T21:23:47.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 -- start start 2024-09-15T21:23:47.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient: init 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa0280a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): picked mon.noname-b con 0x7fa0280a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa02809ffc0 0x7fa0280a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): picked mon.noname-c con 0x7fa02809ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa028135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0280a3130 con 0x7fa028135670 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa0280a16f0 con 0x7fa02809ffc0 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa028073df0 con 0x7fa0280a0890 2024-09-15T21:23:47.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02cb57640 10 monclient(hunting): authenticate will time out at 1350.887130s 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa025d9b640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fa02809ffc0 msgr2=0x7fa0280a11b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa026d9d640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fa028135670 legacy=0x7fa028135aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa0280a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa025d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa02809ffc0 0x7fa0280a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa0280a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49892/0 (socket says 172.21.15.17:49892) 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 -- 172.21.15.17:0/2438016255 learned_addr learned my addr 172.21.15.17:0/2438016255 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): get_auth_request con 0x7fa0280a0890 auth_method 0 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient(hunting): handle_auth_done global_id 44786 payload 274 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 -- 172.21.15.17:0/2438016255 >> v2:172.21.15.119:3300/0 conn(0x7fa02809ffc0 msgr2=0x7fa0280a11b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 --2- 172.21.15.17:0/2438016255 >> v2:172.21.15.119:3300/0 conn(0x7fa02809ffc0 0x7fa0280a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 -- 172.21.15.17:0/2438016255 >> v1:172.21.15.119:6789/0 conn(0x7fa028135670 legacy=0x7fa028135aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 monclient: found mon.noname-b 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.488+0000 7fa02659c640 1 -- 172.21.15.17:0/2438016255 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0280a1bf0 con 0x7fa0280a0890 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02659c640 1 --2- 172.21.15.17:0/2438016255 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa0280a0c70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa0280a1870 tx=0x7fa01801c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ddc3d8db52faae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 1 -- 172.21.15.17:0/2438016255 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180a5640 con 0x7fa0280a0890 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: dump: 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.504 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 1 -- 172.21.15.17:0/2438016255 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0180a57e0 con 0x7fa0280a0890 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 1 -- 172.21.15.17:0/2438016255 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180a5bb0 con 0x7fa0280a0890 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 5 monclient: authenticate success, global_id 44786 2024-09-15T21:23:47.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 4 set_mon_vals no callback set 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02559a640 10 monclient: dump: 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.506 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 10 monclient: shutdown 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 -- 172.21.15.17:0/2438016255 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 msgr2=0x7fa0280a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 --2- 172.21.15.17:0/2438016255 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa0280a0c70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa0280a1870 tx=0x7fa01801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 -- 172.21.15.17:0/2438016255 shutdown_connections 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 --2- 172.21.15.17:0/2438016255 >> v2:172.21.15.119:3300/0 conn(0x7fa02809ffc0 0x7fa0280a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 --2- 172.21.15.17:0/2438016255 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa0280a0c70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 -- 172.21.15.17:0/2438016255 >> 172.21.15.17:0/2438016255 conn(0x7fa0280a66a0 msgr2=0x7fa0280a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 -- 172.21.15.17:0/2438016255 shutdown_connections 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 -- 172.21.15.17:0/2438016255 wait complete. 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 10 monclient: monmap: 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:47.507 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 Processor -- start 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 1 -- start start 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.492+0000 7fa02cb57640 10 monclient: init 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02809ffc0 0x7fa0281cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): picked mon.noname-a con 0x7fa02809ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa028134740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): picked mon.noname-c con 0x7fa0280a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa028135670 0x7fa028134c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): picked mon.noname-b con 0x7fa028135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa028140690 con 0x7fa02809ffc0 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa028140510 con 0x7fa028135670 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa028140810 con 0x7fa0280a0890 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): authenticate will time out at 1350.892298s 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa026d9d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa028135670 msgr2=0x7fa028134c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa028134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa026d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa028135670 0x7fa028134c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa028134740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49908/0 (socket says 172.21.15.17:49908) 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 -- 172.21.15.17:0/3748882125 learned_addr learned my addr 172.21.15.17:0/3748882125 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): get_auth_request con 0x7fa0280a0890 auth_method 0 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02809ffc0 0x7fa0281cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): get_auth_request con 0x7fa02809ffc0 auth_method 0 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient(hunting): handle_auth_done global_id 44789 payload 824 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa028135670 msgr2=0x7fa028134c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa028135670 0x7fa028134c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02809ffc0 msgr2=0x7fa0281cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02809ffc0 0x7fa0281cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 monclient: found mon.noname-c 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0281210f0 con 0x7fa0280a0890 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa025d9b640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa028134740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa02000b530 tx=0x7fa02000b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020013020 con 0x7fa0280a0890 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: dump: 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.512 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa020003df0 con 0x7fa0280a0890 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 5 monclient: authenticate success, global_id 44789 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient: _renew_subs 2024-09-15T21:23:47.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa028120850 con 0x7fa0280a0890 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200052b0 con 0x7fa0280a0890 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: dump: 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.514 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01ffff640 4 set_mon_vals no callback set 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient: _renew_subs 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02811d2d0 con 0x7fa0280a0890 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 0x7fa02813b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa028135aa0 con 0x7fa02813aeb0 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 0x7fa02813b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa020016020 con 0x7fa0280a0890 2024-09-15T21:23:47.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient: get_auth_request con 0x7fa02813aeb0 auth_method 0 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 10 monclient(hunting): handle_auth_done global_id 74217 payload 824 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa028120d00 con 0x7fa02813aeb0 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa00c03b970 0x7fa00c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7fa020050b60 con 0x7fa0280a0890 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa02659c640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 0x7fa02813b290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa02806f1c0 tx=0x7fa018019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa026d9d640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa00c03b970 0x7fa00c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa026d9d640 10 monclient: get_auth_request con 0x7fa00c03b970 auth_method 0 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180c0640 con 0x7fa02813aeb0 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: dump: 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa0180c0640 con 0x7fa02813aeb0 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 msgr2=0x7fa02813b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 0x7fa02813b290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa02806f1c0 tx=0x7fa018019690 comp rx=0 tx=0).stop 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 msgr2=0x7fa02813b290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa01effd640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813aeb0 0x7fa02813b290 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa02806f1c0 tx=0x7fa018019690 comp rx=0 tx=0).stop 2024-09-15T21:23:47.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.496+0000 7fa026d9d640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa00c03b970 0x7fa00c03de30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa02811f680 tx=0x7fa010006c70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 0x7fa02811d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa02806d020 con 0x7fa02813b7d0 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 0x7fa02811d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient: get_auth_request con 0x7fa02813b7d0 auth_method 0 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 10 monclient(hunting): handle_auth_done global_id 74220 payload 824 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 1 -- 172.21.15.17:0/3748882125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa02813bbe0 con 0x7fa02813b7d0 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02659c640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 0x7fa02811d590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa01801c720 tx=0x7fa0180a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018003f50 con 0x7fa02813b7d0 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 10 monclient: dump: 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.521 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7fa018003f50 con 0x7fa02813b7d0 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 msgr2=0x7fa02811d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 0x7fa02811d590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa01801c720 tx=0x7fa0180a7000 comp rx=0 tx=0).stop 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 msgr2=0x7fa02811d590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:47.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa01effd640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa02813b7d0 0x7fa02811d590 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa01801c720 tx=0x7fa0180a7000 comp rx=0 tx=0).stop 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 64, 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 40287, 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:47.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.536 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:47.536 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a67869640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a67869640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a67869640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f5a6013bfc0 con 0x7f5a600a3600 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2320 (secure 0 0 0) 0x7f5a50018740 con 0x7f5a600a3600 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a55ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.530+0000 7f5a55ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:47.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa00c03b970 msgr2=0x7fa00c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa00c03b970 0x7fa00c03de30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa02811f680 tx=0x7fa010006c70 comp rx=0 tx=0).stop 2024-09-15T21:23:47.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 10 monclient: shutdown 2024-09-15T21:23:47.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 msgr2=0x7fa028134740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa028134740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa02000b530 tx=0x7fa02000b9f0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa026d9d640 1 -- 172.21.15.17:0/3748882125 reap_dead start 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 shutdown_connections 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 --2- 172.21.15.17:0/3748882125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0280a0890 0x7fa028134740 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 >> 172.21.15.17:0/3748882125 conn(0x7fa0280a66a0 msgr2=0x7fa0280a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 shutdown_connections 2024-09-15T21:23:47.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.504+0000 7fa02cb57640 1 -- 172.21.15.17:0/3748882125 wait complete. 2024-09-15T21:23:47.539 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:23:47.540 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:23:47.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:23:47.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a67869640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a67869640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a67869640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f5a600a2480 con 0x7f5a600a3600 2024-09-15T21:23:47.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a55ffb640 1 -- 172.21.15.119:0/2405793317 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2320 (secure 0 0 0) 0x7f5a500b9d90 con 0x7f5a600a3600 2024-09-15T21:23:47.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a55ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.630+0000 7f5a55ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a4403b9a0 msgr2=0x7f5a4403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a4403b9a0 0x7f5a4403de60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5a60138680 tx=0x7f5a5c007f40 comp rx=0 tx=0).stop 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 10 monclient: shutdown 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 msgr2=0x7f5a6013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a6013ae50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a500ae000 tx=0x7f5a5001b7a0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 shutdown_connections 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a4403b9a0 0x7f5a4403de60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a60136df0 0x7f5a60137230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a60134370 0x7f5a601368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 --2- 172.21.15.119:0/2405793317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600a3600 0x7f5a6013ae50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 >> 172.21.15.119:0/2405793317 conn(0x7f5a600a2ae0 msgr2=0x7f5a601325e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 shutdown_connections 2024-09-15T21:23:47.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.634+0000 7f5a67869640 1 -- 172.21.15.119:0/2405793317 wait complete. 2024-09-15T21:23:47.651 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-09-15T21:23:47.651 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-09-15T21:23:47.652 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T21:23:47.652 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep 'osd\.0' 2024-09-15T21:23:47.652 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep globster 2024-09-15T21:23:47.652 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep '6\.' 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient: get_monmap_and_config 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient: monmap: 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:23:47.655377+0000 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:23:47.655377+0000 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.656 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 Processor -- start 2024-09-15T21:23:47.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 -- start start 2024-09-15T21:23:47.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient: init 2024-09-15T21:23:47.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:23:47.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd0581378c0 0x7fd058137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): picked mon.noname-a con 0x7fd0581378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd0581381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0580a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): picked mon.noname-b con 0x7fd0580a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0580a1d00 con 0x7fd0581381e0 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd0580a1fe0 con 0x7fd0581378c0 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580a22c0 con 0x7fd0580a5210 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05eb9c640 10 monclient(hunting): authenticate will time out at 1351.051532s 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd057fff640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fd0581381e0 legacy=0x7fd0580a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd0581378c0 0x7fd058137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0580a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd0581378c0 0x7fd058137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53978/0 (socket says 172.21.15.17:53978) 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 1 -- 172.21.15.17:0/1186064495 learned_addr learned my addr 172.21.15.17:0/1186064495 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): get_auth_request con 0x7fd0581378c0 auth_method 0 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0580a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:54572/0 (socket says 172.21.15.17:54572) 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): get_auth_request con 0x7fd0580a5210 auth_method 0 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05c938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.652+0000 7fd05d139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 10 monclient(hunting): handle_auth_done global_id 74223 payload 274 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 1 -- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 msgr2=0x7fd0580a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0580a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 1 -- 172.21.15.17:0/1186064495 >> v1:172.21.15.119:6789/0 conn(0x7fd0581381e0 legacy=0x7fd0580a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 1 monclient: found mon.noname-a 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 1 -- 172.21.15.17:0/1186064495 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580a4490 con 0x7fd0581378c0 2024-09-15T21:23:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05d139640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0580a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05c938640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581378c0 0x7fd058137ca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098a0 tx=0x7fd04801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570f0478d9a295e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 1 -- 172.21.15.17:0/1186064495 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480b1070 con 0x7fd0581378c0 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: dump: 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 1 -- 172.21.15.17:0/1186064495 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0480ac600 con 0x7fd0581378c0 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 1 -- 172.21.15.17:0/1186064495 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480ac9d0 con 0x7fd0581378c0 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 5 monclient: authenticate success, global_id 74223 2024-09-15T21:23:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 4 set_mon_vals no callback set 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd0577fe640 10 monclient: dump: 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.670 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 10 monclient: shutdown 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 -- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581378c0 msgr2=0x7fd058137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581378c0 0x7fd058137ca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098a0 tx=0x7fd04801c400 comp rx=0 tx=0).stop 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 -- 172.21.15.17:0/1186064495 shutdown_connections 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0580a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/1186064495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581378c0 0x7fd058137ca0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 -- 172.21.15.17:0/1186064495 >> 172.21.15.17:0/1186064495 conn(0x7fd0580aac40 msgr2=0x7fd0580a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 -- 172.21.15.17:0/1186064495 shutdown_connections 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.656+0000 7fd05eb9c640 1 -- 172.21.15.17:0/1186064495 wait complete. 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: monmap: 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:23:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 Processor -- start 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- start start 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: init 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0581cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): picked mon.noname-c con 0x7fd0580a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581378c0 0x7fd058136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): picked mon.noname-b con 0x7fd0581378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581381e0 0x7fd05813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): picked mon.noname-a con 0x7fd0581381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd05806f340 con 0x7fd0581381e0 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd05806f1c0 con 0x7fd0581378c0 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd0581425c0 con 0x7fd0580a5210 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): authenticate will time out at 1351.057371s 2024-09-15T21:23:47.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581378c0 msgr2=0x7fd058136b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581378c0 0x7fd058136b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0581cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581381e0 0x7fd05813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0581cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:54580/0 (socket says 172.21.15.17:54580) 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 -- 172.21.15.17:0/3292886976 learned_addr learned my addr 172.21.15.17:0/3292886976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): get_auth_request con 0x7fd0581381e0 auth_method 0 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): get_auth_request con 0x7fd0580a5210 auth_method 0 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient(hunting): handle_auth_done global_id 44795 payload 824 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581378c0 msgr2=0x7fd058136b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581378c0 0x7fd058136b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581381e0 msgr2=0x7fd05813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581381e0 0x7fd05813cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 monclient: found mon.noname-c 2024-09-15T21:23:47.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05813f320 con 0x7fd0580a5210 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0581381e0 0x7fd05813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05c938640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0581cfa30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd04801c230 tx=0x7fd04801ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480bb020 con 0x7fd0580a5210 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: dump: 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.676 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0480b7420 con 0x7fd0580a5210 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480b1040 con 0x7fd0580a5210 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:47.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 10 monclient: dump: 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 5 monclient: authenticate success, global_id 44795 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: _renew_subs 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0580a12f0 con 0x7fd0580a5210 2024-09-15T21:23:47.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd056ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: _renew_subs 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd058121520 con 0x7fd0580a5210 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 0x7fd05813d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd05813db80 con 0x7fd05813d240 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 0x7fd05813d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient: get_auth_request con 0x7fd05813d240 auth_method 0 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd0480be020 con 0x7fd0580a5210 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 10 monclient(hunting): handle_auth_done global_id 44798 payload 824 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0580a8a30 con 0x7fd05813d240 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd04403b970 0x7fd04403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7fd0480eb250 con 0x7fd0580a5210 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd057fff640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 0x7fd05813d640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b450 tx=0x7fd04c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.660+0000 7fd05d139640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd04403b970 0x7fd04403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd05d139640 10 monclient: get_auth_request con 0x7fd04403b970 auth_method 0 2024-09-15T21:23:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c003dd0 con 0x7fd05813d240 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 10 monclient: dump: 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd04c015070 con 0x7fd05813d240 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 msgr2=0x7fd05813d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 0x7fd05813d640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b450 tx=0x7fd04c00b910 comp rx=0 tx=0).stop 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 msgr2=0x7fd05813d640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd0557fa640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd05813d240 0x7fd05813d640 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b450 tx=0x7fd04c00b910 comp rx=0 tx=0).stop 2024-09-15T21:23:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.664+0000 7fd05d139640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd04403b970 0x7fd04403de30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd05813f0b0 tx=0x7fd05000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 0x7fd058139140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580a56a0 con 0x7fd058138d60 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 0x7fd058139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient: get_auth_request con 0x7fd058138d60 auth_method 0 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 10 monclient(hunting): handle_auth_done global_id 44801 payload 824 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 1 -- 172.21.15.17:0/3292886976 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd058069270 con 0x7fd058138d60 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd057fff640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 0x7fd058139140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd058121320 tx=0x7fd04c0059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c005610 con 0x7fd058138d60 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 10 monclient: dump: 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7fd04c005610 con 0x7fd058138d60 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 msgr2=0x7fd058139140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 0x7fd058139140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd058121320 tx=0x7fd04c0059e0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 msgr2=0x7fd058139140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd0557fa640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd058138d60 0x7fd058139140 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd058121320 tx=0x7fd04c0059e0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:23:47.686 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 64, 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 196266, 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:23:47.687 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:23:47.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:47.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:23:47.690 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:23:47.691 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:23:47.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:47.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:23:47.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:23:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:23:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:23:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:23:47.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:23:47.699 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:23:47.699 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd04403b970 msgr2=0x7fd04403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd04403b970 0x7fd04403de30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd05813f0b0 tx=0x7fd05000b040 comp rx=0 tx=0).stop 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 10 monclient: shutdown 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 msgr2=0x7fd0581cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 --2- 172.21.15.17:0/3292886976 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd0580a5210 0x7fd0581cfa30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd04801c230 tx=0x7fd04801ac40 comp rx=0 tx=0).stop 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05d139640 1 -- 172.21.15.17:0/3292886976 reap_dead start 2024-09-15T21:23:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05d139640 1 -- 172.21.15.17:0/3292886976 reap_dead start 2024-09-15T21:23:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 shutdown_connections 2024-09-15T21:23:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 >> 172.21.15.17:0/3292886976 conn(0x7fd0580aac40 msgr2=0x7fd05806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 shutdown_connections 2024-09-15T21:23:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:23:47.668+0000 7fd05eb9c640 1 -- 172.21.15.17:0/3292886976 wait complete. 2024-09-15T21:23:47.701 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:23:47.701 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.762+0000 7f84bfe41640 10 monclient: get_monmap_and_config 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.762+0000 7f84bfe41640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.762+0000 7f84bfe41640 10 monclient: monmap: 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:47.767440+0000 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:47.767440+0000 2024-09-15T21:23:47.768 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:47.769 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.762+0000 7f84bfe41640 1 Processor -- start 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.762+0000 7f84bfe41640 1 -- start start 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.762+0000 7f84bfe41640 10 monclient: init 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84b80a4e60 0x7f84b80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): picked mon.noname-a con 0x7f84b80a4e60 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f84b80a5780 0x7f84b8137c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): picked mon.noname-c con 0x7f84b80a5780 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f84b81381d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80a1150 con 0x7f84b81381d0 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f84b80a3c80 con 0x7f84b80a4e60 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f84b80a0ef0 con 0x7f84b80a5780 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient(hunting): authenticate will time out at 1346.564271s 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84be3de640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f84b81381d0 legacy=0x7f84b813ef50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bd3dc640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f84b80a5780 msgr2=0x7f84b8137c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bd3dc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f84b80a5780 0x7f84b8137c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84b80a4e60 0x7f84b80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84b80a4e60 0x7f84b80a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:37026/0 (socket says 172.21.15.119:37026) 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3512631110 learned_addr learned my addr 172.21.15.119:0/3512631110 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): get_auth_request con 0x7f84b80a4e60 auth_method 0 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_done global_id 74229 payload 274 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3512631110 >> v2:172.21.15.119:3300/0 conn(0x7f84b80a5780 msgr2=0x7f84b8137c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 --2- 172.21.15.119:0/3512631110 >> v2:172.21.15.119:3300/0 conn(0x7f84b80a5780 0x7f84b8137c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3512631110 >> v1:172.21.15.119:6789/0 conn(0x7f84b81381d0 legacy=0x7f84b813ef50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:47.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 monclient: found mon.noname-a 2024-09-15T21:23:47.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:47.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3512631110 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b80a6cf0 con 0x7f84b80a4e60 2024-09-15T21:23:47.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bdbdd640 1 --2- 172.21.15.119:0/3512631110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b80a4e60 0x7f84b80a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84b80a1c70 tx=0x7f84b401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501ebaa5e2663ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 1 -- 172.21.15.119:0/3512631110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40b0070 con 0x7f84b80a4e60 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: dump: 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.779 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 5 monclient: authenticate success, global_id 74229 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 1 -- 172.21.15.119:0/3512631110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b40a4e50 con 0x7f84b80a4e60 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bfe41640 4 set_mon_vals no callback set 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 1 -- 172.21.15.119:0/3512631110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40ab820 con 0x7f84b80a4e60 2024-09-15T21:23:47.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.766+0000 7f84bcbdb640 10 monclient: dump: 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: shutdown 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3512631110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b80a4e60 msgr2=0x7f84b80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3512631110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b80a4e60 0x7f84b80a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84b80a1c70 tx=0x7f84b401c400 comp rx=0 tx=0).stop 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3512631110 shutdown_connections 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3512631110 >> v2:172.21.15.119:3300/0 conn(0x7f84b80a5780 0x7f84b8137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3512631110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b80a4e60 0x7f84b80a5240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3512631110 >> 172.21.15.119:0/3512631110 conn(0x7f84b80a6f60 msgr2=0x7f84b80a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3512631110 shutdown_connections 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3512631110 wait complete. 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: build_initial_monmap 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: monmap: 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:47.782 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 Processor -- start 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- start start 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: init 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:47.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 0x7f84b8139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): picked mon.noname-c con 0x7f84b80a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84b80a5780 0x7f84b81396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): picked mon.noname-b con 0x7f84b80a5780 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b81381d0 0x7f84b81cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): picked mon.noname-a con 0x7f84b81381d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f84b80a7cb0 con 0x7f84b81381d0 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84b80a5c50 con 0x7f84b80a5780 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f84b80a7e30 con 0x7f84b80a4e60 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:47.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient(hunting): authenticate will time out at 1346.570104s 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bd3dc640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84b80a5780 msgr2=0x7f84b81396c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bd3dc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84b80a5780 0x7f84b81396c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 0x7f84b8139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b81381d0 0x7f84b81cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 0x7f84b8139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:41766/0 (socket says 172.21.15.119:41766) 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3294587920 learned_addr learned my addr 172.21.15.119:0/3294587920 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): get_auth_request con 0x7f84b80a4e60 auth_method 0 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): get_auth_request con 0x7f84b81381d0 auth_method 0 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84be3de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient(hunting): handle_auth_done global_id 44804 payload 824 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84b80a5780 msgr2=0x7f84b81396c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84b80a5780 0x7f84b81396c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b81381d0 msgr2=0x7f84b81cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:47.786 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b81381d0 0x7f84b81cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 monclient: found mon.noname-c 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 -- 172.21.15.119:0/3294587920 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b813c8d0 con 0x7f84b80a4e60 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bdbdd640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 0x7f84b8139180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f84b40ae040 tx=0x7f84b401ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40ba020 con 0x7f84b80a4e60 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: dump: 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.787 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b40b6420 con 0x7f84b80a4e60 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:47.788 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40b0040 con 0x7f84b80a4e60 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 5 monclient: authenticate success, global_id 44804 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: _renew_subs 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b813cbb0 con 0x7f84b80a4e60 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84abfff640 4 set_mon_vals no callback set 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84aa7fc640 10 monclient: dump: 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.789 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: _renew_subs 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.770+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b80a9690 con 0x7f84b80a4e60 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bfe41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bfe41640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bfe41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b8077180 con 0x7f84b80a4e60 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f84b40bd020 con 0x7f84b80a4e60 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84aa7fc640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f849c03b970 0x7f849c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7f84b40eb200 con 0x7f84b80a4e60 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bd3dc640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f849c03b970 0x7f849c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bd3dc640 10 monclient: get_auth_request con 0x7f849c03b970 auth_method 0 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.774+0000 7f84bd3dc640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f849c03b970 0x7f849c03de30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84ac006fd0 tx=0x7f84ac006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.778+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f84b40c4030 con 0x7f84b80a4e60 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.778+0000 7f84aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:47.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.778+0000 7f84aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:47.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.954+0000 7f84bfe41640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.954+0000 7f84bfe41640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.954+0000 7f84bfe41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:47.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.954+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f84b80a5240 con 0x7f84b80a4e60 2024-09-15T21:23:47.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.958+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2320 (secure 0 0 0) 0x7f84b4016110 con 0x7f84b80a4e60 2024-09-15T21:23:47.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.958+0000 7f84aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:47.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:47.958+0000 7f84aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:48.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84bfe41640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:48.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84bfe41640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:48.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84bfe41640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:48.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f84b813faf0 con 0x7f84b80a4e60 2024-09-15T21:23:48.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84aa7fc640 1 -- 172.21.15.119:0/3294587920 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2320 (secure 0 0 0) 0x7f84b40a4df0 con 0x7f84b80a4e60 2024-09-15T21:23:48.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84aa7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T21:23:48.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.058+0000 7f84aa7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f849c03b970 msgr2=0x7f849c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f849c03b970 0x7f849c03de30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84ac006fd0 tx=0x7f84ac006d40 comp rx=0 tx=0).stop 2024-09-15T21:23:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 10 monclient: shutdown 2024-09-15T21:23:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 msgr2=0x7f84b8139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 0x7f84b8139180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f84b40ae040 tx=0x7f84b401ac40 comp rx=0 tx=0).stop 2024-09-15T21:23:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 shutdown_connections 2024-09-15T21:23:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f849c03b970 0x7f849c03de30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84b81381d0 0x7f84b81cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84b80a5780 0x7f84b81396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 --2- 172.21.15.119:0/3294587920 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f84b80a4e60 0x7f84b8139180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 >> 172.21.15.119:0/3294587920 conn(0x7f84b80a6f60 msgr2=0x7f84b80a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 shutdown_connections 2024-09-15T21:23:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.062+0000 7f84bfe41640 1 -- 172.21.15.119:0/3294587920 wait complete. 2024-09-15T21:23:48.080 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-09-15T21:23:48.080 INFO:tasks.workunit.client.0.smithi119.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-09-15T21:23:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm-device-class osd.0 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient: get_monmap_and_config 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient: build_initial_monmap 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient: monmap: 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:48.194569+0000 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:48.194569+0000 2024-09-15T21:23:48.195 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:48.196 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 Processor -- start 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 -- start start 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient: init 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15e8137bc0 0x7f15e8137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:48.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): picked mon.noname-a con 0x7f15e8137bc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f15e80a09e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15e80a0ea0 0x7f15e8073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): picked mon.noname-c con 0x7f15e80a0ea0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e809cfa0 con 0x7f15e80a09e0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f15e809cd20 con 0x7f15e8137bc0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f15e80a1820 con 0x7f15e80a0ea0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15eec21640 10 monclient(hunting): authenticate will time out at 1346.991093s 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15e7fff640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f15e80a09e0 legacy=0x7f15e8073040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ed1be640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f15e80a0ea0 msgr2=0x7f15e8073750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:48.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ed1be640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15e80a0ea0 0x7f15e8073750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15e8137bc0 0x7f15e8137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15e8137bc0 0x7f15e8137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:37052/0 (socket says 172.21.15.119:37052) 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 1 -- 172.21.15.119:0/222217733 learned_addr learned my addr 172.21.15.119:0/222217733 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): get_auth_request con 0x7f15e8137bc0 auth_method 0 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.190+0000 7f15ec9bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 10 monclient(hunting): handle_auth_done global_id 74235 payload 274 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 1 -- 172.21.15.119:0/222217733 >> v2:172.21.15.119:3300/0 conn(0x7f15e80a0ea0 msgr2=0x7f15e8073750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:48.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 1 --2- 172.21.15.119:0/222217733 >> v2:172.21.15.119:3300/0 conn(0x7f15e80a0ea0 0x7f15e8073750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 1 -- 172.21.15.119:0/222217733 >> v1:172.21.15.119:6789/0 conn(0x7f15e80a09e0 legacy=0x7f15e8073040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:48.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:48.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 1 monclient: found mon.noname-a 2024-09-15T21:23:48.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:48.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:48.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 1 -- 172.21.15.119:0/222217733 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e80a1b00 con 0x7f15e8137bc0 2024-09-15T21:23:48.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15ec9bd640 1 --2- 172.21.15.119:0/222217733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e8137bc0 0x7f15e8137fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15e806a1c0 tx=0x7f15d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec8290e52deaaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:48.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 1 -- 172.21.15.119:0/222217733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d80a7070 con 0x7f15e8137bc0 2024-09-15T21:23:48.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:48.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:48.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: dump: 2024-09-15T21:23:48.206 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:48.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 1 -- 172.21.15.119:0/222217733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d80b0070 con 0x7f15e8137bc0 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 1 -- 172.21.15.119:0/222217733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d80ab890 con 0x7f15e8137bc0 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15e77fe640 10 monclient: dump: 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:48.208 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 5 monclient: authenticate success, global_id 74235 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 4 set_mon_vals no callback set 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 10 monclient: shutdown 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 1 -- 172.21.15.119:0/222217733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e8137bc0 msgr2=0x7f15e8137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.194+0000 7f15eec21640 1 --2- 172.21.15.119:0/222217733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e8137bc0 0x7f15e8137fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15e806a1c0 tx=0x7f15d801c490 comp rx=0 tx=0).stop 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/222217733 shutdown_connections 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 --2- 172.21.15.119:0/222217733 >> v2:172.21.15.119:3300/0 conn(0x7f15e80a0ea0 0x7f15e8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 --2- 172.21.15.119:0/222217733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e8137bc0 0x7f15e8137fa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/222217733 >> 172.21.15.119:0/222217733 conn(0x7f15e80a2ae0 msgr2=0x7f15e80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/222217733 shutdown_connections 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/222217733 wait complete. 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: build_initial_monmap 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: monmap: 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:48.210 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 Processor -- start 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- start start 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: init 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e80a09e0 0x7f15e811d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): picked mon.noname-b con 0x7f15e80a09e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 0x7f15e811fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): picked mon.noname-a con 0x7f15e80a0ea0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f15e8137bc0 0x7f15e81368f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): picked mon.noname-c con 0x7f15e8137bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:48.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8073e70 con 0x7f15e80a0ea0 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8073cf0 con 0x7f15e80a09e0 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f15e81422d0 con 0x7f15e8137bc0 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient(hunting): authenticate will time out at 1346.997542s 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ec9bd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e80a09e0 msgr2=0x7f15e811d920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ec9bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e80a09e0 0x7f15e811d920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 0x7f15e811fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f15e8137bc0 0x7f15e81368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:48.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 0x7f15e811fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:37056/0 (socket says 172.21.15.119:37056) 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 -- 172.21.15.119:0/2067786796 learned_addr learned my addr 172.21.15.119:0/2067786796 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): get_auth_request con 0x7f15e8137bc0 auth_method 0 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): get_auth_request con 0x7f15e80a0ea0 auth_method 0 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:48.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15ed1be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient(hunting): handle_auth_done global_id 74238 payload 824 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 -- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f15e8137bc0 msgr2=0x7f15e81368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f15e8137bc0 0x7f15e81368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 -- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e80a09e0 msgr2=0x7f15e811d920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e80a09e0 0x7f15e811d920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 monclient: found mon.noname-a 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 -- 172.21.15.119:0/2067786796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e809d1d0 con 0x7f15e80a0ea0 2024-09-15T21:23:48.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e7fff640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 0x7f15e811fa80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b530 tx=0x7f15dc00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc013020 con 0x7f15e80a0ea0 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: dump: 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:48.215 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 5 monclient: authenticate success, global_id 74238 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _renew_subs 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e81209e0 con 0x7f15e80a0ea0 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc003df0 con 0x7f15e80a0ea0 2024-09-15T21:23:48.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0052b0 con 0x7f15e80a0ea0 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e57fa640 10 monclient: dump: 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:48.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _renew_subs 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e8137590 con 0x7f15e80a0ea0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15e6ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.198+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e8137fa0 con 0x7f15e80a0ea0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.202+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f15dc016020 con 0x7f15e80a0ea0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.202+0000 7f15e57fa640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f15d403b970 0x7f15d403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.202+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 9921+0+0 (secure 0 0 0) 0x7f15dc051190 con 0x7f15e80a0ea0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.202+0000 7f15ec9bd640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f15d403b970 0x7f15d403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:48.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.202+0000 7f15ec9bd640 10 monclient: get_auth_request con 0x7f15d403b970 auth_method 0 2024-09-15T21:23:48.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.202+0000 7f15ec9bd640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f15d403b970 0x7f15d403de30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f15d80af000 tx=0x7f15d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:48.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.214+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f15dc01d020 con 0x7f15e80a0ea0 2024-09-15T21:23:48.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.214+0000 7f15e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:48.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.214+0000 7f15e57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:48.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.386+0000 7f15eec21640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:48.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.386+0000 7f15eec21640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:48.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.386+0000 7f15eec21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:48.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:48.386+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7f15e806c460 con 0x7f15e80a0ea0 2024-09-15T21:23:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.238+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v168) ==== 123+0+0 (secure 0 0 0) 0x7f15dc00fe20 con 0x7f15e80a0ea0 2024-09-15T21:23:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.238+0000 7f15e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:49.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.238+0000 7f15e57fa640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-09-15T21:23:49.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.346+0000 7f15eec21640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:49.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.346+0000 7f15eec21640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:49.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.346+0000 7f15eec21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:49.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:49.346+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7f15e8136e30 con 0x7f15e80a0ea0 2024-09-15T21:23:50.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.246+0000 7f15e57fa640 1 -- 172.21.15.119:0/2067786796 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v169) ==== 149+0+0 (secure 0 0 0) 0x7f15dc0173b0 con 0x7f15e80a0ea0 2024-09-15T21:23:50.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.246+0000 7f15e57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T21:23:50.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.246+0000 7f15e57fa640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:23:50.251 INFO:tasks.workunit.client.0.smithi119.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f15d403b970 msgr2=0x7f15d403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f15d403b970 0x7f15d403de30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f15d80af000 tx=0x7f15d8008040 comp rx=0 tx=0).stop 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 10 monclient: shutdown 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 msgr2=0x7f15e811fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 0x7f15e811fa80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b530 tx=0x7f15dc00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 shutdown_connections 2024-09-15T21:23:50.255 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f15d403b970 0x7f15d403de30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f15e8137bc0 0x7f15e81368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15e80a0ea0 0x7f15e811fa80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 --2- 172.21.15.119:0/2067786796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e80a09e0 0x7f15e811d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 >> 172.21.15.119:0/2067786796 conn(0x7f15e80a2ae0 msgr2=0x7f15e80a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:50.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 shutdown_connections 2024-09-15T21:23:50.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.250+0000 7f15eec21640 1 -- 172.21.15.119:0/2067786796 wait complete. 2024-09-15T21:23:50.273 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool rm cool cool --yes-i-really-really-mean-it 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient: get_monmap_and_config 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient: build_initial_monmap 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient: monmap: 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:50.385504+0000 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:50.385504+0000 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:50.386 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:50.387 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:50.387 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:50.387 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:50.387 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 1 Processor -- start 2024-09-15T21:23:50.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 1 -- start start 2024-09-15T21:23:50.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient: init 2024-09-15T21:23:50.390 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0450135bf0 0x7f045009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient(hunting): picked mon.noname-c con 0x7f0450135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0450135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0450136460 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04500a1870 con 0x7f0450135fd0 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04500a1d80 con 0x7f0450136460 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f04500a3600 con 0x7f0450135bf0 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f04581a9640 10 monclient(hunting): authenticate will time out at 1349.182102s 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0455f45640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f0450135bf0 msgr2=0x7f045009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:50.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0456746640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f0450136460 legacy=0x7f04500a1160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0455f45640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0450135bf0 0x7f045009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0455744640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0450135fd0 0x7f04500a0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:44094/0 (socket says 172.21.15.119:44094) 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0455744640 1 -- 172.21.15.119:0/2646246045 learned_addr learned my addr 172.21.15.119:0/2646246045 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393554476 0 0) 0x7f04500a1870 con 0x7f0450135fd0 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): _negotiate 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): my global_id is 74241 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f043c003530 con 0x7f0450135fd0 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 852127433 0 0) 0x7f043c003530 con 0x7f0450135fd0 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 >> v2:172.21.15.119:3300/0 conn(0x7f0450135bf0 msgr2=0x7f045009fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 --2- 172.21.15.119:0/2646246045 >> v2:172.21.15.119:3300/0 conn(0x7f0450135bf0 0x7f045009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.392 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 >> v1:172.21.15.119:6789/0 conn(0x7f0450136460 legacy=0x7f04500a1160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: _finish_hunting 0 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04500a2060 con 0x7f0450135fd0 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: _finish_auth 0 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: _check_auth_tickets 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0440001700 con 0x7f0450135fd0 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: dump: 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:50.393 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:23:50.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 >> v1:172.21.15.17:6789/0 conn(0x7f0450135fd0 legacy=0x7f04500a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c003a10 0x7f043c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): picked mon.b con 0x7f043c003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f043c004810 0x7f043c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): picked mon.c con 0x7f043c004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c005420 0x7f043c00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): picked mon.a con 0x7f043c005420 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f04500a3600 con 0x7f043c005420 2024-09-15T21:23:50.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f043c00c360 con 0x7f043c003a10 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f043c00c560 con 0x7f043c004810 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0454f43640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0455f45640 1 -- 172.21.15.119:0/2646246045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c003a10 msgr2=0x7f043c003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.382+0000 7f0455f45640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c003a10 0x7f043c003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c005420 0x7f043c00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:50.396 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): get_auth_request con 0x7f043c005420 auth_method 0 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f043c004810 0x7f043c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): get_auth_request con 0x7f043c004810 auth_method 0 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:50.397 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0455744640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient(hunting): handle_auth_done global_id 74241 payload 293 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 -- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f043c004810 msgr2=0x7f043c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f043c004810 0x7f043c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 -- 172.21.15.119:0/2646246045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c003a10 msgr2=0x7f043c003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c003a10 0x7f043c003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient: _finish_hunting 0 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 monclient: found mon.a 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 -- 172.21.15.119:0/2646246045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c003530 con 0x7f043c005420 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient: _finish_auth 0 2024-09-15T21:23:50.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 10 monclient: _check_auth_tickets 2024-09-15T21:23:50.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:50.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 5 monclient: authenticate success, global_id 74241 2024-09-15T21:23:50.399 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:50.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 -- 172.21.15.119:0/2646246045 reap_dead start 2024-09-15T21:23:50.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0456746640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c005420 0x7f043c00ba80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e30 tx=0x7f044801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418d706e536c13b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:50.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448009a30 con 0x7f043c005420 2024-09-15T21:23:50.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: dump: 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:50.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04480a9070 con 0x7f043c005420 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 4 set_mon_vals no callback set 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 1 -- 172.21.15.119:0/2646246045 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480a5b60 con 0x7f043c005420 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f0454f43640 10 monclient: dump: 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:50.406 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 10 monclient: shutdown 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 -- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c005420 msgr2=0x7f043c00ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c005420 0x7f043c00ba80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e30 tx=0x7f044801c8e0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 -- 172.21.15.119:0/2646246045 shutdown_connections 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 --2- 172.21.15.119:0/2646246045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c005420 0x7f043c00ba80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 -- 172.21.15.119:0/2646246045 >> 172.21.15.119:0/2646246045 conn(0x7f04500a66e0 msgr2=0x7f04500a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 -- 172.21.15.119:0/2646246045 shutdown_connections 2024-09-15T21:23:50.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.386+0000 7f04581a9640 1 -- 172.21.15.119:0/2646246045 wait complete. 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: build_initial_monmap 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: monmap: 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.408 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 Processor -- start 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- start start 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: init 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c005420 0x7f04501cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): picked mon.noname-b con 0x7f043c005420 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 0x7f045009d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): picked mon.noname-c con 0x7f04501cfe20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04501d0240 0x7f045009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): picked mon.noname-a con 0x7f04501d0240 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f045006edb0 con 0x7f04501d0240 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.409 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f045006ec30 con 0x7f043c005420 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f045009fe70 con 0x7f04501cfe20 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient(hunting): authenticate will time out at 1349.189338s 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455f45640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c005420 msgr2=0x7f04501cf650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455f45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c005420 0x7f04501cf650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 0x7f045009d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04501d0240 0x7f045009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 0x7f045009d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:41802/0 (socket says 172.21.15.119:41802) 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 -- 172.21.15.119:0/4141093126 learned_addr learned my addr 172.21.15.119:0/4141093126 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): get_auth_request con 0x7f04501cfe20 auth_method 0 2024-09-15T21:23:50.410 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): get_auth_request con 0x7f04501d0240 auth_method 0 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient(hunting): handle_auth_done global_id 44810 payload 824 2024-09-15T21:23:50.411 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 -- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c005420 msgr2=0x7f04501cf650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c005420 0x7f04501cf650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 -- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04501d0240 msgr2=0x7f045009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04501d0240 0x7f045009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient: _finish_hunting 0 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 monclient: found mon.noname-c 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 -- 172.21.15.119:0/4141093126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045013e120 con 0x7f04501cfe20 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0456746640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04501d0240 0x7f045009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0455744640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 0x7f045009d580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0440001870 tx=0x7f0440007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000f7b0 con 0x7f04501cfe20 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:50.412 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: dump: 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:50.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 5 monclient: authenticate success, global_id 44810 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _renew_subs 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045013e400 con 0x7f04501cfe20 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0440018070 con 0x7f04501cfe20 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _renew_subs 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0450072720 con 0x7f04501cfe20 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000fbd0 con 0x7f04501cfe20 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:50.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04467fc640 10 monclient: dump: 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f0447fff640 4 set_mon_vals no callback set 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:50.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.390+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04500a0a50 con 0x7f04501cfe20 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.394+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f044001d460 con 0x7f04501cfe20 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.394+0000 7f04467fc640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f043403b9a0 0x7f043403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.394+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(169..169 src has 1..169) ==== 9232+0+0 (secure 0 0 0) 0x7f0440051630 con 0x7f04501cfe20 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.394+0000 7f0455f45640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f043403b9a0 0x7f043403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.394+0000 7f0455f45640 10 monclient: get_auth_request con 0x7f043403b9a0 auth_method 0 2024-09-15T21:23:50.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.394+0000 7f0455f45640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f043403b9a0 0x7f043403de60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:50.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.406+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f044000fd70 con 0x7f04501cfe20 2024-09-15T21:23:50.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.406+0000 7f04467fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:50.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.406+0000 7f04467fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:50.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.566+0000 7f04581a9640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:50.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.566+0000 7f04581a9640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:50.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.566+0000 7f04581a9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:50.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:50.566+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) -- 0x7f045013cf60 con 0x7f04501cfe20 2024-09-15T21:23:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.254+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' removed v170) ==== 148+0+0 (secure 0 0 0) 0x7f044001bd50 con 0x7f04501cfe20 2024-09-15T21:23:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.254+0000 7f04467fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.254+0000 7f04467fc640 10 monclient: _finish_command 2 = system:0 pool 'cool' removed 2024-09-15T21:23:51.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04581a9640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:51.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04581a9640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:51.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04581a9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:51.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0450135210 con 0x7f04501cfe20 2024-09-15T21:23:51.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04467fc640 1 -- 172.21.15.119:0/4141093126 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' does not exist v170) ==== 155+0+0 (secure 0 0 0) 0x7f044001bd50 con 0x7f04501cfe20 2024-09-15T21:23:51.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04467fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:51.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.354+0000 7f04467fc640 10 monclient: _finish_command 3 = system:0 pool 'cool' does not exist 2024-09-15T21:23:51.360 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'cool' does not exist 2024-09-15T21:23:51.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f043403b9a0 msgr2=0x7f043403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f043403b9a0 0x7f043403de60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006da0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 10 monclient: shutdown 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 msgr2=0x7f045009d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 0x7f045009d580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0440001870 tx=0x7f0440007430 comp rx=0 tx=0).stop 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 shutdown_connections 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f043403b9a0 0x7f043403de60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04501d0240 0x7f045009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f04501cfe20 0x7f045009d580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 --2- 172.21.15.119:0/4141093126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f043c005420 0x7f04501cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 >> 172.21.15.119:0/4141093126 conn(0x7f04500a66e0 msgr2=0x7f04500a3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 shutdown_connections 2024-09-15T21:23:51.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.358+0000 7f04581a9640 1 -- 172.21.15.119:0/4141093126 wait complete. 2024-09-15T21:23:51.377 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd pool rm cold cold --yes-i-really-really-mean-it 2024-09-15T21:23:51.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient: get_monmap_and_config 2024-09-15T21:23:51.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient: build_initial_monmap 2024-09-15T21:23:51.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient: monmap: 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:51.497026+0000 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:51.497026+0000 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:51.498 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:51.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 Processor -- start 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 -- start start 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient: init 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f25ac135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25ac09fe70 0x7f25ac13aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): picked mon.noname-a con 0x7f25ac09fe70 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25ac13afb0 0x7f25ac13d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): picked mon.noname-c con 0x7f25ac13afb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac0a0300 con 0x7f25ac135bf0 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:51.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f25ac0a3600 con 0x7f25ac09fe70 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f25ac0a1d80 con 0x7f25ac13afb0 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b2909640 10 monclient(hunting): authenticate will time out at 1350.293874s 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b0ea6640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f25ac13afb0 msgr2=0x7f25ac13d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25b0ea6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25ac13afb0 0x7f25ac13d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25ac09fe70 0x7f25ac13aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25ac09fe70 0x7f25ac13aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:37084/0 (socket says 172.21.15.119:37084) 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25abfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f25ac135bf0 0x7f25ac135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:44108/0 (socket says 172.21.15.119:44108) 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 -- 172.21.15.119:0/286226262 learned_addr learned my addr 172.21.15.119:0/286226262 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): get_auth_request con 0x7f25ac09fe70 auth_method 0 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:51.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_done global_id 74247 payload 274 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 -- 172.21.15.119:0/286226262 >> v2:172.21.15.119:3300/0 conn(0x7f25ac13afb0 msgr2=0x7f25ac13d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 --2- 172.21.15.119:0/286226262 >> v2:172.21.15.119:3300/0 conn(0x7f25ac13afb0 0x7f25ac13d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 -- 172.21.15.119:0/286226262 >> v1:172.21.15.17:6789/0 conn(0x7f25ac135bf0 legacy=0x7f25ac135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 monclient: found mon.noname-a 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:51.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.494+0000 7f25ab7fe640 1 -- 172.21.15.119:0/286226262 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac0a2060 con 0x7f25ac09fe70 2024-09-15T21:23:51.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25ab7fe640 1 --2- 172.21.15.119:0/286226262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac09fe70 0x7f25ac13aa70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f259c009e50 tx=0x7f259c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360f4fbf763cc69b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:51.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 1 -- 172.21.15.119:0/286226262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c0a5780 con 0x7f25ac09fe70 2024-09-15T21:23:51.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:51.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:51.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: dump: 2024-09-15T21:23:51.509 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:51.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 5 monclient: authenticate success, global_id 74247 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 1 -- 172.21.15.119:0/286226262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c0a5920 con 0x7f25ac09fe70 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 1 -- 172.21.15.119:0/286226262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c0a5cf0 con 0x7f25ac09fe70 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 4 set_mon_vals no callback set 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25aaffd640 10 monclient: dump: 2024-09-15T21:23:51.511 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: shutdown 2024-09-15T21:23:51.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- 172.21.15.119:0/286226262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac09fe70 msgr2=0x7f25ac13aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 --2- 172.21.15.119:0/286226262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac09fe70 0x7f25ac13aa70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f259c009e50 tx=0x7f259c01c8a0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- 172.21.15.119:0/286226262 shutdown_connections 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 --2- 172.21.15.119:0/286226262 >> v2:172.21.15.119:3300/0 conn(0x7f25ac13afb0 0x7f25ac13d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 --2- 172.21.15.119:0/286226262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac09fe70 0x7f25ac13aa70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- 172.21.15.119:0/286226262 >> 172.21.15.119:0/286226262 conn(0x7f25ac0a66e0 msgr2=0x7f25ac0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- 172.21.15.119:0/286226262 shutdown_connections 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- 172.21.15.119:0/286226262 wait complete. 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: build_initial_monmap 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: monmap: 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:51.513 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 Processor -- start 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- start start 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: init 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ac09fe70 0x7f25ac132150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:51.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): picked mon.noname-b con 0x7f25ac09fe70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 0x7f25ac139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): picked mon.noname-c con 0x7f25ac135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac13afb0 0x7f25ac09d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): picked mon.noname-a con 0x7f25ac13afb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac140100 con 0x7f25ac13afb0 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac06ecc0 con 0x7f25ac09fe70 2024-09-15T21:23:51.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f25ac140280 con 0x7f25ac135bf0 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25b2909640 10 monclient(hunting): authenticate will time out at 1350.299764s 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25abfff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ac09fe70 msgr2=0x7f25ac132150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.498+0000 7f25abfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ac09fe70 0x7f25ac132150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac13afb0 0x7f25ac09d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 0x7f25ac139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 0x7f25ac139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:41814/0 (socket says 172.21.15.119:41814) 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac13afb0 0x7f25ac09d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:37086/0 (socket says 172.21.15.119:37086) 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 -- 172.21.15.119:0/3334783072 learned_addr learned my addr 172.21.15.119:0/3334783072 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): get_auth_request con 0x7f25ac135bf0 auth_method 0 2024-09-15T21:23:51.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): get_auth_request con 0x7f25ac13afb0 auth_method 0 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b0ea6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient(hunting): handle_auth_done global_id 44813 payload 824 2024-09-15T21:23:51.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 -- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ac09fe70 msgr2=0x7f25ac132150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ac09fe70 0x7f25ac132150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 -- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac13afb0 msgr2=0x7f25ac09d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac13afb0 0x7f25ac09d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 monclient: found mon.noname-c 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 -- 172.21.15.119:0/3334783072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac137e10 con 0x7f25ac135bf0 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25ab7fe640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 0x7f25ac139bc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f259c0af040 tx=0x7f259c019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c0bc020 con 0x7f25ac135bf0 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: dump: 2024-09-15T21:23:51.518 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: _finish_auth 0 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c0b8880 con 0x7f25ac135bf0 2024-09-15T21:23:51.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 5 monclient: authenticate success, global_id 44813 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c0b8c50 con 0x7f25ac135bf0 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 10 monclient: dump: 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:51.520 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 monclient: _renew_subs 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25aa7fc640 4 set_mon_vals no callback set 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1380f0 con 0x7f25ac135bf0 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 monclient: _renew_subs 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac09ef70 con 0x7f25ac135bf0 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac135fd0 con 0x7f25ac135bf0 2024-09-15T21:23:51.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f259c0bf020 con 0x7f25ac135bf0 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f258c03b9a0 0x7f258c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(170..170 src has 1..170) ==== 8561+0+0 (secure 0 0 0) 0x7f259c0ec220 con 0x7f25ac135bf0 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25abfff640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f258c03b9a0 0x7f258c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25abfff640 10 monclient: get_auth_request con 0x7f258c03b9a0 auth_method 0 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.502+0000 7f25abfff640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f258c03b9a0 0x7f258c03de60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f25ac09eb20 tx=0x7f2594007a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.514+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f259c0165e0 con 0x7f25ac135bf0 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.514+0000 7f25a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:51.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.514+0000 7f25a8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:51.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.686+0000 7f25b2909640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:51.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.686+0000 7f25b2909640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:51.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.686+0000 7f25b2909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:51.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:51.686+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25ac06a830 con 0x7f25ac135bf0 2024-09-15T21:23:52.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.242+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' removed v171) ==== 148+0+0 (secure 0 0 0) 0x7f259c0b8df0 con 0x7f25ac135bf0 2024-09-15T21:23:52.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.242+0000 7f25a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:52.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.242+0000 7f25a8ff9640 10 monclient: _finish_command 2 = system:0 pool 'cold' removed 2024-09-15T21:23:52.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.342+0000 7f25b2909640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:52.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.342+0000 7f25b2909640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:52.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.346+0000 7f25b2909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.346+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25ac138330 con 0x7f25ac135bf0 2024-09-15T21:23:52.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.346+0000 7f25a8ff9640 1 -- 172.21.15.119:0/3334783072 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' does not exist v171) ==== 155+0+0 (secure 0 0 0) 0x7f259c0e9d30 con 0x7f25ac135bf0 2024-09-15T21:23:52.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.346+0000 7f25a8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T21:23:52.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.346+0000 7f25a8ff9640 10 monclient: _finish_command 3 = system:0 pool 'cold' does not exist 2024-09-15T21:23:52.351 INFO:tasks.workunit.client.0.smithi119.stderr:pool 'cold' does not exist 2024-09-15T21:23:52.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f258c03b9a0 msgr2=0x7f258c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:52.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f258c03b9a0 0x7f258c03de60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f25ac09eb20 tx=0x7f2594007a90 comp rx=0 tx=0).stop 2024-09-15T21:23:52.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 10 monclient: shutdown 2024-09-15T21:23:52.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:52.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 msgr2=0x7f25ac139bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:52.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 0x7f25ac139bc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f259c0af040 tx=0x7f259c019640 comp rx=0 tx=0).stop 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 shutdown_connections 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f258c03b9a0 0x7f258c03de60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25ac13afb0 0x7f25ac09d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f25ac135bf0 0x7f25ac139bc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 --2- 172.21.15.119:0/3334783072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ac09fe70 0x7f25ac132150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 >> 172.21.15.119:0/3334783072 conn(0x7f25ac0a66e0 msgr2=0x7f25ac0a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 shutdown_connections 2024-09-15T21:23:52.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.350+0000 7f25b2909640 1 -- 172.21.15.119:0/3334783072 wait complete. 2024-09-15T21:23:52.369 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T21:23:52.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient: get_monmap_and_config 2024-09-15T21:23:52.490 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient: build_initial_monmap 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient: monmap: 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:52.490009+0000 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:52.490009+0000 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:52.491 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 1 Processor -- start 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 1 -- start start 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient: init 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7ba8135ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:52.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ba80a0160 0x7f7ba813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient(hunting): picked mon.noname-c con 0x7f7ba80a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7ba806ec30 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bae452640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f7ba8135ca0 legacy=0x7f7ba8136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba809cf30 con 0x7f7ba806ec30 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba80a1ca0 con 0x7f7ba8135ca0 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba80a1f80 con 0x7f7ba80a0160 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7badc51640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f7ba80a0160 msgr2=0x7f7ba813aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bb06b6640 10 monclient(hunting): authenticate will time out at 1351.286880s 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7badc51640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ba80a0160 0x7f7ba813aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7baec53640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7ba806ec30 0x7f7ba813cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:56226/0 (socket says 172.21.15.119:56226) 2024-09-15T21:23:52.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7baec53640 1 -- 172.21.15.119:0/74867547 learned_addr learned my addr 172.21.15.119:0/74867547 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435852455 0 0) 0x7f7ba809cf30 con 0x7f7ba806ec30 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bad450640 10 monclient(hunting): _negotiate 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bad450640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bad450640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bad450640 10 monclient(hunting): my global_id is 74256 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.486+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b94003530 con 0x7f7ba806ec30 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1562143774 0 0) 0x7f7b94003530 con 0x7f7ba806ec30 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 >> v2:172.21.15.119:3300/0 conn(0x7f7ba80a0160 msgr2=0x7f7ba813aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 --2- 172.21.15.119:0/74867547 >> v2:172.21.15.119:3300/0 conn(0x7f7ba80a0160 0x7f7ba813aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 >> v1:172.21.15.119:6789/0 conn(0x7f7ba8135ca0 legacy=0x7f7ba8136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: _finish_hunting 0 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:23:52.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba80a3860 con 0x7f7ba806ec30 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: _finish_auth 0 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: _check_auth_tickets 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7ba0001a80 con 0x7f7ba806ec30 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: dump: 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.500 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 >> v1:172.21.15.17:6789/0 conn(0x7f7ba806ec30 legacy=0x7f7ba813cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b940039f0 0x7f7b94003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): picked mon.b con 0x7f7b940039f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b94004810 0x7f7b94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): picked mon.a con 0x7f7b94004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7b9400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): picked mon.c con 0x7f7b940053b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba80a1f80 con 0x7f7b94004810 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9400c2f0 con 0x7f7b940039f0 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9400c4f0 con 0x7f7b940053b0 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bae452640 1 -- 172.21.15.119:0/74867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b940039f0 msgr2=0x7f7b94003df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bae452640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b940039f0 0x7f7b94003df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:52.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7b9400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b94004810 0x7f7b94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): get_auth_request con 0x7f7b940053b0 auth_method 0 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): get_auth_request con 0x7f7b94004810 auth_method 0 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7badc51640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient(hunting): handle_auth_done global_id 74256 payload 293 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 -- 172.21.15.119:0/74867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b940039f0 msgr2=0x7f7b94003df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b940039f0 0x7f7b94003df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 -- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b94004810 msgr2=0x7f7b94004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b94004810 0x7f7b94004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient: _finish_hunting 0 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 monclient: found mon.c 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 -- 172.21.15.119:0/74867547 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b94003530 con 0x7f7b940053b0 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient: _finish_auth 0 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 10 monclient: _check_auth_tickets 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bb06b6640 5 monclient: authenticate success, global_id 74256 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bb06b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 -- 172.21.15.119:0/74867547 reap_dead start 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7baec53640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7b9400ba10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ba806f0c0 tx=0x7f7ba001d240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fc2a09da6a0bba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000a480 con 0x7f7b940053b0 2024-09-15T21:23:52.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: dump: 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba00ac620 con 0x7f7b940053b0 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:52.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 1 -- 172.21.15.119:0/74867547 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00ac9f0 con 0x7f7b940053b0 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bb06b6640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bb06b6640 4 set_mon_vals no callback set 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.490+0000 7f7bad450640 10 monclient: dump: 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:52.507 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.508 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:52.508 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:52.508 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:52.508 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient: shutdown 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 msgr2=0x7f7b9400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7b9400ba10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ba806f0c0 tx=0x7f7ba001d240 comp rx=0 tx=0).stop 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- 172.21.15.119:0/74867547 shutdown_connections 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/74867547 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7b9400ba10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- 172.21.15.119:0/74867547 >> 172.21.15.119:0/74867547 conn(0x7f7ba80a2ae0 msgr2=0x7f7ba80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- 172.21.15.119:0/74867547 shutdown_connections 2024-09-15T21:23:52.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- 172.21.15.119:0/74867547 wait complete. 2024-09-15T21:23:52.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient: build_initial_monmap 2024-09-15T21:23:52.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient: monmap: 2024-09-15T21:23:52.514 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:52.514 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:52.514 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:52.514 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 Processor -- start 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- start start 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient: init 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:52.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7ba813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): picked mon.noname-c con 0x7f7b940053b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba813afc0 0x7f7ba813b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): picked mon.noname-a con 0x7f7ba813afc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba81cce90 0x7f7ba81cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): picked mon.noname-b con 0x7f7ba81cce90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba8136200 con 0x7f7ba813afc0 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba8136080 con 0x7f7ba81cce90 2024-09-15T21:23:52.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba8136380 con 0x7f7b940053b0 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bb06b6640 10 monclient(hunting): authenticate will time out at 1351.294761s 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7baec53640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba81cce90 msgr2=0x7f7ba81cd320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7baec53640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba81cce90 0x7f7ba81cd320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba813afc0 0x7f7ba813b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7ba813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba813afc0 0x7f7ba813b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36278/0 (socket says 172.21.15.119:36278) 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7ba813d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46384/0 (socket says 172.21.15.119:46384) 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 1 -- 172.21.15.119:0/2469644003 learned_addr learned my addr 172.21.15.119:0/2469644003 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): get_auth_request con 0x7f7ba813afc0 auth_method 0 2024-09-15T21:23:52.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): get_auth_request con 0x7f7b940053b0 auth_method 0 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7badc51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:52.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient(hunting): handle_auth_done global_id 44816 payload 824 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 -- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba81cce90 msgr2=0x7f7ba81cd320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba81cce90 0x7f7ba81cd320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 -- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba813afc0 msgr2=0x7f7ba813b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba813afc0 0x7f7ba813b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient: _finish_hunting 0 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 monclient: found mon.noname-c 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.494+0000 7f7bae452640 1 -- 172.21.15.119:0/2469644003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba811c9f0 con 0x7f7b940053b0 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bae452640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7ba813d0f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800b430 tx=0x7f7b9800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98016070 con 0x7f7b940053b0 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:52.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: dump: 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: _finish_auth 0 2024-09-15T21:23:52.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b98003df0 con 0x7f7b940053b0 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980052b0 con 0x7f7b940053b0 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 10 monclient: dump: 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:52.521 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 5 monclient: authenticate success, global_id 44816 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 monclient: _renew_subs 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba811cf90 con 0x7f7b940053b0 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bacc4f640 4 set_mon_vals no callback set 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 monclient: _renew_subs 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba811d830 con 0x7f7b940053b0 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:52.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba80a09e0 con 0x7f7b940053b0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7b98019070 con 0x7f7b940053b0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(171..171 src has 1..171) ==== 7754+0+0 (secure 0 0 0) 0x7f7b9804fd20 con 0x7f7b940053b0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7badc51640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7badc51640 10 monclient: get_auth_request con 0x7f7b8c03b970 auth_method 0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.498+0000 7f7badc51640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4006d10 tx=0x7f7ba4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:52.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.510+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7b98020020 con 0x7f7b940053b0 2024-09-15T21:23:52.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.510+0000 7f7b9effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:52.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.510+0000 7f7b9effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:52.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.674+0000 7f7bb06b6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:23:52.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.674+0000 7f7bb06b6640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:23:52.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.674+0000 7f7bb06b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:52.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:52.674+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f7ba809f8f0 con 0x7f7b940053b0 2024-09-15T21:23:53.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.278+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v172) ==== 78+0+0 (secure 0 0 0) 0x7f7ba811cf90 con 0x7f7b940053b0 2024-09-15T21:23:53.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.278+0000 7f7b9effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:23:53.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.278+0000 7f7b9effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:53.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.398+0000 7f7bb06b6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:23:53.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.398+0000 7f7bb06b6640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:23:53.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.398+0000 7f7bb06b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:53.404 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:53.398+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f7ba80008e0 con 0x7f7b940053b0 2024-09-15T21:23:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.286+0000 7f7b9effd640 1 -- 172.21.15.119:0/2469644003 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v173) ==== 78+0+0 (secure 0 0 0) 0x7f7b9801a3b0 con 0x7f7b940053b0 2024-09-15T21:23:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.286+0000 7f7b9effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:23:54.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.286+0000 7f7b9effd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:23:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 msgr2=0x7f7b8c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4006d10 tx=0x7f7ba4008040 comp rx=0 tx=0).stop 2024-09-15T21:23:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 10 monclient: shutdown 2024-09-15T21:23:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 msgr2=0x7f7ba813d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:54.299 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7ba813d0f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800b430 tx=0x7f7b9800b8f0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 shutdown_connections 2024-09-15T21:23:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b8c03b970 0x7f7b8c03de30 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4006d10 tx=0x7f7ba4008040 comp rx=0 tx=0).stop 2024-09-15T21:23:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba81cce90 0x7f7ba81cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ba813afc0 0x7f7ba813b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 --2- 172.21.15.119:0/2469644003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b940053b0 0x7f7ba813d0f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.294+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 >> 172.21.15.119:0/2469644003 conn(0x7f7ba80a2ae0 msgr2=0x7f7ba80a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.298+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 shutdown_connections 2024-09-15T21:23:54.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.298+0000 7f7bb06b6640 1 -- 172.21.15.119:0/2469644003 wait complete. 2024-09-15T21:23:54.325 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient: get_monmap_and_config 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient: build_initial_monmap 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient: monmap: 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:54.442561+0000 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:54.442561+0000 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:54.443 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:54.444 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:54.444 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 Processor -- start 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 -- start start 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient: init 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd14137bc0 0x7fbd14137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): picked mon.noname-c con 0x7fbd14137bc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 0x7fbd14073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): picked mon.noname-b con 0x7fbd140a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:54.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbd140a0ea0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd1409cfa0 con 0x7fbd140a0ea0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbd1409cd20 con 0x7fbd14137bc0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbd140a1820 con 0x7fbd140a09e0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd12d9d640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fbd140a0ea0 legacy=0x7fbd14073580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd1259c640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fbd14137bc0 msgr2=0x7fbd14137fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd18d4a640 10 monclient(hunting): authenticate will time out at 1353.239432s 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd1259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd14137bc0 0x7fbd14137fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 0x7fbd14073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 0x7fbd14073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46398/0 (socket says 172.21.15.119:46398) 2024-09-15T21:23:54.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1298777860 learned_addr learned my addr 172.21.15.119:0/1298777860 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 10 monclient(hunting): get_auth_request con 0x7fbd140a09e0 auth_method 0 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.438+0000 7fbd11d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_done global_id 44819 payload 274 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1298777860 >> v2:172.21.15.119:3300/0 conn(0x7fbd14137bc0 msgr2=0x7fbd14137fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 1 --2- 172.21.15.119:0/1298777860 >> v2:172.21.15.119:3300/0 conn(0x7fbd14137bc0 0x7fbd14137fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1298777860 >> v1:172.21.15.119:6789/0 conn(0x7fbd140a0ea0 legacy=0x7fbd14073580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 1 monclient: found mon.noname-b 2024-09-15T21:23:54.448 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:54.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:54.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1298777860 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd140a1b00 con 0x7fbd140a09e0 2024-09-15T21:23:54.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd11d9b640 1 --2- 172.21.15.119:0/1298777860 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 0x7fbd14073040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd140a3ba0 tx=0x7fbd0801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eaf8ceb53dbf1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:54.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 1 -- 172.21.15.119:0/1298777860 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080b0070 con 0x7fbd140a09e0 2024-09-15T21:23:54.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:54.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:54.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: dump: 2024-09-15T21:23:54.453 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: _finish_auth 0 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: _check_auth_tickets 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 1 -- 172.21.15.119:0/1298777860 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd080ab600 con 0x7fbd140a09e0 2024-09-15T21:23:54.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 5 monclient: authenticate success, global_id 44819 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 1 -- 172.21.15.119:0/1298777860 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080ab9d0 con 0x7fbd140a09e0 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 4 set_mon_vals no callback set 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd1159a640 10 monclient: dump: 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:54.455 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 10 monclient: shutdown 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1298777860 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 msgr2=0x7fbd14073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1298777860 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 0x7fbd14073040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd140a3ba0 tx=0x7fbd0801c510 comp rx=0 tx=0).stop 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1298777860 shutdown_connections 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1298777860 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a09e0 0x7fbd14073040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1298777860 >> v2:172.21.15.119:3300/0 conn(0x7fbd14137bc0 0x7fbd14137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.442+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1298777860 >> 172.21.15.119:0/1298777860 conn(0x7fbd140a2ae0 msgr2=0x7fbd140a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1298777860 shutdown_connections 2024-09-15T21:23:54.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1298777860 wait complete. 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: build_initial_monmap 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: monmap: 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 Processor -- start 2024-09-15T21:23:54.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- start start 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: init 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd140a09e0 0x7fbd1411fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): picked mon.noname-a con 0x7fbd140a09e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a0ea0 0x7fbd1411d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): picked mon.noname-c con 0x7fbd140a0ea0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd14137bc0 0x7fbd14136940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): picked mon.noname-b con 0x7fbd14137bc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14073e10 con 0x7fbd140a09e0 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14073c90 con 0x7fbd14137bc0 2024-09-15T21:23:54.458 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbd141422d0 con 0x7fbd140a0ea0 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient(hunting): authenticate will time out at 1353.245911s 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd12d9d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd14137bc0 msgr2=0x7fbd14136940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd12d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd14137bc0 0x7fbd14136940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd140a09e0 0x7fbd1411fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a0ea0 0x7fbd1411d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd140a09e0 0x7fbd1411fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36280/0 (socket says 172.21.15.119:36280) 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 1 -- 172.21.15.119:0/1785512018 learned_addr learned my addr 172.21.15.119:0/1785512018 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): get_auth_request con 0x7fbd140a09e0 auth_method 0 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:54.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): get_auth_request con 0x7fbd140a0ea0 auth_method 0 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd1259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient(hunting): handle_auth_done global_id 44822 payload 824 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd14137bc0 msgr2=0x7fbd14136940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd14137bc0 0x7fbd14136940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd140a09e0 msgr2=0x7fbd1411fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd140a09e0 0x7fbd1411fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 monclient: found mon.noname-c 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 -- 172.21.15.119:0/1785512018 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd141375e0 con 0x7fbd140a0ea0 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd11d9b640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a0ea0 0x7fbd1411d960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ab000 tx=0x7fbd0801ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080ba020 con 0x7fbd140a0ea0 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: dump: 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:54.461 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 5 monclient: authenticate success, global_id 44822 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: _renew_subs 2024-09-15T21:23:54.462 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd080b6620 con 0x7fbd140a0ea0 2024-09-15T21:23:54.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:54.463 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1409d1d0 con 0x7fbd140a0ea0 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080b0040 con 0x7fbd140a0ea0 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd02ffd640 10 monclient: dump: 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:54.464 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd03fff640 4 set_mon_vals no callback set 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: _renew_subs 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.446+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd14120a30 con 0x7fbd140a0ea0 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd18d4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd18d4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd18d4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:54.465 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd14137fa0 con 0x7fbd140a0ea0 2024-09-15T21:23:54.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbd080bd020 con 0x7fbd140a0ea0 2024-09-15T21:23:54.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd02ffd640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcf403b9a0 0x7fbcf403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:54.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(173..173 src has 1..173) ==== 7273+0+0 (secure 0 0 0) 0x7fbd080ead90 con 0x7fbd140a0ea0 2024-09-15T21:23:54.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd1259c640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcf403b9a0 0x7fbcf403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:54.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd1259c640 10 monclient: get_auth_request con 0x7fbcf403b9a0 auth_method 0 2024-09-15T21:23:54.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.450+0000 7fbd1259c640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcf403b9a0 0x7fbcf403de60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd04003ee0 tx=0x7fbd040091c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:54.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.462+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbd080c4030 con 0x7fbd140a0ea0 2024-09-15T21:23:54.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.462+0000 7fbd02ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:54.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.462+0000 7fbd02ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:54.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.638+0000 7fbd18d4a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:23:54.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.638+0000 7fbd18d4a640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:23:54.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.638+0000 7fbd18d4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:54.643 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:54.638+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7fbd1411ef70 con 0x7fbd140a0ea0 2024-09-15T21:23:55.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.294+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v174) ==== 82+0+0 (secure 0 0 0) 0x7fbd08016130 con 0x7fbd140a0ea0 2024-09-15T21:23:55.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.294+0000 7fbd02ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:23:55.301 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.294+0000 7fbd02ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:23:55.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.410+0000 7fbd18d4a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:23:55.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.410+0000 7fbd18d4a640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:23:55.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.410+0000 7fbd18d4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:55.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.410+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7fbd14136e80 con 0x7fbd140a0ea0 2024-09-15T21:23:55.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.414+0000 7fbd02ffd640 1 -- 172.21.15.119:0/1785512018 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v174) ==== 115+0+0 (secure 0 0 0) 0x7fbd080ab480 con 0x7fbd140a0ea0 2024-09-15T21:23:55.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.414+0000 7fbd02ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:23:55.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.414+0000 7fbd02ffd640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T21:23:55.419 INFO:tasks.workunit.client.0.smithi119.stderr:compat weight-set already created 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcf403b9a0 msgr2=0x7fbcf403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcf403b9a0 0x7fbcf403de60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd04003ee0 tx=0x7fbd040091c0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 10 monclient: shutdown 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a0ea0 msgr2=0x7fbd1411d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a0ea0 0x7fbd1411d960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ab000 tx=0x7fbd0801ad50 comp rx=0 tx=0).stop 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 shutdown_connections 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcf403b9a0 0x7fbcf403de60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd14137bc0 0x7fbd14136940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbd140a0ea0 0x7fbd1411d960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 --2- 172.21.15.119:0/1785512018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd140a09e0 0x7fbd1411fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 >> 172.21.15.119:0/1785512018 conn(0x7fbd140a2ae0 msgr2=0x7fbd140a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:55.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 shutdown_connections 2024-09-15T21:23:55.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.418+0000 7fbd18d4a640 1 -- 172.21.15.119:0/1785512018 wait complete. 2024-09-15T21:23:55.437 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket fooo host 2024-09-15T21:23:55.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient: get_monmap_and_config 2024-09-15T21:23:55.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient: build_initial_monmap 2024-09-15T21:23:55.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient: monmap: 2024-09-15T21:23:55.557 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:55.557119+0000 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:55.557119+0000 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:55.558 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 1 Processor -- start 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 1 -- start start 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient: init 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02ec06dc00 0x7f02ec0a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient(hunting): picked mon.noname-c con 0x7f02ec06dc00 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f02ec09ec20 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02ec0a18a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ec09e240 con 0x7f02ec0a18a0 2024-09-15T21:23:55.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ec09e520 con 0x7f02ec09ec20 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f02ec09e800 con 0x7f02ec06dc00 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02f1ade640 10 monclient(hunting): authenticate will time out at 1354.353737s 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f02ec09ec20 legacy=0x7f02ec0a1190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eb7fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f02ec06dc00 msgr2=0x7f02ec0a0010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02ec06dc00 0x7f02ec0a0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f02ec0a18a0 0x7f02ec13b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:56228/0 (socket says 172.21.15.119:56228) 2024-09-15T21:23:55.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 1 -- 172.21.15.119:0/2722514683 learned_addr learned my addr 172.21.15.119:0/2722514683 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 762212495 0 0) 0x7f02ec09e240 con 0x7f02ec0a18a0 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): my global_id is 74265 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02d4003450 con 0x7f02ec0a18a0 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4189388252 0 0) 0x7f02d4003450 con 0x7f02ec0a18a0 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 >> v2:172.21.15.119:3300/0 conn(0x7f02ec06dc00 msgr2=0x7f02ec0a0010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 --2- 172.21.15.119:0/2722514683 >> v2:172.21.15.119:3300/0 conn(0x7f02ec06dc00 0x7f02ec0a0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 >> v1:172.21.15.119:6789/0 conn(0x7f02ec09ec20 legacy=0x7f02ec0a1190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:55.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec074110 con 0x7f02ec0a18a0 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f02e4001a80 con 0x7f02ec0a18a0 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: dump: 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.565 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 >> v1:172.21.15.17:6789/0 conn(0x7f02ec0a18a0 legacy=0x7f02ec13b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02d4003b00 0x7f02d4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): picked mon.b con 0x7f02d4003b00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02d400af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): picked mon.a con 0x7f02d4008ab0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:55.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02d400b7e0 0x7f02d400dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): picked mon.c con 0x7f02d400b7e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec09e800 con 0x7f02d4008ab0 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02d400e580 con 0x7f02d4003b00 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f02d400e780 con 0x7f02d400b7e0 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ea7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eb7fe640 1 -- 172.21.15.119:0/2722514683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02d4003b00 msgr2=0x7f02d4003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eb7fe640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02d4003b00 0x7f02d4003f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02d400af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02d400b7e0 0x7f02d400dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:55.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 10 monclient(hunting): get_auth_request con 0x7f02d4008ab0 auth_method 0 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 10 monclient(hunting): get_auth_request con 0x7f02d400b7e0 auth_method 0 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02ebfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.554+0000 7f02eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:55.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 10 monclient(hunting): handle_auth_done global_id 74265 payload 293 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 -- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02d400b7e0 msgr2=0x7f02d400dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02d400b7e0 0x7f02d400dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 -- 172.21.15.119:0/2722514683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02d4003b00 msgr2=0x7f02d4003f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02d4003b00 0x7f02d4003f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 monclient: found mon.a 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 -- 172.21.15.119:0/2722514683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d4003450 con 0x7f02d4008ab0 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:55.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ebfff640 1 -- 172.21.15.119:0/2722514683 reap_dead start 2024-09-15T21:23:55.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 5 monclient: authenticate success, global_id 74265 2024-09-15T21:23:55.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:55.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02eaffd640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02d400af40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009900 tx=0x7f02e001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1beced9652d13e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:55.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0009500 con 0x7f02d4008ab0 2024-09-15T21:23:55.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:55.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:55.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: dump: 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e0003bb0 con 0x7f02d4008ab0 2024-09-15T21:23:55.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 1 -- 172.21.15.119:0/2722514683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0005070 con 0x7f02d4008ab0 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02ea7fc640 10 monclient: dump: 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:55.577 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 4 set_mon_vals no callback set 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 10 monclient: shutdown 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 -- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 msgr2=0x7f02d400af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02d400af40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009900 tx=0x7f02e001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 -- 172.21.15.119:0/2722514683 shutdown_connections 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 --2- 172.21.15.119:0/2722514683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02d400af40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 -- 172.21.15.119:0/2722514683 >> 172.21.15.119:0/2722514683 conn(0x7f02ec09be60 msgr2=0x7f02ec09c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 -- 172.21.15.119:0/2722514683 shutdown_connections 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.558+0000 7f02f1ade640 1 -- 172.21.15.119:0/2722514683 wait complete. 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: build_initial_monmap 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: monmap: 2024-09-15T21:23:55.578 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 Processor -- start 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- start start 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: init 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02ec133940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): picked mon.noname-a con 0x7f02d4008ab0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02ec1cfc20 0x7f02ec133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): picked mon.noname-c con 0x7f02ec1cfc20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec126570 0x7f02ec126a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): picked mon.noname-b con 0x7f02ec126570 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0a1290 con 0x7f02d4008ab0 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0a1110 con 0x7f02ec126570 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec0a1410 con 0x7f02ec1cfc20 2024-09-15T21:23:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient(hunting): authenticate will time out at 1354.360996s 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02ebfff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec126570 msgr2=0x7f02ec126a00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec126570 0x7f02ec126a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02ec1cfc20 0x7f02ec133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02ec133940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02ec133940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36302/0 (socket says 172.21.15.119:36302) 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 -- 172.21.15.119:0/1603326256 learned_addr learned my addr 172.21.15.119:0/1603326256 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): get_auth_request con 0x7f02d4008ab0 auth_method 0 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:55.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): get_auth_request con 0x7f02ec1cfc20 auth_method 0 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient(hunting): handle_auth_done global_id 74268 payload 824 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 -- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02ec1cfc20 msgr2=0x7f02ec133e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:55.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02ec1cfc20 0x7f02ec133e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 -- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec126570 msgr2=0x7f02ec126a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec126570 0x7f02ec126a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 monclient: found mon.noname-a 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 -- 172.21.15.119:0/1603326256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec09d0f0 con 0x7f02d4008ab0 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eaffd640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02ec1cfc20 0x7f02ec133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02eb7fe640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02ec133940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b580 tx=0x7f02dc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc003d00 con 0x7f02d4008ab0 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: dump: 2024-09-15T21:23:55.583 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: _finish_auth 0 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 5 monclient: authenticate success, global_id 74268 2024-09-15T21:23:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _renew_subs 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec1343c0 con 0x7f02d4008ab0 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02dc015070 con 0x7f02d4008ab0 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc005330 con 0x7f02d4008ab0 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 10 monclient: dump: 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _renew_subs 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec12a0b0 con 0x7f02d4008ab0 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02e9ffb640 4 set_mon_vals no callback set 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec1366e0 con 0x7f02d4008ab0 2024-09-15T21:23:55.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.562+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f02dc0054d0 con 0x7f02d4008ab0 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.566+0000 7f02d3fff640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f02d803b970 0x7f02d803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.566+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 7601+0+0 (secure 0 0 0) 0x7f02dc050fd0 con 0x7f02d4008ab0 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.566+0000 7f02eaffd640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f02d803b970 0x7f02d803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.566+0000 7f02eaffd640 10 monclient: get_auth_request con 0x7f02d803b970 auth_method 0 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.566+0000 7f02eaffd640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f02d803b970 0x7f02d803de30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f02e00aa000 tx=0x7f02e00b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.578+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f02dc023cf0 con 0x7f02d4008ab0 2024-09-15T21:23:55.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.578+0000 7f02d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:55.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.578+0000 7f02d3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:55.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.742+0000 7f02f1ade640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T21:23:55.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.742+0000 7f02f1ade640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T21:23:55.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.742+0000 7f02f1ade640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:55.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:55.742+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) -- 0x7f02ec12a450 con 0x7f02d4008ab0 2024-09-15T21:23:56.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.298+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 added bucket fooo type host to crush map v175) ==== 140+0+0 (secure 0 0 0) 0x7f02dc04ded0 con 0x7f02d4008ab0 2024-09-15T21:23:56.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.298+0000 7f02d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T21:23:56.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.298+0000 7f02d3fff640 10 monclient: _finish_command 2 = system:0 added bucket fooo type host to crush map 2024-09-15T21:23:56.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02f1ade640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T21:23:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02f1ade640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T21:23:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02f1ade640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) -- 0x7f02ec11ec70 con 0x7f02d4008ab0 2024-09-15T21:23:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02d3fff640 1 -- 172.21.15.119:0/1603326256 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 bucket 'fooo' already exists v175) ==== 128+0+0 (secure 0 0 0) 0x7f02dc04e250 con 0x7f02d4008ab0 2024-09-15T21:23:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02d3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T21:23:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.398+0000 7f02d3fff640 10 monclient: _finish_command 3 = system:0 bucket 'fooo' already exists 2024-09-15T21:23:56.403 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'fooo' already exists 2024-09-15T21:23:56.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f02d803b970 msgr2=0x7f02d803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:56.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f02d803b970 0x7f02d803de30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f02e00aa000 tx=0x7f02e00b0040 comp rx=0 tx=0).stop 2024-09-15T21:23:56.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 10 monclient: shutdown 2024-09-15T21:23:56.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:56.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 msgr2=0x7f02ec133940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:56.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02ec133940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b580 tx=0x7f02dc00ba40 comp rx=0 tx=0).stop 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 shutdown_connections 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f02d803b970 0x7f02d803de30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec126570 0x7f02ec126a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02ec1cfc20 0x7f02ec133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 --2- 172.21.15.119:0/1603326256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4008ab0 0x7f02ec133940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 >> 172.21.15.119:0/1603326256 conn(0x7f02ec09be60 msgr2=0x7f02ec073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 shutdown_connections 2024-09-15T21:23:56.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.402+0000 7f02f1ade640 1 -- 172.21.15.119:0/1603326256 wait complete. 2024-09-15T21:23:56.420 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush move fooo root=default 2024-09-15T21:23:56.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.526+0000 7f6258ca8640 10 monclient: get_monmap_and_config 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.526+0000 7f6258ca8640 10 monclient: build_initial_monmap 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.526+0000 7f6258ca8640 10 monclient: monmap: 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:56.532363+0000 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:56.532363+0000 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:56.533 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:56.534 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:56.534 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 Processor -- start 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 -- start start 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient: init 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f62540a79e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 0x7f625413cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): picked mon.noname-b con 0x7f62540a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f625413d0a0 0x7f625413f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): picked mon.noname-c con 0x7f625413d0a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6254135180 con 0x7f62540a79e0 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f625259c640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f62540a79e0 legacy=0x7f62540a4170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:56.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f62540a7dc0 con 0x7f625413d0a0 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f62540a6920 con 0x7f62540a1100 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6258ca8640 10 monclient(hunting): authenticate will time out at 1355.329548s 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6252d9d640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f625413d0a0 msgr2=0x7f625413f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6252d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f625413d0a0 0x7f625413f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 0x7f625413cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 0x7f625413cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46450/0 (socket says 172.21.15.119:46450) 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 -- 172.21.15.119:0/3545117946 learned_addr learned my addr 172.21.15.119:0/3545117946 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): get_auth_request con 0x7f62540a1100 auth_method 0 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:56.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient(hunting): handle_auth_done global_id 44828 payload 274 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 -- 172.21.15.119:0/3545117946 >> v2:172.21.15.119:3300/0 conn(0x7f625413d0a0 msgr2=0x7f625413f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 --2- 172.21.15.119:0/3545117946 >> v2:172.21.15.119:3300/0 conn(0x7f625413d0a0 0x7f625413f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 -- 172.21.15.119:0/3545117946 >> v1:172.21.15.119:6789/0 conn(0x7f62540a79e0 legacy=0x7f62540a4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 monclient: found mon.noname-b 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:23:56.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.530+0000 7f6251d9b640 1 -- 172.21.15.119:0/3545117946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62540a1c30 con 0x7f62540a1100 2024-09-15T21:23:56.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 1 --2- 172.21.15.119:0/3545117946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 0x7f625413cb60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6248009870 tx=0x7f624801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8221089084f0bc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:56.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 1 -- 172.21.15.119:0/3545117946 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480a7070 con 0x7f62540a1100 2024-09-15T21:23:56.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: dump: 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:56.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: _finish_auth 0 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: _check_auth_tickets 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 5 monclient: authenticate success, global_id 44828 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 1 -- 172.21.15.119:0/3545117946 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62480b0070 con 0x7f62540a1100 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:56.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 4 set_mon_vals no callback set 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 1 -- 172.21.15.119:0/3545117946 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480ab890 con 0x7f62540a1100 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625159a640 10 monclient: dump: 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.546 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: shutdown 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- 172.21.15.119:0/3545117946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 msgr2=0x7f625413cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 --2- 172.21.15.119:0/3545117946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 0x7f625413cb60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6248009870 tx=0x7f624801c4d0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- 172.21.15.119:0/3545117946 shutdown_connections 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 --2- 172.21.15.119:0/3545117946 >> v2:172.21.15.119:3300/0 conn(0x7f625413d0a0 0x7f625413f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 --2- 172.21.15.119:0/3545117946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a1100 0x7f625413cb60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- 172.21.15.119:0/3545117946 >> 172.21.15.119:0/3545117946 conn(0x7f62540a6ef0 msgr2=0x7f62540a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:56.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- 172.21.15.119:0/3545117946 shutdown_connections 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- 172.21.15.119:0/3545117946 wait complete. 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: build_initial_monmap 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: monmap: 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:56.548 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 Processor -- start 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- start start 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: init 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 0x7f6254130b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): picked mon.noname-a con 0x7f62540a1100 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 0x7f6254131080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): picked mon.noname-c con 0x7f62540a79e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f625413d0a0 0x7f62541315c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): picked mon.noname-b con 0x7f625413d0a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f62541421f0 con 0x7f62540a1100 2024-09-15T21:23:56.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6254134870 con 0x7f625413d0a0 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6254142370 con 0x7f62540a79e0 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6258ca8640 10 monclient(hunting): authenticate will time out at 1355.335051s 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6252d9d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f625413d0a0 msgr2=0x7f62541315c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6252d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f625413d0a0 0x7f62541315c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 0x7f6254130b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 0x7f6254131080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 0x7f6254130b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36316/0 (socket says 172.21.15.119:36316) 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 0x7f6254131080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46458/0 (socket says 172.21.15.119:46458) 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 1 -- 172.21.15.119:0/2782094620 learned_addr learned my addr 172.21.15.119:0/2782094620 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:56.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): get_auth_request con 0x7f62540a1100 auth_method 0 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): get_auth_request con 0x7f62540a79e0 auth_method 0 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f6251d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.534+0000 7f625259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:56.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 10 monclient(hunting): handle_auth_done global_id 44831 payload 824 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 -- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f625413d0a0 msgr2=0x7f62541315c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f625413d0a0 0x7f62541315c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 -- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 msgr2=0x7f6254130b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 0x7f6254130b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 monclient: found mon.noname-c 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 -- 172.21.15.119:0/2782094620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254137e50 con 0x7f62540a79e0 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f625259c640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 0x7f6254130b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6251d9b640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 0x7f6254131080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62540a6af0 tx=0x7f624801ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480be640 con 0x7f62540a79e0 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:56.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: dump: 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: _finish_auth 0 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:23:56.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 5 monclient: authenticate success, global_id 44831 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 monclient: _renew_subs 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6254132440 con 0x7f62540a79e0 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62480be800 con 0x7f62540a79e0 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480bebf0 con 0x7f62540a79e0 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 10 monclient: dump: 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.554 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 monclient: _renew_subs 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6250d99640 4 set_mon_vals no callback set 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6254137b70 con 0x7f62540a79e0 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:56.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6254137970 con 0x7f62540a79e0 2024-09-15T21:23:56.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f62480b4600 con 0x7f62540a79e0 2024-09-15T21:23:56.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f623403b970 0x7f623403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:56.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(175..175 src has 1..175) ==== 7629+0+0 (secure 0 0 0) 0x7f62480eb930 con 0x7f62540a79e0 2024-09-15T21:23:56.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f625259c640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f623403b970 0x7f623403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:56.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f625259c640 10 monclient: get_auth_request con 0x7f623403b970 auth_method 0 2024-09-15T21:23:56.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.538+0000 7f625259c640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f623403b970 0x7f623403de30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6244003fd0 tx=0x7f6244003ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:56.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.550+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6248016150 con 0x7f62540a79e0 2024-09-15T21:23:56.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.550+0000 7f6242ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:56.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.550+0000 7f6242ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:56.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.710+0000 7f6258ca8640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T21:23:56.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.710+0000 7f6258ca8640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T21:23:56.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.710+0000 7f6258ca8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:56.716 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:56.710+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) -- 0x7f62540a4170 con 0x7f62540a79e0 2024-09-15T21:23:57.315 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.310+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 moved item id -16 name 'fooo' to location {root=default} in crush map v176) ==== 173+0+0 (secure 0 0 0) 0x7f62480aacd0 con 0x7f62540a79e0 2024-09-15T21:23:57.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.310+0000 7f6242ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T21:23:57.316 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.310+0000 7f6242ffd640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {root=default} in crush map 2024-09-15T21:23:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.410+0000 7f6258ca8640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T21:23:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.414+0000 7f6258ca8640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T21:23:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.414+0000 7f6258ca8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:57.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.414+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) -- 0x7f62540a4170 con 0x7f62540a79e0 2024-09-15T21:23:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.414+0000 7f6242ffd640 1 -- 172.21.15.119:0/2782094620 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 no need to move item id -16 name 'fooo' to location {root=default} in crush map v176) ==== 183+0+0 (secure 0 0 0) 0x7f62480bddb0 con 0x7f62540a79e0 2024-09-15T21:23:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.414+0000 7f6242ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T21:23:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.414+0000 7f6242ffd640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-09-15T21:23:57.419 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f623403b970 msgr2=0x7f623403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f623403b970 0x7f623403de30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6244003fd0 tx=0x7f6244003ba0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 10 monclient: shutdown 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 msgr2=0x7f6254131080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 0x7f6254131080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62540a6af0 tx=0x7f624801ad10 comp rx=0 tx=0).stop 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 shutdown_connections 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f623403b970 0x7f623403de30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f625413d0a0 0x7f62541315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f62540a79e0 0x7f6254131080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 --2- 172.21.15.119:0/2782094620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62540a1100 0x7f6254130b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 >> 172.21.15.119:0/2782094620 conn(0x7f62540a6ef0 msgr2=0x7f62540a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 shutdown_connections 2024-09-15T21:23:57.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.418+0000 7f6258ca8640 1 -- 172.21.15.119:0/2782094620 wait complete. 2024-09-15T21:23:57.436 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket barr rack 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient: get_monmap_and_config 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient: build_initial_monmap 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient: monmap: 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:57.549373+0000 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:57.549373+0000 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:57.550 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:57.551 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:57.551 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:57.551 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:57.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 Processor -- start 2024-09-15T21:23:57.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 -- start start 2024-09-15T21:23:57.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient: init 2024-09-15T21:23:57.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:57.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2e380a0900 addr v1:172.21.15.119:6789/0 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e380a13f0 0x7f2e3813ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): picked mon.noname-c con 0x7f2e380a13f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e3813b150 0x7f2e3813d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): picked mon.noname-a con 0x7f2e3813b150 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e380a2240 con 0x7f2e380a0900 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3809fdd0 con 0x7f2e3813b150 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2e380a1c80 con 0x7f2e380a13f0 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e3dfd9640 10 monclient(hunting): authenticate will time out at 1356.346591s 2024-09-15T21:23:57.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e377fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f2e380a0900 legacy=0x7f2e380a0ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e36ffd640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f2e380a13f0 msgr2=0x7f2e3813ac10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e36ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e380a13f0 0x7f2e3813ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e3813b150 0x7f2e3813d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e3813b150 0x7f2e3813d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36324/0 (socket says 172.21.15.119:36324) 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 -- 172.21.15.119:0/2648747782 learned_addr learned my addr 172.21.15.119:0/2648747782 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): get_auth_request con 0x7f2e3813b150 auth_method 0 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient(hunting): handle_auth_done global_id 74274 payload 274 2024-09-15T21:23:57.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 -- 172.21.15.119:0/2648747782 >> v2:172.21.15.119:3300/0 conn(0x7f2e380a13f0 msgr2=0x7f2e3813ac10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 --2- 172.21.15.119:0/2648747782 >> v2:172.21.15.119:3300/0 conn(0x7f2e380a13f0 0x7f2e3813ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 -- 172.21.15.119:0/2648747782 >> v1:172.21.15.119:6789/0 conn(0x7f2e380a0900 legacy=0x7f2e380a0ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient: _finish_hunting 0 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 monclient: found mon.noname-a 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:57.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.546+0000 7f2e37fff640 1 -- 172.21.15.119:0/2648747782 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e380a1f60 con 0x7f2e3813b150 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e37fff640 1 --2- 172.21.15.119:0/2648747782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e3813b150 0x7f2e3813d540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e3806e400 tx=0x7f2e3001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fe25a5bec09e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 1 -- 172.21.15.119:0/2648747782 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300a5770 con 0x7f2e3813b150 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: dump: 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.562 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: _finish_auth 0 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: _check_auth_tickets 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 1 -- 172.21.15.119:0/2648747782 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e300a5910 con 0x7f2e3813b150 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 1 -- 172.21.15.119:0/2648747782 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300a5ce0 con 0x7f2e3813b150 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 5 monclient: authenticate success, global_id 74274 2024-09-15T21:23:57.563 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 4 set_mon_vals no callback set 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e367fc640 10 monclient: dump: 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:57.564 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 10 monclient: shutdown 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/2648747782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e3813b150 msgr2=0x7f2e3813d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/2648747782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e3813b150 0x7f2e3813d540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e3806e400 tx=0x7f2e3001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/2648747782 shutdown_connections 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/2648747782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e3813b150 0x7f2e3813d540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/2648747782 >> v2:172.21.15.119:3300/0 conn(0x7f2e380a13f0 0x7f2e3813ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/2648747782 >> 172.21.15.119:0/2648747782 conn(0x7f2e3809be60 msgr2=0x7f2e3809c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/2648747782 shutdown_connections 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/2648747782 wait complete. 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 10 monclient: build_initial_monmap 2024-09-15T21:23:57.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 10 monclient: monmap: 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 Processor -- start 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 1 -- start start 2024-09-15T21:23:57.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.550+0000 7f2e3dfd9640 10 monclient: init 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e380a0900 0x7f2e381cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): picked mon.noname-b con 0x7f2e380a0900 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e380a13f0 0x7f2e38135990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): picked mon.noname-a con 0x7f2e380a13f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 0x7f2e38135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): picked mon.noname-c con 0x7f2e3813b150 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2e380a01d0 con 0x7f2e380a13f0 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e380a0050 con 0x7f2e380a0900 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:57.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2e381402a0 con 0x7f2e3813b150 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient(hunting): authenticate will time out at 1356.352458s 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e377fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e380a0900 msgr2=0x7f2e381cf670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e377fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e380a0900 0x7f2e381cf670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 0x7f2e38135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e380a13f0 0x7f2e38135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 0x7f2e38135ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46460/0 (socket says 172.21.15.119:46460) 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 1 -- 172.21.15.119:0/4122805874 learned_addr learned my addr 172.21.15.119:0/4122805874 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): get_auth_request con 0x7f2e3813b150 auth_method 0 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:57.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): get_auth_request con 0x7f2e380a13f0 auth_method 0 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient(hunting): handle_auth_done global_id 74277 payload 824 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 -- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 msgr2=0x7f2e38135ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 0x7f2e38135ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 -- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e380a0900 msgr2=0x7f2e381cf670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:57.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e380a0900 0x7f2e381cf670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 monclient: found mon.noname-a 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 -- 172.21.15.119:0/4122805874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3809f370 con 0x7f2e380a13f0 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e37fff640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 0x7f2e38135ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e36ffd640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e380a13f0 0x7f2e38135990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b430 tx=0x7f2e2c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c016070 con 0x7f2e380a13f0 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: dump: 2024-09-15T21:23:57.570 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: _finish_auth 0 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:57.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 5 monclient: authenticate success, global_id 74277 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c003df0 con 0x7f2e380a13f0 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _renew_subs 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3813d570 con 0x7f2e380a13f0 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0052b0 con 0x7f2e380a13f0 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e1ffff640 10 monclient: dump: 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:57.572 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e357fa640 4 set_mon_vals no callback set 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _renew_subs 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e38121d80 con 0x7f2e380a13f0 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:57.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.554+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e380a0ce0 con 0x7f2e380a13f0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.558+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2e2c019070 con 0x7f2e380a13f0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.558+0000 7f2e1ffff640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e2403b970 0x7f2e2403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.558+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 7817+0+0 (secure 0 0 0) 0x7f2e2c04fe60 con 0x7f2e380a13f0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.558+0000 7f2e377fe640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e2403b970 0x7f2e2403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.558+0000 7f2e377fe640 10 monclient: get_auth_request con 0x7f2e2403b970 auth_method 0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.558+0000 7f2e377fe640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e2403b970 0x7f2e2403de30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e38120af0 tx=0x7f2e28006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.570+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2e2c020020 con 0x7f2e380a13f0 2024-09-15T21:23:57.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.570+0000 7f2e1ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:57.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.570+0000 7f2e1ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:57.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.734+0000 7f2e3dfd9640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T21:23:57.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.734+0000 7f2e3dfd9640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T21:23:57.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.734+0000 7f2e3dfd9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:57.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:57.734+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) -- 0x7f2e3806a810 con 0x7f2e380a13f0 2024-09-15T21:23:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.310+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 added bucket barr type rack to crush map v177) ==== 140+0+0 (secure 0 0 0) 0x7f2e2c04de20 con 0x7f2e380a13f0 2024-09-15T21:23:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.310+0000 7f2e1ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T21:23:58.314 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.310+0000 7f2e1ffff640 10 monclient: _finish_command 2 = system:0 added bucket barr type rack to crush map 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e3dfd9640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e3dfd9640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e3dfd9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) -- 0x7f2e38122360 con 0x7f2e380a13f0 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e1ffff640 1 -- 172.21.15.119:0/4122805874 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 bucket 'barr' already exists v177) ==== 128+0+0 (secure 0 0 0) 0x7f2e2c01a3b0 con 0x7f2e380a13f0 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e1ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.410+0000 7f2e1ffff640 10 monclient: _finish_command 3 = system:0 bucket 'barr' already exists 2024-09-15T21:23:58.417 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'barr' already exists 2024-09-15T21:23:58.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e2403b970 msgr2=0x7f2e2403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e2403b970 0x7f2e2403de30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e38120af0 tx=0x7f2e28006d90 comp rx=0 tx=0).stop 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 10 monclient: shutdown 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e380a13f0 msgr2=0x7f2e38135990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e380a13f0 0x7f2e38135990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b430 tx=0x7f2e2c00b8f0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 shutdown_connections 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2e2403b970 0x7f2e2403de30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2e3813b150 0x7f2e38135ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e380a13f0 0x7f2e38135990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 --2- 172.21.15.119:0/4122805874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e380a0900 0x7f2e381cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 >> 172.21.15.119:0/4122805874 conn(0x7f2e3809be60 msgr2=0x7f2e380a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:58.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 shutdown_connections 2024-09-15T21:23:58.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.414+0000 7f2e3dfd9640 1 -- 172.21.15.119:0/4122805874 wait complete. 2024-09-15T21:23:58.433 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush move barr root=default 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient: get_monmap_and_config 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient: build_initial_monmap 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient: monmap: 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:58.548866+0000 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:58.548866+0000 2024-09-15T21:23:58.549 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:58.550 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:58.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 Processor -- start 2024-09-15T21:23:58.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 -- start start 2024-09-15T21:23:58.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient: init 2024-09-15T21:23:58.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:58.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c380a79e0 0x7f8c380a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): picked mon.noname-a con 0x7f8c380a79e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8c380a46b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c380a1100 0x7f8c38137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): picked mon.noname-c con 0x7f8c380a1100 addr v2:172.21.15.119:3300/0 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c38135180 con 0x7f8c380a46b0 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8c380a7dc0 con 0x7f8c380a79e0 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8c380a6920 con 0x7f8c380a1100 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3d018640 10 monclient(hunting): authenticate will time out at 1357.345457s 2024-09-15T21:23:58.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3759e640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f8c380a1100 msgr2=0x7f8c38137ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3759e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c380a1100 0x7f8c38137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c380a79e0 0x7f8c380a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c380a79e0 0x7f8c380a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36342/0 (socket says 172.21.15.119:36342) 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c3659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8c380a46b0 0x7f8c381377e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:56236/0 (socket says 172.21.15.119:56236) 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 -- 172.21.15.119:0/270864096 learned_addr learned my addr 172.21.15.119:0/270864096 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): get_auth_request con 0x7f8c380a79e0 auth_method 0 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:58.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_done global_id 74280 payload 274 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 -- 172.21.15.119:0/270864096 >> v2:172.21.15.119:3300/0 conn(0x7f8c380a1100 msgr2=0x7f8c38137ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 --2- 172.21.15.119:0/270864096 >> v2:172.21.15.119:3300/0 conn(0x7f8c380a1100 0x7f8c38137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 -- 172.21.15.119:0/270864096 >> v1:172.21.15.17:6789/0 conn(0x7f8c380a46b0 legacy=0x7f8c381377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 monclient: found mon.noname-a 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c35d9b640 1 -- 172.21.15.119:0/270864096 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598784970 0 0) 0x7f8c38135180 con 0x7f8c380a46b0 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c36d9d640 1 -- 172.21.15.119:0/270864096 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c380a1c30 con 0x7f8c380a79e0 2024-09-15T21:23:58.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.546+0000 7f8c35d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:23:58.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 1 --2- 172.21.15.119:0/270864096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c380a4170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c38076a00 tx=0x7f8c3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d910865733890af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:58.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 1 -- 172.21.15.119:0/270864096 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38134930 con 0x7f8c380a79e0 2024-09-15T21:23:58.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:58.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:58.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: dump: 2024-09-15T21:23:58.557 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: _finish_auth 0 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:23:58.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 1 -- 172.21.15.119:0/270864096 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c300a58e0 con 0x7f8c380a79e0 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 1 -- 172.21.15.119:0/270864096 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300a5cb0 con 0x7f8c380a79e0 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c35d9b640 10 monclient: dump: 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:58.559 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:58.560 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:58.560 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:58.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 5 monclient: authenticate success, global_id 74280 2024-09-15T21:23:58.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:58.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 4 set_mon_vals no callback set 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: shutdown 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- 172.21.15.119:0/270864096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 msgr2=0x7f8c380a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 --2- 172.21.15.119:0/270864096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c380a4170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c38076a00 tx=0x7f8c3001c9f0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- 172.21.15.119:0/270864096 shutdown_connections 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 --2- 172.21.15.119:0/270864096 >> v2:172.21.15.119:3300/0 conn(0x7f8c380a1100 0x7f8c38137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.565 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 --2- 172.21.15.119:0/270864096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c380a4170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- 172.21.15.119:0/270864096 >> 172.21.15.119:0/270864096 conn(0x7f8c380a6ef0 msgr2=0x7f8c380a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- 172.21.15.119:0/270864096 shutdown_connections 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- 172.21.15.119:0/270864096 wait complete. 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: build_initial_monmap 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: monmap: 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:58.566 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 Processor -- start 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- start start 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: init 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 0x7f8c3812de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): picked mon.noname-c con 0x7f8c380a1100 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:58.567 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c380a46b0 0x7f8c3812e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): picked mon.noname-b con 0x7f8c380a46b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c3806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): picked mon.noname-a con 0x7f8c380a79e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381385b0 con 0x7f8c380a79e0 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38138430 con 0x7f8c380a46b0 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8c381423a0 con 0x7f8c380a1100 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3d018640 10 monclient(hunting): authenticate will time out at 1357.355385s 2024-09-15T21:23:58.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3659c640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c380a46b0 msgr2=0x7f8c3812e3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c380a46b0 0x7f8c3812e3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 0x7f8c3812de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c3806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 0x7f8c3812de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46466/0 (socket says 172.21.15.119:46466) 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 1 -- 172.21.15.119:0/2089701721 learned_addr learned my addr 172.21.15.119:0/2089701721 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 10 monclient(hunting): get_auth_request con 0x7f8c380a1100 auth_method 0 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c3806f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36352/0 (socket says 172.21.15.119:36352) 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:58.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c36d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3759e640 10 monclient(hunting): get_auth_request con 0x7f8c380a79e0 auth_method 0 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.554+0000 7f8c3759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 10 monclient(hunting): handle_auth_done global_id 44837 payload 824 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 -- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c380a46b0 msgr2=0x7f8c3812e3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:58.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c380a46b0 0x7f8c3812e3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 -- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 msgr2=0x7f8c3806f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c3806f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 monclient: found mon.noname-c 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 -- 172.21.15.119:0/2089701721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c38073030 con 0x7f8c380a1100 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c36d9d640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 0x7f8c3812de60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c300af040 tx=0x7f8c30019790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300bc020 con 0x7f8c380a1100 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: dump: 2024-09-15T21:23:58.571 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:58.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 5 monclient: authenticate success, global_id 44837 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 monclient: _renew_subs 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c380a15e0 con 0x7f8c380a1100 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c300b88a0 con 0x7f8c380a1100 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300b8c90 con 0x7f8c380a1100 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 10 monclient: dump: 2024-09-15T21:23:58.573 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3559a640 4 set_mon_vals no callback set 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 monclient: _renew_subs 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3812e8e0 con 0x7f8c380a1100 2024-09-15T21:23:58.574 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c380719c0 con 0x7f8c380a1100 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8c300bf020 con 0x7f8c380a1100 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8c1803b970 0x7f8c1803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(177..177 src has 1..177) ==== 7845+0+0 (secure 0 0 0) 0x7f8c300eb930 con 0x7f8c380a1100 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3659c640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8c1803b970 0x7f8c1803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3659c640 10 monclient: get_auth_request con 0x7f8c1803b970 auth_method 0 2024-09-15T21:23:58.575 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.558+0000 7f8c3659c640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8c1803b970 0x7f8c1803de30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007fd0 tx=0x7f8c28007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:58.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.570+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8c300c7020 con 0x7f8c380a1100 2024-09-15T21:23:58.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.570+0000 7f8c2f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:58.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.570+0000 7f8c2f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:58.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.738+0000 7f8c3d018640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T21:23:58.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.738+0000 7f8c3d018640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T21:23:58.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.738+0000 7f8c3d018640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:58.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:58.738+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) -- 0x7f8c380a40f0 con 0x7f8c380a1100 2024-09-15T21:23:59.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.334+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 moved item id -19 name 'barr' to location {root=default} in crush map v178) ==== 173+0+0 (secure 0 0 0) 0x7f8c300165e0 con 0x7f8c380a1100 2024-09-15T21:23:59.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.334+0000 7f8c2f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T21:23:59.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.334+0000 7f8c2f7fe640 10 monclient: _finish_command 2 = system:0 moved item id -19 name 'barr' to location {root=default} in crush map 2024-09-15T21:23:59.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.442+0000 7f8c3d018640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T21:23:59.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.442+0000 7f8c3d018640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T21:23:59.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.442+0000 7f8c3d018640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:23:59.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.442+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) -- 0x7f8c380008e0 con 0x7f8c380a1100 2024-09-15T21:23:59.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c2f7fe640 1 -- 172.21.15.119:0/2089701721 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 no need to move item id -19 name 'barr' to location {root=default} in crush map v178) ==== 183+0+0 (secure 0 0 0) 0x7f8c300b73d0 con 0x7f8c380a1100 2024-09-15T21:23:59.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c2f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T21:23:59.449 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c2f7fe640 10 monclient: _finish_command 3 = system:0 no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-09-15T21:23:59.449 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8c1803b970 msgr2=0x7f8c1803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8c1803b970 0x7f8c1803de30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007fd0 tx=0x7f8c28007960 comp rx=0 tx=0).stop 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 10 monclient: shutdown 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 msgr2=0x7f8c3812de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 0x7f8c3812de60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c300af040 tx=0x7f8c30019790 comp rx=0 tx=0).stop 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 shutdown_connections 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8c1803b970 0x7f8c1803de30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c380a79e0 0x7f8c3806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c380a46b0 0x7f8c3812e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 --2- 172.21.15.119:0/2089701721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8c380a1100 0x7f8c3812de60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.452 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 >> 172.21.15.119:0/2089701721 conn(0x7f8c380a6ef0 msgr2=0x7f8c38130d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:59.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 shutdown_connections 2024-09-15T21:23:59.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.446+0000 7f8c3d018640 1 -- 172.21.15.119:0/2089701721 wait complete. 2024-09-15T21:23:59.466 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush move fooo rack=barr 2024-09-15T21:23:59.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.578+0000 7f0c8ac3b640 10 monclient: get_monmap_and_config 2024-09-15T21:23:59.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.578+0000 7f0c8ac3b640 10 monclient: build_initial_monmap 2024-09-15T21:23:59.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.578+0000 7f0c8ac3b640 10 monclient: monmap: 2024-09-15T21:23:59.584 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:59.584 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:59.584 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:23:59.583954+0000 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:23:59.583954+0000 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:23:59.585 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.578+0000 7f0c8ac3b640 1 Processor -- start 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.578+0000 7f0c8ac3b640 1 -- start start 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.578+0000 7f0c8ac3b640 10 monclient: init 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c8409d400 0x7f0c8409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): picked mon.noname-a con 0x7f0c8409d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c8409dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 0x7f0c8413c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:59.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): picked mon.noname-b con 0x7f0c84134de0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8406e240 con 0x7f0c8409dd20 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0c8406e000 con 0x7f0c8409d400 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8409e7f0 con 0x7f0c84134de0 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c8ac3b640 10 monclient(hunting): authenticate will time out at 1358.380662s 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 0x7f0c8413c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c8409d400 0x7f0c8409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 0x7f0c8413c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46474/0 (socket says 172.21.15.119:46474) 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 1 -- 172.21.15.119:0/1870106670 learned_addr learned my addr 172.21.15.119:0/1870106670 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): get_auth_request con 0x7f0c84134de0 auth_method 0 2024-09-15T21:23:59.589 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): get_auth_request con 0x7f0c8409d400 auth_method 0 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient(hunting): handle_auth_done global_id 74289 payload 274 2024-09-15T21:23:59.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 1 -- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 msgr2=0x7f0c8413c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 1 --2- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 0x7f0c8413c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 1 -- 172.21.15.119:0/1870106670 >> v1:172.21.15.17:6789/0 conn(0x7f0c8409dd20 legacy=0x7f0c841346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient: _finish_hunting 0 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 1 monclient: found mon.noname-a 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c889d7640 1 -- 172.21.15.119:0/1870106670 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8409e170 con 0x7f0c8409d400 2024-09-15T21:23:59.591 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.582+0000 7f0c891d8640 1 --2- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 0x7f0c8413c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:23:59.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 1 --2- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c8409d400 0x7f0c8409d7e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0c780097f0 tx=0x7f0c7801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9a6b49498b0555 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:59.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 1 -- 172.21.15.119:0/1870106670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780b0070 con 0x7f0c8409d400 2024-09-15T21:23:59.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:59.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:59.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: dump: 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: _finish_auth 0 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: _check_auth_tickets 2024-09-15T21:23:59.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 1 -- 172.21.15.119:0/1870106670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c780ab600 con 0x7f0c8409d400 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 5 monclient: authenticate success, global_id 74289 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: get_monmap_and_config success 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 4 set_mon_vals no callback set 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 1 -- 172.21.15.119:0/1870106670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780ab9d0 con 0x7f0c8409d400 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c837fe640 10 monclient: dump: 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.598 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: shutdown 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c8409d400 msgr2=0x7f0c8409d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c8409d400 0x7f0c8409d7e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0c780097f0 tx=0x7f0c7801c360 comp rx=0 tx=0).stop 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/1870106670 shutdown_connections 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c84134de0 0x7f0c8413c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.599 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/1870106670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c8409d400 0x7f0c8409d7e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/1870106670 >> 172.21.15.119:0/1870106670 conn(0x7f0c8409be60 msgr2=0x7f0c8409c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/1870106670 shutdown_connections 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/1870106670 wait complete. 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: build_initial_monmap 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: monmap: 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:23:59.600 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 Processor -- start 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- start start 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: init 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: _reopen_session rank -1 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c8409d400 0x7f0c841cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): picked mon.noname-c con 0x7f0c8409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c8409dd20 0x7f0c8412e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): picked mon.noname-b con 0x7f0c8409dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c84134de0 0x7f0c8413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): picked mon.noname-a con 0x7f0c84134de0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:23:59.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840a0360 con 0x7f0c84134de0 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840a01e0 con 0x7f0c8409dd20 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8413f700 con 0x7f0c8409d400 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): _renew_subs 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c8ac3b640 10 monclient(hunting): authenticate will time out at 1358.387265s 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c83fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c8409dd20 msgr2=0x7f0c8412e9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c83fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c8409dd20 0x7f0c8412e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c8409d400 0x7f0c841cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c84134de0 0x7f0c8413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c8409d400 0x7f0c841cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46480/0 (socket says 172.21.15.119:46480) 2024-09-15T21:23:59.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 1 -- 172.21.15.119:0/2938568228 learned_addr learned my addr 172.21.15.119:0/2938568228 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): get_auth_request con 0x7f0c8409d400 auth_method 0 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): get_auth_request con 0x7f0c84134de0 auth_method 0 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c891d8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:23:59.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.586+0000 7f0c889d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 10 monclient(hunting): handle_auth_done global_id 74295 payload 824 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 -- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c8409d400 msgr2=0x7f0c841cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c8409d400 0x7f0c841cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 -- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c8409dd20 msgr2=0x7f0c8412e9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c8409dd20 0x7f0c8412e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 10 monclient: _finish_hunting 0 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 monclient: found mon.noname-a 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 -- 172.21.15.119:0/2938568228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c84070670 con 0x7f0c84134de0 2024-09-15T21:23:59.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c891d8640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c84134de0 0x7f0c8413aa90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00efc0 tx=0x7f0c7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00cb20 con 0x7f0c84134de0 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: dump: 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:59.605 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: _finish_auth 0 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: _check_auth_tickets 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00cce0 con 0x7f0c84134de0 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c003be0 con 0x7f0c84134de0 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 5 monclient: authenticate success, global_id 74295 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 monclient: _renew_subs 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c84070950 con 0x7f0c84134de0 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 10 monclient: dump: 2024-09-15T21:23:59.606 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c82ffd640 4 set_mon_vals no callback set 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 monclient: _renew_subs 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:59.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8409d0f0 con 0x7f0c84134de0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8409d7e0 con 0x7f0c84134de0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0c7c01b030 con 0x7f0c84134de0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0c6803b970 0x7f0c6803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 8033+0+0 (secure 0 0 0) 0x7f0c7c051a30 con 0x7f0c84134de0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c889d7640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0c6803b970 0x7f0c6803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c889d7640 10 monclient: get_auth_request con 0x7f0c6803b970 auth_method 0 2024-09-15T21:23:59.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.590+0000 7f0c889d7640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0c6803b970 0x7f0c6803de30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0c780097f0 tx=0x7f0c78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:23:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.602+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0c7c022020 con 0x7f0c84134de0 2024-09-15T21:23:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.602+0000 7f0c817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:23:59.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.602+0000 7f0c817fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:23:59.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.774+0000 7f0c8ac3b640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T21:23:59.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.774+0000 7f0c8ac3b640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T21:23:59.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.774+0000 7f0c8ac3b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:23:59.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:23:59.774+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) -- 0x7f0c8406f1b0 con 0x7f0c84134de0 2024-09-15T21:24:00.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.338+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 moved item id -16 name 'fooo' to location {rack=barr} in crush map v179) ==== 167+0+0 (secure 0 0 0) 0x7f0c7c010040 con 0x7f0c84134de0 2024-09-15T21:24:00.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.338+0000 7f0c817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T21:24:00.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.338+0000 7f0c817fa640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {rack=barr} in crush map 2024-09-15T21:24:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c8ac3b640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T21:24:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c8ac3b640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T21:24:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c8ac3b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) -- 0x7f0c840008e0 con 0x7f0c84134de0 2024-09-15T21:24:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c817fa640 1 -- 172.21.15.119:0/2938568228 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map v179) ==== 177+0+0 (secure 0 0 0) 0x7f0c7c014e20 con 0x7f0c84134de0 2024-09-15T21:24:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c817fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T21:24:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.438+0000 7f0c817fa640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-09-15T21:24:00.443 INFO:tasks.workunit.client.0.smithi119.stderr:no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-09-15T21:24:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0c6803b970 msgr2=0x7f0c6803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0c6803b970 0x7f0c6803de30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0c780097f0 tx=0x7f0c78008040 comp rx=0 tx=0).stop 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 10 monclient: shutdown 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c84134de0 msgr2=0x7f0c8413aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c84134de0 0x7f0c8413aa90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00efc0 tx=0x7f0c7c00c490 comp rx=0 tx=0).stop 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 shutdown_connections 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0c6803b970 0x7f0c6803de30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c84134de0 0x7f0c8413aa90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c8409dd20 0x7f0c8412e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 --2- 172.21.15.119:0/2938568228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0c8409d400 0x7f0c841cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 >> 172.21.15.119:0/2938568228 conn(0x7f0c8409be60 msgr2=0x7f0c841319b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 shutdown_connections 2024-09-15T21:24:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.442+0000 7f0c8ac3b640 1 -- 172.21.15.119:0/2938568228 wait complete. 2024-09-15T21:24:00.460 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm fooo 2024-09-15T21:24:00.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient: get_monmap_and_config 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient: build_initial_monmap 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient: monmap: 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:00.578165+0000 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:00.578165+0000 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:00.612 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 1 Processor -- start 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 1 -- start start 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient: init 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff85c135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff85c0a0160 0x7ff85c13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient(hunting): picked mon.noname-c con 0x7ff85c0a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff85c06ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85c09cf30 con 0x7ff85c135ca0 2024-09-15T21:24:00.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85c0a1ca0 con 0x7ff85c06ec30 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c0a1f80 con 0x7ff85c0a0160 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff862276640 10 monclient(hunting): authenticate will time out at 1359.375346s 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff860813640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7ff85c06ec30 legacy=0x7ff85c13cf50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff85b7fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7ff85c0a0160 msgr2=0x7ff85c13aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff85b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff85c0a0160 0x7ff85c13aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff85bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff85c135ca0 0x7ff85c136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:56252/0 (socket says 172.21.15.119:56252) 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.574+0000 7ff85bfff640 1 -- 172.21.15.119:0/2660774184 learned_addr learned my addr 172.21.15.119:0/2660774184 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683461746 0 0) 0x7ff85c09cf30 con 0x7ff85c135ca0 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): _negotiate 2024-09-15T21:24:00.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): my global_id is 74298 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff848003530 con 0x7ff85c135ca0 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1797196977 0 0) 0x7ff848003530 con 0x7ff85c135ca0 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 >> v2:172.21.15.119:3300/0 conn(0x7ff85c0a0160 msgr2=0x7ff85c13aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 --2- 172.21.15.119:0/2660774184 >> v2:172.21.15.119:3300/0 conn(0x7ff85c0a0160 0x7ff85c13aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 >> v1:172.21.15.119:6789/0 conn(0x7ff85c06ec30 legacy=0x7ff85c13cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c0a3860 con 0x7ff85c135ca0 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: _finish_auth 0 2024-09-15T21:24:00.615 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff844001700 con 0x7ff85c135ca0 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: dump: 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:00.616 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 >> v1:172.21.15.17:6789/0 conn(0x7ff85c135ca0 legacy=0x7ff85c136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8480039f0 0x7ff848003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): picked mon.c con 0x7ff8480039f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff848004810 0x7ff848004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): picked mon.b con 0x7ff848004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff84800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): picked mon.a con 0x7ff848005400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:00.617 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c0a1f80 con 0x7ff848005400 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff84800c350 con 0x7ff848004810 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff84800c550 con 0x7ff8480039f0 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85affd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:00.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85b7fe640 1 -- 172.21.15.119:0/2660774184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff848004810 msgr2=0x7ff848004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85b7fe640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff848004810 0x7ff848004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff84800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): get_auth_request con 0x7ff848005400 auth_method 0 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8480039f0 0x7ff848003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): get_auth_request con 0x7ff8480039f0 auth_method 0 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:00.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff85bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient(hunting): handle_auth_done global_id 74298 payload 293 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 -- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8480039f0 msgr2=0x7ff848003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8480039f0 0x7ff848003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 -- 172.21.15.119:0/2660774184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff848004810 msgr2=0x7ff848004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff848004810 0x7ff848004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient: _finish_hunting 0 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 monclient: found mon.a 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 -- 172.21.15.119:0/2660774184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff848003530 con 0x7ff848005400 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient: _finish_auth 0 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 10 monclient: _check_auth_tickets 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff862276640 5 monclient: authenticate success, global_id 74298 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff862276640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 -- 172.21.15.119:0/2660774184 reap_dead start 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.578+0000 7ff860813640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff84800ba70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff854009900 tx=0x7ff85401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c4de8176c59d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854009500 con 0x7ff848005400 2024-09-15T21:24:00.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: dump: 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff854003b90 con 0x7ff848005400 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 4 set_mon_vals no callback set 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 1 -- 172.21.15.119:0/2660774184 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854005070 con 0x7ff848005400 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85affd640 10 monclient: dump: 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:00.623 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: shutdown 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 msgr2=0x7ff84800ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff84800ba70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff854009900 tx=0x7ff85401c400 comp rx=0 tx=0).stop 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- 172.21.15.119:0/2660774184 shutdown_connections 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 --2- 172.21.15.119:0/2660774184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff84800ba70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- 172.21.15.119:0/2660774184 >> 172.21.15.119:0/2660774184 conn(0x7ff85c0a2ae0 msgr2=0x7ff85c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- 172.21.15.119:0/2660774184 shutdown_connections 2024-09-15T21:24:00.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- 172.21.15.119:0/2660774184 wait complete. 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: build_initial_monmap 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: monmap: 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 Processor -- start 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- start start 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: init 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff85c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): picked mon.noname-a con 0x7ff848005400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff85c13b0a0 0x7ff85c133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): picked mon.noname-c con 0x7ff85c13b0a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c1cccf0 0x7ff85c1cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): picked mon.noname-b con 0x7ff85c1cccf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c136200 con 0x7ff848005400 2024-09-15T21:24:00.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c136080 con 0x7ff85c1cccf0 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c136380 con 0x7ff85c13b0a0 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff862276640 10 monclient(hunting): authenticate will time out at 1359.383257s 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff860813640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c1cccf0 msgr2=0x7ff85c1cd150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff860813640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c1cccf0 0x7ff85c1cd150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff85c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff85c13b0a0 0x7ff85c133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff85c13d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36390/0 (socket says 172.21.15.119:36390) 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 1 -- 172.21.15.119:0/23777183 learned_addr learned my addr 172.21.15.119:0/23777183 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): get_auth_request con 0x7ff848005400 auth_method 0 2024-09-15T21:24:00.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): get_auth_request con 0x7ff85c13b0a0 auth_method 0 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.582+0000 7ff85b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 10 monclient(hunting): handle_auth_done global_id 74301 payload 824 2024-09-15T21:24:00.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 -- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff85c13b0a0 msgr2=0x7ff85c133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff85c13b0a0 0x7ff85c133e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 -- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c1cccf0 msgr2=0x7ff85c1cd150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c1cccf0 0x7ff85c1cd150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 monclient: found mon.noname-a 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 -- 172.21.15.119:0/23777183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c13c680 con 0x7ff848005400 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85bfff640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff85c13d1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff844004ee0 tx=0x7ff844007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84400f720 con 0x7ff848005400 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:00.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: dump: 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:00.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff844018070 con 0x7ff848005400 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84400fb20 con 0x7ff848005400 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 5 monclient: authenticate success, global_id 74301 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 monclient: _renew_subs 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c09d1d0 con 0x7ff848005400 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff859ffb640 4 set_mon_vals no callback set 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 10 monclient: dump: 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:00.631 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 monclient: _renew_subs 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c13bfb0 con 0x7ff848005400 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:00.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c13b480 con 0x7ff848005400 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff84400fcc0 con 0x7ff848005400 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff83c03b970 0x7ff83c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 8033+0+0 (secure 0 0 0) 0x7ff844050e10 con 0x7ff848005400 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85b7fe640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff83c03b970 0x7ff83c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85b7fe640 10 monclient: get_auth_request con 0x7ff83c03b970 auth_method 0 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.586+0000 7ff85b7fe640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff83c03b970 0x7ff83c03de30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff85c11ce40 tx=0x7ff84c00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.598+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff85c13b480 con 0x7ff848005400 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.598+0000 7ff858ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:00.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.598+0000 7ff858ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:00.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.758+0000 7ff862276640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T21:24:00.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.758+0000 7ff862276640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T21:24:00.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.758+0000 7ff862276640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:00.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:00.758+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) -- 0x7ff85c0a09e0 con 0x7ff848005400 2024-09-15T21:24:01.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.346+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 removed item id -16 name 'fooo' from crush map v180) ==== 122+0+0 (secure 0 0 0) 0x7ff844026dc0 con 0x7ff848005400 2024-09-15T21:24:01.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.346+0000 7ff858ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T21:24:01.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.346+0000 7ff858ff9640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'fooo' from crush map 2024-09-15T21:24:01.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.446+0000 7ff862276640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T21:24:01.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T21:24:01.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:01.453 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) -- 0x7ff85c09f8f0 con 0x7ff848005400 2024-09-15T21:24:01.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff858ff9640 1 -- 172.21.15.119:0/23777183 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 device 'fooo' does not appear in the crush map v180) ==== 122+0+0 (secure 0 0 0) 0x7ff84401aa40 con 0x7ff848005400 2024-09-15T21:24:01.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff858ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T21:24:01.454 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff858ff9640 10 monclient: _finish_command 3 = system:0 device 'fooo' does not appear in the crush map 2024-09-15T21:24:01.454 INFO:tasks.workunit.client.0.smithi119.stderr:device 'fooo' does not appear in the crush map 2024-09-15T21:24:01.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff83c03b970 msgr2=0x7ff83c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:01.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff83c03b970 0x7ff83c03de30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff85c11ce40 tx=0x7ff84c00d040 comp rx=0 tx=0).stop 2024-09-15T21:24:01.456 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 10 monclient: shutdown 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 msgr2=0x7ff85c13d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff85c13d1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff844004ee0 tx=0x7ff844007430 comp rx=0 tx=0).stop 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 shutdown_connections 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff83c03b970 0x7ff83c03de30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c1cccf0 0x7ff85c1cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff85c13b0a0 0x7ff85c133e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 --2- 172.21.15.119:0/23777183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848005400 0x7ff85c13d1e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 >> 172.21.15.119:0/23777183 conn(0x7ff85c0a2ae0 msgr2=0x7ff85c0a4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 shutdown_connections 2024-09-15T21:24:01.457 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.450+0000 7ff862276640 1 -- 172.21.15.119:0/23777183 wait complete. 2024-09-15T21:24:01.471 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm barr 2024-09-15T21:24:01.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient: get_monmap_and_config 2024-09-15T21:24:01.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient: build_initial_monmap 2024-09-15T21:24:01.590 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient: monmap: 2024-09-15T21:24:01.590 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:01.590176+0000 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:01.590176+0000 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:01.591 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 Processor -- start 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 -- start start 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient: init 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24400a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): picked mon.noname-b con 0x7f24400a3600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24400a1270 0x7f2440133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): picked mon.noname-a con 0x7f24400a1270 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f24401341a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400a1700 con 0x7f24401341a0 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:01.594 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f24400a39e0 con 0x7f24400a1270 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24400a27a0 con 0x7f24400a3600 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2447062640 10 monclient(hunting): authenticate will time out at 1360.387409s 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f24455ff640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f24401341a0 legacy=0x7f244013aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24400a1270 0x7f2440133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2444dfe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24400a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24400a1270 0x7f2440133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36396/0 (socket says 172.21.15.119:36396) 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 1 -- 172.21.15.119:0/1586496281 learned_addr learned my addr 172.21.15.119:0/1586496281 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 10 monclient(hunting): get_auth_request con 0x7f24400a1270 auth_method 0 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:01.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f243ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2444dfe640 10 monclient(hunting): get_auth_request con 0x7f24400a3600 auth_method 0 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2444dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2444dfe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.586+0000 7f2444dfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2444dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2444dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2444dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 10 monclient(hunting): handle_auth_done global_id 74304 payload 274 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 1 -- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 msgr2=0x7f24400a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:01.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 1 --2- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24400a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 1 -- 172.21.15.119:0/1586496281 >> v1:172.21.15.119:6789/0 conn(0x7f24401341a0 legacy=0x7f244013aff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:01.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:01.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 1 monclient: found mon.noname-a 2024-09-15T21:24:01.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:01.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:01.597 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 1 -- 172.21.15.119:0/1586496281 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244009df80 con 0x7f24400a1270 2024-09-15T21:24:01.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243ffff640 1 --2- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24400a1270 0x7f2440133c60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2434009870 tx=0x7f243401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e602ca84ea745784 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:01.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 1 -- 172.21.15.119:0/1586496281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340b0070 con 0x7f24400a1270 2024-09-15T21:24:01.600 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: dump: 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:01.601 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2447062640 5 monclient: authenticate success, global_id 74304 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 1 -- 172.21.15.119:0/1586496281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24340ab600 con 0x7f24400a1270 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2447062640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2447062640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f2447062640 4 set_mon_vals no callback set 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 1 -- 172.21.15.119:0/1586496281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340ab9d0 con 0x7f24400a1270 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.590+0000 7f243f7fe640 10 monclient: dump: 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:01.602 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient: shutdown 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24400a1270 msgr2=0x7f2440133c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 --2- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24400a1270 0x7f2440133c60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2434009870 tx=0x7f243401c4b0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- 172.21.15.119:0/1586496281 shutdown_connections 2024-09-15T21:24:01.603 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 --2- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24400a1270 0x7f2440133c60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 --2- 172.21.15.119:0/1586496281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24400a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- 172.21.15.119:0/1586496281 >> 172.21.15.119:0/1586496281 conn(0x7f24400a2ae0 msgr2=0x7f24400a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- 172.21.15.119:0/1586496281 shutdown_connections 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- 172.21.15.119:0/1586496281 wait complete. 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient: build_initial_monmap 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient: monmap: 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:01.604 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 Processor -- start 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- start start 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient: init 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24401ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): picked mon.noname-c con 0x7f24400a3600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 0x7f244013b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): picked mon.noname-a con 0x7f24401341a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f244013b6c0 0x7f2440070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:01.605 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): picked mon.noname-b con 0x7f244013b6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f24400a2070 con 0x7f24401341a0 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2440134a90 con 0x7f244013b6c0 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24400a21f0 con 0x7f24400a3600 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2447062640 10 monclient(hunting): authenticate will time out at 1360.394225s 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f24455ff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f244013b6c0 msgr2=0x7f2440070050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f24455ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f244013b6c0 0x7f2440070050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24401ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 0x7f244013b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24401ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:46516/0 (socket says 172.21.15.119:46516) 2024-09-15T21:24:01.606 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 -- 172.21.15.119:0/2002766705 learned_addr learned my addr 172.21.15.119:0/2002766705 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 0x7f244013b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:36408/0 (socket says 172.21.15.119:36408) 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): get_auth_request con 0x7f24400a3600 auth_method 0 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): get_auth_request con 0x7f24401341a0 auth_method 0 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:01.607 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient(hunting): handle_auth_done global_id 44852 payload 824 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 -- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f244013b6c0 msgr2=0x7f2440070050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f244013b6c0 0x7f2440070050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 -- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 msgr2=0x7f244013b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 0x7f244013b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 monclient: found mon.noname-c 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 -- 172.21.15.119:0/2002766705 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2440071ac0 con 0x7f24400a3600 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243ffff640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 0x7f244013b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:01.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f2444dfe640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24401ccab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f243000e870 tx=0x7f243000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:01.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.594+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430017070 con 0x7f24400a3600 2024-09-15T21:24:01.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:01.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:01.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: dump: 2024-09-15T21:24:01.610 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:01.610 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 5 monclient: authenticate success, global_id 44852 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 monclient: _renew_subs 2024-09-15T21:24:01.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24401ccff0 con 0x7f24400a3600 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243000c940 con 0x7f24400a3600 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243000ccf0 con 0x7f24400a3600 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 10 monclient: dump: 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:01.612 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 monclient: _renew_subs 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2440073580 con 0x7f24400a3600 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243effd640 4 set_mon_vals no callback set 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2440133c60 con 0x7f24400a3600 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f243001a070 con 0x7f24400a3600 2024-09-15T21:24:01.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f242c03b970 0x7f242c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:01.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(180..180 src has 1..180) ==== 7817+0+0 (secure 0 0 0) 0x7f2430050eb0 con 0x7f24400a3600 2024-09-15T21:24:01.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243ffff640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f242c03b970 0x7f242c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:01.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243ffff640 10 monclient: get_auth_request con 0x7f242c03b970 auth_method 0 2024-09-15T21:24:01.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.598+0000 7f243ffff640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f242c03b970 0x7f242c03de30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24340ad000 tx=0x7f2434008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:01.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.610+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f243004dce0 con 0x7f24400a3600 2024-09-15T21:24:01.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.610+0000 7f243d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:01.616 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.610+0000 7f243d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:01.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.786+0000 7f2447062640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T21:24:01.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.786+0000 7f2447062640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T21:24:01.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.786+0000 7f2447062640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:01.791 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:01.786+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) -- 0x7f24400a2480 con 0x7f24400a3600 2024-09-15T21:24:02.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.242+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 removed item id -19 name 'barr' from crush map v181) ==== 122+0+0 (secure 0 0 0) 0x7f243004e060 con 0x7f24400a3600 2024-09-15T21:24:02.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.242+0000 7f243d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T21:24:02.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.242+0000 7f243d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'barr' from crush map 2024-09-15T21:24:02.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.350+0000 7f2447062640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T21:24:02.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.350+0000 7f2447062640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T21:24:02.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.350+0000 7f2447062640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:02.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.350+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) -- 0x7f24400008e0 con 0x7f24400a3600 2024-09-15T21:24:02.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f243d7fa640 1 -- 172.21.15.119:0/2002766705 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 device 'barr' does not appear in the crush map v181) ==== 122+0+0 (secure 0 0 0) 0x7f243001a8f0 con 0x7f24400a3600 2024-09-15T21:24:02.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f243d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T21:24:02.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f243d7fa640 10 monclient: _finish_command 3 = system:0 device 'barr' does not appear in the crush map 2024-09-15T21:24:02.358 INFO:tasks.workunit.client.0.smithi119.stderr:device 'barr' does not appear in the crush map 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f242c03b970 msgr2=0x7f242c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f242c03b970 0x7f242c03de30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24340ad000 tx=0x7f2434008040 comp rx=0 tx=0).stop 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 10 monclient: shutdown 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 msgr2=0x7f24401ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24401ccab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f243000e870 tx=0x7f243000ec70 comp rx=0 tx=0).stop 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 shutdown_connections 2024-09-15T21:24:02.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f242c03b970 0x7f242c03de30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f244013b6c0 0x7f2440070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24401341a0 0x7f244013b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 --2- 172.21.15.119:0/2002766705 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24400a3600 0x7f24401ccab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.354+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 >> 172.21.15.119:0/2002766705 conn(0x7f24400a2ae0 msgr2=0x7f244009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:02.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.358+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 shutdown_connections 2024-09-15T21:24:02.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.358+0000 7f2447062640 1 -- 172.21.15.119:0/2002766705 wait complete. 2024-09-15T21:24:02.376 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T21:24:02.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient: get_monmap_and_config 2024-09-15T21:24:02.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient: build_initial_monmap 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient: monmap: 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:02.494262+0000 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:02.494262+0000 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:02.495 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:02.496 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 1 Processor -- start 2024-09-15T21:24:02.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 1 -- start start 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient: init 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bd4135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd413aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient(hunting): picked mon.noname-b con 0x7f0bd40a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0bd406ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd409cf30 con 0x7f0bd4135ca0 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd40a1ca0 con 0x7f0bd406ec30 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd40a1f80 con 0x7f0bd40a0160 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:02.498 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bdadc6640 10 monclient(hunting): authenticate will time out at 1361.290920s 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd9363640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f0bd406ec30 legacy=0x7f0bd413cf50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd413aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd8b62640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0bd4135ca0 0x7f0bd4136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:57200/0 (socket says 172.21.15.119:57200) 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd8b62640 1 -- 172.21.15.119:0/1200719696 learned_addr learned my addr 172.21.15.119:0/1200719696 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): get_auth_request con 0x7f0bd40a0160 auth_method 0 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd37fe640 1 -- 172.21.15.119:0/1200719696 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3764902926 0 0) 0x7f0bd409cf30 con 0x7f0bd4135ca0 2024-09-15T21:24:02.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd37fe640 10 monclient(hunting): my global_id is 74310 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd37fe640 1 -- 172.21.15.119:0/1200719696 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bbc003530 con 0x7f0bd4135ca0 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_done global_id 44855 payload 274 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 1 -- 172.21.15.119:0/1200719696 >> v1:172.21.15.119:6789/0 conn(0x7f0bd406ec30 legacy=0x7f0bd413cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 1 -- 172.21.15.119:0/1200719696 >> v1:172.21.15.17:6789/0 conn(0x7f0bd4135ca0 legacy=0x7f0bd4136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 1 monclient: found mon.noname-b 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:02.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.490+0000 7f0bd3fff640 1 -- 172.21.15.119:0/1200719696 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd40a3860 con 0x7f0bd40a0160 2024-09-15T21:24:02.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd3fff640 1 --2- 172.21.15.119:0/1200719696 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd413aa00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009dc0 tx=0x7f0bc8020c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a736b809561279c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:02.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 1 -- 172.21.15.119:0/1200719696 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80b6070 con 0x7f0bd40a0160 2024-09-15T21:24:02.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: dump: 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:02.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 5 monclient: authenticate success, global_id 44855 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 1 -- 172.21.15.119:0/1200719696 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc80a98f0 con 0x7f0bd40a0160 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 1 -- 172.21.15.119:0/1200719696 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80a9ca0 con 0x7f0bd40a0160 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 4 set_mon_vals no callback set 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bd37fe640 10 monclient: dump: 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:02.507 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 10 monclient: shutdown 2024-09-15T21:24:02.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 -- 172.21.15.119:0/1200719696 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 msgr2=0x7f0bd413aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/1200719696 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd413aa00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009dc0 tx=0x7f0bc8020c60 comp rx=0 tx=0).stop 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 -- 172.21.15.119:0/1200719696 shutdown_connections 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/1200719696 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd413aa00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 -- 172.21.15.119:0/1200719696 >> 172.21.15.119:0/1200719696 conn(0x7f0bd40a2ae0 msgr2=0x7f0bd40a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 -- 172.21.15.119:0/1200719696 shutdown_connections 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 1 -- 172.21.15.119:0/1200719696 wait complete. 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 10 monclient: build_initial_monmap 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.494+0000 7f0bdadc6640 10 monclient: monmap: 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:02.509 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 Processor -- start 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- start start 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: init 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0bd406ec30 0x7f0bd4132f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): picked mon.noname-a con 0x7f0bd406ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:02.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd4133460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): picked mon.noname-c con 0x7f0bd40a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd4135ca0 0x7f0bd41339a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): picked mon.noname-b con 0x7f0bd4135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd413f040 con 0x7f0bd406ec30 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd413eec0 con 0x7f0bd4135ca0 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd413f1c0 con 0x7f0bd40a0160 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient(hunting): authenticate will time out at 1361.296581s 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd9363640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd4135ca0 msgr2=0x7f0bd41339a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd9363640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd4135ca0 0x7f0bd41339a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:02.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd4133460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0bd406ec30 0x7f0bd4132f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd4133460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:44862/0 (socket says 172.21.15.119:44862) 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 1 -- 172.21.15.119:0/3114589608 learned_addr learned my addr 172.21.15.119:0/3114589608 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): get_auth_request con 0x7f0bd40a0160 auth_method 0 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): get_auth_request con 0x7f0bd406ec30 auth_method 0 2024-09-15T21:24:02.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient(hunting): handle_auth_done global_id 74313 payload 824 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 -- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 msgr2=0x7f0bd4133460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd4133460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 -- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd4135ca0 msgr2=0x7f0bd41339a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:02.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd4135ca0 0x7f0bd41339a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient: _finish_hunting 0 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 monclient: found mon.noname-a 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 -- 172.21.15.119:0/3114589608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd411d240 con 0x7f0bd406ec30 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd8b62640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0bd406ec30 0x7f0bd4132f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c400 tx=0x7f0bc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4017070 con 0x7f0bd406ec30 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: dump: 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:02.514 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 5 monclient: authenticate success, global_id 74313 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _renew_subs 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:02.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd4134fa0 con 0x7f0bd406ec30 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _renew_subs 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd413ce00 con 0x7f0bd406ec30 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc4005cf0 con 0x7f0bd406ec30 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40072d0 con 0x7f0bd406ec30 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 10 monclient: dump: 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.516 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd2ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd4137200 con 0x7f0bd406ec30 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0bc401a070 con 0x7f0bd406ec30 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0bc003b9a0 0x7f0bc003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:02.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.498+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 7601+0+0 (secure 0 0 0) 0x7f0bc404f9d0 con 0x7f0bd406ec30 2024-09-15T21:24:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.502+0000 7f0bd3fff640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0bc003b9a0 0x7f0bc003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.502+0000 7f0bd3fff640 10 monclient: get_auth_request con 0x7f0bc003b9a0 auth_method 0 2024-09-15T21:24:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.502+0000 7f0bd3fff640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0bc003b9a0 0x7f0bc003de60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80b4040 tx=0x7f0bc80ab070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.514+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0bc4012400 con 0x7f0bd406ec30 2024-09-15T21:24:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.514+0000 7f0bd17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:02.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.514+0000 7f0bd17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:02.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.694+0000 7f0bdadc6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:24:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.694+0000 7f0bdadc6640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:24:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.694+0000 7f0bdadc6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:02.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:02.694+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f0bd409f8f0 con 0x7f0bd406ec30 2024-09-15T21:24:03.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.354+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v182) ==== 78+0+0 (secure 0 0 0) 0x7f0bc4012780 con 0x7f0bd406ec30 2024-09-15T21:24:03.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.354+0000 7f0bd17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:24:03.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.354+0000 7f0bd17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:03.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.462+0000 7f0bdadc6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:24:03.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.462+0000 7f0bdadc6640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:24:03.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.462+0000 7f0bdadc6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:03.466 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:03.462+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f0bd409f610 con 0x7f0bd406ec30 2024-09-15T21:24:04.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.354+0000 7f0bd17fa640 1 -- 172.21.15.119:0/3114589608 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v183) ==== 78+0+0 (secure 0 0 0) 0x7f0bc4010300 con 0x7f0bd406ec30 2024-09-15T21:24:04.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.354+0000 7f0bd17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:24:04.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.354+0000 7f0bd17fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:24:04.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0bc003b9a0 msgr2=0x7f0bc003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:04.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0bc003b9a0 0x7f0bc003de60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80b4040 tx=0x7f0bc80ab070 comp rx=0 tx=0).stop 2024-09-15T21:24:04.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 10 monclient: shutdown 2024-09-15T21:24:04.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:04.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0bd406ec30 msgr2=0x7f0bd4132f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:04.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0bd406ec30 0x7f0bd4132f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c400 tx=0x7f0bc400c8c0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 shutdown_connections 2024-09-15T21:24:04.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0bc003b9a0 0x7f0bc003de60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd4135ca0 0x7f0bd41339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0bd40a0160 0x7f0bd4133460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 --2- 172.21.15.119:0/3114589608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0bd406ec30 0x7f0bd4132f20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.368 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 >> 172.21.15.119:0/3114589608 conn(0x7f0bd40a2ae0 msgr2=0x7f0bd40a4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:04.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 shutdown_connections 2024-09-15T21:24:04.369 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.358+0000 7f0bdadc6640 1 -- 172.21.15.119:0/3114589608 wait complete. 2024-09-15T21:24:04.382 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T21:24:04.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient: get_monmap_and_config 2024-09-15T21:24:04.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient: build_initial_monmap 2024-09-15T21:24:04.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient: monmap: 2024-09-15T21:24:04.502 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:04.502118+0000 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:04.502118+0000 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:04.503 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:04.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 1 Processor -- start 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 1 -- start start 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient: init 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd254137bc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2540a09e0 0x7fd25413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient(hunting): picked mon.noname-a con 0x7fd2540a09e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd2540a0ea0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25409cfa0 con 0x7fd254137bc0 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25409cd20 con 0x7fd2540a0ea0 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd2541386b0 con 0x7fd2540a09e0 2024-09-15T21:24:04.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd25af8b640 10 monclient(hunting): authenticate will time out at 1363.298762s 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd259528640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fd2540a0ea0 legacy=0x7fd25413efe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2540a09e0 0x7fd25413ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2540a09e0 0x7fd25413ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54614/0 (socket says 172.21.15.119:54614) 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 -- 172.21.15.119:0/3432912700 learned_addr learned my addr 172.21.15.119:0/3432912700 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): get_auth_request con 0x7fd2540a09e0 auth_method 0 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:04.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient(hunting): handle_auth_done global_id 74316 payload 274 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 -- 172.21.15.119:0/3432912700 >> v1:172.21.15.119:6789/0 conn(0x7fd2540a0ea0 legacy=0x7fd25413efe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 -- 172.21.15.119:0/3432912700 >> v1:172.21.15.17:6789/0 conn(0x7fd254137bc0 legacy=0x7fd254137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 monclient: found mon.noname-a 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:04.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.498+0000 7fd253fff640 1 -- 172.21.15.119:0/3432912700 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2540a1820 con 0x7fd2540a09e0 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd253fff640 1 --2- 172.21.15.119:0/3432912700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2540a09e0 0x7fd25413ca90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd240009870 tx=0x7fd24001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2fba3490156460 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 1 -- 172.21.15.119:0/3432912700 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400b0070 con 0x7fd2540a09e0 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: dump: 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:04.512 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 1 -- 172.21.15.119:0/3432912700 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2400ab600 con 0x7fd2540a09e0 2024-09-15T21:24:04.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 1 -- 172.21.15.119:0/3432912700 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400ab9d0 con 0x7fd2540a09e0 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 5 monclient: authenticate success, global_id 74316 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 4 set_mon_vals no callback set 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd2537fe640 10 monclient: dump: 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:04.514 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 10 monclient: shutdown 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 -- 172.21.15.119:0/3432912700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2540a09e0 msgr2=0x7fd25413ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 --2- 172.21.15.119:0/3432912700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2540a09e0 0x7fd25413ca90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd240009870 tx=0x7fd24001c4b0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 -- 172.21.15.119:0/3432912700 shutdown_connections 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 --2- 172.21.15.119:0/3432912700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2540a09e0 0x7fd25413ca90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 -- 172.21.15.119:0/3432912700 >> 172.21.15.119:0/3432912700 conn(0x7fd2540a2ae0 msgr2=0x7fd2540a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 -- 172.21.15.119:0/3432912700 shutdown_connections 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 1 -- 172.21.15.119:0/3432912700 wait complete. 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 10 monclient: build_initial_monmap 2024-09-15T21:24:04.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.502+0000 7fd25af8b640 10 monclient: monmap: 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 Processor -- start 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- start start 2024-09-15T21:24:04.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: init 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2540a09e0 0x7fd2541cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): picked mon.noname-c con 0x7fd2540a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2540a0ea0 0x7fd254132890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): picked mon.noname-b con 0x7fd2540a0ea0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd254137bc0 0x7fd25413be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): picked mon.noname-a con 0x7fd254137bc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd254143100 con 0x7fd254137bc0 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd254142f80 con 0x7fd2540a0ea0 2024-09-15T21:24:04.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd254143280 con 0x7fd2540a09e0 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient(hunting): authenticate will time out at 1363.304869s 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd253fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2540a0ea0 msgr2=0x7fd254132890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd253fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2540a0ea0 0x7fd254132890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2540a09e0 0x7fd2541cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd254137bc0 0x7fd25413be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2540a09e0 0x7fd2541cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:44876/0 (socket says 172.21.15.119:44876) 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 1 -- 172.21.15.119:0/2565959966 learned_addr learned my addr 172.21.15.119:0/2565959966 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): get_auth_request con 0x7fd254137bc0 auth_method 0 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:04.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): get_auth_request con 0x7fd2540a09e0 auth_method 0 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd258d27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient(hunting): handle_auth_done global_id 74322 payload 824 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 -- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2540a09e0 msgr2=0x7fd2541cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2540a09e0 0x7fd2541cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 -- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2540a0ea0 msgr2=0x7fd254132890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2540a0ea0 0x7fd254132890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient: _finish_hunting 0 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 monclient: found mon.noname-a 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 -- 172.21.15.119:0/2565959966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25413f670 con 0x7fd254137bc0 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd259528640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd254137bc0 0x7fd25413be10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00bd90 tx=0x7fd24c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c0064e0 con 0x7fd254137bc0 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: dump: 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:04.520 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 5 monclient: authenticate success, global_id 74322 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _renew_subs 2024-09-15T21:24:04.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2540a59f0 con 0x7fd254137bc0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24c006680 con 0x7fd254137bc0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _renew_subs 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25413fc30 con 0x7fd254137bc0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd252ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c01a020 con 0x7fd254137bc0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 10 monclient: dump: 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:04.522 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:04.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25413e840 con 0x7fd254137bc0 2024-09-15T21:24:04.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.506+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd24c006a30 con 0x7fd254137bc0 2024-09-15T21:24:04.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.510+0000 7fd2517fa640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd23803b970 0x7fd23803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:04.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.510+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 7273+0+0 (secure 0 0 0) 0x7fd24c051460 con 0x7fd254137bc0 2024-09-15T21:24:04.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.510+0000 7fd258d27640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd23803b970 0x7fd23803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:04.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.510+0000 7fd258d27640 10 monclient: get_auth_request con 0x7fd23803b970 auth_method 0 2024-09-15T21:24:04.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.510+0000 7fd258d27640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd23803b970 0x7fd23803de30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd254140070 tx=0x7fd248009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:04.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.522+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd25413e840 con 0x7fd254137bc0 2024-09-15T21:24:04.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.522+0000 7fd2517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:04.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.522+0000 7fd2517fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:04.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.690+0000 7fd25af8b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:24:04.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.690+0000 7fd25af8b640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:24:04.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.690+0000 7fd25af8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:04.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:04.690+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7fd254137fa0 con 0x7fd254137bc0 2024-09-15T21:24:05.376 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.370+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v184) ==== 82+0+0 (secure 0 0 0) 0x7fd24c024080 con 0x7fd254137bc0 2024-09-15T21:24:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.370+0000 7fd2517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:24:05.377 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.370+0000 7fd2517fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:05.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.474+0000 7fd25af8b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:24:05.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:24:05.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:05.481 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7fd25413e430 con 0x7fd254137bc0 2024-09-15T21:24:05.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd2517fa640 1 -- 172.21.15.119:0/2565959966 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v184) ==== 115+0+0 (secure 0 0 0) 0x7fd24c04e5b0 con 0x7fd254137bc0 2024-09-15T21:24:05.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd2517fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T21:24:05.482 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd2517fa640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T21:24:05.482 INFO:tasks.workunit.client.0.smithi119.stderr:compat weight-set already created 2024-09-15T21:24:05.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd23803b970 msgr2=0x7fd23803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:05.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd23803b970 0x7fd23803de30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd254140070 tx=0x7fd248009040 comp rx=0 tx=0).stop 2024-09-15T21:24:05.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 10 monclient: shutdown 2024-09-15T21:24:05.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:05.484 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd254137bc0 msgr2=0x7fd25413be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd254137bc0 0x7fd25413be10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00bd90 tx=0x7fd24c009eb0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 shutdown_connections 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd23803b970 0x7fd23803de30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd254137bc0 0x7fd25413be10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2540a0ea0 0x7fd254132890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 --2- 172.21.15.119:0/2565959966 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2540a09e0 0x7fd2541cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 >> 172.21.15.119:0/2565959966 conn(0x7fd2540a2ae0 msgr2=0x7fd2540a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:05.485 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 shutdown_connections 2024-09-15T21:24:05.486 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.478+0000 7fd25af8b640 1 -- 172.21.15.119:0/2565959966 wait complete. 2024-09-15T21:24:05.499 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket r1 rack root=default 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient: get_monmap_and_config 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient: build_initial_monmap 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient: monmap: 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:05.621999+0000 2024-09-15T21:24:05.622 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:05.621999+0000 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:05.623 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 1 Processor -- start 2024-09-15T21:24:05.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 1 -- start start 2024-09-15T21:24:05.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient: init 2024-09-15T21:24:05.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f55b40a0830 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55b4138c40 0x7f55b413b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient(hunting): picked mon.noname-c con 0x7f55b4138c40 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f55b413b570 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b409f320 con 0x7f55b40a0830 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b409ea80 con 0x7f55b413b570 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f55b406d0b0 con 0x7f55b4138c40 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55ba794640 10 monclient(hunting): authenticate will time out at 1364.419590s 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55b8d31640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f55b413b570 legacy=0x7f55b413d960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55b37fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f55b4138c40 msgr2=0x7f55b413b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:05.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55b37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55b4138c40 0x7f55b413b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:05.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55b3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f55b40a0830 0x7f55b40a0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:57204/0 (socket says 172.21.15.119:57204) 2024-09-15T21:24:05.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.618+0000 7f55b3fff640 1 -- 172.21.15.119:0/2066739837 learned_addr learned my addr 172.21.15.119:0/2066739837 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:05.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125463806 0 0) 0x7f55b409f320 con 0x7f55b40a0830 2024-09-15T21:24:05.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:24:05.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:05.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:05.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): my global_id is 74325 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f559c003530 con 0x7f55b40a0830 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1311354931 0 0) 0x7f559c003530 con 0x7f55b40a0830 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 >> v2:172.21.15.119:3300/0 conn(0x7f55b4138c40 msgr2=0x7f55b413b030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 --2- 172.21.15.119:0/2066739837 >> v2:172.21.15.119:3300/0 conn(0x7f55b4138c40 0x7f55b413b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 >> v1:172.21.15.119:6789/0 conn(0x7f55b413b570 legacy=0x7f55b413d960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40a14a0 con 0x7f55b40a0830 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:05.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f55a4001a80 con 0x7f55b40a0830 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: dump: 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:05.632 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 >> v1:172.21.15.17:6789/0 conn(0x7f55b40a0830 legacy=0x7f55b40a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f559c003a40 0x7f559c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): picked mon.c con 0x7f559c003a40 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f559c004810 0x7f559c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): picked mon.b con 0x7f559c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c0053b0 0x7f559c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): picked mon.a con 0x7f559c0053b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f55b406d0b0 con 0x7f559c0053b0 2024-09-15T21:24:05.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f559c00c2f0 con 0x7f559c004810 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f559c00c4f0 con 0x7f559c003a40 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b37fe640 1 -- 172.21.15.119:0/2066739837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f559c004810 msgr2=0x7f559c004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b37fe640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f559c004810 0x7f559c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f559c003a40 0x7f559c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): get_auth_request con 0x7f559c003a40 auth_method 0 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:05.634 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c0053b0 0x7f559c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): get_auth_request con 0x7f559c0053b0 auth_method 0 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient(hunting): handle_auth_done global_id 74325 payload 293 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 -- 172.21.15.119:0/2066739837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f559c004810 msgr2=0x7f559c004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f559c004810 0x7f559c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 -- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c0053b0 msgr2=0x7f559c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c0053b0 0x7f559c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 monclient: found mon.c 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b8d31640 1 -- 172.21.15.119:0/2066739837 reap_dead start 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 1 -- 172.21.15.119:0/2066739837 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c003530 con 0x7f559c003a40 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient: _finish_auth 0 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55ba794640 5 monclient: authenticate success, global_id 74325 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.622+0000 7f55ba794640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:05.636 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f559c003a40 0x7f559c003e40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55a400a7a0 tx=0x7f55a401f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cae26ffe84ca512 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a400a3f0 con 0x7f559c003a40 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: dump: 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:05.637 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a40aa630 con 0x7f559c003a40 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 4 set_mon_vals no callback set 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 1 -- 172.21.15.119:0/2066739837 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40aaa40 con 0x7f559c003a40 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b2ffd640 10 monclient: dump: 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.638 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: shutdown 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f559c003a40 msgr2=0x7f559c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f559c003a40 0x7f559c003e40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55a400a7a0 tx=0x7f55a401f960 comp rx=0 tx=0).stop 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- 172.21.15.119:0/2066739837 shutdown_connections 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 --2- 172.21.15.119:0/2066739837 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f559c003a40 0x7f559c003e40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- 172.21.15.119:0/2066739837 >> 172.21.15.119:0/2066739837 conn(0x7f55b40a3bd0 msgr2=0x7f55b40a3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- 172.21.15.119:0/2066739837 shutdown_connections 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- 172.21.15.119:0/2066739837 wait complete. 2024-09-15T21:24:05.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: build_initial_monmap 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: monmap: 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:05.640 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 Processor -- start 2024-09-15T21:24:05.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- start start 2024-09-15T21:24:05.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: init 2024-09-15T21:24:05.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:05.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:05.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 0x7f55b4134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): picked mon.noname-a con 0x7f559c003a40 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b4134b40 0x7f55b409ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): picked mon.noname-b con 0x7f55b4134b40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f55b4134fc0 0x7f55b409bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): picked mon.noname-c con 0x7f55b4134fc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f55b40a4aa0 con 0x7f559c003a40 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55b40a4920 con 0x7f55b4134b40 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f55b406ccd0 con 0x7f55b4134fc0 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55ba794640 10 monclient(hunting): authenticate will time out at 1364.426706s 2024-09-15T21:24:05.645 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b37fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b4134b40 msgr2=0x7f55b409ba60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b4134b40 0x7f55b409ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 0x7f55b4134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f55b4134fc0 0x7f55b409bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 0x7f55b4134600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54650/0 (socket says 172.21.15.119:54650) 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 -- 172.21.15.119:0/1972431115 learned_addr learned my addr 172.21.15.119:0/1972431115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): get_auth_request con 0x7f559c003a40 auth_method 0 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): get_auth_request con 0x7f55b4134fc0 auth_method 0 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:05.646 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient(hunting): handle_auth_done global_id 74328 payload 824 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 -- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f55b4134fc0 msgr2=0x7f55b409bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f55b4134fc0 0x7f55b409bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 -- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b4134b40 msgr2=0x7f55b409ba60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b4134b40 0x7f55b409ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 monclient: found mon.noname-a 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:05.647 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 -- 172.21.15.119:0/1972431115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b41233e0 con 0x7f559c003a40 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b8d31640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f55b4134fc0 0x7f55b409bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.626+0000 7f55b3fff640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 0x7f55b4134600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55a401e060 tx=0x7f55a401c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40c0070 con 0x7f559c003a40 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: dump: 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:05.648 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 5 monclient: authenticate success, global_id 74328 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 monclient: _renew_subs 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b40a0260 con 0x7f559c003a40 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a40aa7e0 con 0x7f559c003a40 2024-09-15T21:24:05.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40aab90 con 0x7f559c003a40 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 10 monclient: dump: 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:05.650 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b27fc640 4 set_mon_vals no callback set 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 monclient: _renew_subs 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b409e2c0 con 0x7f559c003a40 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b4135df0 con 0x7f559c003a40 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f55a40070c0 con 0x7f559c003a40 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f55a003b9a0 0x7f55a003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 7601+0+0 (secure 0 0 0) 0x7f55a40eeab0 con 0x7f559c003a40 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b37fe640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f55a003b9a0 0x7f55a003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b37fe640 10 monclient: get_auth_request con 0x7f55a003b9a0 auth_method 0 2024-09-15T21:24:05.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.630+0000 7f55b37fe640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f55a003b9a0 0x7f55a003de60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006fd0 tx=0x7f55a8006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:05.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.642+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f55a40af020 con 0x7f559c003a40 2024-09-15T21:24:05.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.642+0000 7f55b0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:05.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.642+0000 7f55b0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:05.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.802+0000 7f55ba794640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T21:24:05.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.802+0000 7f55ba794640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T21:24:05.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.802+0000 7f55ba794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:05.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:05.802+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) -- 0x7f55b406a760 con 0x7f559c003a40 2024-09-15T21:24:06.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.382+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 added bucket r1 type rack to location {root=default} v185) ==== 176+0+0 (secure 0 0 0) 0x7f55a4019710 con 0x7f559c003a40 2024-09-15T21:24:06.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.382+0000 7f55b0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T21:24:06.386 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.382+0000 7f55b0ff9640 10 monclient: _finish_command 2 = system:0 added bucket r1 type rack to location {root=default} 2024-09-15T21:24:06.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T21:24:06.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T21:24:06.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:06.493 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) -- 0x7f55b4121be0 con 0x7f559c003a40 2024-09-15T21:24:06.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55b0ff9640 1 -- 172.21.15.119:0/1972431115 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 bucket 'r1' already exists v185) ==== 150+0+0 (secure 0 0 0) 0x7f55a40aad30 con 0x7f559c003a40 2024-09-15T21:24:06.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55b0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T21:24:06.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55b0ff9640 10 monclient: _finish_command 3 = system:0 bucket 'r1' already exists 2024-09-15T21:24:06.494 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'r1' already exists 2024-09-15T21:24:06.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f55a003b9a0 msgr2=0x7f55a003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f55a003b9a0 0x7f55a003de60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006fd0 tx=0x7f55a8006d40 comp rx=0 tx=0).stop 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 10 monclient: shutdown 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 msgr2=0x7f55b4134600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 0x7f55b4134600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55a401e060 tx=0x7f55a401c7a0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 shutdown_connections 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f55a003b9a0 0x7f55a003de60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f55b4134fc0 0x7f55b409bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b4134b40 0x7f55b409ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 --2- 172.21.15.119:0/1972431115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f559c003a40 0x7f55b4134600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 >> 172.21.15.119:0/1972431115 conn(0x7f55b40a3bd0 msgr2=0x7f55b40a5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 shutdown_connections 2024-09-15T21:24:06.497 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.490+0000 7f55ba794640 1 -- 172.21.15.119:0/1972431115 wait complete. 2024-09-15T21:24:06.512 INFO:tasks.workunit.client.0.smithi119.stderr:++ seq 1 32 2024-09-15T21:24:06.519 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:06.520 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h1 host rack=r1 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.630+0000 7f865a06a640 10 monclient: get_monmap_and_config 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.630+0000 7f865a06a640 10 monclient: build_initial_monmap 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.630+0000 7f865a06a640 10 monclient: monmap: 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:06.634700+0000 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:06.634700+0000 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:06.635 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:06.636 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:06.636 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:06.636 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:06.636 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:06.636 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.630+0000 7f865a06a640 1 Processor -- start 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.630+0000 7f865a06a640 1 -- start start 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.630+0000 7f865a06a640 10 monclient: init 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8654132680 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f86540a0e00 0x7f865413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:06.639 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): picked mon.noname-a con 0x7f86540a0e00 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f865413afd0 0x7f865413d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): picked mon.noname-b con 0x7f865413afd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8654133170 con 0x7f8654132680 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f865406f5a0 con 0x7f86540a0e00 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8654072ff0 con 0x7f865413afd0 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f865a06a640 10 monclient(hunting): authenticate will time out at 1365.432208s 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f86537fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f8654132680 legacy=0x7f8654132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f86540a0e00 0x7f865413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f865413afd0 0x7f865413d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:06.640 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f86540a0e00 0x7f865413aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54654/0 (socket says 172.21.15.119:54654) 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 -- 172.21.15.119:0/831949667 learned_addr learned my addr 172.21.15.119:0/831949667 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): get_auth_request con 0x7f86540a0e00 auth_method 0 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): get_auth_request con 0x7f865413afd0 auth_method 0 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:06.641 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient(hunting): handle_auth_done global_id 74331 payload 274 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 -- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f865413afd0 msgr2=0x7f865413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 --2- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f865413afd0 0x7f865413d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 -- 172.21.15.119:0/831949667 >> v1:172.21.15.119:6789/0 conn(0x7f8654132680 legacy=0x7f8654132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 monclient: found mon.noname-a 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 -- 172.21.15.119:0/831949667 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8654072a30 con 0x7f86540a0e00 2024-09-15T21:24:06.642 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8653fff640 1 --2- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f865413afd0 0x7f865413d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:06.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f864bfff640 1 --2- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86540a0e00 0x7f865413aa90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f863c009870 tx=0x7f863c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb539f1a912ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:06.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 1 -- 172.21.15.119:0/831949667 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c0b2070 con 0x7f86540a0e00 2024-09-15T21:24:06.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:06.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:06.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: dump: 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:06.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 1 -- 172.21.15.119:0/831949667 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f863c0ad640 con 0x7f86540a0e00 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 1 -- 172.21.15.119:0/831949667 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c0ada10 con 0x7f86540a0e00 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.634+0000 7f8652ffd640 10 monclient: dump: 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:06.650 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 5 monclient: authenticate success, global_id 74331 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 4 set_mon_vals no callback set 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: shutdown 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86540a0e00 msgr2=0x7f865413aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 --2- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86540a0e00 0x7f865413aa90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f863c009870 tx=0x7f863c01c3f0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- 172.21.15.119:0/831949667 shutdown_connections 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 --2- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f865413afd0 0x7f865413d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 --2- 172.21.15.119:0/831949667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86540a0e00 0x7f865413aa90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- 172.21.15.119:0/831949667 >> 172.21.15.119:0/831949667 conn(0x7f865406cbf0 msgr2=0x7f865406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- 172.21.15.119:0/831949667 shutdown_connections 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- 172.21.15.119:0/831949667 wait complete. 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: build_initial_monmap 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: monmap: 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:06.652 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 Processor -- start 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- start start 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: init 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f86540a0e00 0x7f865409fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): picked mon.noname-c con 0x7f86540a0e00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 0x7f86541397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): picked mon.noname-a con 0x7f8654132680 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f865413afd0 0x7f86541cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:06.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): picked mon.noname-b con 0x7f865413afd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8654140120 con 0x7f8654132680 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8654073d70 con 0x7f865413afd0 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f86541402a0 con 0x7f86540a0e00 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f865a06a640 10 monclient(hunting): authenticate will time out at 1365.438698s 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f8653fff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f865413afd0 msgr2=0x7f86541cd400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f8653fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f865413afd0 0x7f86541cd400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 0x7f86541397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f86540a0e00 0x7f865409fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:06.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 0x7f86541397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54668/0 (socket says 172.21.15.119:54668) 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 1 -- 172.21.15.119:0/1872449105 learned_addr learned my addr 172.21.15.119:0/1872449105 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): get_auth_request con 0x7f86540a0e00 auth_method 0 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): get_auth_request con 0x7f8654132680 auth_method 0 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:06.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient(hunting): handle_auth_done global_id 44870 payload 824 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 -- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f865413afd0 msgr2=0x7f86541cd400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f865413afd0 0x7f86541cd400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 -- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 msgr2=0x7f86541397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 0x7f86541397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 monclient: found mon.noname-c 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f86537fe640 1 -- 172.21.15.119:0/1872449105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86540a0540 con 0x7f86540a0e00 2024-09-15T21:24:06.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.638+0000 7f864bfff640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 0x7f86541397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f86537fe640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f86540a0e00 0x7f865409fae0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f864400b400 tx=0x7f864400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644016070 con 0x7f86540a0e00 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: dump: 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:06.657 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8644003df0 con 0x7f86540a0e00 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440052b0 con 0x7f86540a0e00 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 10 monclient: dump: 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:06.658 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f86527fc640 4 set_mon_vals no callback set 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 5 monclient: authenticate success, global_id 44870 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 monclient: _renew_subs 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86541357a0 con 0x7f86540a0e00 2024-09-15T21:24:06.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 monclient: _renew_subs 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8654134840 con 0x7f86540a0e00 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8654132a60 con 0x7f86540a0e00 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8644019070 con 0x7f86540a0e00 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f863403b970 0x7f863403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(185..185 src has 1..185) ==== 7811+0+0 (secure 0 0 0) 0x7f864404fbe0 con 0x7f86540a0e00 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f864bfff640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f863403b970 0x7f863403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f864bfff640 10 monclient: get_auth_request con 0x7f863403b970 auth_method 0 2024-09-15T21:24:06.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.642+0000 7f864bfff640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f863403b970 0x7f863403de30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f863c01cc90 tx=0x7f863c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:06.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.654+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f864400f300 con 0x7f86540a0e00 2024-09-15T21:24:06.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.654+0000 7f8650ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:06.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.654+0000 7f8650ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:06.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.826+0000 7f865a06a640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:06.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.826+0000 7f865a06a640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:06.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.826+0000 7f865a06a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:06.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:06.826+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8654074bd0 con 0x7f86540a0e00 2024-09-15T21:24:07.394 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.390+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 added bucket h1 type host to location {rack=r1} v186) ==== 166+0+0 (secure 0 0 0) 0x7f86440200b0 con 0x7f86540a0e00 2024-09-15T21:24:07.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.390+0000 7f8650ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:07.395 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.390+0000 7f8650ff9640 10 monclient: _finish_command 2 = system:0 added bucket h1 type host to location {rack=r1} 2024-09-15T21:24:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f865a06a640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f865a06a640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f865a06a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:07.494 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8654133880 con 0x7f86540a0e00 2024-09-15T21:24:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f8650ff9640 1 -- 172.21.15.119:0/1872449105 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h1' already exists v186) ==== 145+0+0 (secure 0 0 0) 0x7f864400fe20 con 0x7f86540a0e00 2024-09-15T21:24:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f8650ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.490+0000 7f8650ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h1' already exists 2024-09-15T21:24:07.496 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h1' already exists 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f863403b970 msgr2=0x7f863403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f863403b970 0x7f863403de30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f863c01cc90 tx=0x7f863c009040 comp rx=0 tx=0).stop 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 10 monclient: shutdown 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f86540a0e00 msgr2=0x7f865409fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f86540a0e00 0x7f865409fae0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f864400b400 tx=0x7f864400b8c0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 shutdown_connections 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f863403b970 0x7f863403de30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.499 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f865413afd0 0x7f86541cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8654132680 0x7f86541397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 --2- 172.21.15.119:0/1872449105 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f86540a0e00 0x7f865409fae0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 >> 172.21.15.119:0/1872449105 conn(0x7f865406cbf0 msgr2=0x7f86540a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 shutdown_connections 2024-09-15T21:24:07.500 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.494+0000 7f865a06a640 1 -- 172.21.15.119:0/1872449105 wait complete. 2024-09-15T21:24:07.514 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:07.514 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h2 host rack=r1 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.630+0000 7f8a06719640 10 monclient: get_monmap_and_config 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.630+0000 7f8a06719640 10 monclient: build_initial_monmap 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.630+0000 7f8a06719640 10 monclient: monmap: 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:07.636686+0000 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:07.636686+0000 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:07.637 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:07.638 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:07.638 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:07.638 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:07.638 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 Processor -- start 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 -- start start 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient: init 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 0x7f8a0009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): picked mon.noname-b con 0x7f8a00135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a00135fd0 0x7f8a000a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): picked mon.noname-c con 0x7f8a00135fd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a00136430 0x7f8a000a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): picked mon.noname-a con 0x7f8a00136430 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8a000a3600 con 0x7f8a00136430 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a000a1d80 con 0x7f8a00135fd0 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000a2060 con 0x7f8a00135bf0 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a06719640 10 monclient(hunting): authenticate will time out at 1366.433499s 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89ff7fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f8a00135fd0 msgr2=0x7f8a000a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89ff7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a00135fd0 0x7f8a000a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 0x7f8a0009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a00136430 0x7f8a000a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 0x7f8a0009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:44924/0 (socket says 172.21.15.119:44924) 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a00136430 0x7f8a000a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54674/0 (socket says 172.21.15.119:54674) 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 1 -- 172.21.15.119:0/2875330954 learned_addr learned my addr 172.21.15.119:0/2875330954 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:07.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): get_auth_request con 0x7f8a00135bf0 auth_method 0 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): get_auth_request con 0x7f8a00136430 auth_method 0 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_done global_id 74337 payload 274 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 -- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 msgr2=0x7f8a0009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 --2- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 0x7f8a0009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 -- 172.21.15.119:0/2875330954 >> v2:172.21.15.119:3300/0 conn(0x7f8a00135fd0 msgr2=0x7f8a000a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 --2- 172.21.15.119:0/2875330954 >> v2:172.21.15.119:3300/0 conn(0x7f8a00135fd0 0x7f8a000a0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient: _finish_hunting 0 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 monclient: found mon.noname-a 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f8a04cb6640 1 -- 172.21.15.119:0/2875330954 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000a2340 con 0x7f8a00136430 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.634+0000 7f89fffff640 1 --2- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 0x7f8a0009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a04cb6640 1 --2- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00136430 0x7f8a000a0f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a0009d110 tx=0x7f89f801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3718405556273e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:07.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 1 -- 172.21.15.119:0/2875330954 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80ac070 con 0x7f8a00136430 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: dump: 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.655 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 1 -- 172.21.15.119:0/2875330954 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f80a66a0 con 0x7f8a00136430 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 1 -- 172.21.15.119:0/2875330954 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80a6a70 con 0x7f8a00136430 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89feffd640 10 monclient: dump: 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.656 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 5 monclient: authenticate success, global_id 74337 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 4 set_mon_vals no callback set 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: shutdown 2024-09-15T21:24:07.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00136430 msgr2=0x7f8a000a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00136430 0x7f8a000a0f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a0009d110 tx=0x7f89f801c810 comp rx=0 tx=0).stop 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- 172.21.15.119:0/2875330954 shutdown_connections 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- 172.21.15.119:0/2875330954 >> v2:172.21.15.119:3300/0 conn(0x7f8a00135fd0 0x7f8a000a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00136430 0x7f8a000a0f90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- 172.21.15.119:0/2875330954 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00135bf0 0x7f8a0009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- 172.21.15.119:0/2875330954 >> 172.21.15.119:0/2875330954 conn(0x7f8a000a66e0 msgr2=0x7f8a000a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- 172.21.15.119:0/2875330954 shutdown_connections 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- 172.21.15.119:0/2875330954 wait complete. 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: build_initial_monmap 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: monmap: 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 Processor -- start 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- start start 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: init 2024-09-15T21:24:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a00135bf0 0x7f8a001cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): picked mon.noname-b con 0x7f8a00135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 0x7f8a00134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): picked mon.noname-a con 0x7f8a00135fd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 0x7f8a00135210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): picked mon.noname-c con 0x7f8a00136430 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0013f2a0 con 0x7f8a00135fd0 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000a18e0 con 0x7f8a00135bf0 2024-09-15T21:24:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0013f420 con 0x7f8a00136430 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f8a06719640 10 monclient(hunting): authenticate will time out at 1366.439741s 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89fffff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a00135bf0 msgr2=0x7f8a001cf930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.638+0000 7f89fffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a00135bf0 0x7f8a001cf930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 0x7f8a00135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 0x7f8a00134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 0x7f8a00135210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:44926/0 (socket says 172.21.15.119:44926) 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 1 -- 172.21.15.119:0/2183493644 learned_addr learned my addr 172.21.15.119:0/2183493644 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 0x7f8a00134cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:54684/0 (socket says 172.21.15.119:54684) 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): get_auth_request con 0x7f8a00136430 auth_method 0 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): get_auth_request con 0x7f8a00135fd0 auth_method 0 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient(hunting): handle_auth_done global_id 74340 payload 824 2024-09-15T21:24:07.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 -- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 msgr2=0x7f8a00135210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 0x7f8a00135210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 -- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a00135bf0 msgr2=0x7f8a001cf930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a00135bf0 0x7f8a001cf930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 monclient: found mon.noname-a 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 -- 172.21.15.119:0/2183493644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000a5920 con 0x7f8a00135fd0 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a04cb6640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 0x7f8a00135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89ff7fe640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 0x7f8a00134cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89f000b400 tx=0x7f89f000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0016070 con 0x7f8a00135fd0 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.663 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: dump: 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:07.664 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 5 monclient: authenticate success, global_id 74340 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 monclient: _renew_subs 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001223b0 con 0x7f8a00135fd0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f0003df0 con 0x7f8a00135fd0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 monclient: _renew_subs 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0005270 con 0x7f8a00135fd0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a00120330 con 0x7f8a00135fd0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fe7fc640 4 set_mon_vals no callback set 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:07.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 10 monclient: dump: 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0009fe70 con 0x7f8a00135fd0 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f89f0019070 con 0x7f8a00135fd0 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89e003b970 0x7f89e003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 8021+0+0 (secure 0 0 0) 0x7f89f004fbb0 con 0x7f8a00135fd0 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fffff640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89e003b970 0x7f89e003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fffff640 10 monclient: get_auth_request con 0x7f89e003b970 auth_method 0 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.642+0000 7f89fffff640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89e003b970 0x7f89e003de30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f89e8006c30 tx=0x7f89e8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:07.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.654+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8a000a5920 con 0x7f8a00135fd0 2024-09-15T21:24:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.654+0000 7f89fcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.654+0000 7f89fcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:07.702 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:24:07.702 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:24:07.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:24:07.705 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:24:07.705 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:24:07.705 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:07.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.818+0000 7f8a06719640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:07.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.818+0000 7f8a06719640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:07.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.818+0000 7f8a06719640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.823 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:07.818+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8a00121eb0 con 0x7f8a00135fd0 2024-09-15T21:24:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient: get_monmap_and_config 2024-09-15T21:24:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient: build_initial_monmap 2024-09-15T21:24:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient: monmap: 2024-09-15T21:24:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:07.835423+0000 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:07.835423+0000 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:07.837 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 1 Processor -- start 2024-09-15T21:24:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 1 -- start start 2024-09-15T21:24:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient: init 2024-09-15T21:24:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9f6409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9f6409def0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 0x7f9f6413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient(hunting): picked mon.noname-c con 0x7f9f6413b1d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640a3130 con 0x7f9f6409def0 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6409e7a0 con 0x7f9f6409d400 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9f640a1bc0 con 0x7f9f6413b1d0 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 10 monclient(hunting): authenticate will time out at 1371.230993s 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f637fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9f6409def0 0x7f9f64139ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45556/0 (socket says 172.21.15.17:45556) 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f637fe640 1 -- 172.21.15.17:0/970742242 learned_addr learned my addr 172.21.15.17:0/970742242 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 0x7f9f6413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 10 monclient(hunting): get_auth_request con 0x7f9f6413b1d0 auth_method 0 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 510991650 0 0) 0x7f9f640a3130 con 0x7f9f6409def0 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): my global_id is 74343 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f50003530 con 0x7f9f6409def0 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 1 -- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 msgr2=0x7f9f6413d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:24:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 1 -- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 msgr2=0x7f9f6413d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 0x7f9f6413d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 0x7f9f6413d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 494068907 0 0) 0x7f9f50003530 con 0x7f9f6409def0 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 msgr2=0x7f9f6413d5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/970742242 >> v2:172.21.15.119:3300/0 conn(0x7f9f6413b1d0 0x7f9f6413d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 >> v1:172.21.15.119:6789/0 conn(0x7f9f6409d400 legacy=0x7f9f6409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640a0320 con 0x7f9f6409def0 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 5 monclient: authenticate success, global_id 74343 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f9f54001a80 con 0x7f9f6409def0 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 >> v1:172.21.15.17:6789/0 conn(0x7f9f6409def0 legacy=0x7f9f64139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f50003a60 0x7f9f50003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): picked mon.c con 0x7f9f50003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50004810 0x7f9f50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): picked mon.b con 0x7f9f50004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500053b0 0x7f9f5000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): picked mon.a con 0x7f9f500053b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5000e4d0 con 0x7f9f500053b0 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5000e680 con 0x7f9f50004810 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5000e880 con 0x7f9f50003a60 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f62ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:07.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.832+0000 7f9f6a399640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:07.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500053b0 0x7f9f5000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): get_auth_request con 0x7f9f500053b0 auth_method 0 2024-09-15T21:24:07.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:07.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50004810 0x7f9f50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f50003a60 0x7f9f50003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): get_auth_request con 0x7f9f50004810 auth_method 0 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): get_auth_request con 0x7f9f50003a60 auth_method 0 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f637fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f63fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient(hunting): handle_auth_done global_id 74343 payload 293 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 -- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f50003a60 msgr2=0x7f9f50003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f50003a60 0x7f9f50003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 -- 172.21.15.17:0/970742242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50004810 msgr2=0x7f9f50004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50004810 0x7f9f50004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient: _finish_hunting 0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 monclient: found mon.a 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 -- 172.21.15.17:0/970742242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f50003530 con 0x7f9f500053b0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient: _finish_auth 0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 10 monclient: _check_auth_tickets 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 -- 172.21.15.17:0/970742242 reap_dead start 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f68936640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500053b0 0x7f9f5000dc20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00c820 tx=0x7f9f5c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8220d6c56b6734f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0b0070 con 0x7f9f500053b0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c0ab600 con 0x7f9f500053b0 2024-09-15T21:24:07.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 4 set_mon_vals no callback set 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 1 -- 172.21.15.17:0/970742242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00f040 con 0x7f9f500053b0 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 10 monclient: shutdown 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 -- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500053b0 msgr2=0x7f9f5000dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500053b0 0x7f9f5000dc20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00c820 tx=0x7f9f5c01c510 comp rx=0 tx=0).stop 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 -- 172.21.15.17:0/970742242 shutdown_connections 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 --2- 172.21.15.17:0/970742242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500053b0 0x7f9f5000dc20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 -- 172.21.15.17:0/970742242 >> 172.21.15.17:0/970742242 conn(0x7f9f640a66a0 msgr2=0x7f9f640a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 -- 172.21.15.17:0/970742242 shutdown_connections 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.836+0000 7f9f6a399640 1 -- 172.21.15.17:0/970742242 wait complete. 2024-09-15T21:24:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: build_initial_monmap 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: monmap: 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 Processor -- start 2024-09-15T21:24:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- start start 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: init 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 0x7f9f6413a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): picked mon.noname-c con 0x7f9f500053b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 0x7f9f64134200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): picked mon.noname-b con 0x7f9f64133da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f64134740 0x7f9f641d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): picked mon.noname-a con 0x7f9f64134740 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6409db70 con 0x7f9f64134740 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6409d9f0 con 0x7f9f64133da0 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6409dcf0 con 0x7f9f500053b0 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient(hunting): authenticate will time out at 1371.237215s 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 0x7f9f6413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 0x7f9f6413a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37908/0 (socket says 172.21.15.17:37908) 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 1 -- 172.21.15.17:0/2377233 learned_addr learned my addr 172.21.15.17:0/2377233 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): get_auth_request con 0x7f9f500053b0 auth_method 0 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f64134740 0x7f9f641d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): get_auth_request con 0x7f9f64134740 auth_method 0 2024-09-15T21:24:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 0x7f9f64134200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 10 monclient(hunting): get_auth_request con 0x7f9f64133da0 auth_method 0 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:07.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient(hunting): handle_auth_done global_id 74346 payload 824 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 1 -- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 msgr2=0x7f9f64134200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 1 -- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 msgr2=0x7f9f64134200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 -- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 msgr2=0x7f9f6413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 0x7f9f64134200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 0x7f9f6413a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 -- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 msgr2=0x7f9f64134200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:07.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f637fe640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 0x7f9f64134200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 0x7f9f64134200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient: _finish_hunting 0 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 monclient: found mon.noname-a 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 -- 172.21.15.17:0/2377233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f64137ad0 con 0x7f9f64134740 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 0x7f9f6413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f68936640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f64134740 0x7f9f641d18c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f6406f1e0 tx=0x7f9f50005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0b0040 con 0x7f9f64134740 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: dump: 2024-09-15T21:24:07.870 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c0ab400 con 0x7f9f64134740 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0b5480 con 0x7f9f64134740 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 5 monclient: authenticate success, global_id 74346 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _renew_subs 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f64134bc0 con 0x7f9f64134740 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f627fc640 4 set_mon_vals no callback set 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 10 monclient: dump: 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _renew_subs 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6413ab00 con 0x7f9f64134740 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6409c470 con 0x7f9f64134740 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9f5c0be030 con 0x7f9f64134740 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4403b9c0 0x7f9f4403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 8021+0+0 (secure 0 0 0) 0x7f9f5c0e9c80 con 0x7f9f64134740 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4403b9c0 0x7f9f4403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.840+0000 7f9f63fff640 10 monclient: get_auth_request con 0x7f9f4403b9c0 auth_method 0 2024-09-15T21:24:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.844+0000 7f9f63fff640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4403b9c0 0x7f9f4403de80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0079b0 tx=0x7f9f4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:07.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.844+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9f5c0ef020 con 0x7f9f64134740 2024-09-15T21:24:07.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.844+0000 7f9f60ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:07.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:07.844+0000 7f9f60ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.076+0000 7f9f6a399640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.076+0000 7f9f6a399640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.076+0000 7f9f6a399640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.076+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9f640a10a0 con 0x7f9f64134740 2024-09-15T21:24:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f60ff9640 1 -- 172.21.15.17:0/2377233 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f9f5c0c5060 con 0x7f9f64134740 2024-09-15T21:24:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f60ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f60ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:08.084 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":64,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4403b9c0 msgr2=0x7f9f4403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4403b9c0 0x7f9f4403de80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0079b0 tx=0x7f9f4c009040 comp rx=0 tx=0).stop 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 10 monclient: shutdown 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f64134740 msgr2=0x7f9f641d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f64134740 0x7f9f641d18c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f6406f1e0 tx=0x7f9f50005840 comp rx=0 tx=0).stop 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 shutdown_connections 2024-09-15T21:24:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4403b9c0 0x7f9f4403de80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f64134740 0x7f9f641d18c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f64133da0 0x7f9f64134200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 --2- 172.21.15.17:0/2377233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500053b0 0x7f9f6413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 >> 172.21.15.17:0/2377233 conn(0x7f9f640a66a0 msgr2=0x7f9f6413c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 shutdown_connections 2024-09-15T21:24:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:08.080+0000 7f9f6a399640 1 -- 172.21.15.17:0/2377233 wait complete. 2024-09-15T21:24:08.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.398+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 added bucket h2 type host to location {rack=r1} v187) ==== 166+0+0 (secure 0 0 0) 0x7f89f004ca10 con 0x7f8a00135fd0 2024-09-15T21:24:08.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.398+0000 7f89fcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:08.405 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.398+0000 7f89fcff9640 10 monclient: _finish_command 2 = system:0 added bucket h2 type host to location {rack=r1} 2024-09-15T21:24:08.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f8a06719640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:08.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f8a06719640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:08.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f8a06719640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8a00120e90 con 0x7f8a00135fd0 2024-09-15T21:24:08.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f89fcff9640 1 -- 172.21.15.119:0/2183493644 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h2' already exists v187) ==== 145+0+0 (secure 0 0 0) 0x7f89f003f020 con 0x7f8a00135fd0 2024-09-15T21:24:08.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f89fcff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:08.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.502+0000 7f89fcff9640 10 monclient: _finish_command 3 = system:0 bucket 'h2' already exists 2024-09-15T21:24:08.507 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h2' already exists 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89e003b970 msgr2=0x7f89e003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89e003b970 0x7f89e003de30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f89e8006c30 tx=0x7f89e8009240 comp rx=0 tx=0).stop 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 10 monclient: shutdown 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 msgr2=0x7f8a00134cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 0x7f8a00134cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89f000b400 tx=0x7f89f000b8c0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 shutdown_connections 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89e003b970 0x7f89e003de30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a00136430 0x7f8a00135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.510 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a00135fd0 0x7f8a00134cd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 --2- 172.21.15.119:0/2183493644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a00135bf0 0x7f8a001cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 >> 172.21.15.119:0/2183493644 conn(0x7f8a000a66e0 msgr2=0x7f8a000a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:08.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 shutdown_connections 2024-09-15T21:24:08.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.506+0000 7f8a06719640 1 -- 172.21.15.119:0/2183493644 wait complete. 2024-09-15T21:24:08.525 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:08.525 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h3 host rack=r1 2024-09-15T21:24:08.648 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.642+0000 7f01cf208640 10 monclient: get_monmap_and_config 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.642+0000 7f01cf208640 10 monclient: build_initial_monmap 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.642+0000 7f01cf208640 10 monclient: monmap: 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:08.648610+0000 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:08.648610+0000 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:08.649 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:08.650 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:08.650 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:08.650 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:08.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 1 Processor -- start 2024-09-15T21:24:08.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 1 -- start start 2024-09-15T21:24:08.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient: init 2024-09-15T21:24:08.652 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01c8137bb0 0x7f01c8137f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient(hunting): picked mon.noname-a con 0x7f01c8137bb0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f01c81384d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01c80a5500 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c80a3e60 con 0x7f01c80a5500 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c80a1c70 con 0x7f01c81384d0 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f01c80a6c40 con 0x7f01c8137bb0 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01cf208640 10 monclient(hunting): authenticate will time out at 1367.445375s 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01c7fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f01c81384d0 0x7f01c80a4df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50622/0 (socket says 172.21.15.119:50622) 2024-09-15T21:24:08.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01c7fff640 1 -- 172.21.15.119:0/318021240 learned_addr learned my addr 172.21.15.119:0/318021240 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 1 --2- 172.21.15.119:0/318021240 >> v2:172.21.15.17:3300/0 conn(0x7f01c8137bb0 0x7f01c8137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): get_auth_request con 0x7f01c8137bb0 auth_method 0 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_done global_id 74349 payload 274 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 1 -- 172.21.15.119:0/318021240 >> v1:172.21.15.119:6789/0 conn(0x7f01c81384d0 legacy=0x7f01c80a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 1 -- 172.21.15.119:0/318021240 >> v1:172.21.15.17:6789/0 conn(0x7f01c80a5500 legacy=0x7f01c80a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient: _finish_hunting 0 2024-09-15T21:24:08.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 1 monclient: found mon.noname-a 2024-09-15T21:24:08.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:08.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.646+0000 7f01ccfa4640 1 -- 172.21.15.119:0/318021240 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c80a1990 con 0x7f01c8137bb0 2024-09-15T21:24:08.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 1 --2- 172.21.15.119:0/318021240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c8137bb0 0x7f01c8137f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f01c80a6dc0 tx=0x7f01b801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e5299ae6890d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:08.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 1 -- 172.21.15.119:0/318021240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80b0070 con 0x7f01c8137bb0 2024-09-15T21:24:08.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:08.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: dump: 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:08.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 1 -- 172.21.15.119:0/318021240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b80ab600 con 0x7f01c8137bb0 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 5 monclient: authenticate success, global_id 74349 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 4 set_mon_vals no callback set 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 1 -- 172.21.15.119:0/318021240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80ab9d0 con 0x7f01c8137bb0 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c77fe640 10 monclient: dump: 2024-09-15T21:24:08.662 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:08.664 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: shutdown 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- 172.21.15.119:0/318021240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c8137bb0 msgr2=0x7f01c8137f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 --2- 172.21.15.119:0/318021240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c8137bb0 0x7f01c8137f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f01c80a6dc0 tx=0x7f01b801c360 comp rx=0 tx=0).stop 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- 172.21.15.119:0/318021240 shutdown_connections 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 --2- 172.21.15.119:0/318021240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c8137bb0 0x7f01c8137f90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- 172.21.15.119:0/318021240 >> 172.21.15.119:0/318021240 conn(0x7f01c80a6ef0 msgr2=0x7f01c80a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- 172.21.15.119:0/318021240 shutdown_connections 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- 172.21.15.119:0/318021240 wait complete. 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: build_initial_monmap 2024-09-15T21:24:08.665 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: monmap: 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 Processor -- start 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- start start 2024-09-15T21:24:08.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: init 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c80a5500 0x7f01c8136540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): picked mon.noname-a con 0x7f01c80a5500 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01c8137bb0 0x7f01c8136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): picked mon.noname-c con 0x7f01c8137bb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): picked mon.noname-b con 0x7f01c81384d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80a1260 con 0x7f01c80a5500 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80a10e0 con 0x7f01c81384d0 2024-09-15T21:24:08.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:08.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01c80a62c0 con 0x7f01c8137bb0 2024-09-15T21:24:08.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:08.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cf208640 10 monclient(hunting): authenticate will time out at 1367.451229s 2024-09-15T21:24:08.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:08.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36198/0 (socket says 172.21.15.119:36198) 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 -- 172.21.15.119:0/4206943602 learned_addr learned my addr 172.21.15.119:0/4206943602 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 10 monclient(hunting): get_auth_request con 0x7f01c81384d0 auth_method 0 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 -- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 msgr2=0x7f01c813cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 -- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 msgr2=0x7f01c813cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01cd7a5640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c80a5500 0x7f01c8136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01c8137bb0 0x7f01c8136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:08.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): get_auth_request con 0x7f01c80a5500 auth_method 0 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): get_auth_request con 0x7f01c8137bb0 auth_method 0 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:08.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.650+0000 7f01c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 10 monclient(hunting): handle_auth_done global_id 74355 payload 824 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 -- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01c8137bb0 msgr2=0x7f01c8136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01c8137bb0 0x7f01c8136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 -- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 msgr2=0x7f01c813cce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 10 monclient: _finish_hunting 0 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 monclient: found mon.noname-a 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 -- 172.21.15.119:0/4206943602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c8140170 con 0x7f01c80a5500 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01ccfa4640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c80a5500 0x7f01c8136540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01b80ab000 tx=0x7f01b8019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80b8020 con 0x7f01c80a5500 2024-09-15T21:24:08.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: dump: 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:08.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:08.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:08.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b80b4610 con 0x7f01c80a5500 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 5 monclient: authenticate success, global_id 74355 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 monclient: _renew_subs 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80b0040 con 0x7f01c80a5500 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c8073a10 con 0x7f01c80a5500 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c6ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 10 monclient: dump: 2024-09-15T21:24:08.675 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 monclient: _renew_subs 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c8136fc0 con 0x7f01c80a5500 2024-09-15T21:24:08.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8137f90 con 0x7f01c80a5500 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f01b80bb020 con 0x7f01c80a5500 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01b403b9c0 0x7f01b403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 8231+0+0 (secure 0 0 0) 0x7f01b80eab90 con 0x7f01c80a5500 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c7fff640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01b403b9c0 0x7f01b403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c7fff640 10 monclient: get_auth_request con 0x7f01b403b9c0 auth_method 0 2024-09-15T21:24:08.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.654+0000 7f01c7fff640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01b403b9c0 0x7f01b403de80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f01c8137b30 tx=0x7f01bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:08.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.666+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f01b80c2030 con 0x7f01c80a5500 2024-09-15T21:24:08.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.666+0000 7f01c57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:08.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.666+0000 7f01c57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:08.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.842+0000 7f01cf208640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:08.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.842+0000 7f01cf208640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:08.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.842+0000 7f01cf208640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:08.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:08.842+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f01c806a880 con 0x7f01c80a5500 2024-09-15T21:24:09.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.406+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 added bucket h3 type host to location {rack=r1} v188) ==== 166+0+0 (secure 0 0 0) 0x7f01b8016070 con 0x7f01c80a5500 2024-09-15T21:24:09.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.406+0000 7f01c57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:09.407 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.406+0000 7f01c57fa640 10 monclient: _finish_command 2 = system:0 added bucket h3 type host to location {rack=r1} 2024-09-15T21:24:09.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01cf208640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01cf208640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01cf208640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f01c813ef80 con 0x7f01c80a5500 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01c57fa640 1 -- 172.21.15.119:0/4206943602 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h3' already exists v188) ==== 145+0+0 (secure 0 0 0) 0x7f01b80df070 con 0x7f01c80a5500 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01c57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.510+0000 7f01c57fa640 10 monclient: _finish_command 3 = system:0 bucket 'h3' already exists 2024-09-15T21:24:09.514 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h3' already exists 2024-09-15T21:24:09.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01b403b9c0 msgr2=0x7f01b403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:09.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01b403b9c0 0x7f01b403de80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f01c8137b30 tx=0x7f01bc009040 comp rx=0 tx=0).stop 2024-09-15T21:24:09.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 10 monclient: shutdown 2024-09-15T21:24:09.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:09.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c80a5500 msgr2=0x7f01c8136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c80a5500 0x7f01c8136540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01b80ab000 tx=0x7f01b8019100 comp rx=0 tx=0).stop 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 shutdown_connections 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f01b403b9c0 0x7f01b403de80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01c81384d0 0x7f01c813cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01c8137bb0 0x7f01c8136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 --2- 172.21.15.119:0/4206943602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c80a5500 0x7f01c8136540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 >> 172.21.15.119:0/4206943602 conn(0x7f01c80a6ef0 msgr2=0x7f01c80a0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 shutdown_connections 2024-09-15T21:24:09.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.514+0000 7f01cf208640 1 -- 172.21.15.119:0/4206943602 wait complete. 2024-09-15T21:24:09.532 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:09.532 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h4 host rack=r1 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient: get_monmap_and_config 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient: build_initial_monmap 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient: monmap: 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:09.652944+0000 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:09.652944+0000 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:09.653 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:09.654 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:09.654 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:09.654 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:09.654 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:09.654 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:09.654 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:09.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 Processor -- start 2024-09-15T21:24:09.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 -- start start 2024-09-15T21:24:09.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient: init 2024-09-15T21:24:09.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 0x7f7634137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): picked mon.noname-b con 0x7f76340a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f76340a41b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76340a4590 0x7f7634134a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): picked mon.noname-a con 0x7f76340a4590 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76340a19d0 con 0x7f76340a41b0 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f76340a4a40 con 0x7f76340a4590 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f76340a7ea0 con 0x7f76340a7ac0 2024-09-15T21:24:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7639110640 10 monclient(hunting): authenticate will time out at 1368.449539s 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 0x7f7634137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76340a4590 0x7f7634134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 0x7f7634137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:44986/0 (socket says 172.21.15.119:44986) 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 -- 172.21.15.119:0/3156940744 learned_addr learned my addr 172.21.15.119:0/3156940744 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): get_auth_request con 0x7f76340a7ac0 auth_method 0 2024-09-15T21:24:09.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): get_auth_request con 0x7f76340a4590 auth_method 0 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f763359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient(hunting): handle_auth_done global_id 44885 payload 274 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 -- 172.21.15.119:0/3156940744 >> v2:172.21.15.17:3300/0 conn(0x7f76340a4590 msgr2=0x7f7634134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 --2- 172.21.15.119:0/3156940744 >> v2:172.21.15.17:3300/0 conn(0x7f76340a4590 0x7f7634134a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7631d9b640 1 -- 172.21.15.119:0/3156940744 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068765548 0 0) 0x7f76340a19d0 con 0x7f76340a41b0 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 -- 172.21.15.119:0/3156940744 >> v1:172.21.15.17:6789/0 conn(0x7f76340a41b0 legacy=0x7f7634134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 monclient: found mon.noname-b 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7632d9d640 1 -- 172.21.15.119:0/3156940744 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76340a6960 con 0x7f76340a7ac0 2024-09-15T21:24:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.650+0000 7f7631d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 1 --2- 172.21.15.119:0/3156940744 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 0x7f7634137cd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f762c009e50 tx=0x7f762c01c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a639816f79f0e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 1 -- 172.21.15.119:0/3156940744 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c0a56c0 con 0x7f76340a7ac0 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: dump: 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.666 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: _finish_auth 0 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:09.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 5 monclient: authenticate success, global_id 44885 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 1 -- 172.21.15.119:0/3156940744 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f762c0a5860 con 0x7f76340a7ac0 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 1 -- 172.21.15.119:0/3156940744 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c0a5c30 con 0x7f76340a7ac0 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 4 set_mon_vals no callback set 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:09.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7631d9b640 10 monclient: dump: 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: shutdown 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:09.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- 172.21.15.119:0/3156940744 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 msgr2=0x7f7634137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 --2- 172.21.15.119:0/3156940744 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 0x7f7634137cd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f762c009e50 tx=0x7f762c01c8f0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- 172.21.15.119:0/3156940744 shutdown_connections 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 --2- 172.21.15.119:0/3156940744 >> v2:172.21.15.17:3300/0 conn(0x7f76340a4590 0x7f7634134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 --2- 172.21.15.119:0/3156940744 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a7ac0 0x7f7634137cd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- 172.21.15.119:0/3156940744 >> 172.21.15.119:0/3156940744 conn(0x7f76340a6fd0 msgr2=0x7f76340a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- 172.21.15.119:0/3156940744 shutdown_connections 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- 172.21.15.119:0/3156940744 wait complete. 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: build_initial_monmap 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: monmap: 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:09.670 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 Processor -- start 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- start start 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: init 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76340a41b0 0x7f7634130e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): picked mon.noname-a con 0x7f76340a41b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:09.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a4590 0x7f76341313c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): picked mon.noname-c con 0x7f76340a4590 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): picked mon.noname-b con 0x7f76340a7ac0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f76341350d0 con 0x7f76340a41b0 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7634134f50 con 0x7f76340a7ac0 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f76341423c0 con 0x7f76340a4590 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7639110640 10 monclient(hunting): authenticate will time out at 1368.455391s 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36204/0 (socket says 172.21.15.119:36204) 2024-09-15T21:24:09.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 -- 172.21.15.119:0/1999442050 learned_addr learned my addr 172.21.15.119:0/1999442050 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 10 monclient(hunting): get_auth_request con 0x7f76340a7ac0 auth_method 0 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76340a41b0 0x7f7634130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 -- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 msgr2=0x7f7634072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): get_auth_request con 0x7f76340a41b0 auth_method 0 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 -- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 msgr2=0x7f7634072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:09.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a4590 0x7f76341313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763359e640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): get_auth_request con 0x7f76340a4590 auth_method 0 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f7632d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.654+0000 7f763259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 10 monclient(hunting): handle_auth_done global_id 44888 payload 824 2024-09-15T21:24:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 -- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 msgr2=0x7f7634072280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 -- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76340a41b0 msgr2=0x7f7634130e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76340a41b0 0x7f7634130e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 monclient: found mon.noname-c 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:09.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:09.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 -- 172.21.15.119:0/1999442050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7634139f40 con 0x7f76340a4590 2024-09-15T21:24:09.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7632d9d640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76340a41b0 0x7f7634130e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:09.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f763259c640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a4590 0x7f76341313c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f761c00c440 tx=0x7f761c00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:09.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c017070 con 0x7f76340a4590 2024-09-15T21:24:09.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: dump: 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:09.677 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761c003600 con 0x7f76340a4590 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 5 monclient: authenticate success, global_id 44888 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 monclient: _renew_subs 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763413a220 con 0x7f76340a4590 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0039f0 con 0x7f76340a4590 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 10 monclient: dump: 2024-09-15T21:24:09.678 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7630d99640 4 set_mon_vals no callback set 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 monclient: _renew_subs 2024-09-15T21:24:09.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f763413a500 con 0x7f76340a4590 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7634077180 con 0x7f76340a4590 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f761c01a070 con 0x7f76340a4590 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f761403b9f0 0x7f761403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(188..188 src has 1..188) ==== 8441+0+0 (secure 0 0 0) 0x7f761c0501a0 con 0x7f76340a4590 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7632d9d640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f761403b9f0 0x7f761403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7632d9d640 10 monclient: get_auth_request con 0x7f761403b9f0 auth_method 0 2024-09-15T21:24:09.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.658+0000 7f7632d9d640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f761403b9f0 0x7f761403deb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f762c0ae000 tx=0x7f762c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:09.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.670+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f761c021020 con 0x7f76340a4590 2024-09-15T21:24:09.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.670+0000 7f762b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:09.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.670+0000 7f762b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:09.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.846+0000 7f7639110640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:09.850 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.846+0000 7f7639110640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:09.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.846+0000 7f7639110640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:09.851 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:09.846+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f763406a760 con 0x7f76340a4590 2024-09-15T21:24:10.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.422+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 added bucket h4 type host to location {rack=r1} v189) ==== 166+0+0 (secure 0 0 0) 0x7f761c01ad70 con 0x7f76340a4590 2024-09-15T21:24:10.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.422+0000 7f762b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:10.429 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.422+0000 7f762b7fe640 10 monclient: _finish_command 2 = system:0 added bucket h4 type host to location {rack=r1} 2024-09-15T21:24:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f7639110640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f7639110640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f7639110640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:10.534 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7634131900 con 0x7f76340a4590 2024-09-15T21:24:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f762b7fe640 1 -- 172.21.15.119:0/1999442050 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h4' already exists v189) ==== 145+0+0 (secure 0 0 0) 0x7f761c04cd50 con 0x7f76340a4590 2024-09-15T21:24:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f762b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.530+0000 7f762b7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h4' already exists 2024-09-15T21:24:10.536 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h4' already exists 2024-09-15T21:24:10.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f761403b9f0 msgr2=0x7f761403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f761403b9f0 0x7f761403deb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f762c0ae000 tx=0x7f762c0a7070 comp rx=0 tx=0).stop 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 10 monclient: shutdown 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a4590 msgr2=0x7f76341313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a4590 0x7f76341313c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f761c00c440 tx=0x7f761c00c900 comp rx=0 tx=0).stop 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 shutdown_connections 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f761403b9f0 0x7f761403deb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76340a7ac0 0x7f7634072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76340a4590 0x7f76341313c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 --2- 172.21.15.119:0/1999442050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76340a41b0 0x7f7634130e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 >> 172.21.15.119:0/1999442050 conn(0x7f76340a6fd0 msgr2=0x7f76340a2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 shutdown_connections 2024-09-15T21:24:10.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.534+0000 7f7639110640 1 -- 172.21.15.119:0/1999442050 wait complete. 2024-09-15T21:24:10.553 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:10.553 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h5 host rack=r1 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient: get_monmap_and_config 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient: build_initial_monmap 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient: monmap: 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:10.670827+0000 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:10.670827+0000 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.671 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:10.672 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:10.672 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:10.672 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:10.672 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:10.672 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 1 Processor -- start 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 1 -- start start 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient: init 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa65c135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 0x7fa65c13aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient(hunting): picked mon.noname-c con 0x7fa65c09fe70 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa65c13afb0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:10.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c0a0300 con 0x7fa65c135bf0 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c0a1d80 con 0x7fa65c13afb0 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa65c0a3600 con 0x7fa65c09fe70 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa662184640 10 monclient(hunting): authenticate will time out at 1369.467437s 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 0x7fa65c13aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa65bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa65c13afb0 0x7fa65c13d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50634/0 (socket says 172.21.15.119:50634) 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 0x7fa65c13aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36208/0 (socket says 172.21.15.119:36208) 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa65bfff640 1 -- 172.21.15.119:0/2441544942 learned_addr learned my addr 172.21.15.119:0/2441544942 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 10 monclient(hunting): get_auth_request con 0x7fa65c09fe70 auth_method 0 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 1 -- 172.21.15.119:0/2441544942 >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 msgr2=0x7fa65c13aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 1 -- 172.21.15.119:0/2441544942 >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 msgr2=0x7fa65c13aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 1 --2- 172.21.15.119:0/2441544942 >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 0x7fa65c13aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.666+0000 7fa653fff640 1 --2- 172.21.15.119:0/2441544942 >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 0x7fa65c13aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265889966 0 0) 0x7fa64c001120 con 0x7fa65c135bf0 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): _negotiate 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): my global_id is 74367 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa648003530 con 0x7fa65c135bf0 2024-09-15T21:24:10.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1918497459 0 0) 0x7fa648003530 con 0x7fa65c135bf0 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 msgr2=0x7fa65c13aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 --2- 172.21.15.119:0/2441544942 >> v2:172.21.15.119:3300/0 conn(0x7fa65c09fe70 0x7fa65c13aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 >> v1:172.21.15.119:6789/0 conn(0x7fa65c13afb0 legacy=0x7fa65c13d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c0a2060 con 0x7fa65c135bf0 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: _finish_auth 0 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa64c0018d0 con 0x7fa65c135bf0 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa662184640 5 monclient: authenticate success, global_id 74367 2024-09-15T21:24:10.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa662184640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: dump: 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:10.682 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 >> v1:172.21.15.17:6789/0 conn(0x7fa65c135bf0 legacy=0x7fa65c135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa648003ae0 0x7fa648003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): picked mon.a con 0x7fa648003ae0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa648004810 0x7fa648004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): picked mon.c con 0x7fa648004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa64800bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): picked mon.b con 0x7fa6480099c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:10.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c0a3600 con 0x7fa648003ae0 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa64800c3a0 con 0x7fa6480099c0 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa64800c5a0 con 0x7fa648004810 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa662184640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa64800bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): get_auth_request con 0x7fa6480099c0 auth_method 0 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65b7fe640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa648003ae0 0x7fa648003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65b7fe640 10 monclient(hunting): get_auth_request con 0x7fa648003ae0 auth_method 0 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa653fff640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa648004810 0x7fa648004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa653fff640 10 monclient(hunting): get_auth_request con 0x7fa648004810 auth_method 0 2024-09-15T21:24:10.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa653fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa653fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa653fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient(hunting): handle_auth_done global_id 74367 payload 293 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 -- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa648004810 msgr2=0x7fa648004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa648004810 0x7fa648004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 -- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa648003ae0 msgr2=0x7fa648003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa648003ae0 0x7fa648003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 monclient: found mon.b 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:10.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 -- 172.21.15.119:0/2441544942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa648003530 con 0x7fa6480099c0 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient: _finish_auth 0 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 -- 172.21.15.119:0/2441544942 reap_dead start 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65bfff640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa64800bdb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c190 tx=0x7fa65401cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdaf62a7b4319e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654001f30 con 0x7fa6480099c0 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: dump: 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:10.687 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6540a9070 con 0x7fa6480099c0 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa662184640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:10.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa662184640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa662184640 4 set_mon_vals no callback set 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 1 -- 172.21.15.119:0/2441544942 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540a5c60 con 0x7fa6480099c0 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.670+0000 7fa65affd640 10 monclient: dump: 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.689 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: shutdown 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2441544942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 msgr2=0x7fa64800bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa64800bdb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c190 tx=0x7fa65401cae0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2441544942 shutdown_connections 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 --2- 172.21.15.119:0/2441544942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa64800bdb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2441544942 >> 172.21.15.119:0/2441544942 conn(0x7fa65c0a66e0 msgr2=0x7fa65c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2441544942 shutdown_connections 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2441544942 wait complete. 2024-09-15T21:24:10.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: build_initial_monmap 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: monmap: 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 Processor -- start 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- start start 2024-09-15T21:24:10.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: init 2024-09-15T21:24:10.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:10.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:10.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): picked mon.noname-b con 0x7fa6480099c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c0705e0 0x7fa65c070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): picked mon.noname-c con 0x7fa65c0705e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1cd6f0 0x7fa65c1cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): picked mon.noname-a con 0x7fa65c1cd6f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c09ce50 con 0x7fa65c1cd6f0 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c09ccd0 con 0x7fa6480099c0 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c135fd0 con 0x7fa65c0705e0 2024-09-15T21:24:10.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient(hunting): authenticate will time out at 1369.473838s 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36222/0 (socket says 172.21.15.119:36222) 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 -- 172.21.15.119:0/2142137470 learned_addr learned my addr 172.21.15.119:0/2142137470 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 10 monclient(hunting): get_auth_request con 0x7fa6480099c0 auth_method 0 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 -- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 msgr2=0x7fa65c0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 -- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 msgr2=0x7fa65c0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:10.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65b7fe640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c0705e0 0x7fa65c070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1cd6f0 0x7fa65c1cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): get_auth_request con 0x7fa65c0705e0 auth_method 0 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): get_auth_request con 0x7fa65c1cd6f0 auth_method 0 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:10.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient(hunting): handle_auth_done global_id 44891 payload 824 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 -- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 msgr2=0x7fa65c0700a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 -- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1cd6f0 msgr2=0x7fa65c1cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1cd6f0 0x7fa65c1cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 monclient: found mon.noname-c 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:10.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 -- 172.21.15.119:0/2142137470 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c1370c0 con 0x7fa65c0705e0 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa653fff640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c0705e0 0x7fa65c070a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa644010860 tx=0x7fa644010d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64401a070 con 0x7fa65c0705e0 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: dump: 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.701 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 5 monclient: authenticate success, global_id 44891 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: _renew_subs 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c1373a0 con 0x7fa65c0705e0 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa644004220 con 0x7fa65c0705e0 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:10.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: _renew_subs 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa65a7fc640 4 set_mon_vals no callback set 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c0a5920 con 0x7fa65c0705e0 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64400f040 con 0x7fa65c0705e0 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.674+0000 7fa658ff9640 10 monclient: dump: 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:10.703 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa662184640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa662184640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa662184640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65c13ac50 con 0x7fa65c0705e0 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa64400c730 con 0x7fa65c0705e0 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa658ff9640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9c0 0x7fa63c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(189..189 src has 1..189) ==== 8651+0+0 (secure 0 0 0) 0x7fa6440519d0 con 0x7fa65c0705e0 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa65b7fe640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9c0 0x7fa63c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:10.704 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa65b7fe640 10 monclient: get_auth_request con 0x7fa63c03b9c0 auth_method 0 2024-09-15T21:24:10.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.678+0000 7fa65b7fe640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9c0 0x7fa63c03de80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000fc0 tx=0x7fa64c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:10.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.690+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa644014de0 con 0x7fa65c0705e0 2024-09-15T21:24:10.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.690+0000 7fa658ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:10.705 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.690+0000 7fa658ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:10.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.854+0000 7fa662184640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:10.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.854+0000 7fa662184640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:10.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.854+0000 7fa662184640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:10.860 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:10.854+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fa65c1363b0 con 0x7fa65c0705e0 2024-09-15T21:24:11.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:11.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient: get_monmap_and_config 2024-09-15T21:24:11.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient: build_initial_monmap 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient: monmap: 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:11.235729+0000 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:11.235729+0000 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:11.238 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:11.238 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 Processor -- start 2024-09-15T21:24:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 -- start start 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient: init 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcf5c1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf5c1383b0 0x7fcf5c13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): picked mon.noname-c con 0x7fcf5c1383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcf5c13d2c0 0x7fcf5c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): picked mon.noname-a con 0x7fcf5c13d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf5c0a1a20 con 0x7fcf5c1378c0 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcf5c0a1d00 con 0x7fcf5c13d2c0 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcf5c0a1fe0 con 0x7fcf5c1383b0 2024-09-15T21:24:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf60bcb640 10 monclient(hunting): authenticate will time out at 1374.631188s 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcf5c1378c0 0x7fcf5c137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45560/0 (socket says 172.21.15.17:45560) 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5a59c640 1 -- 172.21.15.17:0/3192817695 learned_addr learned my addr 172.21.15.17:0/3192817695 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 1 --2- 172.21.15.17:0/3192817695 >> v2:172.21.15.17:3300/0 conn(0x7fcf5c13d2c0 0x7fcf5c13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): get_auth_request con 0x7fcf5c13d2c0 auth_method 0 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf59d9b640 1 --2- 172.21.15.17:0/3192817695 >> v2:172.21.15.119:3300/0 conn(0x7fcf5c1383b0 0x7fcf5c13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf59d9b640 10 monclient(hunting): get_auth_request con 0x7fcf5c1383b0 auth_method 0 2024-09-15T21:24:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf59d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf59d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf59d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5959a640 1 -- 172.21.15.17:0/3192817695 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3227343093 0 0) 0x7fcf4c001120 con 0x7fcf5c1378c0 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5959a640 10 monclient(hunting): _negotiate 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5959a640 10 monclient(hunting): my global_id is 74376 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5959a640 1 -- 172.21.15.17:0/3192817695 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf44003530 con 0x7fcf5c1378c0 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient(hunting): handle_auth_done global_id 74373 payload 274 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 1 -- 172.21.15.17:0/3192817695 >> v2:172.21.15.119:3300/0 conn(0x7fcf5c1383b0 msgr2=0x7fcf5c13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 1 --2- 172.21.15.17:0/3192817695 >> v2:172.21.15.119:3300/0 conn(0x7fcf5c1383b0 0x7fcf5c13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 1 -- 172.21.15.17:0/3192817695 >> v1:172.21.15.17:6789/0 conn(0x7fcf5c1378c0 legacy=0x7fcf5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 1 monclient: found mon.noname-a 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.232+0000 7fcf5ad9d640 1 -- 172.21.15.17:0/3192817695 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c0a22c0 con 0x7fcf5c13d2c0 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5ad9d640 1 --2- 172.21.15.17:0/3192817695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c13f6b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009d90 tx=0x7fcf5401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af6364be8f54556 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 1 -- 172.21.15.17:0/3192817695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf540b1070 con 0x7fcf5c13d2c0 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: dump: 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: _finish_auth 0 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 1 -- 172.21.15.17:0/3192817695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf540a4d90 con 0x7fcf5c13d2c0 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 5 monclient: authenticate success, global_id 74373 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 1 -- 172.21.15.17:0/3192817695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf540ac660 con 0x7fcf5c13d2c0 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 4 set_mon_vals no callback set 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf5959a640 10 monclient: dump: 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.248 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 10 monclient: shutdown 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 -- 172.21.15.17:0/3192817695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c13d2c0 msgr2=0x7fcf5c13f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/3192817695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c13f6b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009d90 tx=0x7fcf5401c4d0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 -- 172.21.15.17:0/3192817695 shutdown_connections 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/3192817695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c13f6b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/3192817695 >> v2:172.21.15.119:3300/0 conn(0x7fcf5c1383b0 0x7fcf5c13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 -- 172.21.15.17:0/3192817695 >> 172.21.15.17:0/3192817695 conn(0x7fcf5c0aac40 msgr2=0x7fcf5c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 -- 172.21.15.17:0/3192817695 shutdown_connections 2024-09-15T21:24:11.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 -- 172.21.15.17:0/3192817695 wait complete. 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 10 monclient: build_initial_monmap 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 10 monclient: monmap: 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 Processor -- start 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 1 -- start start 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.236+0000 7fcf60bcb640 10 monclient: init 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf5c1378c0 0x7fcf5c1d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): picked mon.noname-c con 0x7fcf5c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 0x7fcf5c1d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): picked mon.noname-a con 0x7fcf5c1383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c1d1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): picked mon.noname-b con 0x7fcf5c13d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c143020 con 0x7fcf5c1383b0 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c142ea0 con 0x7fcf5c13d2c0 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5c1431a0 con 0x7fcf5c1378c0 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient(hunting): authenticate will time out at 1374.636217s 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 0x7fcf5c1d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf5c1378c0 0x7fcf5c1d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 0x7fcf5c1d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47600/0 (socket says 172.21.15.17:47600) 2024-09-15T21:24:11.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 -- 172.21.15.17:0/1255043367 learned_addr learned my addr 172.21.15.17:0/1255043367 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): get_auth_request con 0x7fcf5c1383b0 auth_method 0 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf5c1378c0 0x7fcf5c1d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37912/0 (socket says 172.21.15.17:37912) 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): get_auth_request con 0x7fcf5c1378c0 auth_method 0 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:11.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5ad9d640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c1d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5ad9d640 10 monclient(hunting): get_auth_request con 0x7fcf5c13d2c0 auth_method 0 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient(hunting): handle_auth_done global_id 74379 payload 824 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 -- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf5c1378c0 msgr2=0x7fcf5c1d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf5c1378c0 0x7fcf5c1d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 -- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf5c13d2c0 msgr2=0x7fcf5c1d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c1d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 monclient: found mon.noname-a 2024-09-15T21:24:11.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 -- 172.21.15.17:0/1255043367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c0a1320 con 0x7fcf5c1383b0 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf59d9b640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 0x7fcf5c1d0cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d800 tx=0x7fcf5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000b6c0 con 0x7fcf5c1383b0 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: dump: 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:11.258 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: _finish_auth 0 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50019070 con 0x7fcf5c1383b0 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 5 monclient: authenticate success, global_id 74379 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _renew_subs 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c13f6c0 con 0x7fcf5c1383b0 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000ba60 con 0x7fcf5c1383b0 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 10 monclient: dump: 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.260 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf58d99640 4 set_mon_vals no callback set 2024-09-15T21:24:11.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _renew_subs 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c09eac0 con 0x7fcf5c1383b0 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5c137ca0 con 0x7fcf5c1383b0 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcf5000bc00 con 0x7fcf5c1383b0 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf3c03b9f0 0x7fcf3c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 8651+0+0 (secure 0 0 0) 0x7fcf50050f20 con 0x7fcf5c1383b0 2024-09-15T21:24:11.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf3c03b9f0 0x7fcf3c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 10 monclient: get_auth_request con 0x7fcf3c03b9f0 auth_method 0 2024-09-15T21:24:11.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.240+0000 7fcf5a59c640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf3c03b9f0 0x7fcf3c03deb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005890 tx=0x7fcf4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:11.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.244+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcf5c0a1320 con 0x7fcf5c1383b0 2024-09-15T21:24:11.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.244+0000 7fcf4affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:11.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.244+0000 7fcf4affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:11.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.430+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 added bucket h5 type host to location {rack=r1} v190) ==== 166+0+0 (secure 0 0 0) 0x7fa64404e510 con 0x7fa65c0705e0 2024-09-15T21:24:11.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.430+0000 7fa658ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:11.435 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.430+0000 7fa658ff9640 10 monclient: _finish_command 2 = system:0 added bucket h5 type host to location {rack=r1} 2024-09-15T21:24:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcf5c09e840 con 0x7fcf5c1383b0 2024-09-15T21:24:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf4affd640 1 -- 172.21.15.17:0/1255043367 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fcf5004ea10 con 0x7fcf5c1383b0 2024-09-15T21:24:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf4affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf4affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:11.492 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":64,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf3c03b9f0 msgr2=0x7fcf3c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf3c03b9f0 0x7fcf3c03deb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005890 tx=0x7fcf4c009040 comp rx=0 tx=0).stop 2024-09-15T21:24:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 10 monclient: shutdown 2024-09-15T21:24:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 msgr2=0x7fcf5c1d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 0x7fcf5c1d0cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d800 tx=0x7fcf5000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 shutdown_connections 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf3c03b9f0 0x7fcf3c03deb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf5c13d2c0 0x7fcf5c1d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5c1383b0 0x7fcf5c1d0cd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 --2- 172.21.15.17:0/1255043367 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf5c1378c0 0x7fcf5c1d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 >> 172.21.15.17:0/1255043367 conn(0x7fcf5c0aac40 msgr2=0x7fcf5c136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 shutdown_connections 2024-09-15T21:24:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:11.488+0000 7fcf60bcb640 1 -- 172.21.15.17:0/1255043367 wait complete. 2024-09-15T21:24:11.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.530+0000 7fa662184640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:11.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.530+0000 7fa662184640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:11.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.530+0000 7fa662184640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.537 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.530+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fa65c06a830 con 0x7fa65c0705e0 2024-09-15T21:24:11.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.534+0000 7fa658ff9640 1 -- 172.21.15.119:0/2142137470 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h5' already exists v190) ==== 145+0+0 (secure 0 0 0) 0x7fa64404e890 con 0x7fa65c0705e0 2024-09-15T21:24:11.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.534+0000 7fa658ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:11.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.534+0000 7fa658ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h5' already exists 2024-09-15T21:24:11.538 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h5' already exists 2024-09-15T21:24:11.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9c0 msgr2=0x7fa63c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9c0 0x7fa63c03de80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000fc0 tx=0x7fa64c009040 comp rx=0 tx=0).stop 2024-09-15T21:24:11.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 10 monclient: shutdown 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c0705e0 msgr2=0x7fa65c070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c0705e0 0x7fa65c070a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa644010860 tx=0x7fa644010d20 comp rx=0 tx=0).stop 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 shutdown_connections 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9c0 0x7fa63c03de80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1cd6f0 0x7fa65c1cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c0705e0 0x7fa65c070a60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 --2- 172.21.15.119:0/2142137470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6480099c0 0x7fa65c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 >> 172.21.15.119:0/2142137470 conn(0x7fa65c0a66e0 msgr2=0x7fa65c0a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 shutdown_connections 2024-09-15T21:24:11.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.538+0000 7fa662184640 1 -- 172.21.15.119:0/2142137470 wait complete. 2024-09-15T21:24:11.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:11.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h6 host rack=r1 2024-09-15T21:24:11.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient: get_monmap_and_config 2024-09-15T21:24:11.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient: build_initial_monmap 2024-09-15T21:24:11.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient: monmap: 2024-09-15T21:24:11.677 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:11.677341+0000 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:11.677341+0000 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:11.678 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:11.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 1 Processor -- start 2024-09-15T21:24:11.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 1 -- start start 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient: init 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fefe0135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fefe009fe70 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe013e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient(hunting): picked mon.noname-b con 0x7fefe013c190 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe00a0300 con 0x7fefe0135bf0 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe00a1d80 con 0x7fefe009fe70 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefe00a3600 con 0x7fefe013c190 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:11.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe87aa640 10 monclient(hunting): authenticate will time out at 1370.474012s 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe5d45640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fefe009fe70 0x7fefe013aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50640/0 (socket says 172.21.15.119:50640) 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe5d45640 1 -- 172.21.15.119:0/4160684461 learned_addr learned my addr 172.21.15.119:0/4160684461 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 1 --2- 172.21.15.119:0/4160684461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe013e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): get_auth_request con 0x7fefe013c190 auth_method 0 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient(hunting): handle_auth_done global_id 44897 payload 274 2024-09-15T21:24:11.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe5544640 1 -- 172.21.15.119:0/4160684461 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932611143 0 0) 0x7fefdc001120 con 0x7fefe0135bf0 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 1 -- 172.21.15.119:0/4160684461 >> v1:172.21.15.119:6789/0 conn(0x7fefe009fe70 legacy=0x7fefe013aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 1 -- 172.21.15.119:0/4160684461 >> v1:172.21.15.17:6789/0 conn(0x7fefe0135bf0 legacy=0x7fefe0135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient: _finish_hunting 0 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 1 monclient: found mon.noname-b 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe6d47640 1 -- 172.21.15.119:0/4160684461 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe00a2060 con 0x7fefe013c190 2024-09-15T21:24:11.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.674+0000 7fefe5544640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:24:11.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe6d47640 1 --2- 172.21.15.119:0/4160684461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe013e580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009df0 tx=0x7fefd801c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbdd2128761926 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:11.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 1 -- 172.21.15.119:0/4160684461 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80b2070 con 0x7fefe013c190 2024-09-15T21:24:11.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: dump: 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: _finish_auth 0 2024-09-15T21:24:11.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: _check_auth_tickets 2024-09-15T21:24:11.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:11.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 1 -- 172.21.15.119:0/4160684461 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd80a5900 con 0x7fefe013c190 2024-09-15T21:24:11.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:11.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 5 monclient: authenticate success, global_id 44897 2024-09-15T21:24:11.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:11.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 4 set_mon_vals no callback set 2024-09-15T21:24:11.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 1 -- 172.21.15.119:0/4160684461 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80a5cd0 con 0x7fefe013c190 2024-09-15T21:24:11.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:11.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe5544640 10 monclient: dump: 2024-09-15T21:24:11.691 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:11.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 10 monclient: shutdown 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 -- 172.21.15.119:0/4160684461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 msgr2=0x7fefe013e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 --2- 172.21.15.119:0/4160684461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe013e580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009df0 tx=0x7fefd801c820 comp rx=0 tx=0).stop 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 -- 172.21.15.119:0/4160684461 shutdown_connections 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 --2- 172.21.15.119:0/4160684461 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe013e580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 -- 172.21.15.119:0/4160684461 >> 172.21.15.119:0/4160684461 conn(0x7fefe00a66e0 msgr2=0x7fefe00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 -- 172.21.15.119:0/4160684461 shutdown_connections 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 -- 172.21.15.119:0/4160684461 wait complete. 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 10 monclient: build_initial_monmap 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 10 monclient: monmap: 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:11.693 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 Processor -- start 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 1 -- start start 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.678+0000 7fefe87aa640 10 monclient: init 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:11.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): picked mon.noname-b con 0x7fefe009fe70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0135bf0 0x7fefe0134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): picked mon.noname-a con 0x7fefe0135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe0139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): picked mon.noname-c con 0x7fefe013c190 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01416f0 con 0x7fefe0135bf0 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006ece0 con 0x7fefe009fe70 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:11.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefe0141870 con 0x7fefe013c190 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient(hunting): authenticate will time out at 1370.480197s 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36226/0 (socket says 172.21.15.119:36226) 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 -- 172.21.15.119:0/3041987698 learned_addr learned my addr 172.21.15.119:0/3041987698 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 10 monclient(hunting): get_auth_request con 0x7fefe009fe70 auth_method 0 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 -- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 msgr2=0x7fefe01cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 -- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 msgr2=0x7fefe01cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe0139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0135bf0 0x7fefe0134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): get_auth_request con 0x7fefe013c190 auth_method 0 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): get_auth_request con 0x7fefe0135bf0 auth_method 0 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:11.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe5d45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient(hunting): handle_auth_done global_id 44900 payload 824 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 -- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 msgr2=0x7fefe01cda00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 -- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0135bf0 msgr2=0x7fefe0134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0135bf0 0x7fefe0134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient: _finish_hunting 0 2024-09-15T21:24:11.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 monclient: found mon.noname-c 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 -- 172.21.15.119:0/3041987698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe011e040 con 0x7fefe013c190 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6d47640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe0139880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fefd80ac000 tx=0x7fefd80195c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80b2040 con 0x7fefe013c190 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: dump: 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd8003ba0 con 0x7fefe013c190 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 5 monclient: authenticate success, global_id 44900 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _renew_subs 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe011e320 con 0x7fefe013c190 2024-09-15T21:24:11.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8004030 con 0x7fefe013c190 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 10 monclient: dump: 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:11.701 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _renew_subs 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd7fff640 4 set_mon_vals no callback set 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe013bc50 con 0x7fefe013c190 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe009d240 con 0x7fefe013c190 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fefd80041d0 con 0x7fefe013c190 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9f0 0x7fefc403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(190..190 src has 1..190) ==== 8989+0+0 (secure 0 0 0) 0x7fefd80ec640 con 0x7fefe013c190 2024-09-15T21:24:11.702 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9f0 0x7fefc403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:11.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.682+0000 7fefe6546640 10 monclient: get_auth_request con 0x7fefc403b9f0 auth_method 0 2024-09-15T21:24:11.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.686+0000 7fefe6546640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9f0 0x7fefc403deb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fefe013af70 tx=0x7fefdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:11.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.698+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fefd80bd020 con 0x7fefe013c190 2024-09-15T21:24:11.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.698+0000 7fefd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:11.703 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.698+0000 7fefd6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.862+0000 7fefe87aa640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.862+0000 7fefe87aa640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.862+0000 7fefe87aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:11.866 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:11.862+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fefe0135fd0 con 0x7fefe013c190 2024-09-15T21:24:12.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.246+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 added bucket h6 type host to location {rack=r1} v191) ==== 166+0+0 (secure 0 0 0) 0x7fefd8016560 con 0x7fefe013c190 2024-09-15T21:24:12.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.246+0000 7fefd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:12.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.246+0000 7fefd6ffd640 10 monclient: _finish_command 2 = system:0 added bucket h6 type host to location {rack=r1} 2024-09-15T21:24:12.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.354+0000 7fefe87aa640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.354+0000 7fefe87aa640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.354+0000 7fefe87aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:12.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.354+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fefe01ce880 con 0x7fefe013c190 2024-09-15T21:24:12.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefd6ffd640 1 -- 172.21.15.119:0/3041987698 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h6' already exists v191) ==== 145+0+0 (secure 0 0 0) 0x7fefd80ec400 con 0x7fefe013c190 2024-09-15T21:24:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefd6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefd6ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h6' already exists 2024-09-15T21:24:12.362 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h6' already exists 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9f0 msgr2=0x7fefc403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9f0 0x7fefc403deb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fefe013af70 tx=0x7fefdc009040 comp rx=0 tx=0).stop 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 10 monclient: shutdown 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 msgr2=0x7fefe0139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe0139880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fefd80ac000 tx=0x7fefd80195c0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 shutdown_connections 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9f0 0x7fefc403deb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefe013c190 0x7fefe0139880 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefe0135bf0 0x7fefe0134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 --2- 172.21.15.119:0/3041987698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe009fe70 0x7fefe01cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 >> 172.21.15.119:0/3041987698 conn(0x7fefe00a66e0 msgr2=0x7fefe013d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 shutdown_connections 2024-09-15T21:24:12.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.358+0000 7fefe87aa640 1 -- 172.21.15.119:0/3041987698 wait complete. 2024-09-15T21:24:12.379 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:12.380 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h7 host rack=r1 2024-09-15T21:24:12.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.494+0000 7fa75f236640 10 monclient: get_monmap_and_config 2024-09-15T21:24:12.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.494+0000 7fa75f236640 10 monclient: build_initial_monmap 2024-09-15T21:24:12.501 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.494+0000 7fa75f236640 10 monclient: monmap: 2024-09-15T21:24:12.501 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:12.501 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:12.501 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:12.500648+0000 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:12.500648+0000 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 Processor -- start 2024-09-15T21:24:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 -- start start 2024-09-15T21:24:12.503 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient: init 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa75809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): picked mon.noname-b con 0x7fa758135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa758135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): picked mon.noname-c con 0x7fa758136430 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7580a1d80 con 0x7fa758135fd0 2024-09-15T21:24:12.505 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa7580a3600 con 0x7fa758136430 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580a2060 con 0x7fa758135bf0 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75f236640 10 monclient(hunting): authenticate will time out at 1371.297320s 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59830/0 (socket says 172.21.15.119:59830) 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 -- 172.21.15.119:0/2816291590 learned_addr learned my addr 172.21.15.119:0/2816291590 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 10 monclient(hunting): get_auth_request con 0x7fa758136430 auth_method 0 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:12.506 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 -- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 msgr2=0x7fa7580a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 -- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 msgr2=0x7fa7580a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 --2- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75d7d3640 1 --2- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 1 --2- 172.21.15.119:0/2816291590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa75809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): get_auth_request con 0x7fa758135bf0 auth_method 0 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:12.507 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa7577fe640 1 -- 172.21.15.119:0/2816291590 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1071072076 0 0) 0x7fa7580a1d80 con 0x7fa758135fd0 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa7577fe640 10 monclient(hunting): _negotiate 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa7577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa7577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa7577fe640 10 monclient(hunting): my global_id is 74388 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa7577fe640 1 -- 172.21.15.119:0/2816291590 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa744003530 con 0x7fa758135fd0 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_done global_id 44903 payload 274 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 1 -- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 msgr2=0x7fa7580a1160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 1 --2- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 1 -- 172.21.15.119:0/2816291590 >> v1:172.21.15.17:6789/0 conn(0x7fa758135fd0 legacy=0x7fa7580a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient: _finish_hunting 0 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 1 monclient: found mon.noname-b 2024-09-15T21:24:12.508 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:12.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:12.509 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.498+0000 7fa75cfd2640 1 -- 172.21.15.119:0/2816291590 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580a2340 con 0x7fa758135bf0 2024-09-15T21:24:12.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 1 --2- 172.21.15.119:0/2816291590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa75809fe70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7400097c0 tx=0x7fa74001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608b4934aebab1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:12.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 1 -- 172.21.15.119:0/2816291590 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7400b0070 con 0x7fa758135bf0 2024-09-15T21:24:12.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:12.515 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: dump: 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:12.516 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 1 -- 172.21.15.119:0/2816291590 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7400ab600 con 0x7fa758135bf0 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 1 -- 172.21.15.119:0/2816291590 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7400ab9d0 con 0x7fa758135bf0 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 5 monclient: authenticate success, global_id 44903 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 4 set_mon_vals no callback set 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa7577fe640 10 monclient: dump: 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:12.517 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: shutdown 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- 172.21.15.119:0/2816291590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 msgr2=0x7fa75809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:12.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 --2- 172.21.15.119:0/2816291590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa75809fe70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7400097c0 tx=0x7fa74001c360 comp rx=0 tx=0).stop 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- 172.21.15.119:0/2816291590 shutdown_connections 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 --2- 172.21.15.119:0/2816291590 >> v2:172.21.15.119:3300/0 conn(0x7fa758136430 0x7fa7580a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 --2- 172.21.15.119:0/2816291590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa75809fe70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- 172.21.15.119:0/2816291590 >> 172.21.15.119:0/2816291590 conn(0x7fa7580a66e0 msgr2=0x7fa7580a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- 172.21.15.119:0/2816291590 shutdown_connections 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- 172.21.15.119:0/2816291590 wait complete. 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: build_initial_monmap 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: monmap: 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:12.519 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 Processor -- start 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- start start 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: init 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa7581c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): picked mon.noname-c con 0x7fa758135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:12.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): picked mon.noname-b con 0x7fa758135fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa758136430 0x7fa758134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): picked mon.noname-a con 0x7fa758136430 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580a1820 con 0x7fa758136430 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580a16a0 con 0x7fa758135fd0 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:12.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa7581402b0 con 0x7fa758135bf0 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75f236640 10 monclient(hunting): authenticate will time out at 1371.303218s 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59844/0 (socket says 172.21.15.119:59844) 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 -- 172.21.15.119:0/3604847458 learned_addr learned my addr 172.21.15.119:0/3604847458 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 10 monclient(hunting): get_auth_request con 0x7fa758135fd0 auth_method 0 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 -- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 msgr2=0x7fa75813c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 -- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 msgr2=0x7fa75813c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:12.522 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa757fff640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa758136430 0x7fa758134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa7581c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): get_auth_request con 0x7fa758136430 auth_method 0 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): get_auth_request con 0x7fa758135bf0 auth_method 0 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:12.523 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75d7d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.502+0000 7fa75cfd2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 10 monclient(hunting): handle_auth_done global_id 74391 payload 824 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 -- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 msgr2=0x7fa7581c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa7581c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 -- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 msgr2=0x7fa75813c770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 10 monclient: _finish_hunting 0 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 monclient: found mon.noname-a 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:12.524 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 -- 172.21.15.119:0/3604847458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa758072b80 con 0x7fa758136430 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75d7d3640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa758136430 0x7fa758134780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa750007a60 tx=0x7fa750007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75001a070 con 0x7fa758136430 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: dump: 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.525 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:12.526 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:12.526 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:12.526 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 5 monclient: authenticate success, global_id 74391 2024-09-15T21:24:12.527 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 monclient: _renew_subs 2024-09-15T21:24:12.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:12.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75809d240 con 0x7fa758136430 2024-09-15T21:24:12.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75000dbe0 con 0x7fa758136430 2024-09-15T21:24:12.528 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 monclient: _renew_subs 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa758134cc0 con 0x7fa758136430 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa756ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750024060 con 0x7fa758136430 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 10 monclient: dump: 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.529 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:12.530 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:12.530 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:12.530 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:12.530 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:12.530 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:12.530 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:12.531 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75809fe70 con 0x7fa758136430 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa75001d070 con 0x7fa758136430 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa73803b9f0 0x7fa73803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 9199+0+0 (secure 0 0 0) 0x7fa7500513a0 con 0x7fa758136430 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75cfd2640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa73803b9f0 0x7fa73803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75cfd2640 10 monclient: get_auth_request con 0x7fa73803b9f0 auth_method 0 2024-09-15T21:24:12.532 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.506+0000 7fa75cfd2640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa73803b9f0 0x7fa73803deb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa7400ad000 tx=0x7fa7400a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:12.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.518+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa750010070 con 0x7fa758136430 2024-09-15T21:24:12.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.518+0000 7fa7557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:12.533 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.518+0000 7fa7557fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:12.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.690+0000 7fa75f236640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:12.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.690+0000 7fa75f236640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:12.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.690+0000 7fa75f236640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:12.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:12.690+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fa75813d0e0 con 0x7fa758136430 2024-09-15T21:24:13.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.434+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 added bucket h7 type host to location {rack=r1} v192) ==== 166+0+0 (secure 0 0 0) 0x7fa75004dc60 con 0x7fa758136430 2024-09-15T21:24:13.439 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.434+0000 7fa7557fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:13.440 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.434+0000 7fa7557fa640 10 monclient: _finish_command 2 = system:0 added bucket h7 type host to location {rack=r1} 2024-09-15T21:24:13.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa75f236640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:13.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa75f236640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:13.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa75f236640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:13.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fa758135680 con 0x7fa758136430 2024-09-15T21:24:13.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa7557fa640 1 -- 172.21.15.119:0/3604847458 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h7' already exists v192) ==== 145+0+0 (secure 0 0 0) 0x7fa75004dfe0 con 0x7fa758136430 2024-09-15T21:24:13.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa7557fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:13.543 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.538+0000 7fa7557fa640 10 monclient: _finish_command 3 = system:0 bucket 'h7' already exists 2024-09-15T21:24:13.543 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h7' already exists 2024-09-15T21:24:13.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa73803b9f0 msgr2=0x7fa73803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa73803b9f0 0x7fa73803deb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa7400ad000 tx=0x7fa7400a6000 comp rx=0 tx=0).stop 2024-09-15T21:24:13.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 10 monclient: shutdown 2024-09-15T21:24:13.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa758136430 msgr2=0x7fa758134780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa758136430 0x7fa758134780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa750007a60 tx=0x7fa750007f20 comp rx=0 tx=0).stop 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 shutdown_connections 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa73803b9f0 0x7fa73803deb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa758136430 0x7fa758134780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa758135fd0 0x7fa75813c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 --2- 172.21.15.119:0/3604847458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa758135bf0 0x7fa7581c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 >> 172.21.15.119:0/3604847458 conn(0x7fa7580a66e0 msgr2=0x7fa75806f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 shutdown_connections 2024-09-15T21:24:13.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.542+0000 7fa75f236640 1 -- 172.21.15.119:0/3604847458 wait complete. 2024-09-15T21:24:13.561 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:13.561 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h8 host rack=r1 2024-09-15T21:24:13.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.662+0000 7f051752c640 10 monclient: get_monmap_and_config 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.662+0000 7f051752c640 10 monclient: build_initial_monmap 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.662+0000 7f051752c640 10 monclient: monmap: 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:13.668753+0000 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:13.668753+0000 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:13.669 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:13.670 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:13.670 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:13.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 Processor -- start 2024-09-15T21:24:13.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 -- start start 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient: init 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f051009ca70 0x7f05100a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): picked mon.noname-a con 0x7f051009ca70 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f051009fd80 0x7f0510133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): picked mon.noname-c con 0x7f051009fd80 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f05100a01f0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051009dfd0 con 0x7f05100a01f0 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f05100a3970 con 0x7f051009ca70 2024-09-15T21:24:13.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f051009ce50 con 0x7f051009fd80 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f051752c640 10 monclient(hunting): authenticate will time out at 1372.465333s 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f051009fd80 0x7f0510133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0515ac9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f05100a01f0 0x7f05100a3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55966/0 (socket says 172.21.15.119:55966) 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f051009fd80 0x7f0510133ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59856/0 (socket says 172.21.15.119:59856) 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0515ac9640 1 -- 172.21.15.119:0/2953196960 learned_addr learned my addr 172.21.15.119:0/2953196960 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): get_auth_request con 0x7f051009fd80 auth_method 0 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:13.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f05152c8640 1 --2- 172.21.15.119:0/2953196960 >> v2:172.21.15.17:3300/0 conn(0x7f051009ca70 0x7f05100a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f05152c8640 10 monclient(hunting): get_auth_request con 0x7f051009ca70 auth_method 0 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f05152c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f05152c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f05152c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient(hunting): handle_auth_done global_id 84100 payload 274 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 -- 172.21.15.119:0/2953196960 >> v2:172.21.15.17:3300/0 conn(0x7f051009ca70 msgr2=0x7f05100a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 --2- 172.21.15.119:0/2953196960 >> v2:172.21.15.17:3300/0 conn(0x7f051009ca70 0x7f05100a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 -- 172.21.15.119:0/2953196960 >> v1:172.21.15.119:6789/0 conn(0x7f05100a01f0 legacy=0x7f05100a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient: _finish_hunting 0 2024-09-15T21:24:13.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 monclient: found mon.noname-c 2024-09-15T21:24:13.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:13.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f0514ac7640 1 -- 172.21.15.119:0/2953196960 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051009e2b0 con 0x7f051009fd80 2024-09-15T21:24:13.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.666+0000 7f05152c8640 1 --2- 172.21.15.119:0/2953196960 >> v2:172.21.15.17:3300/0 conn(0x7f051009ca70 0x7f05100a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0514ac7640 1 --2- 172.21.15.119:0/2953196960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f051009fd80 0x7f0510133ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f051006a380 tx=0x7f050001c080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4e11be5fa7fb1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 1 -- 172.21.15.119:0/2953196960 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000b0070 con 0x7f051009fd80 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: dump: 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:13.683 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: _finish_auth 0 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 5 monclient: authenticate success, global_id 84100 2024-09-15T21:24:13.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 1 -- 172.21.15.119:0/2953196960 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05000ab600 con 0x7f051009fd80 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 4 set_mon_vals no callback set 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 1 -- 172.21.15.119:0/2953196960 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000ab9d0 con 0x7f051009fd80 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f0507fff640 10 monclient: dump: 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.685 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: shutdown 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- 172.21.15.119:0/2953196960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f051009fd80 msgr2=0x7f0510133ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 --2- 172.21.15.119:0/2953196960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f051009fd80 0x7f0510133ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f051006a380 tx=0x7f050001c080 comp rx=0 tx=0).stop 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- 172.21.15.119:0/2953196960 shutdown_connections 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 --2- 172.21.15.119:0/2953196960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f051009fd80 0x7f0510133ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 --2- 172.21.15.119:0/2953196960 >> v2:172.21.15.17:3300/0 conn(0x7f051009ca70 0x7f05100a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- 172.21.15.119:0/2953196960 >> 172.21.15.119:0/2953196960 conn(0x7f05100a66e0 msgr2=0x7f05100a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:13.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- 172.21.15.119:0/2953196960 shutdown_connections 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- 172.21.15.119:0/2953196960 wait complete. 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: build_initial_monmap 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: monmap: 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:13.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 Processor -- start 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- start start 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: init 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f05101cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient(hunting): picked mon.noname-c con 0x7f051009ca70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f0510130a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient(hunting): picked mon.noname-a con 0x7f051009fd80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:13.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient(hunting): picked mon.noname-b con 0x7f05100a01f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.670+0000 7f051752c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f051013f320 con 0x7f051009fd80 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f051013f1a0 con 0x7f05100a01f0 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f051013f4a0 con 0x7f051009ca70 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient(hunting): authenticate will time out at 1372.472018s 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59866/0 (socket says 172.21.15.119:59866) 2024-09-15T21:24:13.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 -- 172.21.15.119:0/3123461468 learned_addr learned my addr 172.21.15.119:0/3123461468 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): get_auth_request con 0x7f05100a01f0 auth_method 0 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f05101cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f0510130a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): get_auth_request con 0x7f051009ca70 auth_method 0 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:13.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient(hunting): handle_auth_done global_id 84106 payload 824 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 msgr2=0x7f05101cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f05101cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 msgr2=0x7f0510130a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f0510130a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient: _finish_hunting 0 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 monclient: found mon.noname-b 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0510141260 con 0x7f05100a01f0 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 10 monclient: get_auth_request con 0x7f051009fd80 auth_method 0 2024-09-15T21:24:13.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f0510130a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f050800cbf0 tx=0x7f050800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 msgr2=0x7f0510130f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 msgr2=0x7f0510130f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f050800cbf0 tx=0x7f050800a060 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0515ac9640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f050800cbf0 tx=0x7f050800a060 comp rx=0 tx=0).stop 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 msgr2=0x7f0510130f90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05100a01f0 0x7f0510130f90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:13.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04f40034b0 0x7f04f4005900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f04f40034b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f04f4008470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f051009ca70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f04f400af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f051009fd80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f04f400b8a0 con 0x7f051009fd80 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04f400ba20 con 0x7f04f40034b0 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f04f400bbf0 con 0x7f051009ca70 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:13.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04f40034b0 0x7f04f4005900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): get_auth_request con 0x7f04f40034b0 auth_method 0 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient(hunting): handle_auth_done global_id 84106 payload 293 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f04f4008470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 msgr2=0x7f04f4008470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f04f4008470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 msgr2=0x7f04f400af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:13.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f04f400af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient: _finish_hunting 0 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 monclient: found mon.noname-b 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f400beb0 con 0x7f04f40034b0 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05152c8640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04f40034b0 0x7f04f4005900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f050c00da90 tx=0x7f0508001aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c019020 con 0x7f04f40034b0 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: dump: 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.695 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 5 monclient: authenticate success, global_id 84106 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient: _renew_subs 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050c00fb40 con 0x7f04f40034b0 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.696 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0510141540 con 0x7f04f40034b0 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c00d660 con 0x7f04f40034b0 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 10 monclient: dump: 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:13.697 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient: _renew_subs 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f05077fe640 4 set_mon_vals no callback set 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0510140f80 con 0x7f04f40034b0 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05100a0f60 con 0x7f04f40034b0 2024-09-15T21:24:13.698 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f050c01c020 con 0x7f04f40034b0 2024-09-15T21:24:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05100a01f0 0x7f04f40460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 9409+0+0 (secure 0 0 0) 0x7f050c0506d0 con 0x7f04f40034b0 2024-09-15T21:24:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05100a01f0 0x7f04f40460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 10 monclient: get_auth_request con 0x7f05100a01f0 auth_method 0 2024-09-15T21:24:13.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.674+0000 7f0514ac7640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05100a01f0 0x7f04f40460d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f051006a380 tx=0x7f0500008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.678+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f050c023020 con 0x7f04f40034b0 2024-09-15T21:24:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.678+0000 7f0505ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:13.700 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.678+0000 7f0505ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:13.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.838+0000 7f051752c640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:13.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.838+0000 7f051752c640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:13.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.838+0000 7f051752c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:13.844 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:13.838+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f05100a4240 con 0x7f04f40034b0 2024-09-15T21:24:14.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:14.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient: get_monmap_and_config 2024-09-15T21:24:14.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient: build_initial_monmap 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient: monmap: 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:14.643717+0000 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:14.643717+0000 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:14.645 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:14.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 Processor -- start 2024-09-15T21:24:14.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 -- start start 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient: init 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ba41378c0 0x7f5ba4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): picked mon.noname-a con 0x7f5ba41378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 0x7f5ba40a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): picked mon.noname-c con 0x7f5ba41381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba40a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): picked mon.noname-b con 0x7f5ba40a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba40a1d00 con 0x7f5ba41378c0 2024-09-15T21:24:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba40a1fe0 con 0x7f5ba41381e0 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40a22c0 con 0x7f5ba40a5040 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5baa5db640 10 monclient(hunting): authenticate will time out at 1378.039392s 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ba41378c0 0x7f5ba4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ba41378c0 0x7f5ba4137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47606/0 (socket says 172.21.15.17:47606) 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 1 -- 172.21.15.17:0/4206720008 learned_addr learned my addr 172.21.15.17:0/4206720008 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): get_auth_request con 0x7f5ba41378c0 auth_method 0 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 1 --2- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba40a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): get_auth_request con 0x7f5ba40a5040 auth_method 0 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba8b78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 0x7f5ba40a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba37fe640 10 monclient(hunting): get_auth_request con 0x7f5ba41381e0 auth_method 0 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.639+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_done global_id 74397 payload 274 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 -- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 msgr2=0x7f5ba40a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 --2- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba40a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba37fe640 1 -- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 msgr2=0x7f5ba40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba37fe640 1 -- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 msgr2=0x7f5ba40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 -- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 msgr2=0x7f5ba40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 0x7f5ba40a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 0x7f5ba40a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 --2- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 0x7f5ba40a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 monclient: found mon.noname-a 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:14.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:14.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 -- 172.21.15.17:0/4206720008 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba40a4490 con 0x7f5ba41378c0 2024-09-15T21:24:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba3fff640 1 --2- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41378c0 0x7f5ba4137ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009870 tx=0x7f5b8c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de1a646db522ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 1 -- 172.21.15.17:0/4206720008 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0b1070 con 0x7f5ba41378c0 2024-09-15T21:24:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: dump: 2024-09-15T21:24:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:14.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 5 monclient: authenticate success, global_id 74397 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 1 -- 172.21.15.17:0/4206720008 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c0ac620 con 0x7f5ba41378c0 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 4 set_mon_vals no callback set 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 1 -- 172.21.15.17:0/4206720008 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0ac9f0 con 0x7f5ba41378c0 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5ba2ffd640 10 monclient: dump: 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.661 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 10 monclient: shutdown 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 -- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41378c0 msgr2=0x7f5ba4137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 --2- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41378c0 0x7f5ba4137ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009870 tx=0x7f5b8c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 -- 172.21.15.17:0/4206720008 shutdown_connections 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 --2- 172.21.15.17:0/4206720008 >> v2:172.21.15.119:3300/0 conn(0x7f5ba41381e0 0x7f5ba40a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 --2- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba40a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 --2- 172.21.15.17:0/4206720008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41378c0 0x7f5ba4137ca0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.643+0000 7f5baa5db640 1 -- 172.21.15.17:0/4206720008 >> 172.21.15.17:0/4206720008 conn(0x7f5ba40aac40 msgr2=0x7f5ba40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- 172.21.15.17:0/4206720008 shutdown_connections 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- 172.21.15.17:0/4206720008 wait complete. 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: build_initial_monmap 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: monmap: 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:14.663 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 Processor -- start 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- start start 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: init 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba41cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): picked mon.noname-c con 0x7f5ba40a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba41378c0 0x7f5ba4136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): picked mon.noname-b con 0x7f5ba41378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41381e0 0x7f5ba413cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): picked mon.noname-a con 0x7f5ba41381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4141730 con 0x7f5ba41381e0 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41415b0 con 0x7f5ba41378c0 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba41418b0 con 0x7f5ba40a5040 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient(hunting): authenticate will time out at 1378.045613s 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba41cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41381e0 0x7f5ba413cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba41cfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37940/0 (socket says 172.21.15.17:37940) 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 -- 172.21.15.17:0/790743843 learned_addr learned my addr 172.21.15.17:0/790743843 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): get_auth_request con 0x7f5ba40a5040 auth_method 0 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): get_auth_request con 0x7f5ba41381e0 auth_method 0 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient(hunting): handle_auth_done global_id 44915 payload 824 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 -- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba41378c0 msgr2=0x7f5ba4136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba41378c0 0x7f5ba4136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba41378c0 0x7f5ba4136ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 -- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41381e0 msgr2=0x7f5ba413cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41381e0 0x7f5ba413cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 monclient: found mon.noname-c 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 -- 172.21.15.17:0/790743843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41223a0 con 0x7f5ba40a5040 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba37fe640 10 monclient: get_auth_request con 0x7f5ba41378c0 auth_method 0 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba8b78640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41381e0 0x7f5ba413cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba41378c0 0x7f5ba4136ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba3fff640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba41cfd00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0af040 tx=0x7f5b8c019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0b9070 con 0x7f5ba40a5040 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: dump: 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.668 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 5 monclient: authenticate success, global_id 44915 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c0b5410 con 0x7f5ba40a5040 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _renew_subs 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:14.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba40a06b0 con 0x7f5ba40a5040 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0b1040 con 0x7f5ba40a5040 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 10 monclient: dump: 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:14.670 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba27fc640 4 set_mon_vals no callback set 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _renew_subs 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba4120e80 con 0x7f5ba40a5040 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba41210b0 con 0x7f5ba40a5040 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5b8c0bc070 con 0x7f5ba40a5040 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8403b9f0 0x7f5b8403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:14.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.647+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(192..192 src has 1..192) ==== 9409+0+0 (secure 0 0 0) 0x7f5b8c0eac40 con 0x7f5ba40a5040 2024-09-15T21:24:14.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.651+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8403b9f0 0x7f5b8403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:14.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.651+0000 7f5ba37fe640 10 monclient: get_auth_request con 0x7f5b8403b9f0 auth_method 0 2024-09-15T21:24:14.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.651+0000 7f5ba37fe640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8403b9f0 0x7f5b8403deb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4121320 tx=0x7f5b9400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:14.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.655+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5ba41210b0 con 0x7f5ba40a5040 2024-09-15T21:24:14.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.655+0000 7f5ba0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:14.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.655+0000 7f5ba0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.790+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 added bucket h8 type host to location {rack=r1} v193) ==== 166+0+0 (secure 0 0 0) 0x7f050c00d420 con 0x7f04f40034b0 2024-09-15T21:24:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.790+0000 7f0505ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:14.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.790+0000 7f0505ffb640 10 monclient: _finish_command 2 = system:0 added bucket h8 type host to location {rack=r1} 2024-09-15T21:24:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.890+0000 7f051752c640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.890+0000 7f051752c640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.890+0000 7f051752c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:14.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.890+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f051006a830 con 0x7f04f40034b0 2024-09-15T21:24:14.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5baa5db640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:14.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5baa5db640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:14.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5baa5db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:14.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5ba4137ca0 con 0x7f5ba40a5040 2024-09-15T21:24:14.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5ba0ff9640 1 -- 172.21.15.17:0/790743843 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f5b8c0160f0 con 0x7f5ba40a5040 2024-09-15T21:24:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5ba0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.891+0000 7f5ba0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:14.897 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:14.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f0505ffb640 1 -- 172.21.15.119:0/3123461468 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h8' already exists v193) ==== 145+0+0 (secure 0 0 0) 0x7f050c012d70 con 0x7f04f40034b0 2024-09-15T21:24:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f0505ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f0505ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h8' already exists 2024-09-15T21:24:14.898 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h8' already exists 2024-09-15T21:24:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8403b9f0 msgr2=0x7f5b8403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8403b9f0 0x7f5b8403deb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4121320 tx=0x7f5b9400b040 comp rx=0 tx=0).stop 2024-09-15T21:24:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 10 monclient: shutdown 2024-09-15T21:24:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 msgr2=0x7f5ba41cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba41cfd00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0af040 tx=0x7f5b8c019180 comp rx=0 tx=0).stop 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 shutdown_connections 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5b8403b9f0 0x7f5b8403deb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ba41381e0 0x7f5ba413cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba41378c0 0x7f5ba4136ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 --2- 172.21.15.17:0/790743843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ba40a5040 0x7f5ba41cfd00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 >> 172.21.15.17:0/790743843 conn(0x7f5ba40aac40 msgr2=0x7f5ba40a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 shutdown_connections 2024-09-15T21:24:14.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:14.895+0000 7f5baa5db640 1 -- 172.21.15.17:0/790743843 wait complete. 2024-09-15T21:24:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05100a01f0 msgr2=0x7f04f40460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05100a01f0 0x7f04f40460d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f051006a380 tx=0x7f0500008040 comp rx=0 tx=0).stop 2024-09-15T21:24:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 10 monclient: shutdown 2024-09-15T21:24:14.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04f40034b0 msgr2=0x7f04f4005900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04f40034b0 0x7f04f4005900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f050c00da90 tx=0x7f0508001aa0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 shutdown_connections 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05100a01f0 0x7f04f40460d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f051009fd80 0x7f04f400af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f051009ca70 0x7f04f4008470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 --2- 172.21.15.119:0/3123461468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04f40034b0 0x7f04f4005900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 >> 172.21.15.119:0/3123461468 conn(0x7f05100a66e0 msgr2=0x7f0510132bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 shutdown_connections 2024-09-15T21:24:14.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:14.894+0000 7f051752c640 1 -- 172.21.15.119:0/3123461468 wait complete. 2024-09-15T21:24:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:14.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h9 host rack=r1 2024-09-15T21:24:14.915 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:24:14.915 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:24:14.915 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:24:15.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient: get_monmap_and_config 2024-09-15T21:24:15.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient: monmap: 2024-09-15T21:24:15.033 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:15.033431+0000 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:15.033431+0000 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:15.034 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.027+0000 7fd166595640 10 monclient: get_monmap_and_config 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.027+0000 7fd166595640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient: monmap: 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:15.034206+0000 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:15.034206+0000 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.036 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 Processor -- start 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 -- start start 2024-09-15T21:24:15.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient: init 2024-09-15T21:24:15.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 Processor -- start 2024-09-15T21:24:15.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 -- start start 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient: init 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6da00a7ac0 0x7f6da0137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): picked mon.noname-a con 0x7f6da00a7ac0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a41b0 0x7f6da00a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): picked mon.noname-b con 0x7f6da00a41b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6da0134300 0x7f6da01347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): picked mon.noname-c con 0x7f6da0134300 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f6da00a7ea0 con 0x7f6da00a7ac0 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6da00a19d0 con 0x7f6da0134300 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6da00a6960 con 0x7f6da00a41b0 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da6bbb640 10 monclient(hunting): authenticate will time out at 1373.830663s 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6da0134300 0x7f6da01347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6da0134300 0x7f6da01347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59880/0 (socket says 172.21.15.119:59880) 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 -- 172.21.15.119:0/2570999192 learned_addr learned my addr 172.21.15.119:0/2570999192 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): get_auth_request con 0x7f6da0134300 auth_method 0 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 1 --2- 172.21.15.119:0/2570999192 >> v2:172.21.15.17:3300/0 conn(0x7f6da00a7ac0 0x7f6da0137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6d97fff640 1 --2- 172.21.15.119:0/2570999192 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a41b0 0x7f6da00a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): get_auth_request con 0x7f6da00a7ac0 auth_method 0 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6d97fff640 10 monclient(hunting): get_auth_request con 0x7f6da00a41b0 auth_method 0 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6d97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6d97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6d97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da4957640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient(hunting): handle_auth_done global_id 84109 payload 274 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 -- 172.21.15.119:0/2570999192 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a41b0 msgr2=0x7f6da00a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 --2- 172.21.15.119:0/2570999192 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a41b0 0x7f6da00a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 -- 172.21.15.119:0/2570999192 >> v2:172.21.15.17:3300/0 conn(0x7f6da00a7ac0 msgr2=0x7f6da0137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 --2- 172.21.15.119:0/2570999192 >> v2:172.21.15.17:3300/0 conn(0x7f6da00a7ac0 0x7f6da0137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 monclient: found mon.noname-c 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.030+0000 7f6da5158640 1 -- 172.21.15.119:0/2570999192 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da00a6c40 con 0x7f6da0134300 2024-09-15T21:24:15.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da5158640 1 --2- 172.21.15.119:0/2570999192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da01347b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6da00a8360 tx=0x7f6d9801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5829eb6cefd53de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 1 -- 172.21.15.119:0/2570999192 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980a56d0 con 0x7f6da0134300 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: dump: 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.051 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 5 monclient: authenticate success, global_id 84109 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 1 -- 172.21.15.119:0/2570999192 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d980a5870 con 0x7f6da0134300 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 1 -- 172.21.15.119:0/2570999192 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980a5c40 con 0x7f6da0134300 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6d977fe640 10 monclient: dump: 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 4 set_mon_vals no callback set 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: shutdown 2024-09-15T21:24:15.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2570999192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 msgr2=0x7f6da01347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2570999192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da01347b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6da00a8360 tx=0x7f6d9801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2570999192 shutdown_connections 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2570999192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da01347b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2570999192 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a41b0 0x7f6da00a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2570999192 >> v2:172.21.15.17:3300/0 conn(0x7f6da00a7ac0 0x7f6da0137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2570999192 >> 172.21.15.119:0/2570999192 conn(0x7f6da00a6fd0 msgr2=0x7f6da00a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2570999192 shutdown_connections 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2570999192 wait complete. 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: monmap: 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:15.054 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 Processor -- start 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- start start 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: init 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 0x7f6da0130f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): picked mon.noname-a con 0x7f6da00a41b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a7ac0 0x7f6da01314c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): picked mon.noname-c con 0x7f6da00a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da00722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): picked mon.noname-b con 0x7f6da0134300 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01382a0 con 0x7f6da00a41b0 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01413b0 con 0x7f6da0134300 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6da00a16c0 con 0x7f6da00a7ac0 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da6bbb640 10 monclient(hunting): authenticate will time out at 1373.835884s 2024-09-15T21:24:15.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da4957640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 0x7f6da0130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.034+0000 7f6da5158640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da00722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 0x7f6da0130f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:58384/0 (socket says 172.21.15.119:58384) 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da00722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59892/0 (socket says 172.21.15.119:59892) 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 1 -- 172.21.15.119:0/2745078684 learned_addr learned my addr 172.21.15.119:0/2745078684 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): get_auth_request con 0x7f6da00a41b0 auth_method 0 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): get_auth_request con 0x7f6da0134300 auth_method 0 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a7ac0 0x7f6da01314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): get_auth_request con 0x7f6da00a7ac0 auth_method 0 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient(hunting): handle_auth_done global_id 84115 payload 824 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 -- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a7ac0 msgr2=0x7f6da01314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a7ac0 0x7f6da01314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 -- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 msgr2=0x7f6da0130f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 0x7f6da0130f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 monclient: found mon.noname-b 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 -- 172.21.15.119:0/2745078684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da013a5d0 con 0x7f6da0134300 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 0x7f6da0130f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da5158640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da00722a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6d980ab000 tx=0x7f6d98019650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980bc030 con 0x7f6da0134300 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: dump: 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.060 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 5 monclient: authenticate success, global_id 84115 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d980b8470 con 0x7f6da0134300 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98009510 con 0x7f6da0134300 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 10 monclient: dump: 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.061 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d96ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 monclient: _renew_subs 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da0073d60 con 0x7f6da0134300 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 monclient: _renew_subs 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da0138dd0 con 0x7f6da0134300 2024-09-15T21:24:15.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da0077180 con 0x7f6da0134300 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6d980bf070 con 0x7f6da0134300 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d8c03b9c0 0x7f6d8c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 9619+0+0 (secure 0 0 0) 0x7f6d980ebaa0 con 0x7f6da0134300 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d8c03b9c0 0x7f6d8c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 10 monclient: get_auth_request con 0x7f6d8c03b9c0 auth_method 0 2024-09-15T21:24:15.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.038+0000 7f6da4957640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d8c03b9c0 0x7f6d8c03de80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009f90 tx=0x7f6d9c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.042+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6d980c6070 con 0x7f6da0134300 2024-09-15T21:24:15.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.042+0000 7f6d957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.042+0000 7f6d957fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): picked mon.noname-b con 0x7fd1601378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1601381e0 0x7fd1600a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): picked mon.noname-c con 0x7fd1601381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd1600a5040 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600a1d00 con 0x7fd1600a5040 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd1600a1fe0 con 0x7fd1601381e0 2024-09-15T21:24:15.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd1600a22c0 con 0x7fd1601378c0 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd166595640 10 monclient(hunting): authenticate will time out at 1378.429893s 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd164b32640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1600a5040 0x7fd1600a54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45568/0 (socket says 172.21.15.17:45568) 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd164b32640 1 -- 172.21.15.17:0/3370131338 learned_addr learned my addr 172.21.15.17:0/3370131338 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): get_auth_request con 0x7fd1601378c0 auth_method 0 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 1 --2- 172.21.15.17:0/3370131338 >> v2:172.21.15.119:3300/0 conn(0x7fd1601381e0 0x7fd1600a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): get_auth_request con 0x7fd1601381e0 auth_method 0 2024-09-15T21:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient(hunting): handle_auth_done global_id 44921 payload 274 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 1 -- 172.21.15.17:0/3370131338 >> v2:172.21.15.119:3300/0 conn(0x7fd1601381e0 msgr2=0x7fd1600a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 1 --2- 172.21.15.17:0/3370131338 >> v2:172.21.15.119:3300/0 conn(0x7fd1601381e0 0x7fd1600a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 1 -- 172.21.15.17:0/3370131338 >> v1:172.21.15.17:6789/0 conn(0x7fd1600a5040 legacy=0x7fd1600a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 1 monclient: found mon.noname-b 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15ffff640 1 -- 172.21.15.17:0/3370131338 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1600a5c00 con 0x7fd1601378c0 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15effd640 1 -- 172.21.15.17:0/3370131338 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608283671 0 0) 0x7fd1600a1d00 con 0x7fd1600a5040 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.031+0000 7fd15effd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 --2- 172.21.15.17:0/3370131338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160137ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1500097c0 tx=0x7fd15001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255ff1d7e67dd86d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 1 -- 172.21.15.17:0/3370131338 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500b0070 con 0x7fd1601378c0 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: dump: 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 1 -- 172.21.15.17:0/3370131338 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1500ab600 con 0x7fd1601378c0 2024-09-15T21:24:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 1 -- 172.21.15.17:0/3370131338 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500ab9d0 con 0x7fd1601378c0 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 5 monclient: authenticate success, global_id 44921 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 4 set_mon_vals no callback set 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15effd640 10 monclient: dump: 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: shutdown 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/3370131338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 msgr2=0x7fd160137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 --2- 172.21.15.17:0/3370131338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160137ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1500097c0 tx=0x7fd15001c3f0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/3370131338 shutdown_connections 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 --2- 172.21.15.17:0/3370131338 >> v2:172.21.15.119:3300/0 conn(0x7fd1601381e0 0x7fd1600a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 --2- 172.21.15.17:0/3370131338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160137ca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/3370131338 >> 172.21.15.17:0/3370131338 conn(0x7fd1600aac40 msgr2=0x7fd1600a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/3370131338 shutdown_connections 2024-09-15T21:24:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/3370131338 wait complete. 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: monmap: 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 Processor -- start 2024-09-15T21:24:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- start start 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: init 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1600a5040 0x7fd1601cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): picked mon.noname-a con 0x7fd1600a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): picked mon.noname-c con 0x7fd1601378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1601381e0 0x7fd16013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): picked mon.noname-b con 0x7fd1601381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601425c0 con 0x7fd1600a5040 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601386d0 con 0x7fd1601381e0 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd160142740 con 0x7fd1601378c0 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient(hunting): authenticate will time out at 1378.434951s 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1600a5040 0x7fd1601cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160136960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37960/0 (socket says 172.21.15.17:37960) 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 1 -- 172.21.15.17:0/1722277391 learned_addr learned my addr 172.21.15.17:0/1722277391 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): get_auth_request con 0x7fd1601378c0 auth_method 0 2024-09-15T21:24:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): get_auth_request con 0x7fd1600a5040 auth_method 0 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1601381e0 0x7fd16013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): get_auth_request con 0x7fd1601381e0 auth_method 0 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd164b32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient(hunting): handle_auth_done global_id 74412 payload 824 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 msgr2=0x7fd160136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1601378c0 0x7fd160136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1601381e0 msgr2=0x7fd16013cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1601381e0 0x7fd16013cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 monclient: found mon.noname-a 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd160121010 con 0x7fd1600a5040 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15ffff640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1600a5040 0x7fd1601cf8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1500ab000 tx=0x7fd150019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500b8020 con 0x7fd1600a5040 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: dump: 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1500b4410 con 0x7fd1600a5040 2024-09-15T21:24:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500b0040 con 0x7fd1600a5040 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 5 monclient: authenticate success, global_id 74412 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: _renew_subs 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1600a88a0 con 0x7fd1600a5040 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15cff9640 10 monclient: dump: 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd15e7fc640 4 set_mon_vals no callback set 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: _renew_subs 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.035+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1600a12f0 con 0x7fd1600a5040 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd166595640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd166595640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 0x7fd16013d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd160120750 con 0x7fd160136ea0 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 0x7fd16013d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient: get_auth_request con 0x7fd160136ea0 auth_method 0 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd1500bb020 con 0x7fd1600a5040 2024-09-15T21:24:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_done global_id 74415 payload 824 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd160137ca0 con 0x7fd160136ea0 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd14c03b9c0 0x7fd14c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 9619+0+0 (secure 0 0 0) 0x7fd1500eab10 con 0x7fd1600a5040 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd164b32640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd14c03b9c0 0x7fd14c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd164b32640 10 monclient: get_auth_request con 0x7fd14c03b9c0 auth_method 0 2024-09-15T21:24:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15f7fe640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 0x7fd16013d0b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd160121220 tx=0x7fd154011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154019070 con 0x7fd160136ea0 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 10 monclient: dump: 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd154004300 con 0x7fd160136ea0 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd164b32640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd14c03b9c0 0x7fd14c03de80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd158004fd0 tx=0x7fd158001850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 msgr2=0x7fd16013d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 0x7fd16013d0b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd160121220 tx=0x7fd154011e50 comp rx=0 tx=0).stop 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 msgr2=0x7fd16013d0b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.039+0000 7fd15cff9640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd160136ea0 0x7fd16013d0b0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd160121220 tx=0x7fd154011e50 comp rx=0 tx=0).stop 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.043+0000 7fd166595640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.043+0000 7fd166595640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 0x7fd160138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.043+0000 7fd166595640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.043+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd16013da00 con 0x7fd16013d5f0 2024-09-15T21:24:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 0x7fd160138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient: get_auth_request con 0x7fd16013d5f0 auth_method 0 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 10 monclient(hunting): handle_auth_done global_id 74418 payload 824 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 1 -- 172.21.15.17:0/1722277391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd16013f330 con 0x7fd16013d5f0 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15f7fe640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 0x7fd160138d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd160121220 tx=0x7fd1540046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154019040 con 0x7fd16013d5f0 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 10 monclient: dump: 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7fd15401e450 con 0x7fd16013d5f0 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 msgr2=0x7fd160138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 0x7fd160138d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd160121220 tx=0x7fd1540046d0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 msgr2=0x7fd160138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd15cff9640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd16013d5f0 0x7fd160138d60 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd160121220 tx=0x7fd1540046d0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:15.093 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 66, 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 67831, 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:15.094 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:15.095 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:15.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.098 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:15.099 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.100 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.102 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:15.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:15.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.105 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.106 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.107 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:15.107 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd14c03b9c0 msgr2=0x7fd14c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd14c03b9c0 0x7fd14c03de80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd158004fd0 tx=0x7fd158001850 comp rx=0 tx=0).stop 2024-09-15T21:24:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 10 monclient: shutdown 2024-09-15T21:24:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1600a5040 msgr2=0x7fd1601cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 --2- 172.21.15.17:0/1722277391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1600a5040 0x7fd1601cf8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1500ab000 tx=0x7fd150019190 comp rx=0 tx=0).stop 2024-09-15T21:24:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd164b32640 1 -- 172.21.15.17:0/1722277391 reap_dead start 2024-09-15T21:24:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd164b32640 1 -- 172.21.15.17:0/1722277391 reap_dead start 2024-09-15T21:24:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 shutdown_connections 2024-09-15T21:24:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 >> 172.21.15.17:0/1722277391 conn(0x7fd1600aac40 msgr2=0x7fd16006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 shutdown_connections 2024-09-15T21:24:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.047+0000 7fd166595640 1 -- 172.21.15.17:0/1722277391 wait complete. 2024-09-15T21:24:15.109 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:24:15.109 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:24:15.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:24:15.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.218+0000 7f6da6bbb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:15.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.218+0000 7f6da6bbb640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:15.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.218+0000 7f6da6bbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.218+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f6da0137cd0 con 0x7f6da0134300 2024-09-15T21:24:15.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient: get_monmap_and_config 2024-09-15T21:24:15.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient: monmap: 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:15.234840+0000 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:15.234840+0000 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:15.236 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 Processor -- start 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 -- start start 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient: init 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f8009cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): picked mon.noname-b con 0x7f8f800a4c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8f800a4fe0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f8013ce10 0x7f8f8013f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): picked mon.noname-c con 0x7f8f8013ce10 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f8009c700 con 0x7f8f800a4fe0 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8006f820 con 0x7f8f8013ce10 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8f800a5fc0 con 0x7f8f800a4c00 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f84acd640 10 monclient(hunting): authenticate will time out at 1378.630900s 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f8009cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f8009cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37964/0 (socket says 172.21.15.17:37964) 2024-09-15T21:24:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 -- 172.21.15.17:0/2373178740 learned_addr learned my addr 172.21.15.17:0/2373178740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): get_auth_request con 0x7f8f800a4c00 auth_method 0 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 1 --2- 172.21.15.17:0/2373178740 >> v2:172.21.15.119:3300/0 conn(0x7f8f8013ce10 0x7f8f8013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): get_auth_request con 0x7f8f8013ce10 auth_method 0 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_done global_id 44930 payload 274 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 -- 172.21.15.17:0/2373178740 >> v2:172.21.15.119:3300/0 conn(0x7f8f8013ce10 msgr2=0x7f8f8013f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/2373178740 >> v2:172.21.15.119:3300/0 conn(0x7f8f8013ce10 0x7f8f8013f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 -- 172.21.15.17:0/2373178740 >> v1:172.21.15.119:6789/0 conn(0x7f8f800a4fe0 legacy=0x7f8f800a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 monclient: found mon.noname-b 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.231+0000 7f8f7e59c640 1 -- 172.21.15.17:0/2373178740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f800a5d10 con 0x7f8f800a4c00 2024-09-15T21:24:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/2373178740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f8009cb20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009870 tx=0x7f8f7001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30ee6e062766341 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 1 -- 172.21.15.17:0/2373178740 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f700b1070 con 0x7f8f800a4c00 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: dump: 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.248 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 1 -- 172.21.15.17:0/2373178740 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f700ac620 con 0x7f8f800a4c00 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 5 monclient: authenticate success, global_id 44930 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 4 set_mon_vals no callback set 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 1 -- 172.21.15.17:0/2373178740 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f700ac9f0 con 0x7f8f800a4c00 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f7dd9b640 10 monclient: dump: 2024-09-15T21:24:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 10 monclient: shutdown 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 1 -- 172.21.15.17:0/2373178740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 msgr2=0x7f8f8009cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 1 --2- 172.21.15.17:0/2373178740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f8009cb20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009870 tx=0x7f8f7001c4b0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 1 -- 172.21.15.17:0/2373178740 shutdown_connections 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 1 --2- 172.21.15.17:0/2373178740 >> v2:172.21.15.119:3300/0 conn(0x7f8f8013ce10 0x7f8f8013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 1 --2- 172.21.15.17:0/2373178740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f8009cb20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.235+0000 7f8f84acd640 1 -- 172.21.15.17:0/2373178740 >> 172.21.15.17:0/2373178740 conn(0x7f8f800aaeb0 msgr2=0x7f8f800a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- 172.21.15.17:0/2373178740 shutdown_connections 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- 172.21.15.17:0/2373178740 wait complete. 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: monmap: 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 Processor -- start 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- start start 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: init 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f80131db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): picked mon.noname-c con 0x7f8f800a4c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f800a4fe0 0x7f8f801322f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): picked mon.noname-a con 0x7f8f800a4fe0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8013ce10 0x7f8f801cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): picked mon.noname-b con 0x7f8f8013ce10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f801422f0 con 0x7f8f800a4fe0 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80142170 con 0x7f8f8013ce10 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8f80142470 con 0x7f8f800a4c00 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): authenticate will time out at 1378.637610s 2024-09-15T21:24:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f800a4fe0 0x7f8f801322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f800a4fe0 0x7f8f801322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47644/0 (socket says 172.21.15.17:47644) 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 -- 172.21.15.17:0/1901872688 learned_addr learned my addr 172.21.15.17:0/1901872688 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): get_auth_request con 0x7f8f800a4fe0 auth_method 0 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f80131db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): get_auth_request con 0x7f8f800a4c00 auth_method 0 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7ed9d640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8013ce10 0x7f8f801cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7ed9d640 10 monclient(hunting): get_auth_request con 0x7f8f8013ce10 auth_method 0 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient(hunting): handle_auth_done global_id 74421 payload 824 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 msgr2=0x7f8f80131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a4c00 0x7f8f80131db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8013ce10 msgr2=0x7f8f801cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8013ce10 0x7f8f801cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 monclient: found mon.noname-a 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f800a0100 con 0x7f8f800a4fe0 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f77fff640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f800a4fe0 0x7f8f801322f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c530 tx=0x7f8f6800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68017070 con 0x7f8f800a4fe0 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: dump: 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f680052d0 con 0x7f8f800a4fe0 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 5 monclient: authenticate success, global_id 74421 2024-09-15T21:24:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: _renew_subs 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8013d720 con 0x7f8f800a4fe0 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68012870 con 0x7f8f800a4fe0 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 10 monclient: dump: 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7d59a640 4 set_mon_vals no callback set 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: _renew_subs 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8009fe20 con 0x7f8f800a4fe0 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 0x7f8f801cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8006a080 con 0x7f8f800a0b60 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 0x7f8f801cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient: get_auth_request con 0x7f8f800a0b60 auth_method 0 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8f6801a070 con 0x7f8f800a4fe0 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_done global_id 44936 payload 824 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f8009cb20 con 0x7f8f800a0b60 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f6003b9f0 0x7f8f6003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 9619+0+0 (secure 0 0 0) 0x7f8f680500e0 con 0x7f8f800a4fe0 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.239+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 0x7f8f801cfcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f8006f5a0 tx=0x7f8f7001acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f700bc070 con 0x7f8f800a0b60 2024-09-15T21:24:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 10 monclient: dump: 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f7ed9d640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f6003b9f0 0x7f8f6003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f7ed9d640 10 monclient: get_auth_request con 0x7f8f6003b9f0 auth_method 0 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8f700b7440 con 0x7f8f800a0b60 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 msgr2=0x7f8f801cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 0x7f8f801cfcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f8006f5a0 tx=0x7f8f7001acf0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 msgr2=0x7f8f801cfcb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f76ffd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f800a0b60 0x7f8f801cfcb0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f8006f5a0 tx=0x7f8f7001acf0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.243+0000 7f8f7ed9d640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f6003b9f0 0x7f8f6003deb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8f800a08d0 tx=0x7f8f7800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 0x7f8f80135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8f801d0600 con 0x7f8f801d01f0 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 0x7f8f80135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient: get_auth_request con 0x7f8f801d01f0 auth_method 0 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 10 monclient(hunting): handle_auth_done global_id 44939 payload 824 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 1 -- 172.21.15.17:0/1901872688 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8f80132830 con 0x7f8f801d01f0 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7e59c640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 0x7f8f80135ed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f7001a0e0 tx=0x7f8f70019250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f700bc050 con 0x7f8f801d01f0 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 10 monclient: dump: 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f8f70003ae0 con 0x7f8f801d01f0 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 msgr2=0x7f8f80135ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 0x7f8f80135ed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f7001a0e0 tx=0x7f8f70019250 comp rx=0 tx=0).stop 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 msgr2=0x7f8f80135ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:15.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f76ffd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f801d01f0 0x7f8f80135ed0 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f7001a0e0 tx=0x7f8f70019250 comp rx=0 tx=0).stop 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 66, 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 223846, 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.267 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.268 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.269 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:15.270 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.271 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:15.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.276 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.277 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.278 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:15.279 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f6003b9f0 msgr2=0x7f8f6003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f6003b9f0 0x7f8f6003deb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8f800a08d0 tx=0x7f8f7800b040 comp rx=0 tx=0).stop 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 10 monclient: shutdown 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f800a4fe0 msgr2=0x7f8f801322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 --2- 172.21.15.17:0/1901872688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f800a4fe0 0x7f8f801322f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c530 tx=0x7f8f6800c9f0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7ed9d640 1 -- 172.21.15.17:0/1901872688 reap_dead start 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f7ed9d640 1 -- 172.21.15.17:0/1901872688 reap_dead start 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 shutdown_connections 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 >> 172.21.15.17:0/1901872688 conn(0x7f8f800aaeb0 msgr2=0x7f8f8013c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 shutdown_connections 2024-09-15T21:24:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.247+0000 7f8f84acd640 1 -- 172.21.15.17:0/1901872688 wait complete. 2024-09-15T21:24:15.282 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:24:15.282 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:24:15.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:24:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.383+0000 7f2b962ff640 10 monclient: get_monmap_and_config 2024-09-15T21:24:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.383+0000 7f2b962ff640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.383+0000 7f2b962ff640 10 monclient: monmap: 2024-09-15T21:24:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:15.389849+0000 2024-09-15T21:24:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:15.389849+0000 2024-09-15T21:24:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:15.393 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 1 Processor -- start 2024-09-15T21:24:15.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 1 -- start start 2024-09-15T21:24:15.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient: init 2024-09-15T21:24:15.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2024-09-15T21:24:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2b9009d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2b9009def0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b9013d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient(hunting): picked mon.noname-b con 0x7f2b9013afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b900a3490 con 0x7f2b9009d400 2024-09-15T21:24:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b900a0320 con 0x7f2b9009def0 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2b900a1bc0 con 0x7f2b9013afc0 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient(hunting): authenticate will time out at 1378.785137s 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b9013d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2b9009d400 0x7f2b9009d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45578/0 (socket says 172.21.15.17:45578) 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): get_auth_request con 0x7f2b9013afc0 auth_method 0 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8ffff640 1 -- 172.21.15.17:0/2053025649 learned_addr learned my addr 172.21.15.17:0/2053025649 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient(hunting): handle_auth_done global_id 44942 payload 274 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 1 -- 172.21.15.17:0/2053025649 >> v1:172.21.15.119:6789/0 conn(0x7f2b9009def0 legacy=0x7f2b901398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 1 -- 172.21.15.17:0/2053025649 >> v1:172.21.15.17:6789/0 conn(0x7f2b9009d400 legacy=0x7f2b9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 1 -- 172.21.15.17:0/2053025649 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619171317 0 0) 0x7f2b900a3490 con 0x7f2b9009d400 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 1 monclient: found mon.noname-b 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 1 -- 172.21.15.17:0/2053025649 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9009e7a0 con 0x7f2b9013afc0 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b9489c640 1 --2- 172.21.15.17:0/2053025649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b9013d3b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009dc0 tx=0x7f2b8801d280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a98411959539d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 1 -- 172.21.15.17:0/2053025649 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b880b2070 con 0x7f2b9013afc0 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: dump: 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 1 -- 172.21.15.17:0/2053025649 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b880a5800 con 0x7f2b9013afc0 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 5 monclient: authenticate success, global_id 44942 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b962ff640 4 set_mon_vals no callback set 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 1 -- 172.21.15.17:0/2053025649 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b880a5bd0 con 0x7f2b9013afc0 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.387+0000 7f2b8effd640 10 monclient: dump: 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: shutdown 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/2053025649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 msgr2=0x7f2b9013d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 --2- 172.21.15.17:0/2053025649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b9013d3b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009dc0 tx=0x7f2b8801d280 comp rx=0 tx=0).stop 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/2053025649 shutdown_connections 2024-09-15T21:24:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 --2- 172.21.15.17:0/2053025649 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b9013d3b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/2053025649 >> 172.21.15.17:0/2053025649 conn(0x7f2b900a66a0 msgr2=0x7f2b900a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/2053025649 shutdown_connections 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/2053025649 wait complete. 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: monmap: 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:15.425 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 Processor -- start 2024-09-15T21:24:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- start start 2024-09-15T21:24:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: init 2024-09-15T21:24:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b9009d400 0x7f2b9013f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): picked mon.noname-b con 0x7f2b9009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9009def0 0x7f2b9013fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): picked mon.noname-a con 0x7f2b9009def0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b901403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): picked mon.noname-c con 0x7f2b9013afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9009ce50 con 0x7f2b9009def0 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9009ccd0 con 0x7f2b9009d400 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2b900a10a0 con 0x7f2b9013afc0 2024-09-15T21:24:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): authenticate will time out at 1378.789864s 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b901403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9009def0 0x7f2b9013fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b901403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38002/0 (socket says 172.21.15.17:38002) 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 1 -- 172.21.15.17:0/3020197366 learned_addr learned my addr 172.21.15.17:0/3020197366 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): get_auth_request con 0x7f2b9013afc0 auth_method 0 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): get_auth_request con 0x7f2b9009def0 auth_method 0 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b9489c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b9009d400 0x7f2b9013f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient(hunting): handle_auth_done global_id 74427 payload 824 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 msgr2=0x7f2b901403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2b9013afc0 0x7f2b901403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b9009d400 msgr2=0x7f2b9013f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b9009d400 0x7f2b9013f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9009d0f0 con 0x7f2b9009def0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 10 monclient: get_auth_request con 0x7f2b9009d400 auth_method 0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b9009d400 0x7f2b9013f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8f7fe640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9009def0 0x7f2b9013fe80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400c3a0 tx=0x7f2b8400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84017070 con 0x7f2b9009def0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: dump: 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b840052d0 con 0x7f2b9009def0 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 5 monclient: authenticate success, global_id 74427 2024-09-15T21:24:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: _renew_subs 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b900729b0 con 0x7f2b9009def0 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84003820 con 0x7f2b9009def0 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 10 monclient: dump: 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8e7fc640 4 set_mon_vals no callback set 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: _renew_subs 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9013cfe0 con 0x7f2b9009def0 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 0x7f2b901cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2b900a1460 con 0x7f2b901cf8c0 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2b8401a070 con 0x7f2b9009def0 2024-09-15T21:24:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b7c03b9f0 0x7f2b7c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 0x7f2b901cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 9619+0+0 (secure 0 0 0) 0x7f2b84050310 con 0x7f2b9009def0 2024-09-15T21:24:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 10 monclient: get_auth_request con 0x7f2b901cf8c0 auth_method 0 2024-09-15T21:24:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.391+0000 7f2b8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b9489c640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b7c03b9f0 0x7f2b7c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b9489c640 10 monclient: get_auth_request con 0x7f2b7c03b9f0 auth_method 0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_done global_id 84130 payload 824 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8ffff640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b901365b0 con 0x7f2b901cf8c0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8ffff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 0x7f2b901cfca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b80000f70 tx=0x7f2b8000db80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b9489c640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b7c03b9f0 0x7f2b7c03deb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b8801a450 tx=0x7f2b88004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80005ae0 con 0x7f2b901cf8c0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 10 monclient: dump: 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2b80017070 con 0x7f2b901cf8c0 2024-09-15T21:24:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 msgr2=0x7f2b901cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 0x7f2b901cfca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b80000f70 tx=0x7f2b8000db80 comp rx=0 tx=0).stop 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 msgr2=0x7f2b901cfca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.395+0000 7f2b8cff9640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901cf8c0 0x7f2b901cfca0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b80000f70 tx=0x7f2b8000db80 comp rx=0 tx=0).stop 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.399+0000 7f2b962ff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.399+0000 7f2b962ff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 0x7f2b90138a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.399+0000 7f2b962ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.399+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9006d020 con 0x7f2b901d01e0 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 0x7f2b90138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient: get_auth_request con 0x7f2b901d01e0 auth_method 0 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 10 monclient(hunting): handle_auth_done global_id 84133 payload 824 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 1 -- 172.21.15.17:0/3020197366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2b900726d0 con 0x7f2b901d01e0 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8ffff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 0x7f2b90138a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b80000fc0 tx=0x7f2b80000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8001ce10 con 0x7f2b901d01e0 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 10 monclient: dump: 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f2b80007280 con 0x7f2b901d01e0 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 msgr2=0x7f2b90138a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 0x7f2b90138a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b80000fc0 tx=0x7f2b80000ee0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 msgr2=0x7f2b90138a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b8cff9640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b901d01e0 0x7f2b90138a30 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b80000fc0 tx=0x7f2b80000ee0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 66, 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7631, 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:15.442 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.443 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.444 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:15.445 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:15.446 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:15.447 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.450 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:15.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.452 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:15.453 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:15.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:15.454 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:15.454 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b7c03b9f0 msgr2=0x7f2b7c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2b7c03b9f0 0x7f2b7c03deb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b8801a450 tx=0x7f2b88004ac0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 10 monclient: shutdown 2024-09-15T21:24:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9009def0 msgr2=0x7f2b9013fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 --2- 172.21.15.17:0/3020197366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9009def0 0x7f2b9013fe80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400c3a0 tx=0x7f2b8400c860 comp rx=0 tx=0).stop 2024-09-15T21:24:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b9489c640 1 -- 172.21.15.17:0/3020197366 reap_dead start 2024-09-15T21:24:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b9489c640 1 -- 172.21.15.17:0/3020197366 reap_dead start 2024-09-15T21:24:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 shutdown_connections 2024-09-15T21:24:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 >> 172.21.15.17:0/3020197366 conn(0x7f2b900a66a0 msgr2=0x7f2b901356e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 shutdown_connections 2024-09-15T21:24:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.403+0000 7f2b962ff640 1 -- 172.21.15.17:0/3020197366 wait complete. 2024-09-15T21:24:15.456 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:24:15.456 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:24:15.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:24:15.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.567+0000 7f042a425640 10 monclient: get_monmap_and_config 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.567+0000 7f042a425640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.567+0000 7f042a425640 10 monclient: monmap: 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:15.574159+0000 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:15.574159+0000 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:15.576 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:15.576 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:15.576 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 Processor -- start 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 -- start start 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient: init 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0424073640 0x7f0424073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): picked mon.noname-a con 0x7f0424073640 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0424073f60 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 0x7f042413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): picked mon.noname-b con 0x7f0424137da0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04240a0890 con 0x7f0424073f60 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f04240746e0 con 0x7f0424073640 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f04240a1640 con 0x7f0424137da0 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f042a425640 10 monclient(hunting): authenticate will time out at 1378.969518s 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 0x7f042413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0424073640 0x7f0424073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 0x7f042413fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38016/0 (socket says 172.21.15.17:38016) 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 -- 172.21.15.17:0/2644286848 learned_addr learned my addr 172.21.15.17:0/2644286848 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): get_auth_request con 0x7f0424137da0 auth_method 0 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): get_auth_request con 0x7f0424073640 auth_method 0 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient(hunting): handle_auth_done global_id 44948 payload 274 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 -- 172.21.15.17:0/2644286848 >> v2:172.21.15.17:3300/0 conn(0x7f0424073640 msgr2=0x7f0424073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 --2- 172.21.15.17:0/2644286848 >> v2:172.21.15.17:3300/0 conn(0x7f0424073640 0x7f0424073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 -- 172.21.15.17:0/2644286848 >> v1:172.21.15.119:6789/0 conn(0x7f0424073f60 legacy=0x7f0424137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 monclient: found mon.noname-b 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f04289c2640 1 -- 172.21.15.17:0/2644286848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04240a1920 con 0x7f0424137da0 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.571+0000 7f0423fff640 1 --2- 172.21.15.17:0/2644286848 >> v2:172.21.15.17:3300/0 conn(0x7f0424073640 0x7f0424073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04289c2640 1 --2- 172.21.15.17:0/2644286848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 0x7f042413fc20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f041c009e50 tx=0x7f041c01c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3690be8c5759b562 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 1 -- 172.21.15.17:0/2644286848 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0a5720 con 0x7f0424137da0 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: dump: 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 1 -- 172.21.15.17:0/2644286848 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c0a58c0 con 0x7f0424137da0 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 5 monclient: authenticate success, global_id 44948 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:15.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 1 -- 172.21.15.17:0/2644286848 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c0a5c90 con 0x7f0424137da0 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 4 set_mon_vals no callback set 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04237fe640 10 monclient: dump: 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.585 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.586 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: shutdown 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/2644286848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 msgr2=0x7f042413fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 --2- 172.21.15.17:0/2644286848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 0x7f042413fc20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f041c009e50 tx=0x7f041c01c8c0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/2644286848 shutdown_connections 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 --2- 172.21.15.17:0/2644286848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424137da0 0x7f042413fc20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 --2- 172.21.15.17:0/2644286848 >> v2:172.21.15.17:3300/0 conn(0x7f0424073640 0x7f0424073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/2644286848 >> 172.21.15.17:0/2644286848 conn(0x7f04240a66a0 msgr2=0x7f04240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/2644286848 shutdown_connections 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/2644286848 wait complete. 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: build_initial_monmap 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: monmap: 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 Processor -- start 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- start start 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: init 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 0x7f04241cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): picked mon.noname-a con 0x7f0424073640 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424073f60 0x7f042413ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): picked mon.noname-c con 0x7f0424073f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0424137da0 0x7f042413f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): picked mon.noname-b con 0x7f0424137da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f04240a0e80 con 0x7f0424073640 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04240a0d00 con 0x7f0424137da0 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0424142980 con 0x7f0424073f60 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient(hunting): authenticate will time out at 1378.974224s 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 0x7f04241cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424073f60 0x7f042413ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 0x7f04241cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47654/0 (socket says 172.21.15.17:47654) 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 -- 172.21.15.17:0/3560480490 learned_addr learned my addr 172.21.15.17:0/3560480490 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): get_auth_request con 0x7f0424073640 auth_method 0 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): get_auth_request con 0x7f0424073f60 auth_method 0 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04289c2640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0424137da0 0x7f042413f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04289c2640 10 monclient(hunting): get_auth_request con 0x7f0424137da0 auth_method 0 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04289c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04289c2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:15.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04289c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient(hunting): handle_auth_done global_id 74433 payload 824 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 -- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424073f60 msgr2=0x7f042413ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424073f60 0x7f042413ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 -- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0424137da0 msgr2=0x7f042413f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0424137da0 0x7f042413f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 monclient: found mon.noname-a 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 -- 172.21.15.17:0/3560480490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042413a680 con 0x7f0424073640 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f041bfff640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424073f60 0x7f042413ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f0423fff640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 0x7f04241cfc70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e980 tx=0x7f040c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c017070 con 0x7f0424073640 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: dump: 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 5 monclient: authenticate success, global_id 74433 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _renew_subs 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042409d0f0 con 0x7f0424073640 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _renew_subs 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241d0320 con 0x7f0424073640 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c00c9a0 con 0x7f0424073640 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c021060 con 0x7f0424073640 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 10 monclient: dump: 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04227fc640 4 set_mon_vals no callback set 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0408005090 con 0x7f0424073640 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.575+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f040c01a070 con 0x7f0424073640 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.579+0000 7f04217fa640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f040403b9c0 0x7f040403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.579+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 9619+0+0 (secure 0 0 0) 0x7f040c050ff0 con 0x7f0424073640 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.579+0000 7f041bfff640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f040403b9c0 0x7f040403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.579+0000 7f041bfff640 10 monclient: get_auth_request con 0x7f040403b9c0 auth_method 0 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.579+0000 7f041bfff640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f040403b9c0 0x7f040403de80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f04240a12b0 tx=0x7f0414007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.583+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f040c009420 con 0x7f0424073640 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.583+0000 7f04217fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.583+0000 7f04217fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:15.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.798+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 added bucket h9 type host to location {rack=r1} v194) ==== 166+0+0 (secure 0 0 0) 0x7f6d980165c0 con 0x7f6da0134300 2024-09-15T21:24:15.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.798+0000 7f6d957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:15.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.798+0000 7f6d957fa640 10 monclient: _finish_command 2 = system:0 added bucket h9 type host to location {rack=r1} 2024-09-15T21:24:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f042a425640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:24:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f042a425640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:24:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f042a425640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0408005650 con 0x7f0424073640 2024-09-15T21:24:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f04217fa640 1 -- 172.21.15.17:0/3560480490 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f040c04e710 con 0x7f0424073640 2024-09-15T21:24:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f04217fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:24:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.823+0000 7f04217fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:15.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=10,kv=11,logm=62} crc {auth=1861395045,config=3541143443,health=3831889982,kv=3059833255,logm=1921096180}) 2024-09-15T21:24:15.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=12,kv=12,logm=59} crc {auth=2961688805,config=3541143443,health=1016277833,kv=3079713637,logm=3013943068}) 2024-09-15T21:24:15.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=10,kv=11,logm=62} crc {auth=1861395045,config=3541143443,health=3831889982,kv=3059833255,logm=1921096180}) 2024-09-15T21:24:15.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=1,health=12,kv=12,logm=62} crc {auth=3950779225,config=555582061,health=1016277833,kv=3079713637,logm=162520895}) 2024-09-15T21:24:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f040403b9c0 msgr2=0x7f040403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f040403b9c0 0x7f040403de80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f04240a12b0 tx=0x7f0414007e40 comp rx=0 tx=0).stop 2024-09-15T21:24:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 10 monclient: shutdown 2024-09-15T21:24:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 msgr2=0x7f04241cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 0x7f04241cfc70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e980 tx=0x7f040c00ee40 comp rx=0 tx=0).stop 2024-09-15T21:24:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 shutdown_connections 2024-09-15T21:24:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f040403b9c0 0x7f040403de80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0424137da0 0x7f042413f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0424073f60 0x7f042413ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 --2- 172.21.15.17:0/3560480490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0424073640 0x7f04241cfc70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 >> 172.21.15.17:0/3560480490 conn(0x7f04240a66a0 msgr2=0x7f04240a4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 shutdown_connections 2024-09-15T21:24:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:15.827+0000 7f042a425640 1 -- 172.21.15.17:0/3560480490 wait complete. 2024-09-15T21:24:15.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=690125220}) 2024-09-15T21:24:15.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.827+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4172634971}) 2024-09-15T21:24:15.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=690125220}) 2024-09-15T21:24:15.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2062228244}) 2024-09-15T21:24:15.837 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.837 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2283393014}) 2024-09-15T21:24:15.837 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2304248693}) 2024-09-15T21:24:15.837 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.837 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2283393014}) 2024-09-15T21:24:15.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.831+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2140515191}) 2024-09-15T21:24:15.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2300550805}) 2024-09-15T21:24:15.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508915258}) 2024-09-15T21:24:15.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2300550805}) 2024-09-15T21:24:15.839 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2422536052}) 2024-09-15T21:24:15.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=635322533}) 2024-09-15T21:24:15.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1773186453}) 2024-09-15T21:24:15.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=635322533}) 2024-09-15T21:24:15.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.835+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=865397510}) 2024-09-15T21:24:15.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2362345884}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=551834195}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2362345884}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2703446464}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1541137960}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2407881473}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1541137960}) 2024-09-15T21:24:15.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3979907065}) 2024-09-15T21:24:15.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.839+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529913587}) 2024-09-15T21:24:15.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1807139626}) 2024-09-15T21:24:15.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529913587}) 2024-09-15T21:24:15.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=152198848}) 2024-09-15T21:24:15.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2293580317}) 2024-09-15T21:24:15.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=673705059}) 2024-09-15T21:24:15.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2293580317}) 2024-09-15T21:24:15.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306066505}) 2024-09-15T21:24:15.850 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:24:15.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3264902215}) 2024-09-15T21:24:15.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2599582045}) 2024-09-15T21:24:15.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3264902215}) 2024-09-15T21:24:15.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.843+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=392271186}) 2024-09-15T21:24:15.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1247784237}) 2024-09-15T21:24:15.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3213547503}) 2024-09-15T21:24:15.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1247784237}) 2024-09-15T21:24:15.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3724653414}) 2024-09-15T21:24:15.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3798996521}) 2024-09-15T21:24:15.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1778067354}) 2024-09-15T21:24:15.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3798996521}) 2024-09-15T21:24:15.855 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.847+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1274239854}) 2024-09-15T21:24:15.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131789085}) 2024-09-15T21:24:15.857 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2994736392}) 2024-09-15T21:24:15.857 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131789085}) 2024-09-15T21:24:15.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2409060609}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=423451532}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2597867642}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=423451532}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.851+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2842063189}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845822332}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3434847417}) 2024-09-15T21:24:15.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.861 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845822332}) 2024-09-15T21:24:15.861 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276956860}) 2024-09-15T21:24:15.861 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3008533408}) 2024-09-15T21:24:15.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1364406016}) 2024-09-15T21:24:15.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3008533408}) 2024-09-15T21:24:15.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.855+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=929892726}) 2024-09-15T21:24:15.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2620446906}) 2024-09-15T21:24:15.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=408953154}) 2024-09-15T21:24:15.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2620446906}) 2024-09-15T21:24:15.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1466804586}) 2024-09-15T21:24:15.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3610088117}) 2024-09-15T21:24:15.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2445009366}) 2024-09-15T21:24:15.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3610088117}) 2024-09-15T21:24:15.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=879783118}) 2024-09-15T21:24:15.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4036028336}) 2024-09-15T21:24:15.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=49927821}) 2024-09-15T21:24:15.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4036028336}) 2024-09-15T21:24:15.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.859+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2977396539}) 2024-09-15T21:24:15.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257792154}) 2024-09-15T21:24:15.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3874909747}) 2024-09-15T21:24:15.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257792154}) 2024-09-15T21:24:15.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141080380}) 2024-09-15T21:24:15.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=573314997}) 2024-09-15T21:24:15.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1969180491}) 2024-09-15T21:24:15.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=573314997}) 2024-09-15T21:24:15.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1957898475}) 2024-09-15T21:24:15.870 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122211748}) 2024-09-15T21:24:15.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2792581846}) 2024-09-15T21:24:15.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122211748}) 2024-09-15T21:24:15.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.863+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2427173248}) 2024-09-15T21:24:15.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3659020812}) 2024-09-15T21:24:15.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3142475142}) 2024-09-15T21:24:15.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3659020812}) 2024-09-15T21:24:15.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196056515}) 2024-09-15T21:24:15.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1899442677}) 2024-09-15T21:24:15.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4235576815}) 2024-09-15T21:24:15.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1899442677}) 2024-09-15T21:24:15.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.867+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4156628396}) 2024-09-15T21:24:15.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131857599}) 2024-09-15T21:24:15.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400593932}) 2024-09-15T21:24:15.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131857599}) 2024-09-15T21:24:15.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=520803566}) 2024-09-15T21:24:15.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=951259762}) 2024-09-15T21:24:15.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3390745638}) 2024-09-15T21:24:15.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=951259762}) 2024-09-15T21:24:15.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.871+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014009923}) 2024-09-15T21:24:15.879 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=650341961}) 2024-09-15T21:24:15.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2095816621}) 2024-09-15T21:24:15.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=650341961}) 2024-09-15T21:24:15.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3522791943}) 2024-09-15T21:24:15.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2995813435}) 2024-09-15T21:24:15.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3768219398}) 2024-09-15T21:24:15.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2995813435}) 2024-09-15T21:24:15.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.875+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3320358239}) 2024-09-15T21:24:15.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1804825076}) 2024-09-15T21:24:15.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1160280511}) 2024-09-15T21:24:15.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1804825076}) 2024-09-15T21:24:15.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=408819648}) 2024-09-15T21:24:15.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1529743912}) 2024-09-15T21:24:15.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1927598905}) 2024-09-15T21:24:15.885 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.885 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1529743912}) 2024-09-15T21:24:15.885 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225945150}) 2024-09-15T21:24:15.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939274788}) 2024-09-15T21:24:15.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=449785695}) 2024-09-15T21:24:15.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939274788}) 2024-09-15T21:24:15.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.879+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1494121643}) 2024-09-15T21:24:15.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444599795}) 2024-09-15T21:24:15.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=413393135}) 2024-09-15T21:24:15.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=444599795}) 2024-09-15T21:24:15.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753860275}) 2024-09-15T21:24:15.890 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.890 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4248290422}) 2024-09-15T21:24:15.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2775938653}) 2024-09-15T21:24:15.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4248290422}) 2024-09-15T21:24:15.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.883+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617541833}) 2024-09-15T21:24:15.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=17583187}) 2024-09-15T21:24:15.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2689591346}) 2024-09-15T21:24:15.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=17583187}) 2024-09-15T21:24:15.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3662826710}) 2024-09-15T21:24:15.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=74,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13} crc {logm=882138633,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1367745308}) 2024-09-15T21:24:15.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=77,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11} crc {logm=1943491483,mdsmap=173658045,mgr=2659938204,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1489745611}) 2024-09-15T21:24:15.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=74,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13} crc {logm=882138633,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1367745308}) 2024-09-15T21:24:15.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.887+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=73,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=4} crc {logm=2441668288,mdsmap=3009587245,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=333674267,mon_config_key=3231751822}) 2024-09-15T21:24:15.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.891+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.891+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=13,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=73} crc {mon_config_key=2985153859,monmap=2617021892,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=933646749,osd_snap=841190635,osdmap=3203123812}) 2024-09-15T21:24:15.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.891+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=13,monmap=1,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=74} crc {mon_config_key=772490508,monmap=1578004834,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=933646749,osd_snap=841190635,osdmap=2310353186}) 2024-09-15T21:24:15.898 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.891+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.898 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.891+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=13,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=73} crc {mon_config_key=2985153859,monmap=2617021892,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=933646749,osd_snap=841190635,osdmap=3203123812}) 2024-09-15T21:24:15.898 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.891+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=71} crc {mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=933646749,osd_snap=841190635,osdmap=1796449855}) 2024-09-15T21:24:15.900 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.900 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=514412080}) 2024-09-15T21:24:15.900 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2902558334}) 2024-09-15T21:24:15.900 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.900 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=514412080}) 2024-09-15T21:24:15.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1814375380}) 2024-09-15T21:24:15.902 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.902 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1928312873}) 2024-09-15T21:24:15.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2178317029}) 2024-09-15T21:24:15.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1928312873}) 2024-09-15T21:24:15.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.895+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1354561887}) 2024-09-15T21:24:15.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.906+0000 7f6da6bbb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:15.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.906+0000 7f6da6bbb640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:15.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.906+0000 7f6da6bbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:15.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.906+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f6da006a760 con 0x7f6da0134300 2024-09-15T21:24:15.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.907+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.907+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=15} crc {osdmap=3789310465}) 2024-09-15T21:24:15.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.907+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=14} crc {osdmap=1176532748}) 2024-09-15T21:24:15.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.907+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:24:15.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.907+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=15} crc {osdmap=3789310465}) 2024-09-15T21:24:15.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:24:15.907+0000 7fe521e74640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=15} crc {osdmap=767822209}) 2024-09-15T21:24:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6d957fa640 1 -- 172.21.15.119:0/2745078684 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h9' already exists v194) ==== 145+0+0 (secure 0 0 0) 0x7f6d980add10 con 0x7f6da0134300 2024-09-15T21:24:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6d957fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6d957fa640 10 monclient: _finish_command 3 = system:0 bucket 'h9' already exists 2024-09-15T21:24:15.914 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h9' already exists 2024-09-15T21:24:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d8c03b9c0 msgr2=0x7f6d8c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d8c03b9c0 0x7f6d8c03de80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009f90 tx=0x7f6d9c009190 comp rx=0 tx=0).stop 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 10 monclient: shutdown 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 msgr2=0x7f6da00722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da00722a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6d980ab000 tx=0x7f6d98019650 comp rx=0 tx=0).stop 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 shutdown_connections 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d8c03b9c0 0x7f6d8c03de80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da0134300 0x7f6da00722a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6da00a7ac0 0x7f6da01314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 --2- 172.21.15.119:0/2745078684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da00a41b0 0x7f6da0130f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 >> 172.21.15.119:0/2745078684 conn(0x7f6da00a6fd0 msgr2=0x7f6da01323b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 shutdown_connections 2024-09-15T21:24:15.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:15.910+0000 7f6da6bbb640 1 -- 172.21.15.119:0/2745078684 wait complete. 2024-09-15T21:24:15.931 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:15.931 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h10 host rack=r1 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient: get_monmap_and_config 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient: build_initial_monmap 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient: monmap: 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:16.054013+0000 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:16.054013+0000 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:16.054 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.055 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:16.055 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:16.055 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:16.055 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:16.055 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:16.055 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 1 Processor -- start 2024-09-15T21:24:16.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 1 -- start start 2024-09-15T21:24:16.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient: init 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb27c09d550 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb27c09fae0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb27c13d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient(hunting): picked mon.noname-b con 0x7fb27c09e040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27c0a3600 con 0x7fb27c09fae0 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27c0a1d60 con 0x7fb27c09d550 2024-09-15T21:24:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb27c09f1b0 con 0x7fb27c09e040 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 10 monclient(hunting): authenticate will time out at 1374.851004s 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb27c09d550 0x7fb27c09d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55974/0 (socket says 172.21.15.119:55974) 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27bfff640 1 -- 172.21.15.119:0/292764250 learned_addr learned my addr 172.21.15.119:0/292764250 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb27c13d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): get_auth_request con 0x7fb27c09e040 auth_method 0 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1016193799 0 0) 0x7fb27c0a1d60 con 0x7fb27c09d550 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:24:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): my global_id is 84139 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb268005740 con 0x7fb27c09d550 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2871211994 0 0) 0x7fb268005740 con 0x7fb27c09d550 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 msgr2=0x7fb27c13d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb27c13d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 >> v1:172.21.15.17:6789/0 conn(0x7fb27c09fae0 legacy=0x7fb27c139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:16.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c09e880 con 0x7fb27c09d550 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb26c001700 con 0x7fb27c09d550 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 5 monclient: authenticate success, global_id 84139 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb28235b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: dump: 2024-09-15T21:24:16.069 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 >> v1:172.21.15.119:6789/0 conn(0x7fb27c09d550 legacy=0x7fb27c09d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb268005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): picked mon.c con 0x7fb27c09e040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2680061f0 0x7fb268006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): picked mon.a con 0x7fb2680061f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb268006f60 0x7fb2680073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): picked mon.b con 0x7fb268006f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c09f1b0 con 0x7fb2680061f0 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb26800de70 con 0x7fb268006f60 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb26800e070 con 0x7fb27c09e040 2024-09-15T21:24:16.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.050+0000 7fb27b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb268006f60 0x7fb2680073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): get_auth_request con 0x7fb268006f60 auth_method 0 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb273fff640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2680061f0 0x7fb268006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27bfff640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb268005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb273fff640 10 monclient(hunting): get_auth_request con 0x7fb2680061f0 auth_method 0 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb273fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb273fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb273fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient(hunting): handle_auth_done global_id 84139 payload 293 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 -- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 msgr2=0x7fb268005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb268005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 -- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2680061f0 msgr2=0x7fb268006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2680061f0 0x7fb268006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient: _finish_hunting 0 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 monclient: found mon.b 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:16.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 -- 172.21.15.119:0/292764250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb268005740 con 0x7fb268006f60 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient: _finish_auth 0 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 10 monclient: _check_auth_tickets 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27bfff640 10 monclient: get_auth_request con 0x7fb27c09e040 auth_method 0 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27bfff640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb27c09e040 0x7fb268005c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb268006f60 0x7fb2680073e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb274009fc0 tx=0x7fb27401cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ae2eda7455ba9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740b2070 con 0x7fb268006f60 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: dump: 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:16.074 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2740a5a60 con 0x7fb268006f60 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 1 -- 172.21.15.119:0/292764250 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274009c10 con 0x7fb268006f60 2024-09-15T21:24:16.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 4 set_mon_vals no callback set 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb27b7fe640 10 monclient: dump: 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.076 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 10 monclient: shutdown 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 1 -- 172.21.15.119:0/292764250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb268006f60 msgr2=0x7fb2680073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 1 --2- 172.21.15.119:0/292764250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb268006f60 0x7fb2680073e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb274009fc0 tx=0x7fb27401cc60 comp rx=0 tx=0).stop 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb2808f8640 1 -- 172.21.15.119:0/292764250 reap_dead start 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 1 -- 172.21.15.119:0/292764250 shutdown_connections 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 1 -- 172.21.15.119:0/292764250 >> 172.21.15.119:0/292764250 conn(0x7fb27c0a66e0 msgr2=0x7fb27c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 1 -- 172.21.15.119:0/292764250 shutdown_connections 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.054+0000 7fb28235b640 1 -- 172.21.15.119:0/292764250 wait complete. 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: build_initial_monmap 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: monmap: 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:16.077 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 Processor -- start 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- start start 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: init 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:16.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb268006f60 0x7fb27c134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): picked mon.noname-c con 0x7fb268006f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c11eb00 0x7fb27c134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): picked mon.noname-a con 0x7fb27c11eb00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 0x7fb27c11cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): picked mon.noname-b con 0x7fb27c11c6a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c0a1260 con 0x7fb27c11eb00 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c0a2920 con 0x7fb27c11c6a0 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb27c0a13e0 con 0x7fb268006f60 2024-09-15T21:24:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient(hunting): authenticate will time out at 1374.856911s 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 0x7fb27c11cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 0x7fb27c11cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59912/0 (socket says 172.21.15.119:59912) 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 -- 172.21.15.119:0/4015714432 learned_addr learned my addr 172.21.15.119:0/4015714432 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): get_auth_request con 0x7fb27c11c6a0 auth_method 0 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb273fff640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c11eb00 0x7fb27c134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27bfff640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb268006f60 0x7fb27c134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient(hunting): handle_auth_done global_id 84142 payload 824 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 -- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb268006f60 msgr2=0x7fb27c134910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb268006f60 0x7fb27c134910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 -- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c11eb00 msgr2=0x7fb27c134e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c11eb00 0x7fb27c134e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient: _finish_hunting 0 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 monclient: found mon.noname-b 2024-09-15T21:24:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 -- 172.21.15.119:0/4015714432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c120020 con 0x7fb27c11c6a0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27bfff640 10 monclient: get_auth_request con 0x7fb268006f60 auth_method 0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27bfff640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb268006f60 0x7fb27c134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb273fff640 10 monclient: get_auth_request con 0x7fb27c11eb00 auth_method 0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb273fff640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c11eb00 0x7fb27c134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2808f8640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 0x7fb27c11cb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2740b0040 tx=0x7fb274009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740bd070 con 0x7fb27c11c6a0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:16.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: dump: 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:16.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2740a5860 con 0x7fb27c11c6a0 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 5 monclient: authenticate success, global_id 84142 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _renew_subs 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c120300 con 0x7fb27c11c6a0 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740b2040 con 0x7fb27c11c6a0 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 10 monclient: dump: 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.084 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27affd640 4 set_mon_vals no callback set 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _renew_subs 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c135700 con 0x7fb27c11c6a0 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:16.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c13ace0 con 0x7fb27c11c6a0 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb2740c0070 con 0x7fb27c11c6a0 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb25c03b9c0 0x7fb25c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 9829+0+0 (secure 0 0 0) 0x7fb2740eab80 con 0x7fb27c11c6a0 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27bfff640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb25c03b9c0 0x7fb25c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27bfff640 10 monclient: get_auth_request con 0x7fb25c03b9c0 auth_method 0 2024-09-15T21:24:16.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.058+0000 7fb27bfff640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb25c03b9c0 0x7fb25c03de80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb26c007940 tx=0x7fb26c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:16.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.066+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb274016970 con 0x7fb27c11c6a0 2024-09-15T21:24:16.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.066+0000 7fb2797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:16.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.066+0000 7fb2797fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:16.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.242+0000 7fb28235b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:16.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.242+0000 7fb28235b640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:16.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.242+0000 7fb28235b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.242+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fb27c11fc70 con 0x7fb27c11c6a0 2024-09-15T21:24:16.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.826+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 added bucket h10 type host to location {rack=r1} v195) ==== 168+0+0 (secure 0 0 0) 0x7fb274003ba0 con 0x7fb27c11c6a0 2024-09-15T21:24:16.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.826+0000 7fb2797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:16.830 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.826+0000 7fb2797fa640 10 monclient: _finish_command 2 = system:0 added bucket h10 type host to location {rack=r1} 2024-09-15T21:24:16.851 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:24:16.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:16.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.930+0000 7fb28235b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:16.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.934+0000 7fb28235b640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:16.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.934+0000 7fb28235b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:16.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.934+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fb27c11d820 con 0x7fb27c11c6a0 2024-09-15T21:24:16.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.934+0000 7fb2797fa640 1 -- 172.21.15.119:0/4015714432 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h10' already exists v195) ==== 147+0+0 (secure 0 0 0) 0x7fb2740e81d0 con 0x7fb27c11c6a0 2024-09-15T21:24:16.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.934+0000 7fb2797fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:16.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.934+0000 7fb2797fa640 10 monclient: _finish_command 3 = system:0 bucket 'h10' already exists 2024-09-15T21:24:16.939 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h10' already exists 2024-09-15T21:24:16.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb25c03b9c0 msgr2=0x7fb25c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb25c03b9c0 0x7fb25c03de80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb26c007940 tx=0x7fb26c009040 comp rx=0 tx=0).stop 2024-09-15T21:24:16.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 10 monclient: shutdown 2024-09-15T21:24:16.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:16.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 msgr2=0x7fb27c11cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 0x7fb27c11cb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2740b0040 tx=0x7fb274009750 comp rx=0 tx=0).stop 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 shutdown_connections 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb25c03b9c0 0x7fb25c03de80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb27c11c6a0 0x7fb27c11cb20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c11eb00 0x7fb27c134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 --2- 172.21.15.119:0/4015714432 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb268006f60 0x7fb27c134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 >> 172.21.15.119:0/4015714432 conn(0x7fb27c0a66e0 msgr2=0x7fb27c0728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 shutdown_connections 2024-09-15T21:24:16.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:16.938+0000 7fb28235b640 1 -- 172.21.15.119:0/4015714432 wait complete. 2024-09-15T21:24:16.956 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:16.956 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h11 host rack=r1 2024-09-15T21:24:16.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient: get_monmap_and_config 2024-09-15T21:24:16.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient: build_initial_monmap 2024-09-15T21:24:16.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient: monmap: 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:16.983630+0000 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:16.983630+0000 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:16.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 Processor -- start 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 -- start start 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient: init 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17dc1378c0 0x7f17dc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): picked mon.noname-a con 0x7f17dc1378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f17dc1381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17dc0a5210 0x7f17dc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): picked mon.noname-c con 0x7f17dc0a5210 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc0a1d00 con 0x7f17dc1381e0 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f17dc0a1fe0 con 0x7f17dc1378c0 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f17dc0a22c0 con 0x7f17dc0a5210 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e3956640 10 monclient(hunting): authenticate will time out at 1380.379044s 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17dc1378c0 0x7f17dc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17dc1378c0 0x7f17dc137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47660/0 (socket says 172.21.15.17:47660) 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 -- 172.21.15.17:0/4266573703 learned_addr learned my addr 172.21.15.17:0/4266573703 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): get_auth_request con 0x7f17dc1378c0 auth_method 0 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 1 --2- 172.21.15.17:0/4266573703 >> v2:172.21.15.119:3300/0 conn(0x7f17dc0a5210 0x7f17dc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): get_auth_request con 0x7f17dc0a5210 auth_method 0 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e1ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient(hunting): handle_auth_done global_id 74439 payload 274 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 -- 172.21.15.17:0/4266573703 >> v2:172.21.15.119:3300/0 conn(0x7f17dc0a5210 msgr2=0x7f17dc0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 --2- 172.21.15.17:0/4266573703 >> v2:172.21.15.119:3300/0 conn(0x7f17dc0a5210 0x7f17dc0a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 -- 172.21.15.17:0/4266573703 >> v1:172.21.15.119:6789/0 conn(0x7f17dc1381e0 legacy=0x7f17dc0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient: _finish_hunting 0 2024-09-15T21:24:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 monclient: found mon.noname-a 2024-09-15T21:24:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.979+0000 7f17e16f2640 1 -- 172.21.15.17:0/4266573703 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc0a4490 con 0x7f17dc1378c0 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e16f2640 1 --2- 172.21.15.17:0/4266573703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc1378c0 0x7f17dc137ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009870 tx=0x7f17d801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed5704cc22c5c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 1 -- 172.21.15.17:0/4266573703 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80ab600 con 0x7f17dc1378c0 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: dump: 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.997 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: _finish_auth 0 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 1 -- 172.21.15.17:0/4266573703 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d80b0070 con 0x7f17dc1378c0 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 1 -- 172.21.15.17:0/4266573703 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80ab9c0 con 0x7f17dc1378c0 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 5 monclient: authenticate success, global_id 74439 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 4 set_mon_vals no callback set 2024-09-15T21:24:16.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17d3fff640 10 monclient: dump: 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:16.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 10 monclient: shutdown 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 -- 172.21.15.17:0/4266573703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc1378c0 msgr2=0x7f17dc137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 --2- 172.21.15.17:0/4266573703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc1378c0 0x7f17dc137ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009870 tx=0x7f17d801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 -- 172.21.15.17:0/4266573703 shutdown_connections 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 --2- 172.21.15.17:0/4266573703 >> v2:172.21.15.119:3300/0 conn(0x7f17dc0a5210 0x7f17dc0a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 --2- 172.21.15.17:0/4266573703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc1378c0 0x7f17dc137ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 -- 172.21.15.17:0/4266573703 >> 172.21.15.17:0/4266573703 conn(0x7f17dc0aac40 msgr2=0x7f17dc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 -- 172.21.15.17:0/4266573703 shutdown_connections 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 1 -- 172.21.15.17:0/4266573703 wait complete. 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.983+0000 7f17e3956640 10 monclient: monmap: 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 Processor -- start 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- start start 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: init 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 0x7f17dc1cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): picked mon.noname-a con 0x7f17dc0a5210 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc1378c0 0x7f17dc136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): picked mon.noname-b con 0x7f17dc1378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 0x7f17dc13cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): picked mon.noname-c con 0x7f17dc1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc06f340 con 0x7f17dc0a5210 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc06f1c0 con 0x7f17dc1378c0 2024-09-15T21:24:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc1425c0 con 0x7f17dc1381e0 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient(hunting): authenticate will time out at 1380.384717s 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 0x7f17dc13cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 0x7f17dc1cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 0x7f17dc13cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38038/0 (socket says 172.21.15.17:38038) 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 0x7f17dc1cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47676/0 (socket says 172.21.15.17:47676) 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 1 -- 172.21.15.17:0/3128684264 learned_addr learned my addr 172.21.15.17:0/3128684264 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): get_auth_request con 0x7f17dc1381e0 auth_method 0 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): get_auth_request con 0x7f17dc0a5210 auth_method 0 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e0ef1640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc1378c0 0x7f17dc136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient(hunting): handle_auth_done global_id 74442 payload 824 2024-09-15T21:24:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 -- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 msgr2=0x7f17dc13cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 0x7f17dc13cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 -- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc1378c0 msgr2=0x7f17dc136b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc1378c0 0x7f17dc136b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 monclient: found mon.noname-a 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 -- 172.21.15.17:0/3128684264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc0a12f0 con 0x7f17dc0a5210 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e0ef1640 10 monclient: get_auth_request con 0x7f17dc1378c0 auth_method 0 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e1ef3640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 0x7f17dc13cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e0ef1640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc1378c0 0x7f17dc136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e16f2640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 0x7f17dc1cf870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17dc06fa50 tx=0x7f17d801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80be640 con 0x7f17dc0a5210 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: dump: 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d80be7e0 con 0x7f17dc0a5210 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80bebd0 con 0x7f17dc0a5210 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 5 monclient: authenticate success, global_id 74442 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _renew_subs 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1214b0 con 0x7f17dc0a5210 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d37fe640 4 set_mon_vals no callback set 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 10 monclient: dump: 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _renew_subs 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc120c10 con 0x7f17dc0a5210 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f17d80aa070 con 0x7f17dc0a5210 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17dc0a56a0 con 0x7f17dc0a5210 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17c003b9c0 0x7f17c003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7f17d80eb6e0 con 0x7f17dc0a5210 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e0ef1640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17c003b9c0 0x7f17c003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e0ef1640 10 monclient: get_auth_request con 0x7f17c003b9c0 auth_method 0 2024-09-15T21:24:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.987+0000 7f17e0ef1640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17c003b9c0 0x7f17c003de80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17dc137890 tx=0x7f17cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.991+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f17dc0a12f0 con 0x7f17dc0a5210 2024-09-15T21:24:17.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.991+0000 7f17d1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:16.991+0000 7f17d1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient: get_monmap_and_config 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient: monmap: 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:17.065021+0000 2024-09-15T21:24:17.076 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:17.065021+0000 2024-09-15T21:24:17.077 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.077 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:17.077 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:17.077 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 1 Processor -- start 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 1 -- start start 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient: init 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135bf0 0x7fe6e009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient(hunting): picked mon.noname-b con 0x7fe6e0135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6e0135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe6e0136430 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:17.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e00a1870 con 0x7fe6e0135fd0 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e00a1d80 con 0x7fe6e0136430 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e00a3600 con 0x7fe6e0135bf0 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e6614640 10 monclient(hunting): authenticate will time out at 1375.861637s 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e4bb1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe6e0136430 0x7fe6e00a1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55976/0 (socket says 172.21.15.119:55976) 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6e4bb1640 1 -- 172.21.15.119:0/4086012476 learned_addr learned my addr 172.21.15.119:0/4086012476 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 1 --2- 172.21.15.119:0/4086012476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135bf0 0x7fe6e009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): get_auth_request con 0x7fe6e0135bf0 auth_method 0 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6deffd640 1 -- 172.21.15.119:0/4086012476 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520477014 0 0) 0x7fe6e00a1d80 con 0x7fe6e0136430 2024-09-15T21:24:17.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6deffd640 10 monclient(hunting): _negotiate 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6deffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6deffd640 10 monclient(hunting): my global_id is 84148 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6deffd640 1 -- 172.21.15.119:0/4086012476 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c8003530 con 0x7fe6e0136430 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient(hunting): handle_auth_done global_id 44960 payload 274 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 1 -- 172.21.15.119:0/4086012476 >> v1:172.21.15.119:6789/0 conn(0x7fe6e0136430 legacy=0x7fe6e00a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 1 -- 172.21.15.119:0/4086012476 >> v1:172.21.15.17:6789/0 conn(0x7fe6e0135fd0 legacy=0x7fe6e00a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 1 monclient: found mon.noname-b 2024-09-15T21:24:17.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.062+0000 7fe6dffff640 1 -- 172.21.15.119:0/4086012476 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e00a2060 con 0x7fe6e0135bf0 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dffff640 1 --2- 172.21.15.119:0/4086012476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135bf0 0x7fe6e009fe70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00097f0 tx=0x7fe6d001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2da62635382ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 1 -- 172.21.15.119:0/4086012476 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00b1070 con 0x7fe6e0135bf0 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: dump: 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.081 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 5 monclient: authenticate success, global_id 44960 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 1 -- 172.21.15.119:0/4086012476 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d00ac5f0 con 0x7fe6e0135bf0 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 4 set_mon_vals no callback set 2024-09-15T21:24:17.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 1 -- 172.21.15.119:0/4086012476 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00ac9a0 con 0x7fe6e0135bf0 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6deffd640 10 monclient: dump: 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.083 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: shutdown 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/4086012476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135bf0 msgr2=0x7fe6e009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 --2- 172.21.15.119:0/4086012476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135bf0 0x7fe6e009fe70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00097f0 tx=0x7fe6d001c370 comp rx=0 tx=0).stop 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/4086012476 shutdown_connections 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 --2- 172.21.15.119:0/4086012476 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135bf0 0x7fe6e009fe70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/4086012476 >> 172.21.15.119:0/4086012476 conn(0x7fe6e00a66e0 msgr2=0x7fe6e00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/4086012476 shutdown_connections 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/4086012476 wait complete. 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: monmap: 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:17.084 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 Processor -- start 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- start start 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: init 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6e0135bf0 0x7fe6e0133f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): picked mon.noname-a con 0x7fe6e0135bf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135fd0 0x7fe6e0134490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): picked mon.noname-c con 0x7fe6e0135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 0x7fe6e013c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): picked mon.noname-b con 0x7fe6e0136430 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01416d0 con 0x7fe6e0135bf0 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e006ece0 con 0x7fe6e0136430 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e0141850 con 0x7fe6e0135fd0 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient(hunting): authenticate will time out at 1375.867055s 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 0x7fe6e013c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 0x7fe6e013c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59922/0 (socket says 172.21.15.119:59922) 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 -- 172.21.15.119:0/2858773870 learned_addr learned my addr 172.21.15.119:0/2858773870 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): get_auth_request con 0x7fe6e0136430 auth_method 0 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6df7fe640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135fd0 0x7fe6e0134490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6df7fe640 10 monclient(hunting): get_auth_request con 0x7fe6e0135fd0 auth_method 0 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dffff640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6e0135bf0 0x7fe6e0133f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient(hunting): handle_auth_done global_id 84151 payload 824 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 -- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135fd0 msgr2=0x7fe6e0134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135fd0 0x7fe6e0134490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 -- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6e0135bf0 msgr2=0x7fe6e0133f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6e0135bf0 0x7fe6e0133f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 monclient: found mon.noname-b 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:17.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 -- 172.21.15.119:0/2858773870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e011df30 con 0x7fe6e0136430 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dffff640 10 monclient: get_auth_request con 0x7fe6e0135bf0 auth_method 0 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dffff640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6e0135bf0 0x7fe6e0133f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e4bb1640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 0x7fe6e013c520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800cb40 tx=0x7fe6d800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800aca0 con 0x7fe6e0136430 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: dump: 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.089 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d800ae40 con 0x7fe6e0136430 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 5 monclient: authenticate success, global_id 84151 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8012600 con 0x7fe6e0136430 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:17.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6dcff9640 10 monclient: dump: 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.091 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: _renew_subs 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e0072a20 con 0x7fe6e0136430 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6de7fc640 4 set_mon_vals no callback set 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: _renew_subs 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.066+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e011dc50 con 0x7fe6e0136430 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6e6614640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6e6614640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6e6614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e009fe70 con 0x7fe6e0136430 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe6d80127a0 con 0x7fe6e0136430 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6dcff9640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6cc03b9c0 0x7fe6cc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7fe6d80516a0 con 0x7fe6e0136430 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6dffff640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6cc03b9c0 0x7fe6cc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6dffff640 10 monclient: get_auth_request con 0x7fe6cc03b9c0 auth_method 0 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.070+0000 7fe6dffff640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6cc03b9c0 0x7fe6cc03de80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00ae000 tx=0x7fe6d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.074+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe6d801a360 con 0x7fe6e0136430 2024-09-15T21:24:17.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.074+0000 7fe6dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.074+0000 7fe6dcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:17.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17e3956640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:17.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17e3956640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:17.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17e3956640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f17dc120750 con 0x7f17dc0a5210 2024-09-15T21:24:17.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17d1ffb640 1 -- 172.21.15.17:0/3128684264 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f17d80160f0 con 0x7f17dc0a5210 2024-09-15T21:24:17.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17d1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:17.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.227+0000 7f17d1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:17.235 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:17.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17c003b9c0 msgr2=0x7f17c003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17c003b9c0 0x7f17c003de80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17dc137890 tx=0x7f17cc009040 comp rx=0 tx=0).stop 2024-09-15T21:24:17.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 10 monclient: shutdown 2024-09-15T21:24:17.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 msgr2=0x7f17dc1cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 0x7f17dc1cf870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17dc06fa50 tx=0x7f17d801ac20 comp rx=0 tx=0).stop 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 shutdown_connections 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17c003b9c0 0x7f17c003de80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17dc1381e0 0x7f17dc13cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc1378c0 0x7f17dc136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 --2- 172.21.15.17:0/3128684264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17dc0a5210 0x7f17dc1cf870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 >> 172.21.15.17:0/3128684264 conn(0x7f17dc0aac40 msgr2=0x7f17dc09da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 shutdown_connections 2024-09-15T21:24:17.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.231+0000 7f17e3956640 1 -- 172.21.15.17:0/3128684264 wait complete. 2024-09-15T21:24:17.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.246+0000 7fe6e6614640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:17.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.246+0000 7fe6e6614640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:17.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.246+0000 7fe6e6614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:17.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:17.246+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fe6e011d770 con 0x7fe6e0136430 2024-09-15T21:24:17.256 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:24:17.256 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:24:17.256 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:24:17.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:24:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient: get_monmap_and_config 2024-09-15T21:24:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient: monmap: 2024-09-15T21:24:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:17.372483+0000 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:17.372483+0000 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:17.374 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 Processor -- start 2024-09-15T21:24:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 -- start start 2024-09-15T21:24:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient: init 2024-09-15T21:24:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2024-09-15T21:24:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fec2809d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec2809def0 0x7fec281398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): picked mon.noname-c con 0x7fec2809def0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec28139de0 0x7fec2813c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): picked mon.noname-b con 0x7fec28139de0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec280a3490 con 0x7fec2809d400 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fec2809f0b0 con 0x7fec2809def0 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fec280a0320 con 0x7fec28139de0 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.367+0000 7fec2fd26640 10 monclient(hunting): authenticate will time out at 1380.767885s 2024-09-15T21:24:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2dac2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fec2809d400 0x7fec2809d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45590/0 (socket says 172.21.15.17:45590) 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2dac2640 1 -- 172.21.15.17:0/3178923617 learned_addr learned my addr 172.21.15.17:0/3178923617 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 --2- 172.21.15.17:0/3178923617 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec28139de0 0x7fec2813c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): get_auth_request con 0x7fec28139de0 auth_method 0 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2d2c1640 1 --2- 172.21.15.17:0/3178923617 >> v2:172.21.15.119:3300/0 conn(0x7fec2809def0 0x7fec281398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2d2c1640 10 monclient(hunting): get_auth_request con 0x7fec2809def0 auth_method 0 2024-09-15T21:24:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2d2c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2d2c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2d2c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_done global_id 44966 payload 274 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 -- 172.21.15.17:0/3178923617 >> v2:172.21.15.119:3300/0 conn(0x7fec2809def0 msgr2=0x7fec281398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 --2- 172.21.15.17:0/3178923617 >> v2:172.21.15.119:3300/0 conn(0x7fec2809def0 0x7fec281398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 -- 172.21.15.17:0/3178923617 >> v1:172.21.15.17:6789/0 conn(0x7fec2809d400 legacy=0x7fec2809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 monclient: found mon.noname-b 2024-09-15T21:24:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 1 -- 172.21.15.17:0/3178923617 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964081341 0 0) 0x7fec280a3490 con 0x7fec2809d400 2024-09-15T21:24:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 -- 172.21.15.17:0/3178923617 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2809e7a0 con 0x7fec28139de0 2024-09-15T21:24:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2d2c1640 1 --2- 172.21.15.17:0/3178923617 >> v2:172.21.15.119:3300/0 conn(0x7fec2809def0 0x7fec281398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:24:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2e2c3640 1 --2- 172.21.15.17:0/3178923617 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec28139de0 0x7fec2813c1d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec20009d90 tx=0x7fec2001d260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ad852fbd5386d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 1 -- 172.21.15.17:0/3178923617 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200b2070 con 0x7fec28139de0 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: dump: 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 5 monclient: authenticate success, global_id 44966 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 1 -- 172.21.15.17:0/3178923617 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec200a58e0 con 0x7fec28139de0 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 4 set_mon_vals no callback set 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 1 -- 172.21.15.17:0/3178923617 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200a5cb0 con 0x7fec28139de0 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2cac0640 10 monclient: dump: 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.389 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 10 monclient: shutdown 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 -- 172.21.15.17:0/3178923617 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec28139de0 msgr2=0x7fec2813c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 --2- 172.21.15.17:0/3178923617 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec28139de0 0x7fec2813c1d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec20009d90 tx=0x7fec2001d260 comp rx=0 tx=0).stop 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 -- 172.21.15.17:0/3178923617 shutdown_connections 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 --2- 172.21.15.17:0/3178923617 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec28139de0 0x7fec2813c1d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 --2- 172.21.15.17:0/3178923617 >> v2:172.21.15.119:3300/0 conn(0x7fec2809def0 0x7fec281398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 -- 172.21.15.17:0/3178923617 >> 172.21.15.17:0/3178923617 conn(0x7fec280a66a0 msgr2=0x7fec280a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.371+0000 7fec2fd26640 1 -- 172.21.15.17:0/3178923617 shutdown_connections 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- 172.21.15.17:0/3178923617 wait complete. 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: monmap: 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:17.391 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 Processor -- start 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- start start 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: init 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2809d400 0x7fec280a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): picked mon.noname-b con 0x7fec2809d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec2809def0 0x7fec281381b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): picked mon.noname-c con 0x7fec2809def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec28139de0 0x7fec281386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): picked mon.noname-a con 0x7fec28139de0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fec2809ce50 con 0x7fec28139de0 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec2809ccd0 con 0x7fec2809d400 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fec2813ef30 con 0x7fec2809def0 2024-09-15T21:24:17.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): authenticate will time out at 1380.773515s 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec2809def0 0x7fec281381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec28139de0 0x7fec281386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec2809def0 0x7fec281381b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38046/0 (socket says 172.21.15.17:38046) 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 1 -- 172.21.15.17:0/597453353 learned_addr learned my addr 172.21.15.17:0/597453353 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): get_auth_request con 0x7fec2809def0 auth_method 0 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): get_auth_request con 0x7fec28139de0 auth_method 0 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2d2c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2809d400 0x7fec280a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient(hunting): handle_auth_done global_id 74451 payload 824 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec2809def0 msgr2=0x7fec281381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fec2809def0 0x7fec281381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2809d400 msgr2=0x7fec280a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2809d400 0x7fec280a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 monclient: found mon.noname-a 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2813c500 con 0x7fec28139de0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient: get_auth_request con 0x7fec2809d400 auth_method 0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2809d400 0x7fec280a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2e2c3640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec28139de0 0x7fec281386f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec200ac000 tx=0x7fec200195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200b2040 con 0x7fec28139de0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:17.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: dump: 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20003ba0 con 0x7fec28139de0 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 5 monclient: authenticate success, global_id 74451 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: _renew_subs 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2809d0f0 con 0x7fec28139de0 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20004030 con 0x7fec28139de0 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: dump: 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.398 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: _renew_subs 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1f7fe640 4 set_mon_vals no callback set 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2811ef50 con 0x7fec28139de0 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 0x7fec2802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fec28125aa0 con 0x7fec281cf8a0 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 0x7fec2802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient: get_auth_request con 0x7fec281cf8a0 auth_method 0 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fec200041d0 con 0x7fec28139de0 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 10 monclient(hunting): handle_auth_done global_id 74454 payload 824 2024-09-15T21:24:17.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec280a11b0 con 0x7fec281cf8a0 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fec0c03b9c0 0x7fec0c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7fec200eb7d0 con 0x7fec28139de0 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec2dac2640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 0x7fec2802d450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec280a1970 tx=0x7fec1800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18005ac0 con 0x7fec281cf8a0 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.375+0000 7fec1e7fc640 10 monclient: dump: 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.401 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec2d2c1640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fec0c03b9c0 0x7fec0c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec2d2c1640 10 monclient: get_auth_request con 0x7fec0c03b9c0 auth_method 0 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fec18017070 con 0x7fec281cf8a0 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 msgr2=0x7fec2802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 0x7fec2802d450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec280a1970 tx=0x7fec1800db80 comp rx=0 tx=0).stop 2024-09-15T21:24:17.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 msgr2=0x7fec2802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:17.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec1e7fc640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cf8a0 0x7fec2802d450 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec280a1970 tx=0x7fec1800db80 comp rx=0 tx=0).stop 2024-09-15T21:24:17.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.379+0000 7fec2d2c1640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fec0c03b9c0 0x7fec0c03de80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec28120830 tx=0x7fec24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:17.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 0x7fec2802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fec2809d7e0 con 0x7fec281cfc80 2024-09-15T21:24:17.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 0x7fec2802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient: get_auth_request con 0x7fec281cfc80 auth_method 0 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 10 monclient(hunting): handle_auth_done global_id 74457 payload 824 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 1 -- 172.21.15.17:0/597453353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fec28072950 con 0x7fec281cfc80 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2dac2640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 0x7fec2802d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec18016000 tx=0x7fec18005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1801ce10 con 0x7fec281cfc80 2024-09-15T21:24:17.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 10 monclient: dump: 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.406 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7fec18007280 con 0x7fec281cfc80 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 msgr2=0x7fec2802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 0x7fec2802d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec18016000 tx=0x7fec18005800 comp rx=0 tx=0).stop 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 msgr2=0x7fec2802d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec1e7fc640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec281cfc80 0x7fec2802d990 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec18016000 tx=0x7fec18005800 comp rx=0 tx=0).stop 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:17.407 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 66, 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 70167, 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.408 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.409 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.410 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:17.411 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:17.412 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.413 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:17.417 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.419 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.420 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:17.421 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fec0c03b9c0 msgr2=0x7fec0c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fec0c03b9c0 0x7fec0c03de80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec28120830 tx=0x7fec24008040 comp rx=0 tx=0).stop 2024-09-15T21:24:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 10 monclient: shutdown 2024-09-15T21:24:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec28139de0 msgr2=0x7fec281386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec28139de0 0x7fec281386f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec200ac000 tx=0x7fec200195a0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2e2c3640 1 -- 172.21.15.17:0/597453353 reap_dead start 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 shutdown_connections 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 --2- 172.21.15.17:0/597453353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec28139de0 0x7fec281386f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.383+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 >> 172.21.15.17:0/597453353 conn(0x7fec280a66a0 msgr2=0x7fec2813aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.387+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 shutdown_connections 2024-09-15T21:24:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.387+0000 7fec2fd26640 1 -- 172.21.15.17:0/597453353 wait complete. 2024-09-15T21:24:17.425 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:24:17.425 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:24:17.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:24:17.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient: get_monmap_and_config 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient: monmap: 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:17.539752+0000 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:17.539752+0000 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:17.541 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:17.542 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:17.542 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:17.542 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:17.542 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 Processor -- start 2024-09-15T21:24:17.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 -- start start 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient: init 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54d81378c0 0x7f54d8137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): picked mon.noname-c con 0x7f54d81378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f54d81381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 0x7f54d80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): picked mon.noname-b con 0x7f54d80a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80a1d00 con 0x7f54d81381e0 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80a1fe0 con 0x7f54d81378c0 2024-09-15T21:24:17.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f54d80a22c0 con 0x7f54d80a5210 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.535+0000 7f54de298640 10 monclient(hunting): authenticate will time out at 1380.935951s 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 0x7f54d80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 0x7f54d80a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38062/0 (socket says 172.21.15.17:38062) 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 -- 172.21.15.17:0/2608393905 learned_addr learned my addr 172.21.15.17:0/2608393905 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): get_auth_request con 0x7f54d80a5210 auth_method 0 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d7fff640 1 --2- 172.21.15.17:0/2608393905 >> v2:172.21.15.119:3300/0 conn(0x7f54d81378c0 0x7f54d8137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d7fff640 10 monclient(hunting): get_auth_request con 0x7f54d81378c0 auth_method 0 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient(hunting): handle_auth_done global_id 44972 payload 274 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 -- 172.21.15.17:0/2608393905 >> v2:172.21.15.119:3300/0 conn(0x7f54d81378c0 msgr2=0x7f54d8137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 --2- 172.21.15.17:0/2608393905 >> v2:172.21.15.119:3300/0 conn(0x7f54d81378c0 0x7f54d8137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 -- 172.21.15.17:0/2608393905 >> v1:172.21.15.119:6789/0 conn(0x7f54d81381e0 legacy=0x7f54d80a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 monclient: found mon.noname-b 2024-09-15T21:24:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 -- 172.21.15.17:0/2608393905 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d80a4490 con 0x7f54d80a5210 2024-09-15T21:24:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54dc835640 1 --2- 172.21.15.17:0/2608393905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 0x7f54d80a56a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d00098a0 tx=0x7f54d001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8cbe301e01d0309 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 1 -- 172.21.15.17:0/2608393905 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00b1070 con 0x7f54d80a5210 2024-09-15T21:24:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: dump: 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 1 -- 172.21.15.17:0/2608393905 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d00ac600 con 0x7f54d80a5210 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 5 monclient: authenticate success, global_id 44972 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 1 -- 172.21.15.17:0/2608393905 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00ac9d0 con 0x7f54d80a5210 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 4 set_mon_vals no callback set 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54d6ffd640 10 monclient: dump: 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.558 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 10 monclient: shutdown 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 -- 172.21.15.17:0/2608393905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 msgr2=0x7f54d80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 --2- 172.21.15.17:0/2608393905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 0x7f54d80a56a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d00098a0 tx=0x7f54d001c400 comp rx=0 tx=0).stop 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 -- 172.21.15.17:0/2608393905 shutdown_connections 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 --2- 172.21.15.17:0/2608393905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d80a5210 0x7f54d80a56a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 --2- 172.21.15.17:0/2608393905 >> v2:172.21.15.119:3300/0 conn(0x7f54d81378c0 0x7f54d8137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 -- 172.21.15.17:0/2608393905 >> 172.21.15.17:0/2608393905 conn(0x7f54d80aac40 msgr2=0x7f54d80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 -- 172.21.15.17:0/2608393905 shutdown_connections 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.539+0000 7f54de298640 1 -- 172.21.15.17:0/2608393905 wait complete. 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: monmap: 2024-09-15T21:24:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 Processor -- start 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- start start 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: init 2024-09-15T21:24:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54d80a5210 0x7f54d81cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): picked mon.noname-b con 0x7f54d80a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d81378c0 0x7f54d8136a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): picked mon.noname-c con 0x7f54d81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54d81381e0 0x7f54d813cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): picked mon.noname-a con 0x7f54d81381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81425c0 con 0x7f54d81381e0 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8138700 con 0x7f54d80a5210 2024-09-15T21:24:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f54d8142740 con 0x7f54d81378c0 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): authenticate will time out at 1380.941380s 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54d81381e0 0x7f54d813cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54d81381e0 0x7f54d813cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47720/0 (socket says 172.21.15.17:47720) 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 -- 172.21.15.17:0/1572537703 learned_addr learned my addr 172.21.15.17:0/1572537703 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): get_auth_request con 0x7f54d81381e0 auth_method 0 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d81378c0 0x7f54d8136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): get_auth_request con 0x7f54d81378c0 auth_method 0 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54d80a5210 0x7f54d81cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): get_auth_request con 0x7f54d80a5210 auth_method 0 2024-09-15T21:24:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient(hunting): handle_auth_done global_id 74460 payload 824 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d81378c0 msgr2=0x7f54d8136a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d81378c0 0x7f54d8136a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54d80a5210 msgr2=0x7f54d81cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54d80a5210 0x7f54d81cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 monclient: found mon.noname-a 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d81d0900 con 0x7f54d81381e0 2024-09-15T21:24:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54dc835640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54d81381e0 0x7f54d813cc80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54d001c230 tx=0x7f54d001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00b1040 con 0x7f54d81381e0 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: dump: 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d00bc070 con 0x7f54d81381e0 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00b7690 con 0x7f54d81381e0 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 5 monclient: authenticate success, global_id 74460 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: _renew_subs 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d80a12f0 con 0x7f54d81381e0 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d67fc640 4 set_mon_vals no callback set 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 10 monclient: dump: 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: _renew_subs 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d80a8990 con 0x7f54d81381e0 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 0x7f54d813d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f54d80a56a0 con 0x7f54d8136f80 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 0x7f54d813d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient: get_auth_request con 0x7f54d8136f80 auth_method 0 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f54d00b7830 con 0x7f54d81381e0 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 10 monclient(hunting): handle_auth_done global_id 44978 payload 824 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54d813f220 con 0x7f54d8136f80 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54c403b9f0 0x7f54c403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7f54d00eafe0 con 0x7f54d81381e0 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.543+0000 7f54d7fff640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 0x7f54d813d1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54c80045c0 tx=0x7f54c8007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8014610 con 0x7f54d8136f80 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 10 monclient: dump: 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d77fe640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54c403b9f0 0x7f54c403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d77fe640 10 monclient: get_auth_request con 0x7f54c403b9f0 auth_method 0 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f54c8019070 con 0x7f54d8136f80 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 msgr2=0x7f54d813d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 0x7f54d813d1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54c80045c0 tx=0x7f54c8007700 comp rx=0 tx=0).stop 2024-09-15T21:24:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 msgr2=0x7f54d813d1c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d4ff9640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d8136f80 0x7f54d813d1c0 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54c80045c0 tx=0x7f54c8007700 comp rx=0 tx=0).stop 2024-09-15T21:24:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.547+0000 7f54d77fe640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54c403b9f0 0x7f54c403deb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f54d806e7b0 tx=0x7f54cc007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 0x7f54d8138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f54d813db10 con 0x7f54d813d700 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 0x7f54d8138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient: get_auth_request con 0x7f54d813d700 auth_method 0 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 10 monclient(hunting): handle_auth_done global_id 44981 payload 824 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 1 -- 172.21.15.17:0/1572537703 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54d8121150 con 0x7f54d813d700 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d7fff640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 0x7f54d8138d60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54c80045c0 tx=0x7f54c8004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c801e230 con 0x7f54d813d700 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 10 monclient: dump: 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f54c801e230 con 0x7f54d813d700 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 msgr2=0x7f54d8138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 0x7f54d8138d60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54c80045c0 tx=0x7f54c8004a50 comp rx=0 tx=0).stop 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 msgr2=0x7f54d8138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54d4ff9640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54d813d700 0x7f54d8138d60 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54c80045c0 tx=0x7f54c8004a50 comp rx=0 tx=0).stop 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 66, 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 226150, 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:17.576 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.577 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.578 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:17.579 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.580 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:17.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:17.583 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.584 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.585 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.588 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:17.589 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:17.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54c403b9f0 msgr2=0x7f54c403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f54c403b9f0 0x7f54c403deb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f54d806e7b0 tx=0x7f54cc007e40 comp rx=0 tx=0).stop 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 10 monclient: shutdown 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54d81381e0 msgr2=0x7f54d813cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 --2- 172.21.15.17:0/1572537703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54d81381e0 0x7f54d813cc80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54d001c230 tx=0x7f54d001ac40 comp rx=0 tx=0).stop 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54dc835640 1 -- 172.21.15.17:0/1572537703 reap_dead start 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54dc835640 1 -- 172.21.15.17:0/1572537703 reap_dead start 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 shutdown_connections 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 >> 172.21.15.17:0/1572537703 conn(0x7f54d80aac40 msgr2=0x7f54d80a8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 shutdown_connections 2024-09-15T21:24:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.551+0000 7f54de298640 1 -- 172.21.15.17:0/1572537703 wait complete. 2024-09-15T21:24:17.592 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:24:17.592 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:24:17.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:24:17.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient: get_monmap_and_config 2024-09-15T21:24:17.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient: monmap: 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:17.707814+0000 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:17.707814+0000 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:17.709 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:17.710 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:17.710 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 Processor -- start 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 -- start start 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient: init 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b1c09d400 0x7f4b1c09d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): picked mon.noname-a con 0x7f4b1c09d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:17.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4b1c09dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b1c134d70 0x7f4b1c13cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): picked mon.noname-c con 0x7f4b1c134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1c0a3130 con 0x7f4b1c09dd20 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1c0a1bc0 con 0x7f4b1c09d400 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1c0a3490 con 0x7f4b1c134d70 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b2165a640 10 monclient(hunting): authenticate will time out at 1381.103227s 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b1c09d400 0x7f4b1c09d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b1c09d400 0x7f4b1c09d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54230/0 (socket says 172.21.15.17:54230) 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 -- 172.21.15.17:0/363029016 learned_addr learned my addr 172.21.15.17:0/363029016 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): get_auth_request con 0x7f4b1c09d400 auth_method 0 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1b7fe640 1 --2- 172.21.15.17:0/363029016 >> v2:172.21.15.119:3300/0 conn(0x7f4b1c134d70 0x7f4b1c13cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1b7fe640 10 monclient(hunting): get_auth_request con 0x7f4b1c134d70 auth_method 0 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient(hunting): handle_auth_done global_id 74463 payload 274 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 -- 172.21.15.17:0/363029016 >> v2:172.21.15.119:3300/0 conn(0x7f4b1c134d70 msgr2=0x7f4b1c13cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 --2- 172.21.15.17:0/363029016 >> v2:172.21.15.119:3300/0 conn(0x7f4b1c134d70 0x7f4b1c13cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 -- 172.21.15.17:0/363029016 >> v1:172.21.15.119:6789/0 conn(0x7f4b1c09dd20 legacy=0x7f4b1c134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 monclient: found mon.noname-a 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.703+0000 7f4b1affd640 1 -- 172.21.15.17:0/363029016 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c09e7a0 con 0x7f4b1c09d400 2024-09-15T21:24:17.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b1affd640 1 --2- 172.21.15.17:0/363029016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c09d7e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c09f4f0 tx=0x7f4b0401eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c7dad1a8c1389b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 1 -- 172.21.15.17:0/363029016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400b2d0 con 0x7f4b1c09d400 2024-09-15T21:24:17.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: dump: 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 1 -- 172.21.15.17:0/363029016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0400b470 con 0x7f4b1c09d400 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 5 monclient: authenticate success, global_id 74463 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 4 set_mon_vals no callback set 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 1 -- 172.21.15.17:0/363029016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b040a78a0 con 0x7f4b1c09d400 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b19ffb640 10 monclient: dump: 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.724 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 10 monclient: shutdown 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 -- 172.21.15.17:0/363029016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 msgr2=0x7f4b1c09d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 --2- 172.21.15.17:0/363029016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c09d7e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c09f4f0 tx=0x7f4b0401eab0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 -- 172.21.15.17:0/363029016 shutdown_connections 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 --2- 172.21.15.17:0/363029016 >> v2:172.21.15.119:3300/0 conn(0x7f4b1c134d70 0x7f4b1c13cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 --2- 172.21.15.17:0/363029016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c09d7e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 -- 172.21.15.17:0/363029016 >> 172.21.15.17:0/363029016 conn(0x7f4b1c0a66a0 msgr2=0x7f4b1c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 -- 172.21.15.17:0/363029016 shutdown_connections 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 -- 172.21.15.17:0/363029016 wait complete. 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 10 monclient: monmap: 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:17.726 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 Processor -- start 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 1 -- start start 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.707+0000 7f4b2165a640 10 monclient: init 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c137350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): picked mon.noname-a con 0x7f4b1c09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4b1c09dd20 0x7f4b1c137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): picked mon.noname-c con 0x7f4b1c09dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c134d70 0x7f4b1c137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): picked mon.noname-b con 0x7f4b1c134d70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c09fba0 con 0x7f4b1c09d400 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c09fa20 con 0x7f4b1c134d70 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c0a10a0 con 0x7f4b1c09dd20 2024-09-15T21:24:17.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): authenticate will time out at 1381.108394s 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c137350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54232/0 (socket says 172.21.15.17:54232) 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 -- 172.21.15.17:0/2195482833 learned_addr learned my addr 172.21.15.17:0/2195482833 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): get_auth_request con 0x7f4b1c09d400 auth_method 0 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4b1c09dd20 0x7f4b1c137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): get_auth_request con 0x7f4b1c09dd20 auth_method 0 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c134d70 0x7f4b1c137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 10 monclient(hunting): get_auth_request con 0x7f4b1c134d70 auth_method 0 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient(hunting): handle_auth_done global_id 74466 payload 824 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4b1c09dd20 msgr2=0x7f4b1c137890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4b1c09dd20 0x7f4b1c137890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c134d70 msgr2=0x7f4b1c137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c134d70 0x7f4b1c137dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 monclient: found mon.noname-a 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c13ae90 con 0x7f4b1c09d400 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c134d70 0x7f4b1c137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4b1c09dd20 0x7f4b1c137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1affd640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c137350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b0401b6c0 tx=0x7f4b0401b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b040be020 con 0x7f4b1c09d400 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: dump: 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b04009820 con 0x7f4b1c09d400 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 5 monclient: authenticate success, global_id 74466 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400b720 con 0x7f4b1c09d400 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient: _renew_subs 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c13b170 con 0x7f4b1c09d400 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b18ff9640 4 set_mon_vals no callback set 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: dump: 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.733 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient: _renew_subs 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c13cc40 con 0x7f4b1c09d400 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 0x7f4b1c063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c02e9b0 con 0x7f4b1c02e5d0 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 0x7f4b1c063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient: get_auth_request con 0x7f4b1c02e5d0 auth_method 0 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4b040c1020 con 0x7f4b1c09d400 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_done global_id 84169 payload 824 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b1c09d0f0 con 0x7f4b1c02e5d0 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4b0003b9c0 0x7f4b0003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7f4b040ee040 con 0x7f4b1c09d400 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1a7fc640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 0x7f4b1c063e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c13b0a0 tx=0x7f4b0c00ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4b0003b9c0 0x7f4b0003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b1b7fe640 10 monclient: get_auth_request con 0x7f4b0003b9c0 auth_method 0 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0052e0 con 0x7f4b1c02e5d0 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.711+0000 7f4b137fe640 10 monclient: dump: 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.736 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b1b7fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4b0003b9c0 0x7f4b0003de80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ba10 tx=0x7f4b14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4b0c018070 con 0x7f4b1c02e5d0 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 msgr2=0x7f4b1c063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 0x7f4b1c063e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c13b0a0 tx=0x7f4b0c00ff70 comp rx=0 tx=0).stop 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 msgr2=0x7f4b1c063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:17.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.715+0000 7f4b137fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c02e5d0 0x7f4b1c063e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b2165a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b2165a640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 0x7f4b1c1d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b2165a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c132f50 con 0x7f4b1c064340 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 0x7f4b1c1d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient: get_auth_request con 0x7f4b1c064340 auth_method 0 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 10 monclient(hunting): handle_auth_done global_id 84172 payload 824 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 1 -- 172.21.15.17:0/2195482833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4b1c06d020 con 0x7f4b1c064340 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b1a7fc640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 0x7f4b1c1d5e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c017000 tx=0x7f4b0c000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.719+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00dcf0 con 0x7f4b1c064340 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 10 monclient: dump: 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.740 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f4b0c00dcf0 con 0x7f4b1c064340 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 msgr2=0x7f4b1c1d5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 0x7f4b1c1d5e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c017000 tx=0x7f4b0c000ee0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 msgr2=0x7f4b1c1d5e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:17.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b137fe640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b1c064340 0x7f4b1c1d5e00 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c017000 tx=0x7f4b0c000ee0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 66, 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9951, 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.742 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.743 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.744 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:17.745 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:17.746 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.750 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:17.753 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:17.754 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:24:17.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:17.754 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:17.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:17.754 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:17.754 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4b0003b9c0 msgr2=0x7f4b0003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4b0003b9c0 0x7f4b0003de80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ba10 tx=0x7f4b14008040 comp rx=0 tx=0).stop 2024-09-15T21:24:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 10 monclient: shutdown 2024-09-15T21:24:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 msgr2=0x7f4b1c137350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 --2- 172.21.15.17:0/2195482833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b1c09d400 0x7f4b1c137350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b0401b6c0 tx=0x7f4b0401b850 comp rx=0 tx=0).stop 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b1b7fe640 1 -- 172.21.15.17:0/2195482833 reap_dead start 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b1b7fe640 1 -- 172.21.15.17:0/2195482833 reap_dead start 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 shutdown_connections 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 >> 172.21.15.17:0/2195482833 conn(0x7f4b1c0a66a0 msgr2=0x7f4b1c073010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 shutdown_connections 2024-09-15T21:24:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.723+0000 7f4b2165a640 1 -- 172.21.15.17:0/2195482833 wait complete. 2024-09-15T21:24:17.756 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:24:17.757 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:24:17.757 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:24:17.757 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:24:17.757 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:24:17.757 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:24:17.757 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:24:17.769 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:24:17.769 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:24:17.769 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient: get_monmap_and_config 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient: monmap: 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:17.927262+0000 2024-09-15T21:24:17.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:17.927262+0000 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 1 Processor -- start 2024-09-15T21:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 1 -- start start 2024-09-15T21:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient: init 2024-09-15T21:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2024-09-15T21:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff7141378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff7141383b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff71413e4a0 0x7ff714140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient(hunting): picked mon.noname-c con 0x7ff71413e4a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7140a1a20 con 0x7ff7141383b0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7140a1d00 con 0x7ff7141378c0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff7140a1fe0 con 0x7ff71413e4a0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff719104640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7ff7141383b0 legacy=0x7ff71413cd80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71bb69640 10 monclient(hunting): authenticate will time out at 1381.322678s 2024-09-15T21:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff71413e4a0 0x7ff714140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff719905640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff7141378c0 0x7ff714137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:41538/0 (socket says 172.21.15.17:41538) 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff719905640 1 -- 172.21.15.17:0/3028804710 learned_addr learned my addr 172.21.15.17:0/3028804710 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): get_auth_request con 0x7ff71413e4a0 auth_method 0 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient(hunting): handle_auth_done global_id 84175 payload 274 2024-09-15T21:24:17.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 1 -- 172.21.15.17:0/3028804710 >> v1:172.21.15.119:6789/0 conn(0x7ff7141378c0 legacy=0x7ff714137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 1 -- 172.21.15.17:0/3028804710 >> v1:172.21.15.17:6789/0 conn(0x7ff7141383b0 legacy=0x7ff71413cd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:17.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 1 monclient: found mon.noname-c 2024-09-15T21:24:17.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:17.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.923+0000 7ff71a106640 1 -- 172.21.15.17:0/3028804710 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7140a22c0 con 0x7ff71413e4a0 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71a106640 1 --2- 172.21.15.17:0/3028804710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff714140890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009dc0 tx=0x7ff70c01c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df8dd2d031f5cff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 1 -- 172.21.15.17:0/3028804710 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0b1070 con 0x7ff71413e4a0 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: dump: 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 5 monclient: authenticate success, global_id 84175 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 1 -- 172.21.15.17:0/3028804710 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70c0a4bd0 con 0x7ff71413e4a0 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 1 -- 172.21.15.17:0/3028804710 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0ac440 con 0x7ff71413e4a0 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff718903640 10 monclient: dump: 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 4 set_mon_vals no callback set 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 10 monclient: shutdown 2024-09-15T21:24:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 -- 172.21.15.17:0/3028804710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 msgr2=0x7ff714140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 --2- 172.21.15.17:0/3028804710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff714140890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009dc0 tx=0x7ff70c01c310 comp rx=0 tx=0).stop 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 -- 172.21.15.17:0/3028804710 shutdown_connections 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 --2- 172.21.15.17:0/3028804710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff714140890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 -- 172.21.15.17:0/3028804710 >> 172.21.15.17:0/3028804710 conn(0x7ff7140aac40 msgr2=0x7ff7140a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 -- 172.21.15.17:0/3028804710 shutdown_connections 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.927+0000 7ff71bb69640 1 -- 172.21.15.17:0/3028804710 wait complete. 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: build_initial_monmap 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: monmap: 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 Processor -- start 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- start start 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: init 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 0x7ff71413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): picked mon.noname-c con 0x7ff7141378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141383b0 0x7ff71413a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): picked mon.noname-a con 0x7ff7141383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff7141369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): picked mon.noname-b con 0x7ff71413e4a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff714144230 con 0x7ff7141383b0 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff7141440b0 con 0x7ff71413e4a0 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff7141443b0 con 0x7ff7141378c0 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient(hunting): authenticate will time out at 1381.329488s 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719104640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141383b0 msgr2=0x7ff71413a1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719104640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141383b0 0x7ff71413a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 0x7ff71413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 0x7ff71413c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52796/0 (socket says 172.21.15.17:52796) 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 -- 172.21.15.17:0/1460275946 learned_addr learned my addr 172.21.15.17:0/1460275946 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): get_auth_request con 0x7ff7141378c0 auth_method 0 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71a106640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff7141369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71a106640 10 monclient(hunting): get_auth_request con 0x7ff71413e4a0 auth_method 0 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71a106640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71a106640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71a106640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient(hunting): handle_auth_done global_id 44987 payload 824 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 -- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 msgr2=0x7ff7141369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff7141369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 -- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141383b0 msgr2=0x7ff71413a1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141383b0 0x7ff71413a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient: _finish_hunting 0 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 monclient: found mon.noname-c 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 -- 172.21.15.17:0/1460275946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71413ccd0 con 0x7ff7141378c0 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71a106640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff7141369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719905640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 0x7ff71413c340 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff71000c480 tx=0x7ff71000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710017070 con 0x7ff7141378c0 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: dump: 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710005d10 con 0x7ff7141378c0 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 5 monclient: authenticate success, global_id 44987 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _renew_subs 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7140a12f0 con 0x7ff7141378c0 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100072b0 con 0x7ff7141378c0 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 10 monclient: dump: 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70bfff640 4 set_mon_vals no callback set 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _renew_subs 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71413b7e0 con 0x7ff7141378c0 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff714137ca0 con 0x7ff7141378c0 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff71001a070 con 0x7ff7141378c0 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6f803b9d0 0x7ff6f803de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:17.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7ff7100502c0 con 0x7ff7141378c0 2024-09-15T21:24:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719104640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6f803b9d0 0x7ff6f803de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.931+0000 7ff719104640 10 monclient: get_auth_request con 0x7ff6f803b9d0 auth_method 0 2024-09-15T21:24:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.935+0000 7ff719104640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6f803b9d0 0x7ff6f803de90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff71413d1d0 tx=0x7ff704006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.935+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff710010300 con 0x7ff7141378c0 2024-09-15T21:24:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.935+0000 7ff70a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:17.935+0000 7ff70a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.163+0000 7ff71bb69640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.163+0000 7ff71bb69640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.163+0000 7ff71bb69640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.163+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff71413dc00 con 0x7ff7141378c0 2024-09-15T21:24:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff70a7fc640 1 -- 172.21.15.17:0/1460275946 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7ff710010e20 con 0x7ff7141378c0 2024-09-15T21:24:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff70a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff70a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:18.171 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6f803b9d0 msgr2=0x7ff6f803de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6f803b9d0 0x7ff6f803de90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff71413d1d0 tx=0x7ff704006d90 comp rx=0 tx=0).stop 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 10 monclient: shutdown 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 msgr2=0x7ff71413c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 0x7ff71413c340 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff71000c480 tx=0x7ff71000c940 comp rx=0 tx=0).stop 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 shutdown_connections 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff6f803b9d0 0x7ff6f803de90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:18.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71413e4a0 0x7ff7141369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:18.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141383b0 0x7ff71413a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:18.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 --2- 172.21.15.17:0/1460275946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff7141378c0 0x7ff71413c340 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:18.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 >> 172.21.15.17:0/1460275946 conn(0x7ff7140aac40 msgr2=0x7ff71406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:18.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 shutdown_connections 2024-09-15T21:24:18.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:18.167+0000 7ff71bb69640 1 -- 172.21.15.17:0/1460275946 wait complete. 2024-09-15T21:24:20.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:20.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:20.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:20.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:20.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:20.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:20.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:20.070+0000 7fe6c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:21.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:21.066+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:21.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:21.066+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:21.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:21.066+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:21.190 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient: get_monmap_and_config 2024-09-15T21:24:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient: build_initial_monmap 2024-09-15T21:24:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient: monmap: 2024-09-15T21:24:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:21.323749+0000 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:21.323749+0000 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 Processor -- start 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 -- start start 2024-09-15T21:24:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient: init 2024-09-15T21:24:21.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:21.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:24:21.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f11fc0a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 0x7f11fc13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): picked mon.noname-b con 0x7f11fc09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11fc0a1380 0x7f11fc13d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): picked mon.noname-a con 0x7f11fc0a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11fc0a1bf0 con 0x7f11fc0a0890 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc073a50 con 0x7f11fc0a1380 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc073df0 con 0x7f11fc09ffc0 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f1200c06640 10 monclient(hunting): authenticate will time out at 1384.719275s 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11fad9d640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f11fc0a1380 msgr2=0x7f11fc13d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:21.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11fad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11fc0a1380 0x7f11fc13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 0x7f11fc13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 0x7f11fc13aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52806/0 (socket says 172.21.15.17:52806) 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 -- 172.21.15.17:0/2059422891 learned_addr learned my addr 172.21.15.17:0/2059422891 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): get_auth_request con 0x7f11fc09ffc0 auth_method 0 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_done global_id 44990 payload 274 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 -- 172.21.15.17:0/2059422891 >> v2:172.21.15.17:3300/0 conn(0x7f11fc0a1380 msgr2=0x7f11fc13d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 --2- 172.21.15.17:0/2059422891 >> v2:172.21.15.17:3300/0 conn(0x7f11fc0a1380 0x7f11fc13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 -- 172.21.15.17:0/2059422891 >> v1:172.21.15.119:6789/0 conn(0x7f11fc0a0890 legacy=0x7f11fc0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 monclient: found mon.noname-b 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.319+0000 7f11f9d9b640 1 -- 172.21.15.17:0/2059422891 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc0a1ed0 con 0x7f11fc09ffc0 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f9d9b640 1 --2- 172.21.15.17:0/2059422891 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 0x7f11fc13aac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009870 tx=0x7f11f001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f224f32d67225e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 1 -- 172.21.15.17:0/2059422891 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00b0070 con 0x7f11fc09ffc0 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: dump: 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:21.332 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: _finish_auth 0 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 5 monclient: authenticate success, global_id 44990 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:21.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 1 -- 172.21.15.17:0/2059422891 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f0004230 con 0x7f11fc09ffc0 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 1 -- 172.21.15.17:0/2059422891 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0003620 con 0x7f11fc09ffc0 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f11f959a640 10 monclient: dump: 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:21.335 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 4 set_mon_vals no callback set 2024-09-15T21:24:21.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 10 monclient: shutdown 2024-09-15T21:24:21.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 -- 172.21.15.17:0/2059422891 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 msgr2=0x7f11fc13aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 --2- 172.21.15.17:0/2059422891 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 0x7f11fc13aac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009870 tx=0x7f11f001c440 comp rx=0 tx=0).stop 2024-09-15T21:24:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 -- 172.21.15.17:0/2059422891 shutdown_connections 2024-09-15T21:24:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 --2- 172.21.15.17:0/2059422891 >> v2:172.21.15.17:3300/0 conn(0x7f11fc0a1380 0x7f11fc13d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 --2- 172.21.15.17:0/2059422891 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc09ffc0 0x7f11fc13aac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 -- 172.21.15.17:0/2059422891 >> 172.21.15.17:0/2059422891 conn(0x7f11fc0a66a0 msgr2=0x7f11fc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:21.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 -- 172.21.15.17:0/2059422891 shutdown_connections 2024-09-15T21:24:21.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.323+0000 7f1200c06640 1 -- 172.21.15.17:0/2059422891 wait complete. 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: build_initial_monmap 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: monmap: 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:21.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 Processor -- start 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- start start 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: init 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11fc09ffc0 0x7f11fc1cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): picked mon.noname-a con 0x7f11fc09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 0x7f11fc1328b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): picked mon.noname-c con 0x7f11fc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11fc0a1380 0x7f11fc139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): picked mon.noname-b con 0x7f11fc0a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:21.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc140100 con 0x7f11fc09ffc0 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc13ff80 con 0x7f11fc0a1380 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc140280 con 0x7f11fc0a0890 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient(hunting): authenticate will time out at 1384.725674s 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11fa59c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11fc09ffc0 msgr2=0x7f11fc1cd500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11fa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11fc09ffc0 0x7f11fc1cd500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 0x7f11fc1328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 0x7f11fc1328b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52808/0 (socket says 172.21.15.17:52808) 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 -- 172.21.15.17:0/3770556015 learned_addr learned my addr 172.21.15.17:0/3770556015 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): get_auth_request con 0x7f11fc0a0890 auth_method 0 2024-09-15T21:24:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11fad9d640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11fc0a1380 0x7f11fc139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient(hunting): handle_auth_done global_id 44993 payload 824 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 -- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11fc0a1380 msgr2=0x7f11fc139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11fc0a1380 0x7f11fc139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 -- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11fc09ffc0 msgr2=0x7f11fc1cd500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11fc09ffc0 0x7f11fc1cd500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 monclient: found mon.noname-c 2024-09-15T21:24:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 -- 172.21.15.17:0/3770556015 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc09d0f0 con 0x7f11fc0a0890 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11fad9d640 10 monclient: get_auth_request con 0x7f11fc0a1380 auth_method 0 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11fad9d640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11fc0a1380 0x7f11fc139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f9d9b640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 0x7f11fc1328b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f11f00ae040 tx=0x7f11f00191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00ba020 con 0x7f11fc0a0890 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: dump: 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f00043e0 con 0x7f11fc0a0890 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 5 monclient: authenticate success, global_id 44993 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _renew_subs 2024-09-15T21:24:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc11cf60 con 0x7f11fc0a0890 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00b0040 con 0x7f11fc0a0890 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 10 monclient: dump: 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11f8d99640 4 set_mon_vals no callback set 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _renew_subs 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc13b5a0 con 0x7f11fc0a0890 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc0a0c70 con 0x7f11fc0a0890 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.327+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f11f00bd020 con 0x7f11fc0a0890 2024-09-15T21:24:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.331+0000 7f11eaffd640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11dc03b9a0 0x7f11dc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.331+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7f11f00eaff0 con 0x7f11fc0a0890 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.331+0000 7f11fa59c640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11dc03b9a0 0x7f11dc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.331+0000 7f11fa59c640 10 monclient: get_auth_request con 0x7f11dc03b9a0 auth_method 0 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.331+0000 7f11fa59c640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11dc03b9a0 0x7f11dc03de60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000ee0 tx=0x7f11ec0079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.335+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f11f00c4070 con 0x7f11fc0a0890 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.335+0000 7f11eaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.335+0000 7f11eaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.563+0000 7f1200c06640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.563+0000 7f1200c06640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.563+0000 7f1200c06640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.563+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f11fc06d020 con 0x7f11fc0a0890 2024-09-15T21:24:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f11eaffd640 1 -- 172.21.15.17:0/3770556015 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f11fc09d0f0 con 0x7f11fc0a0890 2024-09-15T21:24:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f11eaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f11eaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:21.571 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11dc03b9a0 msgr2=0x7f11dc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11dc03b9a0 0x7f11dc03de60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000ee0 tx=0x7f11ec0079e0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 10 monclient: shutdown 2024-09-15T21:24:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 msgr2=0x7f11fc1328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 0x7f11fc1328b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f11f00ae040 tx=0x7f11f00191e0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 shutdown_connections 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11dc03b9a0 0x7f11dc03de60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11fc0a1380 0x7f11fc139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11fc0a0890 0x7f11fc1328b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 --2- 172.21.15.17:0/3770556015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11fc09ffc0 0x7f11fc1cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 >> 172.21.15.17:0/3770556015 conn(0x7f11fc0a66a0 msgr2=0x7f11fc135990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 shutdown_connections 2024-09-15T21:24:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:21.567+0000 7f1200c06640 1 -- 172.21.15.17:0/3770556015 wait complete. 2024-09-15T21:24:22.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:22.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:22.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:22.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:22.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:22.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:23.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:23.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:23.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:23.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:24.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:24.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:24.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:24.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:24.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:24.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:24.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient: get_monmap_and_config 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient: build_initial_monmap 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient: monmap: 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:24.723860+0000 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:24.723860+0000 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:24.726 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 Processor -- start 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 -- start start 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient: init 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 0x7f4764137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): picked mon.noname-b con 0x7f47641378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f47641381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47640a5210 0x7f47640a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): picked mon.noname-a con 0x7f47640a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47640a1d00 con 0x7f47641381e0 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f47640a1fe0 con 0x7f47640a5210 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f47640a22c0 con 0x7f47641378c0 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f476a1eb640 10 monclient(hunting): authenticate will time out at 1388.119537s 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f4763fff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f47640a5210 msgr2=0x7f47640a56c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 0x7f4764137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f4763fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47640a5210 0x7f47640a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 0x7f4764137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52820/0 (socket says 172.21.15.17:52820) 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 1 -- 172.21.15.17:0/3105373102 learned_addr learned my addr 172.21.15.17:0/3105373102 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): get_auth_request con 0x7f47641378c0 auth_method 0 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.719+0000 7f47637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 10 monclient(hunting): handle_auth_done global_id 44996 payload 274 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 1 -- 172.21.15.17:0/3105373102 >> v2:172.21.15.17:3300/0 conn(0x7f47640a5210 msgr2=0x7f47640a56c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 1 --2- 172.21.15.17:0/3105373102 >> v2:172.21.15.17:3300/0 conn(0x7f47640a5210 0x7f47640a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 1 -- 172.21.15.17:0/3105373102 >> v1:172.21.15.119:6789/0 conn(0x7f47641381e0 legacy=0x7f47640a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 1 monclient: found mon.noname-b 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 1 -- 172.21.15.17:0/3105373102 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47640a5c00 con 0x7f47641378c0 2024-09-15T21:24:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47637fe640 1 --2- 172.21.15.17:0/3105373102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 0x7f4764137ca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4754009870 tx=0x7f475401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c68d51b9b792cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 1 -- 172.21.15.17:0/3105373102 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540b0070 con 0x7f47641378c0 2024-09-15T21:24:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: dump: 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 1 -- 172.21.15.17:0/3105373102 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47540ab600 con 0x7f47641378c0 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 5 monclient: authenticate success, global_id 44996 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 1 -- 172.21.15.17:0/3105373102 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540ab9d0 con 0x7f47641378c0 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f47627fc640 10 monclient: dump: 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 4 set_mon_vals no callback set 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 10 monclient: shutdown 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 -- 172.21.15.17:0/3105373102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 msgr2=0x7f4764137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 --2- 172.21.15.17:0/3105373102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 0x7f4764137ca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4754009870 tx=0x7f475401c440 comp rx=0 tx=0).stop 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 -- 172.21.15.17:0/3105373102 shutdown_connections 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 --2- 172.21.15.17:0/3105373102 >> v2:172.21.15.17:3300/0 conn(0x7f47640a5210 0x7f47640a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 --2- 172.21.15.17:0/3105373102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641378c0 0x7f4764137ca0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 -- 172.21.15.17:0/3105373102 >> 172.21.15.17:0/3105373102 conn(0x7f47640aac40 msgr2=0x7f47640a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 -- 172.21.15.17:0/3105373102 shutdown_connections 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 1 -- 172.21.15.17:0/3105373102 wait complete. 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 10 monclient: build_initial_monmap 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.723+0000 7f476a1eb640 10 monclient: monmap: 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 Processor -- start 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- start start 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: init 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640a5210 0x7f47641cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): picked mon.noname-b con 0x7f47640a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47641378c0 0x7f4764136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): picked mon.noname-a con 0x7f47641378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 0x7f476413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): picked mon.noname-c con 0x7f47641381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f476406f340 con 0x7f47641378c0 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f476406f1c0 con 0x7f47640a5210 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f47641425c0 con 0x7f47641381e0 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient(hunting): authenticate will time out at 1388.124752s 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4762ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47641378c0 msgr2=0x7f4764136c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4762ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47641378c0 0x7f4764136c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 0x7f476413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 0x7f476413cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52822/0 (socket says 172.21.15.17:52822) 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 -- 172.21.15.17:0/2159689553 learned_addr learned my addr 172.21.15.17:0/2159689553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): get_auth_request con 0x7f47641381e0 auth_method 0 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640a5210 0x7f47641cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 10 monclient(hunting): get_auth_request con 0x7f47640a5210 auth_method 0 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient(hunting): handle_auth_done global_id 44999 payload 824 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 -- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640a5210 msgr2=0x7f47641cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640a5210 0x7f47641cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 -- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47641378c0 msgr2=0x7f4764136c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47641378c0 0x7f4764136c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 monclient: found mon.noname-c 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 -- 172.21.15.17:0/2159689553 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4764121630 con 0x7f47641381e0 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4763fff640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 0x7f476413cee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f475c00be60 tx=0x7f475c00bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c016430 con 0x7f47641381e0 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: dump: 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:24.745 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: _finish_auth 0 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 5 monclient: authenticate success, global_id 44999 2024-09-15T21:24:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _renew_subs 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475c010030 con 0x7f47641381e0 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641d09d0 con 0x7f47641381e0 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c016850 con 0x7f47641381e0 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 10 monclient: dump: 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _renew_subs 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f4761ffb640 4 set_mon_vals no callback set 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f476413f470 con 0x7f47641381e0 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47640a56c0 con 0x7f47641381e0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f475c0153e0 con 0x7f47641381e0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f475003b970 0x7f475003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(195..195 src has 1..195) ==== 10042+0+0 (secure 0 0 0) 0x7f475c051500 con 0x7f47641381e0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f475003b970 0x7f475003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.727+0000 7f47637fe640 10 monclient: get_auth_request con 0x7f475003b970 auth_method 0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.731+0000 7f47637fe640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f475003b970 0x7f475003de30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47540ad000 tx=0x7f47540a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.731+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f475c04ea80 con 0x7f47641381e0 2024-09-15T21:24:24.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.731+0000 7f474bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:24.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.731+0000 7f474bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:24.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:24.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:24.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:24.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4764121220 con 0x7f47641381e0 2024-09-15T21:24:24.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f474bfff640 1 -- 172.21.15.17:0/2159689553 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f475c013cf0 con 0x7f47641381e0 2024-09-15T21:24:24.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f474bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:24.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f474bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:24.973 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f475003b970 msgr2=0x7f475003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f475003b970 0x7f475003de30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47540ad000 tx=0x7f47540a6000 comp rx=0 tx=0).stop 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 10 monclient: shutdown 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 msgr2=0x7f476413cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.967+0000 7f476a1eb640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 0x7f476413cee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f475c00be60 tx=0x7f475c00bf50 comp rx=0 tx=0).stop 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 shutdown_connections 2024-09-15T21:24:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f475003b970 0x7f475003de30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f47641381e0 0x7f476413cee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47641378c0 0x7f4764136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 --2- 172.21.15.17:0/2159689553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640a5210 0x7f47641cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 >> 172.21.15.17:0/2159689553 conn(0x7f47640aac40 msgr2=0x7f476406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 shutdown_connections 2024-09-15T21:24:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:24.971+0000 7f476a1eb640 1 -- 172.21.15.17:0/2159689553 wait complete. 2024-09-15T21:24:25.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:25.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:25.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:25.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:25.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:25.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:26.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:26.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:26.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:26.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:26.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:27.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:27.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:27.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:27.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:27.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:27.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:27.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:28.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:28.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:28.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:28.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:28.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:28.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:28.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient: get_monmap_and_config 2024-09-15T21:24:28.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient: build_initial_monmap 2024-09-15T21:24:28.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient: monmap: 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:28.124172+0000 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:28.124172+0000 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:28.126 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:28.127 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:28.127 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:28.127 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:28.127 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:28.127 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:28.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 Processor -- start 2024-09-15T21:24:28.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 -- start start 2024-09-15T21:24:28.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient: init 2024-09-15T21:24:28.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:28.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f67d01340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): picked mon.noname-c con 0x7f67d009d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): picked mon.noname-b con 0x7f67d0139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d009e1b0 con 0x7f67d01340c0 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:28.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f67d009d520 con 0x7f67d009d6c0 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f67d009e490 con 0x7f67d0139390 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d811a640 10 monclient(hunting): authenticate will time out at 1391.519735s 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35460/0 (socket says 172.21.15.17:35460) 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 learned_addr learned my addr 172.21.15.17:0/3875296137 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 10 monclient(hunting): get_auth_request con 0x7f67d0139390 auth_method 0 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.119+0000 7f67d66b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:28.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 10 monclient(hunting): get_auth_request con 0x7f67d009d6c0 auth_method 0 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:28.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:28.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:28.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:28.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.123+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 10 monclient(hunting): get_auth_request con 0x7f67d0139390 auth_method 0 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 10 monclient(hunting): get_auth_request con 0x7f67d009d6c0 auth_method 0 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.327+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 10 monclient(hunting): get_auth_request con 0x7f67d0139390 auth_method 0 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 10 monclient(hunting): get_auth_request con 0x7f67d009d6c0 auth_method 0 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:28.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:28.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:28.731+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:24:29.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:29.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:29.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:29.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:29.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:29.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 10 monclient(hunting): get_auth_request con 0x7f67d0139390 auth_method 0 2024-09-15T21:24:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d66b7640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:24:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 10 monclient(hunting): get_auth_request con 0x7f67d009d6c0 auth_method 0 2024-09-15T21:24:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:24:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:29.535+0000 7f67d56b5640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:24:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:30.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:30.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:30.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:30.070+0000 7fe6c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:31.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:31.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:31.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): tick 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 msgr2=0x7f67d013b780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67d0139390 0x7f67d013b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 msgr2=0x7f67d0138e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67d0138e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.119:6789/0 conn(0x7f67d01340c0 legacy=0x7f67d01344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient: _add_conns ranks=[3,0] 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 0x7f67b4005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): picked mon.noname-a con 0x7f67b40036f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f67b4006240 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d009e770 con 0x7f67b4006240 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f67d009d520 con 0x7f67b40036f0 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67c77fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4006240 legacy=0x7f67b4008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 msgr2=0x7f67b4005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.123+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 0x7f67b4005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.327+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 msgr2=0x7f67b4005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:31.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.327+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4006240 legacy=0x7f67b4008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:31.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.327+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 0x7f67b4005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:24:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.731+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4006240 legacy=0x7f67b4008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.731+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 msgr2=0x7f67b4005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:31.731+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 0x7f67b4005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:24:32.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.070+0000 7fe6c77fe640 10 monclient: tick 2024-09-15T21:24:32.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.070+0000 7fe6c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:32.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.070+0000 7fe6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:32.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.354+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 added bucket h11 type host to location {rack=r1} v196) ==== 168+0+0 (secure 0 0 0) 0x7fe6d8007a90 con 0x7fe6e0136430 2024-09-15T21:24:32.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.354+0000 7fe6dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:32.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.354+0000 7fe6dcff9640 10 monclient: _finish_command 2 = system:0 added bucket h11 type host to location {rack=r1} 2024-09-15T21:24:32.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.462+0000 7fe6e6614640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:32.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.462+0000 7fe6e6614640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:32.467 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.462+0000 7fe6e6614640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.462+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fe6e01355a0 con 0x7fe6e0136430 2024-09-15T21:24:32.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6dcff9640 1 -- 172.21.15.119:0/2858773870 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h11' already exists v196) ==== 147+0+0 (secure 0 0 0) 0x7fe6d804ec20 con 0x7fe6e0136430 2024-09-15T21:24:32.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6dcff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:32.468 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6dcff9640 10 monclient: _finish_command 3 = system:0 bucket 'h11' already exists 2024-09-15T21:24:32.468 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h11' already exists 2024-09-15T21:24:32.470 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6cc03b9c0 msgr2=0x7fe6cc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6cc03b9c0 0x7fe6cc03de80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00ae000 tx=0x7fe6d0008040 comp rx=0 tx=0).stop 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 10 monclient: shutdown 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 msgr2=0x7fe6e013c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 0x7fe6e013c520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800cb40 tx=0x7fe6d800a060 comp rx=0 tx=0).stop 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 shutdown_connections 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6cc03b9c0 0x7fe6cc03de80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0136430 0x7fe6e013c520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6e0135fd0 0x7fe6e0134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 --2- 172.21.15.119:0/2858773870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6e0135bf0 0x7fe6e0133f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.471 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 >> 172.21.15.119:0/2858773870 conn(0x7fe6e00a66e0 msgr2=0x7fe6e00a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:32.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 shutdown_connections 2024-09-15T21:24:32.472 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.466+0000 7fe6e6614640 1 -- 172.21.15.119:0/2858773870 wait complete. 2024-09-15T21:24:32.486 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:32.486 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h12 host rack=r1 2024-09-15T21:24:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:32.535+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 msgr2=0x7f67b4005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:32.535+0000 7f67d56b5640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4006240 legacy=0x7f67b4008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:32.535+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 0x7f67b4005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:24:32.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient: get_monmap_and_config 2024-09-15T21:24:32.608 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient: build_initial_monmap 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient: monmap: 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:32.608912+0000 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:32.608912+0000 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:32.609 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:32.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 1 Processor -- start 2024-09-15T21:24:32.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 1 -- start start 2024-09-15T21:24:32.611 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient: init 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdc9c0a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc9c0a41b0 0x7fdc9c0a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient(hunting): picked mon.noname-c con 0x7fdc9c0a41b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdc9c13cb80 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc9c1383e0 con 0x7fdc9c13cb80 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc9c0a19d0 con 0x7fdc9c0a7ac0 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdc9c0a7ea0 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca40f2640 10 monclient(hunting): authenticate will time out at 1391.405583s 2024-09-15T21:24:32.612 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc9c0a41b0 0x7fdc9c0a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc9c0a41b0 0x7fdc9c0a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48798/0 (socket says 172.21.15.119:48798) 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 -- 172.21.15.119:0/1942852080 learned_addr learned my addr 172.21.15.119:0/1942852080 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): get_auth_request con 0x7fdc9c0a41b0 auth_method 0 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca268f640 1 -- 172.21.15.119:0/1942852080 >> v1:172.21.15.17:6789/0 conn(0x7fdc9c13cb80 legacy=0x7fdc9c13ef70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient(hunting): handle_auth_done global_id 84187 payload 274 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 -- 172.21.15.119:0/1942852080 >> v1:172.21.15.119:6789/0 conn(0x7fdc9c0a7ac0 legacy=0x7fdc9c137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 -- 172.21.15.119:0/1942852080 >> v1:172.21.15.17:6789/0 conn(0x7fdc9c13cb80 legacy=0x7fdc9c13ef70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:32.613 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:32.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 monclient: found mon.noname-c 2024-09-15T21:24:32.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:32.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.614 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.606+0000 7fdca168d640 1 -- 172.21.15.119:0/1942852080 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c0a6960 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca168d640 1 --2- 172.21.15.119:0/1942852080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c0a45c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009e80 tx=0x7fdc9001c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bca6993dc8b0801 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:32.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 1 -- 172.21.15.119:0/1942852080 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900a5700 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.618 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: dump: 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:32.619 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: _finish_auth 0 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: _check_auth_tickets 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 1 -- 172.21.15.119:0/1942852080 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc900a58a0 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 1 -- 172.21.15.119:0/1942852080 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900a5c70 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 5 monclient: authenticate success, global_id 84187 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 4 set_mon_vals no callback set 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:32.620 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca0e8c640 10 monclient: dump: 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: shutdown 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/1942852080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 msgr2=0x7fdc9c0a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:32.621 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 --2- 172.21.15.119:0/1942852080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c0a45c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009e80 tx=0x7fdc9001c8a0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/1942852080 shutdown_connections 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 --2- 172.21.15.119:0/1942852080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c0a45c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/1942852080 >> 172.21.15.119:0/1942852080 conn(0x7fdc9c0a6fd0 msgr2=0x7fdc9c0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/1942852080 shutdown_connections 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/1942852080 wait complete. 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: build_initial_monmap 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: monmap: 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:32.622 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 Processor -- start 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- start start 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: init 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c131210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): picked mon.noname-b con 0x7fdc9c0a41b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc9c0a7ac0 0x7fdc9c131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): picked mon.noname-a con 0x7fdc9c0a7ac0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:32.623 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc9c13cb80 0x7fdc9c1cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): picked mon.noname-c con 0x7fdc9c13cb80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c143140 con 0x7fdc9c0a7ac0 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c142fc0 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9c1432c0 con 0x7fdc9c13cb80 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient(hunting): authenticate will time out at 1391.410530s 2024-09-15T21:24:32.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c131210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca168d640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc9c0a7ac0 msgr2=0x7fdc9c131750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c131210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48804/0 (socket says 172.21.15.119:48804) 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 -- 172.21.15.119:0/707890612 learned_addr learned my addr 172.21.15.119:0/707890612 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): get_auth_request con 0x7fdc9c0a41b0 auth_method 0 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca168d640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc9c0a7ac0 0x7fdc9c131750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:32.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient(hunting): handle_auth_done global_id 84193 payload 824 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 -- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc9c13cb80 msgr2=0x7fdc9c1cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc9c13cb80 0x7fdc9c1cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 -- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc9c0a7ac0 msgr2=0x7fdc9c131750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc9c0a7ac0 0x7fdc9c131750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient: _finish_hunting 0 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 monclient: found mon.noname-b 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 -- 172.21.15.119:0/707890612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c136090 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca1e8e640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c131210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c530 tx=0x7fdc9400c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94017070 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:32.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: dump: 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:32.627 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc940052d0 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 5 monclient: authenticate success, global_id 84193 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: _renew_subs 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9c140060 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94012870 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9a7fc640 10 monclient: dump: 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: _renew_subs 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdc9b7fe640 4 set_mon_vals no callback set 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.610+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9c133d30 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca40f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fdc9401a070 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca40f2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca40f2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9c0a45c0 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdc9a7fc640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc8803b970 0x7fdc8803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(196..196 src has 1..196) ==== 10255+0+0 (secure 0 0 0) 0x7fdc940503a0 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca168d640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc8803b970 0x7fdc8803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca168d640 10 monclient: get_auth_request con 0x7fdc8803b970 auth_method 0 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.614+0000 7fdca168d640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc8803b970 0x7fdc8803de30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc900ae000 tx=0x7fdc900a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:32.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.618+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fdc9404d850 con 0x7fdc9c0a41b0 2024-09-15T21:24:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.618+0000 7fdc9a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.618+0000 7fdc9a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:32.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.790+0000 7fdca40f2640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:32.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.790+0000 7fdca40f2640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:32.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.790+0000 7fdca40f2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:32.796 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:32.790+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fdc9c136eb0 con 0x7fdc9c0a41b0 2024-09-15T21:24:33.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.786+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 added bucket h12 type host to location {rack=r1} v198) ==== 168+0+0 (secure 0 0 0) 0x7fdc94003d60 con 0x7fdc9c0a41b0 2024-09-15T21:24:33.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.786+0000 7fdc9a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:33.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.786+0000 7fdc9a7fc640 10 monclient: _finish_command 2 = system:0 added bucket h12 type host to location {rack=r1} 2024-09-15T21:24:33.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdca40f2640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:33.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdca40f2640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:33.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdca40f2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:33.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fdc9c06a760 con 0x7fdc9c0a41b0 2024-09-15T21:24:33.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdc9a7fc640 1 -- 172.21.15.119:0/707890612 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h12' already exists v198) ==== 147+0+0 (secure 0 0 0) 0x7fdc94019cd0 con 0x7fdc9c0a41b0 2024-09-15T21:24:33.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdc9a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:33.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.894+0000 7fdc9a7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h12' already exists 2024-09-15T21:24:33.900 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h12' already exists 2024-09-15T21:24:33.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc8803b970 msgr2=0x7fdc8803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc8803b970 0x7fdc8803de30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc900ae000 tx=0x7fdc900a7070 comp rx=0 tx=0).stop 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 10 monclient: shutdown 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 msgr2=0x7fdc9c131210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c131210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c530 tx=0x7fdc9400c9f0 comp rx=0 tx=0).stop 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 shutdown_connections 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdc8803b970 0x7fdc8803de30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdc9c13cb80 0x7fdc9c1cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc9c0a7ac0 0x7fdc9c131750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:33.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 --2- 172.21.15.119:0/707890612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc9c0a41b0 0x7fdc9c131210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:33.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 >> 172.21.15.119:0/707890612 conn(0x7fdc9c0a6fd0 msgr2=0x7fdc9c0a99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:33.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 shutdown_connections 2024-09-15T21:24:33.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:33.898+0000 7fdca40f2640 1 -- 172.21.15.119:0/707890612 wait complete. 2024-09-15T21:24:33.916 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:33.916 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h13 host rack=r1 2024-09-15T21:24:34.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient: get_monmap_and_config 2024-09-15T21:24:34.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient: monmap: 2024-09-15T21:24:34.038 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:34.038 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.038 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:34.038601+0000 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:34.038601+0000 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:34.039 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 Processor -- start 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 -- start start 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient: init 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd55c0a0a50 0x7fd55c0a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): picked mon.noname-a con 0x7fd55c0a0a50 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:34.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd55c09ea80 0x7fd55c0a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): picked mon.noname-c con 0x7fd55c09ea80 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd55c13c170 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c0741f0 con 0x7fd55c13c170 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c0a18b0 con 0x7fd55c0a0a50 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c0a35a0 con 0x7fd55c09ea80 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd560dbb640 10 monclient(hunting): authenticate will time out at 1392.835272s 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd55a59c640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fd55c0a0a50 msgr2=0x7fd55c0a0e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd55a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd55c0a0a50 0x7fd55c0a0e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd55c09ea80 0x7fd55c0a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd55c09ea80 0x7fd55c0a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48818/0 (socket says 172.21.15.119:48818) 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 -- 172.21.15.119:0/224489889 learned_addr learned my addr 172.21.15.119:0/224489889 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): get_auth_request con 0x7fd55c09ea80 auth_method 0 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient(hunting): handle_auth_done global_id 84196 payload 274 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 -- 172.21.15.119:0/224489889 >> v2:172.21.15.17:3300/0 conn(0x7fd55c0a0a50 msgr2=0x7fd55c0a0e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 --2- 172.21.15.119:0/224489889 >> v2:172.21.15.17:3300/0 conn(0x7fd55c0a0a50 0x7fd55c0a0e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 -- 172.21.15.119:0/224489889 >> v1:172.21.15.119:6789/0 conn(0x7fd55c13c170 legacy=0x7fd55c13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 monclient: found mon.noname-c 2024-09-15T21:24:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.034+0000 7fd559d9b640 1 -- 172.21.15.119:0/224489889 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c09d800 con 0x7fd55c09ea80 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd559d9b640 1 --2- 172.21.15.119:0/224489889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c0a1370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd550009870 tx=0x7fd55001c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92eab0aa10407162 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 1 -- 172.21.15.119:0/224489889 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500b1070 con 0x7fd55c09ea80 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: dump: 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.049 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 5 monclient: authenticate success, global_id 84196 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 1 -- 172.21.15.119:0/224489889 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5500ac630 con 0x7fd55c09ea80 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:34.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 1 -- 172.21.15.119:0/224489889 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500a8050 con 0x7fd55c09ea80 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 4 set_mon_vals no callback set 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd55959a640 10 monclient: dump: 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.051 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 10 monclient: shutdown 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 -- 172.21.15.119:0/224489889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 msgr2=0x7fd55c0a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 --2- 172.21.15.119:0/224489889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c0a1370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd550009870 tx=0x7fd55001c3d0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 -- 172.21.15.119:0/224489889 shutdown_connections 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 --2- 172.21.15.119:0/224489889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c0a1370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 --2- 172.21.15.119:0/224489889 >> v2:172.21.15.17:3300/0 conn(0x7fd55c0a0a50 0x7fd55c0a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 -- 172.21.15.119:0/224489889 >> 172.21.15.119:0/224489889 conn(0x7fd55c0a66e0 msgr2=0x7fd55c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 -- 172.21.15.119:0/224489889 shutdown_connections 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.038+0000 7fd560dbb640 1 -- 172.21.15.119:0/224489889 wait complete. 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: monmap: 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:34.053 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 Processor -- start 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- start start 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: init 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c13ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): picked mon.noname-b con 0x7fd55c09ea80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c0a0a50 0x7fd55c13b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): picked mon.noname-c con 0x7fd55c0a0a50 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c13c170 0x7fd55c141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): picked mon.noname-a con 0x7fd55c13c170 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c09ccd0 con 0x7fd55c13c170 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0a1d20 con 0x7fd55c09ea80 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c09ce50 con 0x7fd55c0a0a50 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient(hunting): authenticate will time out at 1392.841589s 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c13ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c13ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48820/0 (socket says 172.21.15.119:48820) 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 -- 172.21.15.119:0/277767640 learned_addr learned my addr 172.21.15.119:0/277767640 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): get_auth_request con 0x7fd55c09ea80 auth_method 0 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55ad9d640 1 -- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c13c170 msgr2=0x7fd55c141250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55ad9d640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c13c170 0x7fd55c141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd559d9b640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c0a0a50 0x7fd55c13b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient(hunting): handle_auth_done global_id 84199 payload 824 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 -- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c0a0a50 msgr2=0x7fd55c13b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c0a0a50 0x7fd55c13b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 -- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c13c170 msgr2=0x7fd55c141250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c13c170 0x7fd55c141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 monclient: found mon.noname-b 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 -- 172.21.15.119:0/277767640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c125710 con 0x7fd55c09ea80 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd559d9b640 10 monclient: get_auth_request con 0x7fd55c0a0a50 auth_method 0 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd559d9b640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c0a0a50 0x7fd55c13b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd55a59c640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c13ab30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00b590 tx=0x7fd54c00ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c013020 con 0x7fd55c09ea80 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: dump: 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.057 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c003df0 con 0x7fd55c09ea80 2024-09-15T21:24:34.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0052b0 con 0x7fd55c09ea80 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 10 monclient: dump: 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.059 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd558d99640 4 set_mon_vals no callback set 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 5 monclient: authenticate success, global_id 84199 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _renew_subs 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c1259f0 con 0x7fd55c09ea80 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _renew_subs 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c125430 con 0x7fd55c09ea80 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd54c016020 con 0x7fd55c09ea80 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.042+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55c13c5c0 con 0x7fd55c09ea80 2024-09-15T21:24:34.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.046+0000 7fd54affd640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd53c03b9a0 0x7fd53c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.046+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 10468+0+0 (secure 0 0 0) 0x7fd54c050f00 con 0x7fd55c09ea80 2024-09-15T21:24:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.046+0000 7fd559d9b640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd53c03b9a0 0x7fd53c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.046+0000 7fd559d9b640 10 monclient: get_auth_request con 0x7fd53c03b9a0 auth_method 0 2024-09-15T21:24:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.046+0000 7fd559d9b640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd53c03b9a0 0x7fd53c03de60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ae000 tx=0x7fd550008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.050+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd54c01d020 con 0x7fd55c09ea80 2024-09-15T21:24:34.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.050+0000 7fd54affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.050+0000 7fd54affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:34.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): tick 2024-09-15T21:24:34.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:24:34.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:24:34.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:24:34.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 msgr2=0x7f67b4005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.17:3300/0 conn(0x7f67b40036f0 0x7f67b4005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4006240 legacy=0x7f67b4008700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 reap_dead start 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67b4009f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): picked mon.noname-c con 0x7f67d009d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f67d01340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f67b4008ed0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b400b010 con 0x7f67b4008ed0 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d009e770 con 0x7f67d01340c0 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 1 -- 172.21.15.17:0/3875296137 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f67d009d520 con 0x7f67d009d6c0 2024-09-15T21:24:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67c77fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d66b7640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4008ed0 legacy=0x7f67b4009330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/3875296137 >> v2:172.21.15.119:3300/0 conn(0x7f67d009d6c0 0x7f67b4009f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): get_auth_request con 0x7f67d009d6c0 auth_method 0 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d4eb4640 1 -- 172.21.15.17:0/3875296137 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845753915 0 0) 0x7f67d009e770 con 0x7f67d01340c0 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d4eb4640 10 monclient(hunting): _negotiate 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d4eb4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d4eb4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d4eb4640 10 monclient(hunting): my global_id is 84205 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d4eb4640 1 -- 172.21.15.17:0/3875296137 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67bc003530 con 0x7f67d01340c0 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_done global_id 84202 payload 274 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.119:6789/0 conn(0x7f67d01340c0 legacy=0x7f67b400a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 >> v1:172.21.15.17:6789/0 conn(0x7f67b4008ed0 legacy=0x7f67b4009330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 1 monclient: found mon.noname-c 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.123+0000 7f67d5eb6640 1 -- 172.21.15.17:0/3875296137 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b400adb0 con 0x7f67d009d6c0 2024-09-15T21:24:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67b4009f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0014e0 tx=0x7f67cc01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=775fce69885369b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 1 -- 172.21.15.17:0/3875296137 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc005b50 con 0x7f67d009d6c0 2024-09-15T21:24:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: dump: 2024-09-15T21:24:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 1 -- 172.21.15.17:0/3875296137 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67cc005cf0 con 0x7f67d009d6c0 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 1 -- 172.21.15.17:0/3875296137 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc003ba0 con 0x7f67d009d6c0 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d811a640 5 monclient: authenticate success, global_id 84202 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d811a640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d811a640 4 set_mon_vals no callback set 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.127+0000 7f67d4eb4640 10 monclient: dump: 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.137 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient: shutdown 2024-09-15T21:24:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- 172.21.15.17:0/3875296137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 msgr2=0x7f67b4009f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67b4009f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0014e0 tx=0x7f67cc01c920 comp rx=0 tx=0).stop 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- 172.21.15.17:0/3875296137 shutdown_connections 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 --2- 172.21.15.17:0/3875296137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67b4009f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- 172.21.15.17:0/3875296137 >> 172.21.15.17:0/3875296137 conn(0x7f67d00a66a0 msgr2=0x7f67d00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- 172.21.15.17:0/3875296137 shutdown_connections 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- 172.21.15.17:0/3875296137 wait complete. 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient: monmap: 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 Processor -- start 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- start start 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient: init 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 0x7f67d01cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): picked mon.noname-c con 0x7f67b4008ed0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67d01308c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): picked mon.noname-b con 0x7f67d009d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d01340c0 0x7f67d011c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): picked mon.noname-a con 0x7f67d01340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f67d00a0890 con 0x7f67d01340c0 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f67d00a27a0 con 0x7f67d009d6c0 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f67d00a0a10 con 0x7f67b4008ed0 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.131+0000 7f67d811a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient(hunting): authenticate will time out at 1397.532013s 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d66b7640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d01340c0 msgr2=0x7f67d011c730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d66b7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d01340c0 0x7f67d011c730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 0x7f67d01cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 0x7f67d01cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35486/0 (socket says 172.21.15.17:35486) 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67d01308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 -- 172.21.15.17:0/1857514943 learned_addr learned my addr 172.21.15.17:0/1857514943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): get_auth_request con 0x7f67b4008ed0 auth_method 0 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): get_auth_request con 0x7f67d009d6c0 auth_method 0 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient(hunting): handle_auth_done global_id 45002 payload 824 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 -- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 msgr2=0x7f67d01308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67d01308c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 -- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d01340c0 msgr2=0x7f67d011c730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d01340c0 0x7f67d011c730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 monclient: found mon.noname-c 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 -- 172.21.15.17:0/1857514943 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d01cc540 con 0x7f67b4008ed0 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d5eb6640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 0x7f67d01cb630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67cc01a550 tx=0x7f67cc0196c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0011d0 con 0x7f67b4008ed0 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: dump: 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67cc004230 con 0x7f67b4008ed0 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc004620 con 0x7f67b4008ed0 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 5 monclient: authenticate success, global_id 45002 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient: _renew_subs 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d011ffc0 con 0x7f67b4008ed0 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c7fff640 4 set_mon_vals no callback set 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 10 monclient: dump: 2024-09-15T21:24:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient: _renew_subs 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d00a44e0 con 0x7f67b4008ed0 2024-09-15T21:24:34.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f67cc0ae070 con 0x7f67b4008ed0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d009d0f0 con 0x7f67b4008ed0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f67b403ce10 0x7f67b403f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(198..198 src has 1..198) ==== 10468+0+0 (secure 0 0 0) 0x7f67cc0eb3c0 con 0x7f67b4008ed0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f67b403ce10 0x7f67b403f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.135+0000 7f67d56b5640 10 monclient: get_auth_request con 0x7f67b403ce10 auth_method 0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.139+0000 7f67d56b5640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f67b403ce10 0x7f67b403f2d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f67d011ef00 tx=0x7f67c0006b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.143+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f67d01cc540 con 0x7f67b4008ed0 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.143+0000 7f67c67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.143+0000 7f67c67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:34.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.210+0000 7fd560dbb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:34.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.210+0000 7fd560dbb640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:34.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.210+0000 7fd560dbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.210+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd55c143500 con 0x7fd55c09ea80 2024-09-15T21:24:34.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67d811a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:34.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67d811a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:34.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67d811a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f67d0131330 con 0x7f67b4008ed0 2024-09-15T21:24:34.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67c67fc640 1 -- 172.21.15.17:0/1857514943 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7f67cc0b6020 con 0x7f67b4008ed0 2024-09-15T21:24:34.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67c67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:34.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.379+0000 7f67c67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:34.386 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":68,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f67b403ce10 msgr2=0x7f67b403f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f67b403ce10 0x7f67b403f2d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f67d011ef00 tx=0x7f67c0006b90 comp rx=0 tx=0).stop 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 10 monclient: shutdown 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 msgr2=0x7f67d01cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 0x7f67d01cb630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67cc01a550 tx=0x7f67cc0196c0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 shutdown_connections 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f67b403ce10 0x7f67b403f2d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d01340c0 0x7f67d011c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d009d6c0 0x7f67d01308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 --2- 172.21.15.17:0/1857514943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f67b4008ed0 0x7f67d01cb630 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 >> 172.21.15.17:0/1857514943 conn(0x7f67d00a66a0 msgr2=0x7f67d01320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 shutdown_connections 2024-09-15T21:24:34.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.383+0000 7f67d811a640 1 -- 172.21.15.17:0/1857514943 wait complete. 2024-09-15T21:24:34.404 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:24:34.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:24:34.404 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:24:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient: get_monmap_and_config 2024-09-15T21:24:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient: monmap: 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:34.514968+0000 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:34.514968+0000 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:34.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 Processor -- start 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 -- start start 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient: init 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 0x7f465c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): picked mon.noname-b con 0x7f465c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f465c09ffc0 0x7f465c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): picked mon.noname-c con 0x7f465c09ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f465c135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f465c0a3130 con 0x7f465c135670 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f465c0a16f0 con 0x7f465c09ffc0 2024-09-15T21:24:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f465c073df0 con 0x7f465c0a0890 2024-09-15T21:24:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f46641ae640 10 monclient(hunting): authenticate will time out at 1397.910399s 2024-09-15T21:24:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 0x7f465c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 0x7f465c0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35502/0 (socket says 172.21.15.17:35502) 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 -- 172.21.15.17:0/89808606 learned_addr learned my addr 172.21.15.17:0/89808606 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): get_auth_request con 0x7f465c0a0890 auth_method 0 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient(hunting): handle_auth_done global_id 45005 payload 274 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 -- 172.21.15.17:0/89808606 >> v2:172.21.15.119:3300/0 conn(0x7f465c09ffc0 msgr2=0x7f465c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 --2- 172.21.15.17:0/89808606 >> v2:172.21.15.119:3300/0 conn(0x7f465c09ffc0 0x7f465c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 -- 172.21.15.17:0/89808606 >> v1:172.21.15.119:6789/0 conn(0x7f465c135670 legacy=0x7f465c135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 monclient: found mon.noname-b 2024-09-15T21:24:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.511+0000 7f4661f4a640 1 -- 172.21.15.17:0/89808606 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c0a1bf0 con 0x7f465c0a0890 2024-09-15T21:24:34.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 --2- 172.21.15.17:0/89808606 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 0x7f465c0a0c70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f465c0a1870 tx=0x7f464c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276a16cd2e5629fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 1 -- 172.21.15.17:0/89808606 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c0b0070 con 0x7f465c0a0890 2024-09-15T21:24:34.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:34.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: dump: 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 5 monclient: authenticate success, global_id 45005 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 1 -- 172.21.15.17:0/89808606 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f464c0ab600 con 0x7f465c0a0890 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 1 -- 172.21.15.17:0/89808606 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c0ab9d0 con 0x7f465c0a0890 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 4 set_mon_vals no callback set 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4660f48640 10 monclient: dump: 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.530 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: shutdown 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- 172.21.15.17:0/89808606 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 msgr2=0x7f465c0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 --2- 172.21.15.17:0/89808606 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 0x7f465c0a0c70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f465c0a1870 tx=0x7f464c01c400 comp rx=0 tx=0).stop 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- 172.21.15.17:0/89808606 shutdown_connections 2024-09-15T21:24:34.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 --2- 172.21.15.17:0/89808606 >> v2:172.21.15.119:3300/0 conn(0x7f465c09ffc0 0x7f465c0a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 --2- 172.21.15.17:0/89808606 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c0a0890 0x7f465c0a0c70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- 172.21.15.17:0/89808606 >> 172.21.15.17:0/89808606 conn(0x7f465c0a66a0 msgr2=0x7f465c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- 172.21.15.17:0/89808606 shutdown_connections 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- 172.21.15.17:0/89808606 wait complete. 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: monmap: 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.532 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 Processor -- start 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- start start 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: init 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c09ffc0 0x7f465c1341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): picked mon.noname-c con 0x7f465c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f465c0a0890 0x7f465c134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): picked mon.noname-b con 0x7f465c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c135670 0x7f465c11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): picked mon.noname-a con 0x7f465c135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f465c140690 con 0x7f465c135670 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f465c140510 con 0x7f465c0a0890 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f465c140810 con 0x7f465c09ffc0 2024-09-15T21:24:34.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient(hunting): authenticate will time out at 1397.915126s 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f466274b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c135670 msgr2=0x7f465c11c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f466274b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c135670 0x7f465c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c09ffc0 0x7f465c1341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c09ffc0 0x7f465c1341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35512/0 (socket says 172.21.15.17:35512) 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 -- 172.21.15.17:0/1824604353 learned_addr learned my addr 172.21.15.17:0/1824604353 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): get_auth_request con 0x7f465c09ffc0 auth_method 0 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient(hunting): handle_auth_done global_id 45008 payload 824 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661749640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f465c0a0890 0x7f465c134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f465c0a0890 msgr2=0x7f465c134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f465c0a0890 0x7f465c134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c135670 msgr2=0x7f465c11c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c135670 0x7f465c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 monclient: found mon.noname-c 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c072820 con 0x7f465c09ffc0 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661749640 10 monclient: get_auth_request con 0x7f465c0a0890 auth_method 0 2024-09-15T21:24:34.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661749640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f465c0a0890 0x7f465c134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f4661f4a640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c09ffc0 0x7f465c1341f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f464c01c230 tx=0x7f464c01ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c0ba020 con 0x7f465c09ffc0 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: dump: 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.537 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f464c0b6620 con 0x7f465c09ffc0 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 5 monclient: authenticate success, global_id 45008 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: _renew_subs 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c0a56e0 con 0x7f465c09ffc0 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c0b0040 con 0x7f465c09ffc0 2024-09-15T21:24:34.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46527fc640 10 monclient: dump: 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.515+0000 7f46537fe640 4 set_mon_vals no callback set 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 10 monclient: _renew_subs 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c11ed00 con 0x7f465c09ffc0 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 0x7f465c11ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f465c11d340 con 0x7f465c11ca20 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 0x7f465c11ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient: get_auth_request con 0x7f465c11ca20 auth_method 0 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 10 monclient(hunting): handle_auth_done global_id 45011 payload 824 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f464c0bd020 con 0x7f465c09ffc0 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f465c11ef40 con 0x7f465c11ca20 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f4661749640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 0x7f465c11ce00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f465800c6f0 tx=0x7f465800cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f464003b9a0 0x7f464003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(198..198 src has 1..198) ==== 10468+0+0 (secure 0 0 0) 0x7f464c0eb440 con 0x7f465c09ffc0 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658015070 con 0x7f465c11ca20 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 10 monclient: dump: 2024-09-15T21:24:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f466274b640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f464003b9a0 0x7f464003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4658007890 con 0x7f465c11ca20 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 msgr2=0x7f465c11ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 0x7f465c11ce00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f465800c6f0 tx=0x7f465800cbb0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 msgr2=0x7f465c11ce00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f46527fc640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c11ca20 0x7f465c11ce00 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f465800c6f0 tx=0x7f465800cbb0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f466274b640 10 monclient: get_auth_request con 0x7f464003b9a0 auth_method 0 2024-09-15T21:24:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.519+0000 7f466274b640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f464003b9a0 0x7f464003de60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4654007fd0 tx=0x7f4654007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f46641ae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:34.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f46641ae640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 0x7f465c137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f46641ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f465c11d7e0 con 0x7f465c137ab0 2024-09-15T21:24:34.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 0x7f465c137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient: get_auth_request con 0x7f465c137ab0 auth_method 0 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 10 monclient(hunting): handle_auth_done global_id 45014 payload 824 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.523+0000 7f4661749640 1 -- 172.21.15.17:0/1824604353 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f465c135aa0 con 0x7f465c137ab0 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f4661749640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 0x7f465c137ec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f465800c6f0 tx=0x7f46580047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658028020 con 0x7f465c137ab0 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 10 monclient: dump: 2024-09-15T21:24:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4868 (secure 0 0 0) 0x7f46580113f0 con 0x7f465c137ab0 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 msgr2=0x7f465c137ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 0x7f465c137ec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f465800c6f0 tx=0x7f46580047e0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 msgr2=0x7f465c137ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46527fc640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c137ab0 0x7f465c137ec0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f465800c6f0 tx=0x7f46580047e0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 68, 2024-09-15T21:24:34.548 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 243126, 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:34.549 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:34.550 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.551 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:34.552 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:34.553 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.554 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.556 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:34.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:34.560 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:34.561 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:34.562 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f464003b9a0 msgr2=0x7f464003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f464003b9a0 0x7f464003de60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4654007fd0 tx=0x7f4654007d00 comp rx=0 tx=0).stop 2024-09-15T21:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 10 monclient: shutdown 2024-09-15T21:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c09ffc0 msgr2=0x7f465c1341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 --2- 172.21.15.17:0/1824604353 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f465c09ffc0 0x7f465c1341f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f464c01c230 tx=0x7f464c01ac40 comp rx=0 tx=0).stop 2024-09-15T21:24:34.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f466274b640 1 -- 172.21.15.17:0/1824604353 reap_dead start 2024-09-15T21:24:34.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f466274b640 1 -- 172.21.15.17:0/1824604353 reap_dead start 2024-09-15T21:24:34.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 shutdown_connections 2024-09-15T21:24:34.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 >> 172.21.15.17:0/1824604353 conn(0x7f465c0a66a0 msgr2=0x7f465c072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 shutdown_connections 2024-09-15T21:24:34.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.527+0000 7f46641ae640 1 -- 172.21.15.17:0/1824604353 wait complete. 2024-09-15T21:24:34.566 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:24:34.566 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:24:34.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:24:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient: get_monmap_and_config 2024-09-15T21:24:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient: monmap: 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:34.679630+0000 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:34.679630+0000 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:34.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 Processor -- start 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 -- start start 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient: init 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7301378c0 0x7fc730137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): picked mon.noname-c con 0x7fc7301378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7301381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc7300a5210 0x7fc7300a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): picked mon.noname-a con 0x7fc7300a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7300a1d00 con 0x7fc7301381e0 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc7300a1fe0 con 0x7fc7300a5210 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc7300a22c0 con 0x7fc7301378c0 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc738469640 10 monclient(hunting): authenticate will time out at 1398.075186s 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc735a04640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fc7301381e0 legacy=0x7fc7300a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736a06640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fc7300a5210 msgr2=0x7fc7300a56a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736a06640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc7300a5210 0x7fc7300a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7301378c0 0x7fc730137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7301378c0 0x7fc730137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:58140/0 (socket says 172.21.15.17:58140) 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 1 -- 172.21.15.17:0/59129787 learned_addr learned my addr 172.21.15.17:0/59129787 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): get_auth_request con 0x7fc7301378c0 auth_method 0 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.675+0000 7fc736205640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 10 monclient(hunting): handle_auth_done global_id 84211 payload 274 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 1 -- 172.21.15.17:0/59129787 >> v2:172.21.15.17:3300/0 conn(0x7fc7300a5210 msgr2=0x7fc7300a56a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 1 --2- 172.21.15.17:0/59129787 >> v2:172.21.15.17:3300/0 conn(0x7fc7300a5210 0x7fc7300a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 1 -- 172.21.15.17:0/59129787 >> v1:172.21.15.17:6789/0 conn(0x7fc7301381e0 legacy=0x7fc7300a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 1 monclient: found mon.noname-c 2024-09-15T21:24:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 1 -- 172.21.15.17:0/59129787 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7300a4490 con 0x7fc7301378c0 2024-09-15T21:24:34.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc736205640 1 --2- 172.21.15.17:0/59129787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 0x7fc730137ca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc72401c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887380b38c4165e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 1 -- 172.21.15.17:0/59129787 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240b1070 con 0x7fc7301378c0 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: dump: 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.697 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 5 monclient: authenticate success, global_id 84211 2024-09-15T21:24:34.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 1 -- 172.21.15.17:0/59129787 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7240a8050 con 0x7fc7301378c0 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 1 -- 172.21.15.17:0/59129787 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240ac860 con 0x7fc7301378c0 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc735203640 10 monclient: dump: 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.699 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 4 set_mon_vals no callback set 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 10 monclient: shutdown 2024-09-15T21:24:34.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 -- 172.21.15.17:0/59129787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 msgr2=0x7fc730137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 --2- 172.21.15.17:0/59129787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 0x7fc730137ca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc72401c400 comp rx=0 tx=0).stop 2024-09-15T21:24:34.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 -- 172.21.15.17:0/59129787 shutdown_connections 2024-09-15T21:24:34.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 --2- 172.21.15.17:0/59129787 >> v2:172.21.15.17:3300/0 conn(0x7fc7300a5210 0x7fc7300a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 --2- 172.21.15.17:0/59129787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 0x7fc730137ca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 -- 172.21.15.17:0/59129787 >> 172.21.15.17:0/59129787 conn(0x7fc7300aac40 msgr2=0x7fc7300a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 -- 172.21.15.17:0/59129787 shutdown_connections 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 1 -- 172.21.15.17:0/59129787 wait complete. 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 10 monclient: build_initial_monmap 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.679+0000 7fc738469640 10 monclient: monmap: 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:34.703 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:34.703 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 Processor -- start 2024-09-15T21:24:34.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- start start 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: init 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7300a5210 0x7fc73013f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): picked mon.noname-c con 0x7fc7300a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 0x7fc73013d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): picked mon.noname-b con 0x7fc7301378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:34.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7301381e0 0x7fc730136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): picked mon.noname-a con 0x7fc7301381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc73006f340 con 0x7fc7301381e0 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc73006f1c0 con 0x7fc7301378c0 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc7301425c0 con 0x7fc7300a5210 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): authenticate will time out at 1398.080830s 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736a06640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7301381e0 msgr2=0x7fc730136e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736a06640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7301381e0 0x7fc730136e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7300a5210 0x7fc73013f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7300a5210 0x7fc73013f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35524/0 (socket says 172.21.15.17:35524) 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 -- 172.21.15.17:0/3980085 learned_addr learned my addr 172.21.15.17:0/3980085 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): get_auth_request con 0x7fc7300a5210 auth_method 0 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 0x7fc73013d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): get_auth_request con 0x7fc7301378c0 auth_method 0 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient(hunting): handle_auth_done global_id 45017 payload 824 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 msgr2=0x7fc73013d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301378c0 0x7fc73013d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7301381e0 msgr2=0x7fc730136e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7301381e0 0x7fc730136e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient: _finish_hunting 0 2024-09-15T21:24:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 monclient: found mon.noname-c 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73013ea50 con 0x7fc7300a5210 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736205640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7300a5210 0x7fc73013f5b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc72401ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240bb020 con 0x7fc7300a5210 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: dump: 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.708 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7240b7420 con 0x7fc7300a5210 2024-09-15T21:24:34.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 5 monclient: authenticate success, global_id 45017 2024-09-15T21:24:34.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: _renew_subs 2024-09-15T21:24:34.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc730120f20 con 0x7fc7300a5210 2024-09-15T21:24:34.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:34.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240b1040 con 0x7fc7300a5210 2024-09-15T21:24:34.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:34.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 10 monclient: dump: 2024-09-15T21:24:34.711 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.711 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.711 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc734a02640 4 set_mon_vals no callback set 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: _renew_subs 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73013d9e0 con 0x7fc7300a5210 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 0x7fc7301cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7300a56a0 con 0x7fc730137370 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc7240be020 con 0x7fc7300a5210 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71403b970 0x7fc71403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 0x7fc7301cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient: get_auth_request con 0x7fc730137370 auth_method 0 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(198..198 src has 1..198) ==== 10468+0+0 (secure 0 0 0) 0x7fc7240eb650 con 0x7fc7300a5210 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736a06640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71403b970 0x7fc71403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736a06640 10 monclient: get_auth_request con 0x7fc71403b970 auth_method 0 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 10 monclient(hunting): handle_auth_done global_id 84217 payload 824 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc73013fb80 con 0x7fc730137370 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc735a04640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 0x7fc7301cfb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009fd0 tx=0x7fc72c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.683+0000 7fc736a06640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71403b970 0x7fc71403de30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc728003940 tx=0x7fc728006c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c014410 con 0x7fc730137370 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 10 monclient: dump: 2024-09-15T21:24:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc72c014410 con 0x7fc730137370 2024-09-15T21:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 msgr2=0x7fc7301cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 0x7fc7301cfb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009fd0 tx=0x7fc72c005420 comp rx=0 tx=0).stop 2024-09-15T21:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 msgr2=0x7fc7301cfb00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.687+0000 7fc722ffd640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc730137370 0x7fc7301cfb00 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009fd0 tx=0x7fc72c005420 comp rx=0 tx=0).stop 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc738469640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc738469640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 0x7fc7301d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc738469640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7301d0960 con 0x7fc7301d0040 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 0x7fc7301d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient: get_auth_request con 0x7fc7301d0040 auth_method 0 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 10 monclient(hunting): handle_auth_done global_id 84220 payload 824 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.691+0000 7fc735a04640 1 -- 172.21.15.17:0/3980085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc7301215e0 con 0x7fc7301d0040 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc735a04640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 0x7fc7301d0420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005d30 tx=0x7fc72c005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c014cb0 con 0x7fc7301d0040 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 10 monclient: dump: 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7fc72c014cb0 con 0x7fc7301d0040 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 msgr2=0x7fc7301d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 0x7fc7301d0420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005d30 tx=0x7fc72c005d60 comp rx=0 tx=0).stop 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 msgr2=0x7fc7301d0420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc722ffd640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7301d0040 0x7fc7301d0420 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005d30 tx=0x7fc72c005d60 comp rx=0 tx=0).stop 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 68, 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 26923, 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:24:34.723 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:34.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:34.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:34.726 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:34.726 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:34.726 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.726 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:24:34.727 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:24:34.728 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:24:34.729 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:24:34.729 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:24:34.729 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:24:34.729 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.729 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:24:34.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:34.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:34.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:24:34.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:24:34.732 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:24:34.735 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:24:34.736 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:24:34.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:24:34.738 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:24:34.738 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:24:34.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71403b970 msgr2=0x7fc71403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc71403b970 0x7fc71403de30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc728003940 tx=0x7fc728006c50 comp rx=0 tx=0).stop 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 10 monclient: shutdown 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7300a5210 msgr2=0x7fc73013f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 --2- 172.21.15.17:0/3980085 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc7300a5210 0x7fc73013f5b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc72401ac40 comp rx=0 tx=0).stop 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc736a06640 1 -- 172.21.15.17:0/3980085 reap_dead start 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc736a06640 1 -- 172.21.15.17:0/3980085 reap_dead start 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 shutdown_connections 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 >> 172.21.15.17:0/3980085 conn(0x7fc7300aac40 msgr2=0x7fc73006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 shutdown_connections 2024-09-15T21:24:34.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:34.695+0000 7fc738469640 1 -- 172.21.15.17:0/3980085 wait complete. 2024-09-15T21:24:34.741 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:24:34.741 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:24:34.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.794+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 added bucket h13 type host to location {rack=r1} v199) ==== 168+0+0 (secure 0 0 0) 0x7fd54c00e3c0 con 0x7fd55c09ea80 2024-09-15T21:24:34.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.794+0000 7fd54affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:34.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.794+0000 7fd54affd640 10 monclient: _finish_command 2 = system:0 added bucket h13 type host to location {rack=r1} 2024-09-15T21:24:34.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd560dbb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:34.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd560dbb640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:34.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd560dbb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:34.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd55c13b5b0 con 0x7fd55c09ea80 2024-09-15T21:24:34.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd54affd640 1 -- 172.21.15.119:0/277767640 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h13' already exists v199) ==== 147+0+0 (secure 0 0 0) 0x7fd54c017e20 con 0x7fd55c09ea80 2024-09-15T21:24:34.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd54affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:34.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.902+0000 7fd54affd640 10 monclient: _finish_command 3 = system:0 bucket 'h13' already exists 2024-09-15T21:24:34.907 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h13' already exists 2024-09-15T21:24:34.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd53c03b9a0 msgr2=0x7fd53c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd53c03b9a0 0x7fd53c03de60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ae000 tx=0x7fd550008040 comp rx=0 tx=0).stop 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 10 monclient: shutdown 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 msgr2=0x7fd55c13ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c13ab30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00b590 tx=0x7fd54c00ba50 comp rx=0 tx=0).stop 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 shutdown_connections 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd53c03b9a0 0x7fd53c03de60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd55c13c170 0x7fd55c141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd55c0a0a50 0x7fd55c13b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 --2- 172.21.15.119:0/277767640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd55c09ea80 0x7fd55c13ab30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 >> 172.21.15.119:0/277767640 conn(0x7fd55c0a66e0 msgr2=0x7fd55c0a4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 shutdown_connections 2024-09-15T21:24:34.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:34.906+0000 7fd560dbb640 1 -- 172.21.15.119:0/277767640 wait complete. 2024-09-15T21:24:34.924 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:34.924 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h14 host rack=r1 2024-09-15T21:24:35.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient: get_monmap_and_config 2024-09-15T21:24:35.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient: build_initial_monmap 2024-09-15T21:24:35.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient: monmap: 2024-09-15T21:24:35.038 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:35.038472+0000 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:35.038472+0000 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:35.039 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 1 Processor -- start 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 1 -- start start 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient: init 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.034+0000 7f57aada1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f57a40a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57a40a41b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:35.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f57a41402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 10 monclient(hunting): picked mon.noname-b con 0x7f57a40a45c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a41383e0 con 0x7f57a40a41b0 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a40a19d0 con 0x7f57a40a7ac0 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57a40a7ea0 con 0x7f57a40a45c0 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 10 monclient(hunting): authenticate will time out at 1393.836090s 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a8b3d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f57a40a7ac0 0x7f57a4137cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47154/0 (socket says 172.21.15.119:47154) 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a8b3d640 1 -- 172.21.15.119:0/1465674361 learned_addr learned my addr 172.21.15.119:0/1465674361 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a3fff640 1 -- 172.21.15.119:0/1465674361 >> v1:172.21.15.17:6789/0 conn(0x7f57a40a41b0 legacy=0x7f57a413cb80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601298554 0 0) 0x7f57a40a19d0 con 0x7f57a40a7ac0 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:24:35.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): my global_id is 84223 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f57a41402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f578c003530 con 0x7f57a40a7ac0 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): get_auth_request con 0x7f57a40a45c0 auth_method 0 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4161496285 0 0) 0x7f578c003530 con 0x7f57a40a7ac0 2024-09-15T21:24:35.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 msgr2=0x7f57a41402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f57a41402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 >> v1:172.21.15.17:6789/0 conn(0x7f57a40a41b0 legacy=0x7f57a413cb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a40a6960 con 0x7f57a40a7ac0 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5794001700 con 0x7f57a40a7ac0 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 5 monclient: authenticate success, global_id 84223 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:35.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: dump: 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.048 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 >> v1:172.21.15.119:6789/0 conn(0x7f57a40a7ac0 legacy=0x7f57a4137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f578c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): picked mon.c con 0x7f57a40a45c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578c003f70 0x7f578c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): picked mon.a con 0x7f578c003f70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:35.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c004d50 0x7f578c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): picked mon.b con 0x7f578c004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40a7ea0 con 0x7f578c003f70 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f578c00bc10 con 0x7f578c004d50 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f578c00be10 con 0x7f57a40a45c0 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c004d50 0x7f578c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a3fff640 1 -- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578c003f70 msgr2=0x7f578c004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:35.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): get_auth_request con 0x7f578c004d50 auth_method 0 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a3fff640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578c003f70 0x7f578c004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a8b3d640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f578c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient(hunting): handle_auth_done global_id 84223 payload 293 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 -- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 msgr2=0x7f578c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f578c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 -- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578c003f70 msgr2=0x7f578c004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578c003f70 0x7f578c004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient: _finish_hunting 0 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 monclient: found mon.b 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 -- 172.21.15.119:0/1465674361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c003530 con 0x7f578c004d50 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient: _finish_auth 0 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 10 monclient: _check_auth_tickets 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a8b3d640 10 monclient: get_auth_request con 0x7f57a40a45c0 auth_method 0 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a8b3d640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a40a45c0 0x7f578c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a933e640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c004d50 0x7f578c0051d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f579c009fc0 tx=0x7f579c01cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566b9777099d0ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c0b2070 con 0x7f578c004d50 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:35.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: dump: 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c0a5ae0 con 0x7f578c004d50 2024-09-15T21:24:35.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57aada1640 4 set_mon_vals no callback set 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 1 -- 172.21.15.119:0/1465674361 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c009c10 con 0x7f578c004d50 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.038+0000 7f57a37fe640 10 monclient: dump: 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:35.054 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:35.055 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:35.055 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: shutdown 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/1465674361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c004d50 msgr2=0x7f578c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 --2- 172.21.15.119:0/1465674361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c004d50 0x7f578c0051d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f579c009fc0 tx=0x7f579c01cc60 comp rx=0 tx=0).stop 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a933e640 1 -- 172.21.15.119:0/1465674361 reap_dead start 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/1465674361 shutdown_connections 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/1465674361 >> 172.21.15.119:0/1465674361 conn(0x7f57a40a6fd0 msgr2=0x7f57a40a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/1465674361 shutdown_connections 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/1465674361 wait complete. 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: build_initial_monmap 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: monmap: 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:35.060 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:35.062 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 Processor -- start 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- start start 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: init 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 0x7f57a412f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): picked mon.noname-b con 0x7f57a412ef50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a41cf4e0 0x7f57a41cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): picked mon.noname-c con 0x7f57a41cf4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a41cfea0 0x7f57a41d6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): picked mon.noname-a con 0x7f57a41cfea0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:35.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4140ca0 con 0x7f57a41cfea0 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4140b20 con 0x7f57a412ef50 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57a4140e20 con 0x7f57a41cf4e0 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient(hunting): authenticate will time out at 1393.841914s 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 0x7f57a412f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:35.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 0x7f57a412f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48830/0 (socket says 172.21.15.119:48830) 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 -- 172.21.15.119:0/3859189377 learned_addr learned my addr 172.21.15.119:0/3859189377 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): get_auth_request con 0x7f57a412ef50 auth_method 0 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a933e640 1 -- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a41cfea0 msgr2=0x7f57a41d6870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a933e640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a41cfea0 0x7f57a41d6870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:35.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a3fff640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a41cf4e0 0x7f57a41cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient(hunting): handle_auth_done global_id 84226 payload 824 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 -- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a41cf4e0 msgr2=0x7f57a41cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a41cf4e0 0x7f57a41cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 -- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a41cfea0 msgr2=0x7f57a41d6870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a41cfea0 0x7f57a41d6870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 monclient: found mon.noname-b 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:35.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 -- 172.21.15.119:0/3859189377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a41d0320 con 0x7f57a412ef50 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a3fff640 10 monclient: get_auth_request con 0x7f57a41cf4e0 auth_method 0 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a3fff640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a41cf4e0 0x7f57a41cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a8b3d640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 0x7f57a412f330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5794000fe0 tx=0x7f579400df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794004cc0 con 0x7f57a412ef50 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: dump: 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:35.067 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 5 monclient: authenticate success, global_id 84226 2024-09-15T21:24:35.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _renew_subs 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a40a1680 con 0x7f57a412ef50 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5794007110 con 0x7f57a412ef50 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794004e60 con 0x7f57a412ef50 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 10 monclient: dump: 2024-09-15T21:24:35.069 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _renew_subs 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a4135a60 con 0x7f57a412ef50 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a2ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a4137cd0 con 0x7f57a412ef50 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.042+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f579401b020 con 0x7f57a412ef50 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.046+0000 7f57a17fa640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f579003b9a0 0x7f579003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.046+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(199..199 src has 1..199) ==== 10681+0+0 (secure 0 0 0) 0x7f5794051230 con 0x7f57a412ef50 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.046+0000 7f57a3fff640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f579003b9a0 0x7f579003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.046+0000 7f57a3fff640 10 monclient: get_auth_request con 0x7f579003b9a0 auth_method 0 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.046+0000 7f57a3fff640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f579003b9a0 0x7f579003de60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5798006fd0 tx=0x7f5798008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.050+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f579404e530 con 0x7f57a412ef50 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.050+0000 7f57a17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:35.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.050+0000 7f57a17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:35.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.218+0000 7f57aada1640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:35.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.218+0000 7f57aada1640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:35.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.218+0000 7f57aada1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.218+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f57a40a2890 con 0x7f57a412ef50 2024-09-15T21:24:35.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.802+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 added bucket h14 type host to location {rack=r1} v200) ==== 168+0+0 (secure 0 0 0) 0x7f579400bbd0 con 0x7f57a412ef50 2024-09-15T21:24:35.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.802+0000 7f57a17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:35.805 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.802+0000 7f57a17fa640 10 monclient: _finish_command 2 = system:0 added bucket h14 type host to location {rack=r1} 2024-09-15T21:24:35.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57aada1640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:35.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57aada1640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:35.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57aada1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:35.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f57a406a760 con 0x7f57a412ef50 2024-09-15T21:24:35.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57a17fa640 1 -- 172.21.15.119:0/3859189377 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h14' already exists v200) ==== 147+0+0 (secure 0 0 0) 0x7f5794011510 con 0x7f57a412ef50 2024-09-15T21:24:35.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57a17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:35.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.906+0000 7f57a17fa640 10 monclient: _finish_command 3 = system:0 bucket 'h14' already exists 2024-09-15T21:24:35.914 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h14' already exists 2024-09-15T21:24:35.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f579003b9a0 msgr2=0x7f579003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:35.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f579003b9a0 0x7f579003de60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5798006fd0 tx=0x7f5798008040 comp rx=0 tx=0).stop 2024-09-15T21:24:35.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 10 monclient: shutdown 2024-09-15T21:24:35.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:35.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 msgr2=0x7f57a412f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 0x7f57a412f330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5794000fe0 tx=0x7f579400df70 comp rx=0 tx=0).stop 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 shutdown_connections 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f579003b9a0 0x7f579003de60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a41cfea0 0x7f57a41d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57a41cf4e0 0x7f57a41cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 --2- 172.21.15.119:0/3859189377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a412ef50 0x7f57a412f330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 >> 172.21.15.119:0/3859189377 conn(0x7f57a40a6fd0 msgr2=0x7f57a40a83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 shutdown_connections 2024-09-15T21:24:35.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:35.910+0000 7f57aada1640 1 -- 172.21.15.119:0/3859189377 wait complete. 2024-09-15T21:24:35.929 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:35.929 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h15 host rack=r1 2024-09-15T21:24:36.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.042+0000 7f3c395ed640 10 monclient: get_monmap_and_config 2024-09-15T21:24:36.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.042+0000 7f3c395ed640 10 monclient: build_initial_monmap 2024-09-15T21:24:36.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.042+0000 7f3c395ed640 10 monclient: monmap: 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:36.048726+0000 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:36.048726+0000 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:36.049 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:36.050 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:36.050 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 Processor -- start 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 -- start start 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient: init 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c34135bf0 0x7f3c3409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): picked mon.noname-c con 0x7f3c34135bf0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c34135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:36.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c340a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): picked mon.noname-b con 0x7f3c34136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c340a1d80 con 0x7f3c34135fd0 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3c340a3600 con 0x7f3c34135bf0 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3c340a2060 con 0x7f3c34136460 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c395ed640 10 monclient(hunting): authenticate will time out at 1394.845275s 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c34135bf0 0x7f3c3409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c2a7fc640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f3c34135fd0 legacy=0x7f3c340a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c34135bf0 0x7f3c3409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48846/0 (socket says 172.21.15.119:48846) 2024-09-15T21:24:36.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 -- 172.21.15.119:0/1713660335 learned_addr learned my addr 172.21.15.119:0/1713660335 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): get_auth_request con 0x7f3c34135bf0 auth_method 0 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c337fe640 1 --2- 172.21.15.119:0/1713660335 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c340a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c337fe640 10 monclient(hunting): get_auth_request con 0x7f3c34136460 auth_method 0 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:36.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_done global_id 84229 payload 274 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 -- 172.21.15.119:0/1713660335 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 msgr2=0x7f3c340a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 --2- 172.21.15.119:0/1713660335 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c340a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 -- 172.21.15.119:0/1713660335 >> v1:172.21.15.17:6789/0 conn(0x7f3c34135fd0 legacy=0x7f3c340a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 monclient: found mon.noname-c 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 -- 172.21.15.119:0/1713660335 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c340a2340 con 0x7f3c34135bf0 2024-09-15T21:24:36.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.046+0000 7f3c32ffd640 1 --2- 172.21.15.119:0/1713660335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3409fe70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c240098a0 tx=0x7f3c2401c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d78f5ab6aaf2c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 1 -- 172.21.15.119:0/1713660335 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240b0070 con 0x7f3c34135bf0 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: dump: 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.060 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 1 -- 172.21.15.119:0/1713660335 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c240ab600 con 0x7f3c34135bf0 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 5 monclient: authenticate success, global_id 84229 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:36.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 1 -- 172.21.15.119:0/1713660335 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240ab9d0 con 0x7f3c34135bf0 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 4 set_mon_vals no callback set 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c327fc640 10 monclient: dump: 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:36.062 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: shutdown 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/1713660335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 msgr2=0x7f3c3409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 --2- 172.21.15.119:0/1713660335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3409fe70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c240098a0 tx=0x7f3c2401c490 comp rx=0 tx=0).stop 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/1713660335 shutdown_connections 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 --2- 172.21.15.119:0/1713660335 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c340a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 --2- 172.21.15.119:0/1713660335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3409fe70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/1713660335 >> 172.21.15.119:0/1713660335 conn(0x7f3c340a66e0 msgr2=0x7f3c340a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:36.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/1713660335 shutdown_connections 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/1713660335 wait complete. 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: build_initial_monmap 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: monmap: 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:36.064 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 Processor -- start 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- start start 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: init 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3411c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): picked mon.noname-b con 0x7f3c34135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:36.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c34135fd0 0x7f3c3411cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): picked mon.noname-a con 0x7f3c34135fd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c3411d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): picked mon.noname-c con 0x7f3c34136460 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3c340a1820 con 0x7f3c34135fd0 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c340a16a0 con 0x7f3c34135bf0 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3c341402b0 con 0x7f3c34136460 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient(hunting): authenticate will time out at 1394.850213s 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3411c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:36.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3411c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48858/0 (socket says 172.21.15.119:48858) 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 -- 172.21.15.119:0/9600017 learned_addr learned my addr 172.21.15.119:0/9600017 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): get_auth_request con 0x7f3c34135bf0 auth_method 0 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2a7fc640 1 -- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c34135fd0 msgr2=0x7f3c3411cbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2a7fc640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c34135fd0 0x7f3c3411cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient(hunting): handle_auth_done global_id 84232 payload 824 2024-09-15T21:24:36.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 -- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 msgr2=0x7f3c3411d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c337fe640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c3411d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c3411d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 -- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c34135fd0 msgr2=0x7f3c3411cbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c34135fd0 0x7f3c3411cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 monclient: found mon.noname-b 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 -- 172.21.15.119:0/9600017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3411f100 con 0x7f3c34135bf0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c32ffd640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3411c6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c2401cd30 tx=0x7f3c2401acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240ba020 con 0x7f3c34135bf0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:36.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: dump: 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: _finish_auth 0 2024-09-15T21:24:36.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c240b6640 con 0x7f3c34135bf0 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 5 monclient: authenticate success, global_id 84232 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _renew_subs 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3411f3e0 con 0x7f3c34135bf0 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240b0040 con 0x7f3c34135bf0 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 10 monclient: dump: 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:36.070 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c31ffb640 4 set_mon_vals no callback set 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _renew_subs 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c340a56f0 con 0x7f3c34135bf0 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:36.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3c240bd020 con 0x7f3c34135bf0 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.050+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3409d240 con 0x7f3c34135bf0 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.054+0000 7f3c2bfff640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c1403b970 0x7f3c1403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.054+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(200..200 src has 1..200) ==== 10894+0+0 (secure 0 0 0) 0x7f3c240eb6e0 con 0x7f3c34135bf0 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.054+0000 7f3c2a7fc640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c1403b970 0x7f3c1403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.054+0000 7f3c2a7fc640 10 monclient: get_auth_request con 0x7f3c1403b970 auth_method 0 2024-09-15T21:24:36.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.054+0000 7f3c2a7fc640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c1403b970 0x7f3c1403de30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c006fd0 tx=0x7f3c1c006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:36.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.058+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3c240c4030 con 0x7f3c34135bf0 2024-09-15T21:24:36.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.058+0000 7f3c2bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:36.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.058+0000 7f3c2bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:36.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.218+0000 7f3c395ed640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:36.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.218+0000 7f3c395ed640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:36.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.218+0000 7f3c395ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.218+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f3c3409fe70 con 0x7f3c34135bf0 2024-09-15T21:24:36.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.806+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 added bucket h15 type host to location {rack=r1} v201) ==== 168+0+0 (secure 0 0 0) 0x7f3c24016110 con 0x7f3c34135bf0 2024-09-15T21:24:36.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.806+0000 7f3c2bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:36.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.806+0000 7f3c2bfff640 10 monclient: _finish_command 2 = system:0 added bucket h15 type host to location {rack=r1} 2024-09-15T21:24:36.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c395ed640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:36.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c395ed640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:36.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c395ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:36.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f3c3406a830 con 0x7f3c34135bf0 2024-09-15T21:24:36.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c2bfff640 1 -- 172.21.15.119:0/9600017 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h15' already exists v201) ==== 147+0+0 (secure 0 0 0) 0x7f3c240b43f0 con 0x7f3c34135bf0 2024-09-15T21:24:36.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c2bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:36.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.906+0000 7f3c2bfff640 10 monclient: _finish_command 3 = system:0 bucket 'h15' already exists 2024-09-15T21:24:36.912 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h15' already exists 2024-09-15T21:24:36.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c1403b970 msgr2=0x7f3c1403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:36.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c1403b970 0x7f3c1403de30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c006fd0 tx=0x7f3c1c006d40 comp rx=0 tx=0).stop 2024-09-15T21:24:36.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 10 monclient: shutdown 2024-09-15T21:24:36.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:36.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 msgr2=0x7f3c3411c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:36.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3411c6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c2401cd30 tx=0x7f3c2401acd0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 shutdown_connections 2024-09-15T21:24:36.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c1403b970 0x7f3c1403de30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c34136460 0x7f3c3411d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c34135fd0 0x7f3c3411cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 --2- 172.21.15.119:0/9600017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c34135bf0 0x7f3c3411c6a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:36.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 >> 172.21.15.119:0/9600017 conn(0x7f3c340a66e0 msgr2=0x7f3c340a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:36.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 shutdown_connections 2024-09-15T21:24:36.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:36.910+0000 7f3c395ed640 1 -- 172.21.15.119:0/9600017 wait complete. 2024-09-15T21:24:36.929 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:36.929 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h16 host rack=r1 2024-09-15T21:24:37.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.046+0000 7fc6cb6f3640 10 monclient: get_monmap_and_config 2024-09-15T21:24:37.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.046+0000 7fc6cb6f3640 10 monclient: build_initial_monmap 2024-09-15T21:24:37.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.046+0000 7fc6cb6f3640 10 monclient: monmap: 2024-09-15T21:24:37.051 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:37.051 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:37.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:37.050922+0000 2024-09-15T21:24:37.052 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:37.050922+0000 2024-09-15T21:24:37.052 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:37.052 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:37.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:37.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:37.053 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:37.053 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:37.053 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:37.054 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.046+0000 7fc6cb6f3640 1 Processor -- start 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.046+0000 7fc6cb6f3640 1 -- start start 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.046+0000 7fc6cb6f3640 10 monclient: init 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6c40a2450 0x7fc6c4072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:37.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): picked mon.noname-a con 0x7fc6c40a2450 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc6c406f900 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6c40a2f40 0x7fc6c413b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): picked mon.noname-c con 0x7fc6c40a2f40 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c406d670 con 0x7fc6c406f900 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c4071470 con 0x7fc6c40a2450 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c40716c0 con 0x7fc6c40a2f40 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient(hunting): authenticate will time out at 1395.848646s 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c948f640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fc6c40a2450 msgr2=0x7fc6c4072080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:37.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c8c8e640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fc6c406f900 legacy=0x7fc6c40a2830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:37.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c948f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6c40a2450 0x7fc6c4072080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:37.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6c40a2f40 0x7fc6c413b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:37.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6c40a2f40 0x7fc6c413b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48860/0 (socket says 172.21.15.119:48860) 2024-09-15T21:24:37.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1382218630 learned_addr learned my addr 172.21.15.119:0/1382218630 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:37.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): get_auth_request con 0x7fc6c40a2f40 auth_method 0 2024-09-15T21:24:37.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_done global_id 84235 payload 274 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1382218630 >> v2:172.21.15.17:3300/0 conn(0x7fc6c40a2450 msgr2=0x7fc6c4072080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:37.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 --2- 172.21.15.119:0/1382218630 >> v2:172.21.15.17:3300/0 conn(0x7fc6c40a2450 0x7fc6c4072080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1382218630 >> v1:172.21.15.17:6789/0 conn(0x7fc6c406f900 legacy=0x7fc6c40a2830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient: _finish_hunting 0 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 monclient: found mon.noname-c 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1382218630 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c413ca10 con 0x7fc6c40a2f40 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6c9c90640 1 --2- 172.21.15.119:0/1382218630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c413b4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406f860 tx=0x7fc6bc01d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15363648dfa7f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 1 -- 172.21.15.119:0/1382218630 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00a2d0 con 0x7fc6c40a2f40 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:37.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: dump: 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:37.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: _finish_auth 0 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 5 monclient: authenticate success, global_id 84235 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 1 -- 172.21.15.119:0/1382218630 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6bc0a6700 con 0x7fc6c40a2f40 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 1 -- 172.21.15.119:0/1382218630 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc0a6ad0 con 0x7fc6c40a2f40 2024-09-15T21:24:37.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:37.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 4 set_mon_vals no callback set 2024-09-15T21:24:37.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:37.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:37.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6bbfff640 10 monclient: dump: 2024-09-15T21:24:37.077 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:37.077 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:37.078 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:37.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 10 monclient: shutdown 2024-09-15T21:24:37.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:37.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1382218630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 msgr2=0x7fc6c413b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:37.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.050+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1382218630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c413b4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406f860 tx=0x7fc6bc01d9d0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1382218630 shutdown_connections 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1382218630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c413b4c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1382218630 >> v2:172.21.15.17:3300/0 conn(0x7fc6c40a2450 0x7fc6c4072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1382218630 >> 172.21.15.119:0/1382218630 conn(0x7fc6c40a3d00 msgr2=0x7fc6c406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1382218630 shutdown_connections 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1382218630 wait complete. 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: build_initial_monmap 2024-09-15T21:24:37.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: monmap: 2024-09-15T21:24:37.081 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:37.081 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:37.081 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:37.081 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:37.081 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:37.081 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 Processor -- start 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- start start 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: init 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6c406f900 0x7fc6c4134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): picked mon.noname-a con 0x7fc6c406f900 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6c40a2450 0x7fc6c41345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): picked mon.noname-c con 0x7fc6c40a2450 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:37.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c409ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): picked mon.noname-b con 0x7fc6c40a2f40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c413c390 con 0x7fc6c406f900 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c413c210 con 0x7fc6c40a2f40 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c413c510 con 0x7fc6c40a2450 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient(hunting): authenticate will time out at 1395.853555s 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c409ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c409ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48870/0 (socket says 172.21.15.119:48870) 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1570874601 learned_addr learned my addr 172.21.15.119:0/1570874601 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): get_auth_request con 0x7fc6c40a2f40 auth_method 0 2024-09-15T21:24:37.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c948f640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6c406f900 msgr2=0x7fc6c4134090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c948f640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6c406f900 0x7fc6c4134090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c8c8e640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6c40a2450 0x7fc6c41345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient(hunting): handle_auth_done global_id 84238 payload 824 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6c40a2450 msgr2=0x7fc6c41345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6c40a2450 0x7fc6c41345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6c406f900 msgr2=0x7fc6c4134090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6c406f900 0x7fc6c4134090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient: _finish_hunting 0 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 monclient: found mon.noname-b 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 -- 172.21.15.119:0/1570874601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c40a1c00 con 0x7fc6c40a2f40 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c8c8e640 10 monclient: get_auth_request con 0x7fc6c40a2450 auth_method 0 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c8c8e640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6c40a2450 0x7fc6c41345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c9c90640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c409ba60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0b0040 tx=0x7fc6bc01a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc0bd020 con 0x7fc6c40a2f40 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:37.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: dump: 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:37.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6bc009a40 con 0x7fc6c40a2f40 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc009e70 con 0x7fc6c40a2f40 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 10 monclient: dump: 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:37.087 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 5 monclient: authenticate success, global_id 84238 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _renew_subs 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c40a0570 con 0x7fc6c40a2f40 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6bb7fe640 4 set_mon_vals no callback set 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _renew_subs 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c409d870 con 0x7fc6c40a2f40 2024-09-15T21:24:37.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc6bc0c0020 con 0x7fc6c40a2f40 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c4072080 con 0x7fc6c40a2f40 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc6a803b970 0x7fc6a803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(201..201 src has 1..201) ==== 11107+0+0 (secure 0 0 0) 0x7fc6bc0ec760 con 0x7fc6c40a2f40 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c948f640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc6a803b970 0x7fc6a803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.054+0000 7fc6c948f640 10 monclient: get_auth_request con 0x7fc6a803b970 auth_method 0 2024-09-15T21:24:37.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.058+0000 7fc6c948f640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc6a803b970 0x7fc6a803de30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0003940 tx=0x7fc6c0006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:37.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.058+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc6c40a0570 con 0x7fc6c40a2f40 2024-09-15T21:24:37.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.058+0000 7fc6b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:37.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.058+0000 7fc6b9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:37.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.230+0000 7fc6cb6f3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:37.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.230+0000 7fc6cb6f3640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:37.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.230+0000 7fc6cb6f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.230+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc6c41223c0 con 0x7fc6c40a2f40 2024-09-15T21:24:37.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.810+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 added bucket h16 type host to location {rack=r1} v202) ==== 168+0+0 (secure 0 0 0) 0x7fc6c40a1c00 con 0x7fc6c40a2f40 2024-09-15T21:24:37.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.814+0000 7fc6b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:37.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.814+0000 7fc6b9ffb640 10 monclient: _finish_command 2 = system:0 added bucket h16 type host to location {rack=r1} 2024-09-15T21:24:37.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.914+0000 7fc6cb6f3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:37.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.914+0000 7fc6cb6f3640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:37.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.914+0000 7fc6cb6f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:37.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.914+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc6c4122930 con 0x7fc6c40a2f40 2024-09-15T21:24:37.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6b9ffb640 1 -- 172.21.15.119:0/1570874601 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h16' already exists v202) ==== 147+0+0 (secure 0 0 0) 0x7fc6bc0176e0 con 0x7fc6c40a2f40 2024-09-15T21:24:37.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6b9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:37.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6b9ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h16' already exists 2024-09-15T21:24:37.922 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h16' already exists 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc6a803b970 msgr2=0x7fc6a803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc6a803b970 0x7fc6a803de30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0003940 tx=0x7fc6c0006d90 comp rx=0 tx=0).stop 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 10 monclient: shutdown 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 msgr2=0x7fc6c409ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c409ba60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0b0040 tx=0x7fc6bc01a770 comp rx=0 tx=0).stop 2024-09-15T21:24:37.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 shutdown_connections 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc6a803b970 0x7fc6a803de30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6c40a2f40 0x7fc6c409ba60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc6c40a2450 0x7fc6c41345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 --2- 172.21.15.119:0/1570874601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6c406f900 0x7fc6c4134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 >> 172.21.15.119:0/1570874601 conn(0x7fc6c40a3d00 msgr2=0x7fc6c40a5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 shutdown_connections 2024-09-15T21:24:37.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:37.918+0000 7fc6cb6f3640 1 -- 172.21.15.119:0/1570874601 wait complete. 2024-09-15T21:24:37.938 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:37.938 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h17 host rack=r1 2024-09-15T21:24:38.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.054+0000 7fd49b6fe640 10 monclient: get_monmap_and_config 2024-09-15T21:24:38.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.054+0000 7fd49b6fe640 10 monclient: build_initial_monmap 2024-09-15T21:24:38.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.054+0000 7fd49b6fe640 10 monclient: monmap: 2024-09-15T21:24:38.059 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:38.060 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:38.060 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:38.059134+0000 2024-09-15T21:24:38.060 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:38.059134+0000 2024-09-15T21:24:38.060 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:38.060 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.060 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:38.061 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:38.061 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:38.061 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:38.061 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:38.061 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.054+0000 7fd49b6fe640 1 Processor -- start 2024-09-15T21:24:38.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.054+0000 7fd49b6fe640 1 -- start start 2024-09-15T21:24:38.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.054+0000 7fd49b6fe640 10 monclient: init 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd49406f3f0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd49406fee0 0x7fd494139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): picked mon.noname-a con 0x7fd49406fee0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 0x7fd49413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): picked mon.noname-b con 0x7fd494139ad0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd494071940 con 0x7fd49406f3f0 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd494071660 con 0x7fd49406fee0 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd4940a3220 con 0x7fd494139ad0 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 10 monclient(hunting): authenticate will time out at 1396.856571s 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd49406f3f0 0x7fd49406f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47168/0 (socket says 172.21.15.119:47168) 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd498c99640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fd49406fee0 msgr2=0x7fd494139590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 -- 172.21.15.119:0/928268297 learned_addr learned my addr 172.21.15.119:0/928268297 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd498c99640 1 --2- 172.21.15.119:0/928268297 >> v2:172.21.15.17:3300/0 conn(0x7fd49406fee0 0x7fd494139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 0x7fd49413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): get_auth_request con 0x7fd494139ad0 auth_method 0 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:38.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318965457 0 0) 0x7fd494071940 con 0x7fd49406f3f0 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): _negotiate 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): my global_id is 84241 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd480005740 con 0x7fd49406f3f0 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd499c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1040740310 0 0) 0x7fd480005740 con 0x7fd49406f3f0 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 msgr2=0x7fd49413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:38.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 0x7fd49413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 >> v2:172.21.15.17:3300/0 conn(0x7fd49406fee0 msgr2=0x7fd494139590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 --2- 172.21.15.119:0/928268297 >> v2:172.21.15.17:3300/0 conn(0x7fd49406fee0 0x7fd494139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4940a2fa0 con 0x7fd49406f3f0 2024-09-15T21:24:38.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: _finish_auth 0 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fd490001700 con 0x7fd49406f3f0 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 5 monclient: authenticate success, global_id 84241 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: dump: 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:38.078 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 >> v1:172.21.15.119:6789/0 conn(0x7fd49406f3f0 legacy=0x7fd49406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd480005c00 0x7fd480006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): picked mon.b con 0x7fd480005c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480006a20 0x7fd480006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): picked mon.a con 0x7fd480006a20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 0x7fd48000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): picked mon.c con 0x7fd494139ad0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd494071660 con 0x7fd480006a20 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940a3220 con 0x7fd480005c00 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800077c0 con 0x7fd494139ad0 2024-09-15T21:24:38.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:38.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd48bfff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:38.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49b6fe640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:38.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd498c99640 1 -- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480006a20 msgr2=0x7fd480006ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:38.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd480005c00 0x7fd480006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:38.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd498c99640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480006a20 0x7fd480006ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): get_auth_request con 0x7fd480005c00 auth_method 0 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:38.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient(hunting): handle_auth_done global_id 84241 payload 293 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 -- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 msgr2=0x7fd48000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494139ad0 0x7fd48000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 -- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480006a20 msgr2=0x7fd480006ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480006a20 0x7fd480006ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient: _finish_hunting 0 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 monclient: found mon.b 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:38.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 1 -- 172.21.15.119:0/928268297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd480005740 con 0x7fd480005c00 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient: _finish_auth 0 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.058+0000 7fd49949a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49949a640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd480005c00 0x7fd480006020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c240 tx=0x7fd49001cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63c3f0709aaec099 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490007d40 con 0x7fd480005c00 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: dump: 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:38.084 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:38.085 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4900a9070 con 0x7fd480005c00 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 1 -- 172.21.15.119:0/928268297 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490005d20 con 0x7fd480005c00 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 4 set_mon_vals no callback set 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48bfff640 10 monclient: dump: 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.086 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: shutdown 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:38.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/928268297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd480005c00 msgr2=0x7fd480006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/928268297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd480005c00 0x7fd480006020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c240 tx=0x7fd49001cbc0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd499c9b640 1 -- 172.21.15.119:0/928268297 reap_dead start 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/928268297 shutdown_connections 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/928268297 >> 172.21.15.119:0/928268297 conn(0x7fd49406cbf0 msgr2=0x7fd49406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/928268297 shutdown_connections 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/928268297 wait complete. 2024-09-15T21:24:38.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: build_initial_monmap 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: monmap: 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 Processor -- start 2024-09-15T21:24:38.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- start start 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: init 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480005c00 0x7fd4941c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): picked mon.noname-a con 0x7fd480005c00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 0x7fd4941307d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): picked mon.noname-b con 0x7fd49409f450 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:38.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494130d10 0x7fd4941cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): picked mon.noname-c con 0x7fd494130d10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940a3460 con 0x7fd480005c00 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd494074020 con 0x7fd49409f450 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd49413bec0 con 0x7fd494130d10 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient(hunting): authenticate will time out at 1396.862919s 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 0x7fd4941307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 0x7fd4941307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48888/0 (socket says 172.21.15.119:48888) 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 -- 172.21.15.119:0/2371869637 learned_addr learned my addr 172.21.15.119:0/2371869637 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:38.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): get_auth_request con 0x7fd49409f450 auth_method 0 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49949a640 1 -- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480005c00 msgr2=0x7fd4941c7ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49949a640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480005c00 0x7fd4941c7ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient(hunting): handle_auth_done global_id 84244 payload 824 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 -- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494130d10 msgr2=0x7fd4941cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd499c9b640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494130d10 0x7fd4941cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494130d10 0x7fd4941cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 -- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480005c00 msgr2=0x7fd4941c7ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:38.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480005c00 0x7fd4941c7ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient: _finish_hunting 0 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 monclient: found mon.noname-b 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 -- 172.21.15.119:0/2371869637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49409fc70 con 0x7fd49409f450 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd498c99640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 0x7fd4941307d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd48400b530 tx=0x7fd48400b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484013020 con 0x7fd49409f450 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: dump: 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:38.093 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd484003df0 con 0x7fd49409f450 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 5 monclient: authenticate success, global_id 84244 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: _renew_subs 2024-09-15T21:24:38.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49409e8f0 con 0x7fd49409f450 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840052b0 con 0x7fd49409f450 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd489ffb640 10 monclient: dump: 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:38.095 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd48b7fe640 4 set_mon_vals no callback set 2024-09-15T21:24:38.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: _renew_subs 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.062+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4941cb960 con 0x7fd49409f450 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49b6fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49b6fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd484016020 con 0x7fd49409f450 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49b6fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd494131120 con 0x7fd49409f450 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd489ffb640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd47803b9a0 0x7fd47803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 11576+0+0 (secure 0 0 0) 0x7fd484051260 con 0x7fd49409f450 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49949a640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd47803b9a0 0x7fd47803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49949a640 10 monclient: get_auth_request con 0x7fd47803b9a0 auth_method 0 2024-09-15T21:24:38.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.066+0000 7fd49949a640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd47803b9a0 0x7fd47803de60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd4900ab000 tx=0x7fd4900a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:38.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.070+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd48401d020 con 0x7fd49409f450 2024-09-15T21:24:38.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.070+0000 7fd489ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:38.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.070+0000 7fd489ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:38.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.246+0000 7fd49b6fe640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:38.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.246+0000 7fd49b6fe640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:38.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.246+0000 7fd49b6fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.246+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd49406f7d0 con 0x7fd49409f450 2024-09-15T21:24:38.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.822+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 added bucket h17 type host to location {rack=r1} v203) ==== 168+0+0 (secure 0 0 0) 0x7fd48400d380 con 0x7fd49409f450 2024-09-15T21:24:38.826 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.822+0000 7fd489ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:38.827 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.822+0000 7fd489ffb640 10 monclient: _finish_command 2 = system:0 added bucket h17 type host to location {rack=r1} 2024-09-15T21:24:38.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd4940a6080 con 0x7fd49409f450 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd489ffb640 1 -- 172.21.15.119:0/2371869637 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h17' already exists v203) ==== 147+0+0 (secure 0 0 0) 0x7fd48400fdf0 con 0x7fd49409f450 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd489ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd489ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h17' already exists 2024-09-15T21:24:38.937 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h17' already exists 2024-09-15T21:24:38.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd47803b9a0 msgr2=0x7fd47803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:38.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd47803b9a0 0x7fd47803de60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd4900ab000 tx=0x7fd4900a7070 comp rx=0 tx=0).stop 2024-09-15T21:24:38.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 10 monclient: shutdown 2024-09-15T21:24:38.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:38.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 msgr2=0x7fd4941307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 0x7fd4941307d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd48400b530 tx=0x7fd48400b9f0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 shutdown_connections 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd47803b9a0 0x7fd47803de60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd494130d10 0x7fd4941cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd49409f450 0x7fd4941307d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 --2- 172.21.15.119:0/2371869637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd480005c00 0x7fd4941c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 >> 172.21.15.119:0/2371869637 conn(0x7fd49406cbf0 msgr2=0x7fd494134990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 shutdown_connections 2024-09-15T21:24:38.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:38.934+0000 7fd49b6fe640 1 -- 172.21.15.119:0/2371869637 wait complete. 2024-09-15T21:24:38.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:38.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h18 host rack=r1 2024-09-15T21:24:39.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.070+0000 7f44d7917640 10 monclient: get_monmap_and_config 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.070+0000 7f44d7917640 10 monclient: build_initial_monmap 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.070+0000 7f44d7917640 10 monclient: monmap: 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:39.075825+0000 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:39.075825+0000 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:39.076 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:39.077 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:39.077 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:39.077 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:39.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.070+0000 7f44d7917640 1 Processor -- start 2024-09-15T21:24:39.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 1 -- start start 2024-09-15T21:24:39.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient: init 2024-09-15T21:24:39.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:39.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2024-09-15T21:24:39.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135bf0 0x7f44d009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:39.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): picked mon.noname-b con 0x7f44d0135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:39.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f44d0135fd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44d0136430 0x7f44d00a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): picked mon.noname-c con 0x7f44d0136430 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00a1d80 con 0x7f44d0135fd0 2024-09-15T21:24:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f44d00a3600 con 0x7f44d0136430 2024-09-15T21:24:39.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f44d00a2060 con 0x7f44d0135bf0 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient(hunting): authenticate will time out at 1397.872767s 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44d0136430 0x7f44d00a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d4eb2640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f44d0135fd0 legacy=0x7f44d00a0a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44d0136430 0x7f44d00a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48904/0 (socket says 172.21.15.119:48904) 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 -- 172.21.15.119:0/2685819741 learned_addr learned my addr 172.21.15.119:0/2685819741 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:39.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): get_auth_request con 0x7f44d0136430 auth_method 0 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d56b3640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135bf0 0x7f44d009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient(hunting): handle_auth_done global_id 84247 payload 274 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 -- 172.21.15.119:0/2685819741 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135bf0 msgr2=0x7f44d009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:39.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135bf0 0x7f44d009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 -- 172.21.15.119:0/2685819741 >> v1:172.21.15.17:6789/0 conn(0x7f44d0135fd0 legacy=0x7f44d00a0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient: _finish_hunting 0 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 monclient: found mon.noname-c 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 -- 172.21.15.119:0/2685819741 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d00a2340 con 0x7f44d0136430 2024-09-15T21:24:39.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d56b3640 10 monclient: get_auth_request con 0x7f44d0135bf0 auth_method 0 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d56b3640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135bf0 0x7f44d009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d5eb4640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0136430 0x7f44d00a1160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009e50 tx=0x7f44c801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c27e20d9c348235c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 1 -- 172.21.15.119:0/2685819741 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80a5530 con 0x7f44d0136430 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: dump: 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:39.099 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:39.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: _finish_auth 0 2024-09-15T21:24:39.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:39.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:39.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 1 -- 172.21.15.119:0/2685819741 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c80a56d0 con 0x7f44d0136430 2024-09-15T21:24:39.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 5 monclient: authenticate success, global_id 84247 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44d7917640 4 set_mon_vals no callback set 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 1 -- 172.21.15.119:0/2685819741 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80a5aa0 con 0x7f44d0136430 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.074+0000 7f44c7fff640 10 monclient: dump: 2024-09-15T21:24:39.102 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:39.103 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:39.103 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.103 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.104 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:39.104 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:39.104 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:39.104 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:39.104 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:39.104 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: shutdown 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/2685819741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0136430 msgr2=0x7f44d00a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0136430 0x7f44d00a1160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009e50 tx=0x7f44c801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/2685819741 shutdown_connections 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0136430 0x7f44d00a1160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 --2- 172.21.15.119:0/2685819741 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135bf0 0x7f44d009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/2685819741 >> 172.21.15.119:0/2685819741 conn(0x7f44d00a66e0 msgr2=0x7f44d00a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/2685819741 shutdown_connections 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/2685819741 wait complete. 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: build_initial_monmap 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: monmap: 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:39.106 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:39.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 Processor -- start 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- start start 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: init 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 0x7f44d013c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): picked mon.noname-b con 0x7f44d0135bf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135fd0 0x7f44d0134960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:39.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): picked mon.noname-c con 0x7f44d0135fd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:39.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44d0136430 0x7f44d0134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:39.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): picked mon.noname-a con 0x7f44d0136430 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:39.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:39.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44d00a1820 con 0x7f44d0136430 2024-09-15T21:24:39.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:39.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44d00a16a0 con 0x7f44d0135bf0 2024-09-15T21:24:39.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:39.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f44d01402b0 con 0x7f44d0135fd0 2024-09-15T21:24:39.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:39.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient(hunting): authenticate will time out at 1397.877744s 2024-09-15T21:24:39.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 0x7f44d013c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:39.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d5eb4640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44d0136430 msgr2=0x7f44d0134ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:39.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 0x7f44d013c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48914/0 (socket says 172.21.15.119:48914) 2024-09-15T21:24:39.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d5eb4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44d0136430 0x7f44d0134ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:39.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 -- 172.21.15.119:0/954400038 learned_addr learned my addr 172.21.15.119:0/954400038 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:39.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): get_auth_request con 0x7f44d0135bf0 auth_method 0 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d4eb2640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135fd0 0x7f44d0134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:39.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d4eb2640 10 monclient(hunting): get_auth_request con 0x7f44d0135fd0 auth_method 0 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d4eb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d4eb2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d4eb2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient(hunting): handle_auth_done global_id 84250 payload 824 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 -- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135fd0 msgr2=0x7f44d0134960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135fd0 0x7f44d0134960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 -- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44d0136430 msgr2=0x7f44d0134ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44d0136430 0x7f44d0134ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient: _finish_hunting 0 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 monclient: found mon.noname-b 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 -- 172.21.15.119:0/954400038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d0120ab0 con 0x7f44d0135bf0 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d56b3640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 0x7f44d013c6f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c7c0 tx=0x7f44cc00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc007a30 con 0x7f44d0135bf0 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: dump: 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:39.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:39.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc007bd0 con 0x7f44d0135bf0 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0113f0 con 0x7f44d0135bf0 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c5ffb640 10 monclient: dump: 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:39.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.117 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44c77fe640 4 set_mon_vals no callback set 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 5 monclient: authenticate success, global_id 84250 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: _renew_subs 2024-09-15T21:24:39.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d00a56f0 con 0x7f44d0135bf0 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: _renew_subs 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.078+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d01cf830 con 0x7f44d0135bf0 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d7917640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d7917640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d7917640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f44cc011590 con 0x7f44d0135bf0 2024-09-15T21:24:39.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d009fe70 con 0x7f44d0135bf0 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44c5ffb640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f44b403b970 0x7f44b403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(203..203 src has 1..203) ==== 11789+0+0 (secure 0 0 0) 0x7f44cc051a10 con 0x7f44d0135bf0 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d4eb2640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f44b403b970 0x7f44b403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d4eb2640 10 monclient: get_auth_request con 0x7f44b403b970 auth_method 0 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.082+0000 7f44d4eb2640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f44b403b970 0x7f44b403de30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f44d0073c00 tx=0x7f44c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.086+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f44cc015020 con 0x7f44d0135bf0 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.086+0000 7f44c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:39.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.086+0000 7f44c5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:39.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.266+0000 7f44d7917640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:39.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.266+0000 7f44d7917640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:39.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.266+0000 7f44d7917640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.266+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f44d006a830 con 0x7f44d0135bf0 2024-09-15T21:24:39.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.830+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 added bucket h18 type host to location {rack=r1} v204) ==== 168+0+0 (secure 0 0 0) 0x7f44cc04e8c0 con 0x7f44d0135bf0 2024-09-15T21:24:39.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.830+0000 7f44c5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:39.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.830+0000 7f44c5ffb640 10 monclient: _finish_command 2 = system:0 added bucket h18 type host to location {rack=r1} 2024-09-15T21:24:39.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.938+0000 7f44d7917640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:39.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.938+0000 7f44d7917640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:39.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.938+0000 7f44d7917640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:39.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.938+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f44d0120fb0 con 0x7f44d0135bf0 2024-09-15T21:24:39.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44c5ffb640 1 -- 172.21.15.119:0/954400038 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h18' already exists v204) ==== 147+0+0 (secure 0 0 0) 0x7f44cc011880 con 0x7f44d0135bf0 2024-09-15T21:24:39.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44c5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:39.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44c5ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h18' already exists 2024-09-15T21:24:39.945 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h18' already exists 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f44b403b970 msgr2=0x7f44b403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f44b403b970 0x7f44b403de30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f44d0073c00 tx=0x7f44c0008040 comp rx=0 tx=0).stop 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 10 monclient: shutdown 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 msgr2=0x7f44d013c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 0x7f44d013c6f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c7c0 tx=0x7f44cc00cc80 comp rx=0 tx=0).stop 2024-09-15T21:24:39.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 shutdown_connections 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f44b403b970 0x7f44b403de30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44d0136430 0x7f44d0134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44d0135fd0 0x7f44d0134960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 --2- 172.21.15.119:0/954400038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0135bf0 0x7f44d013c6f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 >> 172.21.15.119:0/954400038 conn(0x7f44d00a66e0 msgr2=0x7f44d00a3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 shutdown_connections 2024-09-15T21:24:39.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:39.942+0000 7f44d7917640 1 -- 172.21.15.119:0/954400038 wait complete. 2024-09-15T21:24:39.962 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:39.962 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h19 host rack=r1 2024-09-15T21:24:40.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 10 monclient: get_monmap_and_config 2024-09-15T21:24:40.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 10 monclient: build_initial_monmap 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 10 monclient: monmap: 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:40.082780+0000 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:40.082780+0000 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:40.083 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:40.084 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:40.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 1 Processor -- start 2024-09-15T21:24:40.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 1 -- start start 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 10 monclient: init 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.078+0000 7f0330866640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f032c0a7ac0 0x7f032c137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): picked mon.noname-a con 0x7f032c0a7ac0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f032c0a41b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:40.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f032c0a45c0 0x7f032c134a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): picked mon.noname-c con 0x7f032c0a45c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c0a19d0 con 0x7f032c0a41b0 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0a7ea0 con 0x7f032c0a7ac0 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0a6960 con 0x7f032c0a45c0 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:40.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient(hunting): authenticate will time out at 1398.880112s 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032a59c640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f032c0a7ac0 msgr2=0x7f032c137cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f032c0a45c0 0x7f032c134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f032c0a7ac0 0x7f032c137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f032c0a45c0 0x7f032c134a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48926/0 (socket says 172.21.15.119:48926) 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 -- 172.21.15.119:0/4082930731 learned_addr learned my addr 172.21.15.119:0/4082930731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): get_auth_request con 0x7f032c0a45c0 auth_method 0 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:40.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient(hunting): handle_auth_done global_id 84253 payload 274 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 -- 172.21.15.119:0/4082930731 >> v2:172.21.15.17:3300/0 conn(0x7f032c0a7ac0 msgr2=0x7f032c137cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 --2- 172.21.15.119:0/4082930731 >> v2:172.21.15.17:3300/0 conn(0x7f032c0a7ac0 0x7f032c137cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 -- 172.21.15.119:0/4082930731 >> v1:172.21.15.119:6789/0 conn(0x7f032c0a41b0 legacy=0x7f032c134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 monclient: found mon.noname-c 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 -- 172.21.15.119:0/4082930731 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c0a6c40 con 0x7f032c0a45c0 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032ad9d640 1 --2- 172.21.15.119:0/4082930731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a45c0 0x7f032c134a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0324009e50 tx=0x7f032401c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461908c792bbeeab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 1 -- 172.21.15.119:0/4082930731 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240a56a0 con 0x7f032c0a45c0 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: dump: 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:40.096 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:40.097 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:40.097 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:40.097 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:40.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: _finish_auth 0 2024-09-15T21:24:40.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 1 -- 172.21.15.119:0/4082930731 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240a5840 con 0x7f032c0a45c0 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 5 monclient: authenticate success, global_id 84253 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 1 -- 172.21.15.119:0/4082930731 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240a5c10 con 0x7f032c0a45c0 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 4 set_mon_vals no callback set 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f032959a640 10 monclient: dump: 2024-09-15T21:24:40.098 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 10 monclient: shutdown 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- 172.21.15.119:0/4082930731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a45c0 msgr2=0x7f032c134a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 --2- 172.21.15.119:0/4082930731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a45c0 0x7f032c134a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0324009e50 tx=0x7f032401c8d0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- 172.21.15.119:0/4082930731 shutdown_connections 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 --2- 172.21.15.119:0/4082930731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a45c0 0x7f032c134a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 --2- 172.21.15.119:0/4082930731 >> v2:172.21.15.17:3300/0 conn(0x7f032c0a7ac0 0x7f032c137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- 172.21.15.119:0/4082930731 >> 172.21.15.119:0/4082930731 conn(0x7f032c0a6fd0 msgr2=0x7f032c0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- 172.21.15.119:0/4082930731 shutdown_connections 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.082+0000 7f0330866640 1 -- 172.21.15.119:0/4082930731 wait complete. 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: build_initial_monmap 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: monmap: 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:40.100 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 Processor -- start 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- start start 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: init 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f032c0a41b0 0x7f032c131260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): picked mon.noname-a con 0x7f032c0a41b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f032c0a45c0 0x7f032c0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:40.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): picked mon.noname-c con 0x7f032c0a45c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 0x7f032c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): picked mon.noname-b con 0x7f032c0a7ac0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f032c135150 con 0x7f032c0a41b0 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f032c134fd0 con 0x7f032c0a7ac0 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f032c1423c0 con 0x7f032c0a45c0 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient(hunting): authenticate will time out at 1398.885309s 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 0x7f032c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032a59c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f032c0a41b0 msgr2=0x7f032c131260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f032c0a41b0 0x7f032c131260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:40.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 0x7f032c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48936/0 (socket says 172.21.15.119:48936) 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 -- 172.21.15.119:0/875255034 learned_addr learned my addr 172.21.15.119:0/875255034 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): get_auth_request con 0x7f032c0a7ac0 auth_method 0 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0329d9b640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f032c0a45c0 0x7f032c0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient(hunting): handle_auth_done global_id 84256 payload 824 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 -- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f032c0a45c0 msgr2=0x7f032c0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f032c0a45c0 0x7f032c0723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 -- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f032c0a41b0 msgr2=0x7f032c131260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f032c0a41b0 0x7f032c131260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 -- 172.21.15.119:0/875255034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c13a530 con 0x7f032c0a7ac0 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032ad9d640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 0x7f032c072900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f03240af040 tx=0x7f0324019670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240bc020 con 0x7f032c0a7ac0 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: dump: 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:40.104 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: _finish_auth 0 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240b8670 con 0x7f032c0a7ac0 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 5 monclient: authenticate success, global_id 84256 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _renew_subs 2024-09-15T21:24:40.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c139c90 con 0x7f032c0a7ac0 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240b8a20 con 0x7f032c0a7ac0 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 10 monclient: dump: 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:40.106 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031bfff640 4 set_mon_vals no callback set 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _renew_subs 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c138d30 con 0x7f032c0a7ac0 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f03240bf020 con 0x7f032c0a7ac0 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c13a1b0 con 0x7f032c0a7ac0 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f030c03b970 0x7f030c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(204..204 src has 1..204) ==== 12002+0+0 (secure 0 0 0) 0x7f03240ec950 con 0x7f032c0a7ac0 2024-09-15T21:24:40.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032a59c640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f030c03b970 0x7f030c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:40.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032a59c640 10 monclient: get_auth_request con 0x7f030c03b970 auth_method 0 2024-09-15T21:24:40.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.086+0000 7f032a59c640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f030c03b970 0x7f030c03de30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f031c003940 tx=0x7f031c006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:40.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.090+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f03240165e0 con 0x7f032c0a7ac0 2024-09-15T21:24:40.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.090+0000 7f031affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:40.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.090+0000 7f031affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:40.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.266+0000 7f0330866640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:40.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.266+0000 7f0330866640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:40.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.266+0000 7f0330866640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.266+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f032c06a760 con 0x7f032c0a7ac0 2024-09-15T21:24:40.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.842+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 added bucket h19 type host to location {rack=r1} v205) ==== 168+0+0 (secure 0 0 0) 0x7f03240e9720 con 0x7f032c0a7ac0 2024-09-15T21:24:40.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.842+0000 7f031affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:40.845 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.842+0000 7f031affd640 10 monclient: _finish_command 2 = system:0 added bucket h19 type host to location {rack=r1} 2024-09-15T21:24:40.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:40.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f032c138990 con 0x7f032c0a7ac0 2024-09-15T21:24:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f031affd640 1 -- 172.21.15.119:0/875255034 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h19' already exists v205) ==== 147+0+0 (secure 0 0 0) 0x7f03240adcf0 con 0x7f032c0a7ac0 2024-09-15T21:24:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f031affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f031affd640 10 monclient: _finish_command 3 = system:0 bucket 'h19' already exists 2024-09-15T21:24:40.953 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h19' already exists 2024-09-15T21:24:40.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f030c03b970 msgr2=0x7f030c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:40.955 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f030c03b970 0x7f030c03de30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f031c003940 tx=0x7f031c006d90 comp rx=0 tx=0).stop 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 10 monclient: shutdown 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 msgr2=0x7f032c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 0x7f032c072900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f03240af040 tx=0x7f0324019670 comp rx=0 tx=0).stop 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 shutdown_connections 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f030c03b970 0x7f030c03de30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f032c0a7ac0 0x7f032c072900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f032c0a45c0 0x7f032c0723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 --2- 172.21.15.119:0/875255034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f032c0a41b0 0x7f032c131260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 >> 172.21.15.119:0/875255034 conn(0x7f032c0a6fd0 msgr2=0x7f032c134210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 shutdown_connections 2024-09-15T21:24:40.956 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:40.950+0000 7f0330866640 1 -- 172.21.15.119:0/875255034 wait complete. 2024-09-15T21:24:40.971 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:40.971 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h20 host rack=r1 2024-09-15T21:24:41.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient: get_monmap_and_config 2024-09-15T21:24:41.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient: build_initial_monmap 2024-09-15T21:24:41.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient: monmap: 2024-09-15T21:24:41.092 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:41.092 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:41.092896+0000 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:41.092896+0000 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:41.093 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 Processor -- start 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 -- start start 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient: init 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd4d40a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4d40a41b0 0x7fd4d40a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): picked mon.noname-c con 0x7fd4d40a41b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:41.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4d413cb80 0x7fd4d413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): picked mon.noname-a con 0x7fd4d413cb80 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d41383e0 con 0x7fd4d40a7ac0 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d40a7ea0 con 0x7fd4d413cb80 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d40a19d0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4da4ee640 10 monclient(hunting): authenticate will time out at 1399.889639s 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4d40a41b0 0x7fd4d40a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d8a8b640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fd4d413cb80 msgr2=0x7fd4d413ef70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d8a8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4d413cb80 0x7fd4d413ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4d40a41b0 0x7fd4d40a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48938/0 (socket says 172.21.15.119:48938) 2024-09-15T21:24:41.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 -- 172.21.15.119:0/2647692152 learned_addr learned my addr 172.21.15.119:0/2647692152 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): get_auth_request con 0x7fd4d40a41b0 auth_method 0 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient(hunting): handle_auth_done global_id 84259 payload 274 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 -- 172.21.15.119:0/2647692152 >> v2:172.21.15.17:3300/0 conn(0x7fd4d413cb80 msgr2=0x7fd4d413ef70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 --2- 172.21.15.119:0/2647692152 >> v2:172.21.15.17:3300/0 conn(0x7fd4d413cb80 0x7fd4d413ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 -- 172.21.15.119:0/2647692152 >> v1:172.21.15.119:6789/0 conn(0x7fd4d40a7ac0 legacy=0x7fd4d4137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 monclient: found mon.noname-c 2024-09-15T21:24:41.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:41.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.090+0000 7fd4d37fe640 1 -- 172.21.15.119:0/2647692152 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d40a6960 con 0x7fd4d40a41b0 2024-09-15T21:24:41.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d37fe640 1 --2- 172.21.15.119:0/2647692152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d40a45c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009870 tx=0x7fd4c801c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a1e18e8b386ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 1 -- 172.21.15.119:0/2647692152 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80b0070 con 0x7fd4d40a41b0 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: dump: 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:41.103 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 1 -- 172.21.15.119:0/2647692152 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c80ab600 con 0x7fd4d40a41b0 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 1 -- 172.21.15.119:0/2647692152 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80ab9d0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 5 monclient: authenticate success, global_id 84259 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 4 set_mon_vals no callback set 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d2ffd640 10 monclient: dump: 2024-09-15T21:24:41.104 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: shutdown 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/2647692152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 msgr2=0x7fd4d40a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/2647692152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d40a45c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009870 tx=0x7fd4c801c3d0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/2647692152 shutdown_connections 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/2647692152 >> v2:172.21.15.17:3300/0 conn(0x7fd4d413cb80 0x7fd4d413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/2647692152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d40a45c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/2647692152 >> 172.21.15.119:0/2647692152 conn(0x7fd4d40a6fd0 msgr2=0x7fd4d40a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/2647692152 shutdown_connections 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/2647692152 wait complete. 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: build_initial_monmap 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: monmap: 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:41.106 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 Processor -- start 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- start start 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: init 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d4139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): picked mon.noname-b con 0x7fd4d40a41b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4d40a7ac0 0x7fd4d4139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): picked mon.noname-c con 0x7fd4d40a7ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4d413cb80 0x7fd4d41cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:41.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): picked mon.noname-a con 0x7fd4d413cb80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4141f30 con 0x7fd4d413cb80 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4141db0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d41420b0 con 0x7fd4d40a7ac0 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient(hunting): authenticate will time out at 1399.894758s 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d8a8b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4d413cb80 msgr2=0x7fd4d41cf6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d4139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d8a8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4d413cb80 0x7fd4d41cf6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d4139650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48942/0 (socket says 172.21.15.119:48942) 2024-09-15T21:24:41.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 -- 172.21.15.119:0/683769370 learned_addr learned my addr 172.21.15.119:0/683769370 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): get_auth_request con 0x7fd4d40a41b0 auth_method 0 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d37fe640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4d40a7ac0 0x7fd4d4139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d37fe640 10 monclient(hunting): get_auth_request con 0x7fd4d40a7ac0 auth_method 0 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient(hunting): handle_auth_done global_id 84262 payload 824 2024-09-15T21:24:41.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 -- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4d40a7ac0 msgr2=0x7fd4d4139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4d40a7ac0 0x7fd4d4139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 -- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4d413cb80 msgr2=0x7fd4d41cf6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4d413cb80 0x7fd4d41cf6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 monclient: found mon.noname-b 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 -- 172.21.15.119:0/683769370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d413a0d0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d3fff640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d4139650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c8a0 tx=0x7fd4cc00cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003b80 con 0x7fd4d40a41b0 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: dump: 2024-09-15T21:24:41.110 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4cc003d40 con 0x7fd4d40a41b0 2024-09-15T21:24:41.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00aa90 con 0x7fd4d40a41b0 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 5 monclient: authenticate success, global_id 84262 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: _renew_subs 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d41390f0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d0ff9640 10 monclient: dump: 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:41.112 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4d27fc640 4 set_mon_vals no callback set 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: _renew_subs 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.094+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d413a670 con 0x7fd4d40a41b0 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4da4ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4da4ee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4da4ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d40a45c0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd4cc00ac30 con 0x7fd4d40a41b0 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4d0ff9640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4c003b970 0x7fd4c003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(205..205 src has 1..205) ==== 12215+0+0 (secure 0 0 0) 0x7fd4cc0517f0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4d37fe640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4c003b970 0x7fd4c003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4d37fe640 10 monclient: get_auth_request con 0x7fd4c003b970 auth_method 0 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.098+0000 7fd4d37fe640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4c003b970 0x7fd4c003de30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80ad000 tx=0x7fd4c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.102+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd4d413a0d0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.102+0000 7fd4d0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:41.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.102+0000 7fd4d0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:41.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.278+0000 7fd4da4ee640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:41.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.278+0000 7fd4da4ee640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:41.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.278+0000 7fd4da4ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.278+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd4d406a760 con 0x7fd4d40a41b0 2024-09-15T21:24:41.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.850+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 added bucket h20 type host to location {rack=r1} v206) ==== 168+0+0 (secure 0 0 0) 0x7fd4cc005090 con 0x7fd4d40a41b0 2024-09-15T21:24:41.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.850+0000 7fd4d0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:41.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.850+0000 7fd4d0ff9640 10 monclient: _finish_command 2 = system:0 added bucket h20 type host to location {rack=r1} 2024-09-15T21:24:41.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4da4ee640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:41.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4da4ee640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:41.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4da4ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:41.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd4d41d03c0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4d0ff9640 1 -- 172.21.15.119:0/683769370 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h20' already exists v206) ==== 147+0+0 (secure 0 0 0) 0x7fd4cc0052b0 con 0x7fd4d40a41b0 2024-09-15T21:24:41.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4d0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:41.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.958+0000 7fd4d0ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h20' already exists 2024-09-15T21:24:41.962 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h20' already exists 2024-09-15T21:24:41.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4c003b970 msgr2=0x7fd4c003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:41.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4c003b970 0x7fd4c003de30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80ad000 tx=0x7fd4c8008040 comp rx=0 tx=0).stop 2024-09-15T21:24:41.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 10 monclient: shutdown 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 msgr2=0x7fd4d4139650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d4139650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c8a0 tx=0x7fd4cc00cd60 comp rx=0 tx=0).stop 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 shutdown_connections 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd4c003b970 0x7fd4c003de30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4d413cb80 0x7fd4d41cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd4d40a7ac0 0x7fd4d4139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 --2- 172.21.15.119:0/683769370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40a41b0 0x7fd4d4139650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 >> 172.21.15.119:0/683769370 conn(0x7fd4d40a6fd0 msgr2=0x7fd4d40a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 shutdown_connections 2024-09-15T21:24:41.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:41.962+0000 7fd4da4ee640 1 -- 172.21.15.119:0/683769370 wait complete. 2024-09-15T21:24:41.979 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:41.979 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h21 host rack=r1 2024-09-15T21:24:42.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient: get_monmap_and_config 2024-09-15T21:24:42.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient: build_initial_monmap 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient: monmap: 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:42.100942+0000 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:42.100942+0000 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:42.101 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:42.102 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:42.102 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:42.102 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:42.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 Processor -- start 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 -- start start 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient: init 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffb640a7ac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb640a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): picked mon.noname-b con 0x7ffb640a41b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb6413cb80 0x7ffb6413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): picked mon.noname-c con 0x7ffb6413cb80 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb641383e0 con 0x7ffb640a7ac0 2024-09-15T21:24:42.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffb640a7ea0 con 0x7ffb6413cb80 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb640a19d0 con 0x7ffb640a41b0 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6adad640 10 monclient(hunting): authenticate will time out at 1400.897583s 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb6413cb80 0x7ffb6413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb6413cb80 0x7ffb6413ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48944/0 (socket says 172.21.15.119:48944) 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 -- 172.21.15.119:0/3604854885 learned_addr learned my addr 172.21.15.119:0/3604854885 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): get_auth_request con 0x7ffb6413cb80 auth_method 0 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:42.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient(hunting): handle_auth_done global_id 84265 payload 274 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb63fff640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb640a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 -- 172.21.15.119:0/3604854885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 msgr2=0x7ffb640a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb640a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 -- 172.21.15.119:0/3604854885 >> v1:172.21.15.119:6789/0 conn(0x7ffb640a7ac0 legacy=0x7ffb64137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient: _finish_hunting 0 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 monclient: found mon.noname-c 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb6934a640 1 -- 172.21.15.119:0/3604854885 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb640a6960 con 0x7ffb6413cb80 2024-09-15T21:24:42.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb63fff640 10 monclient: get_auth_request con 0x7ffb640a41b0 auth_method 0 2024-09-15T21:24:42.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.098+0000 7ffb63fff640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb640a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:42.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6934a640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6413cb80 0x7ffb6413ef70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb540098a0 tx=0x7ffb5401c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=353bbc8674be0c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:42.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 1 -- 172.21.15.119:0/3604854885 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb540b0070 con 0x7ffb6413cb80 2024-09-15T21:24:42.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: dump: 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:42.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 5 monclient: authenticate success, global_id 84265 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 1 -- 172.21.15.119:0/3604854885 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb540ab600 con 0x7ffb6413cb80 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 4 set_mon_vals no callback set 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 1 -- 172.21.15.119:0/3604854885 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb540ab9d0 con 0x7ffb6413cb80 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb637fe640 10 monclient: dump: 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:42.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: shutdown 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/3604854885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6413cb80 msgr2=0x7ffb6413ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6413cb80 0x7ffb6413ef70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb540098a0 tx=0x7ffb5401c3d0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/3604854885 shutdown_connections 2024-09-15T21:24:42.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6413cb80 0x7ffb6413ef70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 --2- 172.21.15.119:0/3604854885 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb640a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/3604854885 >> 172.21.15.119:0/3604854885 conn(0x7ffb640a6fd0 msgr2=0x7ffb640a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/3604854885 shutdown_connections 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/3604854885 wait complete. 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: build_initial_monmap 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: monmap: 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:42.116 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 Processor -- start 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- start start 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: init 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb641311d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): picked mon.noname-c con 0x7ffb640a41b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:42.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 0x7ffb64131710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): picked mon.noname-b con 0x7ffb640a7ac0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6413cb80 0x7ffb64070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): picked mon.noname-a con 0x7ffb6413cb80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64141f90 con 0x7ffb6413cb80 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64141e10 con 0x7ffb640a7ac0 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb64142110 con 0x7ffb640a41b0 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient(hunting): authenticate will time out at 1400.902984s 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 0x7ffb64131710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6934a640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6413cb80 msgr2=0x7ffb64070200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:42.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6934a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6413cb80 0x7ffb64070200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 0x7ffb64131710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48948/0 (socket says 172.21.15.119:48948) 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 -- 172.21.15.119:0/1673964763 learned_addr learned my addr 172.21.15.119:0/1673964763 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): get_auth_request con 0x7ffb640a7ac0 auth_method 0 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb68b49640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb641311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient(hunting): handle_auth_done global_id 84271 payload 824 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 -- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 msgr2=0x7ffb641311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb641311d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 -- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6413cb80 msgr2=0x7ffb64070200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6413cb80 0x7ffb64070200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 monclient: found mon.noname-b 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 -- 172.21.15.119:0/1673964763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641370e0 con 0x7ffb640a7ac0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb68b49640 10 monclient: get_auth_request con 0x7ffb640a41b0 auth_method 0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb68b49640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb641311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb63fff640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 0x7ffb64131710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800cf60 tx=0x7ffb5800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5801a070 con 0x7ffb640a7ac0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:42.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: dump: 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:42.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5800d840 con 0x7ffb640a7ac0 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 5 monclient: authenticate success, global_id 84271 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: _renew_subs 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb640a30f0 con 0x7ffb640a7ac0 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800dc10 con 0x7ffb640a7ac0 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb617fa640 10 monclient: dump: 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.122 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: _renew_subs 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb62ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.102+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6413ef80 con 0x7ffb640a7ac0 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb6adad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb6adad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffb5801d070 con 0x7ffb640a7ac0 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb6adad640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb64077180 con 0x7ffb640a7ac0 2024-09-15T21:24:42.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb617fa640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5003b970 0x7ffb5003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:42.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 12428+0+0 (secure 0 0 0) 0x7ffb58052160 con 0x7ffb640a7ac0 2024-09-15T21:24:42.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb68b49640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5003b970 0x7ffb5003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:42.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb68b49640 10 monclient: get_auth_request con 0x7ffb5003b970 auth_method 0 2024-09-15T21:24:42.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.106+0000 7ffb68b49640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5003b970 0x7ffb5003de30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb64073950 tx=0x7ffb5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:42.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.110+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffb640a30f0 con 0x7ffb640a7ac0 2024-09-15T21:24:42.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.110+0000 7ffb617fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:42.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.110+0000 7ffb617fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:42.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.282+0000 7ffb6adad640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:42.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.282+0000 7ffb6adad640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:42.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.282+0000 7ffb6adad640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.282+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ffb640a4590 con 0x7ffb640a7ac0 2024-09-15T21:24:42.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.786+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 added bucket h21 type host to location {rack=r1} v207) ==== 168+0+0 (secure 0 0 0) 0x7ffb58013340 con 0x7ffb640a7ac0 2024-09-15T21:24:42.789 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.786+0000 7ffb617fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:42.790 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.786+0000 7ffb617fa640 10 monclient: _finish_command 2 = system:0 added bucket h21 type host to location {rack=r1} 2024-09-15T21:24:42.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb6adad640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:42.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb6adad640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:42.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb6adad640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:42.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ffb64073980 con 0x7ffb640a7ac0 2024-09-15T21:24:42.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb617fa640 1 -- 172.21.15.119:0/1673964763 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h21' already exists v207) ==== 147+0+0 (secure 0 0 0) 0x7ffb5800ddb0 con 0x7ffb640a7ac0 2024-09-15T21:24:42.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb617fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:42.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.898+0000 7ffb617fa640 10 monclient: _finish_command 3 = system:0 bucket 'h21' already exists 2024-09-15T21:24:42.903 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h21' already exists 2024-09-15T21:24:42.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5003b970 msgr2=0x7ffb5003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:42.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5003b970 0x7ffb5003de30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb64073950 tx=0x7ffb5c009040 comp rx=0 tx=0).stop 2024-09-15T21:24:42.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 10 monclient: shutdown 2024-09-15T21:24:42.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:42.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 msgr2=0x7ffb64131710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:42.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 0x7ffb64131710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800cf60 tx=0x7ffb5800cf90 comp rx=0 tx=0).stop 2024-09-15T21:24:42.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 shutdown_connections 2024-09-15T21:24:42.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb5003b970 0x7ffb5003de30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6413cb80 0x7ffb64070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb640a7ac0 0x7ffb64131710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 --2- 172.21.15.119:0/1673964763 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb640a41b0 0x7ffb641311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:42.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 >> 172.21.15.119:0/1673964763 conn(0x7ffb640a6fd0 msgr2=0x7ffb640a9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:42.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 shutdown_connections 2024-09-15T21:24:42.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:42.902+0000 7ffb6adad640 1 -- 172.21.15.119:0/1673964763 wait complete. 2024-09-15T21:24:42.920 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:42.920 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h22 host rack=r1 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient: get_monmap_and_config 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient: build_initial_monmap 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient: monmap: 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:43.038530+0000 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:43.038530+0000 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:43.038 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:43.039 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:43.039 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:43.039 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:43.039 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:43.039 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:43.039 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:43.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 Processor -- start 2024-09-15T21:24:43.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 -- start start 2024-09-15T21:24:43.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient: init 2024-09-15T21:24:43.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:43.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0d28136b90 0x7f0d28136f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): picked mon.noname-a con 0x7f0d28136b90 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0d281374b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d280a10d0 0x7f0d28138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): picked mon.noname-c con 0x7f0d280a10d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d280a18f0 con 0x7f0d281374b0 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2809fc40 con 0x7f0d28136b90 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0d280a1bd0 con 0x7f0d280a10d0 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:43.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2c8af640 10 monclient(hunting): authenticate will time out at 1401.835176s 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d280a10d0 0x7f0d28138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2659c640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f0d28136b90 msgr2=0x7f0d28136f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d2659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0d28136b90 0x7f0d28136f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d280a10d0 0x7f0d28138040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43736/0 (socket says 172.21.15.119:43736) 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 -- 172.21.15.119:0/1969152642 learned_addr learned my addr 172.21.15.119:0/1969152642 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): get_auth_request con 0x7f0d280a10d0 auth_method 0 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient(hunting): handle_auth_done global_id 84274 payload 274 2024-09-15T21:24:43.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 -- 172.21.15.119:0/1969152642 >> v2:172.21.15.17:3300/0 conn(0x7f0d28136b90 msgr2=0x7f0d28136f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 --2- 172.21.15.119:0/1969152642 >> v2:172.21.15.17:3300/0 conn(0x7f0d28136b90 0x7f0d28136f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 -- 172.21.15.119:0/1969152642 >> v1:172.21.15.119:6789/0 conn(0x7f0d281374b0 legacy=0x7f0d28137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 monclient: found mon.noname-c 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.034+0000 7f0d26d9d640 1 -- 172.21.15.119:0/1969152642 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d280a3660 con 0x7f0d280a10d0 2024-09-15T21:24:43.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d26d9d640 1 --2- 172.21.15.119:0/1969152642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d28138040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009e50 tx=0x7f0d2001c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c7a58f9770ac15 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:43.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 1 -- 172.21.15.119:0/1969152642 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200a57b0 con 0x7f0d280a10d0 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: dump: 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:43.050 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: _finish_auth 0 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 1 -- 172.21.15.119:0/1969152642 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d28138790 con 0x7f0d280a10d0 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 5 monclient: authenticate success, global_id 84274 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 1 -- 172.21.15.119:0/1969152642 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200a5b70 con 0x7f0d280a10d0 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 4 set_mon_vals no callback set 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2559a640 10 monclient: dump: 2024-09-15T21:24:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 10 monclient: shutdown 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 -- 172.21.15.119:0/1969152642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 msgr2=0x7f0d28138040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/1969152642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d28138040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009e50 tx=0x7f0d2001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 -- 172.21.15.119:0/1969152642 shutdown_connections 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/1969152642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d28138040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/1969152642 >> v2:172.21.15.17:3300/0 conn(0x7f0d28136b90 0x7f0d28136f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 -- 172.21.15.119:0/1969152642 >> 172.21.15.119:0/1969152642 conn(0x7f0d280a66e0 msgr2=0x7f0d280a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 -- 172.21.15.119:0/1969152642 shutdown_connections 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 -- 172.21.15.119:0/1969152642 wait complete. 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 10 monclient: build_initial_monmap 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 10 monclient: monmap: 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 Processor -- start 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 1 -- start start 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.038+0000 7f0d2c8af640 10 monclient: init 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d281cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): picked mon.noname-b con 0x7f0d280a10d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:43.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d28136b90 0x7f0d28132990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): picked mon.noname-a con 0x7f0d28136b90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281374b0 0x7f0d2813c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): picked mon.noname-c con 0x7f0d281374b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0d28142730 con 0x7f0d28136b90 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281425b0 con 0x7f0d280a10d0 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0d281428b0 con 0x7f0d281374b0 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient(hunting): authenticate will time out at 1401.840312s 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d281cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d25d9b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d28136b90 msgr2=0x7f0d28132990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:43.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d25d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d28136b90 0x7f0d28132990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d281cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43746/0 (socket says 172.21.15.119:43746) 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 -- 172.21.15.119:0/3622250290 learned_addr learned my addr 172.21.15.119:0/3622250290 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): get_auth_request con 0x7f0d280a10d0 auth_method 0 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient(hunting): handle_auth_done global_id 84280 payload 824 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d26d9d640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281374b0 0x7f0d2813c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 -- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281374b0 msgr2=0x7f0d2813c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:43.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281374b0 0x7f0d2813c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 -- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d28136b90 msgr2=0x7f0d28132990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d28136b90 0x7f0d28132990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 monclient: found mon.noname-b 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 -- 172.21.15.119:0/3622250290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2811d040 con 0x7f0d280a10d0 2024-09-15T21:24:43.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2659c640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d281cfb50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b560 tx=0x7f0d1800ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18013020 con 0x7f0d280a10d0 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: dump: 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:43.058 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18003df0 con 0x7f0d280a10d0 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180052b0 con 0x7f0d280a10d0 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 10 monclient: dump: 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:43.059 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 5 monclient: authenticate success, global_id 84280 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _renew_subs 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d24d99640 4 set_mon_vals no callback set 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2811d320 con 0x7f0d280a10d0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _renew_subs 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2813e4b0 con 0x7f0d280a10d0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d28136f70 con 0x7f0d280a10d0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0d18016020 con 0x7f0d280a10d0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0803b9a0 0x7f0d0803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 12641+0+0 (secure 0 0 0) 0x7f0d180517b0 con 0x7f0d280a10d0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d25d9b640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0803b9a0 0x7f0d0803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d25d9b640 10 monclient: get_auth_request con 0x7f0d0803b9a0 auth_method 0 2024-09-15T21:24:43.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.042+0000 7f0d25d9b640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0803b9a0 0x7f0d0803de60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c000c10 tx=0x7f0d1c007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:43.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.046+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0d1801d020 con 0x7f0d280a10d0 2024-09-15T21:24:43.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.046+0000 7f0d16ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:43.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.046+0000 7f0d16ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:43.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.218+0000 7f0d2c8af640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:43.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.218+0000 7f0d2c8af640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:43.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.218+0000 7f0d2c8af640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.218+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f0d281d09d0 con 0x7f0d280a10d0 2024-09-15T21:24:43.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.858+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 added bucket h22 type host to location {rack=r1} v208) ==== 168+0+0 (secure 0 0 0) 0x7f0d1804e300 con 0x7f0d280a10d0 2024-09-15T21:24:43.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.858+0000 7f0d16ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:43.863 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.858+0000 7f0d16ffd640 10 monclient: _finish_command 2 = system:0 added bucket h22 type host to location {rack=r1} 2024-09-15T21:24:43.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.958+0000 7f0d2c8af640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:43.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.958+0000 7f0d2c8af640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:43.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.958+0000 7f0d2c8af640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:43.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.958+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f0d28133630 con 0x7f0d280a10d0 2024-09-15T21:24:43.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d16ffd640 1 -- 172.21.15.119:0/3622250290 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h22' already exists v208) ==== 147+0+0 (secure 0 0 0) 0x7f0d1800fdc0 con 0x7f0d280a10d0 2024-09-15T21:24:43.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d16ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:43.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d16ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h22' already exists 2024-09-15T21:24:43.965 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h22' already exists 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0803b9a0 msgr2=0x7f0d0803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0803b9a0 0x7f0d0803de60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c000c10 tx=0x7f0d1c007a00 comp rx=0 tx=0).stop 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 10 monclient: shutdown 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 msgr2=0x7f0d281cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d281cfb50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b560 tx=0x7f0d1800ba20 comp rx=0 tx=0).stop 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 shutdown_connections 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0d0803b9a0 0x7f0d0803de60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0d281374b0 0x7f0d2813c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d28136b90 0x7f0d28132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 --2- 172.21.15.119:0/3622250290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d280a10d0 0x7f0d281cfb50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:43.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 >> 172.21.15.119:0/3622250290 conn(0x7f0d280a66e0 msgr2=0x7f0d2809e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:43.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 shutdown_connections 2024-09-15T21:24:43.968 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:43.962+0000 7f0d2c8af640 1 -- 172.21.15.119:0/3622250290 wait complete. 2024-09-15T21:24:43.981 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:43.981 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h23 host rack=r1 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient: get_monmap_and_config 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient: build_initial_monmap 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient: monmap: 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:44.102840+0000 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:44.102840+0000 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:44.103 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:44.104 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:44.104 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:44.104 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 1 Processor -- start 2024-09-15T21:24:44.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 1 -- start start 2024-09-15T21:24:44.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient: init 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4244135bf0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f424409fe70 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f424413c190 0x7f424413e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient(hunting): picked mon.noname-a con 0x7f424413c190 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42440a0300 con 0x7f4244135bf0 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42440a1d80 con 0x7f424409fe70 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f42440a3600 con 0x7f424413c190 2024-09-15T21:24:44.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4248cec640 10 monclient(hunting): authenticate will time out at 1402.899491s 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4241d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f424409fe70 0x7f424413aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33818/0 (socket says 172.21.15.119:33818) 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4241d9b640 1 -- 172.21.15.119:0/3827061447 learned_addr learned my addr 172.21.15.119:0/3827061447 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4242d9d640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f424413c190 msgr2=0x7f424413e580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424259c640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f4244135bf0 legacy=0x7f4244135fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f4242d9d640 1 --2- 172.21.15.119:0/3827061447 >> v2:172.21.15.17:3300/0 conn(0x7f424413c190 0x7f424413e580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087656077 0 0) 0x7f42440a1d80 con 0x7f424409fe70 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424159a640 10 monclient(hunting): _negotiate 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424159a640 10 monclient(hunting): my global_id is 84283 2024-09-15T21:24:44.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.098+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f422c003530 con 0x7f424409fe70 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 125443590 0 0) 0x7f422c003530 con 0x7f424409fe70 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 >> v2:172.21.15.17:3300/0 conn(0x7f424413c190 msgr2=0x7f424413e580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 --2- 172.21.15.119:0/3827061447 >> v2:172.21.15.17:3300/0 conn(0x7f424413c190 0x7f424413e580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 >> v1:172.21.15.17:6789/0 conn(0x7f4244135bf0 legacy=0x7f4244135fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: _finish_hunting 0 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440a2060 con 0x7f424409fe70 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: _finish_auth 0 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:44.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f4238001700 con 0x7f424409fe70 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 5 monclient: authenticate success, global_id 84283 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: dump: 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:44.121 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:44.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 >> v1:172.21.15.119:6789/0 conn(0x7f424409fe70 legacy=0x7f424413aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0039c0 0x7f422c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): picked mon.b con 0x7f422c0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c004810 0x7f422c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): picked mon.a con 0x7f422c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0053d0 0x7f422c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): picked mon.c con 0x7f422c0053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f42440a3600 con 0x7f422c004810 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f422c00c310 con 0x7f422c0039c0 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f422c00c510 con 0x7f422c0053d0 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:44.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0039c0 0x7f422c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): get_auth_request con 0x7f422c0039c0 auth_method 0 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4241d9b640 1 -- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c004810 msgr2=0x7f422c004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4241d9b640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c004810 0x7f422c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient(hunting): handle_auth_done global_id 84283 payload 293 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 -- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0053d0 msgr2=0x7f422c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:44.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0053d0 0x7f422c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 -- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c004810 msgr2=0x7f422c004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c004810 0x7f422c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 monclient: found mon.b 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 -- 172.21.15.119:0/3827061447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c003530 con 0x7f422c0039c0 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient: _finish_auth 0 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 10 monclient: _check_auth_tickets 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4242d9d640 1 -- 172.21.15.119:0/3827061447 reap_dead start 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424259c640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0039c0 0x7f422c003dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f42340038c0 tx=0x7f423401c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf6c3cb0588dc69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:44.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234009470 con 0x7f422c0039c0 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: dump: 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4234003af0 con 0x7f422c0039c0 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 4 set_mon_vals no callback set 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 1 -- 172.21.15.119:0/3827061447 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42340050d0 con 0x7f422c0039c0 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f424159a640 10 monclient: dump: 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.128 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 10 monclient: shutdown 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 -- 172.21.15.119:0/3827061447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0039c0 msgr2=0x7f422c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0039c0 0x7f422c003dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f42340038c0 tx=0x7f423401c310 comp rx=0 tx=0).stop 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 -- 172.21.15.119:0/3827061447 shutdown_connections 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 --2- 172.21.15.119:0/3827061447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0039c0 0x7f422c003dc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 -- 172.21.15.119:0/3827061447 >> 172.21.15.119:0/3827061447 conn(0x7f42440a66e0 msgr2=0x7f42440a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:44.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 -- 172.21.15.119:0/3827061447 shutdown_connections 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.102+0000 7f4248cec640 1 -- 172.21.15.119:0/3827061447 wait complete. 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: build_initial_monmap 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: monmap: 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:44.130 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 Processor -- start 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- start start 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: init 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0039c0 0x7f42441cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): picked mon.noname-c con 0x7f422c0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4244134cc0 0x7f4244135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): picked mon.noname-a con 0x7f4244134cc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 0x7f4244121d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): picked mon.noname-b con 0x7f4244121890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f424409ccd0 con 0x7f4244134cc0 2024-09-15T21:24:44.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f424406ec90 con 0x7f4244121890 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f424409ce50 con 0x7f422c0039c0 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient(hunting): authenticate will time out at 1402.905001s 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4241d9b640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4244134cc0 msgr2=0x7f4244135140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 0x7f4244121d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4241d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4244134cc0 0x7f4244135140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 0x7f4244121d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43762/0 (socket says 172.21.15.119:43762) 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 -- 172.21.15.119:0/3646864879 learned_addr learned my addr 172.21.15.119:0/3646864879 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): get_auth_request con 0x7f4244121890 auth_method 0 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:44.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f424259c640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0039c0 0x7f42441cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient(hunting): handle_auth_done global_id 84286 payload 824 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 -- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0039c0 msgr2=0x7f42441cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0039c0 0x7f42441cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 -- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4244134cc0 msgr2=0x7f4244135140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4244134cc0 0x7f4244135140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 monclient: found mon.noname-b 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:44.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 -- 172.21.15.119:0/3646864879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4244124e10 con 0x7f4244121890 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f424259c640 10 monclient: get_auth_request con 0x7f422c0039c0 auth_method 0 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f424259c640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0039c0 0x7f42441cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4242d9d640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 0x7f4244121d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f423c003cc0 tx=0x7f423c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c015670 con 0x7f4244121890 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: dump: 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:44.134 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423c010070 con 0x7f4244121890 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c015a70 con 0x7f4244121890 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 5 monclient: authenticate success, global_id 84286 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _renew_subs 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424409d240 con 0x7f4244121890 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 10 monclient: dump: 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:44.136 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4240d99640 4 set_mon_vals no callback set 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _renew_subs 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4244135680 con 0x7f4244121890 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42440a12f0 con 0x7f4244121890 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f423c01e3f0 con 0x7f4244121890 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(208..208 src has 1..208) ==== 12854+0+0 (secure 0 0 0) 0x7f423c051f60 con 0x7f4244121890 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f424259c640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f424259c640 10 monclient: get_auth_request con 0x7f422403b970 auth_method 0 2024-09-15T21:24:44.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.106+0000 7f424259c640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f42340ac000 tx=0x7f42340059c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:44.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.110+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4244124e10 con 0x7f4244121890 2024-09-15T21:24:44.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.110+0000 7f4232ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:44.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.110+0000 7f4232ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:44.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.286+0000 7f4248cec640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:44.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.286+0000 7f4248cec640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:44.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.286+0000 7f4248cec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.286+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f4244125770 con 0x7f4244121890 2024-09-15T21:24:44.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.862+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 added bucket h23 type host to location {rack=r1} v209) ==== 168+0+0 (secure 0 0 0) 0x7f423c04e9e0 con 0x7f4244121890 2024-09-15T21:24:44.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.862+0000 7f4232ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:44.867 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.862+0000 7f4232ffd640 10 monclient: _finish_command 2 = system:0 added bucket h23 type host to location {rack=r1} 2024-09-15T21:24:44.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4248cec640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:44.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4248cec640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:44.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4248cec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:44.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f4244070640 con 0x7f4244121890 2024-09-15T21:24:44.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4232ffd640 1 -- 172.21.15.119:0/3646864879 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h23' already exists v209) ==== 147+0+0 (secure 0 0 0) 0x7f423c013370 con 0x7f4244121890 2024-09-15T21:24:44.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4232ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:44.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.966+0000 7f4232ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h23' already exists 2024-09-15T21:24:44.972 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h23' already exists 2024-09-15T21:24:44.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 msgr2=0x7f422403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:44.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f42340ac000 tx=0x7f42340059c0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.974 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 10 monclient: shutdown 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 msgr2=0x7f4244121d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 0x7f4244121d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f423c003cc0 tx=0x7f423c009e30 comp rx=0 tx=0).stop 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 shutdown_connections 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b970 0x7f422403de30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244121890 0x7f4244121d20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4244134cc0 0x7f4244135140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 --2- 172.21.15.119:0/3646864879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f422c0039c0 0x7f42441cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 >> 172.21.15.119:0/3646864879 conn(0x7f42440a66e0 msgr2=0x7f424413ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 shutdown_connections 2024-09-15T21:24:44.975 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:44.970+0000 7f4248cec640 1 -- 172.21.15.119:0/3646864879 wait complete. 2024-09-15T21:24:44.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:44.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h24 host rack=r1 2024-09-15T21:24:45.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.106+0000 7f332f804640 10 monclient: get_monmap_and_config 2024-09-15T21:24:45.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.106+0000 7f332f804640 10 monclient: build_initial_monmap 2024-09-15T21:24:45.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.106+0000 7f332f804640 10 monclient: monmap: 2024-09-15T21:24:45.112 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:45.112 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:45.112 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:45.112732+0000 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:45.112732+0000 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:45.113 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:45.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 Processor -- start 2024-09-15T21:24:45.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 -- start start 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient: init 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3328135bf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f332809fe70 0x7f332813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): picked mon.noname-c con 0x7f332809fe70 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f332813afb0 0x7f332813d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): picked mon.noname-a con 0x7f332813afb0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33280a0300 con 0x7f3328135bf0 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f33280a3600 con 0x7f332813afb0 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:45.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f33280a1d80 con 0x7f332809fe70 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332f804640 10 monclient(hunting): authenticate will time out at 1403.909490s 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332d5a0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3328135bf0 0x7f3328135fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33832/0 (socket says 172.21.15.119:33832) 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332d5a0640 1 -- 172.21.15.119:0/1706461773 learned_addr learned my addr 172.21.15.119:0/1706461773 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 1 --2- 172.21.15.119:0/1706461773 >> v2:172.21.15.119:3300/0 conn(0x7f332809fe70 0x7f332813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332dda1640 1 -- 172.21.15.119:0/1706461773 >> v2:172.21.15.17:3300/0 conn(0x7f332813afb0 msgr2=0x7f332813d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332dda1640 1 --2- 172.21.15.119:0/1706461773 >> v2:172.21.15.17:3300/0 conn(0x7f332813afb0 0x7f332813d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): get_auth_request con 0x7f332809fe70 auth_method 0 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:45.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f331ffff640 1 -- 172.21.15.119:0/1706461773 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446821310 0 0) 0x7f3324001120 con 0x7f3328135bf0 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f331ffff640 10 monclient(hunting): _negotiate 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f331ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f331ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f331ffff640 10 monclient(hunting): my global_id is 84289 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f331ffff640 1 -- 172.21.15.119:0/1706461773 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3314003530 con 0x7f3328135bf0 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient(hunting): handle_auth_done global_id 84292 payload 274 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 1 -- 172.21.15.119:0/1706461773 >> v2:172.21.15.17:3300/0 conn(0x7f332813afb0 msgr2=0x7f332813d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 1 --2- 172.21.15.119:0/1706461773 >> v2:172.21.15.17:3300/0 conn(0x7f332813afb0 0x7f332813d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 1 -- 172.21.15.119:0/1706461773 >> v1:172.21.15.119:6789/0 conn(0x7f3328135bf0 legacy=0x7f3328135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient: _finish_hunting 0 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 1 monclient: found mon.noname-c 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:45.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.110+0000 7f332cd9f640 1 -- 172.21.15.119:0/1706461773 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33280a2060 con 0x7f332809fe70 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332cd9f640 1 --2- 172.21.15.119:0/1706461773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332809fe70 0x7f332813aa70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33180097c0 tx=0x7f331801c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac85a7064326534 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 1 -- 172.21.15.119:0/1706461773 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180b0070 con 0x7f332809fe70 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: dump: 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:45.123 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: _finish_auth 0 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 5 monclient: authenticate success, global_id 84292 2024-09-15T21:24:45.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 1 -- 172.21.15.119:0/1706461773 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33180ab600 con 0x7f332809fe70 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 1 -- 172.21.15.119:0/1706461773 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180ab9b0 con 0x7f332809fe70 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 4 set_mon_vals no callback set 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331ffff640 10 monclient: dump: 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.125 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: shutdown 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/1706461773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332809fe70 msgr2=0x7f332813aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 --2- 172.21.15.119:0/1706461773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332809fe70 0x7f332813aa70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33180097c0 tx=0x7f331801c310 comp rx=0 tx=0).stop 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/1706461773 shutdown_connections 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 --2- 172.21.15.119:0/1706461773 >> v2:172.21.15.17:3300/0 conn(0x7f332813afb0 0x7f332813d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 --2- 172.21.15.119:0/1706461773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332809fe70 0x7f332813aa70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/1706461773 >> 172.21.15.119:0/1706461773 conn(0x7f33280a66e0 msgr2=0x7f33280a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:45.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/1706461773 shutdown_connections 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/1706461773 wait complete. 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: build_initial_monmap 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: monmap: 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:45.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 Processor -- start 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- start start 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: init 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f332809fe70 0x7f33281320c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): picked mon.noname-a con 0x7f332809fe70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3328135bf0 0x7f3328139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): picked mon.noname-c con 0x7f3328135bf0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 0x7f332809d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): picked mon.noname-b con 0x7f332813afb0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3328140100 con 0x7f332809fe70 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:45.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f332806ecc0 con 0x7f332813afb0 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3328140280 con 0x7f3328135bf0 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient(hunting): authenticate will time out at 1403.914536s 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 0x7f332809d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332d5a0640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f332809fe70 msgr2=0x7f33281320c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:45.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332d5a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f332809fe70 0x7f33281320c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 0x7f332809d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43780/0 (socket says 172.21.15.119:43780) 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 -- 172.21.15.119:0/526389980 learned_addr learned my addr 172.21.15.119:0/526389980 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): get_auth_request con 0x7f332813afb0 auth_method 0 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:45.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient(hunting): handle_auth_done global_id 84295 payload 824 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332cd9f640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3328135bf0 0x7f3328139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 -- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3328135bf0 msgr2=0x7f3328139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3328135bf0 0x7f3328139f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 -- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f332809fe70 msgr2=0x7f33281320c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f332809fe70 0x7f33281320c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient: _finish_hunting 0 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 monclient: found mon.noname-b 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 -- 172.21.15.119:0/526389980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332813a8c0 con 0x7f332813afb0 2024-09-15T21:24:45.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332dda1640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 0x7f332809d550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f332000b790 tx=0x7f332000bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332000c960 con 0x7f332813afb0 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: dump: 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:45.134 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 5 monclient: authenticate success, global_id 84295 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: _renew_subs 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3320010070 con 0x7f332813afb0 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332813acb0 con 0x7f332813afb0 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332000cd80 con 0x7f332813afb0 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:45.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331dffb640 10 monclient: dump: 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f331f7fe640 4 set_mon_vals no callback set 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: _renew_subs 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.114+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281380f0 con 0x7f332813afb0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332f804640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332f804640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332f804640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328135fd0 con 0x7f332813afb0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3320015470 con 0x7f332813afb0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f331dffb640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f330c03b9a0 0x7f330c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 13067+0+0 (secure 0 0 0) 0x7f3320051b20 con 0x7f332813afb0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332d5a0640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f330c03b9a0 0x7f330c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332d5a0640 10 monclient: get_auth_request con 0x7f330c03b9a0 auth_method 0 2024-09-15T21:24:45.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.118+0000 7f332d5a0640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f330c03b9a0 0x7f330c03de60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3324004820 tx=0x7f3324001240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:45.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.122+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f332004e4d0 con 0x7f332813afb0 2024-09-15T21:24:45.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.122+0000 7f331dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:45.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.122+0000 7f331dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:45.300 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.298+0000 7f332f804640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:45.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.298+0000 7f332f804640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:45.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.298+0000 7f332f804640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.302 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.298+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f332806a830 con 0x7f332813afb0 2024-09-15T21:24:45.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.878+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 added bucket h24 type host to location {rack=r1} v210) ==== 168+0+0 (secure 0 0 0) 0x7f3320013c90 con 0x7f332813afb0 2024-09-15T21:24:45.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.878+0000 7f331dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:45.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.878+0000 7f331dffb640 10 monclient: _finish_command 2 = system:0 added bucket h24 type host to location {rack=r1} 2024-09-15T21:24:45.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f332f804640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:45.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f332f804640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:45.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f332f804640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:45.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f3328136d50 con 0x7f332813afb0 2024-09-15T21:24:45.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f331dffb640 1 -- 172.21.15.119:0/526389980 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h24' already exists v210) ==== 147+0+0 (secure 0 0 0) 0x7f332001f3f0 con 0x7f332813afb0 2024-09-15T21:24:45.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f331dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:45.991 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.986+0000 7f331dffb640 10 monclient: _finish_command 3 = system:0 bucket 'h24' already exists 2024-09-15T21:24:45.991 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h24' already exists 2024-09-15T21:24:45.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f330c03b9a0 msgr2=0x7f330c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:45.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f330c03b9a0 0x7f330c03de60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3324004820 tx=0x7f3324001240 comp rx=0 tx=0).stop 2024-09-15T21:24:45.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 10 monclient: shutdown 2024-09-15T21:24:45.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:45.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 msgr2=0x7f332809d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:45.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 0x7f332809d550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f332000b790 tx=0x7f332000bc50 comp rx=0 tx=0).stop 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 shutdown_connections 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f330c03b9a0 0x7f330c03de60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332813afb0 0x7f332809d550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3328135bf0 0x7f3328139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 --2- 172.21.15.119:0/526389980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f332809fe70 0x7f33281320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 >> 172.21.15.119:0/526389980 conn(0x7f33280a66e0 msgr2=0x7f332813c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 shutdown_connections 2024-09-15T21:24:45.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:45.990+0000 7f332f804640 1 -- 172.21.15.119:0/526389980 wait complete. 2024-09-15T21:24:46.008 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:46.008 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h25 host rack=r1 2024-09-15T21:24:46.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.126+0000 7f0b45073640 10 monclient: get_monmap_and_config 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.126+0000 7f0b45073640 10 monclient: build_initial_monmap 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.126+0000 7f0b45073640 10 monclient: monmap: 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:46.131319+0000 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:46.131319+0000 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:46.131 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:46.132 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:46.132 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.126+0000 7f0b45073640 1 Processor -- start 2024-09-15T21:24:46.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 1 -- start start 2024-09-15T21:24:46.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient: init 2024-09-15T21:24:46.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:46.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0b4006d980 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0b40070840 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b40134c80 0x7f0b4013d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient(hunting): picked mon.noname-a con 0x7f0b40134c80 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b40071800 con 0x7f0b4006d980 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b40071ae0 con 0x7f0b40070840 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0b40069c80 con 0x7f0b40134c80 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 10 monclient(hunting): authenticate will time out at 1404.928821s 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3e59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0b40070840 0x7f0b40139b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33836/0 (socket says 172.21.15.119:33836) 2024-09-15T21:24:46.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3e59c640 1 -- 172.21.15.119:0/357525122 learned_addr learned my addr 172.21.15.119:0/357525122 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/357525122 >> v1:172.21.15.17:6789/0 conn(0x7f0b4006d980 legacy=0x7f0b400735d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3f59e640 1 -- 172.21.15.119:0/357525122 >> v2:172.21.15.17:3300/0 conn(0x7f0b40134c80 msgr2=0x7f0b4013d270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3f59e640 1 --2- 172.21.15.119:0/357525122 >> v2:172.21.15.17:3300/0 conn(0x7f0b40134c80 0x7f0b4013d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003889943 0 0) 0x7f0b40071ae0 con 0x7f0b40070840 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): _negotiate 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): my global_id is 84298 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b28003530 con 0x7f0b40070840 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3811606891 0 0) 0x7f0b28003530 con 0x7f0b40070840 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 >> v2:172.21.15.17:3300/0 conn(0x7f0b40134c80 msgr2=0x7f0b4013d270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 --2- 172.21.15.119:0/357525122 >> v2:172.21.15.17:3300/0 conn(0x7f0b40134c80 0x7f0b4013d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 >> v1:172.21.15.17:6789/0 conn(0x7f0b4006d980 legacy=0x7f0b400735d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b40071dc0 con 0x7f0b40070840 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0b34001700 con 0x7f0b40070840 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:46.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: dump: 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:46.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 >> v1:172.21.15.119:6789/0 conn(0x7f0b40070840 legacy=0x7f0b40139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:46.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:46.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): picked mon.b con 0x7f0b280039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:46.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b28004810 0x7f0b28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): picked mon.a con 0x7f0b28004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b28005420 0x7f0b2800ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): picked mon.c con 0x7f0b28005420 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40069c80 con 0x7f0b28004810 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2800c330 con 0x7f0b280039c0 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2800c530 con 0x7f0b28005420 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3dd9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:46.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3e59c640 1 -- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b28004810 msgr2=0x7f0b28004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b28004810 0x7f0b28004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): get_auth_request con 0x7f0b280039c0 auth_method 0 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_done global_id 84298 payload 293 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3f59e640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b28005420 0x7f0b2800ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b28005420 msgr2=0x7f0b2800ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b28005420 0x7f0b2800ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b28004810 msgr2=0x7f0b28004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b28004810 0x7f0b28004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 monclient: found mon.b 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3f59e640 1 -- 172.21.15.119:0/357525122 reap_dead start 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/357525122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b28003530 con 0x7f0b280039c0 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient: _finish_auth 0 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 10 monclient: _check_auth_tickets 2024-09-15T21:24:46.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b3ed9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:46.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 5 monclient: authenticate success, global_id 84298 2024-09-15T21:24:46.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.130+0000 7f0b45073640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b28003de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b30003940 tx=0x7f0b3001c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6feb3f083b430b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30009520 con 0x7f0b280039c0 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: dump: 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:46.150 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b30003bc0 con 0x7f0b280039c0 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 4 set_mon_vals no callback set 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/357525122 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30005070 con 0x7f0b280039c0 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3dd9b640 10 monclient: dump: 2024-09-15T21:24:46.151 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: shutdown 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/357525122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 msgr2=0x7f0b28003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:46.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b28003de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b30003940 tx=0x7f0b3001c420 comp rx=0 tx=0).stop 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/357525122 shutdown_connections 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 --2- 172.21.15.119:0/357525122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b28003de0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/357525122 >> 172.21.15.119:0/357525122 conn(0x7f0b4006cbf0 msgr2=0x7f0b4006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/357525122 shutdown_connections 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/357525122 wait complete. 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: build_initial_monmap 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: monmap: 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.153 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:46.154 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:46.154 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:46.154 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 Processor -- start 2024-09-15T21:24:46.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- start start 2024-09-15T21:24:46.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: init 2024-09-15T21:24:46.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:46.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:46.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b401cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): picked mon.noname-b con 0x7f0b280039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b401233c0 0x7f0b401ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): picked mon.noname-a con 0x7f0b401233c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b401ce630 0x7f0b40120da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): picked mon.noname-c con 0x7f0b401ce630 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400a2de0 con 0x7f0b401233c0 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400a2c60 con 0x7f0b280039c0 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0b400a2f60 con 0x7f0b401ce630 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient(hunting): authenticate will time out at 1404.934527s 2024-09-15T21:24:46.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b401cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b401cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43790/0 (socket says 172.21.15.119:43790) 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3104349415 learned_addr learned my addr 172.21.15.119:0/3104349415 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): get_auth_request con 0x7f0b280039c0 auth_method 0 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3e59c640 1 -- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b401233c0 msgr2=0x7f0b401ce0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b401233c0 0x7f0b401ce0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:46.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient(hunting): handle_auth_done global_id 84301 payload 824 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3f59e640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b401ce630 0x7f0b40120da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b401ce630 msgr2=0x7f0b40120da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b401ce630 0x7f0b40120da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b401233c0 msgr2=0x7f0b401ce0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b401233c0 0x7f0b401ce0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 monclient: found mon.noname-b 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3104349415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b40122860 con 0x7f0b280039c0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b401cdbb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b3001c460 tx=0x7f0b28005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30009500 con 0x7f0b280039c0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:46.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: dump: 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:46.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 5 monclient: authenticate success, global_id 84301 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b300039c0 con 0x7f0b280039c0 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: _renew_subs 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b300b7660 con 0x7f0b280039c0 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b400a5320 con 0x7f0b280039c0 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b277fe640 10 monclient: dump: 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:46.161 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b3d59a640 4 set_mon_vals no callback set 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: _renew_subs 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.134+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b401212e0 con 0x7f0b280039c0 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b45073640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:46.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b45073640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b45073640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b40071ae0 con 0x7f0b280039c0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0b300c0440 con 0x7f0b280039c0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b277fe640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2c03b970 0x7f0b2c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 13280+0+0 (secure 0 0 0) 0x7f0b300ebc40 con 0x7f0b280039c0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2c03b970 0x7f0b2c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b3e59c640 10 monclient: get_auth_request con 0x7f0b2c03b970 auth_method 0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.138+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2c03b970 0x7f0b2c03de30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0b400a3510 tx=0x7f0b34001580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.142+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0b300f2080 con 0x7f0b280039c0 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.142+0000 7f0b277fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:46.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.142+0000 7f0b277fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:46.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.314+0000 7f0b45073640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:46.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.314+0000 7f0b45073640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:46.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.314+0000 7f0b45073640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.314+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f0b40074bd0 con 0x7f0b280039c0 2024-09-15T21:24:46.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.878+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 added bucket h25 type host to location {rack=r1} v211) ==== 168+0+0 (secure 0 0 0) 0x7f0b30016130 con 0x7f0b280039c0 2024-09-15T21:24:46.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.882+0000 7f0b277fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:46.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.882+0000 7f0b277fe640 10 monclient: _finish_command 2 = system:0 added bucket h25 type host to location {rack=r1} 2024-09-15T21:24:46.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:46.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:46.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:46.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f0b401215d0 con 0x7f0b280039c0 2024-09-15T21:24:46.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b277fe640 1 -- 172.21.15.119:0/3104349415 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h25' already exists v211) ==== 147+0+0 (secure 0 0 0) 0x7f0b30003b70 con 0x7f0b280039c0 2024-09-15T21:24:46.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b277fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:46.994 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b277fe640 10 monclient: _finish_command 3 = system:0 bucket 'h25' already exists 2024-09-15T21:24:46.994 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h25' already exists 2024-09-15T21:24:46.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2c03b970 msgr2=0x7f0b2c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:46.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2c03b970 0x7f0b2c03de30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0b400a3510 tx=0x7f0b34001580 comp rx=0 tx=0).stop 2024-09-15T21:24:46.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 10 monclient: shutdown 2024-09-15T21:24:46.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:46.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 msgr2=0x7f0b401cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:46.996 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b401cdbb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b3001c460 tx=0x7f0b28005840 comp rx=0 tx=0).stop 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 shutdown_connections 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2c03b970 0x7f0b2c03de30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b401ce630 0x7f0b40120da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b401233c0 0x7f0b401ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 --2- 172.21.15.119:0/3104349415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b280039c0 0x7f0b401cdbb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.990+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 >> 172.21.15.119:0/3104349415 conn(0x7f0b4006cbf0 msgr2=0x7f0b4009fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.994+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 shutdown_connections 2024-09-15T21:24:46.997 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:46.994+0000 7f0b45073640 1 -- 172.21.15.119:0/3104349415 wait complete. 2024-09-15T21:24:47.011 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:47.011 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h26 host rack=r1 2024-09-15T21:24:47.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.126+0000 7f22bd8e5640 10 monclient: get_monmap_and_config 2024-09-15T21:24:47.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.126+0000 7f22bd8e5640 10 monclient: build_initial_monmap 2024-09-15T21:24:47.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.126+0000 7f22bd8e5640 10 monclient: monmap: 2024-09-15T21:24:47.132 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:47.132 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:47.132756+0000 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:47.132756+0000 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:47.133 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 Processor -- start 2024-09-15T21:24:47.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 -- start start 2024-09-15T21:24:47.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient: init 2024-09-15T21:24:47.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:47.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f22b8136b90 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22b80a10d0 0x7f22b813cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): picked mon.noname-a con 0x7f22b80a10d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22b813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): picked mon.noname-b con 0x7f22b80a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b80a18f0 con 0x7f22b8136b90 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f22b809fc40 con 0x7f22b80a10d0 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f22b80a1bd0 con 0x7f22b80a25b0 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 10 monclient(hunting): authenticate will time out at 1405.929552s 2024-09-15T21:24:47.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f22b8136b90 0x7f22b8136f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33848/0 (socket says 172.21.15.119:33848) 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 -- 172.21.15.119:0/2914593534 learned_addr learned my addr 172.21.15.119:0/2914593534 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b67fc640 1 -- 172.21.15.119:0/2914593534 >> v2:172.21.15.17:3300/0 conn(0x7f22b80a10d0 msgr2=0x7f22b813cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b67fc640 1 --2- 172.21.15.119:0/2914593534 >> v2:172.21.15.17:3300/0 conn(0x7f22b80a10d0 0x7f22b813cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896013582 0 0) 0x7f22a8001120 con 0x7f22b8136b90 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): my global_id is 84304 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22a0003530 con 0x7f22b8136b90 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22b813f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 10 monclient(hunting): get_auth_request con 0x7f22b80a25b0 auth_method 0 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:47.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1223105290 0 0) 0x7f22a0003530 con 0x7f22b8136b90 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 msgr2=0x7f22b813f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22b813f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 >> v2:172.21.15.17:3300/0 conn(0x7f22b80a10d0 msgr2=0x7f22b813cc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 --2- 172.21.15.119:0/2914593534 >> v2:172.21.15.17:3300/0 conn(0x7f22b80a10d0 0x7f22b813cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b80a3660 con 0x7f22b8136b90 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:47.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f22a80018d0 con 0x7f22b8136b90 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 5 monclient: authenticate success, global_id 84304 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: dump: 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.142 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 >> v1:172.21.15.119:6789/0 conn(0x7f22b8136b90 legacy=0x7f22b8136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a00039c0 0x7f22a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): picked mon.b con 0x7f22a00039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): picked mon.c con 0x7f22b80a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a0004dc0 0x7f22a0005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.143 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): picked mon.a con 0x7f22a0004dc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f22a000bc40 con 0x7f22a0004dc0 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22a000bdf0 con 0x7f22a00039c0 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f22a000bff0 con 0x7f22b80a25b0 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b5ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22bd8e5640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a00039c0 0x7f22a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): get_auth_request con 0x7f22a00039c0 auth_method 0 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:47.144 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 1 -- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a0004dc0 msgr2=0x7f22a0005240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b77fe640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a0004dc0 0x7f22a0005240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b67fc640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_done global_id 84304 payload 293 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 -- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 msgr2=0x7f22a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 -- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a0004dc0 msgr2=0x7f22a0005240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a0004dc0 0x7f22a0005240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.145 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 monclient: found mon.b 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 1 -- 172.21.15.119:0/2914593534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0003530 con 0x7f22a00039c0 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b67fc640 10 monclient: get_auth_request con 0x7f22b80a25b0 auth_method 0 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.130+0000 7f22b67fc640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b80a25b0 0x7f22a0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b6ffd640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a00039c0 0x7f22a0003de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c4c0 tx=0x7f22a801cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a5871eec01b90b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:47.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a800c0e0 con 0x7f22a00039c0 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: dump: 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:47.147 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a80a9020 con 0x7f22a00039c0 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 1 -- 172.21.15.119:0/2914593534 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8005b00 con 0x7f22a00039c0 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 4 set_mon_vals no callback set 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b5ffb640 10 monclient: dump: 2024-09-15T21:24:47.148 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: shutdown 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- 172.21.15.119:0/2914593534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a00039c0 msgr2=0x7f22a0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:47.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/2914593534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a00039c0 0x7f22a0003de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c4c0 tx=0x7f22a801cab0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b77fe640 1 -- 172.21.15.119:0/2914593534 reap_dead start 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- 172.21.15.119:0/2914593534 shutdown_connections 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- 172.21.15.119:0/2914593534 >> 172.21.15.119:0/2914593534 conn(0x7f22b80a66e0 msgr2=0x7f22b80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- 172.21.15.119:0/2914593534 shutdown_connections 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- 172.21.15.119:0/2914593534 wait complete. 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: build_initial_monmap 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: monmap: 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:47.150 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.151 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:47.151 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:47.151 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:47.151 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 Processor -- start 2024-09-15T21:24:47.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- start start 2024-09-15T21:24:47.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: init 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a00039c0 0x7f22b81cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): picked mon.noname-a con 0x7f22a00039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 0x7f22b8134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): picked mon.noname-b con 0x7f22b811fe90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b8135220 0x7f22b811d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): picked mon.noname-c con 0x7f22b8135220 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f22b813fc80 con 0x7f22a00039c0 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22b813fb00 con 0x7f22b811fe90 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:47.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f22b813fe00 con 0x7f22b8135220 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22bd8e5640 10 monclient(hunting): authenticate will time out at 1405.935461s 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b6ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a00039c0 msgr2=0x7f22b81cf3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a00039c0 0x7f22b81cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 0x7f22b8134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 0x7f22b8134ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43798/0 (socket says 172.21.15.119:43798) 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 -- 172.21.15.119:0/4276822571 learned_addr learned my addr 172.21.15.119:0/4276822571 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): get_auth_request con 0x7f22b811fe90 auth_method 0 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:47.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient(hunting): handle_auth_done global_id 84307 payload 824 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b77fe640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b8135220 0x7f22b811d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 -- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b8135220 msgr2=0x7f22b811d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b8135220 0x7f22b811d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 -- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a00039c0 msgr2=0x7f22b81cf3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a00039c0 0x7f22b81cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 monclient: found mon.noname-b 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:47.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.134+0000 7f22b67fc640 1 -- 172.21.15.119:0/4276822571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b811faf0 con 0x7f22b811fe90 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22b67fc640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 0x7f22b8134ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c820 tx=0x7f22ac00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac003dd0 con 0x7f22b811fe90 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: dump: 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.159 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: _finish_auth 0 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac016070 con 0x7f22b811fe90 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0052b0 con 0x7f22b811fe90 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:47.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 10 monclient: dump: 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 5 monclient: authenticate success, global_id 84307 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 monclient: _renew_subs 2024-09-15T21:24:47.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22b4ff9640 4 set_mon_vals no callback set 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b80a5790 con 0x7f22b811fe90 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 monclient: _renew_subs 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b809d240 con 0x7f22b811fe90 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f22ac005450 con 0x7f22b811fe90 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b8136f70 con 0x7f22b811fe90 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f22a403b9a0 0x7f22a403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:47.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) ==== 13493+0+0 (secure 0 0 0) 0x7f22ac051d40 con 0x7f22b811fe90 2024-09-15T21:24:47.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22b6ffd640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f22a403b9a0 0x7f22a403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:47.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22b6ffd640 10 monclient: get_auth_request con 0x7f22a403b9a0 auth_method 0 2024-09-15T21:24:47.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.138+0000 7f22b6ffd640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f22a403b9a0 0x7f22a403de60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c490 tx=0x7f22b80a2330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:47.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.142+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f22b80a5790 con 0x7f22b811fe90 2024-09-15T21:24:47.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.142+0000 7f229ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:47.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.142+0000 7f229ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:47.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.302+0000 7f22bd8e5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:47.306 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.302+0000 7f22bd8e5640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:47.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.302+0000 7f22bd8e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:47.307 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.302+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f22b806a830 con 0x7f22b811fe90 2024-09-15T21:24:47.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.898+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 added bucket h26 type host to location {rack=r1} v212) ==== 168+0+0 (secure 0 0 0) 0x7f22ac04e540 con 0x7f22b811fe90 2024-09-15T21:24:47.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.898+0000 7f229ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:47.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:47.898+0000 7f229ffff640 10 monclient: _finish_command 2 = system:0 added bucket h26 type host to location {rack=r1} 2024-09-15T21:24:48.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f22bd8e5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:48.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f22bd8e5640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:48.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f22bd8e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f22b8121310 con 0x7f22b811fe90 2024-09-15T21:24:48.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f229ffff640 1 -- 172.21.15.119:0/4276822571 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h26' already exists v212) ==== 147+0+0 (secure 0 0 0) 0x7f22ac007dd0 con 0x7f22b811fe90 2024-09-15T21:24:48.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f229ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:48.011 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.006+0000 7f229ffff640 10 monclient: _finish_command 3 = system:0 bucket 'h26' already exists 2024-09-15T21:24:48.011 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h26' already exists 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f22a403b9a0 msgr2=0x7f22a403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f22a403b9a0 0x7f22a403de60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c490 tx=0x7f22b80a2330 comp rx=0 tx=0).stop 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 10 monclient: shutdown 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 msgr2=0x7f22b8134ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 0x7f22b8134ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c820 tx=0x7f22ac00cce0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 shutdown_connections 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f22a403b9a0 0x7f22a403de60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.013 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f22b8135220 0x7f22b811d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b811fe90 0x7f22b8134ce0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 --2- 172.21.15.119:0/4276822571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22a00039c0 0x7f22b81cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 >> 172.21.15.119:0/4276822571 conn(0x7f22b80a66e0 msgr2=0x7f22b80a4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:48.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 shutdown_connections 2024-09-15T21:24:48.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.010+0000 7f22bd8e5640 1 -- 172.21.15.119:0/4276822571 wait complete. 2024-09-15T21:24:48.028 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:48.028 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h27 host rack=r1 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient: get_monmap_and_config 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient: build_initial_monmap 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient: monmap: 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:48.146473+0000 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:48.146473+0000 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.146 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:48.147 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:48.147 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:48.147 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:48.147 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:48.147 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 1 Processor -- start 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 1 -- start start 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient: init 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff38c136b90 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:48.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff38c0a10d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 0x7ff38c140380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient(hunting): picked mon.noname-b con 0x7ff38c0a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0a18f0 con 0x7ff38c0a10d0 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0a3660 con 0x7ff38c136b90 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c09fc40 con 0x7ff38c0a25b0 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 10 monclient(hunting): authenticate will time out at 1406.943058s 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38ad9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff38c136b90 0x7ff38c136f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33856/0 (socket says 172.21.15.119:33856) 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38ad9d640 1 -- 172.21.15.119:0/129293003 learned_addr learned my addr 172.21.15.119:0/129293003 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38a59c640 1 -- 172.21.15.119:0/129293003 >> v1:172.21.15.17:6789/0 conn(0x7ff38c0a10d0 legacy=0x7ff38c13cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 0x7ff38c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:48.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245938475 0 0) 0x7ff38c0a3660 con 0x7ff38c136b90 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): my global_id is 84310 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff374003530 con 0x7ff38c136b90 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): get_auth_request con 0x7ff38c0a25b0 auth_method 0 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff38b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:48.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2425460789 0 0) 0x7ff374003530 con 0x7ff38c136b90 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 msgr2=0x7ff38c140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 0x7ff38c140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 >> v1:172.21.15.17:6789/0 conn(0x7ff38c0a10d0 legacy=0x7ff38c13cc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c137680 con 0x7ff38c136b90 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: _finish_auth 0 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff37c001700 con 0x7ff38c136b90 2024-09-15T21:24:48.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 5 monclient: authenticate success, global_id 84310 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: dump: 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:48.156 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 >> v1:172.21.15.119:6789/0 conn(0x7ff38c136b90 legacy=0x7ff38c136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3740039c0 0x7ff374003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): picked mon.b con 0x7ff3740039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 0x7ff374004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): picked mon.c con 0x7ff38c0a25b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff374004d50 0x7ff3740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): picked mon.a con 0x7ff374004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff37400bb60 con 0x7ff374004d50 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff37400bd10 con 0x7ff3740039c0 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff37400bf10 con 0x7ff38c0a25b0 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff389d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.142+0000 7ff3911ef640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38b59e640 1 -- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff374004d50 msgr2=0x7ff3740051d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3740039c0 0x7ff374003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38b59e640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff374004d50 0x7ff3740051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): get_auth_request con 0x7ff3740039c0 auth_method 0 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:48.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38a59c640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 0x7ff374004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient(hunting): handle_auth_done global_id 84310 payload 293 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 -- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 msgr2=0x7ff374004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c0a25b0 0x7ff374004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 -- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff374004d50 msgr2=0x7ff3740051d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff374004d50 0x7ff3740051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 monclient: found mon.b 2024-09-15T21:24:48.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 -- 172.21.15.119:0/129293003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff374003530 con 0x7ff3740039c0 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient: _finish_auth 0 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 10 monclient: _check_auth_tickets 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3740039c0 0x7ff374003dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c210 tx=0x7ff37c01cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2bf24a0db043e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c007d20 con 0x7ff3740039c0 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: dump: 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:48.160 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c0a9020 con 0x7ff3740039c0 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 1 -- 172.21.15.119:0/129293003 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c005ae0 con 0x7ff3740039c0 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:48.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 4 set_mon_vals no callback set 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff389d9b640 10 monclient: dump: 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:48.162 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 10 monclient: shutdown 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 1 -- 172.21.15.119:0/129293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3740039c0 msgr2=0x7ff374003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 1 --2- 172.21.15.119:0/129293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3740039c0 0x7ff374003dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c210 tx=0x7ff37c01cbc0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff38b59e640 1 -- 172.21.15.119:0/129293003 reap_dead start 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 1 -- 172.21.15.119:0/129293003 shutdown_connections 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 1 -- 172.21.15.119:0/129293003 >> 172.21.15.119:0/129293003 conn(0x7ff38c0a66e0 msgr2=0x7ff38c0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 1 -- 172.21.15.119:0/129293003 shutdown_connections 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 1 -- 172.21.15.119:0/129293003 wait complete. 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 10 monclient: build_initial_monmap 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.146+0000 7ff3911ef640 10 monclient: monmap: 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:48.163 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 Processor -- start 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- start start 2024-09-15T21:24:48.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: init 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740039c0 0x7ff38c135830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): picked mon.noname-a con 0x7ff3740039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c135d70 0x7ff38c11d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): picked mon.noname-c con 0x7ff38c135d70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 0x7ff38c11dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): picked mon.noname-b con 0x7ff38c11d8f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c140d70 con 0x7ff3740039c0 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c140bf0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c140ef0 con 0x7ff38c135d70 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient(hunting): authenticate will time out at 1406.948908s 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 0x7ff38c11dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 0x7ff38c11dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43810/0 (socket says 172.21.15.119:43810) 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 -- 172.21.15.119:0/3603837818 learned_addr learned my addr 172.21.15.119:0/3603837818 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): get_auth_request con 0x7ff38c11d8f0 auth_method 0 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:48.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38ad9d640 1 -- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740039c0 msgr2=0x7ff38c135830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740039c0 0x7ff38c135830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient(hunting): handle_auth_done global_id 84313 payload 824 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38a59c640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c135d70 0x7ff38c11d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 -- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c135d70 msgr2=0x7ff38c11d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c135d70 0x7ff38c11d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 -- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740039c0 msgr2=0x7ff38c135830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740039c0 0x7ff38c135830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient: _finish_hunting 0 2024-09-15T21:24:48.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 monclient: found mon.noname-b 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 -- 172.21.15.119:0/3603837818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c1365d0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38b59e640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 0x7ff38c11dda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0743c0 tx=0x7ff38400d560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38401a070 con 0x7ff38c11d8f0 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: dump: 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.172 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840039a0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384003dd0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 10 monclient: dump: 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 5 monclient: authenticate success, global_id 84313 2024-09-15T21:24:48.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _renew_subs 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c120ef0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38959a640 4 set_mon_vals no callback set 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _renew_subs 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c11e2e0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c120cf0 con 0x7ff38c11d8f0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff384004030 con 0x7ff38c11d8f0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff37803b970 0x7ff37803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(212..212 src has 1..212) ==== 13706+0+0 (secure 0 0 0) 0x7ff384052440 con 0x7ff38c11d8f0 2024-09-15T21:24:48.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff37803b970 0x7ff37803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:48.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38ad9d640 10 monclient: get_auth_request con 0x7ff37803b970 auth_method 0 2024-09-15T21:24:48.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.150+0000 7ff38ad9d640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff37803b970 0x7ff37803de30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0aa000 tx=0x7ff38c0a2330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:48.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.154+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff384010070 con 0x7ff38c11d8f0 2024-09-15T21:24:48.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.154+0000 7ff3737fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:48.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.154+0000 7ff3737fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:48.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.314+0000 7ff3911ef640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:48.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.314+0000 7ff3911ef640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:48.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.314+0000 7ff3911ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:48.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.314+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ff38c136f70 con 0x7ff38c11d8f0 2024-09-15T21:24:48.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.906+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 added bucket h27 type host to location {rack=r1} v213) ==== 168+0+0 (secure 0 0 0) 0x7ff38404eb70 con 0x7ff38c11d8f0 2024-09-15T21:24:48.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.910+0000 7ff3737fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:48.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:48.910+0000 7ff3737fe640 10 monclient: _finish_command 2 = system:0 added bucket h27 type host to location {rack=r1} 2024-09-15T21:24:49.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3911ef640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:49.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3911ef640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:49.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3911ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ff38c13afe0 con 0x7ff38c11d8f0 2024-09-15T21:24:49.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3737fe640 1 -- 172.21.15.119:0/3603837818 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h27' already exists v213) ==== 147+0+0 (secure 0 0 0) 0x7ff384015780 con 0x7ff38c11d8f0 2024-09-15T21:24:49.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3737fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:49.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.014+0000 7ff3737fe640 10 monclient: _finish_command 3 = system:0 bucket 'h27' already exists 2024-09-15T21:24:49.020 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h27' already exists 2024-09-15T21:24:49.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff37803b970 msgr2=0x7ff37803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:49.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff37803b970 0x7ff37803de30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0aa000 tx=0x7ff38c0a2330 comp rx=0 tx=0).stop 2024-09-15T21:24:49.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 10 monclient: shutdown 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 msgr2=0x7ff38c11dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 0x7ff38c11dda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0743c0 tx=0x7ff38400d560 comp rx=0 tx=0).stop 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 shutdown_connections 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff37803b970 0x7ff37803de30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff38c11d8f0 0x7ff38c11dda0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff38c135d70 0x7ff38c11d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 --2- 172.21.15.119:0/3603837818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740039c0 0x7ff38c135830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 >> 172.21.15.119:0/3603837818 conn(0x7ff38c0a66e0 msgr2=0x7ff38c09e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 shutdown_connections 2024-09-15T21:24:49.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.018+0000 7ff3911ef640 1 -- 172.21.15.119:0/3603837818 wait complete. 2024-09-15T21:24:49.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:49.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h28 host rack=r1 2024-09-15T21:24:49.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient: get_monmap_and_config 2024-09-15T21:24:49.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient: build_initial_monmap 2024-09-15T21:24:49.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient: monmap: 2024-09-15T21:24:49.160 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:49.160846+0000 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:49.160846+0000 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:49.161 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:49.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 Processor -- start 2024-09-15T21:24:49.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 -- start start 2024-09-15T21:24:49.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient: init 2024-09-15T21:24:49.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:49.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6dfc136b90 0x7f6dfc136f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): picked mon.noname-c con 0x7f6dfc136b90 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc137930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): picked mon.noname-b con 0x7f6dfc1374b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6dfc0a10d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dfc0a18f0 con 0x7f6dfc0a10d0 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6dfc09fc40 con 0x7f6dfc136b90 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc0a1bd0 con 0x7f6dfc1374b0 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:49.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e031f3640 10 monclient(hunting): authenticate will time out at 1407.957493s 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6dfc136b90 0x7f6dfc136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6dfc136b90 0x7f6dfc136f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43824/0 (socket says 172.21.15.119:43824) 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 -- 172.21.15.119:0/4005729697 learned_addr learned my addr 172.21.15.119:0/4005729697 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): get_auth_request con 0x7f6dfc136b90 auth_method 0 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e01790640 1 -- 172.21.15.119:0/4005729697 >> v1:172.21.15.17:6789/0 conn(0x7f6dfc0a10d0 legacy=0x7f6dfc137e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient(hunting): handle_auth_done global_id 84316 payload 274 2024-09-15T21:24:49.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 -- 172.21.15.119:0/4005729697 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 msgr2=0x7f6dfc137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6dfbfff640 1 --2- 172.21.15.119:0/4005729697 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 --2- 172.21.15.119:0/4005729697 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc137930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 -- 172.21.15.119:0/4005729697 >> v1:172.21.15.17:6789/0 conn(0x7f6dfc0a10d0 legacy=0x7f6dfc137e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient: _finish_hunting 0 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 monclient: found mon.noname-c 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.158+0000 7f6e00f8f640 1 -- 172.21.15.119:0/4005729697 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc0a3660 con 0x7f6dfc136b90 2024-09-15T21:24:49.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e00f8f640 1 --2- 172.21.15.119:0/4005729697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc136f70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc09d010 tx=0x7f6df401c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a6d69430081583 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:49.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 1 -- 172.21.15.119:0/4005729697 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40b0070 con 0x7f6dfc136b90 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: dump: 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:49.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 5 monclient: authenticate success, global_id 84316 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 1 -- 172.21.15.119:0/4005729697 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df40a4e50 con 0x7f6dfc136b90 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 4 set_mon_vals no callback set 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 1 -- 172.21.15.119:0/4005729697 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40ab820 con 0x7f6dfc136b90 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfb7fe640 10 monclient: dump: 2024-09-15T21:24:49.173 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: shutdown 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- 172.21.15.119:0/4005729697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 msgr2=0x7f6dfc136f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:49.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 --2- 172.21.15.119:0/4005729697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc136f70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc09d010 tx=0x7f6df401c220 comp rx=0 tx=0).stop 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- 172.21.15.119:0/4005729697 shutdown_connections 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 --2- 172.21.15.119:0/4005729697 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 --2- 172.21.15.119:0/4005729697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc136f70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- 172.21.15.119:0/4005729697 >> 172.21.15.119:0/4005729697 conn(0x7f6dfc0a66e0 msgr2=0x7f6dfc0a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- 172.21.15.119:0/4005729697 shutdown_connections 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- 172.21.15.119:0/4005729697 wait complete. 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: build_initial_monmap 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: monmap: 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:49.175 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 Processor -- start 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- start start 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: init 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc0a10d0 0x7f6dfc13dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): picked mon.noname-a con 0x7f6dfc0a10d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc13be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:49.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): picked mon.noname-b con 0x7f6dfc136b90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc131b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): picked mon.noname-c con 0x7f6dfc1374b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc142700 con 0x7f6dfc0a10d0 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc142580 con 0x7f6dfc136b90 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc142880 con 0x7f6dfc1374b0 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 10 monclient(hunting): authenticate will time out at 1407.962663s 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e00f8f640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc0a10d0 msgr2=0x7f6dfc13dfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e00f8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc0a10d0 0x7f6dfc13dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:49.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc13be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc13be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43828/0 (socket says 172.21.15.119:43828) 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 -- 172.21.15.119:0/2278655943 learned_addr learned my addr 172.21.15.119:0/2278655943 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): get_auth_request con 0x7f6dfc136b90 auth_method 0 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e01790640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc131b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e01790640 10 monclient(hunting): get_auth_request con 0x7f6dfc1374b0 auth_method 0 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e01790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:49.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e01790640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e01790640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient(hunting): handle_auth_done global_id 84319 payload 824 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 -- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 msgr2=0x7f6dfc131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc131b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 -- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc0a10d0 msgr2=0x7f6dfc13dfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc0a10d0 0x7f6dfc13dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 monclient: found mon.noname-b 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 -- 172.21.15.119:0/2278655943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc09d240 con 0x7f6dfc136b90 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfbfff640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc13be70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c7f0 tx=0x7f6de400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4015070 con 0x7f6dfc136b90 2024-09-15T21:24:49.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: dump: 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:49.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6e031f3640 5 monclient: authenticate success, global_id 84319 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de4007820 con 0x7f6dfc136b90 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4007c50 con 0x7f6dfc136b90 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6df97fa640 10 monclient: dump: 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.181 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.162+0000 7f6dfaffd640 4 set_mon_vals no callback set 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 monclient: _renew_subs 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc13d470 con 0x7f6dfc136b90 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 monclient: _renew_subs 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc132080 con 0x7f6dfc136b90 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6de4018070 con 0x7f6dfc136b90 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc13cec0 con 0x7f6dfc136b90 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6df97fa640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6ddc03b970 0x7f6ddc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 13919+0+0 (secure 0 0 0) 0x7f6de4052170 con 0x7f6dfc136b90 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e00f8f640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6ddc03b970 0x7f6ddc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e00f8f640 10 monclient: get_auth_request con 0x7f6ddc03b970 auth_method 0 2024-09-15T21:24:49.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.166+0000 7f6e00f8f640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6ddc03b970 0x7f6ddc03de30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ad000 tx=0x7f6df4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:49.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.170+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6de404e7d0 con 0x7f6dfc136b90 2024-09-15T21:24:49.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.170+0000 7f6df97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:49.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.170+0000 7f6df97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:49.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.338+0000 7f6e031f3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:49.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.338+0000 7f6e031f3640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:49.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.338+0000 7f6e031f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:49.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.338+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f6dfc136f70 con 0x7f6dfc136b90 2024-09-15T21:24:49.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.918+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 added bucket h28 type host to location {rack=r1} v214) ==== 168+0+0 (secure 0 0 0) 0x7f6de4011bf0 con 0x7f6dfc136b90 2024-09-15T21:24:49.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.918+0000 7f6df97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:49.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:49.918+0000 7f6df97fa640 10 monclient: _finish_command 2 = system:0 added bucket h28 type host to location {rack=r1} 2024-09-15T21:24:50.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6e031f3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:50.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6e031f3640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:50.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6e031f3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.031 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f6dfc0008e0 con 0x7f6dfc136b90 2024-09-15T21:24:50.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6df97fa640 1 -- 172.21.15.119:0/2278655943 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h28' already exists v214) ==== 147+0+0 (secure 0 0 0) 0x7f6de4054110 con 0x7f6dfc136b90 2024-09-15T21:24:50.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6df97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:50.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.026+0000 7f6df97fa640 10 monclient: _finish_command 3 = system:0 bucket 'h28' already exists 2024-09-15T21:24:50.032 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h28' already exists 2024-09-15T21:24:50.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6ddc03b970 msgr2=0x7f6ddc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:50.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6ddc03b970 0x7f6ddc03de30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ad000 tx=0x7f6df4008040 comp rx=0 tx=0).stop 2024-09-15T21:24:50.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 10 monclient: shutdown 2024-09-15T21:24:50.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:50.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 msgr2=0x7f6dfc13be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:50.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc13be70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c7f0 tx=0x7f6de400ccb0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 shutdown_connections 2024-09-15T21:24:50.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6ddc03b970 0x7f6ddc03de30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dfc1374b0 0x7f6dfc131b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dfc136b90 0x7f6dfc13be70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 --2- 172.21.15.119:0/2278655943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc0a10d0 0x7f6dfc13dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 >> 172.21.15.119:0/2278655943 conn(0x7f6dfc0a66e0 msgr2=0x7f6dfc073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:50.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 shutdown_connections 2024-09-15T21:24:50.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.030+0000 7f6e031f3640 1 -- 172.21.15.119:0/2278655943 wait complete. 2024-09-15T21:24:50.053 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:50.054 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h29 host rack=r1 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient: get_monmap_and_config 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient: build_initial_monmap 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient: monmap: 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:50.166777+0000 2024-09-15T21:24:50.166 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:50.166777+0000 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:50.167 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 Processor -- start 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 -- start start 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient: init 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f977009d550 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f977009fae0 0x7f9770139c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): picked mon.noname-c con 0x7f977009fae0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:50.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f977009e040 0x7f977013c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): picked mon.noname-a con 0x7f977009e040 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700a3600 con 0x7f977009d550 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f977009f1b0 con 0x7f977009e040 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f97700a2920 con 0x7f977009fae0 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f9775af3640 10 monclient(hunting): authenticate will time out at 1408.963392s 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976ffff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f977009e040 msgr2=0x7f977013c1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976f7fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f977009d550 legacy=0x7f977009d930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f977009e040 0x7f977013c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f977009fae0 0x7f9770139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:50.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f977009fae0 0x7f9770139c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43830/0 (socket says 172.21.15.119:43830) 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 -- 172.21.15.119:0/3847171672 learned_addr learned my addr 172.21.15.119:0/3847171672 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): get_auth_request con 0x7f977009fae0 auth_method 0 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient(hunting): handle_auth_done global_id 84322 payload 274 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 -- 172.21.15.119:0/3847171672 >> v2:172.21.15.17:3300/0 conn(0x7f977009e040 msgr2=0x7f977013c1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 --2- 172.21.15.119:0/3847171672 >> v2:172.21.15.17:3300/0 conn(0x7f977009e040 0x7f977013c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 -- 172.21.15.119:0/3847171672 >> v1:172.21.15.17:6789/0 conn(0x7f977009d550 legacy=0x7f977009d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:50.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:50.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 monclient: found mon.noname-c 2024-09-15T21:24:50.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:50.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.162+0000 7f976effd640 1 -- 172.21.15.119:0/3847171672 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700a1d60 con 0x7f977009fae0 2024-09-15T21:24:50.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976effd640 1 --2- 172.21.15.119:0/3847171672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009fae0 0x7f9770139c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97700a2aa0 tx=0x7f976001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ae9604a73c52b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:50.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 1 -- 172.21.15.119:0/3847171672 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600b1070 con 0x7f977009fae0 2024-09-15T21:24:50.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:50.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:50.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: dump: 2024-09-15T21:24:50.177 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:50.178 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:50.179 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:50.179 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:50.179 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:50.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:50.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 5 monclient: authenticate success, global_id 84322 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 1 -- 172.21.15.119:0/3847171672 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97600a8050 con 0x7f977009fae0 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 1 -- 172.21.15.119:0/3847171672 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600ac860 con 0x7f977009fae0 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:50.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 4 set_mon_vals no callback set 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f976e7fc640 10 monclient: dump: 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:50.181 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 10 monclient: shutdown 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 -- 172.21.15.119:0/3847171672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009fae0 msgr2=0x7f9770139c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 --2- 172.21.15.119:0/3847171672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009fae0 0x7f9770139c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97700a2aa0 tx=0x7f976001c400 comp rx=0 tx=0).stop 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 -- 172.21.15.119:0/3847171672 shutdown_connections 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 --2- 172.21.15.119:0/3847171672 >> v2:172.21.15.17:3300/0 conn(0x7f977009e040 0x7f977013c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 --2- 172.21.15.119:0/3847171672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009fae0 0x7f9770139c70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 -- 172.21.15.119:0/3847171672 >> 172.21.15.119:0/3847171672 conn(0x7f97700a66e0 msgr2=0x7f97700a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 -- 172.21.15.119:0/3847171672 shutdown_connections 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 1 -- 172.21.15.119:0/3847171672 wait complete. 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 10 monclient: build_initial_monmap 2024-09-15T21:24:50.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.166+0000 7f9775af3640 10 monclient: monmap: 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 Processor -- start 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- start start 2024-09-15T21:24:50.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: init 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f977009d550 0x7f9770134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): picked mon.noname-a con 0x7f977009d550 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 0x7f9770134790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): picked mon.noname-b con 0x7f977009e040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f977009fae0 0x7f9770139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): picked mon.noname-c con 0x7f977009fae0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:50.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f977013d090 con 0x7f977009d550 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f977013cf10 con 0x7f977009e040 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f977013d210 con 0x7f977009fae0 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient(hunting): authenticate will time out at 1408.968794s 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 0x7f9770134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976f7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f977009d550 msgr2=0x7f9770134250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 0x7f9770134790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43832/0 (socket says 172.21.15.119:43832) 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 -- 172.21.15.119:0/3209089451 learned_addr learned my addr 172.21.15.119:0/3209089451 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:50.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): get_auth_request con 0x7f977009e040 auth_method 0 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976f7fe640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f977009d550 0x7f9770134250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient(hunting): handle_auth_done global_id 84325 payload 824 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976ffff640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f977009fae0 0x7f9770139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 -- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f977009fae0 msgr2=0x7f9770139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f977009fae0 0x7f9770139b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 -- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f977009d550 msgr2=0x7f9770134250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:50.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f977009d550 0x7f9770134250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient: _finish_hunting 0 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 monclient: found mon.noname-b 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 -- 172.21.15.119:0/3209089451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977011d960 con 0x7f977009e040 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976effd640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 0x7f9770134790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f976001cca0 tx=0x7f97600191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600b1050 con 0x7f977009e040 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: dump: 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:50.187 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: _finish_auth 0 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97600bd070 con 0x7f977009e040 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 5 monclient: authenticate success, global_id 84325 2024-09-15T21:24:50.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _renew_subs 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977013ba70 con 0x7f977009e040 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600b88b0 con 0x7f977009e040 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 10 monclient: dump: 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:50.189 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _renew_subs 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976dffb640 4 set_mon_vals no callback set 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97700a5060 con 0x7f977009e040 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977009d930 con 0x7f977009e040 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f97600b8a50 con 0x7f977009e040 2024-09-15T21:24:50.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f975c03b970 0x7f975c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(214..214 src has 1..214) ==== 14132+0+0 (secure 0 0 0) 0x7f97600eca40 con 0x7f977009e040 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976f7fe640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f975c03b970 0x7f975c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976f7fe640 10 monclient: get_auth_request con 0x7f975c03b970 auth_method 0 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.170+0000 7f976f7fe640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f975c03b970 0x7f975c03de30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9764006fd0 tx=0x7f9764006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.174+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f97600eee20 con 0x7f977009e040 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.174+0000 7f9757fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:50.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.174+0000 7f9757fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:50.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.350+0000 7f9775af3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:50.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.350+0000 7f9775af3640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:50.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.350+0000 7f9775af3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:50.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.350+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f977006a830 con 0x7f977009e040 2024-09-15T21:24:50.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.926+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 added bucket h29 type host to location {rack=r1} v215) ==== 168+0+0 (secure 0 0 0) 0x7f9760016110 con 0x7f977009e040 2024-09-15T21:24:50.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.926+0000 7f9757fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:50.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:50.926+0000 7f9757fff640 10 monclient: _finish_command 2 = system:0 added bucket h29 type host to location {rack=r1} 2024-09-15T21:24:51.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9775af3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:51.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9775af3640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:51.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9775af3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f977011dba0 con 0x7f977009e040 2024-09-15T21:24:51.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9757fff640 1 -- 172.21.15.119:0/3209089451 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h29' already exists v215) ==== 147+0+0 (secure 0 0 0) 0x7f97600e8ee0 con 0x7f977009e040 2024-09-15T21:24:51.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9757fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:51.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.030+0000 7f9757fff640 10 monclient: _finish_command 3 = system:0 bucket 'h29' already exists 2024-09-15T21:24:51.035 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h29' already exists 2024-09-15T21:24:51.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f975c03b970 msgr2=0x7f975c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f975c03b970 0x7f975c03de30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9764006fd0 tx=0x7f9764006d40 comp rx=0 tx=0).stop 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 10 monclient: shutdown 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 msgr2=0x7f9770134790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 0x7f9770134790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f976001cca0 tx=0x7f97600191a0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 shutdown_connections 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f975c03b970 0x7f975c03de30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f977009fae0 0x7f9770139b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977009e040 0x7f9770134790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 --2- 172.21.15.119:0/3209089451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f977009d550 0x7f9770134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 >> 172.21.15.119:0/3209089451 conn(0x7f97700a66e0 msgr2=0x7f97700a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:51.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 shutdown_connections 2024-09-15T21:24:51.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.034+0000 7f9775af3640 1 -- 172.21.15.119:0/3209089451 wait complete. 2024-09-15T21:24:51.052 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:51.052 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h30 host rack=r1 2024-09-15T21:24:51.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient: get_monmap_and_config 2024-09-15T21:24:51.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient: build_initial_monmap 2024-09-15T21:24:51.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient: monmap: 2024-09-15T21:24:51.160 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:51.160945+0000 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:51.160945+0000 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:51.161 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:51.162 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 Processor -- start 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 -- start start 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient: init 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c073570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): picked mon.noname-b con 0x7f690c06dbb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:51.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f690c0a0750 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f690c13ceb0 0x7f690c13f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): picked mon.noname-c con 0x7f690c13ceb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690c0a0410 con 0x7f690c0a0750 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f690c134b00 con 0x7f690c13ceb0 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f690c0a0240 con 0x7f690c06dbb0 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6913123640 10 monclient(hunting): authenticate will time out at 1409.957559s 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f690bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f690c0a0750 0x7f690c073ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33872/0 (socket says 172.21.15.119:33872) 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f690bfff640 1 -- 172.21.15.119:0/596383359 learned_addr learned my addr 172.21.15.119:0/596383359 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:51.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 1 --2- 172.21.15.119:0/596383359 >> v2:172.21.15.119:3300/0 conn(0x7f690c13ceb0 0x7f690c13f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): get_auth_request con 0x7f690c13ceb0 auth_method 0 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6910ebf640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient(hunting): handle_auth_done global_id 84328 payload 274 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 1 -- 172.21.15.119:0/596383359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 msgr2=0x7f690c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:51.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 1 -- 172.21.15.119:0/596383359 >> v1:172.21.15.119:6789/0 conn(0x7f690c0a0750 legacy=0x7f690c073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient: _finish_hunting 0 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 1 monclient: found mon.noname-c 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f69116c0640 1 -- 172.21.15.119:0/596383359 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c134d30 con 0x7f690c13ceb0 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6910ebf640 10 monclient: get_auth_request con 0x7f690c06dbb0 auth_method 0 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.158+0000 7f6910ebf640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c13ceb0 0x7f690c13f2a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6904009d90 tx=0x7f690401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e175500c33f8cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:51.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 1 -- 172.21.15.119:0/596383359 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040b2070 con 0x7f690c13ceb0 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: dump: 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:51.174 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 5 monclient: authenticate success, global_id 84328 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 1 -- 172.21.15.119:0/596383359 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69040a5950 con 0x7f690c13ceb0 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 1 -- 172.21.15.119:0/596383359 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040a5d20 con 0x7f690c13ceb0 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 4 set_mon_vals no callback set 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:51.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690b7fe640 10 monclient: dump: 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: shutdown 2024-09-15T21:24:51.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/596383359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c13ceb0 msgr2=0x7f690c13f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c13ceb0 0x7f690c13f2a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6904009d90 tx=0x7f690401c890 comp rx=0 tx=0).stop 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/596383359 shutdown_connections 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c13ceb0 0x7f690c13f2a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 --2- 172.21.15.119:0/596383359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/596383359 >> 172.21.15.119:0/596383359 conn(0x7f690c06cbf0 msgr2=0x7f690c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/596383359 shutdown_connections 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/596383359 wait complete. 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: build_initial_monmap 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: monmap: 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:51.177 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:51.178 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 Processor -- start 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- start start 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: init 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c13ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): picked mon.noname-c con 0x7f690c06dbb0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f690c13ceb0 0x7f690c13bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): picked mon.noname-a con 0x7f690c13ceb0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:51.179 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 0x7f690c135b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): picked mon.noname-b con 0x7f690c135700 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f690c142210 con 0x7f690c13ceb0 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f690c142090 con 0x7f690c135700 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f690c142390 con 0x7f690c06dbb0 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient(hunting): authenticate will time out at 1409.962374s 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 0x7f690c135b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:51.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 0x7f690c135b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43844/0 (socket says 172.21.15.119:43844) 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 -- 172.21.15.119:0/801754107 learned_addr learned my addr 172.21.15.119:0/801754107 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): get_auth_request con 0x7f690c135700 auth_method 0 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690bfff640 1 -- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f690c13ceb0 msgr2=0x7f690c13bf40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690bfff640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f690c13ceb0 0x7f690c13bf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:51.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6910ebf640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c13ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient(hunting): handle_auth_done global_id 84334 payload 824 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 -- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 msgr2=0x7f690c13ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c13ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 -- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f690c13ceb0 msgr2=0x7f690c13bf40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f690c13ceb0 0x7f690c13bf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient: _finish_hunting 0 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 monclient: found mon.noname-b 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:51.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 -- 172.21.15.119:0/801754107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c074610 con 0x7f690c135700 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69116c0640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 0x7f690c135b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6904009d90 tx=0x7f6904019630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040b2070 con 0x7f690c135700 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: dump: 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:51.183 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 5 monclient: authenticate success, global_id 84334 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: _renew_subs 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69040bc070 con 0x7f690c135700 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c13c480 con 0x7f690c135700 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:51.184 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040adbf0 con 0x7f690c135700 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f69097fa640 10 monclient: dump: 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:51.185 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f690affd640 4 set_mon_vals no callback set 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: _renew_subs 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.162+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c1384b0 con 0x7f690c135700 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6913123640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6913123640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6913123640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f69040c1460 con 0x7f690c135700 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690c073570 con 0x7f690c135700 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f69097fa640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f68f803b970 0x7f68f803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 14345+0+0 (secure 0 0 0) 0x7f69040ecd60 con 0x7f690c135700 2024-09-15T21:24:51.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6910ebf640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f68f803b970 0x7f68f803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:51.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6910ebf640 10 monclient: get_auth_request con 0x7f68f803b970 auth_method 0 2024-09-15T21:24:51.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.166+0000 7f6910ebf640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f68f803b970 0x7f68f803de30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6900006fd0 tx=0x7f6900008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:51.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.170+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6904016540 con 0x7f690c135700 2024-09-15T21:24:51.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.170+0000 7f69097fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:51.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.170+0000 7f69097fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:51.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.330+0000 7f6913123640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:51.333 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.330+0000 7f6913123640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:51.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.330+0000 7f6913123640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:51.334 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.330+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f690c137ff0 con 0x7f690c135700 2024-09-15T21:24:51.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.930+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 added bucket h30 type host to location {rack=r1} v216) ==== 168+0+0 (secure 0 0 0) 0x7f69040e9210 con 0x7f690c135700 2024-09-15T21:24:51.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.930+0000 7f69097fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:51.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:51.930+0000 7f69097fa640 10 monclient: _finish_command 2 = system:0 added bucket h30 type host to location {rack=r1} 2024-09-15T21:24:52.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f690c136cb0 con 0x7f690c135700 2024-09-15T21:24:52.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f69097fa640 1 -- 172.21.15.119:0/801754107 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h30' already exists v216) ==== 147+0+0 (secure 0 0 0) 0x7f69040add90 con 0x7f690c135700 2024-09-15T21:24:52.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f69097fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f69097fa640 10 monclient: _finish_command 3 = system:0 bucket 'h30' already exists 2024-09-15T21:24:52.037 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h30' already exists 2024-09-15T21:24:52.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f68f803b970 msgr2=0x7f68f803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:52.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f68f803b970 0x7f68f803de30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6900006fd0 tx=0x7f6900008040 comp rx=0 tx=0).stop 2024-09-15T21:24:52.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 10 monclient: shutdown 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 msgr2=0x7f690c135b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 0x7f690c135b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6904009d90 tx=0x7f6904019630 comp rx=0 tx=0).stop 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 shutdown_connections 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f68f803b970 0x7f68f803de30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f690c135700 0x7f690c135b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f690c13ceb0 0x7f690c13bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 --2- 172.21.15.119:0/801754107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f690c06dbb0 0x7f690c13ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 >> 172.21.15.119:0/801754107 conn(0x7f690c06cbf0 msgr2=0x7f690c0a2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 shutdown_connections 2024-09-15T21:24:52.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.034+0000 7f6913123640 1 -- 172.21.15.119:0/801754107 wait complete. 2024-09-15T21:24:52.054 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:52.054 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h31 host rack=r1 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient: get_monmap_and_config 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient: build_initial_monmap 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient: monmap: 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:52.177463+0000 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:52.177463+0000 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:52.177 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:52.178 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:52.178 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:52.178 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:52.178 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:52.178 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:52.178 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 Processor -- start 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 -- start start 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient: init 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68809d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.180 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): picked mon.noname-b con 0x7fe68809d550 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe68809fae0 0x7fe68809de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): picked mon.noname-a con 0x7fe68809fae0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe688134820 0x7fe688134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): picked mon.noname-c con 0x7fe688134820 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe68809f1b0 con 0x7fe68809fae0 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe6880a2920 con 0x7fe688134820 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe68809e3b0 con 0x7fe68809d550 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe6901c9640 10 monclient(hunting): authenticate will time out at 1410.974100s 2024-09-15T21:24:52.181 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe688134820 0x7fe688134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68d764640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fe68809fae0 msgr2=0x7fe68809de70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68d764640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe68809fae0 0x7fe68809de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe688134820 0x7fe688134cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43850/0 (socket says 172.21.15.119:43850) 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 -- 172.21.15.119:0/3591891661 learned_addr learned my addr 172.21.15.119:0/3591891661 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): get_auth_request con 0x7fe688134820 auth_method 0 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:52.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68df65640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68809d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient(hunting): handle_auth_done global_id 84337 payload 274 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 -- 172.21.15.119:0/3591891661 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 msgr2=0x7fe68809d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68809d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 -- 172.21.15.119:0/3591891661 >> v2:172.21.15.17:3300/0 conn(0x7fe68809fae0 msgr2=0x7fe68809de70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 --2- 172.21.15.119:0/3591891661 >> v2:172.21.15.17:3300/0 conn(0x7fe68809fae0 0x7fe68809de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient: _finish_hunting 0 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 monclient: found mon.noname-c 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68e766640 1 -- 172.21.15.119:0/3591891661 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6880a3600 con 0x7fe688134820 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68df65640 10 monclient: get_auth_request con 0x7fe68809d550 auth_method 0 2024-09-15T21:24:52.183 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.174+0000 7fe68df65640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68809d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:52.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68e766640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688134820 0x7fe688134cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe6880a37d0 tx=0x7fe68001ead0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f2fd3605cffd9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:52.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 1 -- 172.21.15.119:0/3591891661 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68000b2d0 con 0x7fe688134820 2024-09-15T21:24:52.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: dump: 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:52.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: _finish_auth 0 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: _check_auth_tickets 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 1 -- 172.21.15.119:0/3591891661 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68000b470 con 0x7fe688134820 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 5 monclient: authenticate success, global_id 84337 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 4 set_mon_vals no callback set 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 1 -- 172.21.15.119:0/3591891661 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800a7990 con 0x7fe688134820 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:52.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68cf63640 10 monclient: dump: 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:52.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: shutdown 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- 172.21.15.119:0/3591891661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688134820 msgr2=0x7fe688134cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688134820 0x7fe688134cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe6880a37d0 tx=0x7fe68001ead0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- 172.21.15.119:0/3591891661 shutdown_connections 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688134820 0x7fe688134cd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- 172.21.15.119:0/3591891661 >> v2:172.21.15.17:3300/0 conn(0x7fe68809fae0 0x7fe68809de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- 172.21.15.119:0/3591891661 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68809d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- 172.21.15.119:0/3591891661 >> 172.21.15.119:0/3591891661 conn(0x7fe6880a66e0 msgr2=0x7fe6880a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- 172.21.15.119:0/3591891661 shutdown_connections 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- 172.21.15.119:0/3591891661 wait complete. 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: build_initial_monmap 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: monmap: 2024-09-15T21:24:52.193 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 Processor -- start 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- start start 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: init 2024-09-15T21:24:52.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68811c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): picked mon.noname-c con 0x7fe68809d550 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 0x7fe68811cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): picked mon.noname-b con 0x7fe68809fae0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe688134820 0x7fe68811d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): picked mon.noname-a con 0x7fe688134820 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe68809ce50 con 0x7fe688134820 2024-09-15T21:24:52.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe68809ccd0 con 0x7fe68809fae0 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe6880a1260 con 0x7fe68809d550 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe6901c9640 10 monclient(hunting): authenticate will time out at 1410.979178s 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 0x7fe68811cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 0x7fe68811cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43854/0 (socket says 172.21.15.119:43854) 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 -- 172.21.15.119:0/2487360149 learned_addr learned my addr 172.21.15.119:0/2487360149 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): get_auth_request con 0x7fe68809fae0 auth_method 0 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:52.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68e766640 1 -- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe688134820 msgr2=0x7fe68811d120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68e766640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe688134820 0x7fe68811d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68df65640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68811c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient(hunting): handle_auth_done global_id 84340 payload 824 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 -- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 msgr2=0x7fe68811c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68811c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 -- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe688134820 msgr2=0x7fe68811d120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe688134820 0x7fe68811d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient: _finish_hunting 0 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 monclient: found mon.noname-b 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 -- 172.21.15.119:0/2487360149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68811ee70 con 0x7fe68809fae0 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68df65640 10 monclient: get_auth_request con 0x7fe68809d550 auth_method 0 2024-09-15T21:24:52.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68df65640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68811c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe68d764640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 0x7fe68811cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b550 tx=0x7fe67800ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678013020 con 0x7fe68809fae0 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 10 monclient: dump: 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.199 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.178+0000 7fe67e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:52.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678003de0 con 0x7fe68809fae0 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780052b0 con 0x7fe68809fae0 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 5 monclient: authenticate success, global_id 84340 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 monclient: _renew_subs 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6880a5950 con 0x7fe68809fae0 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67ffff640 4 set_mon_vals no callback set 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 10 monclient: dump: 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.201 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 monclient: _renew_subs 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68811eb90 con 0x7fe68809fae0 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6880a1670 con 0x7fe68809fae0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe678016020 con 0x7fe68809fae0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe66c03b970 0x7fe66c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 14558+0+0 (secure 0 0 0) 0x7fe678051e30 con 0x7fe68809fae0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe68df65640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe66c03b970 0x7fe66c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe68df65640 10 monclient: get_auth_request con 0x7fe66c03b970 auth_method 0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.182+0000 7fe68df65640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe66c03b970 0x7fe66c03de30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe684008930 tx=0x7fe684008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.186+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe67801d020 con 0x7fe68809fae0 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.186+0000 7fe67e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:52.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.186+0000 7fe67e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:52.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.354+0000 7fe6901c9640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.354+0000 7fe6901c9640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.354+0000 7fe6901c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.354+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fe6880a0270 con 0x7fe68809fae0 2024-09-15T21:24:52.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.790+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 added bucket h31 type host to location {rack=r1} v217) ==== 168+0+0 (secure 0 0 0) 0x7fe67800fe20 con 0x7fe68809fae0 2024-09-15T21:24:52.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.790+0000 7fe67e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.790+0000 7fe67e7fc640 10 monclient: _finish_command 2 = system:0 added bucket h31 type host to location {rack=r1} 2024-09-15T21:24:52.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe6901c9640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe6901c9640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe6901c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:52.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fe68806a830 con 0x7fe68809fae0 2024-09-15T21:24:52.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe67e7fc640 1 -- 172.21.15.119:0/2487360149 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h31' already exists v217) ==== 147+0+0 (secure 0 0 0) 0x7fe67800d380 con 0x7fe68809fae0 2024-09-15T21:24:52.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe67e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:52.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.894+0000 7fe67e7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h31' already exists 2024-09-15T21:24:52.899 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h31' already exists 2024-09-15T21:24:52.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe66c03b970 msgr2=0x7fe66c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:52.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe66c03b970 0x7fe66c03de30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe684008930 tx=0x7fe684008040 comp rx=0 tx=0).stop 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 10 monclient: shutdown 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 msgr2=0x7fe68811cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 0x7fe68811cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b550 tx=0x7fe67800ba10 comp rx=0 tx=0).stop 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 shutdown_connections 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe66c03b970 0x7fe66c03de30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe688134820 0x7fe68811d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68809fae0 0x7fe68811cbe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 --2- 172.21.15.119:0/2487360149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe68809d550 0x7fe68811c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 >> 172.21.15.119:0/2487360149 conn(0x7fe6880a66e0 msgr2=0x7fe6880a4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 shutdown_connections 2024-09-15T21:24:52.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:52.898+0000 7fe6901c9640 1 -- 172.21.15.119:0/2487360149 wait complete. 2024-09-15T21:24:52.917 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:52.917 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush add-bucket h32 host rack=r1 2024-09-15T21:24:53.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient: get_monmap_and_config 2024-09-15T21:24:53.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient: build_initial_monmap 2024-09-15T21:24:53.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient: monmap: 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:53.029113+0000 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:53.029113+0000 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:53.029 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 1 Processor -- start 2024-09-15T21:24:53.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 1 -- start start 2024-09-15T21:24:53.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient: init 2024-09-15T21:24:53.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc4e4136b90 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4e40a10d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4e40a25b0 0x7fc4e4140380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient(hunting): picked mon.noname-a con 0x7fc4e40a25b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e40a18f0 con 0x7fc4e40a10d0 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e40a3660 con 0x7fc4e4136b90 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e409fc40 con 0x7fc4e40a25b0 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 10 monclient(hunting): authenticate will time out at 1411.825753s 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e8984640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc4e4136b90 0x7fc4e4136f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52468/0 (socket says 172.21.15.119:52468) 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e8984640 1 -- 172.21.15.119:0/3950486484 learned_addr learned my addr 172.21.15.119:0/3950486484 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:53.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 -- 172.21.15.119:0/3950486484 >> v1:172.21.15.17:6789/0 conn(0x7fc4e40a10d0 legacy=0x7fc4e413cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e9185640 1 -- 172.21.15.119:0/3950486484 >> v2:172.21.15.17:3300/0 conn(0x7fc4e40a25b0 msgr2=0x7fc4e4140380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e9185640 1 --2- 172.21.15.119:0/3950486484 >> v2:172.21.15.17:3300/0 conn(0x7fc4e40a25b0 0x7fc4e4140380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584299010 0 0) 0x7fc4e40a3660 con 0x7fc4e4136b90 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): my global_id is 84343 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4cc003530 con 0x7fc4e4136b90 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 671361976 0 0) 0x7fc4cc003530 con 0x7fc4e4136b90 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 >> v2:172.21.15.17:3300/0 conn(0x7fc4e40a25b0 msgr2=0x7fc4e4140380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 --2- 172.21.15.119:0/3950486484 >> v2:172.21.15.17:3300/0 conn(0x7fc4e40a25b0 0x7fc4e4140380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 >> v1:172.21.15.17:6789/0 conn(0x7fc4e40a10d0 legacy=0x7fc4e413cc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:53.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e4137680 con 0x7fc4e4136b90 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fc4d4001a80 con 0x7fc4e4136b90 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: dump: 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:53.041 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 >> v1:172.21.15.119:6789/0 conn(0x7fc4e4136b90 legacy=0x7fc4e4136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4cc0039c0 0x7fc4cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): picked mon.c con 0x7fc4cc0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:53.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): picked mon.b con 0x7fc4cc004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4cc0053d0 0x7fc4cc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): picked mon.a con 0x7fc4cc0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc00c2e0 con 0x7fc4cc0053d0 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc00c490 con 0x7fc4cc004810 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc00c690 con 0x7fc4cc0039c0 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e37fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:53.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): get_auth_request con 0x7fc4cc004810 auth_method 0 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e9185640 1 -- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4cc0053d0 msgr2=0x7fc4cc00ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e9185640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4cc0053d0 0x7fc4cc00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient(hunting): handle_auth_done global_id 84343 payload 293 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 -- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4cc0039c0 msgr2=0x7fc4cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4cc0039c0 0x7fc4cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 -- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4cc0053d0 msgr2=0x7fc4cc00ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:53.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4cc0053d0 0x7fc4cc00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 monclient: found mon.b 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 1 -- 172.21.15.119:0/3950486484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc003530 con 0x7fc4cc004810 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient: _finish_auth 0 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4e9185640 1 -- 172.21.15.119:0/3950486484 reap_dead start 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 5 monclient: authenticate success, global_id 84343 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.026+0000 7fc4eabe8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4cc004c90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003940 tx=0x7fc4d801c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789237422c8cc4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:53.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8009520 con 0x7fc4cc004810 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: dump: 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d8003bc0 con 0x7fc4cc004810 2024-09-15T21:24:53.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 1 -- 172.21.15.119:0/3950486484 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8005070 con 0x7fc4cc004810 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 4 set_mon_vals no callback set 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e37fe640 10 monclient: dump: 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.047 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: shutdown 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- 172.21.15.119:0/3950486484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 msgr2=0x7fc4cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4cc004c90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003940 tx=0x7fc4d801c420 comp rx=0 tx=0).stop 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- 172.21.15.119:0/3950486484 shutdown_connections 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/3950486484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4cc004c90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- 172.21.15.119:0/3950486484 >> 172.21.15.119:0/3950486484 conn(0x7fc4e40a66e0 msgr2=0x7fc4e40a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- 172.21.15.119:0/3950486484 shutdown_connections 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- 172.21.15.119:0/3950486484 wait complete. 2024-09-15T21:24:53.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: build_initial_monmap 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: monmap: 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 Processor -- start 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- start start 2024-09-15T21:24:53.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: init 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4e41d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): picked mon.noname-b con 0x7fc4cc004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41d05b0 0x7fc4e4123860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): picked mon.noname-a con 0x7fc4e41d05b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4e4123da0 0x7fc4e4124250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): picked mon.noname-c con 0x7fc4e4123da0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41382c0 con 0x7fc4e41d05b0 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4138140 con 0x7fc4cc004810 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:53.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e4138440 con 0x7fc4e4123da0 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4eabe8640 10 monclient(hunting): authenticate will time out at 1411.831488s 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4e41d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4e41d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38598/0 (socket says 172.21.15.119:38598) 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 -- 172.21.15.119:0/2702147390 learned_addr learned my addr 172.21.15.119:0/2702147390 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): get_auth_request con 0x7fc4cc004810 auth_method 0 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e3fff640 1 -- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41d05b0 msgr2=0x7fc4e4123860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:53.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41d05b0 0x7fc4e4123860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient(hunting): handle_auth_done global_id 84346 payload 824 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 -- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4e4123da0 msgr2=0x7fc4e4124250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4e4123da0 0x7fc4e4124250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 -- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41d05b0 msgr2=0x7fc4e4123860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41d05b0 0x7fc4e4123860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient: _finish_hunting 0 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 monclient: found mon.noname-b 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 -- 172.21.15.119:0/2702147390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e409ec60 con 0x7fc4cc004810 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.030+0000 7fc4e8984640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4e41d0070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400ca70 tx=0x7fc4d400cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400a700 con 0x7fc4cc004810 2024-09-15T21:24:53.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: dump: 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:53.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 5 monclient: authenticate success, global_id 84346 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d400a8a0 con 0x7fc4cc004810 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 monclient: _renew_subs 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e40a03b0 con 0x7fc4cc004810 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400ac70 con 0x7fc4cc004810 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 10 monclient: dump: 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:53.054 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e2ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 monclient: _renew_subs 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41273a0 con 0x7fc4cc004810 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:53.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e40a5c50 con 0x7fc4cc004810 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc4d4011070 con 0x7fc4cc004810 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc4d003b970 0x7fc4d003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(217..217 src has 1..217) ==== 14771+0+0 (secure 0 0 0) 0x7fc4d4052190 con 0x7fc4cc004810 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc4d003b970 0x7fc4d003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e3fff640 10 monclient: get_auth_request con 0x7fc4d003b970 auth_method 0 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.034+0000 7fc4e3fff640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc4d003b970 0x7fc4d003de30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80aa000 tx=0x7fc4d8005a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:53.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.038+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc4e40a03b0 con 0x7fc4cc004810 2024-09-15T21:24:53.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.038+0000 7fc4e17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:53.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.038+0000 7fc4e17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:53.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.198+0000 7fc4eabe8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:53.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.198+0000 7fc4eabe8640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:53.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.198+0000 7fc4eabe8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:53.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.198+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc4e409d240 con 0x7fc4cc004810 2024-09-15T21:24:53.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.946+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 added bucket h32 type host to location {rack=r1} v218) ==== 168+0+0 (secure 0 0 0) 0x7fc4e409ec60 con 0x7fc4cc004810 2024-09-15T21:24:53.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.946+0000 7fc4e17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:53.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:53.946+0000 7fc4e17fa640 10 monclient: _finish_command 2 = system:0 added bucket h32 type host to location {rack=r1} 2024-09-15T21:24:54.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:54.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:54.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc4e411fdc0 con 0x7fc4cc004810 2024-09-15T21:24:54.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4e17fa640 1 -- 172.21.15.119:0/2702147390 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h32' already exists v218) ==== 147+0+0 (secure 0 0 0) 0x7fc4d400ae10 con 0x7fc4cc004810 2024-09-15T21:24:54.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4e17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T21:24:54.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4e17fa640 10 monclient: _finish_command 3 = system:0 bucket 'h32' already exists 2024-09-15T21:24:54.054 INFO:tasks.workunit.client.0.smithi119.stderr:bucket 'h32' already exists 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc4d003b970 msgr2=0x7fc4d003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc4d003b970 0x7fc4d003de30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80aa000 tx=0x7fc4d8005a90 comp rx=0 tx=0).stop 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 10 monclient: shutdown 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 msgr2=0x7fc4e41d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4e41d0070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400ca70 tx=0x7fc4d400cf30 comp rx=0 tx=0).stop 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 shutdown_connections 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc4d003b970 0x7fc4d003de30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc4e4123da0 0x7fc4e4124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41d05b0 0x7fc4e4123860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 --2- 172.21.15.119:0/2702147390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc004810 0x7fc4e41d0070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.050+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 >> 172.21.15.119:0/2702147390 conn(0x7fc4e40a66e0 msgr2=0x7fc4e409f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:54.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.054+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 shutdown_connections 2024-09-15T21:24:54.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.054+0000 7fc4eabe8640 1 -- 172.21.15.119:0/2702147390 wait complete. 2024-09-15T21:24:54.071 INFO:tasks.workunit.client.0.smithi119.stderr:++ seq 1 32 2024-09-15T21:24:54.072 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:54.072 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h1 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient: get_monmap_and_config 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient: build_initial_monmap 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient: monmap: 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:54.190612+0000 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:54.190612+0000 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:54.190 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:54.191 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:54.191 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:54.191 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:54.191 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 1 Processor -- start 2024-09-15T21:24:54.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 1 -- start start 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient: init 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4e540a4ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e540a5790 0x7f4e5413ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient(hunting): picked mon.noname-a con 0x7f4e540a5790 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e5413d3f0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e540a23d0 con 0x7f4e5413d3f0 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e540a20f0 con 0x7f4e540a4ca0 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4e540740f0 con 0x7f4e540a5790 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e596ee640 10 monclient(hunting): authenticate will time out at 1412.987223s 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e52ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4e540a4ca0 0x7f4e540a5080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52476/0 (socket says 172.21.15.119:52476) 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e537fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f4e5413d3f0 legacy=0x7f4e5413f7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e52ffd640 1 -- 172.21.15.119:0/1697470514 learned_addr learned my addr 172.21.15.119:0/1697470514 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e527fc640 1 -- 172.21.15.119:0/1697470514 >> v2:172.21.15.17:3300/0 conn(0x7f4e540a5790 msgr2=0x7f4e5413ceb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e527fc640 1 --2- 172.21.15.119:0/1697470514 >> v2:172.21.15.17:3300/0 conn(0x7f4e540a5790 0x7f4e5413ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168572789 0 0) 0x7f4e540a20f0 con 0x7f4e540a4ca0 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e51ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e51ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e51ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e51ffb640 10 monclient(hunting): my global_id is 84349 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.186+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e3c003530 con 0x7f4e540a4ca0 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3858586241 0 0) 0x7f4e3c003530 con 0x7f4e540a4ca0 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 >> v2:172.21.15.17:3300/0 conn(0x7f4e540a5790 msgr2=0x7f4e5413ceb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:54.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 --2- 172.21.15.119:0/1697470514 >> v2:172.21.15.17:3300/0 conn(0x7f4e540a5790 0x7f4e5413ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 >> v1:172.21.15.17:6789/0 conn(0x7f4e5413d3f0 legacy=0x7f4e5413f7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540a5d20 con 0x7f4e540a4ca0 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f4e44001700 con 0x7f4e540a4ca0 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: dump: 2024-09-15T21:24:54.199 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 >> v1:172.21.15.119:6789/0 conn(0x7f4e540a4ca0 legacy=0x7f4e540a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e3c0039c0 0x7f4e3c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): picked mon.c con 0x7f4e3c0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e3c004810 0x7f4e3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): picked mon.b con 0x7f4e3c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0053d0 0x7f4e3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): picked mon.a con 0x7f4e3c0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c00c240 con 0x7f4e3c0053d0 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c00c3f0 con 0x7f4e3c004810 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c00c5f0 con 0x7f4e3c0039c0 2024-09-15T21:24:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e3c004810 0x7f4e3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): get_auth_request con 0x7f4e3c004810 auth_method 0 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e537fe640 1 -- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0053d0 msgr2=0x7f4e3c00b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e537fe640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0053d0 0x7f4e3c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e52ffd640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e3c0039c0 0x7f4e3c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient(hunting): handle_auth_done global_id 84349 payload 293 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 -- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e3c0039c0 msgr2=0x7f4e3c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e3c0039c0 0x7f4e3c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 -- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0053d0 msgr2=0x7f4e3c00b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0053d0 0x7f4e3c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient: _finish_hunting 0 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 monclient: found mon.b 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e537fe640 1 -- 172.21.15.119:0/1697470514 reap_dead start 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 -- 172.21.15.119:0/1697470514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c003530 con 0x7f4e3c004810 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:54.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 5 monclient: authenticate success, global_id 84349 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e52ffd640 10 monclient: get_auth_request con 0x7f4e3c0039c0 auth_method 0 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e52ffd640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e3c0039c0 0x7f4e3c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e527fc640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e3c004810 0x7f4e3c004c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e480038c0 tx=0x7f4e4801c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddc0f6ebd84b08e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48009470 con 0x7f4e3c004810 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: dump: 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.204 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e48003af0 con 0x7f4e3c004810 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 1 -- 172.21.15.119:0/1697470514 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e480050d0 con 0x7f4e3c004810 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 4 set_mon_vals no callback set 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:54.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e51ffb640 10 monclient: dump: 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 10 monclient: shutdown 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 -- 172.21.15.119:0/1697470514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e3c004810 msgr2=0x7f4e3c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e3c004810 0x7f4e3c004c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e480038c0 tx=0x7f4e4801c340 comp rx=0 tx=0).stop 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 -- 172.21.15.119:0/1697470514 shutdown_connections 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 --2- 172.21.15.119:0/1697470514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e3c004810 0x7f4e3c004c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 -- 172.21.15.119:0/1697470514 >> 172.21.15.119:0/1697470514 conn(0x7f4e540a7400 msgr2=0x7f4e5406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 -- 172.21.15.119:0/1697470514 shutdown_connections 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.190+0000 7f4e596ee640 1 -- 172.21.15.119:0/1697470514 wait complete. 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: build_initial_monmap 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: monmap: 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.207 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:54.208 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:54.208 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:54.208 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 Processor -- start 2024-09-15T21:24:54.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- start start 2024-09-15T21:24:54.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: init 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c004810 0x7f4e54136210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): picked mon.noname-a con 0x7f4e3c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e54136750 0x7f4e54136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): picked mon.noname-c con 0x7f4e54136750 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 0x7f4e54120d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): picked mon.noname-b con 0x7f4e541208f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5406dd20 con 0x7f4e3c004810 2024-09-15T21:24:54.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540a5fa0 con 0x7f4e541208f0 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4e540a45e0 con 0x7f4e54136750 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient(hunting): authenticate will time out at 1412.993432s 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 0x7f4e54120d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e52ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c004810 msgr2=0x7f4e54136210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 0x7f4e54120d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38616/0 (socket says 172.21.15.119:38616) 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 -- 172.21.15.119:0/292762200 learned_addr learned my addr 172.21.15.119:0/292762200 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): get_auth_request con 0x7f4e541208f0 auth_method 0 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:54.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e52ffd640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c004810 0x7f4e54136210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e527fc640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e54136750 0x7f4e54136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient(hunting): handle_auth_done global_id 84352 payload 824 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 -- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e54136750 msgr2=0x7f4e54136bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e54136750 0x7f4e54136bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 -- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c004810 msgr2=0x7f4e54136210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c004810 0x7f4e54136210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 monclient: found mon.noname-b 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 -- 172.21.15.119:0/292762200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e541232c0 con 0x7f4e541208f0 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e527fc640 10 monclient: get_auth_request con 0x7f4e54136750 auth_method 0 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e527fc640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e54136750 0x7f4e54136bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e537fe640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 0x7f4e54120d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e540a8130 tx=0x7f4e4c009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c016630 con 0x7f4e541208f0 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: dump: 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.215 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 5 monclient: authenticate success, global_id 84352 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _renew_subs 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e540a9280 con 0x7f4e541208f0 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c010030 con 0x7f4e541208f0 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c016a10 con 0x7f4e541208f0 2024-09-15T21:24:54.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 10 monclient: dump: 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e517fa640 4 set_mon_vals no callback set 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _renew_subs 2024-09-15T21:24:54.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e54122fe0 con 0x7f4e541208f0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5406d560 con 0x7f4e541208f0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4e4c0153e0 con 0x7f4e541208f0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e4003b9a0 0x7f4e4003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(218..218 src has 1..218) ==== 14984+0+0 (secure 0 0 0) 0x7f4e4c052a80 con 0x7f4e541208f0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e52ffd640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e4003b9a0 0x7f4e4003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.194+0000 7f4e52ffd640 10 monclient: get_auth_request con 0x7f4e4003b9a0 auth_method 0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.198+0000 7f4e52ffd640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e4003b9a0 0x7f4e4003de60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4e54052890 tx=0x7f4e44001750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.198+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4e4c04ecb0 con 0x7f4e541208f0 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.198+0000 7f4e3b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:54.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.198+0000 7f4e3b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:54.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.378+0000 7f4e596ee640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T21:24:54.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.378+0000 7f4e596ee640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T21:24:54.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.378+0000 7f4e596ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:54.380 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.378+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) -- 0x7f4e5406c440 con 0x7f4e541208f0 2024-09-15T21:24:54.742 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:24:54.742 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:24:54.742 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:24:54.745 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:24:54.745 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:24:54.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient: get_monmap_and_config 2024-09-15T21:24:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient: build_initial_monmap 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient: monmap: 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:54.871048+0000 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:54.871048+0000 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:54.872 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 Processor -- start 2024-09-15T21:24:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 -- start start 2024-09-15T21:24:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient: init 2024-09-15T21:24:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f30c81378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f30c81383b0 0x7f30c813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): picked mon.noname-c con 0x7f30c81383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c813d2c0 0x7f30c813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): picked mon.noname-b con 0x7f30c813d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c80a1a20 con 0x7f30c81378c0 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f30c80a1d00 con 0x7f30c81383b0 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f30c80a1fe0 con 0x7f30c813d2c0 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30cdb09640 10 monclient(hunting): authenticate will time out at 1418.266028s 2024-09-15T21:24:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c813d2c0 0x7f30c813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f30c81378c0 0x7f30c8137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40236/0 (socket says 172.21.15.17:40236) 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c77fe640 1 -- 172.21.15.17:0/138051465 learned_addr learned my addr 172.21.15.17:0/138051465 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): get_auth_request con 0x7f30c813d2c0 auth_method 0 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c6ffd640 1 --2- 172.21.15.17:0/138051465 >> v2:172.21.15.119:3300/0 conn(0x7f30c81383b0 0x7f30c813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c6ffd640 10 monclient(hunting): get_auth_request con 0x7f30c81383b0 auth_method 0 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient(hunting): handle_auth_done global_id 45044 payload 274 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 1 -- 172.21.15.17:0/138051465 >> v2:172.21.15.119:3300/0 conn(0x7f30c81383b0 msgr2=0x7f30c813cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 1 --2- 172.21.15.17:0/138051465 >> v2:172.21.15.119:3300/0 conn(0x7f30c81383b0 0x7f30c813cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 1 -- 172.21.15.17:0/138051465 >> v1:172.21.15.17:6789/0 conn(0x7f30c81378c0 legacy=0x7f30c8137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 1 monclient: found mon.noname-b 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.867+0000 7f30c7fff640 1 -- 172.21.15.17:0/138051465 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c80a22c0 con 0x7f30c813d2c0 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c7fff640 1 --2- 172.21.15.17:0/138051465 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c813d2c0 0x7f30c813f6b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009dc0 tx=0x7f30c001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8295c13063caee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 1 -- 172.21.15.17:0/138051465 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c00b1070 con 0x7f30c813d2c0 2024-09-15T21:24:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: dump: 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 5 monclient: authenticate success, global_id 45044 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 1 -- 172.21.15.17:0/138051465 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c00a4db0 con 0x7f30c813d2c0 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 4 set_mon_vals no callback set 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 1 -- 172.21.15.17:0/138051465 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c00ac660 con 0x7f30c813d2c0 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c67fc640 10 monclient: dump: 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.881 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: shutdown 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/138051465 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c813d2c0 msgr2=0x7f30c813f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 --2- 172.21.15.17:0/138051465 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c813d2c0 0x7f30c813f6b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009dc0 tx=0x7f30c001c4f0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/138051465 shutdown_connections 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 --2- 172.21.15.17:0/138051465 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c813d2c0 0x7f30c813f6b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 --2- 172.21.15.17:0/138051465 >> v2:172.21.15.119:3300/0 conn(0x7f30c81383b0 0x7f30c813cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/138051465 >> 172.21.15.17:0/138051465 conn(0x7f30c80aac40 msgr2=0x7f30c80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:54.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/138051465 shutdown_connections 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/138051465 wait complete. 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: build_initial_monmap 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: monmap: 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:54.887 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 Processor -- start 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- start start 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: init 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 0x7f30c81d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): picked mon.noname-c con 0x7f30c81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 0x7f30c81d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): picked mon.noname-a con 0x7f30c81383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30c813d2c0 0x7f30c809db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): picked mon.noname-b con 0x7f30c813d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8143130 con 0x7f30c81383b0 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8142fb0 con 0x7f30c813d2c0 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f30c81432b0 con 0x7f30c81378c0 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient(hunting): authenticate will time out at 1418.270953s 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 0x7f30c81d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 0x7f30c81d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33934/0 (socket says 172.21.15.17:33934) 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 0x7f30c81d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 -- 172.21.15.17:0/2008361147 learned_addr learned my addr 172.21.15.17:0/2008361147 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): get_auth_request con 0x7f30c81378c0 auth_method 0 2024-09-15T21:24:54.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 10 monclient(hunting): get_auth_request con 0x7f30c81383b0 auth_method 0 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 1 -- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 msgr2=0x7f30c81d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 1 -- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 msgr2=0x7f30c81d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 0x7f30c81d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c6ffd640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 0x7f30c81d0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient(hunting): handle_auth_done global_id 45047 payload 824 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 -- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30c813d2c0 msgr2=0x7f30c809db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c7fff640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30c813d2c0 0x7f30c809db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30c813d2c0 0x7f30c809db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 -- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 msgr2=0x7f30c81d0f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 0x7f30c81d0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 monclient: found mon.noname-c 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 -- 172.21.15.17:0/2008361147 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c809fab0 con 0x7f30c81378c0 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c77fe640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 0x7f30c81d0760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c480 tx=0x7f30b800c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8017070 con 0x7f30c81378c0 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: dump: 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:54.892 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: _finish_auth 0 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: _check_auth_tickets 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8005d10 con 0x7f30c81378c0 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 5 monclient: authenticate success, global_id 45047 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: _renew_subs 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c8120fe0 con 0x7f30c81378c0 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:54.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80072b0 con 0x7f30c81378c0 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30affff640 10 monclient: dump: 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:54.894 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: _renew_subs 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c813f6c0 con 0x7f30c81378c0 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.871+0000 7f30c5ffb640 4 set_mon_vals no callback set 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30cdb09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30cdb09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30cdb09640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c8137ca0 con 0x7f30c81378c0 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f30b801a070 con 0x7f30c81378c0 2024-09-15T21:24:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30affff640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30b403ba20 0x7f30b403dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(218..218 src has 1..218) ==== 14984+0+0 (secure 0 0 0) 0x7f30b8051630 con 0x7f30c81378c0 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30c6ffd640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30b403ba20 0x7f30b403dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30c6ffd640 10 monclient: get_auth_request con 0x7f30b403ba20 auth_method 0 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.875+0000 7f30c6ffd640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30b403ba20 0x7f30b403dee0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b990 tx=0x7f30bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.879+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f30c80a1a20 con 0x7f30c81378c0 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.879+0000 7f30affff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:54.879+0000 7f30affff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:54.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.958+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 removed item id -19 name 'h1' from crush map v219) ==== 118+0+0 (secure 0 0 0) 0x7f4e4c052e20 con 0x7f4e541208f0 2024-09-15T21:24:54.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.958+0000 7f4e3b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T21:24:54.963 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:54.958+0000 7f4e3b7fe640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'h1' from crush map 2024-09-15T21:24:55.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e596ee640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T21:24:55.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e596ee640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T21:24:55.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e596ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) -- 0x7f4e54072d50 con 0x7f4e541208f0 2024-09-15T21:24:55.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e3b7fe640 1 -- 172.21.15.119:0/292762200 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 device 'h1' does not appear in the crush map v219) ==== 118+0+0 (secure 0 0 0) 0x7f4e4c015bf0 con 0x7f4e541208f0 2024-09-15T21:24:55.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e3b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T21:24:55.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.066+0000 7f4e3b7fe640 10 monclient: _finish_command 3 = system:0 device 'h1' does not appear in the crush map 2024-09-15T21:24:55.071 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h1' does not appear in the crush map 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e4003b9a0 msgr2=0x7f4e4003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e4003b9a0 0x7f4e4003de60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4e54052890 tx=0x7f4e44001750 comp rx=0 tx=0).stop 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 10 monclient: shutdown 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 msgr2=0x7f4e54120d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 0x7f4e54120d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e540a8130 tx=0x7f4e4c009e50 comp rx=0 tx=0).stop 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 shutdown_connections 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e4003b9a0 0x7f4e4003de60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e541208f0 0x7f4e54120d60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e54136750 0x7f4e54136bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 --2- 172.21.15.119:0/292762200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c004810 0x7f4e54136210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 >> 172.21.15.119:0/292762200 conn(0x7f4e540a7400 msgr2=0x7f4e540a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:55.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 shutdown_connections 2024-09-15T21:24:55.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.070+0000 7f4e596ee640 1 -- 172.21.15.119:0/292762200 wait complete. 2024-09-15T21:24:55.088 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:55.089 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h2 2024-09-15T21:24:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.115+0000 7f30cdb09640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.115+0000 7f30cdb09640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.115+0000 7f30cdb09640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.115+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f30c80a1270 con 0x7f30c81378c0 2024-09-15T21:24:55.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30affff640 1 -- 172.21.15.17:0/2008361147 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f30b8010300 con 0x7f30c81378c0 2024-09-15T21:24:55.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30affff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:55.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30affff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:55.123 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":68,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30b403ba20 msgr2=0x7f30b403dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30b403ba20 0x7f30b403dee0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b990 tx=0x7f30bc008040 comp rx=0 tx=0).stop 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 10 monclient: shutdown 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 msgr2=0x7f30c81d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 0x7f30c81d0760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c480 tx=0x7f30b800c940 comp rx=0 tx=0).stop 2024-09-15T21:24:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 shutdown_connections 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30b403ba20 0x7f30b403dee0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30c813d2c0 0x7f30c809db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30c81383b0 0x7f30c81d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 --2- 172.21.15.17:0/2008361147 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30c81378c0 0x7f30c81d0760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 >> 172.21.15.17:0/2008361147 conn(0x7f30c80aac40 msgr2=0x7f30c80a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 shutdown_connections 2024-09-15T21:24:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:55.119+0000 7f30cdb09640 1 -- 172.21.15.17:0/2008361147 wait complete. 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient: get_monmap_and_config 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient: build_initial_monmap 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient: monmap: 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:55.202466+0000 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:55.202466+0000 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:55.202 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:55.203 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:55.203 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:55.203 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:55.203 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:55.203 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 Processor -- start 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 -- start start 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient: init 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff51809f500 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:55.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff51809eb70 0x7ff51809f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): picked mon.noname-a con 0x7ff51809eb70 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5180a10f0 0x7ff518144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): picked mon.noname-c con 0x7ff5180a10f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5180a02f0 con 0x7ff51809f500 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff51809efa0 con 0x7ff51809eb70 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff51809fe20 con 0x7ff5180a10f0 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff520158640 10 monclient(hunting): authenticate will time out at 1413.999222s 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5180a10f0 0x7ff518144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:55.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5180a10f0 0x7ff518144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38626/0 (socket says 172.21.15.119:38626) 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 -- 172.21.15.119:0/4050155562 learned_addr learned my addr 172.21.15.119:0/4050155562 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): get_auth_request con 0x7ff5180a10f0 auth_method 0 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51d6f3640 1 --2- 172.21.15.119:0/4050155562 >> v2:172.21.15.17:3300/0 conn(0x7ff51809eb70 0x7ff51809f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_done global_id 84358 payload 274 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 -- 172.21.15.119:0/4050155562 >> v2:172.21.15.17:3300/0 conn(0x7ff51809eb70 msgr2=0x7ff51809f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:55.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 --2- 172.21.15.119:0/4050155562 >> v2:172.21.15.17:3300/0 conn(0x7ff51809eb70 0x7ff51809f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 -- 172.21.15.119:0/4050155562 >> v1:172.21.15.17:6789/0 conn(0x7ff51809f500 legacy=0x7ff5180a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient: _finish_hunting 0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 monclient: found mon.noname-c 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51e6f5640 1 -- 172.21.15.119:0/4050155562 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff518074260 con 0x7ff5180a10f0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51d6f3640 10 monclient: get_auth_request con 0x7ff51809eb70 auth_method 0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.198+0000 7ff51d6f3640 1 --2- 172.21.15.119:0/4050155562 >> v2:172.21.15.17:3300/0 conn(0x7ff51809eb70 0x7ff51809f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51e6f5640 1 --2- 172.21.15.119:0/4050155562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff518144000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff510009870 tx=0x7ff51001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a45cc794fd2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 1 -- 172.21.15.119:0/4050155562 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b0070 con 0x7ff5180a10f0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: dump: 2024-09-15T21:24:55.209 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:55.210 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: _finish_auth 0 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: _check_auth_tickets 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 5 monclient: authenticate success, global_id 84358 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 1 -- 172.21.15.119:0/4050155562 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100ab600 con 0x7ff5180a10f0 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 1 -- 172.21.15.119:0/4050155562 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100ab9d0 con 0x7ff5180a10f0 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff51cef2640 10 monclient: dump: 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:55.211 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 4 set_mon_vals no callback set 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 10 monclient: shutdown 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 -- 172.21.15.119:0/4050155562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 msgr2=0x7ff518144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 --2- 172.21.15.119:0/4050155562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff518144000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff510009870 tx=0x7ff51001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 -- 172.21.15.119:0/4050155562 shutdown_connections 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 --2- 172.21.15.119:0/4050155562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff518144000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 --2- 172.21.15.119:0/4050155562 >> v2:172.21.15.17:3300/0 conn(0x7ff51809eb70 0x7ff51809f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 -- 172.21.15.119:0/4050155562 >> 172.21.15.119:0/4050155562 conn(0x7ff5180a2ae0 msgr2=0x7ff5180a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 -- 172.21.15.119:0/4050155562 shutdown_connections 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 1 -- 172.21.15.119:0/4050155562 wait complete. 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 10 monclient: build_initial_monmap 2024-09-15T21:24:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.202+0000 7ff520158640 10 monclient: monmap: 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 Processor -- start 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- start start 2024-09-15T21:24:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: init 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51809eb70 0x7ff518128290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): picked mon.noname-c con 0x7ff51809eb70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff51809f500 0x7ff518126170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): picked mon.noname-a con 0x7ff51809f500 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff51813f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): picked mon.noname-b con 0x7ff5180a10f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5180a39e0 con 0x7ff51809f500 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5180a3860 con 0x7ff5180a10f0 2024-09-15T21:24:55.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff518146d60 con 0x7ff51809eb70 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient(hunting): authenticate will time out at 1414.004870s 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff51813f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff51813f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38642/0 (socket says 172.21.15.119:38642) 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 -- 172.21.15.119:0/591268222 learned_addr learned my addr 172.21.15.119:0/591268222 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): get_auth_request con 0x7ff5180a10f0 auth_method 0 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:55.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51d6f3640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff51809f500 0x7ff518126170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51d6f3640 10 monclient(hunting): get_auth_request con 0x7ff51809f500 auth_method 0 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51d6f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51d6f3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51d6f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient(hunting): handle_auth_done global_id 84361 payload 824 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 -- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51809eb70 msgr2=0x7ff518128290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51809eb70 0x7ff518128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 -- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff51809f500 msgr2=0x7ff518126170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff51809f500 0x7ff518126170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient: _finish_hunting 0 2024-09-15T21:24:55.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 monclient: found mon.noname-b 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 -- 172.21.15.119:0/591268222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51813fda0 con 0x7ff5180a10f0 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51e6f5640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff51813f100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff51001aa90 tx=0x7ff510019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b0040 con 0x7ff5180a10f0 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: dump: 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.222 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100b9070 con 0x7ff5180a10f0 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 5 monclient: authenticate success, global_id 84361 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _renew_subs 2024-09-15T21:24:55.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff518073840 con 0x7ff5180a10f0 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b4650 con 0x7ff5180a10f0 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 10 monclient: dump: 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:55.224 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50bfff640 4 set_mon_vals no callback set 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _renew_subs 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff518127730 con 0x7ff5180a10f0 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:55.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51809f8e0 con 0x7ff5180a10f0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff5100b47f0 con 0x7ff5180a10f0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4fc03b9c0 0x7ff4fc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 14774+0+0 (secure 0 0 0) 0x7ff5100ec0f0 con 0x7ff5180a10f0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51def4640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4fc03b9c0 0x7ff4fc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51def4640 10 monclient: get_auth_request con 0x7ff4fc03b9c0 auth_method 0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.206+0000 7ff51def4640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4fc03b9c0 0x7ff4fc03de80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff514003cf0 tx=0x7ff5140079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.210+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff5100160f0 con 0x7ff5180a10f0 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.210+0000 7ff50a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:55.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.210+0000 7ff50a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:55.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.386+0000 7ff520158640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T21:24:55.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.386+0000 7ff520158640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T21:24:55.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.386+0000 7ff520158640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:55.391 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.386+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) -- 0x7ff5180a1bf0 con 0x7ff5180a10f0 2024-09-15T21:24:55.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.966+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 removed item id -22 name 'h2' from crush map v220) ==== 118+0+0 (secure 0 0 0) 0x7ff5100be460 con 0x7ff5180a10f0 2024-09-15T21:24:55.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.966+0000 7ff50a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T21:24:55.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:55.966+0000 7ff50a7fc640 10 monclient: _finish_command 2 = system:0 removed item id -22 name 'h2' from crush map 2024-09-15T21:24:56.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.074+0000 7ff520158640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T21:24:56.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.074+0000 7ff520158640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T21:24:56.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.074+0000 7ff520158640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.074+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) -- 0x7ff518129710 con 0x7ff5180a10f0 2024-09-15T21:24:56.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff50a7fc640 1 -- 172.21.15.119:0/591268222 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 device 'h2' does not appear in the crush map v220) ==== 118+0+0 (secure 0 0 0) 0x7ff5100e83f0 con 0x7ff5180a10f0 2024-09-15T21:24:56.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff50a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T21:24:56.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff50a7fc640 10 monclient: _finish_command 3 = system:0 device 'h2' does not appear in the crush map 2024-09-15T21:24:56.081 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h2' does not appear in the crush map 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4fc03b9c0 msgr2=0x7ff4fc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4fc03b9c0 0x7ff4fc03de80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff514003cf0 tx=0x7ff5140079c0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 10 monclient: shutdown 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 msgr2=0x7ff51813f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff51813f100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff51001aa90 tx=0x7ff510019180 comp rx=0 tx=0).stop 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 shutdown_connections 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff4fc03b9c0 0x7ff4fc03de80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5180a10f0 0x7ff51813f100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff51809f500 0x7ff518126170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 --2- 172.21.15.119:0/591268222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff51809eb70 0x7ff518128290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 >> 172.21.15.119:0/591268222 conn(0x7ff5180a2ae0 msgr2=0x7ff518071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:56.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 shutdown_connections 2024-09-15T21:24:56.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.078+0000 7ff520158640 1 -- 172.21.15.119:0/591268222 wait complete. 2024-09-15T21:24:56.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:56.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h3 2024-09-15T21:24:56.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient: get_monmap_and_config 2024-09-15T21:24:56.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient: build_initial_monmap 2024-09-15T21:24:56.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient: monmap: 2024-09-15T21:24:56.214 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:56.214 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:56.214367+0000 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:56.214367+0000 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:56.216 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 Processor -- start 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 -- start start 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient: init 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7ef8135ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 0x7f7ef813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): picked mon.noname-a con 0x7f7ef80a0160 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 0x7f7ef813cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): picked mon.noname-b con 0x7f7ef806ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef809cf30 con 0x7f7ef8135ca0 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef80a1ca0 con 0x7f7ef80a0160 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef80a1f80 con 0x7f7ef806ec30 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 10 monclient(hunting): authenticate will time out at 1415.011032s 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ef8135ca0 0x7f7ef8136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52478/0 (socket says 172.21.15.119:52478) 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef659c640 1 -- 172.21.15.119:0/1035381256 learned_addr learned my addr 172.21.15.119:0/1035381256 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 0x7f7ef813cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 1 --2- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 0x7f7ef813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 10 monclient(hunting): get_auth_request con 0x7f7ef80a0160 auth_method 0 2024-09-15T21:24:56.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): get_auth_request con 0x7f7ef806ec30 auth_method 0 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2561842504 0 0) 0x7f7ef809cf30 con 0x7f7ef8135ca0 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient(hunting): _negotiate 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient(hunting): my global_id is 84364 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ee4003530 con 0x7f7ef8135ca0 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:56.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 1 -- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 msgr2=0x7f7ef813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 1 -- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 msgr2=0x7f7ef813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 1 --2- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 0x7f7ef813aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef5d9b640 1 --2- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 0x7f7ef813aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1836234139 0 0) 0x7f7ee4003530 con 0x7f7ef8135ca0 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 msgr2=0x7f7ef813cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:56.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 0x7f7ef813cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 msgr2=0x7f7ef813aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 --2- 172.21.15.119:0/1035381256 >> v2:172.21.15.17:3300/0 conn(0x7f7ef80a0160 0x7f7ef813aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: _finish_hunting 0 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef80a3860 con 0x7f7ef8135ca0 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: _finish_auth 0 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: _check_auth_tickets 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7efcfb0640 5 monclient: authenticate success, global_id 84364 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7ee0001700 con 0x7f7ef8135ca0 2024-09-15T21:24:56.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: dump: 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:24:56.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 >> v1:172.21.15.119:6789/0 conn(0x7f7ef8135ca0 legacy=0x7f7ef8136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee4003a10 0x7f7ee4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 10 monclient(hunting): picked mon.b con 0x7f7ee4003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.210+0000 7f7ef559a640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): picked mon.a con 0x7f7ee4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 0x7f7ee4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): picked mon.c con 0x7f7ef806ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef80a1ca0 con 0x7f7ee4004810 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef80a1f80 con 0x7f7ee4003a10 2024-09-15T21:24:56.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee400bbb0 con 0x7f7ef806ec30 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee4003a10 0x7f7ee4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): get_auth_request con 0x7f7ee4003a10 auth_method 0 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:24:56.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef5d9b640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef5d9b640 10 monclient(hunting): get_auth_request con 0x7f7ee4004810 auth_method 0 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient(hunting): handle_auth_done global_id 84364 payload 293 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 -- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 msgr2=0x7f7ee4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ef806ec30 0x7f7ee4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 -- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee4004810 msgr2=0x7f7ee4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 monclient: found mon.b 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 -- 172.21.15.119:0/1035381256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee4003530 con 0x7f7ee4003a10 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient: _finish_auth 0 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 10 monclient: _check_auth_tickets 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef659c640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee4003a10 0x7f7ee4003e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c240 tx=0x7f7ee001cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f1c4515959d3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0007d60 con 0x7f7ee4003a10 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: dump: 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.238 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee00a9070 con 0x7f7ee4003a10 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 1 -- 172.21.15.119:0/1035381256 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0005d40 con 0x7f7ee4003a10 2024-09-15T21:24:56.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 4 set_mon_vals no callback set 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef559a640 10 monclient: dump: 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:56.240 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 10 monclient: shutdown 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 1 -- 172.21.15.119:0/1035381256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee4003a10 msgr2=0x7f7ee4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/1035381256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee4003a10 0x7f7ee4003e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c240 tx=0x7f7ee001cc00 comp rx=0 tx=0).stop 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7ef6d9d640 1 -- 172.21.15.119:0/1035381256 reap_dead start 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 1 -- 172.21.15.119:0/1035381256 shutdown_connections 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 1 -- 172.21.15.119:0/1035381256 >> 172.21.15.119:0/1035381256 conn(0x7f7ef80a2ae0 msgr2=0x7f7ef80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 1 -- 172.21.15.119:0/1035381256 shutdown_connections 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 1 -- 172.21.15.119:0/1035381256 wait complete. 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 10 monclient: build_initial_monmap 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.214+0000 7f7efcfb0640 10 monclient: monmap: 2024-09-15T21:24:56.241 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 Processor -- start 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- start start 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: init 2024-09-15T21:24:56.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ee4003a10 0x7f7ef81cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): picked mon.noname-c con 0x7f7ee4003a10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef811ee50 0x7f7ef8133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): picked mon.noname-a con 0x7f7ef811ee50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 0x7f7ef811c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): picked mon.noname-b con 0x7f7ef81341e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef813d8e0 con 0x7f7ef811ee50 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef813d760 con 0x7f7ef81341e0 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef813da60 con 0x7f7ee4003a10 2024-09-15T21:24:56.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient(hunting): authenticate will time out at 1415.016890s 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 0x7f7ef811c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 0x7f7ef811c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38662/0 (socket says 172.21.15.119:38662) 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 -- 172.21.15.119:0/575014975 learned_addr learned my addr 172.21.15.119:0/575014975 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): get_auth_request con 0x7f7ef81341e0 auth_method 0 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:56.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ee4003a10 0x7f7ef81cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 10 monclient(hunting): get_auth_request con 0x7f7ee4003a10 auth_method 0 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef5d9b640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef811ee50 0x7f7ef8133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient(hunting): handle_auth_done global_id 84367 payload 824 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 -- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ee4003a10 msgr2=0x7f7ef81cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ee4003a10 0x7f7ef81cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 -- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef811ee50 msgr2=0x7f7ef8133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef811ee50 0x7f7ef8133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 monclient: found mon.noname-b 2024-09-15T21:24:56.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 -- 172.21.15.119:0/575014975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef811fdb0 con 0x7f7ef81341e0 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef5d9b640 10 monclient: get_auth_request con 0x7f7ef811ee50 auth_method 0 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef5d9b640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef811ee50 0x7f7ef8133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef6d9d640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 0x7f7ef811c880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000ee50 tx=0x7f7ef000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0018020 con 0x7f7ef81341e0 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:56.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: dump: 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: _finish_auth 0 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:24:56.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef000bc50 con 0x7f7ef81341e0 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 5 monclient: authenticate success, global_id 84367 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0014400 con 0x7f7ef81341e0 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _renew_subs 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef81cd550 con 0x7f7ef81341e0 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef4d99640 4 set_mon_vals no callback set 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 10 monclient: dump: 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:56.248 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _renew_subs 2024-09-15T21:24:56.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef811e2f0 con 0x7f7ef81341e0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef8137200 con 0x7f7ef81341e0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7ef001b020 con 0x7f7ef81341e0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7ed803b9c0 0x7f7ed803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(220..220 src has 1..220) ==== 14564+0+0 (secure 0 0 0) 0x7f7ef0052980 con 0x7f7ef81341e0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7ed803b9c0 0x7f7ed803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 10 monclient: get_auth_request con 0x7f7ed803b9c0 auth_method 0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.218+0000 7f7ef659c640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7ed803b9c0 0x7f7ed803de80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c240 tx=0x7f7ee00b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.222+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7ef0022030 con 0x7f7ef81341e0 2024-09-15T21:24:56.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.222+0000 7f7eeeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:56.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.222+0000 7f7eeeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:56.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.398+0000 7f7efcfb0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T21:24:56.401 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.398+0000 7f7efcfb0640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T21:24:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.398+0000 7f7efcfb0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.398+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) -- 0x7f7ef809f8f0 con 0x7f7ef81341e0 2024-09-15T21:24:56.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.978+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 removed item id -25 name 'h3' from crush map v221) ==== 118+0+0 (secure 0 0 0) 0x7f7ef811fdb0 con 0x7f7ef81341e0 2024-09-15T21:24:56.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.978+0000 7f7eeeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T21:24:56.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:56.978+0000 7f7eeeffd640 10 monclient: _finish_command 2 = system:0 removed item id -25 name 'h3' from crush map 2024-09-15T21:24:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.086+0000 7f7efcfb0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T21:24:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.086+0000 7f7efcfb0640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T21:24:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.086+0000 7f7efcfb0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.086+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) -- 0x7f7ef81205b0 con 0x7f7ef81341e0 2024-09-15T21:24:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7eeeffd640 1 -- 172.21.15.119:0/575014975 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 device 'h3' does not appear in the crush map v221) ==== 118+0+0 (secure 0 0 0) 0x7f7ef004ed50 con 0x7f7ef81341e0 2024-09-15T21:24:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7eeeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T21:24:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7eeeffd640 10 monclient: _finish_command 3 = system:0 device 'h3' does not appear in the crush map 2024-09-15T21:24:57.092 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h3' does not appear in the crush map 2024-09-15T21:24:57.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7ed803b9c0 msgr2=0x7f7ed803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:57.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7ed803b9c0 0x7f7ed803de80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c240 tx=0x7f7ee00b2040 comp rx=0 tx=0).stop 2024-09-15T21:24:57.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 10 monclient: shutdown 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 msgr2=0x7f7ef811c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 0x7f7ef811c880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000ee50 tx=0x7f7ef000b480 comp rx=0 tx=0).stop 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 shutdown_connections 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7ed803b9c0 0x7f7ed803de80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81341e0 0x7f7ef811c880 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef811ee50 0x7f7ef8133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 --2- 172.21.15.119:0/575014975 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7ee4003a10 0x7f7ef81cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 >> 172.21.15.119:0/575014975 conn(0x7f7ef80a2ae0 msgr2=0x7f7ef8072210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 shutdown_connections 2024-09-15T21:24:57.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.090+0000 7f7efcfb0640 1 -- 172.21.15.119:0/575014975 wait complete. 2024-09-15T21:24:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:57.109 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h4 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.218+0000 7f423d0c5640 10 monclient: get_monmap_and_config 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.218+0000 7f423d0c5640 10 monclient: build_initial_monmap 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.218+0000 7f423d0c5640 10 monclient: monmap: 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:57.223735+0000 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:57.223735+0000 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:57.227 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.218+0000 7f423d0c5640 1 Processor -- start 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.218+0000 7f423d0c5640 1 -- start start 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.218+0000 7f423d0c5640 10 monclient: init 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4238135ca0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:57.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42380a0160 0x7f423813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient(hunting): picked mon.noname-c con 0x7f42380a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f423806ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423809cf30 con 0x7f4238135ca0 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42380a1ca0 con 0x7f423806ec30 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f42380a1f80 con 0x7f42380a0160 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient(hunting): authenticate will time out at 1416.020376s 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42380a0160 0x7f423813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42380a0160 0x7f423813aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38678/0 (socket says 172.21.15.119:38678) 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 -- 172.21.15.119:0/786005406 learned_addr learned my addr 172.21.15.119:0/786005406 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): get_auth_request con 0x7f42380a0160 auth_method 0 2024-09-15T21:24:57.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient(hunting): handle_auth_done global_id 84370 payload 274 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 -- 172.21.15.119:0/786005406 >> v1:172.21.15.119:6789/0 conn(0x7f423806ec30 legacy=0x7f423813cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 -- 172.21.15.119:0/786005406 >> v1:172.21.15.17:6789/0 conn(0x7f4238135ca0 legacy=0x7f4238136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 monclient: found mon.noname-c 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 -- 172.21.15.119:0/786005406 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42380a3860 con 0x7f42380a0160 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423659c640 1 --2- 172.21.15.119:0/786005406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f423813aa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e9cb8a2cbe26b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 1 -- 172.21.15.119:0/786005406 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0b1070 con 0x7f42380a0160 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: dump: 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:57.231 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: _finish_auth 0 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 1 -- 172.21.15.119:0/786005406 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c0ac640 con 0x7f42380a0160 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 1 -- 172.21.15.119:0/786005406 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0aca10 con 0x7f42380a0160 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 5 monclient: authenticate success, global_id 84370 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 4 set_mon_vals no callback set 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:57.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f4235d9b640 10 monclient: dump: 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:57.233 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:57.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 10 monclient: shutdown 2024-09-15T21:24:57.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:57.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 1 -- 172.21.15.119:0/786005406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 msgr2=0x7f423813aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:57.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.222+0000 7f423d0c5640 1 --2- 172.21.15.119:0/786005406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f423813aa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c01c400 comp rx=0 tx=0).stop 2024-09-15T21:24:57.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/786005406 shutdown_connections 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 --2- 172.21.15.119:0/786005406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f423813aa00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/786005406 >> 172.21.15.119:0/786005406 conn(0x7f42380a2ae0 msgr2=0x7f42380a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/786005406 shutdown_connections 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/786005406 wait complete. 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: build_initial_monmap 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: monmap: 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:57.239 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 Processor -- start 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- start start 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: init 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f423806ec30 0x7f4238133280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): picked mon.noname-a con 0x7f423806ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f42381337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): picked mon.noname-b con 0x7f42380a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:57.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4238135ca0 0x7f423813b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): picked mon.noname-c con 0x7f4238135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4238141010 con 0x7f423806ec30 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4238140e90 con 0x7f42380a0160 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4238141190 con 0x7f4238135ca0 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient(hunting): authenticate will time out at 1416.025583s 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f42381337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f42381337c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38686/0 (socket says 172.21.15.119:38686) 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 -- 172.21.15.119:0/627199630 learned_addr learned my addr 172.21.15.119:0/627199630 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:57.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): get_auth_request con 0x7f42380a0160 auth_method 0 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient(hunting): handle_auth_done global_id 84376 payload 824 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 -- 172.21.15.119:0/627199630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4238135ca0 msgr2=0x7f423813b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4238135ca0 0x7f423813b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 -- 172.21.15.119:0/627199630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f423806ec30 msgr2=0x7f4238133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f423806ec30 0x7f4238133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient: _finish_hunting 0 2024-09-15T21:24:57.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 monclient: found mon.noname-b 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 -- 172.21.15.119:0/627199630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423811d1d0 con 0x7f42380a0160 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423659c640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f42381337c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f422c01a030 tx=0x7f422c01ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0bb020 con 0x7f42380a0160 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: dump: 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.243 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c0aaa70 con 0x7f42380a0160 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 5 monclient: authenticate success, global_id 84376 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _renew_subs 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423811c630 con 0x7f42380a0160 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0b1040 con 0x7f42380a0160 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 10 monclient: dump: 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:57.245 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f4234d99640 4 set_mon_vals no callback set 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _renew_subs 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423813cbb0 con 0x7f42380a0160 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423809f8f0 con 0x7f42380a0160 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f422c0be020 con 0x7f42380a0160 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b9c0 0x7f422403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 14354+0+0 (secure 0 0 0) 0x7f422c0ec990 con 0x7f42380a0160 2024-09-15T21:24:57.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f4236d9d640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b9c0 0x7f422403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:57.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.226+0000 7f4236d9d640 10 monclient: get_auth_request con 0x7f422403b9c0 auth_method 0 2024-09-15T21:24:57.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.230+0000 7f4236d9d640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b9c0 0x7f422403de80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4228003cf0 tx=0x7f42280079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:57.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.230+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f422c0c5030 con 0x7f42380a0160 2024-09-15T21:24:57.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.230+0000 7f421f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:57.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.230+0000 7f421f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.394+0000 7f423d0c5640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T21:24:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.394+0000 7f423d0c5640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T21:24:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.394+0000 7f423d0c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:57.398 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.394+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) -- 0x7f4238134880 con 0x7f42380a0160 2024-09-15T21:24:57.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.986+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 removed item id -28 name 'h4' from crush map v222) ==== 118+0+0 (secure 0 0 0) 0x7f422c0eb090 con 0x7f42380a0160 2024-09-15T21:24:57.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.986+0000 7f421f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T21:24:57.990 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:57.986+0000 7f421f7fe640 10 monclient: _finish_command 2 = system:0 removed item id -28 name 'h4' from crush map 2024-09-15T21:24:58.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f423d0c5640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T21:24:58.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f423d0c5640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T21:24:58.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f423d0c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) -- 0x7f423806c460 con 0x7f42380a0160 2024-09-15T21:24:58.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f421f7fe640 1 -- 172.21.15.119:0/627199630 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 device 'h4' does not appear in the crush map v222) ==== 118+0+0 (secure 0 0 0) 0x7f422c016110 con 0x7f42380a0160 2024-09-15T21:24:58.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f421f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T21:24:58.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.098+0000 7f421f7fe640 10 monclient: _finish_command 3 = system:0 device 'h4' does not appear in the crush map 2024-09-15T21:24:58.102 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h4' does not appear in the crush map 2024-09-15T21:24:58.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b9c0 msgr2=0x7f422403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b9c0 0x7f422403de80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4228003cf0 tx=0x7f42280079c0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 10 monclient: shutdown 2024-09-15T21:24:58.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 msgr2=0x7f42381337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f42381337c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f422c01a030 tx=0x7f422c01ac40 comp rx=0 tx=0).stop 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 shutdown_connections 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f422403b9c0 0x7f422403de80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4238135ca0 0x7f423813b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42380a0160 0x7f42381337c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 --2- 172.21.15.119:0/627199630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f423806ec30 0x7f4238133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 >> 172.21.15.119:0/627199630 conn(0x7f42380a2ae0 msgr2=0x7f4238071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 shutdown_connections 2024-09-15T21:24:58.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.102+0000 7f423d0c5640 1 -- 172.21.15.119:0/627199630 wait complete. 2024-09-15T21:24:58.120 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:58.120 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h5 2024-09-15T21:24:58.143 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient: get_monmap_and_config 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient: build_initial_monmap 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient: monmap: 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:58.233928+0000 2024-09-15T21:24:58.233 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:58.233928+0000 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:58.234 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 Processor -- start 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 -- start start 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient: init 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcbd8137bc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcbd80a09e0 0x7fcbd813ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): picked mon.noname-a con 0x7fcbd80a09e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbd80a0ea0 0x7fcbd813efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): picked mon.noname-c con 0x7fcbd80a0ea0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd809cfa0 con 0x7fcbd8137bc0 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd81386b0 con 0x7fcbd80a09e0 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd809cd20 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbdd5e5640 10 monclient(hunting): authenticate will time out at 1417.030539s 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbd80a0ea0 0x7fcbd813efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbd80a0ea0 0x7fcbd813efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38688/0 (socket says 172.21.15.119:38688) 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 -- 172.21.15.119:0/3265058309 learned_addr learned my addr 172.21.15.119:0/3265058309 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:58.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): get_auth_request con 0x7fcbd80a0ea0 auth_method 0 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd67fc640 1 --2- 172.21.15.119:0/3265058309 >> v2:172.21.15.17:3300/0 conn(0x7fcbd80a09e0 0x7fcbd813ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd67fc640 10 monclient(hunting): get_auth_request con 0x7fcbd80a09e0 auth_method 0 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient(hunting): handle_auth_done global_id 84379 payload 274 2024-09-15T21:24:58.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 -- 172.21.15.119:0/3265058309 >> v2:172.21.15.17:3300/0 conn(0x7fcbd80a09e0 msgr2=0x7fcbd813ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 --2- 172.21.15.119:0/3265058309 >> v2:172.21.15.17:3300/0 conn(0x7fcbd80a09e0 0x7fcbd813ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 -- 172.21.15.119:0/3265058309 >> v1:172.21.15.17:6789/0 conn(0x7fcbd8137bc0 legacy=0x7fcbd8137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 monclient: found mon.noname-c 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.230+0000 7fcbd77fe640 1 -- 172.21.15.119:0/3265058309 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd80a1820 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd77fe640 1 --2- 172.21.15.119:0/3265058309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd813efe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbd806a1c0 tx=0x7fcbd001c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f9335b8b5370bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 1 -- 172.21.15.119:0/3265058309 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00a57f0 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: dump: 2024-09-15T21:24:58.241 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 5 monclient: authenticate success, global_id 84379 2024-09-15T21:24:58.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 1 -- 172.21.15.119:0/3265058309 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd00a5990 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 4 set_mon_vals no callback set 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 1 -- 172.21.15.119:0/3265058309 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00a5d60 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd5ffb640 10 monclient: dump: 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.243 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: shutdown 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/3265058309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 msgr2=0x7fcbd813efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/3265058309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd813efe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbd806a1c0 tx=0x7fcbd001c910 comp rx=0 tx=0).stop 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/3265058309 shutdown_connections 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/3265058309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd813efe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/3265058309 >> v2:172.21.15.17:3300/0 conn(0x7fcbd80a09e0 0x7fcbd813ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/3265058309 >> 172.21.15.119:0/3265058309 conn(0x7fcbd80a2ae0 msgr2=0x7fcbd80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:58.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/3265058309 shutdown_connections 2024-09-15T21:24:58.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/3265058309 wait complete. 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: build_initial_monmap 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: monmap: 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:58.249 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 Processor -- start 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- start start 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: init 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbd80a09e0 0x7fcbd811fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): picked mon.noname-c con 0x7fcbd80a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd811da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): picked mon.noname-b con 0x7fcbd80a0ea0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd8137bc0 0x7fcbd81369c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): picked mon.noname-a con 0x7fcbd8137bc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8141f80 con 0x7fcbd8137bc0 2024-09-15T21:24:58.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8141e00 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd8142100 con 0x7fcbd80a09e0 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbdd5e5640 10 monclient(hunting): authenticate will time out at 1417.035641s 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd811da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd811da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38702/0 (socket says 172.21.15.119:38702) 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 1 -- 172.21.15.119:0/624272193 learned_addr learned my addr 172.21.15.119:0/624272193 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): get_auth_request con 0x7fcbd80a0ea0 auth_method 0 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd77fe640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd8137bc0 0x7fcbd81369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.234+0000 7fcbd6ffd640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbd80a09e0 0x7fcbd811fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd77fe640 10 monclient(hunting): get_auth_request con 0x7fcbd8137bc0 auth_method 0 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 10 monclient(hunting): get_auth_request con 0x7fcbd80a09e0 auth_method 0 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 10 monclient(hunting): handle_auth_done global_id 84382 payload 824 2024-09-15T21:24:58.252 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 -- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbd80a09e0 msgr2=0x7fcbd811fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbd80a09e0 0x7fcbd811fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 -- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd8137bc0 msgr2=0x7fcbd81369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd8137bc0 0x7fcbd81369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 monclient: found mon.noname-b 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 -- 172.21.15.119:0/624272193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd811eff0 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd67fc640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd811da10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800d800 tx=0x7fcbc800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc800b6c0 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:58.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: dump: 2024-09-15T21:24:58.254 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:58.254 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.254 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.254 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 5 monclient: authenticate success, global_id 84382 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 monclient: _renew_subs 2024-09-15T21:24:58.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc8019070 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd813e880 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc800ba60 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 10 monclient: dump: 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:58.257 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 monclient: _renew_subs 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd8136f00 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd57fa640 4 set_mon_vals no callback set 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd8137130 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcbc800bc00 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbc403b9c0 0x7fcbc403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 14144+0+0 (secure 0 0 0) 0x7fcbc8052580 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbc403b9c0 0x7fcbc403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 10 monclient: get_auth_request con 0x7fcbc403b9c0 auth_method 0 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.238+0000 7fcbd6ffd640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbc403b9c0 0x7fcbc403de80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0016f0 tx=0x7fcbcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.242+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcbc804eb00 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.242+0000 7fcbbf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:58.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.242+0000 7fcbbf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:58.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient: get_monmap_and_config 2024-09-15T21:24:58.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient: build_initial_monmap 2024-09-15T21:24:58.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient: monmap: 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:24:58.275964+0000 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:24:58.275964+0000 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:58.277 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 Processor -- start 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 -- start start 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient: init 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): picked mon.noname-a con 0x7f8d601379c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8d600a63b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:24:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 0x7f8d6013ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): picked mon.noname-b con 0x7f8d601384b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6009c2a0 con 0x7f8d600a63b0 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8d6009c840 con 0x7f8d601379c0 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6009c510 con 0x7f8d601384b0 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d6720a640 10 monclient(hunting): authenticate will time out at 1421.671401s 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 0x7f8d6013ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35686/0 (socket says 172.21.15.17:35686) 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 0x7f8d6013ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36928/0 (socket says 172.21.15.17:36928) 2024-09-15T21:24:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 -- 172.21.15.17:0/2405501260 learned_addr learned my addr 172.21.15.17:0/2405501260 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 10 monclient(hunting): get_auth_request con 0x7f8d601379c0 auth_method 0 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d5ffff640 1 --1- 172.21.15.17:0/2405501260 >> v1:172.21.15.119:6789/0 conn(0x7f8d600a63b0 0x7f8d60137da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:52214/0 (socket says 172.21.15.17:52214) 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): get_auth_request con 0x7f8d601384b0 auth_method 0 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 -- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 msgr2=0x7f8d6006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 -- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 msgr2=0x7f8d6006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 --2- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d657a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.271+0000 7f8d64fa6640 1 --2- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 10 monclient(hunting): handle_auth_done global_id 45059 payload 274 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 1 -- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 msgr2=0x7f8d6006f5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 1 --2- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 1 -- 172.21.15.17:0/2405501260 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635845354 0 0) 0x7f8d6009c2a0 con 0x7f8d600a63b0 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 1 -- 172.21.15.17:0/2405501260 >> v1:172.21.15.119:6789/0 conn(0x7f8d600a63b0 legacy=0x7f8d60137da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 10 monclient: _finish_hunting 0 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 1 monclient: found mon.noname-b 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 1 -- 172.21.15.17:0/2405501260 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6009d1d0 con 0x7f8d601384b0 2024-09-15T21:24:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:24:58.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d657a7640 1 --2- 172.21.15.17:0/2405501260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 0x7f8d6013ef00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009d90 tx=0x7f8d5801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f9680d905849c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:58.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 1 -- 172.21.15.17:0/2405501260 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d580b2070 con 0x7f8d601384b0 2024-09-15T21:24:58.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:58.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: dump: 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:58.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 5 monclient: authenticate success, global_id 45059 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 1 -- 172.21.15.17:0/2405501260 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d580a5800 con 0x7f8d601384b0 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 1 -- 172.21.15.17:0/2405501260 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d580a5bd0 con 0x7f8d601384b0 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 4 set_mon_vals no callback set 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d5f7fe640 10 monclient: dump: 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.290 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 10 monclient: shutdown 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 -- 172.21.15.17:0/2405501260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 msgr2=0x7f8d6013ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 --2- 172.21.15.17:0/2405501260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 0x7f8d6013ef00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009d90 tx=0x7f8d5801c830 comp rx=0 tx=0).stop 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 -- 172.21.15.17:0/2405501260 shutdown_connections 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 --2- 172.21.15.17:0/2405501260 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d601384b0 0x7f8d6013ef00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 --2- 172.21.15.17:0/2405501260 >> v2:172.21.15.17:3300/0 conn(0x7f8d601379c0 0x7f8d6006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 -- 172.21.15.17:0/2405501260 >> 172.21.15.17:0/2405501260 conn(0x7f8d600a7100 msgr2=0x7f8d600a7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 -- 172.21.15.17:0/2405501260 shutdown_connections 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 1 -- 172.21.15.17:0/2405501260 wait complete. 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 10 monclient: build_initial_monmap 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.275+0000 7f8d6720a640 10 monclient: monmap: 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 Processor -- start 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- start start 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: init 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 0x7f8d60136520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): picked mon.noname-c con 0x7f8d600a63b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 0x7f8d60136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): picked mon.noname-a con 0x7f8d601379c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d601384b0 0x7f8d6013bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): picked mon.noname-b con 0x7f8d601384b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8d60142140 con 0x7f8d601379c0 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d60141fc0 con 0x7f8d601384b0 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8d601422c0 con 0x7f8d600a63b0 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient(hunting): authenticate will time out at 1421.676631s 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 0x7f8d60136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 0x7f8d60136520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36938/0 (socket says 172.21.15.17:36938) 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 -- 172.21.15.17:0/1102031149 learned_addr learned my addr 172.21.15.17:0/1102031149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): get_auth_request con 0x7f8d600a63b0 auth_method 0 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 0x7f8d60136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 10 monclient(hunting): get_auth_request con 0x7f8d601379c0 auth_method 0 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d601384b0 0x7f8d6013bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): get_auth_request con 0x7f8d601384b0 auth_method 0 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 -- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 msgr2=0x7f8d60136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 -- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 msgr2=0x7f8d60136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 0x7f8d60136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 0x7f8d60136a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d657a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient(hunting): handle_auth_done global_id 45062 payload 824 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 -- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d601384b0 msgr2=0x7f8d6013bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d601384b0 0x7f8d6013bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 -- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 msgr2=0x7f8d60136a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:24:58.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 0x7f8d60136a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient: _finish_hunting 0 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 monclient: found mon.noname-c 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 -- 172.21.15.17:0/1102031149 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d60121020 con 0x7f8d600a63b0 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d64fa6640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 0x7f8d60136520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e840 tx=0x7f8d5000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50017070 con 0x7f8d600a63b0 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: dump: 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d50004020 con 0x7f8d600a63b0 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000f040 con 0x7f8d600a63b0 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 5 monclient: authenticate success, global_id 45062 2024-09-15T21:24:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _renew_subs 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d60121300 con 0x7f8d600a63b0 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5e7fc640 4 set_mon_vals no callback set 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 10 monclient: dump: 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _renew_subs 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d60120d40 con 0x7f8d600a63b0 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8d5000c6d0 con 0x7f8d600a63b0 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6006f5e0 con 0x7f8d600a63b0 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d4003b9c0 0x7f8d4003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(222..222 src has 1..222) ==== 14144+0+0 (secure 0 0 0) 0x7f8d50051d00 con 0x7f8d600a63b0 2024-09-15T21:24:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d4003b9c0 0x7f8d4003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 10 monclient: get_auth_request con 0x7f8d4003b9c0 auth_method 0 2024-09-15T21:24:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.279+0000 7f8d5ffff640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d4003b9c0 0x7f8d4003de80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d48007fd0 tx=0x7f8d48007b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.283+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8d50004410 con 0x7f8d600a63b0 2024-09-15T21:24:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.283+0000 7f8d5d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.283+0000 7f8d5d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:58.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.410+0000 7fcbdd5e5640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T21:24:58.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.410+0000 7fcbdd5e5640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T21:24:58.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.410+0000 7fcbdd5e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:58.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.410+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) -- 0x7fcbd8137fa0 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8d600a7800 con 0x7f8d600a63b0 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d5d7fa640 1 -- 172.21.15.17:0/1102031149 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f8d5004e240 con 0x7f8d600a63b0 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d5d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d5d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:24:58.533 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":68,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d4003b9c0 msgr2=0x7f8d4003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d4003b9c0 0x7f8d4003de80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d48007fd0 tx=0x7f8d48007b20 comp rx=0 tx=0).stop 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 10 monclient: shutdown 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 msgr2=0x7f8d60136520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 0x7f8d60136520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e840 tx=0x7f8d5000ed00 comp rx=0 tx=0).stop 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.527+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 shutdown_connections 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d4003b9c0 0x7f8d4003de80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d601384b0 0x7f8d6013bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601379c0 0x7f8d60136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 --2- 172.21.15.17:0/1102031149 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d600a63b0 0x7f8d60136520 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 >> 172.21.15.17:0/1102031149 conn(0x7f8d600a7100 msgr2=0x7f8d600a1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:58.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 shutdown_connections 2024-09-15T21:24:58.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:24:58.531+0000 7f8d6720a640 1 -- 172.21.15.17:0/1102031149 wait complete. 2024-09-15T21:24:58.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.990+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 removed item id -31 name 'h5' from crush map v223) ==== 118+0+0 (secure 0 0 0) 0x7fcbc801cd70 con 0x7fcbd80a0ea0 2024-09-15T21:24:58.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.990+0000 7fcbbf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T21:24:58.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:58.990+0000 7fcbbf7fe640 10 monclient: _finish_command 2 = system:0 removed item id -31 name 'h5' from crush map 2024-09-15T21:24:59.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbdd5e5640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T21:24:59.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbdd5e5640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T21:24:59.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbdd5e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) -- 0x7fcbd8120c80 con 0x7fcbd80a0ea0 2024-09-15T21:24:59.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbbf7fe640 1 -- 172.21.15.119:0/624272193 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 device 'h5' does not appear in the crush map v223) ==== 118+0+0 (secure 0 0 0) 0x7fcbc801cad0 con 0x7fcbd80a0ea0 2024-09-15T21:24:59.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbbf7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T21:24:59.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.094+0000 7fcbbf7fe640 10 monclient: _finish_command 3 = system:0 device 'h5' does not appear in the crush map 2024-09-15T21:24:59.100 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h5' does not appear in the crush map 2024-09-15T21:24:59.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbc403b9c0 msgr2=0x7fcbc403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:59.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbc403b9c0 0x7fcbc403de80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0016f0 tx=0x7fcbcc009040 comp rx=0 tx=0).stop 2024-09-15T21:24:59.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 10 monclient: shutdown 2024-09-15T21:24:59.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:59.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 msgr2=0x7fcbd811da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:59.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd811da10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800d800 tx=0x7fcbc800dcc0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 shutdown_connections 2024-09-15T21:24:59.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcbc403b9c0 0x7fcbc403de80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd8137bc0 0x7fcbd81369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd80a0ea0 0x7fcbd811da10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 --2- 172.21.15.119:0/624272193 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcbd80a09e0 0x7fcbd811fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 >> 172.21.15.119:0/624272193 conn(0x7fcbd80a2ae0 msgr2=0x7fcbd80a4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:59.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 shutdown_connections 2024-09-15T21:24:59.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.098+0000 7fcbdd5e5640 1 -- 172.21.15.119:0/624272193 wait complete. 2024-09-15T21:24:59.116 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:24:59.116 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h6 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient: get_monmap_and_config 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient: build_initial_monmap 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient: monmap: 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:24:59.226852+0000 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:24:59.226852+0000 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:59.226 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:59.227 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:24:59.227 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:24:59.227 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:24:59.227 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:24:59.227 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:24:59.227 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:59.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 Processor -- start 2024-09-15T21:24:59.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 -- start start 2024-09-15T21:24:59.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient: init 2024-09-15T21:24:59.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff0f80a4ca0 0x7ff0f80a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): picked mon.noname-a con 0x7ff0f80a4ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0f80a55c0 0x7ff0f8137a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): picked mon.noname-c con 0x7ff0f80a55c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f8137fa0 0x7ff0f8138430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): picked mon.noname-b con 0x7ff0f8137fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f80740f0 con 0x7ff0f80a4ca0 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f80a5fa0 con 0x7ff0f80a55c0 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f806dd20 con 0x7ff0f8137fa0 2024-09-15T21:24:59.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fefac640 10 monclient(hunting): authenticate will time out at 1418.023391s 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0f80a55c0 0x7ff0f8137a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0f80a55c0 0x7ff0f8137a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38708/0 (socket says 172.21.15.119:38708) 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 1 -- 172.21.15.119:0/3350042648 learned_addr learned my addr 172.21.15.119:0/3350042648 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): get_auth_request con 0x7ff0f80a55c0 auth_method 0 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fd549640 1 --2- 172.21.15.119:0/3350042648 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f8137fa0 0x7ff0f8138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fcd48640 1 --2- 172.21.15.119:0/3350042648 >> v2:172.21.15.17:3300/0 conn(0x7ff0f80a4ca0 0x7ff0f80a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fd549640 10 monclient(hunting): get_auth_request con 0x7ff0f8137fa0 auth_method 0 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fd549640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fd549640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fd549640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fcd48640 10 monclient(hunting): get_auth_request con 0x7ff0f80a4ca0 auth_method 0 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fcd48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fcd48640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.222+0000 7ff0fcd48640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 10 monclient(hunting): handle_auth_done global_id 84391 payload 274 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 -- 172.21.15.119:0/3350042648 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f8137fa0 msgr2=0x7ff0f8138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 --2- 172.21.15.119:0/3350042648 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f8137fa0 0x7ff0f8138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 -- 172.21.15.119:0/3350042648 >> v2:172.21.15.17:3300/0 conn(0x7ff0f80a4ca0 msgr2=0x7ff0f80a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 --2- 172.21.15.119:0/3350042648 >> v2:172.21.15.17:3300/0 conn(0x7ff0f80a4ca0 0x7ff0f80a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:24:59.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 monclient: found mon.noname-c 2024-09-15T21:24:59.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:59.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 -- 172.21.15.119:0/3350042648 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f80a23d0 con 0x7ff0f80a55c0 2024-09-15T21:24:59.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f7fff640 1 --2- 172.21.15.119:0/3350042648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8137a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80a1490 tx=0x7ff0ec01c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c404a31008cbac2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:59.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 1 -- 172.21.15.119:0/3350042648 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0b1070 con 0x7ff0f80a55c0 2024-09-15T21:24:59.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:59.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: dump: 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: _finish_auth 0 2024-09-15T21:24:59.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 5 monclient: authenticate success, global_id 84391 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 1 -- 172.21.15.119:0/3350042648 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0ac410 con 0x7ff0f80a55c0 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 10 monclient: get_monmap_and_config success 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 4 set_mon_vals no callback set 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 1 -- 172.21.15.119:0/3350042648 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0ac7e0 con 0x7ff0f80a55c0 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0f77fe640 10 monclient: dump: 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:59.240 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 10 monclient: shutdown 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 -- 172.21.15.119:0/3350042648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f80a55c0 msgr2=0x7ff0f8137a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 --2- 172.21.15.119:0/3350042648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8137a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80a1490 tx=0x7ff0ec01c500 comp rx=0 tx=0).stop 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 -- 172.21.15.119:0/3350042648 shutdown_connections 2024-09-15T21:24:59.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 --2- 172.21.15.119:0/3350042648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8137a60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 --2- 172.21.15.119:0/3350042648 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f8137fa0 0x7ff0f8138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 --2- 172.21.15.119:0/3350042648 >> v2:172.21.15.17:3300/0 conn(0x7ff0f80a4ca0 0x7ff0f80a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.226+0000 7ff0fefac640 1 -- 172.21.15.119:0/3350042648 >> 172.21.15.119:0/3350042648 conn(0x7ff0f80a7400 msgr2=0x7ff0f806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- 172.21.15.119:0/3350042648 shutdown_connections 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- 172.21.15.119:0/3350042648 wait complete. 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: build_initial_monmap 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: monmap: 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:59.242 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 Processor -- start 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- start start 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: init 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _reopen_session rank -1 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f80a4ca0 0x7ff0f8131cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): picked mon.noname-c con 0x7ff0f80a4ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8132230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): picked mon.noname-a con 0x7ff0f80a55c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 0x7ff0f8071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): picked mon.noname-b con 0x7ff0f8137fa0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f80a45e0 con 0x7ff0f80a55c0 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f80a59d0 con 0x7ff0f8137fa0 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): start opening mon connection 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f80a4760 con 0x7ff0f80a4ca0 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): _renew_subs 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient(hunting): authenticate will time out at 1418.029481s 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 0x7ff0f8071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 0x7ff0f8071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38718/0 (socket says 172.21.15.119:38718) 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 -- 172.21.15.119:0/161195389 learned_addr learned my addr 172.21.15.119:0/161195389 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): get_auth_request con 0x7ff0f8137fa0 auth_method 0 2024-09-15T21:24:59.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f7fff640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8132230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fcd48640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f80a4ca0 0x7ff0f8131cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f7fff640 10 monclient(hunting): get_auth_request con 0x7ff0f80a55c0 auth_method 0 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:24:59.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient(hunting): handle_auth_done global_id 84394 payload 824 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 -- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f80a4ca0 msgr2=0x7ff0f8131cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f80a4ca0 0x7ff0f8131cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 -- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0f80a55c0 msgr2=0x7ff0f8132230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8132230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient: _finish_hunting 0 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 monclient: found mon.noname-b 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 -- 172.21.15.119:0/161195389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f813a980 con 0x7ff0f8137fa0 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fd549640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 0x7ff0f8071100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0007ea0 tx=0x7ff0f0007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:59.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0014620 con 0x7ff0f8137fa0 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: dump: 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:59.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: _finish_auth 0 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 5 monclient: authenticate success, global_id 84394 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f0019070 con 0x7ff0f8137fa0 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0014a20 con 0x7ff0f8137fa0 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 10 monclient: dump: 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:24:59.248 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f6ffd640 4 set_mon_vals no callback set 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _renew_subs 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f80a93b0 con 0x7ff0f8137fa0 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _renew_subs 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f813a6a0 con 0x7ff0f8137fa0 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:59.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f806d5f0 con 0x7ff0f8137fa0 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff0f0014bc0 con 0x7ff0f8137fa0 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0e403b9c0 0x7ff0e403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 13934+0+0 (secure 0 0 0) 0x7ff0f00527f0 con 0x7ff0f8137fa0 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fcd48640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0e403b9c0 0x7ff0e403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.230+0000 7ff0fcd48640 10 monclient: get_auth_request con 0x7ff0e403b9c0 auth_method 0 2024-09-15T21:24:59.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.234+0000 7ff0fcd48640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0e403b9c0 0x7ff0e403de80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80096f0 tx=0x7ff0e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:24:59.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.238+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff0f004ee40 con 0x7ff0f8137fa0 2024-09-15T21:24:59.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.238+0000 7ff0f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:24:59.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.238+0000 7ff0f57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:24:59.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.398+0000 7ff0fefac640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T21:24:59.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.398+0000 7ff0fefac640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T21:24:59.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.398+0000 7ff0fefac640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:24:59.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:24:59.398+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) -- 0x7ff0f80a1500 con 0x7ff0f8137fa0 2024-09-15T21:25:00.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.006+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 removed item id -34 name 'h6' from crush map v224) ==== 118+0+0 (secure 0 0 0) 0x7ff0f00124c0 con 0x7ff0f8137fa0 2024-09-15T21:25:00.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.006+0000 7ff0f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T21:25:00.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.006+0000 7ff0f57fa640 10 monclient: _finish_command 2 = system:0 removed item id -34 name 'h6' from crush map 2024-09-15T21:25:00.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0fefac640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T21:25:00.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0fefac640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T21:25:00.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0fefac640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) -- 0x7ff0f80008e0 con 0x7ff0f8137fa0 2024-09-15T21:25:00.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0f57fa640 1 -- 172.21.15.119:0/161195389 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 device 'h6' does not appear in the crush map v224) ==== 118+0+0 (secure 0 0 0) 0x7ff0f0012850 con 0x7ff0f8137fa0 2024-09-15T21:25:00.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0f57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T21:25:00.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.114+0000 7ff0f57fa640 10 monclient: _finish_command 3 = system:0 device 'h6' does not appear in the crush map 2024-09-15T21:25:00.120 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h6' does not appear in the crush map 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0e403b9c0 msgr2=0x7ff0e403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0e403b9c0 0x7ff0e403de80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80096f0 tx=0x7ff0e8008040 comp rx=0 tx=0).stop 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 10 monclient: shutdown 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 msgr2=0x7ff0f8071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 0x7ff0f8071100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0007ea0 tx=0x7ff0f0007f90 comp rx=0 tx=0).stop 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 shutdown_connections 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0e403b9c0 0x7ff0e403de80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0f8137fa0 0x7ff0f8071100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0f80a55c0 0x7ff0f8132230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 --2- 172.21.15.119:0/161195389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0f80a4ca0 0x7ff0f8131cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 >> 172.21.15.119:0/161195389 conn(0x7ff0f80a7400 msgr2=0x7ff0f80a81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:00.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 shutdown_connections 2024-09-15T21:25:00.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.118+0000 7ff0fefac640 1 -- 172.21.15.119:0/161195389 wait complete. 2024-09-15T21:25:00.137 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:00.137 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h7 2024-09-15T21:25:00.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient: get_monmap_and_config 2024-09-15T21:25:00.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient: build_initial_monmap 2024-09-15T21:25:00.253 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient: monmap: 2024-09-15T21:25:00.253 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:00.253 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:00.254015+0000 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:00.254015+0000 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:00.254 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 Processor -- start 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 -- start start 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient: init 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81e0137bc0 0x7f81e0137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:00.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): picked mon.noname-a con 0x7f81e0137bc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f81e00a09e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81e00a0ea0 0x7f81e0073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): picked mon.noname-c con 0x7f81e00a0ea0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e009cfa0 con 0x7f81e00a09e0 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f81e009cd20 con 0x7f81e0137bc0 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f81e00a1820 con 0x7f81e00a0ea0 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e77ab640 10 monclient(hunting): authenticate will time out at 1419.050581s 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e4d46640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f81e00a09e0 0x7f81e0073040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52490/0 (socket says 172.21.15.119:52490) 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e4d46640 1 -- 172.21.15.119:0/2055325326 learned_addr learned my addr 172.21.15.119:0/2055325326 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:00.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 1 --2- 172.21.15.119:0/2055325326 >> v2:172.21.15.119:3300/0 conn(0x7f81e00a0ea0 0x7f81e0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): get_auth_request con 0x7f81e00a0ea0 auth_method 0 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5547640 1 --2- 172.21.15.119:0/2055325326 >> v2:172.21.15.17:3300/0 conn(0x7f81e0137bc0 0x7f81e0137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient(hunting): handle_auth_done global_id 84397 payload 274 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 1 -- 172.21.15.119:0/2055325326 >> v2:172.21.15.17:3300/0 conn(0x7f81e0137bc0 msgr2=0x7f81e0137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 1 --2- 172.21.15.119:0/2055325326 >> v2:172.21.15.17:3300/0 conn(0x7f81e0137bc0 0x7f81e0137fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 1 -- 172.21.15.119:0/2055325326 >> v1:172.21.15.119:6789/0 conn(0x7f81e00a09e0 legacy=0x7f81e0073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:00.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient: _finish_hunting 0 2024-09-15T21:25:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 1 monclient: found mon.noname-c 2024-09-15T21:25:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5d48640 1 -- 172.21.15.119:0/2055325326 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e00a1b00 con 0x7f81e00a0ea0 2024-09-15T21:25:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5547640 10 monclient: get_auth_request con 0x7f81e0137bc0 auth_method 0 2024-09-15T21:25:00.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.250+0000 7f81e5547640 1 --2- 172.21.15.119:0/2055325326 >> v2:172.21.15.17:3300/0 conn(0x7f81e0137bc0 0x7f81e0137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:00.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5d48640 1 --2- 172.21.15.119:0/2055325326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a0ea0 0x7f81e0073750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81e006a1c0 tx=0x7f81d801c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f10e6b060fa89c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 1 -- 172.21.15.119:0/2055325326 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80b2070 con 0x7f81e00a0ea0 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: dump: 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:00.264 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 1 -- 172.21.15.119:0/2055325326 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d80a5960 con 0x7f81e00a0ea0 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 1 -- 172.21.15.119:0/2055325326 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80a5d30 con 0x7f81e00a0ea0 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81d3fff640 10 monclient: dump: 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:00.265 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 5 monclient: authenticate success, global_id 84397 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 4 set_mon_vals no callback set 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: shutdown 2024-09-15T21:25:00.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- 172.21.15.119:0/2055325326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a0ea0 msgr2=0x7f81e0073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 --2- 172.21.15.119:0/2055325326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a0ea0 0x7f81e0073750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81e006a1c0 tx=0x7f81d801c880 comp rx=0 tx=0).stop 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- 172.21.15.119:0/2055325326 shutdown_connections 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 --2- 172.21.15.119:0/2055325326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a0ea0 0x7f81e0073750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 --2- 172.21.15.119:0/2055325326 >> v2:172.21.15.17:3300/0 conn(0x7f81e0137bc0 0x7f81e0137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- 172.21.15.119:0/2055325326 >> 172.21.15.119:0/2055325326 conn(0x7f81e00a2ae0 msgr2=0x7f81e00a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- 172.21.15.119:0/2055325326 shutdown_connections 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- 172.21.15.119:0/2055325326 wait complete. 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: build_initial_monmap 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: monmap: 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:00.267 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 Processor -- start 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- start start 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: init 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:00.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 0x7f81e011fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): picked mon.noname-b con 0x7f81e00a09e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81e00a0ea0 0x7f81e011dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): picked mon.noname-c con 0x7f81e00a0ea0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81e0137bc0 0x7f81e0136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): picked mon.noname-a con 0x7f81e0137bc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0073e40 con 0x7f81e0137bc0 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0073cc0 con 0x7f81e00a09e0 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f81e01422d0 con 0x7f81e00a0ea0 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e77ab640 10 monclient(hunting): authenticate will time out at 1419.055734s 2024-09-15T21:25:00.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 0x7f81e011fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 0x7f81e011fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38736/0 (socket says 172.21.15.119:38736) 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 1 -- 172.21.15.119:0/3509438761 learned_addr learned my addr 172.21.15.119:0/3509438761 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): get_auth_request con 0x7f81e00a09e0 auth_method 0 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.254+0000 7f81e5547640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 10 monclient(hunting): handle_auth_done global_id 84403 payload 824 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 -- 172.21.15.119:0/3509438761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81e00a0ea0 msgr2=0x7f81e011dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81e00a0ea0 0x7f81e011dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.270 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 -- 172.21.15.119:0/3509438761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81e0137bc0 msgr2=0x7f81e0136bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81e0137bc0 0x7f81e0136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 10 monclient: _finish_hunting 0 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 monclient: found mon.noname-b 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 -- 172.21.15.119:0/3509438761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e009d1d0 con 0x7f81e00a09e0 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e5547640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 0x7f81e011fd50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c7c0 tx=0x7f81dc00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc007a30 con 0x7f81e00a09e0 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: dump: 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:00.271 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81dc007bd0 con 0x7f81e00a09e0 2024-09-15T21:25:00.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 5 monclient: authenticate success, global_id 84403 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0113f0 con 0x7f81e00a09e0 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 10 monclient: dump: 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:00.273 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d37fe640 4 set_mon_vals no callback set 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 monclient: _renew_subs 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e011f1f0 con 0x7f81e00a09e0 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 monclient: _renew_subs 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e0120cb0 con 0x7f81e00a09e0 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f81dc011590 con 0x7f81e00a09e0 2024-09-15T21:25:00.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e0137fa0 con 0x7f81e00a09e0 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81c403b9f0 0x7f81c403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(224..224 src has 1..224) ==== 13724+0+0 (secure 0 0 0) 0x7f81dc0521a0 con 0x7f81e00a09e0 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e4d46640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81c403b9f0 0x7f81c403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e4d46640 10 monclient: get_auth_request con 0x7f81c403b9f0 auth_method 0 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.258+0000 7f81e4d46640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81c403b9f0 0x7f81c403deb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81d40015f0 tx=0x7f81d4007a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.262+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f81dc015020 con 0x7f81e00a09e0 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.262+0000 7f81d1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:00.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.262+0000 7f81d1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.442+0000 7f81e77ab640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T21:25:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.442+0000 7f81e77ab640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T21:25:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.442+0000 7f81e77ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:00.447 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:00.442+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) -- 0x7f81e0121050 con 0x7f81e00a09e0 2024-09-15T21:25:01.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.010+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 removed item id -37 name 'h7' from crush map v225) ==== 118+0+0 (secure 0 0 0) 0x7f81dc04e8c0 con 0x7f81e00a09e0 2024-09-15T21:25:01.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.010+0000 7f81d1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T21:25:01.014 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.010+0000 7f81d1ffb640 10 monclient: _finish_command 2 = system:0 removed item id -37 name 'h7' from crush map 2024-09-15T21:25:01.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.122+0000 7f81e77ab640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T21:25:01.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.122+0000 7f81e77ab640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T21:25:01.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.122+0000 7f81e77ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:01.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:01.122+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) -- 0x7f81e00008e0 con 0x7f81e00a09e0 2024-09-15T21:25:01.552 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:01.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.675+0000 7f5c38d03640 10 monclient: get_monmap_and_config 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.675+0000 7f5c38d03640 10 monclient: build_initial_monmap 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.675+0000 7f5c38d03640 10 monclient: monmap: 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:01.680225+0000 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:01.680225+0000 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:01.681 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:01.682 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.675+0000 7f5c38d03640 1 Processor -- start 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.675+0000 7f5c38d03640 1 -- start start 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.675+0000 7f5c38d03640 10 monclient: init 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c340a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:01.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): picked mon.noname-a con 0x7f5c340a5580 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): picked mon.noname-b con 0x7f5c340a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c340a2c30 con 0x7f5c340a4a90 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5c340a0f50 con 0x7f5c340a5580 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c340a1a00 con 0x7f5c340a59d0 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c38d03640 10 monclient(hunting): authenticate will time out at 1425.076224s 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:01.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c3259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5c340a4a90 0x7f5c340a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52518/0 (socket says 172.21.15.17:52518) 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 learned_addr learned my addr 172.21.15.17:0/3501843762 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request con 0x7f5c340a5580 auth_method 0 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request con 0x7f5c340a59d0 auth_method 0 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:01.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:25:01.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:01.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:01.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:01.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.679+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request con 0x7f5c340a59d0 auth_method 0 2024-09-15T21:25:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request con 0x7f5c340a5580 auth_method 0 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:01.883+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:25:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request con 0x7f5c340a59d0 auth_method 0 2024-09-15T21:25:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request con 0x7f5c340a5580 auth_method 0 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:25:02.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:02.287+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:25:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request con 0x7f5c340a59d0 auth_method 0 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request con 0x7f5c340a5580 auth_method 0 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:25:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:03.091+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:25:03.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:03.258+0000 7f81d0ff9640 10 monclient: tick 2024-09-15T21:25:03.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:03.258+0000 7f81d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:03.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:03.258+0000 7f81d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:03.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:03.258+0000 7f81d0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:04.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:04.258+0000 7f81d0ff9640 10 monclient: tick 2024-09-15T21:25:04.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:04.258+0000 7f81d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:04.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:04.258+0000 7f81d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): tick 2024-09-15T21:25:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:25:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 monclient(hunting): continuing hunt 2024-09-15T21:25:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 msgr2=0x7f5c3413f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c340a59d0 0x7f5c3413f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 msgr2=0x7f5c3413cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.17:3300/0 conn(0x7f5c340a5580 0x7f5c3413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 -- 172.21.15.17:0/3501843762 >> v1:172.21.15.17:6789/0 conn(0x7f5c340a4a90 legacy=0x7f5c340a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient: _add_conns ranks=[4,1] 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): picked mon.noname-c con 0x7f5c14003700 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5c14006250 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 -- 172.21.15.17:0/3501843762 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c340a1c90 con 0x7f5c14006250 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 1 -- 172.21.15.17:0/3501843762 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5c340a0f50 con 0x7f5c14003700 2024-09-15T21:25:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c23fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 10 monclient(hunting): get_auth_request con 0x7f5c14003700 auth_method 0 2024-09-15T21:25:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.679+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:04.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:04.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 10 monclient(hunting): get_auth_request con 0x7f5c14003700 auth_method 0 2024-09-15T21:25:04.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:04.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:04.883+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:25:05.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:05.258+0000 7f81d0ff9640 10 monclient: tick 2024-09-15T21:25:05.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:05.258+0000 7f81d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:05.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:05.258+0000 7f81d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 10 monclient(hunting): get_auth_request con 0x7f5c14003700 auth_method 0 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:05.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:05.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:05.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:05.287+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:25:06.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:06.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 10 monclient(hunting): get_auth_request con 0x7f5c14003700 auth_method 0 2024-09-15T21:25:06.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:06.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:06.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:06.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:06.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:06.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:06.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:06.091+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:25:06.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:06.258+0000 7f81d0ff9640 10 monclient: tick 2024-09-15T21:25:06.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:06.258+0000 7f81d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:06.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:06.258+0000 7f81d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.258+0000 7f81d0ff9640 10 monclient: tick 2024-09-15T21:25:07.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.258+0000 7f81d0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.258+0000 7f81d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207294671 0 0) 0x7f5c340a1c90 con 0x7f5c14006250 2024-09-15T21:25:07.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): _negotiate 2024-09-15T21:25:07.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): my global_id is 84406 2024-09-15T21:25:07.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c1c003530 con 0x7f5c14006250 2024-09-15T21:25:07.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2970236475 0 0) 0x7f5c1c003530 con 0x7f5c14006250 2024-09-15T21:25:07.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 msgr2=0x7f5c14005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:07.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 --2- 172.21.15.17:0/3501843762 >> v2:172.21.15.119:3300/0 conn(0x7f5c14003700 0x7f5c14005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1400afc0 con 0x7f5c14006250 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5c2800f040 con 0x7f5c14006250 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:07.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: dump: 2024-09-15T21:25:07.499 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.499 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 >> v1:172.21.15.119:6789/0 conn(0x7f5c14006250 legacy=0x7f5c14008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:07.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 1 -- 172.21.15.17:0/3501843762 reap_dead start 2024-09-15T21:25:07.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c1c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): picked mon.c con 0x7f5c1c003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c004810 0x7f5c1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): picked mon.a con 0x7f5c1c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c0053d0 0x7f5c1c00b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): picked mon.b con 0x7f5c1c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c340a0f50 con 0x7f5c1c004810 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c00c2d0 con 0x7f5c1c0053d0 2024-09-15T21:25:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1c00c4d0 con 0x7f5c1c003a60 2024-09-15T21:25:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3159a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c1c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c004810 0x7f5c1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request con 0x7f5c1c004810 auth_method 0 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c0053d0 0x7f5c1c00b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 10 monclient(hunting): get_auth_request con 0x7f5c1c003a60 auth_method 0 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request con 0x7f5c1c0053d0 auth_method 0 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.483+0000 7f5c3259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c3259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_done global_id 84406 payload 293 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 msgr2=0x7f5c1c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c1c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c0053d0 msgr2=0x7f5c1c00b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c0053d0 0x7f5c1c00b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 1 monclient: found mon.a 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 1 -- 172.21.15.17:0/3501843762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c003530 con 0x7f5c1c004810 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c31d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c38d03640 5 monclient: authenticate success, global_id 84406 2024-09-15T21:25:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c38d03640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.487+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c0053d0 0x7f5c1c00b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.491+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c004810 0x7f5c1c004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c28014370 tx=0x7f5c2801d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905dbd98b3203735 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.506+0000 7f81d1ffb640 1 -- 172.21.15.119:0/3509438761 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 device 'h7' does not appear in the crush map v225) ==== 118+0+0 (secure 0 0 0) 0x7f81dc018ac0 con 0x7f81e00a09e0 2024-09-15T21:25:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.506+0000 7f81d1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T21:25:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.506+0000 7f81d1ffb640 10 monclient: _finish_command 3 = system:0 device 'h7' does not appear in the crush map 2024-09-15T21:25:07.511 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h7' does not appear in the crush map 2024-09-15T21:25:07.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280af020 con 0x7f5c1c004810 2024-09-15T21:25:07.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:07.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: dump: 2024-09-15T21:25:07.515 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.515 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.515 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.516 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.516 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c280a7680 con 0x7f5c1c004810 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 1 -- 172.21.15.17:0/3501843762 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800ecd0 con 0x7f5c1c004810 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 4 set_mon_vals no callback set 2024-09-15T21:25:07.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c3159a640 10 monclient: dump: 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.518 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81c403b9f0 msgr2=0x7f81c403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81c403b9f0 0x7f81c403deb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81d40015f0 tx=0x7f81d4007a50 comp rx=0 tx=0).stop 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 10 monclient: shutdown 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 msgr2=0x7f81e011fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 0x7f81e011fd50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c7c0 tx=0x7f81dc00cc80 comp rx=0 tx=0).stop 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 shutdown_connections 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81c403b9f0 0x7f81c403deb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81e0137bc0 0x7f81e0136bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81e00a0ea0 0x7f81e011dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 --2- 172.21.15.119:0/3509438761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e00a09e0 0x7f81e011fd50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 >> 172.21.15.119:0/3509438761 conn(0x7f81e00a2ae0 msgr2=0x7f81e009d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:07.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 shutdown_connections 2024-09-15T21:25:07.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.510+0000 7f81e77ab640 1 -- 172.21.15.119:0/3509438761 wait complete. 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 10 monclient: shutdown 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 -- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c004810 msgr2=0x7f5c1c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c004810 0x7f5c1c004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c28014370 tx=0x7f5c2801d850 comp rx=0 tx=0).stop 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 -- 172.21.15.17:0/3501843762 shutdown_connections 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c0053d0 0x7f5c1c00b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c004810 0x7f5c1c004c90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 --2- 172.21.15.17:0/3501843762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c1c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.507+0000 7f5c38d03640 1 -- 172.21.15.17:0/3501843762 >> 172.21.15.17:0/3501843762 conn(0x7f5c340aabb0 msgr2=0x7f5c340a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 -- 172.21.15.17:0/3501843762 shutdown_connections 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 -- 172.21.15.17:0/3501843762 wait complete. 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient: build_initial_monmap 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient: monmap: 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:07.527 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 Processor -- start 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 -- start start 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient: init 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c341cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): picked mon.noname-c con 0x7f5c1c003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c004810 0x7f5c34136d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): picked mon.noname-b con 0x7f5c1c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c0053d0 0x7f5c341372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): picked mon.noname-a con 0x7f5c1c0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3406f2d0 con 0x7f5c1c0053d0 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3406f150 con 0x7f5c1c004810 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c34138060 con 0x7f5c1c003a60 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c38d03640 10 monclient(hunting): authenticate will time out at 1430.910826s 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c341cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c341cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37010/0 (socket says 172.21.15.17:37010) 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c0053d0 0x7f5c341372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 1 -- 172.21.15.17:0/158201760 learned_addr learned my addr 172.21.15.17:0/158201760 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): get_auth_request con 0x7f5c1c003a60 auth_method 0 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request con 0x7f5c1c0053d0 auth_method 0 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c004810 0x7f5c34136d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request con 0x7f5c1c004810 auth_method 0 2024-09-15T21:25:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c31d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient(hunting): handle_auth_done global_id 94098 payload 824 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 -- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 msgr2=0x7f5c341cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c341cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 -- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c004810 msgr2=0x7f5c34136d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c004810 0x7f5c34136d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 monclient: found mon.noname-a 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 -- 172.21.15.17:0/158201760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c34123410 con 0x7f5c1c0053d0 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c3259c640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c341cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.511+0000 7f5c32d9d640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c0053d0 0x7f5c341372c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c340a2420 tx=0x7f5c2c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c01c070 con 0x7f5c1c0053d0 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: dump: 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c016ba0 con 0x7f5c1c0053d0 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 5 monclient: authenticate success, global_id 94098 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 monclient: _renew_subs 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c340a9130 con 0x7f5c1c0053d0 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c021460 con 0x7f5c1c0053d0 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 10 monclient: dump: 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c23fff640 4 set_mon_vals no callback set 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 monclient: _renew_subs 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c34123f70 con 0x7f5c1c0053d0 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5c2c015040 con 0x7f5c1c0053d0 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:07.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c340a1c90 con 0x7f5c1c0053d0 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c1403c2e0 0x7f5c1403e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 13514+0+0 (secure 0 0 0) 0x7f5c2c052dd0 con 0x7f5c1c0053d0 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c3259c640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c1403c2e0 0x7f5c1403e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c3259c640 10 monclient: get_auth_request con 0x7f5c1403c2e0 auth_method 0 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.515+0000 7f5c3259c640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c1403c2e0 0x7f5c1403e7a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c24007ca0 tx=0x7f5c24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.519+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5c2c026060 con 0x7f5c1c0053d0 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.519+0000 7f5c22ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.519+0000 7f5c22ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:07.536 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:07.537 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h8 2024-09-15T21:25:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient: get_monmap_and_config 2024-09-15T21:25:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient: build_initial_monmap 2024-09-15T21:25:07.658 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient: monmap: 2024-09-15T21:25:07.659 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:07.658186+0000 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:07.658186+0000 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:07.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 Processor -- start 2024-09-15T21:25:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 -- start start 2024-09-15T21:25:07.661 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient: init 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb820135ca0 0x7fb820136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): picked mon.noname-a con 0x7fb820135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb8200a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): picked mon.noname-b con 0x7fb8200a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb82006ec30 0x7fb8200a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.668 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): picked mon.noname-c con 0x7fb82006ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb8201365c0 con 0x7fb820135ca0 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb82009cf30 con 0x7fb82006ec30 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200a1ca0 con 0x7fb8200a0160 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb828417640 10 monclient(hunting): authenticate will time out at 1426.454755s 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb82006ec30 0x7fb8200a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb82006ec30 0x7fb8200a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38842/0 (socket says 172.21.15.119:38842) 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 -- 172.21.15.119:0/1740895711 learned_addr learned my addr 172.21.15.119:0/1740895711 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:07.669 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): get_auth_request con 0x7fb82006ec30 auth_method 0 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8261b3640 1 --2- 172.21.15.119:0/1740895711 >> v2:172.21.15.17:3300/0 conn(0x7fb820135ca0 0x7fb820136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 1 --2- 172.21.15.119:0/1740895711 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb8200a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8261b3640 10 monclient(hunting): get_auth_request con 0x7fb820135ca0 auth_method 0 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8261b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8261b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8261b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): get_auth_request con 0x7fb8200a0160 auth_method 0 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8259b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient(hunting): handle_auth_done global_id 84412 payload 274 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 -- 172.21.15.119:0/1740895711 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 msgr2=0x7fb8200a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 --2- 172.21.15.119:0/1740895711 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb8200a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 -- 172.21.15.119:0/1740895711 >> v2:172.21.15.17:3300/0 conn(0x7fb820135ca0 msgr2=0x7fb820136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 --2- 172.21.15.119:0/1740895711 >> v2:172.21.15.17:3300/0 conn(0x7fb820135ca0 0x7fb820136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 monclient: found mon.noname-c 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8269b4640 1 -- 172.21.15.119:0/1740895711 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8200a1f80 con 0x7fb82006ec30 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.654+0000 7fb8261b3640 1 --2- 172.21.15.119:0/1740895711 >> v2:172.21.15.17:3300/0 conn(0x7fb820135ca0 0x7fb820136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8269b4640 1 --2- 172.21.15.119:0/1740895711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb8200a0f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8200a29c0 tx=0x7fb81801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68c506871b87654 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 1 -- 172.21.15.119:0/1740895711 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8180a5530 con 0x7fb82006ec30 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: dump: 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.672 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 1 -- 172.21.15.119:0/1740895711 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8180a56d0 con 0x7fb82006ec30 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:07.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 5 monclient: authenticate success, global_id 84412 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 1 -- 172.21.15.119:0/1740895711 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8180a5aa0 con 0x7fb82006ec30 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb8251b1640 10 monclient: dump: 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.674 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 4 set_mon_vals no callback set 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 10 monclient: shutdown 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 -- 172.21.15.119:0/1740895711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 msgr2=0x7fb8200a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 --2- 172.21.15.119:0/1740895711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb8200a0f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8200a29c0 tx=0x7fb81801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 -- 172.21.15.119:0/1740895711 shutdown_connections 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 --2- 172.21.15.119:0/1740895711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb8200a0f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 --2- 172.21.15.119:0/1740895711 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb8200a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 --2- 172.21.15.119:0/1740895711 >> v2:172.21.15.17:3300/0 conn(0x7fb820135ca0 0x7fb820136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 -- 172.21.15.119:0/1740895711 >> 172.21.15.119:0/1740895711 conn(0x7fb8200a2ae0 msgr2=0x7fb8200a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:07.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 -- 172.21.15.119:0/1740895711 shutdown_connections 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 1 -- 172.21.15.119:0/1740895711 wait complete. 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 10 monclient: build_initial_monmap 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.658+0000 7fb828417640 10 monclient: monmap: 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:07.676 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 Processor -- start 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- start start 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: init 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb82013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): picked mon.noname-b con 0x7fb82006ec30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb820138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): picked mon.noname-c con 0x7fb8200a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb820135ca0 0x7fb820130920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): picked mon.noname-a con 0x7fb820135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.677 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb82013f230 con 0x7fb820135ca0 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200a1870 con 0x7fb82006ec30 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb82013f3b0 con 0x7fb8200a0160 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient(hunting): authenticate will time out at 1426.460967s 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb82013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb82013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38852/0 (socket says 172.21.15.119:38852) 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 -- 172.21.15.119:0/2853724868 learned_addr learned my addr 172.21.15.119:0/2853724868 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): get_auth_request con 0x7fb82006ec30 auth_method 0 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.678 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb820138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8269b4640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb820135ca0 0x7fb820130920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 10 monclient(hunting): get_auth_request con 0x7fb8200a0160 auth_method 0 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8269b4640 10 monclient(hunting): get_auth_request con 0x7fb820135ca0 auth_method 0 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8269b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8269b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.679 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8269b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient(hunting): handle_auth_done global_id 84415 payload 824 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 -- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 msgr2=0x7fb820138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb820138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 -- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb820135ca0 msgr2=0x7fb820130920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb820135ca0 0x7fb820130920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 monclient: found mon.noname-b 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 -- 172.21.15.119:0/2853724868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82009d1d0 con 0x7fb82006ec30 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8261b3640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb82013ae50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb81400e810 tx=0x7fb81400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814017070 con 0x7fb82006ec30 2024-09-15T21:25:07.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: dump: 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.681 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81400c8d0 con 0x7fb82006ec30 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81400f040 con 0x7fb82006ec30 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 10 monclient: dump: 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.682 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 5 monclient: authenticate success, global_id 84415 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _renew_subs 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8249b0640 4 set_mon_vals no callback set 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb820134ce0 con 0x7fb82006ec30 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _renew_subs 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:07.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82013c650 con 0x7fb82006ec30 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb820136080 con 0x7fb82006ec30 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb81401a070 con 0x7fb82006ec30 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb80403b9c0 0x7fb80403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 13514+0+0 (secure 0 0 0) 0x7fb814051b10 con 0x7fb82006ec30 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb80403b9c0 0x7fb80403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 10 monclient: get_auth_request con 0x7fb80403b9c0 auth_method 0 2024-09-15T21:25:07.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.662+0000 7fb8259b2640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb80403b9c0 0x7fb80403de80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb82013c140 tx=0x7fb81c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.666+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb814024350 con 0x7fb82006ec30 2024-09-15T21:25:07.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.666+0000 7fb812ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.666+0000 7fb812ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c38d03640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c38d03640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c38d03640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5c34069200 con 0x7f5c1c0053d0 2024-09-15T21:25:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c22ffd640 1 -- 172.21.15.17:0/158201760 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f5c2c04f560 con 0x7f5c1c0053d0 2024-09-15T21:25:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c22ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.759+0000 7f5c22ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:07.767 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c1403c2e0 msgr2=0x7f5c1403e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c1403c2e0 0x7f5c1403e7a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c24007ca0 tx=0x7f5c24009040 comp rx=0 tx=0).stop 2024-09-15T21:25:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 10 monclient: shutdown 2024-09-15T21:25:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c0053d0 msgr2=0x7f5c341372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c0053d0 0x7f5c341372c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c340a2420 tx=0x7f5c2c009f10 comp rx=0 tx=0).stop 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 shutdown_connections 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c1403c2e0 0x7f5c1403e7a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c0053d0 0x7f5c341372c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c1c004810 0x7f5c34136d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 --2- 172.21.15.17:0/158201760 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c1c003a60 0x7f5c341cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 >> 172.21.15.17:0/158201760 conn(0x7f5c340aabb0 msgr2=0x7f5c341c1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 shutdown_connections 2024-09-15T21:25:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.763+0000 7f5c38d03640 1 -- 172.21.15.17:0/158201760 wait complete. 2024-09-15T21:25:07.786 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:25:07.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:25:07.787 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:25:07.848 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.846+0000 7fb828417640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T21:25:07.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.846+0000 7fb828417640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T21:25:07.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.846+0000 7fb828417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:07.849 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:07.846+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) -- 0x7fb82006c460 con 0x7fb82006ec30 2024-09-15T21:25:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient: get_monmap_and_config 2024-09-15T21:25:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient: build_initial_monmap 2024-09-15T21:25:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient: monmap: 2024-09-15T21:25:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.912 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:07.911627+0000 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:07.911627+0000 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:07.913 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 Processor -- start 2024-09-15T21:25:07.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 -- start start 2024-09-15T21:25:07.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient: init 2024-09-15T21:25:07.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:07.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:25:07.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 0x7fbfe40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): picked mon.noname-b con 0x7fbfe40a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbfe409ffc0 0x7fbfe40a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): picked mon.noname-c con 0x7fbfe409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbfe4135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe40a3130 con 0x7fbfe4135670 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe40a16f0 con 0x7fbfe409ffc0 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe4073df0 con 0x7fbfe40a0890 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe9213640 10 monclient(hunting): authenticate will time out at 1431.307155s 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 0x7fbfe40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 0x7fbfe40a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39042/0 (socket says 172.21.15.17:39042) 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 -- 172.21.15.17:0/2254924733 learned_addr learned my addr 172.21.15.17:0/2254924733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): get_auth_request con 0x7fbfe40a0890 auth_method 0 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe259c640 1 --2- 172.21.15.17:0/2254924733 >> v2:172.21.15.119:3300/0 conn(0x7fbfe409ffc0 0x7fbfe40a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_done global_id 45077 payload 274 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 -- 172.21.15.17:0/2254924733 >> v2:172.21.15.119:3300/0 conn(0x7fbfe409ffc0 msgr2=0x7fbfe40a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/2254924733 >> v2:172.21.15.119:3300/0 conn(0x7fbfe409ffc0 0x7fbfe40a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 -- 172.21.15.17:0/2254924733 >> v1:172.21.15.119:6789/0 conn(0x7fbfe4135670 legacy=0x7fbfe4135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:07.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 monclient: found mon.noname-b 2024-09-15T21:25:07.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:07.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe2d9d640 1 -- 172.21.15.17:0/2254924733 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe40a1bf0 con 0x7fbfe40a0890 2024-09-15T21:25:07.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe259c640 10 monclient: get_auth_request con 0x7fbfe409ffc0 auth_method 0 2024-09-15T21:25:07.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.907+0000 7fbfe259c640 1 --2- 172.21.15.17:0/2254924733 >> v2:172.21.15.119:3300/0 conn(0x7fbfe409ffc0 0x7fbfe40a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:07.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/2254924733 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 0x7fbfe40a0c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40a1870 tx=0x7fbfd401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8acd81cd48d5c62d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 1 -- 172.21.15.17:0/2254924733 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40a5620 con 0x7fbfe40a0890 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: dump: 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 5 monclient: authenticate success, global_id 45077 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 1 -- 172.21.15.17:0/2254924733 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd40a57c0 con 0x7fbfe40a0890 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 1 -- 172.21.15.17:0/2254924733 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40a5b90 con 0x7fbfe40a0890 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe1d9b640 10 monclient: dump: 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 4 set_mon_vals no callback set 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 10 monclient: shutdown 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 -- 172.21.15.17:0/2254924733 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 msgr2=0x7fbfe40a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 --2- 172.21.15.17:0/2254924733 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 0x7fbfe40a0c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40a1870 tx=0x7fbfd401c8d0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 -- 172.21.15.17:0/2254924733 shutdown_connections 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 --2- 172.21.15.17:0/2254924733 >> v2:172.21.15.119:3300/0 conn(0x7fbfe409ffc0 0x7fbfe40a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 --2- 172.21.15.17:0/2254924733 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe40a0890 0x7fbfe40a0c70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 -- 172.21.15.17:0/2254924733 >> 172.21.15.17:0/2254924733 conn(0x7fbfe40a66a0 msgr2=0x7fbfe40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 -- 172.21.15.17:0/2254924733 shutdown_connections 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 -- 172.21.15.17:0/2254924733 wait complete. 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 10 monclient: build_initial_monmap 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 10 monclient: monmap: 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:07.928 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 Processor -- start 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 1 -- start start 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.911+0000 7fbfe9213640 10 monclient: init 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe409ffc0 0x7fbfe4133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): picked mon.noname-b con 0x7fbfe409ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40a0890 0x7fbfe41344f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): picked mon.noname-a con 0x7fbfe40a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe4135670 0x7fbfe411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): picked mon.noname-c con 0x7fbfe4135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4140690 con 0x7fbfe40a0890 2024-09-15T21:25:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4140510 con 0x7fbfe409ffc0 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe4140810 con 0x7fbfe4135670 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): authenticate will time out at 1431.312420s 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe4135670 0x7fbfe411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40a0890 0x7fbfe41344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40a0890 0x7fbfe41344f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56672/0 (socket says 172.21.15.17:56672) 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 1 -- 172.21.15.17:0/948574339 learned_addr learned my addr 172.21.15.17:0/948574339 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): get_auth_request con 0x7fbfe40a0890 auth_method 0 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe4135670 0x7fbfe411c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39058/0 (socket says 172.21.15.17:39058) 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): get_auth_request con 0x7fbfe4135670 auth_method 0 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe409ffc0 0x7fbfe4133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): get_auth_request con 0x7fbfe409ffc0 auth_method 0 2024-09-15T21:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient(hunting): handle_auth_done global_id 45080 payload 824 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe409ffc0 msgr2=0x7fbfe4133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe409ffc0 0x7fbfe4133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40a0890 msgr2=0x7fbfe41344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40a0890 0x7fbfe41344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient: _finish_hunting 0 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 monclient: found mon.noname-c 2024-09-15T21:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe411d870 con 0x7fbfe4135670 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40a0890 0x7fbfe41344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe359e640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe4135670 0x7fbfe411c4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00c440 tx=0x7fbfdc00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc011790 con 0x7fbfe4135670 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: dump: 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfdc017070 con 0x7fbfe4135670 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc011b70 con 0x7fbfe4135670 2024-09-15T21:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 5 monclient: authenticate success, global_id 45080 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient: _renew_subs 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe411eda0 con 0x7fbfe4135670 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: dump: 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe0d99640 4 set_mon_vals no callback set 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient: _renew_subs 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe413cfd0 con 0x7fbfe4135670 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 0x7fbfe411ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4135aa0 con 0x7fbfe411ca20 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 0x7fbfe411ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient: get_auth_request con 0x7fbfe411ca20 auth_method 0 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbfdc011d10 con 0x7fbfe4135670 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_done global_id 94110 payload 824 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbfe411d340 con 0x7fbfe411ca20 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfd003b9f0 0x7fbfd003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(225..225 src has 1..225) ==== 13514+0+0 (secure 0 0 0) 0x7fbfdc0519f0 con 0x7fbfe4135670 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 0x7fbfe411ce00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbfe406f1c0 tx=0x7fbfd4019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40c0640 con 0x7fbfe411ca20 2024-09-15T21:25:07.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: dump: 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfd003b9f0 0x7fbfd003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 10 monclient: get_auth_request con 0x7fbfd003b9f0 auth_method 0 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbfd40c0640 con 0x7fbfe411ca20 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 msgr2=0x7fbfe411ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 0x7fbfe411ce00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbfe406f1c0 tx=0x7fbfd4019670 comp rx=0 tx=0).stop 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 msgr2=0x7fbfe411ce00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfcb7fe640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe411ca20 0x7fbfe411ce00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.915+0000 7fbfe259c640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfd003b9f0 0x7fbfd003deb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4135640 tx=0x7fbfd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 0x7fbfe4137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe411d5a0 con 0x7fbfe4137ab0 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 0x7fbfe4137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient: get_auth_request con 0x7fbfe4137ab0 auth_method 0 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 10 monclient(hunting): handle_auth_done global_id 94113 payload 824 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 1 -- 172.21.15.17:0/948574339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbfe406d020 con 0x7fbfe4137ab0 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe2d9d640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 0x7fbfe4137ec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbfd401c910 tx=0x7fbfd40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4004040 con 0x7fbfe4137ab0 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 10 monclient: dump: 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.942 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7fbfd4003ee0 con 0x7fbfe4137ab0 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 msgr2=0x7fbfe4137ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 0x7fbfe4137ec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbfd401c910 tx=0x7fbfd40a7000 comp rx=0 tx=0).stop 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 msgr2=0x7fbfe4137ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfcb7fe640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe4137ab0 0x7fbfe4137ec0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbfd401c910 tx=0x7fbfd40a7000 comp rx=0 tx=0).stop 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 76, 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 13107, 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:07.954 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:07.955 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:07.956 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfd003b9f0 msgr2=0x7fbfd003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfd003b9f0 0x7fbfd003deb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4135640 tx=0x7fbfd8008040 comp rx=0 tx=0).stop 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 10 monclient: shutdown 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe4135670 msgr2=0x7fbfe411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 --2- 172.21.15.17:0/948574339 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfe4135670 0x7fbfe411c4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00c440 tx=0x7fbfdc00c900 comp rx=0 tx=0).stop 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe359e640 1 -- 172.21.15.17:0/948574339 reap_dead start 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe359e640 1 -- 172.21.15.17:0/948574339 reap_dead start 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 shutdown_connections 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 >> 172.21.15.17:0/948574339 conn(0x7fbfe40a66a0 msgr2=0x7fbfe40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 shutdown_connections 2024-09-15T21:25:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:07.923+0000 7fbfe9213640 1 -- 172.21.15.17:0/948574339 wait complete. 2024-09-15T21:25:07.959 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:07.959 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:25:07.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:25:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient: get_monmap_and_config 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient: monmap: 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:08.075139+0000 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:08.075139+0000 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:08.077 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 Processor -- start 2024-09-15T21:25:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 -- start start 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient: init 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6b009d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6b009def0 0x7fb6b0139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): picked mon.noname-a con 0x7fb6b009def0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b013c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): picked mon.noname-b con 0x7fb6b0139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b00a3130 con 0x7fb6b009d400 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b00a1bc0 con 0x7fb6b009def0 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b00a3490 con 0x7fb6b0139ff0 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b6dc8640 10 monclient(hunting): authenticate will time out at 1431.470983s 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b013c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b013c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39064/0 (socket says 172.21.15.17:39064) 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6b009def0 0x7fb6b0139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 -- 172.21.15.17:0/909940701 learned_addr learned my addr 172.21.15.17:0/909940701 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): get_auth_request con 0x7fb6b0139ff0 auth_method 0 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): get_auth_request con 0x7fb6b009def0 auth_method 0 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient(hunting): handle_auth_done global_id 45083 payload 274 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 -- 172.21.15.17:0/909940701 >> v2:172.21.15.17:3300/0 conn(0x7fb6b009def0 msgr2=0x7fb6b0139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 --2- 172.21.15.17:0/909940701 >> v2:172.21.15.17:3300/0 conn(0x7fb6b009def0 0x7fb6b0139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 -- 172.21.15.17:0/909940701 >> v1:172.21.15.119:6789/0 conn(0x7fb6b009d400 legacy=0x7fb6b009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 monclient: found mon.noname-b 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.071+0000 7fb6b5365640 1 -- 172.21.15.17:0/909940701 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b009e7a0 con 0x7fb6b0139ff0 2024-09-15T21:25:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b5365640 1 --2- 172.21.15.17:0/909940701 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b013c3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009e50 tx=0x7fb6a801c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59070f52591715f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 1 -- 172.21.15.17:0/909940701 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80a55b0 con 0x7fb6b0139ff0 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: dump: 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 1 -- 172.21.15.17:0/909940701 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a80a5750 con 0x7fb6b0139ff0 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 1 -- 172.21.15.17:0/909940701 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80a5b20 con 0x7fb6b0139ff0 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6af7fe640 10 monclient: dump: 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 5 monclient: authenticate success, global_id 45083 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 4 set_mon_vals no callback set 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 10 monclient: shutdown 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/909940701 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 msgr2=0x7fb6b013c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/909940701 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b013c3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009e50 tx=0x7fb6a801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/909940701 shutdown_connections 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/909940701 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b013c3e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/909940701 >> v2:172.21.15.17:3300/0 conn(0x7fb6b009def0 0x7fb6b0139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/909940701 >> 172.21.15.17:0/909940701 conn(0x7fb6b00a66a0 msgr2=0x7fb6b00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/909940701 shutdown_connections 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/909940701 wait complete. 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.075+0000 7fb6b6dc8640 10 monclient: monmap: 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 Processor -- start 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- start start 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: init 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b009d400 0x7fb6b01cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): picked mon.noname-a con 0x7fb6b009d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b009def0 0x7fb6b0134900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): picked mon.noname-b con 0x7fb6b009def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b0134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): picked mon.noname-c con 0x7fb6b0139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b013f140 con 0x7fb6b009d400 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b013f2c0 con 0x7fb6b009def0 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b013f440 con 0x7fb6b0139ff0 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): authenticate will time out at 1431.476631s 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b009d400 0x7fb6b01cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b0134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b009d400 0x7fb6b01cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56692/0 (socket says 172.21.15.17:56692) 2024-09-15T21:25:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 -- 172.21.15.17:0/3062681 learned_addr learned my addr 172.21.15.17:0/3062681 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): get_auth_request con 0x7fb6b009d400 auth_method 0 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): get_auth_request con 0x7fb6b0139ff0 auth_method 0 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b009def0 0x7fb6b0134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): get_auth_request con 0x7fb6b009def0 auth_method 0 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient(hunting): handle_auth_done global_id 94119 payload 824 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 msgr2=0x7fb6b0134e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b0134e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b009def0 msgr2=0x7fb6b0134900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b009def0 0x7fb6b0134900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 monclient: found mon.noname-a 2024-09-15T21:25:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b01206b0 con 0x7fb6b009d400 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b0139ff0 0x7fb6b0134e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b4b64640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b009d400 0x7fb6b01cf700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c500 tx=0x7fb6a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0017070 con 0x7fb6b009d400 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: dump: 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00052d0 con 0x7fb6b009d400 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0012870 con 0x7fb6b009d400 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: dump: 2024-09-15T21:25:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6aeffd640 4 set_mon_vals no callback set 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 5 monclient: authenticate success, global_id 94119 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: _renew_subs 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b011ff60 con 0x7fb6b009d400 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: _renew_subs 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b0121fe0 con 0x7fb6b009d400 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 0x7fb6b011e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b009fc10 con 0x7fb6b011e4f0 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 0x7fb6b011e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient: get_auth_request con 0x7fb6b011e4f0 auth_method 0 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb6a001a070 con 0x7fb6b009d400 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 10 monclient(hunting): handle_auth_done global_id 45089 payload 824 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6b009d7e0 con 0x7fb6b011e4f0 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb69c03b9f0 0x7fb69c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 13514+0+0 (secure 0 0 0) 0x7fb6a0050ff0 con 0x7fb6b009d400 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6affff640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 0x7fb6b011e8d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40115d0 tx=0x7fb6a4011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4004230 con 0x7fb6b011e4f0 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6ad7fa640 10 monclient: dump: 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb69c03b9f0 0x7fb69c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.079+0000 7fb6b5365640 10 monclient: get_auth_request con 0x7fb69c03b9f0 auth_method 0 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb6a4004230 con 0x7fb6b011e4f0 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 msgr2=0x7fb6b011e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 0x7fb6b011e8d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40115d0 tx=0x7fb6a4011a90 comp rx=0 tx=0).stop 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 msgr2=0x7fb6b011e8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6ad7fa640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011e4f0 0x7fb6b011e8d0 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40115d0 tx=0x7fb6a4011a90 comp rx=0 tx=0).stop 2024-09-15T21:25:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.083+0000 7fb6b5365640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb69c03b9f0 0x7fb69c03deb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80ae000 tx=0x7fb6a80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 0x7fb6b0137aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b01cfc40 con 0x7fb6b011ee10 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 0x7fb6b0137aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient: get_auth_request con 0x7fb6b011ee10 auth_method 0 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 10 monclient(hunting): handle_auth_done global_id 45092 payload 824 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 1 -- 172.21.15.17:0/3062681 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6b006d020 con 0x7fb6b011ee10 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6affff640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 0x7fb6b0137aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40115d0 tx=0x7fb6a40046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4003e00 con 0x7fb6b011ee10 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 10 monclient: dump: 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7fb6a4003e00 con 0x7fb6b011ee10 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 msgr2=0x7fb6b0137aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 0x7fb6b0137aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40115d0 tx=0x7fb6a40046d0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 msgr2=0x7fb6b0137aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6ad7fa640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6b011ee10 0x7fb6b0137aa0 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40115d0 tx=0x7fb6a40046d0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 76, 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 276685, 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:08.109 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:08.110 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:08.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:08.112 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:08.113 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.114 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.115 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:08.117 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.119 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.120 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.121 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.122 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:08.122 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:08.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb69c03b9f0 msgr2=0x7fb69c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb69c03b9f0 0x7fb69c03deb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80ae000 tx=0x7fb6a80a7070 comp rx=0 tx=0).stop 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 10 monclient: shutdown 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b009d400 msgr2=0x7fb6b01cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 --2- 172.21.15.17:0/3062681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b009d400 0x7fb6b01cf700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c500 tx=0x7fb6a000c9c0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b5365640 1 -- 172.21.15.17:0/3062681 reap_dead start 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b5365640 1 -- 172.21.15.17:0/3062681 reap_dead start 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 shutdown_connections 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 >> 172.21.15.17:0/3062681 conn(0x7fb6b00a66a0 msgr2=0x7fb6b0073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 shutdown_connections 2024-09-15T21:25:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.087+0000 7fb6b6dc8640 1 -- 172.21.15.17:0/3062681 wait complete. 2024-09-15T21:25:08.125 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:08.125 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:25:08.125 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:25:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient: get_monmap_and_config 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient: monmap: 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:08.243164+0000 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:08.243164+0000 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:08.245 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 Processor -- start 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 -- start start 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient: init 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa041378c0 0x7faa04137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): picked mon.noname-a con 0x7faa041378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faa041381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa040a5210 0x7faa040a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): picked mon.noname-c con 0x7faa040a5210 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa040a1d00 con 0x7faa041381e0 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faa040a1fe0 con 0x7faa041378c0 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faa040a22c0 con 0x7faa040a5210 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0c5d6640 10 monclient(hunting): authenticate will time out at 1431.638686s 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa041378c0 0x7faa04137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa041378c0 0x7faa04137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56704/0 (socket says 172.21.15.17:56704) 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 -- 172.21.15.17:0/2321235538 learned_addr learned my addr 172.21.15.17:0/2321235538 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): get_auth_request con 0x7faa041378c0 auth_method 0 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0ab73640 1 --2- 172.21.15.17:0/2321235538 >> v2:172.21.15.119:3300/0 conn(0x7faa040a5210 0x7faa040a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0ab73640 10 monclient(hunting): get_auth_request con 0x7faa040a5210 auth_method 0 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0ab73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0ab73640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0ab73640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient(hunting): handle_auth_done global_id 94122 payload 274 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 -- 172.21.15.17:0/2321235538 >> v2:172.21.15.119:3300/0 conn(0x7faa040a5210 msgr2=0x7faa040a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 --2- 172.21.15.17:0/2321235538 >> v2:172.21.15.119:3300/0 conn(0x7faa040a5210 0x7faa040a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 -- 172.21.15.17:0/2321235538 >> v1:172.21.15.119:6789/0 conn(0x7faa041381e0 legacy=0x7faa040a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 monclient: found mon.noname-a 2024-09-15T21:25:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.239+0000 7faa0a372640 1 -- 172.21.15.17:0/2321235538 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa040a4490 con 0x7faa041378c0 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0a372640 1 --2- 172.21.15.17:0/2321235538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04137ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa0001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c936e73342366da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 1 -- 172.21.15.17:0/2321235538 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000b1070 con 0x7faa041378c0 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: dump: 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 1 -- 172.21.15.17:0/2321235538 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa000ac620 con 0x7faa041378c0 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 5 monclient: authenticate success, global_id 94122 2024-09-15T21:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 4 set_mon_vals no callback set 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 1 -- 172.21.15.17:0/2321235538 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000ac9f0 con 0x7faa041378c0 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa09370640 10 monclient: dump: 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.259 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 10 monclient: shutdown 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 -- 172.21.15.17:0/2321235538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 msgr2=0x7faa04137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/2321235538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04137ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa0001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 -- 172.21.15.17:0/2321235538 shutdown_connections 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/2321235538 >> v2:172.21.15.119:3300/0 conn(0x7faa040a5210 0x7faa040a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/2321235538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04137ca0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 -- 172.21.15.17:0/2321235538 >> 172.21.15.17:0/2321235538 conn(0x7faa040aac40 msgr2=0x7faa040a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 -- 172.21.15.17:0/2321235538 shutdown_connections 2024-09-15T21:25:08.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 -- 172.21.15.17:0/2321235538 wait complete. 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 10 monclient: monmap: 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 Processor -- start 2024-09-15T21:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 1 -- start start 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.243+0000 7faa0c5d6640 10 monclient: init 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa040a5210 0x7faa041cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): picked mon.noname-b con 0x7faa040a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): picked mon.noname-a con 0x7faa041378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faa041381e0 0x7faa0413bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): picked mon.noname-c con 0x7faa041381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faa0406f340 con 0x7faa041378c0 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa0406f1c0 con 0x7faa040a5210 2024-09-15T21:25:08.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faa041425c0 con 0x7faa041381e0 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): authenticate will time out at 1431.644453s 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faa041381e0 0x7faa0413bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04136c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56710/0 (socket says 172.21.15.17:56710) 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faa041381e0 0x7faa0413bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39096/0 (socket says 172.21.15.17:39096) 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 -- 172.21.15.17:0/1927744273 learned_addr learned my addr 172.21.15.17:0/1927744273 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): get_auth_request con 0x7faa041378c0 auth_method 0 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): get_auth_request con 0x7faa041381e0 auth_method 0 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa040a5210 0x7faa041cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): get_auth_request con 0x7faa040a5210 auth_method 0 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient(hunting): handle_auth_done global_id 94125 payload 824 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faa041381e0 msgr2=0x7faa0413bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faa041381e0 0x7faa0413bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa040a5210 msgr2=0x7faa041cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa040a5210 0x7faa041cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 monclient: found mon.noname-a 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0413e480 con 0x7faa041378c0 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faa041381e0 0x7faa0413bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa040a5210 0x7faa041cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa09b71640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04136c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c450 tx=0x7fa9f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4011790 con 0x7faa041378c0 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: dump: 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.266 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4017070 con 0x7faa041378c0 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4011b90 con 0x7faa041378c0 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 5 monclient: authenticate success, global_id 94125 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient: _renew_subs 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa040a12f0 con 0x7faa041378c0 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa08b6f640 4 set_mon_vals no callback set 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:08.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: dump: 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient: _renew_subs 2024-09-15T21:25:08.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa04121610 con 0x7faa041378c0 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 0x7faa0402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa040a56a0 con 0x7faa0402d450 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 0x7faa0402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient: get_auth_request con 0x7faa0402d450 auth_method 0 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa9f4011d30 con 0x7faa041378c0 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 10 monclient(hunting): handle_auth_done global_id 84433 payload 824 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa0409faf0 con 0x7faa0402d450 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9e803b9f0 0x7fa9e803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 13514+0+0 (secure 0 0 0) 0x7fa9f4051b50 con 0x7faa041378c0 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0a372640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 0x7faa0402d830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa040a6150 tx=0x7faa00019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9e803b9f0 0x7fa9e803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7faa0ab73640 10 monclient: get_auth_request con 0x7fa9e803b9f0 auth_method 0 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000bc070 con 0x7faa0402d450 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.247+0000 7fa9faffd640 10 monclient: dump: 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7faa0ab73640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9e803b9f0 0x7fa9e803deb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0077c0 tx=0x7fa9fc009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faa000acd20 con 0x7faa0402d450 2024-09-15T21:25:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 msgr2=0x7faa0402d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 0x7faa0402d830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa040a6150 tx=0x7faa00019180 comp rx=0 tx=0).stop 2024-09-15T21:25:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 msgr2=0x7faa0402d830 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.251+0000 7fa9faffd640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402d450 0x7faa0402d830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0c5d6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 0x7faa0402e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0c5d6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa04069270 con 0x7faa0402dd70 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 0x7faa0402e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient: get_auth_request con 0x7faa0402dd70 auth_method 0 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 10 monclient(hunting): handle_auth_done global_id 84436 payload 824 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.255+0000 7faa0a372640 1 -- 172.21.15.17:0/1927744273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faa0413ccf0 con 0x7faa0402dd70 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0a372640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 0x7faa0402e180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa00008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000b1040 con 0x7faa0402dd70 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 10 monclient: dump: 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7faa000b6900 con 0x7faa0402dd70 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 msgr2=0x7faa0402e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 0x7faa0402e180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa00008040 comp rx=0 tx=0).stop 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 msgr2=0x7faa0402e180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7fa9faffd640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa0402dd70 0x7faa0402e180 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa00008040 comp rx=0 tx=0).stop 2024-09-15T21:25:08.276 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 76, 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 60487, 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:08.287 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:08.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:08.289 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:08.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9e803b9f0 msgr2=0x7fa9e803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9e803b9f0 0x7fa9e803deb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0077c0 tx=0x7fa9fc009820 comp rx=0 tx=0).stop 2024-09-15T21:25:08.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 10 monclient: shutdown 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 msgr2=0x7faa04136c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 --2- 172.21.15.17:0/1927744273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa041378c0 0x7faa04136c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c450 tx=0x7fa9f400c910 comp rx=0 tx=0).stop 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0ab73640 1 -- 172.21.15.17:0/1927744273 reap_dead start 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0ab73640 1 -- 172.21.15.17:0/1927744273 reap_dead start 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 shutdown_connections 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 >> 172.21.15.17:0/1927744273 conn(0x7faa040aac40 msgr2=0x7faa0413efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 shutdown_connections 2024-09-15T21:25:08.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.259+0000 7faa0c5d6640 1 -- 172.21.15.17:0/1927744273 wait complete. 2024-09-15T21:25:08.291 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:08.291 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:25:08.292 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:25:08.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient: get_monmap_and_config 2024-09-15T21:25:08.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient: monmap: 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:08.403883+0000 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:08.403883+0000 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:08.406 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:08.406 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:08.406 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 Processor -- start 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 -- start start 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient: init 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f65541378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): picked mon.noname-b con 0x7f65541383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 0x7f655413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): picked mon.noname-a con 0x7f655413d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65540a1a20 con 0x7f65541378c0 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f65540a1d00 con 0x7f655413d2c0 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f65540a1fe0 con 0x7f65541383b0 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f6559597640 10 monclient(hunting): authenticate will time out at 1431.799276s 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 0x7f655413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 0x7f655413f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56722/0 (socket says 172.21.15.17:56722) 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 1 -- 172.21.15.17:0/749472902 learned_addr learned my addr 172.21.15.17:0/749472902 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): get_auth_request con 0x7f655413d2c0 auth_method 0 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 1 --2- 172.21.15.17:0/749472902 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): get_auth_request con 0x7f65541383b0 auth_method 0 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient(hunting): handle_auth_done global_id 45098 payload 274 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 1 -- 172.21.15.17:0/749472902 >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 msgr2=0x7f655413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 1 --2- 172.21.15.17:0/749472902 >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 0x7f655413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 1 -- 172.21.15.17:0/749472902 >> v1:172.21.15.119:6789/0 conn(0x7f65541378c0 legacy=0x7f6554137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 1 monclient: found mon.noname-b 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65527fc640 1 -- 172.21.15.17:0/749472902 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65540a22c0 con 0x7f65541383b0 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.399+0000 7f65537fe640 1 --2- 172.21.15.17:0/749472902 >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 0x7f655413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f65527fc640 1 --2- 172.21.15.17:0/749472902 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655413cd80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6544009d90 tx=0x7f654401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2583987caf127344 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 1 -- 172.21.15.17:0/749472902 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440b1070 con 0x7f65541383b0 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: dump: 2024-09-15T21:25:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 1 -- 172.21.15.17:0/749472902 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65440a4dd0 con 0x7f65541383b0 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 1 -- 172.21.15.17:0/749472902 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440ac660 con 0x7f65541383b0 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6551ffb640 10 monclient: dump: 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.415 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 5 monclient: authenticate success, global_id 45098 2024-09-15T21:25:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 4 set_mon_vals no callback set 2024-09-15T21:25:08.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 10 monclient: shutdown 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 -- 172.21.15.17:0/749472902 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 msgr2=0x7f655413cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 --2- 172.21.15.17:0/749472902 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655413cd80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6544009d90 tx=0x7f654401c510 comp rx=0 tx=0).stop 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 -- 172.21.15.17:0/749472902 shutdown_connections 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 --2- 172.21.15.17:0/749472902 >> v2:172.21.15.17:3300/0 conn(0x7f655413d2c0 0x7f655413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 --2- 172.21.15.17:0/749472902 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655413cd80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 -- 172.21.15.17:0/749472902 >> 172.21.15.17:0/749472902 conn(0x7f65540aac40 msgr2=0x7f65540a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 -- 172.21.15.17:0/749472902 shutdown_connections 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 -- 172.21.15.17:0/749472902 wait complete. 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 10 monclient: monmap: 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.403+0000 7f6559597640 1 Processor -- start 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- start start 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: init 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 0x7f65540a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): picked mon.noname-a con 0x7f65541378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655409df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): picked mon.noname-c con 0x7f65541383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f655413d2c0 0x7f655409e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): picked mon.noname-b con 0x7f655413d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6554143040 con 0x7f65541378c0 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6554142ec0 con 0x7f655413d2c0 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f65541431c0 con 0x7f65541383b0 2024-09-15T21:25:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient(hunting): authenticate will time out at 1431.804528s 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 0x7f65540a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655409df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 0x7f65540a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56726/0 (socket says 172.21.15.17:56726) 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 -- 172.21.15.17:0/3895814416 learned_addr learned my addr 172.21.15.17:0/3895814416 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): get_auth_request con 0x7f65541378c0 auth_method 0 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): get_auth_request con 0x7f65541383b0 auth_method 0 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65537fe640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f655413d2c0 0x7f655409e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient(hunting): handle_auth_done global_id 94131 payload 824 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 -- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 msgr2=0x7f655409df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655409df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 -- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f655413d2c0 msgr2=0x7f655409e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f655413d2c0 0x7f655409e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 monclient: found mon.noname-a 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 -- 172.21.15.17:0/3895814416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65540a12f0 con 0x7f65541378c0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65537fe640 10 monclient: get_auth_request con 0x7f655413d2c0 auth_method 0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655409df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65537fe640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f655413d2c0 0x7f655409e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6552ffd640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 0x7f65540a0550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c4e0 tx=0x7f654c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0052d0 con 0x7f65541378c0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: dump: 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c005470 con 0x7f65541378c0 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c012850 con 0x7f65541378c0 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 5 monclient: authenticate success, global_id 94131 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _renew_subs 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65541219c0 con 0x7f65541378c0 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 10 monclient: dump: 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:08.426 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65517fa640 4 set_mon_vals no callback set 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _renew_subs 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541216e0 con 0x7f65541378c0 2024-09-15T21:25:08.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f654c0129f0 con 0x7f65541378c0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6554138190 con 0x7f65541378c0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f654003b9c0 0x7f654003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 13514+0+0 (secure 0 0 0) 0x7f654c051990 con 0x7f65541378c0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f654003b9c0 0x7f654003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 10 monclient: get_auth_request con 0x7f654003b9c0 auth_method 0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.407+0000 7f65527fc640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f654003b9c0 0x7f654003de80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f65440ae000 tx=0x7f65440a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.411+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f65540a12f0 con 0x7f65541378c0 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.411+0000 7f653b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.411+0000 7f653b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:08.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.514+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 removed item id -40 name 'h8' from crush map v226) ==== 118+0+0 (secure 0 0 0) 0x7fb81404e300 con 0x7fb82006ec30 2024-09-15T21:25:08.518 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.514+0000 7fb812ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T21:25:08.519 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.514+0000 7fb812ffd640 10 monclient: _finish_command 2 = system:0 removed item id -40 name 'h8' from crush map 2024-09-15T21:25:08.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb828417640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T21:25:08.624 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb828417640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T21:25:08.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb828417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:08.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) -- 0x7fb820130f80 con 0x7fb82006ec30 2024-09-15T21:25:08.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb812ffd640 1 -- 172.21.15.119:0/2853724868 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 device 'h8' does not appear in the crush map v226) ==== 118+0+0 (secure 0 0 0) 0x7fb8140234c0 con 0x7fb82006ec30 2024-09-15T21:25:08.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb812ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T21:25:08.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.622+0000 7fb812ffd640 10 monclient: _finish_command 3 = system:0 device 'h8' does not appear in the crush map 2024-09-15T21:25:08.626 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h8' does not appear in the crush map 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb80403b9c0 msgr2=0x7fb80403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb80403b9c0 0x7fb80403de80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb82013c140 tx=0x7fb81c008040 comp rx=0 tx=0).stop 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 10 monclient: shutdown 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 msgr2=0x7fb82013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb82013ae50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb81400e810 tx=0x7fb81400ecd0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 shutdown_connections 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb80403b9c0 0x7fb80403de80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb820135ca0 0x7fb820130920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb8200a0160 0x7fb820138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.629 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 --2- 172.21.15.119:0/2853724868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb82006ec30 0x7fb82013ae50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 >> 172.21.15.119:0/2853724868 conn(0x7fb8200a2ae0 msgr2=0x7fb820072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 shutdown_connections 2024-09-15T21:25:08.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.626+0000 7fb828417640 1 -- 172.21.15.119:0/2853724868 wait complete. 2024-09-15T21:25:08.644 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:08.644 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h9 2024-09-15T21:25:08.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.655+0000 7f6559597640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:25:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.655+0000 7f6559597640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:25:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.655+0000 7f6559597640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.655+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f6554137ca0 con 0x7f65541378c0 2024-09-15T21:25:08.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f653b7fe640 1 -- 172.21.15.17:0/3895814416 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f65540a1a20 con 0x7f65541378c0 2024-09-15T21:25:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f653b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:25:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f653b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:08.664 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=1990424466,config=3541143443,health=2199285752,kv=3079713638,logm=2504054459}) 2024-09-15T21:25:08.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=10,kv=9,logm=64} crc {auth=1638665723,config=3541143443,health=3428897959,kv=4121287724,logm=3282554353}) 2024-09-15T21:25:08.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=1990424466,config=3541143443,health=2199285752,kv=3079713638,logm=2504054459}) 2024-09-15T21:25:08.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=16,config=1,health=10,kv=12,logm=61} crc {auth=1589209448,config=555582061,health=4177428808,kv=3079713637,logm=999781211}) 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f654003b9c0 msgr2=0x7f654003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f654003b9c0 0x7f654003de80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f65440ae000 tx=0x7f65440a6070 comp rx=0 tx=0).stop 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 10 monclient: shutdown 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 msgr2=0x7f65540a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 0x7f65540a0550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c4e0 tx=0x7f654c00c9a0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 shutdown_connections 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f654003b9c0 0x7f654003de80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f655413d2c0 0x7f655409e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f65541383b0 0x7f655409df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 --2- 172.21.15.17:0/3895814416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65541378c0 0x7f65540a0550 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 >> 172.21.15.17:0/3895814416 conn(0x7f65540aac40 msgr2=0x7f655406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 shutdown_connections 2024-09-15T21:25:08.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:08.659+0000 7f6559597640 1 -- 172.21.15.17:0/3895814416 wait complete. 2024-09-15T21:25:08.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.663+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.663+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899959967}) 2024-09-15T21:25:08.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.663+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3727845106}) 2024-09-15T21:25:08.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.663+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.663+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899959967}) 2024-09-15T21:25:08.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.663+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1137633442}) 2024-09-15T21:25:08.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.667+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.667+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3286441562}) 2024-09-15T21:25:08.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.667+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3266932955}) 2024-09-15T21:25:08.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.667+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.667+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3286441562}) 2024-09-15T21:25:08.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.667+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1107184544}) 2024-09-15T21:25:08.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722850837}) 2024-09-15T21:25:08.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2473209489}) 2024-09-15T21:25:08.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722850837}) 2024-09-15T21:25:08.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=275132420}) 2024-09-15T21:25:08.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1396989645}) 2024-09-15T21:25:08.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213495231}) 2024-09-15T21:25:08.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1396989645}) 2024-09-15T21:25:08.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.671+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1051868470}) 2024-09-15T21:25:08.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.675+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.675+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=651211980}) 2024-09-15T21:25:08.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.675+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3454759957}) 2024-09-15T21:25:08.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.675+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.675+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=651211980}) 2024-09-15T21:25:08.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.675+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1407182921}) 2024-09-15T21:25:08.682 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:25:08.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.679+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.679+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1954395802}) 2024-09-15T21:25:08.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.679+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1036290137}) 2024-09-15T21:25:08.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.679+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.679+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1954395802}) 2024-09-15T21:25:08.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.679+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1209686681}) 2024-09-15T21:25:08.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2204619436}) 2024-09-15T21:25:08.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2982082602}) 2024-09-15T21:25:08.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2204619436}) 2024-09-15T21:25:08.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=71134268}) 2024-09-15T21:25:08.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343448169}) 2024-09-15T21:25:08.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1679231905}) 2024-09-15T21:25:08.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343448169}) 2024-09-15T21:25:08.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.683+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3739523365}) 2024-09-15T21:25:08.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=208229795}) 2024-09-15T21:25:08.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728571231}) 2024-09-15T21:25:08.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=208229795}) 2024-09-15T21:25:08.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3662627875}) 2024-09-15T21:25:08.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2309737601}) 2024-09-15T21:25:08.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=725385398}) 2024-09-15T21:25:08.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2309737601}) 2024-09-15T21:25:08.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.687+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=654250324}) 2024-09-15T21:25:08.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1673097275}) 2024-09-15T21:25:08.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2839156920}) 2024-09-15T21:25:08.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1673097275}) 2024-09-15T21:25:08.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=725856226}) 2024-09-15T21:25:08.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1438187907}) 2024-09-15T21:25:08.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2189864546}) 2024-09-15T21:25:08.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1438187907}) 2024-09-15T21:25:08.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.691+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1207617232}) 2024-09-15T21:25:08.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891363865}) 2024-09-15T21:25:08.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841755312}) 2024-09-15T21:25:08.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891363865}) 2024-09-15T21:25:08.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2323296162}) 2024-09-15T21:25:08.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3443696519}) 2024-09-15T21:25:08.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=795711498}) 2024-09-15T21:25:08.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3443696519}) 2024-09-15T21:25:08.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1439542919}) 2024-09-15T21:25:08.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=476885525}) 2024-09-15T21:25:08.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2959931171}) 2024-09-15T21:25:08.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=476885525}) 2024-09-15T21:25:08.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.695+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2670144771}) 2024-09-15T21:25:08.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198228161}) 2024-09-15T21:25:08.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3251315597}) 2024-09-15T21:25:08.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198228161}) 2024-09-15T21:25:08.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3445613146}) 2024-09-15T21:25:08.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034798398}) 2024-09-15T21:25:08.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2070290693}) 2024-09-15T21:25:08.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034798398}) 2024-09-15T21:25:08.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.699+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2050897580}) 2024-09-15T21:25:08.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4161620660}) 2024-09-15T21:25:08.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2929263548}) 2024-09-15T21:25:08.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4161620660}) 2024-09-15T21:25:08.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3702853893}) 2024-09-15T21:25:08.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248359321}) 2024-09-15T21:25:08.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3680465177}) 2024-09-15T21:25:08.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248359321}) 2024-09-15T21:25:08.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=666543962}) 2024-09-15T21:25:08.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3419090624}) 2024-09-15T21:25:08.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2839737639}) 2024-09-15T21:25:08.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3419090624}) 2024-09-15T21:25:08.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.703+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3854075432}) 2024-09-15T21:25:08.712 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.712 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1629595440}) 2024-09-15T21:25:08.712 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1827099646}) 2024-09-15T21:25:08.712 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.712 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1629595440}) 2024-09-15T21:25:08.712 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2204060827}) 2024-09-15T21:25:08.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1887698645}) 2024-09-15T21:25:08.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=588076121}) 2024-09-15T21:25:08.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1887698645}) 2024-09-15T21:25:08.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.707+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1015245655}) 2024-09-15T21:25:08.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3324419941}) 2024-09-15T21:25:08.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=25360212}) 2024-09-15T21:25:08.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3324419941}) 2024-09-15T21:25:08.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1169050711}) 2024-09-15T21:25:08.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1396718592}) 2024-09-15T21:25:08.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=488351666}) 2024-09-15T21:25:08.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1396718592}) 2024-09-15T21:25:08.719 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.711+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2285821421}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869961117}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3638565584}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869961117}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4207651707}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632994334}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1740858682}) 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632994334}) 2024-09-15T21:25:08.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2604224996}) 2024-09-15T21:25:08.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927537639}) 2024-09-15T21:25:08.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3651354109}) 2024-09-15T21:25:08.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927537639}) 2024-09-15T21:25:08.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.715+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=298575873}) 2024-09-15T21:25:08.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2472512489}) 2024-09-15T21:25:08.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=397443508}) 2024-09-15T21:25:08.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2472512489}) 2024-09-15T21:25:08.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898512433}) 2024-09-15T21:25:08.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2003446964}) 2024-09-15T21:25:08.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4245069479}) 2024-09-15T21:25:08.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2003446964}) 2024-09-15T21:25:08.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.719+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1539999447}) 2024-09-15T21:25:08.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1825132084}) 2024-09-15T21:25:08.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3040274684}) 2024-09-15T21:25:08.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1825132084}) 2024-09-15T21:25:08.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3220347651}) 2024-09-15T21:25:08.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3020905925}) 2024-09-15T21:25:08.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3714087606}) 2024-09-15T21:25:08.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3020905925}) 2024-09-15T21:25:08.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.723+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1795554250}) 2024-09-15T21:25:08.731 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.731 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3922642825}) 2024-09-15T21:25:08.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4064412363}) 2024-09-15T21:25:08.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3922642825}) 2024-09-15T21:25:08.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3995628477}) 2024-09-15T21:25:08.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=917772943}) 2024-09-15T21:25:08.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2459374824}) 2024-09-15T21:25:08.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=917772943}) 2024-09-15T21:25:08.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.727+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3008548978}) 2024-09-15T21:25:08.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716807005}) 2024-09-15T21:25:08.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1360802472}) 2024-09-15T21:25:08.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716807005}) 2024-09-15T21:25:08.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1316283762}) 2024-09-15T21:25:08.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2256595894}) 2024-09-15T21:25:08.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1730317449}) 2024-09-15T21:25:08.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2256595894}) 2024-09-15T21:25:08.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.731+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3372430621}) 2024-09-15T21:25:08.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=422384626}) 2024-09-15T21:25:08.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=788152172}) 2024-09-15T21:25:08.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=422384626}) 2024-09-15T21:25:08.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2135557431}) 2024-09-15T21:25:08.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465054739}) 2024-09-15T21:25:08.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3018435878}) 2024-09-15T21:25:08.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465054739}) 2024-09-15T21:25:08.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.735+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4126572409}) 2024-09-15T21:25:08.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845817593}) 2024-09-15T21:25:08.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3895703901}) 2024-09-15T21:25:08.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.744 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845817593}) 2024-09-15T21:25:08.744 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1998211971}) 2024-09-15T21:25:08.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2840365423}) 2024-09-15T21:25:08.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=588598337}) 2024-09-15T21:25:08.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2840365423}) 2024-09-15T21:25:08.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.739+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2707601913}) 2024-09-15T21:25:08.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=483896902}) 2024-09-15T21:25:08.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=520429073}) 2024-09-15T21:25:08.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=483896902}) 2024-09-15T21:25:08.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3133209835}) 2024-09-15T21:25:08.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=14,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=14} crc {logm=1659894567,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=503318008,mon_config_key=521969046,monmap=2617021892,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3165743630,osd_snap=841190635,osdmap=627071438}) 2024-09-15T21:25:08.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=32,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=14} crc {logm=570874249,mdsmap=3140514009,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=503318008,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3165743630,osd_snap=841190635,osdmap=85443423}) 2024-09-15T21:25:08.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=14,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=14} crc {logm=1659894567,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=503318008,mon_config_key=521969046,monmap=2617021892,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3165743630,osd_snap=841190635,osdmap=627071438}) 2024-09-15T21:25:08.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.743+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=15} crc {logm=3856101938,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=4022465175,mon_config_key=3131655263,monmap=186996059,nvmeofgw=598337686,osd_metadata=1154780841,osd_pg_creating=3165743630,osd_snap=841190635,osdmap=2887598059}) 2024-09-15T21:25:08.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.747+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.747+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2010817622}) 2024-09-15T21:25:08.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.747+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2954390959}) 2024-09-15T21:25:08.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.747+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.747+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2010817622}) 2024-09-15T21:25:08.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.747+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=603476868}) 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient: get_monmap_and_config 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient: monmap: 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:08.754536+0000 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:08.754536+0000 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.754 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 1 Processor -- start 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 1 -- start start 2024-09-15T21:25:08.755 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient: init 2024-09-15T21:25:08.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.751+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.751+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3264728996}) 2024-09-15T21:25:08.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.751+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1654118017}) 2024-09-15T21:25:08.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.751+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.751+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3264728996}) 2024-09-15T21:25:08.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.751+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1766476091}) 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82800a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient(hunting): picked mon.noname-b con 0x7f82800a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82800a5780 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f82801383a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82800a1150 con 0x7f82800a5780 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82800a0ef0 con 0x7f82801383a0 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.759 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f82800a3c80 con 0x7f82800a4e60 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f828821b640 10 monclient(hunting): authenticate will time out at 1427.551147s 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f82867b8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f82801383a0 0x7f828013ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48958/0 (socket says 172.21.15.119:48958) 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f82867b8640 1 -- 172.21.15.119:0/1982095828 learned_addr learned my addr 172.21.15.119:0/1982095828 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 1 --2- 172.21.15.119:0/1982095828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82800a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): get_auth_request con 0x7f82800a4e60 auth_method 0 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8285fb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.749+0000 7f8284fb5640 1 -- 172.21.15.119:0/1982095828 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359779595 0 0) 0x7f82800a0ef0 con 0x7f82801383a0 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8284fb5640 10 monclient(hunting): _negotiate 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8284fb5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8284fb5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8284fb5640 10 monclient(hunting): my global_id is 84442 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8284fb5640 1 -- 172.21.15.119:0/1982095828 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f826c003530 con 0x7f82801383a0 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 10 monclient(hunting): handle_auth_done global_id 45104 payload 274 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 1 -- 172.21.15.119:0/1982095828 >> v1:172.21.15.119:6789/0 conn(0x7f82801383a0 legacy=0x7f828013ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 1 -- 172.21.15.119:0/1982095828 >> v1:172.21.15.17:6789/0 conn(0x7f82800a5780 legacy=0x7f8280137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 1 monclient: found mon.noname-b 2024-09-15T21:25:08.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:08.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.753+0000 7f8285fb7640 1 -- 172.21.15.119:0/1982095828 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82800a6cf0 con 0x7f82800a4e60 2024-09-15T21:25:08.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.757+0000 7f8285fb7640 1 --2- 172.21.15.119:0/1982095828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82800a5240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f827c0097f0 tx=0x7f827c01c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feccb847e97df44f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.755+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.755+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1536380387}) 2024-09-15T21:25:08.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.755+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1211790383}) 2024-09-15T21:25:08.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.755+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.755+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1536380387}) 2024-09-15T21:25:08.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.755+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1925896936}) 2024-09-15T21:25:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 1 -- 172.21.15.119:0/1982095828 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0b0070 con 0x7f82800a4e60 2024-09-15T21:25:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: dump: 2024-09-15T21:25:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:08.767 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 1 -- 172.21.15.119:0/1982095828 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c0ab600 con 0x7f82800a4e60 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 1 -- 172.21.15.119:0/1982095828 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0ab9d0 con 0x7f82800a4e60 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f828821b640 5 monclient: authenticate success, global_id 45104 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f828821b640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f828821b640 4 set_mon_vals no callback set 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.761+0000 7f8284fb5640 10 monclient: dump: 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.769 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.770 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:08.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: shutdown 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/1982095828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 msgr2=0x7f82800a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 --2- 172.21.15.119:0/1982095828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82800a5240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f827c0097f0 tx=0x7f827c01c3a0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/1982095828 shutdown_connections 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 --2- 172.21.15.119:0/1982095828 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82800a5240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/1982095828 >> 172.21.15.119:0/1982095828 conn(0x7f82800a6f60 msgr2=0x7f82800a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/1982095828 shutdown_connections 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/1982095828 wait complete. 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: build_initial_monmap 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: monmap: 2024-09-15T21:25:08.775 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 Processor -- start 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- start start 2024-09-15T21:25:08.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: init 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82801cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): picked mon.noname-c con 0x7f82800a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 0x7f82801d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): picked mon.noname-b con 0x7f82800a5780 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82801383a0 0x7f82801d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): picked mon.noname-a con 0x7f82801383a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82800a7e30 con 0x7f82801383a0 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82800a7cb0 con 0x7f82800a5780 2024-09-15T21:25:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f82800a2190 con 0x7f82800a4e60 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient(hunting): authenticate will time out at 1427.566017s 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 0x7f82801d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 0x7f82801d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38862/0 (socket says 172.21.15.119:38862) 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 -- 172.21.15.119:0/519849431 learned_addr learned my addr 172.21.15.119:0/519849431 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): get_auth_request con 0x7f82800a5780 auth_method 0 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82801383a0 0x7f82801d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): get_auth_request con 0x7f82801383a0 auth_method 0 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f8285fb7640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82801cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f8285fb7640 10 monclient(hunting): get_auth_request con 0x7f82800a4e60 auth_method 0 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f8285fb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f8285fb7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f8285fb7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82867b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient(hunting): handle_auth_done global_id 84445 payload 824 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 -- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 msgr2=0x7f82801cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82801cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 -- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82801383a0 msgr2=0x7f82801d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82801383a0 0x7f82801d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient: _finish_hunting 0 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 monclient: found mon.noname-b 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 -- 172.21.15.119:0/519849431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8280140db0 con 0x7f82800a5780 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82857b6640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 0x7f82801d0260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f827000c4b0 tx=0x7f827000c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82700052d0 con 0x7f82800a5780 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: dump: 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:08.781 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 5 monclient: authenticate success, global_id 84445 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270005470 con 0x7f82800a5780 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _renew_subs 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82800a9370 con 0x7f82800a5780 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270012850 con 0x7f82800a5780 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 10 monclient: dump: 2024-09-15T21:25:08.782 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82777fe640 4 set_mon_vals no callback set 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _renew_subs 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:08.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8280140ad0 con 0x7f82800a5780 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f82700129f0 con 0x7f82800a5780 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.765+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8280077180 con 0x7f82800a5780 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.769+0000 7f82767fc640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f826403b9c0 0x7f826403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.769+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(226..226 src has 1..226) ==== 13304+0+0 (secure 0 0 0) 0x7f8270051990 con 0x7f82800a5780 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.769+0000 7f8285fb7640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f826403b9c0 0x7f826403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.769+0000 7f8285fb7640 10 monclient: get_auth_request con 0x7f826403b9c0 auth_method 0 2024-09-15T21:25:08.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.769+0000 7f8285fb7640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f826403b9c0 0x7f826403de80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f827c0ad000 tx=0x7f827c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.773+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f827004e250 con 0x7f82800a5780 2024-09-15T21:25:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.773+0000 7f82767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:08.785 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.773+0000 7f82767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:08.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.767+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.767+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=26} crc {osdmap=1786505599}) 2024-09-15T21:25:08.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.767+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=25} crc {osdmap=201277917}) 2024-09-15T21:25:08.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.767+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:08.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.767+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=26} crc {osdmap=1786505599}) 2024-09-15T21:25:08.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:08.767+0000 7f5ffa1aa640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=25} crc {osdmap=2103284144}) 2024-09-15T21:25:08.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.929+0000 7f828821b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T21:25:08.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.929+0000 7f828821b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T21:25:08.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.929+0000 7f828821b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:08.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:08.929+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) -- 0x7f828006f550 con 0x7f82800a5780 2024-09-15T21:25:09.544 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.541+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 removed item id -43 name 'h9' from crush map v227) ==== 118+0+0 (secure 0 0 0) 0x7f8270010cf0 con 0x7f82800a5780 2024-09-15T21:25:09.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.541+0000 7f82767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T21:25:09.545 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.541+0000 7f82767fc640 10 monclient: _finish_command 2 = system:0 removed item id -43 name 'h9' from crush map 2024-09-15T21:25:09.654 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.649+0000 7f828821b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T21:25:09.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.649+0000 7f828821b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T21:25:09.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.649+0000 7f828821b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.649+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) -- 0x7f82800008e0 con 0x7f82800a5780 2024-09-15T21:25:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f82767fc640 1 -- 172.21.15.119:0/519849431 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 device 'h9' does not appear in the crush map v227) ==== 118+0+0 (secure 0 0 0) 0x7f8270010a50 con 0x7f82800a5780 2024-09-15T21:25:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f82767fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T21:25:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f82767fc640 10 monclient: _finish_command 3 = system:0 device 'h9' does not appear in the crush map 2024-09-15T21:25:09.657 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h9' does not appear in the crush map 2024-09-15T21:25:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f826403b9c0 msgr2=0x7f826403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.659 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f826403b9c0 0x7f826403de80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f827c0ad000 tx=0x7f827c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 10 monclient: shutdown 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 msgr2=0x7f82801d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 0x7f82801d0260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f827000c4b0 tx=0x7f827000c970 comp rx=0 tx=0).stop 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 shutdown_connections 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f826403b9c0 0x7f826403de80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82801383a0 0x7f82801d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82800a5780 0x7f82801d0260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 --2- 172.21.15.119:0/519849431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82800a4e60 0x7f82801cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 >> 172.21.15.119:0/519849431 conn(0x7f82800a6f60 msgr2=0x7f8280071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 shutdown_connections 2024-09-15T21:25:09.660 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.653+0000 7f828821b640 1 -- 172.21.15.119:0/519849431 wait complete. 2024-09-15T21:25:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:09.674 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h10 2024-09-15T21:25:09.684 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:25:09.684 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:09.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient: get_monmap_and_config 2024-09-15T21:25:09.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient: build_initial_monmap 2024-09-15T21:25:09.793 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient: monmap: 2024-09-15T21:25:09.793 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:09.793 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.793 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:09.793994+0000 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:09.793994+0000 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:09.794 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 Processor -- start 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 -- start start 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient: init 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5cb01324a0 0x7f5cb0132880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): picked mon.noname-c con 0x7f5cb01324a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb0132dc0 0x7f5cb00a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.797 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): picked mon.noname-b con 0x7f5cb0132dc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5cb00739b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb0072730 con 0x7f5cb00739b0 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb01320e0 con 0x7f5cb01324a0 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb0072a10 con 0x7f5cb0132dc0 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb7b89640 10 monclient(hunting): authenticate will time out at 1428.591092s 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5cb01324a0 0x7f5cb0132880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb6126640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5cb00739b0 0x7f5cb00a5170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48972/0 (socket says 172.21.15.119:48972) 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb6126640 1 -- 172.21.15.119:0/960591516 learned_addr learned my addr 172.21.15.119:0/960591516 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:09.798 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): get_auth_request con 0x7f5cb01324a0 auth_method 0 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5124640 1 --2- 172.21.15.119:0/960591516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb0132dc0 0x7f5cb00a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5124640 10 monclient(hunting): get_auth_request con 0x7f5cb0132dc0 auth_method 0 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5124640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5124640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.799 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5124640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient(hunting): handle_auth_done global_id 84448 payload 274 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 1 -- 172.21.15.119:0/960591516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb0132dc0 msgr2=0x7f5cb00a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 1 --2- 172.21.15.119:0/960591516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb0132dc0 0x7f5cb00a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 1 -- 172.21.15.119:0/960591516 >> v1:172.21.15.119:6789/0 conn(0x7f5cb00739b0 legacy=0x7f5cb00a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient: _finish_hunting 0 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 1 monclient: found mon.noname-c 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.789+0000 7f5cb5925640 1 -- 172.21.15.119:0/960591516 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0072cf0 con 0x7f5cb01324a0 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb5925640 1 --2- 172.21.15.119:0/960591516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0132880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0097c0 tx=0x7f5cac01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a9cd6bcf3714fa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 1 -- 172.21.15.119:0/960591516 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0b0070 con 0x7f5cb01324a0 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: dump: 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.806 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: _finish_auth 0 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: _check_auth_tickets 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:09.807 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 5 monclient: authenticate success, global_id 84448 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 1 -- 172.21.15.119:0/960591516 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac0ab600 con 0x7f5cb01324a0 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 1 -- 172.21.15.119:0/960591516 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0ab9d0 con 0x7f5cb01324a0 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 4 set_mon_vals no callback set 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb4923640 10 monclient: dump: 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.808 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 10 monclient: shutdown 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 -- 172.21.15.119:0/960591516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 msgr2=0x7f5cb0132880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/960591516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0132880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0097c0 tx=0x7f5cac01c330 comp rx=0 tx=0).stop 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 -- 172.21.15.119:0/960591516 shutdown_connections 2024-09-15T21:25:09.809 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/960591516 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb0132dc0 0x7f5cb00a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/960591516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0132880 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 -- 172.21.15.119:0/960591516 >> 172.21.15.119:0/960591516 conn(0x7f5cb006cbf0 msgr2=0x7f5cb006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 -- 172.21.15.119:0/960591516 shutdown_connections 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 -- 172.21.15.119:0/960591516 wait complete. 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 10 monclient: build_initial_monmap 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 10 monclient: monmap: 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:09.810 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 Processor -- start 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 1 -- start start 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.793+0000 7f5cb7b89640 10 monclient: init 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:09.811 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb00739b0 0x7f5cb0131420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): picked mon.noname-c con 0x7f5cb00739b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0131960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): picked mon.noname-b con 0x7f5cb01324a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cb0132dc0 0x7f5cb01cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): picked mon.noname-a con 0x7f5cb0132dc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.812 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00a5880 con 0x7f5cb0132dc0 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00698f0 con 0x7f5cb01324a0 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb00a5a00 con 0x7f5cb00739b0 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient(hunting): authenticate will time out at 1428.596343s 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0131960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0131960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38880/0 (socket says 172.21.15.119:38880) 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 -- 172.21.15.119:0/358562790 learned_addr learned my addr 172.21.15.119:0/358562790 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): get_auth_request con 0x7f5cb01324a0 auth_method 0 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb6126640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cb0132dc0 0x7f5cb01cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient(hunting): handle_auth_done global_id 84454 payload 824 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 -- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb00739b0 msgr2=0x7f5cb0131420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb00739b0 0x7f5cb0131420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 -- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cb0132dc0 msgr2=0x7f5cb01cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cb0132dc0 0x7f5cb01cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient: _finish_hunting 0 2024-09-15T21:25:09.814 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 monclient: found mon.noname-b 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 -- 172.21.15.119:0/358562790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb013bec0 con 0x7f5cb01324a0 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb6126640 10 monclient: get_auth_request con 0x7f5cb0132dc0 auth_method 0 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb6126640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cb0132dc0 0x7f5cb01cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5124640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0131960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d7d0 tx=0x7f5ca000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000b6c0 con 0x7f5cb01324a0 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: dump: 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.815 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca0019070 con 0x7f5cb01324a0 2024-09-15T21:25:09.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000baa0 con 0x7f5cb01324a0 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 10 monclient: dump: 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.817 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 5 monclient: authenticate success, global_id 84454 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _renew_subs 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca7fff640 4 set_mon_vals no callback set 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb013c1a0 con 0x7f5cb01324a0 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _renew_subs 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb0139e40 con 0x7f5cb01324a0 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.818 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb0132880 con 0x7f5cb01324a0 2024-09-15T21:25:09.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5ca000bc40 con 0x7f5cb01324a0 2024-09-15T21:25:09.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c9403b9c0 0x7f5c9403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f5ca00520a0 con 0x7f5cb01324a0 2024-09-15T21:25:09.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5925640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c9403b9c0 0x7f5c9403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5925640 10 monclient: get_auth_request con 0x7f5c9403b9c0 auth_method 0 2024-09-15T21:25:09.819 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.797+0000 7f5cb5925640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c9403b9c0 0x7f5c9403de80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0ad000 tx=0x7f5cac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:09.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.801+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5ca004ea30 con 0x7f5cb01324a0 2024-09-15T21:25:09.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.801+0000 7f5ca67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:09.820 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.801+0000 7f5ca67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:09.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient: get_monmap_and_config 2024-09-15T21:25:09.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient: build_initial_monmap 2024-09-15T21:25:09.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient: monmap: 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:09.815420+0000 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:09.815420+0000 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:09.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 1 Processor -- start 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 1 -- start start 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient: init 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f983c1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f983c1383b0 0x7f983c13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient(hunting): picked mon.noname-c con 0x7f983c1383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f983c13d2c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f983c0a1a20 con 0x7f983c1378c0 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f983c0a1d00 con 0x7f983c13d2c0 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f983c0a1fe0 con 0x7f983c1383b0 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f9842912640 10 monclient(hunting): authenticate will time out at 1433.210854s 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f983c1378c0 0x7f983c137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35930/0 (socket says 172.21.15.17:35930) 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983bfff640 1 -- 172.21.15.17:0/3657579663 learned_addr learned my addr 172.21.15.17:0/3657579663 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 1 --2- 172.21.15.17:0/3657579663 >> v2:172.21.15.119:3300/0 conn(0x7f983c1383b0 0x7f983c13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): get_auth_request con 0x7f983c1383b0 auth_method 0 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422539937 0 0) 0x7f982c0014a0 con 0x7f983c1378c0 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): _negotiate 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): my global_id is 94137 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f981c003530 con 0x7f983c1378c0 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:09.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619329832 0 0) 0x7f983c0a1d00 con 0x7f983c13d2c0 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): _negotiate 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient(hunting): my global_id is 84460 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c0014a0 con 0x7f983c13d2c0 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient(hunting): handle_auth_done global_id 84457 payload 274 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1185180011 0 0) 0x7f981c003530 con 0x7f983c1378c0 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 1 -- 172.21.15.17:0/3657579663 >> v1:172.21.15.119:6789/0 conn(0x7f983c13d2c0 legacy=0x7f983c13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 1 -- 172.21.15.17:0/3657579663 >> v1:172.21.15.17:6789/0 conn(0x7f983c1378c0 legacy=0x7f983c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:09.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:09.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:09.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983b7fe640 1 -- 172.21.15.17:0/3657579663 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c0a22c0 con 0x7f983c1383b0 2024-09-15T21:25:09.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.811+0000 7f983affd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:09.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983b7fe640 1 --2- 172.21.15.17:0/3657579663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c1383b0 0x7f983c13cd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98240097f0 tx=0x7f982401bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58674fba6be40c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:09.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c001c50 con 0x7f983c1383b0 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: dump: 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: _finish_auth 0 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 5 monclient: authenticate success, global_id 84457 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98240a54e0 con 0x7f983c1383b0 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 1 -- 172.21.15.17:0/3657579663 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240a5890 con 0x7f983c1383b0 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f983affd640 10 monclient: dump: 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 4 set_mon_vals no callback set 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 10 monclient: shutdown 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 -- 172.21.15.17:0/3657579663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c1383b0 msgr2=0x7f983c13cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 --2- 172.21.15.17:0/3657579663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c1383b0 0x7f983c13cd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98240097f0 tx=0x7f982401bf60 comp rx=0 tx=0).stop 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 -- 172.21.15.17:0/3657579663 shutdown_connections 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 --2- 172.21.15.17:0/3657579663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c1383b0 0x7f983c13cd80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 -- 172.21.15.17:0/3657579663 >> 172.21.15.17:0/3657579663 conn(0x7f983c0aac40 msgr2=0x7f983c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 -- 172.21.15.17:0/3657579663 shutdown_connections 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 -- 172.21.15.17:0/3657579663 wait complete. 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 10 monclient: build_initial_monmap 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 10 monclient: monmap: 2024-09-15T21:25:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.815+0000 7f9842912640 1 Processor -- start 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- start start 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: init 2024-09-15T21:25:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 0x7f983c1d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): picked mon.noname-a con 0x7f983c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f983c1383b0 0x7f983c136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): picked mon.noname-c con 0x7f983c1383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c13d2c0 0x7f983c09c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): picked mon.noname-b con 0x7f983c13d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f983c144200 con 0x7f983c1378c0 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f983c144080 con 0x7f983c13d2c0 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f983c144380 con 0x7f983c1383b0 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient(hunting): authenticate will time out at 1433.216547s 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 0x7f983c1d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 0x7f983c1d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56740/0 (socket says 172.21.15.17:56740) 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 -- 172.21.15.17:0/341584834 learned_addr learned my addr 172.21.15.17:0/341584834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): get_auth_request con 0x7f983c1378c0 auth_method 0 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f983c1383b0 0x7f983c136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c13d2c0 0x7f983c09c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): get_auth_request con 0x7f983c1383b0 auth_method 0 2024-09-15T21:25:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): get_auth_request con 0x7f983c13d2c0 auth_method 0 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient(hunting): handle_auth_done global_id 94140 payload 824 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 -- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f983c1383b0 msgr2=0x7f983c136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f983c1383b0 0x7f983c136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 -- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c13d2c0 msgr2=0x7f983c09c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c13d2c0 0x7f983c09c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 monclient: found mon.noname-a 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 -- 172.21.15.17:0/341584834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c0a04a0 con 0x7f983c1378c0 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9840eaf640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c13d2c0 0x7f983c09c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983bfff640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 0x7f983c1d06a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f982c0018e0 tx=0x7f982c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c017070 con 0x7f983c1378c0 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: dump: 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: _finish_auth 0 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 5 monclient: authenticate success, global_id 94140 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982c012400 con 0x7f983c1378c0 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c0127f0 con 0x7f983c1378c0 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _renew_subs 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983c0a7f00 con 0x7f983c1378c0 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 10 monclient: dump: 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:09.846 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983a7fc640 4 set_mon_vals no callback set 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _renew_subs 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983c1406d0 con 0x7f983c1378c0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983c0a12f0 con 0x7f983c1378c0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f982c01a070 con 0x7f983c1378c0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f982003b9c0 0x7f982003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f982c051d50 con 0x7f983c1378c0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f982003b9c0 0x7f982003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 10 monclient: get_auth_request con 0x7f982003b9c0 auth_method 0 2024-09-15T21:25:09.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.819+0000 7f983b7fe640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f982003b9c0 0x7f982003de80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f98240ae000 tx=0x7f98240a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:09.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.827+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f983c0a1a20 con 0x7f983c1378c0 2024-09-15T21:25:09.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.827+0000 7f9838ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:09.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:09.827+0000 7f9838ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.981+0000 7f5cb7b89640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T21:25:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.981+0000 7f5cb7b89640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T21:25:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.981+0000 7f5cb7b89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:09.986 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:09.981+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) -- 0x7f5cb013a9a0 con 0x7f5cb01324a0 2024-09-15T21:25:10.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9842912640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9842912640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9842912640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f983c0008e0 con 0x7f983c1378c0 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9838ff9640 1 -- 172.21.15.17:0/341584834 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f982c04e6e0 con 0x7f983c1378c0 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9838ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.055+0000 7f9838ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:10.065 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f982003b9c0 msgr2=0x7f982003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f982003b9c0 0x7f982003de80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f98240ae000 tx=0x7f98240a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 10 monclient: shutdown 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 msgr2=0x7f983c1d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 0x7f983c1d06a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f982c0018e0 tx=0x7f982c00cf90 comp rx=0 tx=0).stop 2024-09-15T21:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 shutdown_connections 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f982003b9c0 0x7f982003de80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f983c13d2c0 0x7f983c09c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f983c1383b0 0x7f983c136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 --2- 172.21.15.17:0/341584834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f983c1378c0 0x7f983c1d06a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 >> 172.21.15.17:0/341584834 conn(0x7f983c0aac40 msgr2=0x7f983c0a82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 shutdown_connections 2024-09-15T21:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.059+0000 7f9842912640 1 -- 172.21.15.17:0/341584834 wait complete. 2024-09-15T21:25:10.079 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:25:10.080 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:25:10.080 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:25:10.080 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:25:10.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient: get_monmap_and_config 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient: monmap: 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:10.202307+0000 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:10.202307+0000 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:10.203 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:10.204 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 Processor -- start 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 -- start start 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient: init 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f098c1378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 0x7f098c13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): picked mon.noname-b con 0x7f098c1383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f098c13d2c0 0x7f098c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): picked mon.noname-a con 0x7f098c13d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f098c0a1a20 con 0x7f098c1378c0 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f098c0a1d00 con 0x7f098c13d2c0 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f098c0a1fe0 con 0x7f098c1383b0 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0990aa2640 10 monclient(hunting): authenticate will time out at 1433.597641s 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f098c13d2c0 0x7f098c13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 0x7f098c13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 0x7f098c13cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39128/0 (socket says 172.21.15.17:39128) 2024-09-15T21:25:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f098c13d2c0 0x7f098c13f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56746/0 (socket says 172.21.15.17:56746) 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 1 -- 172.21.15.17:0/1580623704 learned_addr learned my addr 172.21.15.17:0/1580623704 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): get_auth_request con 0x7f098c1383b0 auth_method 0 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): get_auth_request con 0x7f098c13d2c0 auth_method 0 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient(hunting): handle_auth_done global_id 94143 payload 274 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 -- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 msgr2=0x7f098c13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 --2- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 0x7f098c13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 -- 172.21.15.17:0/1580623704 >> v1:172.21.15.119:6789/0 conn(0x7f098c1378c0 legacy=0x7f098c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 monclient: found mon.noname-a 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f098ad9d640 1 -- 172.21.15.17:0/1580623704 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c0a22c0 con 0x7f098c13d2c0 2024-09-15T21:25:10.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.199+0000 7f0989d9b640 1 --2- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 0x7f098c13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 --2- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c13f6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f097c009d90 tx=0x7f097c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd80b72ba509d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 1 -- 172.21.15.17:0/1580623704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0b1070 con 0x7f098c13d2c0 2024-09-15T21:25:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: dump: 2024-09-15T21:25:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 5 monclient: authenticate success, global_id 94143 2024-09-15T21:25:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 1 -- 172.21.15.17:0/1580623704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c0a4dd0 con 0x7f098c13d2c0 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 1 -- 172.21.15.17:0/1580623704 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0ac660 con 0x7f098c13d2c0 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 4 set_mon_vals no callback set 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098959a640 10 monclient: dump: 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: shutdown 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 msgr2=0x7f098c13f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c13f6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f097c009d90 tx=0x7f097c01c510 comp rx=0 tx=0).stop 2024-09-15T21:25:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1580623704 shutdown_connections 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c13f6b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1580623704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1383b0 0x7f098c13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1580623704 >> 172.21.15.17:0/1580623704 conn(0x7f098c0aac40 msgr2=0x7f098c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1580623704 shutdown_connections 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1580623704 wait complete. 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: monmap: 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 Processor -- start 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- start start 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: init 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1378c0 0x7f098c0a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): picked mon.noname-c con 0x7f098c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098c1383b0 0x7f098c1cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): picked mon.noname-b con 0x7f098c1383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): picked mon.noname-a con 0x7f098c13d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f098c143020 con 0x7f098c13d2c0 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f098c142ea0 con 0x7f098c1383b0 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f098c1431a0 con 0x7f098c1378c0 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient(hunting): authenticate will time out at 1433.602791s 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1378c0 0x7f098c0a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c136d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56756/0 (socket says 172.21.15.17:56756) 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1378c0 0x7f098c0a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39140/0 (socket says 172.21.15.17:39140) 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 -- 172.21.15.17:0/1853731228 learned_addr learned my addr 172.21.15.17:0/1853731228 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): get_auth_request con 0x7f098c13d2c0 auth_method 0 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): get_auth_request con 0x7f098c1378c0 auth_method 0 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0989d9b640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098c1383b0 0x7f098c1cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0989d9b640 10 monclient(hunting): get_auth_request con 0x7f098c1383b0 auth_method 0 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0989d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0989d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0989d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient(hunting): handle_auth_done global_id 94146 payload 824 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1378c0 msgr2=0x7f098c0a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1378c0 0x7f098c0a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098c1383b0 msgr2=0x7f098c1cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098c1383b0 0x7f098c1cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 monclient: found mon.noname-a 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c09f140 con 0x7f098c13d2c0 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098a59c640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f098c1378c0 0x7f098c0a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f098ad9d640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c136d50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f097c0af040 tx=0x7f097c0195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0b1050 con 0x7f098c13d2c0 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: dump: 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c0bd070 con 0x7f098c13d2c0 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 5 monclient: authenticate success, global_id 94146 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: _renew_subs 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c0a8000 con 0x7f098c13d2c0 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c009450 con 0x7f098c13d2c0 2024-09-15T21:25:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0982ffd640 10 monclient: dump: 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0988d99640 4 set_mon_vals no callback set 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: _renew_subs 2024-09-15T21:25:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.203+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c13f6c0 con 0x7f098c13d2c0 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0990aa2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 0x7f098c1d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f098c137ca0 con 0x7f098c1d07f0 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 0x7f098c1d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f097c0c2460 con 0x7f098c13d2c0 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient: get_auth_request con 0x7f098c1d07f0 auth_method 0 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f096c03b9c0 0x7f096c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 10 monclient(hunting): handle_auth_done global_id 94149 payload 824 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f097c0ec080 con 0x7f098c13d2c0 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f098c0008e0 con 0x7f098c1d07f0 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f098a59c640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 0x7f098c1d0bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0984000f70 tx=0x7f098400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09840052e0 con 0x7f098c1d07f0 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0989d9b640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f096c03b9c0 0x7f096c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 10 monclient: dump: 2024-09-15T21:25:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0989d9b640 10 monclient: get_auth_request con 0x7f096c03b9c0 auth_method 0 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0984018070 con 0x7f098c1d07f0 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 msgr2=0x7f098c1d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 0x7f098c1d0bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0984000f70 tx=0x7f098400ff70 comp rx=0 tx=0).stop 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 msgr2=0x7f098c1d0bd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0982ffd640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d07f0 0x7f098c1d0bd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.207+0000 7f0989d9b640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f096c03b9c0 0x7f096c03de80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f098c13f0f0 tx=0x7f0974008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f0990aa2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 0x7f098c13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f0990aa2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1d1520 con 0x7f098c1d1110 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 0x7f098c13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient: get_auth_request con 0x7f098c1d1110 auth_method 0 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.211+0000 7f098a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f098a59c640 10 monclient(hunting): handle_auth_done global_id 94152 payload 824 2024-09-15T21:25:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f098a59c640 1 -- 172.21.15.17:0/1853731228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f098c06cba0 con 0x7f098c1d1110 2024-09-15T21:25:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f098a59c640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 0x7f098c13ad70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0984017000 tx=0x7f0984001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098401ed30 con 0x7f098c1d1110 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 10 monclient: dump: 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7f0984003600 con 0x7f098c1d1110 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 msgr2=0x7f098c13ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 0x7f098c13ad70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0984017000 tx=0x7f0984001130 comp rx=0 tx=0).stop 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 msgr2=0x7f098c13ad70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0982ffd640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c1d1110 0x7f098c13ad70 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0984017000 tx=0x7f0984001130 comp rx=0 tx=0).stop 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 76, 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 15399, 2024-09-15T21:25:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:25:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:10.251 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f096c03b9c0 msgr2=0x7f096c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f096c03b9c0 0x7f096c03de80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f098c13f0f0 tx=0x7f0974008040 comp rx=0 tx=0).stop 2024-09-15T21:25:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 10 monclient: shutdown 2024-09-15T21:25:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 msgr2=0x7f098c136d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 --2- 172.21.15.17:0/1853731228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f098c13d2c0 0x7f098c136d50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f097c0af040 tx=0x7f097c0195d0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f098ad9d640 1 -- 172.21.15.17:0/1853731228 reap_dead start 2024-09-15T21:25:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f098ad9d640 1 -- 172.21.15.17:0/1853731228 reap_dead start 2024-09-15T21:25:10.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 shutdown_connections 2024-09-15T21:25:10.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 >> 172.21.15.17:0/1853731228 conn(0x7f098c0aac40 msgr2=0x7f098c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 shutdown_connections 2024-09-15T21:25:10.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.215+0000 7f0990aa2640 1 -- 172.21.15.17:0/1853731228 wait complete. 2024-09-15T21:25:10.255 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:10.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:25:10.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient: get_monmap_and_config 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient: monmap: 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:10.366710+0000 2024-09-15T21:25:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:10.366710+0000 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:10.368 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 1 Processor -- start 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 1 -- start start 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient: init 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4fcc0a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4fcc09ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc13e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient(hunting): picked mon.noname-b con 0x7f4fcc0a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fcc0a3130 con 0x7f4fcc0a0890 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fcc073df0 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc0a1bf0 con 0x7f4fcc0a1380 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fd1df0640 10 monclient(hunting): authenticate will time out at 1433.762118s 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4fcc0a0890 0x7f4fcc0a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35942/0 (socket says 172.21.15.17:35942) 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcb7fe640 1 -- 172.21.15.17:0/2747743893 learned_addr learned my addr 172.21.15.17:0/2747743893 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 1 --2- 172.21.15.17:0/2747743893 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc13e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): get_auth_request con 0x7f4fcc0a1380 auth_method 0 2024-09-15T21:25:10.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_done global_id 45122 payload 274 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 1 -- 172.21.15.17:0/2747743893 >> v1:172.21.15.119:6789/0 conn(0x7f4fcc09ffc0 legacy=0x7f4fcc13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fca7fc640 1 -- 172.21.15.17:0/2747743893 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 434978936 0 0) 0x7f4fcc0a3130 con 0x7f4fcc0a0890 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 1 -- 172.21.15.17:0/2747743893 >> v1:172.21.15.17:6789/0 conn(0x7f4fcc0a0890 legacy=0x7f4fcc0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 1 monclient: found mon.noname-b 2024-09-15T21:25:10.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fcbfff640 1 -- 172.21.15.17:0/2747743893 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc09ccd0 con 0x7f4fcc0a1380 2024-09-15T21:25:10.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.363+0000 7f4fca7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:10.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 1 --2- 172.21.15.17:0/2747743893 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc13e1f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009820 tx=0x7f4fc401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c58645dccea306f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 1 -- 172.21.15.17:0/2747743893 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40b0070 con 0x7f4fcc0a1380 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: dump: 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.385 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 5 monclient: authenticate success, global_id 45122 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 1 -- 172.21.15.17:0/2747743893 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc40ab600 con 0x7f4fcc0a1380 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 4 set_mon_vals no callback set 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 1 -- 172.21.15.17:0/2747743893 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40ab9d0 con 0x7f4fcc0a1380 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fca7fc640 10 monclient: dump: 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: shutdown 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2747743893 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 msgr2=0x7f4fcc13e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 --2- 172.21.15.17:0/2747743893 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc13e1f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009820 tx=0x7f4fc401c380 comp rx=0 tx=0).stop 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2747743893 shutdown_connections 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 --2- 172.21.15.17:0/2747743893 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc13e1f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2747743893 >> 172.21.15.17:0/2747743893 conn(0x7f4fcc0a66a0 msgr2=0x7f4fcc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2747743893 shutdown_connections 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2747743893 wait complete. 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: monmap: 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 Processor -- start 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- start start 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: init 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc09ffc0 0x7f4fcc11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): picked mon.noname-a con 0x7f4fcc09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fcc0a0890 0x7f4fcc11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): picked mon.noname-b con 0x7f4fcc0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc11cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): picked mon.noname-c con 0x7f4fcc0a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1412e0 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc141160 con 0x7f4fcc0a0890 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc141460 con 0x7f4fcc0a1380 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient(hunting): authenticate will time out at 1433.766734s 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc09ffc0 0x7f4fcc11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc11cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc09ffc0 0x7f4fcc11c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56790/0 (socket says 172.21.15.17:56790) 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc11cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39164/0 (socket says 172.21.15.17:39164) 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 -- 172.21.15.17:0/2674948747 learned_addr learned my addr 172.21.15.17:0/2674948747 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): get_auth_request con 0x7f4fcc09ffc0 auth_method 0 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): get_auth_request con 0x7f4fcc0a1380 auth_method 0 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcaffd640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fcc0a0890 0x7f4fcc11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient(hunting): handle_auth_done global_id 94158 payload 824 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 msgr2=0x7f4fcc11cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc11cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fcc0a0890 msgr2=0x7f4fcc11ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fcc0a0890 0x7f4fcc11ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 monclient: found mon.noname-a 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc11ed80 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcaffd640 10 monclient: get_auth_request con 0x7f4fcc0a0890 auth_method 0 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcbfff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc0a1380 0x7f4fcc11cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcaffd640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fcc0a0890 0x7f4fcc11ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fcb7fe640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc09ffc0 0x7f4fcc11c4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c490 tx=0x7f4fb400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4018070 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: dump: 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 5 monclient: authenticate success, global_id 94158 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: _renew_subs 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc06f6e0 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb4007500 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4004a30 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc3fff640 10 monclient: dump: 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.395 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fc9ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: _renew_subs 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.367+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc11eaa0 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fd1df0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fd1df0640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 0x7f4fcc1cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fd1df0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc0a0c70 con 0x7f4fcc11d790 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 0x7f4fcc1cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient: get_auth_request con 0x7f4fcc11d790 auth_method 0 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4fb4004bd0 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_done global_id 45128 payload 824 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4fcc0008e0 con 0x7f4fcc11d790 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fac03b9c0 0x7f4fac03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f4fb4050ff0 con 0x7f4fcc09ffc0 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcaffd640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 0x7f4fcc1cdb20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc1364c0 tx=0x7f4fbc00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc006d60 con 0x7f4fcc11d790 2024-09-15T21:25:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 10 monclient: dump: 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcbfff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fac03b9c0 0x7f4fac03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcbfff640 10 monclient: get_auth_request con 0x7f4fac03b9c0 auth_method 0 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4fbc006d60 con 0x7f4fcc11d790 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 msgr2=0x7f4fcc1cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 0x7f4fcc1cdb20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc1364c0 tx=0x7f4fbc00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 msgr2=0x7f4fcc1cdb20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fc3fff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc11d790 0x7f4fcc1cdb20 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc1364c0 tx=0x7f4fbc00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.371+0000 7f4fcbfff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fac03b9c0 0x7f4fac03de80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40ad000 tx=0x7f4fc40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fd1df0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fd1df0640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 0x7f4fcc02d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fd1df0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc1391c0 con 0x7f4fcc02d450 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 0x7f4fcc02d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient: get_auth_request con 0x7f4fcc02d450 auth_method 0 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 10 monclient(hunting): handle_auth_done global_id 45131 payload 824 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.375+0000 7f4fcaffd640 1 -- 172.21.15.17:0/2674948747 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4fcc1ce060 con 0x7f4fcc02d450 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fcaffd640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 0x7f4fcc02d860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc017000 tx=0x7f4fbc003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc007890 con 0x7f4fcc02d450 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 10 monclient: dump: 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.401 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7f4fbc007890 con 0x7f4fcc02d450 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 msgr2=0x7f4fcc02d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 0x7f4fcc02d860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc017000 tx=0x7f4fbc003af0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 msgr2=0x7f4fcc02d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fc3fff640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4fcc02d450 0x7f4fcc02d860 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc017000 tx=0x7f4fbc003af0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 76, 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 278977, 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:10.417 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fac03b9c0 msgr2=0x7f4fac03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4fac03b9c0 0x7f4fac03de80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40ad000 tx=0x7f4fc40a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 10 monclient: shutdown 2024-09-15T21:25:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc09ffc0 msgr2=0x7f4fcc11c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 --2- 172.21.15.17:0/2674948747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc09ffc0 0x7f4fcc11c4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c490 tx=0x7f4fb400c950 comp rx=0 tx=0).stop 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fcbfff640 1 -- 172.21.15.17:0/2674948747 reap_dead start 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fcbfff640 1 -- 172.21.15.17:0/2674948747 reap_dead start 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 shutdown_connections 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 >> 172.21.15.17:0/2674948747 conn(0x7f4fcc0a66a0 msgr2=0x7f4fcc0a4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 shutdown_connections 2024-09-15T21:25:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.379+0000 7f4fd1df0640 1 -- 172.21.15.17:0/2674948747 wait complete. 2024-09-15T21:25:10.419 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:10.419 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:25:10.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient: get_monmap_and_config 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient: monmap: 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:10.539576+0000 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:10.539576+0000 2024-09-15T21:25:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:10.541 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 Processor -- start 2024-09-15T21:25:10.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 -- start start 2024-09-15T21:25:10.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient: init 2024-09-15T21:25:10.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:25:10.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 0x7f8fd00a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): picked mon.noname-a con 0x7f8fd00a4a90 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8fd00a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 0x7f8fd0137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): picked mon.noname-b con 0x7f8fd00a5800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fd00a2c30 con 0x7f8fd00a53b0 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8fd00a0f00 con 0x7f8fd00a4a90 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd00a19b0 con 0x7f8fd00a5800 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fd48fd640 10 monclient(hunting): authenticate will time out at 1433.935069s 2024-09-15T21:25:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 0x7f8fd00a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 0x7f8fd0137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 0x7f8fd0137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39192/0 (socket says 172.21.15.17:39192) 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 0x7f8fd00a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56792/0 (socket says 172.21.15.17:56792) 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 -- 172.21.15.17:0/3478171343 learned_addr learned my addr 172.21.15.17:0/3478171343 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): get_auth_request con 0x7f8fd00a4a90 auth_method 0 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): get_auth_request con 0x7f8fd00a5800 auth_method 0 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient(hunting): handle_auth_done global_id 45134 payload 274 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 -- 172.21.15.17:0/3478171343 >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 msgr2=0x7f8fd00a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 --2- 172.21.15.17:0/3478171343 >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 0x7f8fd00a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 -- 172.21.15.17:0/3478171343 >> v1:172.21.15.119:6789/0 conn(0x7f8fd00a53b0 legacy=0x7f8fd0137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 monclient: found mon.noname-b 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fced9d640 1 -- 172.21.15.17:0/3478171343 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd00a1c90 con 0x7f8fd00a5800 2024-09-15T21:25:10.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.535+0000 7f8fce59c640 1 --2- 172.21.15.17:0/3478171343 >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 0x7f8fd00a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fced9d640 1 --2- 172.21.15.17:0/3478171343 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 0x7f8fd0137f60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00098a0 tx=0x7f8fc001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eba19abdcf039f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 1 -- 172.21.15.17:0/3478171343 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00b0070 con 0x7f8fd00a5800 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: dump: 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 1 -- 172.21.15.17:0/3478171343 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc00ab600 con 0x7f8fd00a5800 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 5 monclient: authenticate success, global_id 45134 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 4 set_mon_vals no callback set 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 1 -- 172.21.15.17:0/3478171343 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00ab9d0 con 0x7f8fd00a5800 2024-09-15T21:25:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fcd59a640 10 monclient: dump: 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 10 monclient: shutdown 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 -- 172.21.15.17:0/3478171343 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 msgr2=0x7f8fd0137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/3478171343 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 0x7f8fd0137f60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00098a0 tx=0x7f8fc001c400 comp rx=0 tx=0).stop 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 -- 172.21.15.17:0/3478171343 shutdown_connections 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/3478171343 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a5800 0x7f8fd0137f60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/3478171343 >> v2:172.21.15.17:3300/0 conn(0x7f8fd00a4a90 0x7f8fd00a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 -- 172.21.15.17:0/3478171343 >> 172.21.15.17:0/3478171343 conn(0x7f8fd00aabb0 msgr2=0x7f8fd00a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 -- 172.21.15.17:0/3478171343 shutdown_connections 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 1 -- 172.21.15.17:0/3478171343 wait complete. 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.539+0000 7f8fd48fd640 10 monclient: monmap: 2024-09-15T21:25:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 Processor -- start 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- start start 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: init 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a4a90 0x7f8fd01cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): picked mon.noname-c con 0x7f8fd00a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd00a53b0 0x7f8fd0137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): picked mon.noname-a con 0x7f8fd00a53b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd00a5800 0x7f8fd013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): picked mon.noname-b con 0x7f8fd00a5800 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0138620 con 0x7f8fd00a53b0 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01384a0 con 0x7f8fd00a5800 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd0142550 con 0x7f8fd00a4a90 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): authenticate will time out at 1433.940846s 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a4a90 0x7f8fd01cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd00a53b0 0x7f8fd0137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a4a90 0x7f8fd01cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39196/0 (socket says 172.21.15.17:39196) 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 -- 172.21.15.17:0/2169453406 learned_addr learned my addr 172.21.15.17:0/2169453406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): get_auth_request con 0x7f8fd00a4a90 auth_method 0 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): get_auth_request con 0x7f8fd00a53b0 auth_method 0 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fced9d640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd00a5800 0x7f8fd013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient(hunting): handle_auth_done global_id 45137 payload 824 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd00a5800 msgr2=0x7f8fd013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd00a5800 0x7f8fd013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd00a53b0 msgr2=0x7f8fd0137630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd00a53b0 0x7f8fd0137630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 monclient: found mon.noname-c 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd0121720 con 0x7f8fd00a4a90 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fced9d640 10 monclient: get_auth_request con 0x7f8fd00a5800 auth_method 0 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fced9d640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd00a5800 0x7f8fd013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd00a53b0 0x7f8fd0137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fce59c640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a4a90 0x7f8fd01cf840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800efc0 tx=0x7f8fc800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8018020 con 0x7f8fd00a4a90 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: dump: 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.562 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.562 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.562 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.562 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.562 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.563 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.563 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc8014680 con 0x7f8fd00a4a90 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8014a50 con 0x7f8fd00a4a90 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 5 monclient: authenticate success, global_id 45137 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 10 monclient: dump: 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: _renew_subs 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fccd99640 4 set_mon_vals no callback set 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd00a9680 con 0x7f8fd00a4a90 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: _renew_subs 2024-09-15T21:25:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd0121fc0 con 0x7f8fd00a4a90 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 0x7f8fd013dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd00a4e70 con 0x7f8fd0137b70 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8fc801b020 con 0x7f8fd00a4a90 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 0x7f8fd013dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient: get_auth_request con 0x7f8fd0137b70 auth_method 0 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fb003b9f0 0x7f8fb003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f8fc8052ae0 con 0x7f8fd00a4a90 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fced9d640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fb003b9f0 0x7f8fb003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fced9d640 10 monclient: get_auth_request con 0x7f8fb003b9f0 auth_method 0 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_done global_id 84475 payload 824 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fd00008e0 con 0x7f8fd0137b70 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.543+0000 7f8fcdd9b640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 0x7f8fd013dd30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8007fd0 tx=0x7f8fb800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80052e0 con 0x7f8fd0137b70 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fced9d640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fb003b9f0 0x7f8fb003deb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00ad000 tx=0x7f8fc00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 10 monclient: dump: 2024-09-15T21:25:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8fb8018070 con 0x7f8fd0137b70 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 msgr2=0x7f8fd013dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 0x7f8fd013dd30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8007fd0 tx=0x7f8fb800ff70 comp rx=0 tx=0).stop 2024-09-15T21:25:10.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 msgr2=0x7f8fd013dd30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:10.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.547+0000 7f8fc6ffd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0137b70 0x7f8fd013dd30 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8007fd0 tx=0x7f8fb800ff70 comp rx=0 tx=0).stop 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fd48fd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 0x7f8fd0138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fd48fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd013e680 con 0x7f8fd013e270 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 0x7f8fd0138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient: get_auth_request con 0x7f8fd013e270 auth_method 0 2024-09-15T21:25:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 10 monclient(hunting): handle_auth_done global_id 84478 payload 824 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 1 -- 172.21.15.17:0/2169453406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8fd01209d0 con 0x7f8fd013e270 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.551+0000 7f8fcdd9b640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 0x7f8fd0138cf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8017000 tx=0x7f8fb8001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb801d9d0 con 0x7f8fd013e270 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 10 monclient: dump: 2024-09-15T21:25:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f8fb801d9d0 con 0x7f8fd013e270 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 msgr2=0x7f8fd0138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 0x7f8fd0138cf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8017000 tx=0x7f8fb8001130 comp rx=0 tx=0).stop 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 msgr2=0x7f8fd0138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fc6ffd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd013e270 0x7f8fd0138cf0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8017000 tx=0x7f8fb8001130 comp rx=0 tx=0).stop 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 76, 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 62783, 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:10.585 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fb003b9f0 msgr2=0x7f8fb003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fb003b9f0 0x7f8fb003deb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00ad000 tx=0x7f8fc00a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 10 monclient: shutdown 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a4a90 msgr2=0x7f8fd01cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 --2- 172.21.15.17:0/2169453406 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd00a4a90 0x7f8fd01cf840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800efc0 tx=0x7f8fc800c490 comp rx=0 tx=0).stop 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fced9d640 1 -- 172.21.15.17:0/2169453406 reap_dead start 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fced9d640 1 -- 172.21.15.17:0/2169453406 reap_dead start 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 shutdown_connections 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 >> 172.21.15.17:0/2169453406 conn(0x7f8fd00aabb0 msgr2=0x7f8fd009e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 shutdown_connections 2024-09-15T21:25:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.555+0000 7f8fd48fd640 1 -- 172.21.15.17:0/2169453406 wait complete. 2024-09-15T21:25:10.588 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:10.588 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:25:10.588 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:25:10.588 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:25:10.588 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:25:10.588 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:25:10.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:25:10.598 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:25:10.598 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:25:10.599 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:10.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient: get_monmap_and_config 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient: monmap: 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:10.752240+0000 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:10.752240+0000 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:10.754 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 1 Processor -- start 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 1 -- start start 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient: init 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42480a4a90 0x7f42480a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient(hunting): picked mon.noname-a con 0x7f42480a4a90 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f42480a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f42480a5800 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42480a2c30 con 0x7f42480a5800 2024-09-15T21:25:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4248138670 con 0x7f42480a53b0 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f42480a0f00 con 0x7f42480a4a90 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424ffae640 10 monclient(hunting): authenticate will time out at 1434.147588s 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424dd4a640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f42480a4a90 msgr2=0x7f42480a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424dd4a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42480a4a90 0x7f42480a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424e54b640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f42480a5800 legacy=0x7f4248137f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424d549640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f42480a53b0 0x7f4248137850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:33822/0 (socket says 172.21.15.17:33822) 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.747+0000 7f424d549640 1 -- 172.21.15.17:0/2728262738 learned_addr learned my addr 172.21.15.17:0/2728262738 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744306056 0 0) 0x7f4248138670 con 0x7f42480a53b0 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): _negotiate 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): my global_id is 84481 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4230003530 con 0x7f42480a53b0 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 35883642 0 0) 0x7f4230003530 con 0x7f42480a53b0 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 >> v2:172.21.15.17:3300/0 conn(0x7f42480a4a90 msgr2=0x7f42480a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 --2- 172.21.15.17:0/2728262738 >> v2:172.21.15.17:3300/0 conn(0x7f42480a4a90 0x7f42480a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 >> v1:172.21.15.17:6789/0 conn(0x7f42480a5800 legacy=0x7f4248137f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42480a19b0 con 0x7f42480a53b0 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f423c001700 con 0x7f42480a53b0 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: dump: 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 >> v1:172.21.15.119:6789/0 conn(0x7f42480a53b0 legacy=0x7f4248137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42300039c0 0x7f4230003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): picked mon.a con 0x7f42300039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4230004810 0x7f4230004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): picked mon.b con 0x7f4230004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42300052f0 0x7f423000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): picked mon.c con 0x7f42300052f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f42480a0f00 con 0x7f42300039c0 2024-09-15T21:25:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f423000c240 con 0x7f4230004810 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f423000c440 con 0x7f42300052f0 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424dd4a640 1 -- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42300039c0 msgr2=0x7f4230003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424dd4a640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42300039c0 0x7f4230003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42300052f0 0x7f423000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): get_auth_request con 0x7f42300052f0 auth_method 0 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4230004810 0x7f4230004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): get_auth_request con 0x7f4230004810 auth_method 0 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424d549640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient(hunting): handle_auth_done global_id 84481 payload 293 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 -- 172.21.15.17:0/2728262738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4230004810 msgr2=0x7f4230004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4230004810 0x7f4230004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 -- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42300039c0 msgr2=0x7f4230003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42300039c0 0x7f4230003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 monclient: found mon.c 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 -- 172.21.15.17:0/2728262738 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4230003530 con 0x7f42300052f0 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424ffae640 5 monclient: authenticate success, global_id 84481 2024-09-15T21:25:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424ffae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 -- 172.21.15.17:0/2728262738 reap_dead start 2024-09-15T21:25:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424e54b640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42300052f0 0x7f423000b9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4238006fd0 tx=0x7f423801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39458b0d2480227 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238009520 con 0x7f42300052f0 2024-09-15T21:25:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: dump: 2024-09-15T21:25:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424ffae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4238003bc0 con 0x7f42300052f0 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424ffae640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424ffae640 4 set_mon_vals no callback set 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 1 -- 172.21.15.17:0/2728262738 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238005070 con 0x7f42300052f0 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.751+0000 7f424cd48640 10 monclient: dump: 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: shutdown 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42300052f0 msgr2=0x7f423000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42300052f0 0x7f423000b9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4238006fd0 tx=0x7f423801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/2728262738 shutdown_connections 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 --2- 172.21.15.17:0/2728262738 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f42300052f0 0x7f423000b9d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/2728262738 >> 172.21.15.17:0/2728262738 conn(0x7f42480aabb0 msgr2=0x7f42480a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/2728262738 shutdown_connections 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/2728262738 wait complete. 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: build_initial_monmap 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: monmap: 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 Processor -- start 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- start start 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: init 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42300052f0 0x7f424813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): picked mon.noname-b con 0x7f42300052f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42481cf830 0x7f42481cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): picked mon.noname-a con 0x7f42481cf830 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 0x7f424813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): picked mon.noname-c con 0x7f424813c890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f424806f2d0 con 0x7f42481cf830 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f424806f150 con 0x7f42300052f0 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f42480a4e70 con 0x7f424813c890 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient(hunting): authenticate will time out at 1434.153899s 2024-09-15T21:25:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424d549640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42481cf830 msgr2=0x7f42481cfc90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424d549640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42481cf830 0x7f42481cfc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 0x7f424813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 0x7f424813cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39216/0 (socket says 172.21.15.17:39216) 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 -- 172.21.15.17:0/4058650188 learned_addr learned my addr 172.21.15.17:0/4058650188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): get_auth_request con 0x7f424813c890 auth_method 0 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424dd4a640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42300052f0 0x7f424813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424dd4a640 10 monclient(hunting): get_auth_request con 0x7f42300052f0 auth_method 0 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424dd4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424dd4a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424dd4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient(hunting): handle_auth_done global_id 45140 payload 824 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 -- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42300052f0 msgr2=0x7f424813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42300052f0 0x7f424813ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 -- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42481cf830 msgr2=0x7f42481cfc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42481cf830 0x7f42481cfc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 monclient: found mon.noname-c 2024-09-15T21:25:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 -- 172.21.15.17:0/4058650188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424813eac0 con 0x7f424813c890 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424e54b640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 0x7f424813cd10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4238006fd0 tx=0x7f4230005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238009500 con 0x7f424813c890 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: dump: 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4238003d70 con 0x7f424813c890 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 5 monclient: authenticate success, global_id 45140 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _renew_subs 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424809ffd0 con 0x7f424813c890 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380ac460 con 0x7f424813c890 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 10 monclient: dump: 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _renew_subs 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4248140660 con 0x7f424813c890 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f4247fff640 4 set_mon_vals no callback set 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42480a8a10 con 0x7f424813c890 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.755+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f42380ac600 con 0x7f424813c890 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.759+0000 7f42467fc640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f423403b9a0 0x7f423403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.759+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f42380eb5a0 con 0x7f424813c890 2024-09-15T21:25:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.759+0000 7f424dd4a640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f423403b9a0 0x7f423403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.759+0000 7f424dd4a640 10 monclient: get_auth_request con 0x7f423403b9a0 auth_method 0 2024-09-15T21:25:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.759+0000 7f424dd4a640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f423403b9a0 0x7f423403de60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4240003d10 tx=0x7f42400094d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.763+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f424809ffd0 con 0x7f424813c890 2024-09-15T21:25:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.763+0000 7f42467fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.763+0000 7f42467fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.991+0000 7f424ffae640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.991+0000 7f424ffae640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.991+0000 7f424ffae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.991+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f42480a1280 con 0x7f424813c890 2024-09-15T21:25:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f42467fc640 1 -- 172.21.15.17:0/4058650188 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f42380160f0 con 0x7f424813c890 2024-09-15T21:25:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f42467fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f42467fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:10.999 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:11.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f423403b9a0 msgr2=0x7f423403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:11.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f423403b9a0 0x7f423403de60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4240003d10 tx=0x7f42400094d0 comp rx=0 tx=0).stop 2024-09-15T21:25:11.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 10 monclient: shutdown 2024-09-15T21:25:11.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:11.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 msgr2=0x7f424813cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 0x7f424813cd10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4238006fd0 tx=0x7f4230005840 comp rx=0 tx=0).stop 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 shutdown_connections 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f423403b9a0 0x7f423403de60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f424813c890 0x7f424813cd10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42481cf830 0x7f42481cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 --2- 172.21.15.17:0/4058650188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42300052f0 0x7f424813ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 >> 172.21.15.17:0/4058650188 conn(0x7f42480aabb0 msgr2=0x7f42480a2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 shutdown_connections 2024-09-15T21:25:11.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:10.995+0000 7f424ffae640 1 -- 172.21.15.17:0/4058650188 wait complete. 2024-09-15T21:25:12.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:12.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:12.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:12.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:12.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:12.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:12.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:12.797+0000 7f5ca57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:13.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:13.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:13.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:13.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:13.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:13.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:14.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.143+0000 7feb79a46640 10 monclient: get_monmap_and_config 2024-09-15T21:25:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.143+0000 7feb79a46640 10 monclient: build_initial_monmap 2024-09-15T21:25:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.143+0000 7feb79a46640 10 monclient: monmap: 2024-09-15T21:25:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:14.149705+0000 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:14.149705+0000 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:14.151 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 Processor -- start 2024-09-15T21:25:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 -- start start 2024-09-15T21:25:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient: init 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7feb7409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb7409def0 0x7feb741398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): picked mon.noname-a con 0x7feb7409def0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb7413c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): picked mon.noname-b con 0x7feb74139de0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb740a3490 con 0x7feb7409d400 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7feb7409f0b0 con 0x7feb7409def0 2024-09-15T21:25:14.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb740a0320 con 0x7feb74139de0 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb79a46640 10 monclient(hunting): authenticate will time out at 1437.545057s 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb6bfff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7feb7409def0 msgr2=0x7feb741398a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb6bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb7409def0 0x7feb741398a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb7413c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb7413c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39218/0 (socket says 172.21.15.17:39218) 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 -- 172.21.15.17:0/2808118341 learned_addr learned my addr 172.21.15.17:0/2808118341 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): get_auth_request con 0x7feb74139de0 auth_method 0 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient(hunting): handle_auth_done global_id 45143 payload 274 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 -- 172.21.15.17:0/2808118341 >> v2:172.21.15.17:3300/0 conn(0x7feb7409def0 msgr2=0x7feb741398a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 --2- 172.21.15.17:0/2808118341 >> v2:172.21.15.17:3300/0 conn(0x7feb7409def0 0x7feb741398a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 -- 172.21.15.17:0/2808118341 >> v1:172.21.15.119:6789/0 conn(0x7feb7409d400 legacy=0x7feb7409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.147+0000 7feb737fe640 1 -- 172.21.15.17:0/2808118341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7409e7a0 con 0x7feb74139de0 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 --2- 172.21.15.17:0/2808118341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb7413c1d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009e50 tx=0x7feb6c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fca638cb9720a443 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 1 -- 172.21.15.17:0/2808118341 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0a57c0 con 0x7feb74139de0 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: dump: 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 5 monclient: authenticate success, global_id 45143 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 1 -- 172.21.15.17:0/2808118341 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c0a5960 con 0x7feb74139de0 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 1 -- 172.21.15.17:0/2808118341 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0a5d30 con 0x7feb74139de0 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 4 set_mon_vals no callback set 2024-09-15T21:25:14.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb727fc640 10 monclient: dump: 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: shutdown 2024-09-15T21:25:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/2808118341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 msgr2=0x7feb7413c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 --2- 172.21.15.17:0/2808118341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb7413c1d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009e50 tx=0x7feb6c01c8e0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/2808118341 shutdown_connections 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 --2- 172.21.15.17:0/2808118341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb7413c1d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 --2- 172.21.15.17:0/2808118341 >> v2:172.21.15.17:3300/0 conn(0x7feb7409def0 0x7feb741398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/2808118341 >> 172.21.15.17:0/2808118341 conn(0x7feb740a66a0 msgr2=0x7feb740a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/2808118341 shutdown_connections 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/2808118341 wait complete. 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: build_initial_monmap 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: monmap: 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 Processor -- start 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- start start 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: init 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb7409d400 0x7feb741cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): picked mon.noname-a con 0x7feb7409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb7409def0 0x7feb74073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): picked mon.noname-b con 0x7feb7409def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb74134130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): picked mon.noname-c con 0x7feb74139de0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb7413ef30 con 0x7feb7409d400 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb740a32a0 con 0x7feb7409def0 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb7413f0b0 con 0x7feb74139de0 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient(hunting): authenticate will time out at 1437.550497s 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb72ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb7409d400 msgr2=0x7feb741cf550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb72ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb7409d400 0x7feb741cf550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb74134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb74134130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39230/0 (socket says 172.21.15.17:39230) 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 -- 172.21.15.17:0/1124959158 learned_addr learned my addr 172.21.15.17:0/1124959158 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): get_auth_request con 0x7feb74139de0 auth_method 0 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:14.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:14.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:14.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient(hunting): handle_auth_done global_id 45146 payload 824 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb6bfff640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb7409def0 0x7feb74073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 -- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb7409def0 msgr2=0x7feb74073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb7409def0 0x7feb74073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 -- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb7409d400 msgr2=0x7feb741cf550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb7409d400 0x7feb741cf550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:14.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 -- 172.21.15.17:0/1124959158 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7409d0f0 con 0x7feb74139de0 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb6bfff640 10 monclient: get_auth_request con 0x7feb7409def0 auth_method 0 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb6bfff640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb7409def0 0x7feb74073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb737fe640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb74134130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb6c01a510 tx=0x7feb6c019680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0bc070 con 0x7feb74139de0 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: dump: 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: _finish_auth 0 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c0b8850 con 0x7feb74139de0 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0b8c40 con 0x7feb74139de0 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 5 monclient: authenticate success, global_id 45146 2024-09-15T21:25:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: _renew_subs 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7413b260 con 0x7feb74139de0 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb70ff9640 10 monclient: dump: 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:14.174 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb71ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: _renew_subs 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.151+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741d03a0 con 0x7feb74139de0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb79a46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb79a46640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb79a46640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7409d7e0 con 0x7feb74139de0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7feb6c0b6410 con 0x7feb74139de0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb70ff9640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb5403b9a0 0x7feb5403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7feb6c0ec440 con 0x7feb74139de0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb72ffd640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb5403b9a0 0x7feb5403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb72ffd640 10 monclient: get_auth_request con 0x7feb5403b9a0 auth_method 0 2024-09-15T21:25:14.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.155+0000 7feb72ffd640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb5403b9a0 0x7feb5403de60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c10 tx=0x7feb64007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:14.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.159+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7feb6c0c5020 con 0x7feb74139de0 2024-09-15T21:25:14.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.159+0000 7feb70ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:14.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.159+0000 7feb70ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb79a46640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb79a46640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb79a46640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7feb74134670 con 0x7feb74139de0 2024-09-15T21:25:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb70ff9640 1 -- 172.21.15.17:0/1124959158 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7feb6c016620 con 0x7feb74139de0 2024-09-15T21:25:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb70ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.391+0000 7feb70ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:14.398 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb5403b9a0 msgr2=0x7feb5403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb5403b9a0 0x7feb5403de60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c10 tx=0x7feb64007a20 comp rx=0 tx=0).stop 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 10 monclient: shutdown 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 msgr2=0x7feb74134130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb74134130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb6c01a510 tx=0x7feb6c019680 comp rx=0 tx=0).stop 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 shutdown_connections 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb5403b9a0 0x7feb5403de60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb74139de0 0x7feb74134130 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb7409def0 0x7feb74073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 --2- 172.21.15.17:0/1124959158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb7409d400 0x7feb741cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 >> 172.21.15.17:0/1124959158 conn(0x7feb740a66a0 msgr2=0x7feb740a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 shutdown_connections 2024-09-15T21:25:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:14.395+0000 7feb79a46640 1 -- 172.21.15.17:0/1124959158 wait complete. 2024-09-15T21:25:14.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:14.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:14.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:14.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:14.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:14.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:15.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:15.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:15.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:15.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:15.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:15.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:16.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:16.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:16.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:16.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:16.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:16.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:17.416 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient: get_monmap_and_config 2024-09-15T21:25:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient: build_initial_monmap 2024-09-15T21:25:17.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient: monmap: 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:17.547412+0000 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:17.547412+0000 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:17.549 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:17.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 Processor -- start 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 -- start start 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient: init 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c040a0890 0x7f4c040a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): picked mon.noname-a con 0x7f4c040a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4c0409ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 0x7f4c04135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): picked mon.noname-b con 0x7f4c040a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c040a3130 con 0x7f4c0409ffc0 2024-09-15T21:25:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4c04073df0 con 0x7f4c040a0890 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4c040a1bf0 con 0x7f4c040a11b0 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c0b756640 10 monclient(hunting): authenticate will time out at 1440.943584s 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c08cf1640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f4c0409ffc0 legacy=0x7f4c04135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c094f2640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f4c040a0890 msgr2=0x7f4c040a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c094f2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c040a0890 0x7f4c040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 0x7f4c04135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 0x7f4c04135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39240/0 (socket says 172.21.15.17:39240) 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 1 -- 172.21.15.17:0/3387779642 learned_addr learned my addr 172.21.15.17:0/3387779642 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 10 monclient(hunting): get_auth_request con 0x7f4c040a11b0 auth_method 0 2024-09-15T21:25:17.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.543+0000 7f4c09cf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 10 monclient(hunting): handle_auth_done global_id 45149 payload 274 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 1 -- 172.21.15.17:0/3387779642 >> v2:172.21.15.17:3300/0 conn(0x7f4c040a0890 msgr2=0x7f4c040a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 1 --2- 172.21.15.17:0/3387779642 >> v2:172.21.15.17:3300/0 conn(0x7f4c040a0890 0x7f4c040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 1 -- 172.21.15.17:0/3387779642 >> v1:172.21.15.17:6789/0 conn(0x7f4c0409ffc0 legacy=0x7f4c04135670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 10 monclient: _finish_hunting 0 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 1 monclient: found mon.noname-b 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 1 -- 172.21.15.17:0/3387779642 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c040a15e0 con 0x7f4c040a11b0 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c09cf3640 1 --2- 172.21.15.17:0/3387779642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 0x7f4c04135d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0098a0 tx=0x7f4bfc01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6b203612770956 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 1 -- 172.21.15.17:0/3387779642 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0b0070 con 0x7f4c040a11b0 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: dump: 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: _finish_auth 0 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 5 monclient: authenticate success, global_id 45149 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 1 -- 172.21.15.17:0/3387779642 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bfc0ab600 con 0x7f4c040a11b0 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 1 -- 172.21.15.17:0/3387779642 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0ab9d0 con 0x7f4c040a11b0 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4bfbfff640 10 monclient: dump: 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:17.564 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 4 set_mon_vals no callback set 2024-09-15T21:25:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 10 monclient: shutdown 2024-09-15T21:25:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:17.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 -- 172.21.15.17:0/3387779642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 msgr2=0x7f4c04135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 --2- 172.21.15.17:0/3387779642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 0x7f4c04135d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0098a0 tx=0x7f4bfc01c420 comp rx=0 tx=0).stop 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 -- 172.21.15.17:0/3387779642 shutdown_connections 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 --2- 172.21.15.17:0/3387779642 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c040a11b0 0x7f4c04135d80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 --2- 172.21.15.17:0/3387779642 >> v2:172.21.15.17:3300/0 conn(0x7f4c040a0890 0x7f4c040a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 -- 172.21.15.17:0/3387779642 >> 172.21.15.17:0/3387779642 conn(0x7f4c040a66a0 msgr2=0x7f4c040a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 -- 172.21.15.17:0/3387779642 shutdown_connections 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 1 -- 172.21.15.17:0/3387779642 wait complete. 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 10 monclient: build_initial_monmap 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.547+0000 7f4c0b756640 10 monclient: monmap: 2024-09-15T21:25:17.565 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 Processor -- start 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- start start 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: init 2024-09-15T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 0x7f4c041cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): picked mon.noname-c con 0x7f4c0409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c040a0890 0x7f4c04135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): picked mon.noname-a con 0x7f4c040a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c040a11b0 0x7f4c0411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): picked mon.noname-b con 0x7f4c040a11b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04140510 con 0x7f4c040a0890 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041364d0 con 0x7f4c040a11b0 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4c04140690 con 0x7f4c0409ffc0 2024-09-15T21:25:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient(hunting): authenticate will time out at 1440.948966s 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c08cf1640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c040a0890 msgr2=0x7f4c04135390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c08cf1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c040a0890 0x7f4c04135390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 0x7f4c041cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 0x7f4c041cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39242/0 (socket says 172.21.15.17:39242) 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 -- 172.21.15.17:0/4073267604 learned_addr learned my addr 172.21.15.17:0/4073267604 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): get_auth_request con 0x7f4c0409ffc0 auth_method 0 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient(hunting): handle_auth_done global_id 45152 payload 824 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 -- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c040a11b0 msgr2=0x7f4c0411c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c040a11b0 0x7f4c0411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 -- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c040a0890 msgr2=0x7f4c04135390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c040a0890 0x7f4c04135390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient: _finish_hunting 0 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 monclient: found mon.noname-c 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:17.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 -- 172.21.15.17:0/4073267604 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0411fa40 con 0x7f4c0409ffc0 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c094f2640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 0x7f4c041cd610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000b9e0 tx=0x7f4c0000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000cba0 con 0x7f4c0409ffc0 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: dump: 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 5 monclient: authenticate success, global_id 45152 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _renew_subs 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:17.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0411fd20 con 0x7f4c0409ffc0 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c00010070 con 0x7f4c0409ffc0 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00015470 con 0x7f4c0409ffc0 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 10 monclient: dump: 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _renew_subs 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0411e520 con 0x7f4c0409ffc0 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bfb7fe640 4 set_mon_vals no callback set 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c040a0c70 con 0x7f4c0409ffc0 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4c0001e3a0 con 0x7f4c0409ffc0 2024-09-15T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4be803b970 0x7f4be803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7f4c00051e30 con 0x7f4c0409ffc0 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c08cf1640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4be803b970 0x7f4be803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.551+0000 7f4c08cf1640 10 monclient: get_auth_request con 0x7f4be803b970 auth_method 0 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.555+0000 7f4c08cf1640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4be803b970 0x7f4be803de30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c0411e000 tx=0x7f4bf4006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.555+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4c0004e7c0 con 0x7f4c0409ffc0 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.555+0000 7f4bf9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.555+0000 7f4bf9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4c0b756640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:17.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4c0b756640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:17.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4c0b756640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:17.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4c0413cad0 con 0x7f4c0409ffc0 2024-09-15T21:25:17.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:17.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:17.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:17.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:17.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:17.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4bf9ffb640 1 -- 172.21.15.17:0/4073267604 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f4c00005a80 con 0x7f4c0409ffc0 2024-09-15T21:25:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4bf9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.795+0000 7f4bf9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:17.802 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4be803b970 msgr2=0x7f4be803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4be803b970 0x7f4be803de30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c0411e000 tx=0x7f4bf4006d90 comp rx=0 tx=0).stop 2024-09-15T21:25:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 10 monclient: shutdown 2024-09-15T21:25:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 msgr2=0x7f4c041cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 0x7f4c041cd610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000b9e0 tx=0x7f4c0000bea0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 shutdown_connections 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4be803b970 0x7f4be803de30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c040a11b0 0x7f4c0411c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c040a0890 0x7f4c04135390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 --2- 172.21.15.17:0/4073267604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4c0409ffc0 0x7f4c041cd610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 >> 172.21.15.17:0/4073267604 conn(0x7f4c040a66a0 msgr2=0x7f4c0406fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 shutdown_connections 2024-09-15T21:25:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:17.799+0000 7f4c0b756640 1 -- 172.21.15.17:0/4073267604 wait complete. 2024-09-15T21:25:18.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:18.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:18.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:18.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:18.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:18.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:19.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:19.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:19.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:19.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:19.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:19.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:20.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:20.801+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:20.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:20.801+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:20.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:20.801+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:20.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:20.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient: get_monmap_and_config 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient: build_initial_monmap 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient: monmap: 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:20.952105+0000 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:20.952105+0000 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:20.954 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 Processor -- start 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 -- start start 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient: init 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:20.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): picked mon.noname-c con 0x7fcf9009d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): picked mon.noname-b con 0x7fcf9009dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcf90134ba0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf900a3130 con 0x7fcf90134ba0 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcf900a1bc0 con 0x7fcf9009d400 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcf900a3490 con 0x7fcf9009dd20 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.947+0000 7fcf96612640 10 monclient(hunting): authenticate will time out at 1444.347965s 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf94baf640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fcf90134ba0 legacy=0x7fcf9013bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55774/0 (socket says 172.21.15.17:55774) 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 learned_addr learned my addr 172.21.15.17:0/1898643125 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request con 0x7fcf9009dd20 auth_method 0 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request con 0x7fcf9009d400 auth_method 0 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:20.951+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request con 0x7fcf9009d400 auth_method 0 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf94baf640 1 -- 172.21.15.17:0/1898643125 >> v1:172.21.15.17:6789/0 conn(0x7fcf90134ba0 legacy=0x7fcf9013bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request con 0x7fcf9009dd20 auth_method 0 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.155+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:25:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf94baf640 1 -- 172.21.15.17:0/1898643125 >> v1:172.21.15.17:6789/0 conn(0x7fcf90134ba0 legacy=0x7fcf9013bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request con 0x7fcf9009d400 auth_method 0 2024-09-15T21:25:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request con 0x7fcf9009dd20 auth_method 0 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:21.559+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:25:21.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:21.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:21.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:21.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:21.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:21.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf94baf640 1 -- 172.21.15.17:0/1898643125 >> v1:172.21.15.17:6789/0 conn(0x7fcf90134ba0 legacy=0x7fcf9013bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request con 0x7fcf9009d400 auth_method 0 2024-09-15T21:25:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request con 0x7fcf9009dd20 auth_method 0 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:25:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:22.363+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:25:22.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:22.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:22.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:22.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:22.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:22.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:22.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:22.797+0000 7f5ca57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:23.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:23.797+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:23.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:23.797+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:23.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:23.797+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.947+0000 7fcf8dffb640 10 monclient(hunting): tick 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 monclient(hunting): continuing hunt 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 msgr2=0x7fcf90134660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf9009dd20 0x7fcf90134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 msgr2=0x7fcf9009d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.119:3300/0 conn(0x7fcf9009d400 0x7fcf9009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 -- 172.21.15.17:0/1898643125 >> v1:172.21.15.17:6789/0 conn(0x7fcf90134ba0 legacy=0x7fcf9013bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient: _add_conns ranks=[1,3] 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcf7c0058c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 0x7fcf7c00bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient(hunting): picked mon.noname-a con 0x7fcf7c0095f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 -- 172.21.15.17:0/1898643125 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9009e7a0 con 0x7fcf7c0058c0 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 1 -- 172.21.15.17:0/1898643125 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcf900a3490 con 0x7fcf7c0095f0 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8dffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 msgr2=0x7fcf7c00bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:23.951+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 0x7fcf7c00bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:24.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:24.155+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 msgr2=0x7fcf7c00bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:24.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:24.155+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 0x7fcf7c00bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:25:24.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:24.559+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 msgr2=0x7fcf7c00bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:24.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:24.559+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 0x7fcf7c00bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:25:24.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:24.801+0000 7f5ca57fa640 10 monclient: tick 2024-09-15T21:25:24.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:24.801+0000 7f5ca57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:24.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:24.801+0000 7f5ca57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.363+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 msgr2=0x7fcf7c00bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.363+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 0x7fcf7c00bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:25:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.567+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 267577217 0 0) 0x7fcf9009e7a0 con 0x7fcf7c0058c0 2024-09-15T21:25:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.567+0000 7fcf8effd640 10 monclient(hunting): _negotiate 2024-09-15T21:25:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.567+0000 7fcf8effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.567+0000 7fcf8effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.567+0000 7fcf8effd640 10 monclient(hunting): my global_id is 84487 2024-09-15T21:25:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.567+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf78003530 con 0x7fcf7c0058c0 2024-09-15T21:25:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1820166660 0 0) 0x7fcf78003530 con 0x7fcf7c0058c0 2024-09-15T21:25:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 msgr2=0x7fcf7c00bab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 --2- 172.21.15.17:0/1898643125 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c0095f0 0x7fcf7c00bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf900a3130 con 0x7fcf7c0058c0 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fcf80009980 con 0x7fcf7c0058c0 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: dump: 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.585 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 >> v1:172.21.15.119:6789/0 conn(0x7fcf7c0058c0 legacy=0x7fcf7c007d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf94baf640 1 -- 172.21.15.17:0/1898643125 reap_dead start 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf780039c0 0x7fcf78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): picked mon.c con 0x7fcf780039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf78004810 0x7fcf78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): picked mon.b con 0x7fcf78004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf7800b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): picked mon.a con 0x7fcf78005380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcf900a3490 con 0x7fcf78005380 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7800c2d0 con 0x7fcf78004810 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7800c4d0 con 0x7fcf780039c0 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf8effd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.575+0000 7fcf94baf640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 msgr2=0x7fcf7800b9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf94baf640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf7800b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf780039c0 0x7fcf78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request con 0x7fcf780039c0 auth_method 0 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf78004810 0x7fcf78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request con 0x7fcf78004810 auth_method 0 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_done global_id 84487 payload 293 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf78004810 msgr2=0x7fcf78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf78004810 0x7fcf78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 msgr2=0x7fcf7800b9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf7800b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 monclient: found mon.c 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 1 -- 172.21.15.17:0/1898643125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf78003530 con 0x7fcf780039c0 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf96612640 5 monclient: authenticate success, global_id 84487 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf96612640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:25.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.579+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf78004810 0x7fcf78004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:25.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf780039c0 0x7fcf78003dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009610 tx=0x7fcf80020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93aa37a749b12fee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf800af070 con 0x7fcf780039c0 2024-09-15T21:25:25.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:25.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: dump: 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf96612640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf800a9c70 con 0x7fcf780039c0 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:25.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf96612640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 1 -- 172.21.15.17:0/1898643125 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80003b80 con 0x7fcf780039c0 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf96612640 4 set_mon_vals no callback set 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.583+0000 7fcf8effd640 10 monclient: dump: 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.594 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 10 monclient: shutdown 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 -- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf780039c0 msgr2=0x7fcf78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf780039c0 0x7fcf78003dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009610 tx=0x7fcf80020000 comp rx=0 tx=0).stop 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 -- 172.21.15.17:0/1898643125 shutdown_connections 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf7800b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf78004810 0x7fcf78004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 --2- 172.21.15.17:0/1898643125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf780039c0 0x7fcf78003dc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 -- 172.21.15.17:0/1898643125 >> 172.21.15.17:0/1898643125 conn(0x7fcf900a66a0 msgr2=0x7fcf900a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 -- 172.21.15.17:0/1898643125 shutdown_connections 2024-09-15T21:25:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 -- 172.21.15.17:0/1898643125 wait complete. 2024-09-15T21:25:25.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 10 monclient: build_initial_monmap 2024-09-15T21:25:25.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 10 monclient: monmap: 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 Processor -- start 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 1 -- start start 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.587+0000 7fcf96612640 10 monclient: init 2024-09-15T21:25:25.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf780039c0 0x7fcf90122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): picked mon.noname-b con 0x7fcf780039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 0x7fcf90123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): picked mon.noname-c con 0x7fcf78004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf90123880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): picked mon.noname-a con 0x7fcf78005380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9009ccd0 con 0x7fcf78005380 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9009e160 con 0x7fcf780039c0 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9009ce50 con 0x7fcf78004810 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient(hunting): authenticate will time out at 1448.988290s 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf94baf640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 msgr2=0x7fcf90123880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf94baf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf90123880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 0x7fcf90123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 0x7fcf90123340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55824/0 (socket says 172.21.15.17:55824) 2024-09-15T21:25:25.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/3528222282 learned_addr learned my addr 172.21.15.17:0/3528222282 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request con 0x7fcf78004810 auth_method 0 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf780039c0 0x7fcf90122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request con 0x7fcf780039c0 auth_method 0 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient(hunting): handle_auth_done global_id 45155 payload 824 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf780039c0 msgr2=0x7fcf90122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf780039c0 0x7fcf90122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 msgr2=0x7fcf90123880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf90123880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 -- 172.21.15.17:0/3528222282 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90126940 con 0x7fcf78004810 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8f7fe640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 0x7fcf90123340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf84017a80 tx=0x7fcf84017f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84015ae0 con 0x7fcf78004810 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: dump: 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 5 monclient: authenticate success, global_id 45155 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _renew_subs 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf901270d0 con 0x7fcf78004810 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf84015c80 con 0x7fcf78004810 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84018650 con 0x7fcf78004810 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 10 monclient: dump: 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.607 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _renew_subs 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf900a4510 con 0x7fcf78004810 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8e7fc640 4 set_mon_vals no callback set 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcf840187f0 con 0x7fcf78004810 2024-09-15T21:25:25.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9009fa20 con 0x7fcf78004810 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf7c03c090 0x7fcf7c03e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 13094+0+0 (secure 0 0 0) 0x7fcf84052de0 con 0x7fcf78004810 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf7c03c090 0x7fcf7c03e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.591+0000 7fcf8ffff640 10 monclient: get_auth_request con 0x7fcf7c03c090 auth_method 0 2024-09-15T21:25:25.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.595+0000 7fcf8ffff640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf7c03c090 0x7fcf7c03e550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcf800b4000 tx=0x7fcf80004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.599+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcf8401f380 con 0x7fcf78004810 2024-09-15T21:25:25.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.599+0000 7fcf8cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.599+0000 7fcf8cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:25.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.629+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 removed item id -46 name 'h10' from crush map v228) ==== 120+0+0 (secure 0 0 0) 0x7f5ca001cd70 con 0x7f5cb01324a0 2024-09-15T21:25:25.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.629+0000 7f5ca67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T21:25:25.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.629+0000 7f5ca67fc640 10 monclient: _finish_command 2 = system:0 removed item id -46 name 'h10' from crush map 2024-09-15T21:25:25.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T21:25:25.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T21:25:25.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) -- 0x7f5cb013d140 con 0x7f5cb01324a0 2024-09-15T21:25:25.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5ca67fc640 1 -- 172.21.15.119:0/358562790 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 device 'h10' does not appear in the crush map v228) ==== 120+0+0 (secure 0 0 0) 0x7f5ca001cad0 con 0x7f5cb01324a0 2024-09-15T21:25:25.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5ca67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T21:25:25.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5ca67fc640 10 monclient: _finish_command 3 = system:0 device 'h10' does not appear in the crush map 2024-09-15T21:25:25.745 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h10' does not appear in the crush map 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c9403b9c0 msgr2=0x7f5c9403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c9403b9c0 0x7f5c9403de80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0ad000 tx=0x7f5cac008040 comp rx=0 tx=0).stop 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 10 monclient: shutdown 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 msgr2=0x7f5cb0131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0131960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d7d0 tx=0x7f5ca000dc90 comp rx=0 tx=0).stop 2024-09-15T21:25:25.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 shutdown_connections 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c9403b9c0 0x7f5c9403de80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cb0132dc0 0x7f5cb01cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cb01324a0 0x7f5cb0131960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 --2- 172.21.15.119:0/358562790 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cb00739b0 0x7f5cb0131420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 >> 172.21.15.119:0/358562790 conn(0x7f5cb006cbf0 msgr2=0x7f5cb009eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 shutdown_connections 2024-09-15T21:25:25.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.741+0000 7f5cb7b89640 1 -- 172.21.15.119:0/358562790 wait complete. 2024-09-15T21:25:25.762 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:25.762 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h11 2024-09-15T21:25:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf96612640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf96612640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf96612640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcf90125db0 con 0x7fcf78004810 2024-09-15T21:25:25.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf8cff9640 1 -- 172.21.15.17:0/3528222282 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7fcf8402a440 con 0x7fcf78004810 2024-09-15T21:25:25.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf8cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:25.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.835+0000 7fcf8cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:25.841 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf7c03c090 msgr2=0x7fcf7c03e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf7c03c090 0x7fcf7c03e550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcf800b4000 tx=0x7fcf80004ac0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 10 monclient: shutdown 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 msgr2=0x7fcf90123340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 0x7fcf90123340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf84017a80 tx=0x7fcf84017f40 comp rx=0 tx=0).stop 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 shutdown_connections 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf7c03c090 0x7fcf7c03e550 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf78005380 0x7fcf90123880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcf78004810 0x7fcf90123340 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 --2- 172.21.15.17:0/3528222282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf780039c0 0x7fcf90122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 >> 172.21.15.17:0/3528222282 conn(0x7fcf900a66a0 msgr2=0x7fcf90133e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 shutdown_connections 2024-09-15T21:25:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.839+0000 7fcf96612640 1 -- 172.21.15.17:0/3528222282 wait complete. 2024-09-15T21:25:25.859 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:25:25.859 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:25:25.860 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient: get_monmap_and_config 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient: build_initial_monmap 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient: monmap: 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:25.882388+0000 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:25.882388+0000 2024-09-15T21:25:25.881 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:25.882 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:25.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 Processor -- start 2024-09-15T21:25:25.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 -- start start 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient: init 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d58135ca0 0x7f8d58136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): picked mon.noname-a con 0x7f8d58135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d580a0160 0x7f8d580a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): picked mon.noname-c con 0x7f8d580a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d5806ec30 0x7f8d580a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): picked mon.noname-b con 0x7f8d5806ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.885 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8d581365c0 con 0x7f8d58135ca0 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5809cf30 con 0x7f8d580a0160 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8d580a1ca0 con 0x7f8d5806ec30 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d603ec640 10 monclient(hunting): authenticate will time out at 1444.679124s 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e188640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f8d58135ca0 msgr2=0x7f8d58136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e188640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d58135ca0 0x7f8d58136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d580a0160 0x7f8d580a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d580a0160 0x7f8d580a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41888/0 (socket says 172.21.15.119:41888) 2024-09-15T21:25:25.886 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 -- 172.21.15.119:0/2913318768 learned_addr learned my addr 172.21.15.119:0/2913318768 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): get_auth_request con 0x7f8d580a0160 auth_method 0 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 1 --2- 172.21.15.119:0/2913318768 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d5806ec30 0x7f8d580a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): get_auth_request con 0x7f8d5806ec30 auth_method 0 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5e989640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient(hunting): handle_auth_done global_id 84493 payload 274 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 -- 172.21.15.119:0/2913318768 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d5806ec30 msgr2=0x7f8d580a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 --2- 172.21.15.119:0/2913318768 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d5806ec30 0x7f8d580a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 -- 172.21.15.119:0/2913318768 >> v2:172.21.15.17:3300/0 conn(0x7f8d58135ca0 msgr2=0x7f8d58136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 --2- 172.21.15.119:0/2913318768 >> v2:172.21.15.17:3300/0 conn(0x7f8d58135ca0 0x7f8d58136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 monclient: found mon.noname-c 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.877+0000 7f8d5d987640 1 -- 172.21.15.119:0/2913318768 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d580a1f80 con 0x7f8d580a0160 2024-09-15T21:25:25.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 --2- 172.21.15.119:0/2913318768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d580a09e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d581367a0 tx=0x7f8d5401c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65557edf8aa4bb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 1 -- 172.21.15.119:0/2913318768 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540b0070 con 0x7f8d580a0160 2024-09-15T21:25:25.893 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: dump: 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.894 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 1 -- 172.21.15.119:0/2913318768 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d540a4e50 con 0x7f8d580a0160 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 5 monclient: authenticate success, global_id 84493 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 4 set_mon_vals no callback set 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 1 -- 172.21.15.119:0/2913318768 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540ab820 con 0x7f8d580a0160 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d186640 10 monclient: dump: 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:25.896 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: shutdown 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- 172.21.15.119:0/2913318768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 msgr2=0x7f8d580a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- 172.21.15.119:0/2913318768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d580a09e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d581367a0 tx=0x7f8d5401c400 comp rx=0 tx=0).stop 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- 172.21.15.119:0/2913318768 shutdown_connections 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- 172.21.15.119:0/2913318768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d580a09e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- 172.21.15.119:0/2913318768 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d5806ec30 0x7f8d580a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- 172.21.15.119:0/2913318768 >> v2:172.21.15.17:3300/0 conn(0x7f8d58135ca0 0x7f8d58136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- 172.21.15.119:0/2913318768 >> 172.21.15.119:0/2913318768 conn(0x7f8d580a2ae0 msgr2=0x7f8d580a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:25.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- 172.21.15.119:0/2913318768 shutdown_connections 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- 172.21.15.119:0/2913318768 wait complete. 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: build_initial_monmap 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: monmap: 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:25.898 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 Processor -- start 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- start start 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: init 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d5806ec30 0x7f8d581cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): picked mon.noname-a con 0x7f8d5806ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d5813bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): picked mon.noname-b con 0x7f8d580a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d58135ca0 0x7f8d5813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): picked mon.noname-c con 0x7f8d58135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5813f230 con 0x7f8d5806ec30 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d580a1870 con 0x7f8d580a0160 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5813f3b0 con 0x7f8d58135ca0 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d603ec640 10 monclient(hunting): authenticate will time out at 1444.687188s 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e188640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d5806ec30 msgr2=0x7f8d581cd700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e188640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d5806ec30 0x7f8d581cd700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:25.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d5813bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d5813bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41902/0 (socket says 172.21.15.119:41902) 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 -- 172.21.15.119:0/85510040 learned_addr learned my addr 172.21.15.119:0/85510040 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): get_auth_request con 0x7f8d580a0160 auth_method 0 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d58135ca0 0x7f8d5813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): get_auth_request con 0x7f8d58135ca0 auth_method 0 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5e989640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient(hunting): handle_auth_done global_id 84496 payload 824 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 -- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d58135ca0 msgr2=0x7f8d5813c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d58135ca0 0x7f8d5813c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 -- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d5806ec30 msgr2=0x7f8d581cd700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d5806ec30 0x7f8d581cd700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 monclient: found mon.noname-b 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 -- 172.21.15.119:0/85510040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5809d1d0 con 0x7f8d580a0160 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.885+0000 7f8d5d987640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d5813bb70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d540ae040 tx=0x7f8d5401ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540ba020 con 0x7f8d580a0160 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: dump: 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.903 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d540b6420 con 0x7f8d580a0160 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 5 monclient: authenticate success, global_id 84496 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 monclient: _renew_subs 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5813c5f0 con 0x7f8d580a0160 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540b0040 con 0x7f8d580a0160 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 10 monclient: dump: 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:25.905 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d5c985640 4 set_mon_vals no callback set 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 monclient: _renew_subs 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5813a640 con 0x7f8d580a0160 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d58136080 con 0x7f8d580a0160 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8d540bd020 con 0x7f8d580a0160 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d3c03b970 0x7f8d3c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 12881+0+0 (secure 0 0 0) 0x7f8d540ebdd0 con 0x7f8d580a0160 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d5e188640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d3c03b970 0x7f8d3c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d5e188640 10 monclient: get_auth_request con 0x7f8d3c03b970 auth_method 0 2024-09-15T21:25:25.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.889+0000 7f8d5e188640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d3c03b970 0x7f8d3c03de30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d48003940 tx=0x7f8d48006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.893+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8d540c4030 con 0x7f8d580a0160 2024-09-15T21:25:25.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.893+0000 7f8d4effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:25.893+0000 7f8d4effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.963+0000 7f0e21d04640 10 monclient: get_monmap_and_config 2024-09-15T21:25:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.963+0000 7f0e21d04640 10 monclient: build_initial_monmap 2024-09-15T21:25:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.963+0000 7f0e21d04640 10 monclient: monmap: 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:25.970134+0000 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:25.970134+0000 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:25.971 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 Processor -- start 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 -- start start 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient: init 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient: _add_conns ranks=[4,0,3,2,1] 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e1c0a4a90 0x7f0e1c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): picked mon.noname-c con 0x7f0e1c0a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0e1c0a53b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e1c0a5800 0x7f0e1c137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): picked mon.noname-a con 0x7f0e1c0a5800 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c0a2c30 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0a5cb0 con 0x7f0e1c0a5800 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0a0f00 con 0x7f0e1c0a4a90 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e21d04640 10 monclient(hunting): authenticate will time out at 1449.365697s 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1bfff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f0e1c0a5800 msgr2=0x7f0e1c137f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e1c0a5800 0x7f0e1c137f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1affd640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f0e1c0a53b0 legacy=0x7f0e1c137850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e1c0a4a90 0x7f0e1c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e1c0a4a90 0x7f0e1c0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:45542/0 (socket says 172.21.15.17:45542) 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 -- 172.21.15.17:0/3327193192 learned_addr learned my addr 172.21.15.17:0/3327193192 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): get_auth_request con 0x7f0e1c0a4a90 auth_method 0 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_done global_id 84499 payload 274 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 -- 172.21.15.17:0/3327193192 >> v2:172.21.15.17:3300/0 conn(0x7f0e1c0a5800 msgr2=0x7f0e1c137f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/3327193192 >> v2:172.21.15.17:3300/0 conn(0x7f0e1c0a5800 0x7f0e1c137f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 -- 172.21.15.17:0/3327193192 >> v1:172.21.15.17:6789/0 conn(0x7f0e1c0a53b0 legacy=0x7f0e1c137850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.967+0000 7f0e1b7fe640 1 -- 172.21.15.17:0/3327193192 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0a19b0 con 0x7f0e1c0a4a90 2024-09-15T21:25:25.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/3327193192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 0x7f0e1c0a4e70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009e50 tx=0x7f0e0c01c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92744f9fda898f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 1 -- 172.21.15.17:0/3327193192 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0a55d0 con 0x7f0e1c0a4a90 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: dump: 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.983 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 5 monclient: authenticate success, global_id 84499 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 1 -- 172.21.15.17:0/3327193192 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0a5770 con 0x7f0e1c0a4a90 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 1 -- 172.21.15.17:0/3327193192 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0a5b40 con 0x7f0e1c0a4a90 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1a7fc640 10 monclient: dump: 2024-09-15T21:25:25.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 4 set_mon_vals no callback set 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: shutdown 2024-09-15T21:25:25.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/3327193192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 msgr2=0x7f0e1c0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 --2- 172.21.15.17:0/3327193192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 0x7f0e1c0a4e70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009e50 tx=0x7f0e0c01c750 comp rx=0 tx=0).stop 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/3327193192 shutdown_connections 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 --2- 172.21.15.17:0/3327193192 >> v2:172.21.15.17:3300/0 conn(0x7f0e1c0a5800 0x7f0e1c137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 --2- 172.21.15.17:0/3327193192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 0x7f0e1c0a4e70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/3327193192 >> 172.21.15.17:0/3327193192 conn(0x7f0e1c0aabb0 msgr2=0x7f0e1c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/3327193192 shutdown_connections 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/3327193192 wait complete. 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: build_initial_monmap 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: monmap: 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:25.986 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 Processor -- start 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- start start 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: init 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 0x7f0e1c1cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): picked mon.noname-b con 0x7f0e1c0a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:25.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a53b0 0x7f0e1c1cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): picked mon.noname-c con 0x7f0e1c0a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c0a5800 0x7f0e1c13da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): picked mon.noname-a con 0x7f0e1c0a5800 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c06f2d0 con 0x7f0e1c0a5800 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c06f150 con 0x7f0e1c0a4a90 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c1384a0 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient(hunting): authenticate will time out at 1449.370676s 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1bfff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c0a5800 msgr2=0x7f0e1c13da00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:25.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c0a5800 0x7f0e1c13da00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a53b0 0x7f0e1c1cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a53b0 0x7f0e1c1cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55834/0 (socket says 172.21.15.17:55834) 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 -- 172.21.15.17:0/1099943746 learned_addr learned my addr 172.21.15.17:0/1099943746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): get_auth_request con 0x7f0e1c0a53b0 auth_method 0 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 0x7f0e1c1cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): get_auth_request con 0x7f0e1c0a4a90 auth_method 0 2024-09-15T21:25:25.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient(hunting): handle_auth_done global_id 45164 payload 824 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 msgr2=0x7f0e1c1cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c0a4a90 0x7f0e1c1cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c0a5800 msgr2=0x7f0e1c13da00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e1c0a5800 0x7f0e1c13da00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 monclient: found mon.noname-c 2024-09-15T21:25:25.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0a1280 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e1affd640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a53b0 0x7f0e1c1cfec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400b580 tx=0x7f0e0400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04016070 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: dump: 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.991 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 5 monclient: authenticate success, global_id 45164 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: _renew_subs 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c121e20 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e04003df0 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:25.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04005310 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e13fff640 10 monclient: dump: 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e197fa640 4 set_mon_vals no callback set 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: _renew_subs 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.971+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c13f3f0 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e21d04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e21d04640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 0x7f0e1c13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c1219b0 con 0x7f0e1c1d0400 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 0x7f0e1c13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient: get_auth_request con 0x7f0e1c1d0400 auth_method 0 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:25.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0e040054b0 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_done global_id 45167 payload 824 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c0a4e70 con 0x7f0e1c1d0400 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(228..228 src has 1..228) ==== 12881+0+0 (secure 0 0 0) 0x7f0e04050f40 con 0x7f0e1c0a53b0 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 0x7f0e1c13df40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0a7900 tx=0x7f0e0c0194f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1bfff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c009510 con 0x7f0e1c1d0400 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:25.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 10 monclient: dump: 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1bfff640 10 monclient: get_auth_request con 0x7f0dfc03b970 auth_method 0 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0e0c0c0670 con 0x7f0e1c1d0400 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:25.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:25.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 msgr2=0x7f0e1c13df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:25.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 0x7f0e1c13df40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0a7900 tx=0x7f0e0c0194f0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 msgr2=0x7f0e1c13df40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:25.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e13fff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c1d0400 0x7f0e1c13df40 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0a7900 tx=0x7f0e0c0194f0 comp rx=0 tx=0).stop 2024-09-15T21:25:25.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.975+0000 7f0e1bfff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c120720 tx=0x7f0e14006cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e21d04640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e21d04640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 0x7f0e1c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e21d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c13e890 con 0x7f0e1c13e480 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 0x7f0e1c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient: get_auth_request con 0x7f0e1c13e480 auth_method 0 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 10 monclient(hunting): handle_auth_done global_id 45170 payload 824 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 1 -- 172.21.15.17:0/1099943746 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e1c137db0 con 0x7f0e1c13e480 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e1b7fe640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 0x7f0e1c138cf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c01c580 tx=0x7f0e0c0a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0c1da0 con 0x7f0e1c13e480 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e13fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e13fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.979+0000 7f0e13fff640 10 monclient: dump: 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4868 (secure 0 0 0) 0x7f0e0c0c1d80 con 0x7f0e1c13e480 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 msgr2=0x7f0e1c138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 0x7f0e1c138cf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c01c580 tx=0x7f0e0c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 msgr2=0x7f0e1c138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:26.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e13fff640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c13e480 0x7f0e1c138cf0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c01c580 tx=0x7f0e0c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 78, 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 294577, 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:26.002 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:26.003 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:26.004 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:26.005 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:26.006 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:26.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:26.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:26.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:26.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.014 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:26.015 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 msgr2=0x7f0dfc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0dfc03b970 0x7f0dfc03de30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c120720 tx=0x7f0e14006cd0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 10 monclient: shutdown 2024-09-15T21:25:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a53b0 msgr2=0x7f0e1c1cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 --2- 172.21.15.17:0/1099943746 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e1c0a53b0 0x7f0e1c1cfec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400b580 tx=0x7f0e0400ba40 comp rx=0 tx=0).stop 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e1bfff640 1 -- 172.21.15.17:0/1099943746 reap_dead start 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e1bfff640 1 -- 172.21.15.17:0/1099943746 reap_dead start 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 shutdown_connections 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 >> 172.21.15.17:0/1099943746 conn(0x7f0e1c0aabb0 msgr2=0x7f0e1c0a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 shutdown_connections 2024-09-15T21:25:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:25.983+0000 7f0e21d04640 1 -- 172.21.15.17:0/1099943746 wait complete. 2024-09-15T21:25:26.017 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:25:26.017 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:25:26.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:25:26.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.065+0000 7f8d603ec640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T21:25:26.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.065+0000 7f8d603ec640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T21:25:26.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.065+0000 7f8d603ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.065+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) -- 0x7f8d5809f8f0 con 0x7f8d580a0160 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient: get_monmap_and_config 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient: build_initial_monmap 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient: monmap: 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:26.126771+0000 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:26.126771+0000 2024-09-15T21:25:26.130 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 1 Processor -- start 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 1 -- start start 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient: init 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f14901378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:26.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14901383b0 0x7f149013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient(hunting): picked mon.noname-c con 0x7f14901383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f149013d2c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14900a1a20 con 0x7f149013d2c0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14900a1d00 con 0x7f14901378c0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f14900a1fe0 con 0x7f14901383b0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f1495fdf640 10 monclient(hunting): authenticate will time out at 1449.522233s 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148ffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f149013d2c0 legacy=0x7f149013f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14901383b0 0x7f149013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148f7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f14901378c0 0x7f1490137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:35512/0 (socket says 172.21.15.17:35512) 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148f7fe640 1 -- 172.21.15.17:0/3883687677 learned_addr learned my addr 172.21.15.17:0/3883687677 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:26.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): get_auth_request con 0x7f14901383b0 auth_method 0 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148e7fc640 1 -- 172.21.15.17:0/3883687677 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545842094 0 0) 0x7f14900a1d00 con 0x7f14901378c0 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148e7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148e7fc640 10 monclient(hunting): my global_id is 84508 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148e7fc640 1 -- 172.21.15.17:0/3883687677 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f147c003530 con 0x7f14901378c0 2024-09-15T21:25:26.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient(hunting): handle_auth_done global_id 84505 payload 274 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 1 -- 172.21.15.17:0/3883687677 >> v1:172.21.15.119:6789/0 conn(0x7f14901378c0 legacy=0x7f1490137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 1 -- 172.21.15.17:0/3883687677 >> v1:172.21.15.17:6789/0 conn(0x7f149013d2c0 legacy=0x7f149013f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 1 monclient: found mon.noname-c 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.123+0000 7f148effd640 1 -- 172.21.15.17:0/3883687677 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14900a22c0 con 0x7f14901383b0 2024-09-15T21:25:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148effd640 1 --2- 172.21.15.17:0/3883687677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901383b0 0x7f149013cd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1480009dc0 tx=0x7f148001c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68bc0d6e013cdf79 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 1 -- 172.21.15.17:0/3883687677 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14800b1070 con 0x7f14901383b0 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: dump: 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 1 -- 172.21.15.17:0/3883687677 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14800a4bd0 con 0x7f14901383b0 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 1 -- 172.21.15.17:0/3883687677 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14800ac440 con 0x7f14901383b0 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148e7fc640 10 monclient: dump: 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.142 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 5 monclient: authenticate success, global_id 84505 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 4 set_mon_vals no callback set 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: shutdown 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- 172.21.15.17:0/3883687677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901383b0 msgr2=0x7f149013cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 --2- 172.21.15.17:0/3883687677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901383b0 0x7f149013cd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1480009dc0 tx=0x7f148001c310 comp rx=0 tx=0).stop 2024-09-15T21:25:26.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- 172.21.15.17:0/3883687677 shutdown_connections 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 --2- 172.21.15.17:0/3883687677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901383b0 0x7f149013cd80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- 172.21.15.17:0/3883687677 >> 172.21.15.17:0/3883687677 conn(0x7f14900aac40 msgr2=0x7f14900a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- 172.21.15.17:0/3883687677 shutdown_connections 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- 172.21.15.17:0/3883687677 wait complete. 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: build_initial_monmap 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: monmap: 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.144 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 Processor -- start 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- start start 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: init 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:26.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14901378c0 0x7f14901cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): picked mon.noname-c con 0x7f14901378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14901383b0 0x7f149009f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): picked mon.noname-a con 0x7f14901383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149013d2c0 0x7f1490136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): picked mon.noname-b con 0x7f149013d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1490144280 con 0x7f14901383b0 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1490144100 con 0x7f149013d2c0 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1490144400 con 0x7f14901378c0 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f1495fdf640 10 monclient(hunting): authenticate will time out at 1449.527652s 2024-09-15T21:25:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148effd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14901383b0 msgr2=0x7f149009f240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14901383b0 0x7f149009f240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14901378c0 0x7f14901cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14901378c0 0x7f14901cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55844/0 (socket says 172.21.15.17:55844) 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 1 -- 172.21.15.17:0/3043416560 learned_addr learned my addr 172.21.15.17:0/3043416560 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): get_auth_request con 0x7f14901378c0 auth_method 0 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.127+0000 7f148f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149013d2c0 0x7f1490136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 10 monclient(hunting): get_auth_request con 0x7f149013d2c0 auth_method 0 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 10 monclient(hunting): handle_auth_done global_id 45173 payload 824 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149013d2c0 msgr2=0x7f1490136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149013d2c0 0x7f1490136cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14901383b0 msgr2=0x7f149009f240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14901383b0 0x7f149009f240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14900a12f0 con 0x7f14901378c0 2024-09-15T21:25:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148f7fe640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14901378c0 0x7f14901cfe20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f147800c4d0 tx=0x7f147800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478017070 con 0x7f14901378c0 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: dump: 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.150 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 5 monclient: authenticate success, global_id 45173 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 10 monclient: _renew_subs 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14901d0400 con 0x7f14901378c0 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1478005cf0 con 0x7f14901378c0 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478007270 con 0x7f14901378c0 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: dump: 2024-09-15T21:25:26.151 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 10 monclient: _renew_subs 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1490137210 con 0x7f14901378c0 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148d7fa640 4 set_mon_vals no callback set 2024-09-15T21:25:26.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 0x7f149002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1490137ca0 con 0x7f14901cf080 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f147801a070 con 0x7f14901378c0 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 0x7f149002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient: get_auth_request con 0x7f14901cf080 auth_method 0 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f147003b970 0x7f147003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(228..228 src has 1..228) ==== 12881+0+0 (secure 0 0 0) 0x7f1478050d40 con 0x7f14901378c0 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f147003b970 0x7f147003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 10 monclient: get_auth_request con 0x7f147003b970 auth_method 0 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 10 monclient(hunting): handle_auth_done global_id 84514 payload 824 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f149009e6e0 con 0x7f14901cf080 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148effd640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 0x7f149002d450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14900a6150 tx=0x7f14800193d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14800bc070 con 0x7f14901cf080 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f148ffff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f147003b970 0x7f147003de30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1488004fd0 tx=0x7f1488006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: dump: 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.154 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f14800ac820 con 0x7f14901cf080 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 msgr2=0x7f149002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 0x7f149002d450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14900a6150 tx=0x7f14800193d0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 msgr2=0x7f149002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:26.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.131+0000 7f1487fff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf080 0x7f149002d450 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14900a6150 tx=0x7f14800193d0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 0x7f149002d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f149013a470 con 0x7f14901cf460 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 0x7f149002d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient: get_auth_request con 0x7f14901cf460 auth_method 0 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 10 monclient(hunting): handle_auth_done global_id 84517 payload 824 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 1 -- 172.21.15.17:0/3043416560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f14900a9200 con 0x7f14901cf460 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148effd640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 0x7f149002d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f148001a9c0 tx=0x7f14800a6000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14800cc060 con 0x7f14901cf460 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 10 monclient: dump: 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.158 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7f1480003e80 con 0x7f14901cf460 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 msgr2=0x7f149002d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 0x7f149002d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f148001a9c0 tx=0x7f14800a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 msgr2=0x7f149002d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:26.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1487fff640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14901cf460 0x7f149002d990 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f148001a9c0 tx=0x7f14800a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 78, 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 78371, 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.160 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:26.161 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:26.162 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:26.163 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:26.164 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:26.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:26.166 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:26.168 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:26.170 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:26.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:26.172 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f147003b970 msgr2=0x7f147003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f147003b970 0x7f147003de30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1488004fd0 tx=0x7f1488006e20 comp rx=0 tx=0).stop 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 10 monclient: shutdown 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14901378c0 msgr2=0x7f14901cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 --2- 172.21.15.17:0/3043416560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14901378c0 0x7f14901cfe20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f147800c4d0 tx=0x7f147800c990 comp rx=0 tx=0).stop 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148ffff640 1 -- 172.21.15.17:0/3043416560 reap_dead start 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f148ffff640 1 -- 172.21.15.17:0/3043416560 reap_dead start 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 shutdown_connections 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 >> 172.21.15.17:0/3043416560 conn(0x7f14900aac40 msgr2=0x7f14900a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:26.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 shutdown_connections 2024-09-15T21:25:26.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:26.139+0000 7f1495fdf640 1 -- 172.21.15.17:0/3043416560 wait complete. 2024-09-15T21:25:26.175 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:25:26.175 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:25:26.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.641+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 removed item id -49 name 'h11' from crush map v229) ==== 120+0+0 (secure 0 0 0) 0x7f8d54016110 con 0x7f8d580a0160 2024-09-15T21:25:26.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.641+0000 7f8d4effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T21:25:26.644 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.641+0000 7f8d4effd640 10 monclient: _finish_command 2 = system:0 removed item id -49 name 'h11' from crush map 2024-09-15T21:25:26.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T21:25:26.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T21:25:26.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) -- 0x7f8d580008e0 con 0x7f8d580a0160 2024-09-15T21:25:26.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d4effd640 1 -- 172.21.15.119:0/85510040 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 device 'h11' does not appear in the crush map v229) ==== 120+0+0 (secure 0 0 0) 0x7f8d540e8830 con 0x7f8d580a0160 2024-09-15T21:25:26.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d4effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T21:25:26.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d4effd640 10 monclient: _finish_command 3 = system:0 device 'h11' does not appear in the crush map 2024-09-15T21:25:26.750 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h11' does not appear in the crush map 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d3c03b970 msgr2=0x7f8d3c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d3c03b970 0x7f8d3c03de30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d48003940 tx=0x7f8d48006d90 comp rx=0 tx=0).stop 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 10 monclient: shutdown 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 msgr2=0x7f8d5813bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d5813bb70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d540ae040 tx=0x7f8d5401ac40 comp rx=0 tx=0).stop 2024-09-15T21:25:26.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 shutdown_connections 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d3c03b970 0x7f8d3c03de30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d58135ca0 0x7f8d5813c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d580a0160 0x7f8d5813bb70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 --2- 172.21.15.119:0/85510040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d5806ec30 0x7f8d581cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 >> 172.21.15.119:0/85510040 conn(0x7f8d580a2ae0 msgr2=0x7f8d581318f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 shutdown_connections 2024-09-15T21:25:26.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.745+0000 7f8d603ec640 1 -- 172.21.15.119:0/85510040 wait complete. 2024-09-15T21:25:26.766 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:26.766 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h12 2024-09-15T21:25:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient: get_monmap_and_config 2024-09-15T21:25:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient: build_initial_monmap 2024-09-15T21:25:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient: monmap: 2024-09-15T21:25:26.884 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:26.885795+0000 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:26.885795+0000 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:26.885 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 Processor -- start 2024-09-15T21:25:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 -- start start 2024-09-15T21:25:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient: init 2024-09-15T21:25:26.887 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98d00a4ed0 0x7f98d00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): picked mon.noname-c con 0x7f98d00a4ed0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98d00a57f0 0x7f98d00a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): picked mon.noname-a con 0x7f98d00a57f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f98d0137c90 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d00a6db0 con 0x7f98d0137c90 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f98d00a1210 con 0x7f98d00a57f0 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f98d00a1b00 con 0x7f98d00a4ed0 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:26.888 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98d5374640 10 monclient(hunting): authenticate will time out at 1445.682425s 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98d00a4ed0 0x7f98d00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98cf7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f98d0137c90 0x7f98d01380c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60298/0 (socket says 172.21.15.119:60298) 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98cf7fe640 1 -- 172.21.15.119:0/994446141 learned_addr learned my addr 172.21.15.119:0/994446141 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ce7fc640 1 -- 172.21.15.119:0/994446141 >> v2:172.21.15.17:3300/0 conn(0x7f98d00a57f0 msgr2=0x7f98d00a5c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): get_auth_request con 0x7f98d00a4ed0 auth_method 0 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ce7fc640 1 --2- 172.21.15.119:0/994446141 >> v2:172.21.15.17:3300/0 conn(0x7f98d00a57f0 0x7f98d00a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient(hunting): handle_auth_done global_id 84520 payload 274 2024-09-15T21:25:26.889 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 1 -- 172.21.15.119:0/994446141 >> v2:172.21.15.17:3300/0 conn(0x7f98d00a57f0 msgr2=0x7f98d00a5c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 1 --2- 172.21.15.119:0/994446141 >> v2:172.21.15.17:3300/0 conn(0x7f98d00a57f0 0x7f98d00a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 1 -- 172.21.15.119:0/994446141 >> v1:172.21.15.119:6789/0 conn(0x7f98d0137c90 legacy=0x7f98d01380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 1 monclient: found mon.noname-c 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98cdffb640 1 -- 172.21.15.119:0/994446141 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411615896 0 0) 0x7f98d00a6db0 con 0x7f98d0137c90 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98ceffd640 1 -- 172.21.15.119:0/994446141 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d00a6ad0 con 0x7f98d00a4ed0 2024-09-15T21:25:26.890 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.881+0000 7f98cdffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:26.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 --2- 172.21.15.119:0/994446141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d00a52b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f98d006f060 tx=0x7f98c401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e7ca7a5da0a0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 1 -- 172.21.15.119:0/994446141 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40b0070 con 0x7f98d00a4ed0 2024-09-15T21:25:26.895 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: dump: 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 1 -- 172.21.15.119:0/994446141 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c40ab400 con 0x7f98d00a4ed0 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 5 monclient: authenticate success, global_id 84520 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 1 -- 172.21.15.119:0/994446141 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40ab7d0 con 0x7f98d00a4ed0 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98cdffb640 10 monclient: dump: 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 4 set_mon_vals no callback set 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: shutdown 2024-09-15T21:25:26.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- 172.21.15.119:0/994446141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 msgr2=0x7f98d00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 --2- 172.21.15.119:0/994446141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d00a52b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f98d006f060 tx=0x7f98c401c360 comp rx=0 tx=0).stop 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- 172.21.15.119:0/994446141 shutdown_connections 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 --2- 172.21.15.119:0/994446141 >> v2:172.21.15.17:3300/0 conn(0x7f98d00a57f0 0x7f98d00a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 --2- 172.21.15.119:0/994446141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d00a52b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- 172.21.15.119:0/994446141 >> 172.21.15.119:0/994446141 conn(0x7f98d00a6fd0 msgr2=0x7f98d00a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- 172.21.15.119:0/994446141 shutdown_connections 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- 172.21.15.119:0/994446141 wait complete. 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: build_initial_monmap 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: monmap: 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:26.899 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 Processor -- start 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- start start 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: init 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d01364e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): picked mon.noname-b con 0x7f98d00a4ed0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d00a57f0 0x7f98d0136a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): picked mon.noname-a con 0x7f98d00a57f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98d0137c90 0x7f98d013cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): picked mon.noname-c con 0x7f98d0137c90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00a7ea0 con 0x7f98d00a57f0 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00a7d20 con 0x7f98d00a4ed0 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f98d0142710 con 0x7f98d0137c90 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98d5374640 10 monclient(hunting): authenticate will time out at 1445.687452s 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d01364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.901 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d01364e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41908/0 (socket says 172.21.15.119:41908) 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 -- 172.21.15.119:0/1045127016 learned_addr learned my addr 172.21.15.119:0/1045127016 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): get_auth_request con 0x7f98d00a4ed0 auth_method 0 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ce7fc640 1 -- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d00a57f0 msgr2=0x7f98d0136a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ce7fc640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d00a57f0 0x7f98d0136a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient(hunting): handle_auth_done global_id 84526 payload 824 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 -- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98d0137c90 msgr2=0x7f98d013cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98d0137c90 0x7f98d013cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 -- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d00a57f0 msgr2=0x7f98d0136a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d00a57f0 0x7f98d0136a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 monclient: found mon.noname-b 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.885+0000 7f98ceffd640 1 -- 172.21.15.119:0/1045127016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d0121210 con 0x7f98d00a4ed0 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98ceffd640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d01364e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98c4018f70 tx=0x7f98c401aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40bb070 con 0x7f98d00a4ed0 2024-09-15T21:25:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: dump: 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c4004020 con 0x7f98d00a4ed0 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 5 monclient: authenticate success, global_id 84526 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 monclient: _renew_subs 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d00a0830 con 0x7f98d00a4ed0 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40b0040 con 0x7f98d00a4ed0 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 10 monclient: dump: 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:26.905 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 monclient: _renew_subs 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98cd7fa640 4 set_mon_vals no callback set 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d0120c50 con 0x7f98d00a4ed0 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:26.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d00a52b0 con 0x7f98d00a4ed0 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f98c4003620 con 0x7f98d00a4ed0 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98bc03b970 0x7f98bc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(229..229 src has 1..229) ==== 12668+0+0 (secure 0 0 0) 0x7f98c40ec0b0 con 0x7f98d00a4ed0 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98ce7fc640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98bc03b970 0x7f98bc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98ce7fc640 10 monclient: get_auth_request con 0x7f98bc03b970 auth_method 0 2024-09-15T21:25:26.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.889+0000 7f98ce7fc640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98bc03b970 0x7f98bc03de30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98d013eb70 tx=0x7f98c0006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:26.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.893+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f98c4016070 con 0x7f98d00a4ed0 2024-09-15T21:25:26.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.893+0000 7f98b77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:26.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:26.893+0000 7f98b77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:27.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.057+0000 7f98d5374640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T21:25:27.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.057+0000 7f98d5374640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T21:25:27.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.057+0000 7f98d5374640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.057+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) -- 0x7f98d013ede0 con 0x7f98d00a4ed0 2024-09-15T21:25:27.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.649+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 removed item id -52 name 'h12' from crush map v230) ==== 120+0+0 (secure 0 0 0) 0x7f98c40e8bf0 con 0x7f98d00a4ed0 2024-09-15T21:25:27.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.649+0000 7f98b77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T21:25:27.653 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.649+0000 7f98b77fe640 10 monclient: _finish_command 2 = system:0 removed item id -52 name 'h12' from crush map 2024-09-15T21:25:27.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98d5374640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T21:25:27.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98d5374640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T21:25:27.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98d5374640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) -- 0x7f98d0136f60 con 0x7f98d00a4ed0 2024-09-15T21:25:27.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98b77fe640 1 -- 172.21.15.119:0/1045127016 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 device 'h12' does not appear in the crush map v230) ==== 120+0+0 (secure 0 0 0) 0x7f98c40bfaf0 con 0x7f98d00a4ed0 2024-09-15T21:25:27.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98b77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T21:25:27.758 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.753+0000 7f98b77fe640 10 monclient: _finish_command 3 = system:0 device 'h12' does not appear in the crush map 2024-09-15T21:25:27.759 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h12' does not appear in the crush map 2024-09-15T21:25:27.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98bc03b970 msgr2=0x7f98bc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:27.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98bc03b970 0x7f98bc03de30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98d013eb70 tx=0x7f98c0006d90 comp rx=0 tx=0).stop 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 10 monclient: shutdown 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 msgr2=0x7f98d01364e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d01364e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98c4018f70 tx=0x7f98c401aba0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 shutdown_connections 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98bc03b970 0x7f98bc03de30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98d0137c90 0x7f98d013cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d00a57f0 0x7f98d0136a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 --2- 172.21.15.119:0/1045127016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98d00a4ed0 0x7f98d01364e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 >> 172.21.15.119:0/1045127016 conn(0x7f98d00a6fd0 msgr2=0x7f98d00a8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 shutdown_connections 2024-09-15T21:25:27.762 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.757+0000 7f98d5374640 1 -- 172.21.15.119:0/1045127016 wait complete. 2024-09-15T21:25:27.776 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:27.776 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h13 2024-09-15T21:25:27.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient: get_monmap_and_config 2024-09-15T21:25:27.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient: build_initial_monmap 2024-09-15T21:25:27.897 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient: monmap: 2024-09-15T21:25:27.897 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:27.897 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:27.898258+0000 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:27.898258+0000 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 1 Processor -- start 2024-09-15T21:25:27.898 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 1 -- start start 2024-09-15T21:25:27.899 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient: init 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fafb0135ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fafb00a0160 0x7fafb013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient(hunting): picked mon.noname-c con 0x7fafb00a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fafb006ec30 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafb009cf30 con 0x7fafb006ec30 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafb00a1ca0 con 0x7fafb0135ca0 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:27.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fafb00a1f80 con 0x7fafb00a0160 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafb5f95640 10 monclient(hunting): authenticate will time out at 1446.694863s 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fafb00a0160 0x7fafb013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fafb00a0160 0x7fafb013aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41914/0 (socket says 172.21.15.119:41914) 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fafb006ec30 legacy=0x7fafb013cf50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaf7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fafb0135ca0 0x7fafb0136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60302/0 (socket says 172.21.15.119:60302) 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 -- 172.21.15.119:0/3760606863 learned_addr learned my addr 172.21.15.119:0/3760606863 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): get_auth_request con 0x7fafb00a0160 auth_method 0 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:27.904 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient(hunting): handle_auth_done global_id 84529 payload 274 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 -- 172.21.15.119:0/3760606863 >> v1:172.21.15.119:6789/0 conn(0x7fafb0135ca0 legacy=0x7fafb0136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 -- 172.21.15.119:0/3760606863 >> v1:172.21.15.17:6789/0 conn(0x7fafb006ec30 legacy=0x7fafb013cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 monclient: found mon.noname-c 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.893+0000 7fafaeffd640 1 -- 172.21.15.119:0/3760606863 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb00a3860 con 0x7fafb00a0160 2024-09-15T21:25:27.905 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafaeffd640 1 --2- 172.21.15.119:0/3760606863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb00a0160 0x7fafb013aa00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafa00098a0 tx=0x7fafa001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808bab7ba1221399 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 1 -- 172.21.15.119:0/3760606863 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00b1070 con 0x7fafb00a0160 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: dump: 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:27.906 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 1 -- 172.21.15.119:0/3760606863 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa00ac640 con 0x7fafb00a0160 2024-09-15T21:25:27.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 1 -- 172.21.15.119:0/3760606863 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00aca10 con 0x7fafb00a0160 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 5 monclient: authenticate success, global_id 84529 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 4 set_mon_vals no callback set 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafae7fc640 10 monclient: dump: 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:27.908 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 10 monclient: shutdown 2024-09-15T21:25:27.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 -- 172.21.15.119:0/3760606863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb00a0160 msgr2=0x7fafb013aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 --2- 172.21.15.119:0/3760606863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb00a0160 0x7fafb013aa00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafa00098a0 tx=0x7fafa001c400 comp rx=0 tx=0).stop 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 -- 172.21.15.119:0/3760606863 shutdown_connections 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 --2- 172.21.15.119:0/3760606863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb00a0160 0x7fafb013aa00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 -- 172.21.15.119:0/3760606863 >> 172.21.15.119:0/3760606863 conn(0x7fafb00a2ae0 msgr2=0x7fafb00a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 -- 172.21.15.119:0/3760606863 shutdown_connections 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 -- 172.21.15.119:0/3760606863 wait complete. 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 10 monclient: build_initial_monmap 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 10 monclient: monmap: 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:27.910 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:27.911 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:27.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 Processor -- start 2024-09-15T21:25:27.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 1 -- start start 2024-09-15T21:25:27.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.897+0000 7fafb5f95640 10 monclient: init 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fafb006ec30 0x7fafb013d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): picked mon.noname-c con 0x7fafb006ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafb00a0160 0x7fafb009d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): picked mon.noname-a con 0x7fafb00a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 0x7fafb009da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): picked mon.noname-b con 0x7fafb0135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:27.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fafb0141010 con 0x7fafb00a0160 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fafb0140e90 con 0x7fafb0135ca0 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fafb0141190 con 0x7fafb006ec30 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient(hunting): authenticate will time out at 1446.700492s 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 0x7fafb009da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaeffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafb00a0160 msgr2=0x7fafb009d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafb00a0160 0x7fafb009d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 0x7fafb009da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41928/0 (socket says 172.21.15.119:41928) 2024-09-15T21:25:27.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 -- 172.21.15.119:0/68479703 learned_addr learned my addr 172.21.15.119:0/68479703 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): get_auth_request con 0x7fafb0135ca0 auth_method 0 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fafb006ec30 0x7fafb013d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 10 monclient(hunting): get_auth_request con 0x7fafb006ec30 auth_method 0 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:27.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient(hunting): handle_auth_done global_id 84535 payload 824 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 -- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fafb006ec30 msgr2=0x7fafb013d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fafb006ec30 0x7fafb013d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 -- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafb00a0160 msgr2=0x7fafb009d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafb00a0160 0x7fafb009d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 monclient: found mon.noname-b 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 -- 172.21.15.119:0/68479703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb009ef90 con 0x7fafb0135ca0 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaffff640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 0x7fafb009da50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b990 tx=0x7fafa800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800cb20 con 0x7fafb0135ca0 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:27.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: dump: 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:27.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa8010070 con 0x7fafb0135ca0 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8015470 con 0x7fafb0135ca0 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 10 monclient: dump: 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:27.922 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 5 monclient: authenticate success, global_id 84535 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _renew_subs 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb013c870 con 0x7fafb0135ca0 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafadffb640 4 set_mon_vals no callback set 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _renew_subs 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb013b210 con 0x7fafb0135ca0 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb009f8f0 con 0x7fafb0135ca0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fafa8015610 con 0x7fafb0135ca0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf9c03b970 0x7faf9c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(231..231 src has 1..231) ==== 12455+0+0 (secure 0 0 0) 0x7fafa8051b40 con 0x7fafb0135ca0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf9c03b970 0x7faf9c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 10 monclient: get_auth_request con 0x7faf9c03b970 auth_method 0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.901+0000 7fafaf7fe640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf9c03b970 0x7faf9c03de30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f70 tx=0x7fafa4007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.905+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fafa8014de0 con 0x7fafb0135ca0 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.905+0000 7faf97fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:27.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:27.905+0000 7faf97fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:28.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.065+0000 7fafb5f95640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T21:25:28.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.065+0000 7fafb5f95640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T21:25:28.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.065+0000 7fafb5f95640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:28.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.065+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) -- 0x7fafb006c460 con 0x7fafb0135ca0 2024-09-15T21:25:28.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.797+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 removed item id -55 name 'h13' from crush map v232) ==== 120+0+0 (secure 0 0 0) 0x7fafa804e6f0 con 0x7fafb0135ca0 2024-09-15T21:25:28.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.797+0000 7faf97fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T21:25:28.802 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.797+0000 7faf97fff640 10 monclient: _finish_command 2 = system:0 removed item id -55 name 'h13' from crush map 2024-09-15T21:25:28.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7fafb5f95640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T21:25:28.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7fafb5f95640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T21:25:28.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7fafb5f95640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:28.906 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) -- 0x7fafb013b630 con 0x7fafb0135ca0 2024-09-15T21:25:28.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7faf97fff640 1 -- 172.21.15.119:0/68479703 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 device 'h13' does not appear in the crush map v232) ==== 120+0+0 (secure 0 0 0) 0x7fafa8014b40 con 0x7fafb0135ca0 2024-09-15T21:25:28.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7faf97fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T21:25:28.907 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.901+0000 7faf97fff640 10 monclient: _finish_command 3 = system:0 device 'h13' does not appear in the crush map 2024-09-15T21:25:28.907 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h13' does not appear in the crush map 2024-09-15T21:25:28.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf9c03b970 msgr2=0x7faf9c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:28.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf9c03b970 0x7faf9c03de30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f70 tx=0x7fafa4007c90 comp rx=0 tx=0).stop 2024-09-15T21:25:28.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 10 monclient: shutdown 2024-09-15T21:25:28.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:28.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 msgr2=0x7fafb009da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 0x7fafb009da50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b990 tx=0x7fafa800be50 comp rx=0 tx=0).stop 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 shutdown_connections 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf9c03b970 0x7faf9c03de30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb0135ca0 0x7fafb009da50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafb00a0160 0x7fafb009d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 --2- 172.21.15.119:0/68479703 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fafb006ec30 0x7fafb013d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 >> 172.21.15.119:0/68479703 conn(0x7fafb00a2ae0 msgr2=0x7fafb00722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 shutdown_connections 2024-09-15T21:25:28.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:28.905+0000 7fafb5f95640 1 -- 172.21.15.119:0/68479703 wait complete. 2024-09-15T21:25:28.924 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:28.924 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h14 2024-09-15T21:25:29.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient: get_monmap_and_config 2024-09-15T21:25:29.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient: build_initial_monmap 2024-09-15T21:25:29.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient: monmap: 2024-09-15T21:25:29.037 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:29.037 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:29.038373+0000 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:29.038373+0000 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:29.038 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 Processor -- start 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 -- start start 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient: init 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f960009f640 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96000a0130 0x7f960009d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): picked mon.noname-c con 0x7f96000a0130 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9600139c00 0x7f960013bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:29.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): picked mon.noname-b con 0x7f9600139c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960009e880 con 0x7f960009f640 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f960009eb60 con 0x7f96000a0130 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f960009ee40 con 0x7f9600139c00 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f96067ff640 10 monclient(hunting): authenticate will time out at 1447.835401s 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95fffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f960009f640 0x7f960009fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60318/0 (socket says 172.21.15.119:60318) 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95fffff640 1 -- 172.21.15.119:0/67124650 learned_addr learned my addr 172.21.15.119:0/67124650 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 1 --2- 172.21.15.119:0/67124650 >> v2:172.21.15.119:3300/0 conn(0x7f96000a0130 0x7f960009d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): get_auth_request con 0x7f96000a0130 auth_method 0 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:29.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f9604d9c640 1 --2- 172.21.15.119:0/67124650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9600139c00 0x7f960013bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f9604d9c640 10 monclient(hunting): get_auth_request con 0x7f9600139c00 auth_method 0 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f9604d9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f9604d9c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f9604d9c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95ff7fe640 1 -- 172.21.15.119:0/67124650 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385317505 0 0) 0x7f960009e880 con 0x7f960009f640 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95ff7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:29.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95ff7fe640 10 monclient(hunting): my global_id is 84541 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.033+0000 7f95ff7fe640 1 -- 172.21.15.119:0/67124650 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95ec003530 con 0x7f960009f640 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 10 monclient(hunting): handle_auth_done global_id 84538 payload 274 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 1 -- 172.21.15.119:0/67124650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9600139c00 msgr2=0x7f960013bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 1 --2- 172.21.15.119:0/67124650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9600139c00 0x7f960013bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 1 -- 172.21.15.119:0/67124650 >> v1:172.21.15.119:6789/0 conn(0x7f960009f640 legacy=0x7f960009fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 1 -- 172.21.15.119:0/67124650 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96000a2840 con 0x7f96000a0130 2024-09-15T21:25:29.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95f77fe640 1 --2- 172.21.15.119:0/67124650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f960009d9a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95e80097c0 tx=0x7f95e801c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e45e6fbe57f5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:29.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 1 -- 172.21.15.119:0/67124650 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b0070 con 0x7f96000a0130 2024-09-15T21:25:29.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: dump: 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:29.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 5 monclient: authenticate success, global_id 84538 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 1 -- 172.21.15.119:0/67124650 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e80ab600 con 0x7f96000a0130 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 4 set_mon_vals no callback set 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 1 -- 172.21.15.119:0/67124650 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80ab9d0 con 0x7f96000a0130 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f95ff7fe640 10 monclient: dump: 2024-09-15T21:25:29.051 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 10 monclient: shutdown 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 -- 172.21.15.119:0/67124650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 msgr2=0x7f960009d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:29.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 --2- 172.21.15.119:0/67124650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f960009d9a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95e80097c0 tx=0x7f95e801c360 comp rx=0 tx=0).stop 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 -- 172.21.15.119:0/67124650 shutdown_connections 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 --2- 172.21.15.119:0/67124650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9600139c00 0x7f960013bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 --2- 172.21.15.119:0/67124650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f960009d9a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 -- 172.21.15.119:0/67124650 >> 172.21.15.119:0/67124650 conn(0x7f96000a2ae0 msgr2=0x7f96000a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 -- 172.21.15.119:0/67124650 shutdown_connections 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 -- 172.21.15.119:0/67124650 wait complete. 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 10 monclient: build_initial_monmap 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 10 monclient: monmap: 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:29.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.037+0000 7f96067ff640 1 Processor -- start 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- start start 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: init 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f960009f640 0x7f9600134810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): picked mon.noname-c con 0x7f960009f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:29.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f9600134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): picked mon.noname-b con 0x7f96000a0130 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9600139c00 0x7f960011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): picked mon.noname-a con 0x7f9600139c00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f960013eed0 con 0x7f9600139c00 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f960013ed50 con 0x7f96000a0130 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f960013f050 con 0x7f960009f640 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient(hunting): authenticate will time out at 1447.840479s 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f9600134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:29.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f9604d9c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9600139c00 msgr2=0x7f960011c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f9600134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41954/0 (socket says 172.21.15.119:41954) 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f9604d9c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9600139c00 0x7f960011c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 -- 172.21.15.119:0/26740961 learned_addr learned my addr 172.21.15.119:0/26740961 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): get_auth_request con 0x7f96000a0130 auth_method 0 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f960009f640 0x7f9600134810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 10 monclient(hunting): get_auth_request con 0x7f960009f640 auth_method 0 2024-09-15T21:25:29.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient(hunting): handle_auth_done global_id 84544 payload 824 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 -- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f960009f640 msgr2=0x7f9600134810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f960009f640 0x7f9600134810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 -- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9600139c00 msgr2=0x7f960011c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9600139c00 0x7f960011c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 -- 172.21.15.119:0/26740961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960009d1d0 con 0x7f96000a0130 2024-09-15T21:25:29.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95f77fe640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f9600134d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95e80ae040 tx=0x7f95e8019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b0040 con 0x7f96000a0130 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: dump: 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:29.058 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 5 monclient: authenticate success, global_id 84544 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _renew_subs 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e80b43f0 con 0x7f96000a0130 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960009c850 con 0x7f96000a0130 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b4800 con 0x7f96000a0130 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:29.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 10 monclient: dump: 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:29.060 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _renew_subs 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960011ff40 con 0x7f96000a0130 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95feffd640 4 set_mon_vals no callback set 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f95e80b49a0 con 0x7f96000a0130 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960009d9a0 con 0x7f96000a0130 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f95e003b9a0 0x7f95e003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 12242+0+0 (secure 0 0 0) 0x7f95e80eb6d0 con 0x7f96000a0130 2024-09-15T21:25:29.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f95e003b9a0 0x7f95e003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:29.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 10 monclient: get_auth_request con 0x7f95e003b9a0 auth_method 0 2024-09-15T21:25:29.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.041+0000 7f95fffff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f95e003b9a0 0x7f95e003de60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f95f0007a70 tx=0x7f95f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:29.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.045+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f95e8016070 con 0x7f96000a0130 2024-09-15T21:25:29.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.045+0000 7f95fd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:29.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.045+0000 7f95fd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.217+0000 7f96067ff640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T21:25:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.217+0000 7f96067ff640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T21:25:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.217+0000 7f96067ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.223 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.217+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) -- 0x7f960006c460 con 0x7f96000a0130 2024-09-15T21:25:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.809+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 removed item id -58 name 'h14' from crush map v233) ==== 120+0+0 (secure 0 0 0) 0x7f95e80aade0 con 0x7f96000a0130 2024-09-15T21:25:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.809+0000 7f95fd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T21:25:29.813 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.809+0000 7f95fd7fa640 10 monclient: _finish_command 2 = system:0 removed item id -58 name 'h14' from crush map 2024-09-15T21:25:29.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T21:25:29.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T21:25:29.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:29.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) -- 0x7f960011ea20 con 0x7f96000a0130 2024-09-15T21:25:29.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f95fd7fa640 1 -- 172.21.15.119:0/26740961 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 device 'h14' does not appear in the crush map v233) ==== 120+0+0 (secure 0 0 0) 0x7f95e80b4d40 con 0x7f96000a0130 2024-09-15T21:25:29.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f95fd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T21:25:29.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f95fd7fa640 10 monclient: _finish_command 3 = system:0 device 'h14' does not appear in the crush map 2024-09-15T21:25:29.925 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h14' does not appear in the crush map 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f95e003b9a0 msgr2=0x7f95e003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f95e003b9a0 0x7f95e003de60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f95f0007a70 tx=0x7f95f0009040 comp rx=0 tx=0).stop 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 10 monclient: shutdown 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 msgr2=0x7f9600134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f9600134d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95e80ae040 tx=0x7f95e8019100 comp rx=0 tx=0).stop 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 shutdown_connections 2024-09-15T21:25:29.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f95e003b9a0 0x7f95e003de60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9600139c00 0x7f960011c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96000a0130 0x7f9600134d50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 --2- 172.21.15.119:0/26740961 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f960009f640 0x7f9600134810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:29.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 >> 172.21.15.119:0/26740961 conn(0x7f96000a2ae0 msgr2=0x7f96001382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:29.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 shutdown_connections 2024-09-15T21:25:29.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:29.921+0000 7f96067ff640 1 -- 172.21.15.119:0/26740961 wait complete. 2024-09-15T21:25:29.942 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:29.942 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h15 2024-09-15T21:25:30.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient: get_monmap_and_config 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient: build_initial_monmap 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient: monmap: 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:30.062033+0000 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:30.062033+0000 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:30.061 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:30.062 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:30.062 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:30.062 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 1 Processor -- start 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 1 -- start start 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient: init 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5af4136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient(hunting): picked mon.noname-b con 0x7f5af4135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5af40a0160 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:30.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5af406ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af40a1800 con 0x7f5af40a0160 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af41365c0 con 0x7f5af406ec30 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5af409cf30 con 0x7f5af4135ca0 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5afb99a640 10 monclient(hunting): authenticate will time out at 1448.858646s 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9f37640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5af406ec30 0x7f5af40a10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60322/0 (socket says 172.21.15.119:60322) 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9f37640 1 -- 172.21.15.119:0/3089394627 learned_addr learned my addr 172.21.15.119:0/3089394627 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af8f35640 1 -- 172.21.15.119:0/3089394627 >> v1:172.21.15.17:6789/0 conn(0x7f5af40a0160 legacy=0x7f5af40a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5af4136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): get_auth_request con 0x7f5af4135ca0 auth_method 0 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:30.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638151483 0 0) 0x7f5af41365c0 con 0x7f5af406ec30 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): _negotiate 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): my global_id is 84547 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5adc003530 con 0x7f5af406ec30 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1000064110 0 0) 0x7f5adc003530 con 0x7f5af406ec30 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 msgr2=0x7f5af4136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:30.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5af4136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 >> v1:172.21.15.17:6789/0 conn(0x7f5af40a0160 legacy=0x7f5af40a09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af40a1ca0 con 0x7f5af406ec30 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5ae4001a80 con 0x7f5af406ec30 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:30.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: dump: 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:30.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 >> v1:172.21.15.119:6789/0 conn(0x7f5af406ec30 legacy=0x7f5af40a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc0039c0 0x7f5adc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): picked mon.b con 0x7f5adc0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5adc004810 0x7f5adc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): picked mon.a con 0x7f5adc004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5adc005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): picked mon.c con 0x7f5af4135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5af409cf30 con 0x7f5adc004810 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc00bb60 con 0x7f5adc0039c0 2024-09-15T21:25:30.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc00bd60 con 0x7f5af4135ca0 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af3fff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc0039c0 0x7f5adc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af8f35640 1 -- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5adc004810 msgr2=0x7f5adc004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af8f35640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5adc004810 0x7f5adc004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): get_auth_request con 0x7f5adc0039c0 auth_method 0 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:30.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.057+0000 7f5af9736640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 10 monclient(hunting): handle_auth_done global_id 84547 payload 293 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9f37640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5adc005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 -- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 msgr2=0x7f5adc005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5adc005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 -- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5adc004810 msgr2=0x7f5adc004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5adc004810 0x7f5adc004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 10 monclient: _finish_hunting 0 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 monclient: found mon.b 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 -- 172.21.15.119:0/3089394627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc003530 con 0x7f5adc0039c0 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 10 monclient: _finish_auth 0 2024-09-15T21:25:30.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 10 monclient: _check_auth_tickets 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 5 monclient: authenticate success, global_id 84547 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9f37640 10 monclient: get_auth_request con 0x7f5af4135ca0 auth_method 0 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9f37640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5af4135ca0 0x7f5adc005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9736640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc0039c0 0x7f5adc003dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009b10 tx=0x7f5aec01ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0941095ef4ca1bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0096b0 con 0x7f5adc0039c0 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: dump: 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:30.075 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec0a6430 con 0x7f5adc0039c0 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:30.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 4 set_mon_vals no callback set 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 1 -- 172.21.15.119:0/3089394627 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0a6820 con 0x7f5adc0039c0 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af3fff640 10 monclient: dump: 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:30.077 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 10 monclient: shutdown 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 1 -- 172.21.15.119:0/3089394627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc0039c0 msgr2=0x7f5adc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 1 --2- 172.21.15.119:0/3089394627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5adc0039c0 0x7f5adc003dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009b10 tx=0x7f5aec01ca50 comp rx=0 tx=0).stop 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5af9f37640 1 -- 172.21.15.119:0/3089394627 reap_dead start 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 1 -- 172.21.15.119:0/3089394627 shutdown_connections 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 1 -- 172.21.15.119:0/3089394627 >> 172.21.15.119:0/3089394627 conn(0x7f5af40a2ae0 msgr2=0x7f5af40a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 1 -- 172.21.15.119:0/3089394627 shutdown_connections 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 1 -- 172.21.15.119:0/3089394627 wait complete. 2024-09-15T21:25:30.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 10 monclient: build_initial_monmap 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.061+0000 7f5afb99a640 10 monclient: monmap: 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 Processor -- start 2024-09-15T21:25:30.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- start start 2024-09-15T21:25:30.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: init 2024-09-15T21:25:30.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:30.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5adc0039c0 0x7f5af411dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): picked mon.noname-c con 0x7f5adc0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5af4133d40 0x7f5af41341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): picked mon.noname-a con 0x7f5af4133d40 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 0x7f5af41cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): picked mon.noname-b con 0x7f5af41cd870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5af413f3c0 con 0x7f5af4133d40 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5af413f240 con 0x7f5af41cd870 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:30.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5af413f540 con 0x7f5adc0039c0 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient(hunting): authenticate will time out at 1448.864572s 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af8f35640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5af4133d40 msgr2=0x7f5af41341c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af8f35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5af4133d40 0x7f5af41341c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 0x7f5af41cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 0x7f5af41cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41972/0 (socket says 172.21.15.119:41972) 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 -- 172.21.15.119:0/928259134 learned_addr learned my addr 172.21.15.119:0/928259134 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): get_auth_request con 0x7f5af41cd870 auth_method 0 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:30.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5adc0039c0 0x7f5af411dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 10 monclient(hunting): get_auth_request con 0x7f5adc0039c0 auth_method 0 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient(hunting): handle_auth_done global_id 84550 payload 824 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 -- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5adc0039c0 msgr2=0x7f5af411dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5adc0039c0 0x7f5af411dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 -- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5af4133d40 msgr2=0x7f5af41341c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5af4133d40 0x7f5af41341c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient: _finish_hunting 0 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 monclient: found mon.noname-b 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 -- 172.21.15.119:0/928259134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af409d1d0 con 0x7f5af41cd870 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9f37640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 0x7f5af41cdd00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5af406f0c0 tx=0x7f5ae400b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400de70 con 0x7f5af41cd870 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:30.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: dump: 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:30.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae400bb80 con 0x7f5af41cd870 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 5 monclient: authenticate success, global_id 84550 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _renew_subs 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af411c9a0 con 0x7f5af41cd870 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4007110 con 0x7f5af41cd870 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 10 monclient: dump: 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:30.089 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _renew_subs 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af411f6e0 con 0x7f5af41cd870 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af37fe640 4 set_mon_vals no callback set 2024-09-15T21:25:30.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af40a09e0 con 0x7f5af41cd870 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5ae401b020 con 0x7f5af41cd870 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae003b970 0x7f5ae003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 12029+0+0 (secure 0 0 0) 0x7f5ae4051b30 con 0x7f5af41cd870 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae003b970 0x7f5ae003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 10 monclient: get_auth_request con 0x7f5ae003b970 auth_method 0 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.065+0000 7f5af9736640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae003b970 0x7f5ae003de30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0ad000 tx=0x7f5aec0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.069+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5ae404e8f0 con 0x7f5af41cd870 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.069+0000 7f5af1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:30.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.069+0000 7f5af1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:30.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.237+0000 7f5afb99a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T21:25:30.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.237+0000 7f5afb99a640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T21:25:30.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.237+0000 7f5afb99a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.237+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) -- 0x7f5af409f8f0 con 0x7f5af41cd870 2024-09-15T21:25:30.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.813+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 removed item id -61 name 'h15' from crush map v234) ==== 120+0+0 (secure 0 0 0) 0x7f5ae4007490 con 0x7f5af41cd870 2024-09-15T21:25:30.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.813+0000 7f5af1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T21:25:30.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.813+0000 7f5af1ffb640 10 monclient: _finish_command 2 = system:0 removed item id -61 name 'h15' from crush map 2024-09-15T21:25:30.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5afb99a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T21:25:30.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5afb99a640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T21:25:30.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5afb99a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:30.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) -- 0x7f5af40008e0 con 0x7f5af41cd870 2024-09-15T21:25:30.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5af1ffb640 1 -- 172.21.15.119:0/928259134 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 device 'h15' does not appear in the crush map v234) ==== 120+0+0 (secure 0 0 0) 0x7f5ae4011db0 con 0x7f5af41cd870 2024-09-15T21:25:30.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5af1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T21:25:30.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.917+0000 7f5af1ffb640 10 monclient: _finish_command 3 = system:0 device 'h15' does not appear in the crush map 2024-09-15T21:25:30.922 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h15' does not appear in the crush map 2024-09-15T21:25:30.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae003b970 msgr2=0x7f5ae003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae003b970 0x7f5ae003de30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0ad000 tx=0x7f5aec0b2040 comp rx=0 tx=0).stop 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 10 monclient: shutdown 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 msgr2=0x7f5af41cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 0x7f5af41cdd00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5af406f0c0 tx=0x7f5ae400b4a0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 shutdown_connections 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae003b970 0x7f5ae003de30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af41cd870 0x7f5af41cdd00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5af4133d40 0x7f5af41341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 --2- 172.21.15.119:0/928259134 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5adc0039c0 0x7f5af411dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 >> 172.21.15.119:0/928259134 conn(0x7f5af40a2ae0 msgr2=0x7f5af40a4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 shutdown_connections 2024-09-15T21:25:30.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:30.921+0000 7f5afb99a640 1 -- 172.21.15.119:0/928259134 wait complete. 2024-09-15T21:25:30.939 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:30.939 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h16 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient: get_monmap_and_config 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient: build_initial_monmap 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient: monmap: 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:31.058233+0000 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:31.058233+0000 2024-09-15T21:25:31.057 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:31.058 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:31.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 Processor -- start 2024-09-15T21:25:31.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 -- start start 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient: init 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3640135ca0 0x7f3640136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): picked mon.noname-a con 0x7f3640135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36400a0160 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f364006ec30 0x7f36400a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:31.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): picked mon.noname-c con 0x7f364006ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:31.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36401365c0 con 0x7f36400a0160 2024-09-15T21:25:31.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:31.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f364009cf30 con 0x7f3640135ca0 2024-09-15T21:25:31.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:31.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f36400a1ca0 con 0x7f364006ec30 2024-09-15T21:25:31.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f3645a1f640 10 monclient(hunting): authenticate will time out at 1449.855088s 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363e7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f36400a0160 0x7f36400a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60326/0 (socket says 172.21.15.119:60326) 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363e7fc640 1 -- 172.21.15.119:0/3996669594 learned_addr learned my addr 172.21.15.119:0/3996669594 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 1 --2- 172.21.15.119:0/3996669594 >> v2:172.21.15.119:3300/0 conn(0x7f364006ec30 0x7f36400a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363effd640 1 -- 172.21.15.119:0/3996669594 >> v2:172.21.15.17:3300/0 conn(0x7f3640135ca0 msgr2=0x7f3640136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363effd640 1 --2- 172.21.15.119:0/3996669594 >> v2:172.21.15.17:3300/0 conn(0x7f3640135ca0 0x7f3640136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): get_auth_request con 0x7f364006ec30 auth_method 0 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:31.063 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient(hunting): handle_auth_done global_id 84553 payload 274 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 1 -- 172.21.15.119:0/3996669594 >> v2:172.21.15.17:3300/0 conn(0x7f3640135ca0 msgr2=0x7f3640136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 1 --2- 172.21.15.119:0/3996669594 >> v2:172.21.15.17:3300/0 conn(0x7f3640135ca0 0x7f3640136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 1 -- 172.21.15.119:0/3996669594 >> v1:172.21.15.119:6789/0 conn(0x7f36400a0160 legacy=0x7f36400a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363f7fe640 1 -- 172.21.15.119:0/3996669594 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36400a1f80 con 0x7f364006ec30 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363dffb640 1 -- 172.21.15.119:0/3996669594 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701404690 0 0) 0x7f36401365c0 con 0x7f36400a0160 2024-09-15T21:25:31.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.053+0000 7f363dffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363f7fe640 1 --2- 172.21.15.119:0/3996669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364006ec30 0x7f36400a10f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36400a29c0 tx=0x7f363001c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d48b18f4a2907 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 1 -- 172.21.15.119:0/3996669594 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300b0070 con 0x7f364006ec30 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: dump: 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:31.071 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:31.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 1 -- 172.21.15.119:0/3996669594 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36300ab600 con 0x7f364006ec30 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 1 -- 172.21.15.119:0/3996669594 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300ab9d0 con 0x7f364006ec30 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f363dffb640 10 monclient: dump: 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:31.073 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 5 monclient: authenticate success, global_id 84553 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 4 set_mon_vals no callback set 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 10 monclient: shutdown 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 -- 172.21.15.119:0/3996669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364006ec30 msgr2=0x7f36400a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 --2- 172.21.15.119:0/3996669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364006ec30 0x7f36400a10f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36400a29c0 tx=0x7f363001c360 comp rx=0 tx=0).stop 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 -- 172.21.15.119:0/3996669594 shutdown_connections 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 --2- 172.21.15.119:0/3996669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364006ec30 0x7f36400a10f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 --2- 172.21.15.119:0/3996669594 >> v2:172.21.15.17:3300/0 conn(0x7f3640135ca0 0x7f3640136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 -- 172.21.15.119:0/3996669594 >> 172.21.15.119:0/3996669594 conn(0x7f36400a2ae0 msgr2=0x7f36400a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 -- 172.21.15.119:0/3996669594 shutdown_connections 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 -- 172.21.15.119:0/3996669594 wait complete. 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 10 monclient: build_initial_monmap 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 10 monclient: monmap: 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:31.075 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 Processor -- start 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 1 -- start start 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.057+0000 7f3645a1f640 10 monclient: init 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f364006ec30 0x7f36401331d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): picked mon.noname-a con 0x7f364006ec30 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f36400a0160 0x7f3640133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): picked mon.noname-c con 0x7f36400a0160 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 0x7f364013b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:31.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): picked mon.noname-b con 0x7f3640135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3640140240 con 0x7f364006ec30 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36400a1840 con 0x7f3640135ca0 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f36401403c0 con 0x7f36400a0160 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient(hunting): authenticate will time out at 1449.860321s 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 0x7f364013b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363effd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f364006ec30 msgr2=0x7f36401331d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:31.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f364006ec30 0x7f36401331d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 0x7f364013b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42000/0 (socket says 172.21.15.119:42000) 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 -- 172.21.15.119:0/2607472018 learned_addr learned my addr 172.21.15.119:0/2607472018 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): get_auth_request con 0x7f3640135ca0 auth_method 0 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363e7fc640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f36400a0160 0x7f3640133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363e7fc640 10 monclient(hunting): get_auth_request con 0x7f36400a0160 auth_method 0 2024-09-15T21:25:31.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient(hunting): handle_auth_done global_id 84559 payload 824 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 -- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f36400a0160 msgr2=0x7f3640133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f36400a0160 0x7f3640133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 -- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f364006ec30 msgr2=0x7f36401331d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f364006ec30 0x7f36401331d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 -- 172.21.15.119:0/2607472018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364011dbd0 con 0x7f3640135ca0 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363f7fe640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 0x7f364013b8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3630019f90 tx=0x7f363001aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300b0040 con 0x7f3640135ca0 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: dump: 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:31.080 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3630004230 con 0x7f3640135ca0 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 5 monclient: authenticate success, global_id 84559 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _renew_subs 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36400a5730 con 0x7f3640135ca0 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630003620 con 0x7f3640135ca0 2024-09-15T21:25:31.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 10 monclient: dump: 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:31.082 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363d7fa640 4 set_mon_vals no callback set 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _renew_subs 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364011e190 con 0x7f3640135ca0 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3640136080 con 0x7f3640135ca0 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f36300ab4c0 con 0x7f3640135ca0 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f362403b970 0x7f362403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:31.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(234..234 src has 1..234) ==== 11816+0+0 (secure 0 0 0) 0x7f36300ec500 con 0x7f3640135ca0 2024-09-15T21:25:31.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363effd640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f362403b970 0x7f362403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:31.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363effd640 10 monclient: get_auth_request con 0x7f362403b970 auth_method 0 2024-09-15T21:25:31.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.061+0000 7f363effd640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f362403b970 0x7f362403de30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3638003940 tx=0x7f3638006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:31.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.065+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3630016070 con 0x7f3640135ca0 2024-09-15T21:25:31.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.065+0000 7f3637fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:31.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.065+0000 7f3637fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:31.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.241+0000 7f3645a1f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T21:25:31.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.241+0000 7f3645a1f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T21:25:31.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.241+0000 7f3645a1f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.241+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) -- 0x7f364011d950 con 0x7f3640135ca0 2024-09-15T21:25:31.834 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.829+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 removed item id -64 name 'h16' from crush map v235) ==== 120+0+0 (secure 0 0 0) 0x7f36300e9390 con 0x7f3640135ca0 2024-09-15T21:25:31.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.829+0000 7f3637fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T21:25:31.835 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.829+0000 7f3637fff640 10 monclient: _finish_command 2 = system:0 removed item id -64 name 'h16' from crush map 2024-09-15T21:25:31.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.937+0000 7f3645a1f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T21:25:31.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.937+0000 7f3645a1f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T21:25:31.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.937+0000 7f3645a1f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:31.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.937+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) -- 0x7f36400008e0 con 0x7f3640135ca0 2024-09-15T21:25:31.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3637fff640 1 -- 172.21.15.119:0/2607472018 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 device 'h16' does not appear in the crush map v235) ==== 120+0+0 (secure 0 0 0) 0x7f36300c0cf0 con 0x7f3640135ca0 2024-09-15T21:25:31.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3637fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T21:25:31.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3637fff640 10 monclient: _finish_command 3 = system:0 device 'h16' does not appear in the crush map 2024-09-15T21:25:31.944 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h16' does not appear in the crush map 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f362403b970 msgr2=0x7f362403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f362403b970 0x7f362403de30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3638003940 tx=0x7f3638006d90 comp rx=0 tx=0).stop 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 10 monclient: shutdown 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 msgr2=0x7f364013b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 0x7f364013b8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3630019f90 tx=0x7f363001aba0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 shutdown_connections 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f362403b970 0x7f362403de30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3640135ca0 0x7f364013b8e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.947 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f36400a0160 0x7f3640133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 --2- 172.21.15.119:0/2607472018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f364006ec30 0x7f36401331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:31.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 >> 172.21.15.119:0/2607472018 conn(0x7f36400a2ae0 msgr2=0x7f36400a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:31.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 shutdown_connections 2024-09-15T21:25:31.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:31.941+0000 7f3645a1f640 1 -- 172.21.15.119:0/2607472018 wait complete. 2024-09-15T21:25:31.961 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:31.961 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h17 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient: get_monmap_and_config 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient: build_initial_monmap 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient: monmap: 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:32.082185+0000 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:32.082185+0000 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:32.081 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:32.082 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:32.082 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:32.082 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 1 Processor -- start 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 1 -- start start 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient: init 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f45ec0a4ed0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f45ec0a59c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45ec0a5dd0 0x7f45ec140600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:32.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient(hunting): picked mon.noname-c con 0x7f45ec0a5dd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45ec0a6db0 con 0x7f45ec0a59c0 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45ec0a6ad0 con 0x7f45ec0a4ed0 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f45ec0a1210 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f3975640 10 monclient(hunting): authenticate will time out at 1450.879665s 2024-09-15T21:25:32.085 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f1711640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f45ec0a4ed0 0x7f45ec0a52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60332/0 (socket says 172.21.15.119:60332) 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f1711640 1 -- 172.21.15.119:0/1359385674 learned_addr learned my addr 172.21.15.119:0/1359385674 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f1f12640 1 --2- 172.21.15.119:0/1359385674 >> v2:172.21.15.119:3300/0 conn(0x7f45ec0a5dd0 0x7f45ec140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.077+0000 7f45f0f10640 1 -- 172.21.15.119:0/1359385674 >> v1:172.21.15.17:6789/0 conn(0x7f45ec0a59c0 legacy=0x7f45ec13ced0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): get_auth_request con 0x7f45ec0a5dd0 auth_method 0 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 1 -- 172.21.15.119:0/1359385674 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490184743 0 0) 0x7f45ec0a6ad0 con 0x7f45ec0a4ed0 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient(hunting): _negotiate 2024-09-15T21:25:32.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient(hunting): my global_id is 84562 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 1 -- 172.21.15.119:0/1359385674 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d8003530 con 0x7f45ec0a4ed0 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient(hunting): handle_auth_done global_id 84565 payload 274 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 1 -- 172.21.15.119:0/1359385674 >> v1:172.21.15.119:6789/0 conn(0x7f45ec0a4ed0 legacy=0x7f45ec0a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 1 -- 172.21.15.119:0/1359385674 >> v1:172.21.15.17:6789/0 conn(0x7f45ec0a59c0 legacy=0x7f45ec13ced0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient: _finish_hunting 0 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 1 monclient: found mon.noname-c 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 1 -- 172.21.15.119:0/1359385674 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec0a1450 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f1f12640 1 --2- 172.21.15.119:0/1359385674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec140600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009d90 tx=0x7f45e401c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee5f6c75bc543c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 1 -- 172.21.15.119:0/1359385674 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40b2070 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: dump: 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:32.092 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 1 -- 172.21.15.119:0/1359385674 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e40a5800 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.093 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 5 monclient: authenticate success, global_id 84565 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 1 -- 172.21.15.119:0/1359385674 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40a5bd0 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 4 set_mon_vals no callback set 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45e3fff640 10 monclient: dump: 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:32.094 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 10 monclient: shutdown 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 -- 172.21.15.119:0/1359385674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 msgr2=0x7f45ec140600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 --2- 172.21.15.119:0/1359385674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec140600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009d90 tx=0x7f45e401c830 comp rx=0 tx=0).stop 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 -- 172.21.15.119:0/1359385674 shutdown_connections 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 --2- 172.21.15.119:0/1359385674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec140600 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 -- 172.21.15.119:0/1359385674 >> 172.21.15.119:0/1359385674 conn(0x7f45ec0a6fd0 msgr2=0x7f45ec0a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 -- 172.21.15.119:0/1359385674 shutdown_connections 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 1 -- 172.21.15.119:0/1359385674 wait complete. 2024-09-15T21:25:32.095 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 10 monclient: build_initial_monmap 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.081+0000 7f45f3975640 10 monclient: monmap: 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 Processor -- start 2024-09-15T21:25:32.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- start start 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: init 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45ec0a4ed0 0x7f45ec127bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): picked mon.noname-a con 0x7f45ec0a4ed0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45ec0a59c0 0x7f45ec13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): picked mon.noname-c con 0x7f45ec0a59c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec125680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:32.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): picked mon.noname-b con 0x7f45ec0a5dd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec1438f0 con 0x7f45ec0a4ed0 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec143770 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec143a70 con 0x7f45ec0a59c0 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient(hunting): authenticate will time out at 1450.884820s 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1711640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45ec0a4ed0 msgr2=0x7f45ec127bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec125680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1711640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45ec0a4ed0 0x7f45ec127bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec125680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42020/0 (socket says 172.21.15.119:42020) 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 -- 172.21.15.119:0/933032436 learned_addr learned my addr 172.21.15.119:0/933032436 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:32.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): get_auth_request con 0x7f45ec0a5dd0 auth_method 0 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f0f10640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45ec0a59c0 0x7f45ec13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient(hunting): handle_auth_done global_id 84568 payload 824 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 -- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45ec0a59c0 msgr2=0x7f45ec13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45ec0a59c0 0x7f45ec13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 -- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45ec0a4ed0 msgr2=0x7f45ec127bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45ec0a4ed0 0x7f45ec127bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient: _finish_hunting 0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 monclient: found mon.noname-b 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 -- 172.21.15.119:0/933032436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec1d09d0 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f0f10640 10 monclient: get_auth_request con 0x7f45ec0a59c0 auth_method 0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f0f10640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45ec0a59c0 0x7f45ec13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1f12640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec125680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f45e401aee0 tx=0x7f45e40195d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40b2050 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: dump: 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:32.100 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e40be070 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 5 monclient: authenticate success, global_id 84568 2024-09-15T21:25:32.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _renew_subs 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec0a9c80 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40ad850 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 10 monclient: dump: 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:32.102 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _renew_subs 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e37fe640 4 set_mon_vals no callback set 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec128540 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f45e40ad9f0 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ec0a52b0 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f45d003b970 0x7f45d003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 11603+0+0 (secure 0 0 0) 0x7f45e40ec340 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1711640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f45d003b970 0x7f45d003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1711640 10 monclient: get_auth_request con 0x7f45d003b970 auth_method 0 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.085+0000 7f45f1711640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f45d003b970 0x7f45d003de30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f45e8004820 tx=0x7f45e8007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:32.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.089+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f45e4016540 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.089+0000 7f45e1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:32.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.089+0000 7f45e1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:32.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.265+0000 7f45f3975640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T21:25:32.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.265+0000 7f45f3975640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T21:25:32.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.265+0000 7f45f3975640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.265+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) -- 0x7f45ec06c480 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.833+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 removed item id -67 name 'h17' from crush map v236) ==== 120+0+0 (secure 0 0 0) 0x7f45e40e92a0 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.833+0000 7f45e1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T21:25:32.838 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.833+0000 7f45e1ffb640 10 monclient: _finish_command 2 = system:0 removed item id -67 name 'h17' from crush map 2024-09-15T21:25:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45f3975640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T21:25:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45f3975640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T21:25:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45f3975640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:32.949 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) -- 0x7f45ec126380 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45e1ffb640 1 -- 172.21.15.119:0/933032436 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 device 'h17' does not appear in the crush map v236) ==== 120+0+0 (secure 0 0 0) 0x7f45e40ec120 con 0x7f45ec0a5dd0 2024-09-15T21:25:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45e1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T21:25:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.945+0000 7f45e1ffb640 10 monclient: _finish_command 3 = system:0 device 'h17' does not appear in the crush map 2024-09-15T21:25:32.950 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h17' does not appear in the crush map 2024-09-15T21:25:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f45d003b970 msgr2=0x7f45d003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f45d003b970 0x7f45d003de30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f45e8004820 tx=0x7f45e8007a20 comp rx=0 tx=0).stop 2024-09-15T21:25:32.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 10 monclient: shutdown 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 msgr2=0x7f45ec125680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec125680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f45e401aee0 tx=0x7f45e40195d0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 shutdown_connections 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f45d003b970 0x7f45d003de30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ec0a5dd0 0x7f45ec125680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f45ec0a59c0 0x7f45ec13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 --2- 172.21.15.119:0/933032436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45ec0a4ed0 0x7f45ec127bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 >> 172.21.15.119:0/933032436 conn(0x7f45ec0a6fd0 msgr2=0x7f45ec0a8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 shutdown_connections 2024-09-15T21:25:32.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:32.949+0000 7f45f3975640 1 -- 172.21.15.119:0/933032436 wait complete. 2024-09-15T21:25:32.967 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:32.968 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h18 2024-09-15T21:25:33.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.081+0000 7f0e039bf640 10 monclient: get_monmap_and_config 2024-09-15T21:25:33.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.081+0000 7f0e039bf640 10 monclient: build_initial_monmap 2024-09-15T21:25:33.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.081+0000 7f0e039bf640 10 monclient: monmap: 2024-09-15T21:25:33.086 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:33.087773+0000 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:33.087773+0000 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:33.087 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:33.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 Processor -- start 2024-09-15T21:25:33.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 -- start start 2024-09-15T21:25:33.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient: init 2024-09-15T21:25:33.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:33.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2024-09-15T21:25:33.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0dfc0696f0 0x7f0dfc073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): picked mon.noname-c con 0x7f0dfc0696f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc073980 0x7f0dfc134550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): picked mon.noname-b con 0x7f0dfc073980 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0dfc134a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dfc06dc30 con 0x7f0dfc134a90 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:33.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0dfc069ad0 con 0x7f0dfc0696f0 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc073d60 con 0x7f0dfc073980 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e039bf640 10 monclient(hunting): authenticate will time out at 1451.885066s 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0dfc0696f0 0x7f0dfc073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e01f5c640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f0dfc134a90 legacy=0x7f0dfc13b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0dfc0696f0 0x7f0dfc073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45602/0 (socket says 172.21.15.119:45602) 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 -- 172.21.15.119:0/3418336031 learned_addr learned my addr 172.21.15.119:0/3418336031 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): get_auth_request con 0x7f0dfc0696f0 auth_method 0 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:33.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:33.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:33.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:33.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:33.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e00f5a640 1 --2- 172.21.15.119:0/3418336031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc073980 0x7f0dfc134550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:33.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e00f5a640 10 monclient(hunting): get_auth_request con 0x7f0dfc073980 auth_method 0 2024-09-15T21:25:33.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e00f5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e00f5a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e00f5a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient(hunting): handle_auth_done global_id 84571 payload 274 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 -- 172.21.15.119:0/3418336031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc073980 msgr2=0x7f0dfc134550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 --2- 172.21.15.119:0/3418336031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc073980 0x7f0dfc134550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 -- 172.21.15.119:0/3418336031 >> v1:172.21.15.17:6789/0 conn(0x7f0dfc134a90 legacy=0x7f0dfc13b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 monclient: found mon.noname-c 2024-09-15T21:25:33.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 -- 172.21.15.119:0/3418336031 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc071d30 con 0x7f0dfc0696f0 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0e0175b640 1 --2- 172.21.15.119:0/3418336031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc0696f0 0x7f0dfc073440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0df80098a0 tx=0x7f0df801c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56059223b766093e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0df3fff640 1 -- 172.21.15.119:0/3418336031 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80b0070 con 0x7f0dfc0696f0 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0df3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0df3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.085+0000 7f0df3fff640 10 monclient: dump: 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:33.105 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 5 monclient: authenticate success, global_id 84571 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 1 -- 172.21.15.119:0/3418336031 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df80ab600 con 0x7f0dfc0696f0 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 4 set_mon_vals no callback set 2024-09-15T21:25:33.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 1 -- 172.21.15.119:0/3418336031 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80ab9d0 con 0x7f0dfc0696f0 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df3fff640 10 monclient: dump: 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:33.107 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: shutdown 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/3418336031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc0696f0 msgr2=0x7f0dfc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 --2- 172.21.15.119:0/3418336031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc0696f0 0x7f0dfc073440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0df80098a0 tx=0x7f0df801c400 comp rx=0 tx=0).stop 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/3418336031 shutdown_connections 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 --2- 172.21.15.119:0/3418336031 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc073980 0x7f0dfc134550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 --2- 172.21.15.119:0/3418336031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc0696f0 0x7f0dfc073440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/3418336031 >> 172.21.15.119:0/3418336031 conn(0x7f0dfc0a3b80 msgr2=0x7f0dfc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/3418336031 shutdown_connections 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/3418336031 wait complete. 2024-09-15T21:25:33.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: build_initial_monmap 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: monmap: 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 Processor -- start 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- start start 2024-09-15T21:25:33.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: init 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc0696f0 0x7f0dfc1cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): picked mon.noname-c con 0x7f0dfc0696f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 0x7f0dfc1314b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): picked mon.noname-b con 0x7f0dfc073980 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dfc134a90 0x7f0dfc13a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): picked mon.noname-a con 0x7f0dfc134a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0a60d0 con 0x7f0dfc134a90 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0a5ca0 con 0x7f0dfc073980 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:33.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc0a5e20 con 0x7f0dfc0696f0 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient(hunting): authenticate will time out at 1451.890275s 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 0x7f0dfc1314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 0x7f0dfc1314b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45614/0 (socket says 172.21.15.119:45614) 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e01f5c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dfc134a90 msgr2=0x7f0dfc13a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 -- 172.21.15.119:0/64002560 learned_addr learned my addr 172.21.15.119:0/64002560 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): get_auth_request con 0x7f0dfc073980 auth_method 0 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:33.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e01f5c640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dfc134a90 0x7f0dfc13a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e0175b640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc0696f0 0x7f0dfc1cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e0175b640 10 monclient(hunting): get_auth_request con 0x7f0dfc0696f0 auth_method 0 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e0175b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e0175b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e0175b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient(hunting): handle_auth_done global_id 84574 payload 824 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 -- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc0696f0 msgr2=0x7f0dfc1cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc0696f0 0x7f0dfc1cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 -- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dfc134a90 msgr2=0x7f0dfc13a8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dfc134a90 0x7f0dfc13a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient: _finish_hunting 0 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 monclient: found mon.noname-b 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 -- 172.21.15.119:0/64002560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc0a3020 con 0x7f0dfc073980 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e00f5a640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 0x7f0dfc1314b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00d800 tx=0x7f0dec00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:33.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00b6c0 con 0x7f0dfc073980 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: dump: 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:33.114 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 5 monclient: authenticate success, global_id 84574 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: _renew_subs 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec019070 con 0x7f0dfc073980 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc0744c0 con 0x7f0dfc073980 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00ba60 con 0x7f0dfc073980 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:33.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df1ffb640 10 monclient: dump: 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: _renew_subs 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0df37fe640 4 set_mon_vals no callback set 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.089+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc11ef70 con 0x7f0dfc073980 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e039bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e039bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e039bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc073440 con 0x7f0dfc073980 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0dec00bc00 con 0x7f0dfc073980 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0df1ffb640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0de003b9a0 0x7f0de003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 11390+0+0 (secure 0 0 0) 0x7f0dec051a30 con 0x7f0dfc073980 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e0175b640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0de003b9a0 0x7f0de003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e0175b640 10 monclient: get_auth_request con 0x7f0de003b9a0 auth_method 0 2024-09-15T21:25:33.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.093+0000 7f0e0175b640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0de003b9a0 0x7f0de003de60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0df80ad000 tx=0x7f0df8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:33.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.097+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0dec04ea70 con 0x7f0dfc073980 2024-09-15T21:25:33.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.097+0000 7f0df1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:33.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.097+0000 7f0df1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:33.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.273+0000 7f0e039bf640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T21:25:33.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.273+0000 7f0e039bf640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T21:25:33.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.273+0000 7f0e039bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.273+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) -- 0x7f0dfc11d8f0 con 0x7f0dfc073980 2024-09-15T21:25:33.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.849+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 removed item id -70 name 'h18' from crush map v237) ==== 120+0+0 (secure 0 0 0) 0x7f0dec01cd70 con 0x7f0dfc073980 2024-09-15T21:25:33.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.849+0000 7f0df1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T21:25:33.853 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.849+0000 7f0df1ffb640 10 monclient: _finish_command 2 = system:0 removed item id -70 name 'h18' from crush map 2024-09-15T21:25:33.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0e039bf640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T21:25:33.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0e039bf640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T21:25:33.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0e039bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:33.961 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) -- 0x7f0dfc0008e0 con 0x7f0dfc073980 2024-09-15T21:25:33.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0df1ffb640 1 -- 172.21.15.119:0/64002560 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 device 'h18' does not appear in the crush map v237) ==== 120+0+0 (secure 0 0 0) 0x7f0dec01cad0 con 0x7f0dfc073980 2024-09-15T21:25:33.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0df1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T21:25:33.962 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.957+0000 7f0df1ffb640 10 monclient: _finish_command 3 = system:0 device 'h18' does not appear in the crush map 2024-09-15T21:25:33.962 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h18' does not appear in the crush map 2024-09-15T21:25:33.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0de003b9a0 msgr2=0x7f0de003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:33.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0de003b9a0 0x7f0de003de60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0df80ad000 tx=0x7f0df8008040 comp rx=0 tx=0).stop 2024-09-15T21:25:33.964 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 10 monclient: shutdown 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 msgr2=0x7f0dfc1314b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 0x7f0dfc1314b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00d800 tx=0x7f0dec00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 shutdown_connections 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0de003b9a0 0x7f0de003de60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dfc134a90 0x7f0dfc13a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc073980 0x7f0dfc1314b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 --2- 172.21.15.119:0/64002560 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dfc0696f0 0x7f0dfc1cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 >> 172.21.15.119:0/64002560 conn(0x7f0dfc0a3b80 msgr2=0x7f0dfc06f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 shutdown_connections 2024-09-15T21:25:33.965 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:33.961+0000 7f0e039bf640 1 -- 172.21.15.119:0/64002560 wait complete. 2024-09-15T21:25:33.978 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:33.979 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h19 2024-09-15T21:25:34.096 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient: get_monmap_and_config 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient: build_initial_monmap 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient: monmap: 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:34.097964+0000 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:34.097964+0000 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:34.097 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:34.098 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:34.098 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:34.098 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:34.098 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 Processor -- start 2024-09-15T21:25:34.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 -- start start 2024-09-15T21:25:34.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient: init 2024-09-15T21:25:34.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:34.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 0x7f6b2409d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): picked mon.noname-b con 0x7f6b2409f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b2409fa20 0x7f6b2409fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): picked mon.noname-a con 0x7f6b2409fa20 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6b241347b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b2409e880 con 0x7f6b241347b0 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2409eb60 con 0x7f6b2409fa20 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2409ee40 con 0x7f6b2409f640 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b28e2e640 10 monclient(hunting): authenticate will time out at 1452.895244s 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b19d9b640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f6b2409fa20 msgr2=0x7f6b2409fe80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b19d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b2409fa20 0x7f6b2409fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b22d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6b241347b0 0x7f6b24134c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40768/0 (socket says 172.21.15.119:40768) 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b22d9d640 1 -- 172.21.15.119:0/1023792713 learned_addr learned my addr 172.21.15.119:0/1023792713 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 0x7f6b2409d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): get_auth_request con 0x7f6b2409f640 auth_method 0 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:34.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051174769 0 0) 0x7f6b2409e880 con 0x7f6b241347b0 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 10 monclient(hunting): my global_id is 84577 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b10005740 con 0x7f6b241347b0 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.093+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2713598652 0 0) 0x7f6b10005740 con 0x7f6b241347b0 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 msgr2=0x7f6b2409d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 0x7f6b2409d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 >> v2:172.21.15.17:3300/0 conn(0x7f6b2409fa20 msgr2=0x7f6b2409fe80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:34.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 --2- 172.21.15.119:0/1023792713 >> v2:172.21.15.17:3300/0 conn(0x7f6b2409fa20 0x7f6b2409fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b240a2840 con 0x7f6b241347b0 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: _finish_auth 0 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f6b1c001a80 con 0x7f6b241347b0 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 5 monclient: authenticate success, global_id 84577 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:34.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: dump: 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 >> v1:172.21.15.119:6789/0 conn(0x7f6b241347b0 legacy=0x7f6b24134c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b10005bd0 0x7f6b10005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): picked mon.a con 0x7f6b10005bd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 0x7f6b10006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): picked mon.c con 0x7f6b2409f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10006f60 0x7f6b100073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): picked mon.b con 0x7f6b10006f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2409eb60 con 0x7f6b10005bd0 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2409ee40 con 0x7f6b10006f60 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1000bbd0 con 0x7f6b2409f640 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b2259c640 1 -- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b10005bd0 msgr2=0x7f6b10005ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b2259c640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b10005bd0 0x7f6b10005ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10006f60 0x7f6b100073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:34.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): get_auth_request con 0x7f6b10006f60 auth_method 0 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b19d9b640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 0x7f6b10006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b19d9b640 10 monclient(hunting): get_auth_request con 0x7f6b2409f640 auth_method 0 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b19d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b19d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_done global_id 84577 payload 293 2024-09-15T21:25:34.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 -- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 msgr2=0x7f6b10006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2409f640 0x7f6b10006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 -- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b10005bd0 msgr2=0x7f6b10005ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b10005bd0 0x7f6b10005ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 monclient: found mon.b 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 -- 172.21.15.119:0/1023792713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b10005740 con 0x7f6b10006f60 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient: _finish_auth 0 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 10 monclient: _check_auth_tickets 2024-09-15T21:25:34.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10006f60 0x7f6b100073e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00a880 tx=0x7f6b1c01d2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c890ac2d01f02a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00a4a0 con 0x7f6b10006f60 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: dump: 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:34.123 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c0a9070 con 0x7f6b10006f60 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 1 -- 172.21.15.119:0/1023792713 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c0ae6b0 con 0x7f6b10006f60 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 4 set_mon_vals no callback set 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b21d9b640 10 monclient: dump: 2024-09-15T21:25:34.124 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 10 monclient: shutdown 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 1 -- 172.21.15.119:0/1023792713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10006f60 msgr2=0x7f6b100073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:34.125 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/1023792713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10006f60 0x7f6b100073e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00a880 tx=0x7f6b1c01d2a0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b22d9d640 1 -- 172.21.15.119:0/1023792713 reap_dead start 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 1 -- 172.21.15.119:0/1023792713 shutdown_connections 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 1 -- 172.21.15.119:0/1023792713 >> 172.21.15.119:0/1023792713 conn(0x7f6b240a2ae0 msgr2=0x7f6b240a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 1 -- 172.21.15.119:0/1023792713 shutdown_connections 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.097+0000 7f6b28e2e640 1 -- 172.21.15.119:0/1023792713 wait complete. 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: build_initial_monmap 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: monmap: 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:34.126 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 Processor -- start 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- start start 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: init 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b2411eb40 0x7f6b24133970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): picked mon.noname-a con 0x7f6b2411eb40 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2411cb10 0x7f6b24133eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): picked mon.noname-c con 0x7f6b2411cb10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 0x7f6b241d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): picked mon.noname-b con 0x7f6b241cffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6b240a1a80 con 0x7f6b2411eb40 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2409ddb0 con 0x7f6b241cffc0 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6b240a1c00 con 0x7f6b2411cb10 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient(hunting): authenticate will time out at 1452.901118s 2024-09-15T21:25:34.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b2259c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b2411eb40 msgr2=0x7f6b24133970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b2259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b2411eb40 0x7f6b24133970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 0x7f6b241d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 0x7f6b241d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45630/0 (socket says 172.21.15.119:45630) 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 -- 172.21.15.119:0/2076282293 learned_addr learned my addr 172.21.15.119:0/2076282293 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): get_auth_request con 0x7f6b241cffc0 auth_method 0 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b19d9b640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2411cb10 0x7f6b24133eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:34.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b19d9b640 10 monclient(hunting): get_auth_request con 0x7f6b2411cb10 auth_method 0 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b19d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient(hunting): handle_auth_done global_id 84580 payload 824 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 -- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2411cb10 msgr2=0x7f6b24133eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2411cb10 0x7f6b24133eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 -- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b2411eb40 msgr2=0x7f6b24133970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b2411eb40 0x7f6b24133970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 monclient: found mon.noname-b 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 -- 172.21.15.119:0/2076282293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b240a5760 con 0x7f6b241cffc0 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b22d9d640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 0x7f6b241d03d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c098b00 tx=0x7f6b10007a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c001ff0 con 0x7f6b241cffc0 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: dump: 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.131 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c0ae670 con 0x7f6b241cffc0 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 5 monclient: authenticate success, global_id 84580 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _renew_subs 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b24120340 con 0x7f6b241cffc0 2024-09-15T21:25:34.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c0aea20 con 0x7f6b241cffc0 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 10 monclient: dump: 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:34.133 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b2159a640 4 set_mon_vals no callback set 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _renew_subs 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2411faa0 con 0x7f6b241cffc0 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b240a1f60 con 0x7f6b241cffc0 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6b1c0aebc0 con 0x7f6b241cffc0 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6b0403b970 0x7f6b0403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(237..237 src has 1..237) ==== 11177+0+0 (secure 0 0 0) 0x7f6b1c0eb370 con 0x7f6b241cffc0 2024-09-15T21:25:34.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b2259c640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6b0403b970 0x7f6b0403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:34.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b2259c640 10 monclient: get_auth_request con 0x7f6b0403b970 auth_method 0 2024-09-15T21:25:34.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.101+0000 7f6b2259c640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6b0403b970 0x7f6b0403de30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6b140094c0 tx=0x7f6b14009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:34.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.105+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6b240a5760 con 0x7f6b241cffc0 2024-09-15T21:25:34.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.105+0000 7f6b1b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:34.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.105+0000 7f6b1b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:34.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.281+0000 7f6b28e2e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T21:25:34.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.281+0000 7f6b28e2e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T21:25:34.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.281+0000 7f6b28e2e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.281+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) -- 0x7f6b24070320 con 0x7f6b241cffc0 2024-09-15T21:25:34.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.857+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 removed item id -73 name 'h19' from crush map v238) ==== 120+0+0 (secure 0 0 0) 0x7f6b1c017050 con 0x7f6b241cffc0 2024-09-15T21:25:34.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.857+0000 7f6b1b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T21:25:34.862 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.857+0000 7f6b1b7fe640 10 monclient: _finish_command 2 = system:0 removed item id -73 name 'h19' from crush map 2024-09-15T21:25:34.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b28e2e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T21:25:34.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b28e2e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T21:25:34.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b28e2e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:34.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) -- 0x7f6b2411fc70 con 0x7f6b241cffc0 2024-09-15T21:25:34.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b1b7fe640 1 -- 172.21.15.119:0/2076282293 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 device 'h19' does not appear in the crush map v238) ==== 120+0+0 (secure 0 0 0) 0x7f6b1c0e8480 con 0x7f6b241cffc0 2024-09-15T21:25:34.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b1b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T21:25:34.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.965+0000 7f6b1b7fe640 10 monclient: _finish_command 3 = system:0 device 'h19' does not appear in the crush map 2024-09-15T21:25:34.970 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h19' does not appear in the crush map 2024-09-15T21:25:34.971 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6b0403b970 msgr2=0x7f6b0403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6b0403b970 0x7f6b0403de30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6b140094c0 tx=0x7f6b14009190 comp rx=0 tx=0).stop 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 10 monclient: shutdown 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 msgr2=0x7f6b241d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 0x7f6b241d03d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c098b00 tx=0x7f6b10007a50 comp rx=0 tx=0).stop 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 shutdown_connections 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6b0403b970 0x7f6b0403de30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b241cffc0 0x7f6b241d03d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6b2411cb10 0x7f6b24133eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 --2- 172.21.15.119:0/2076282293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b2411eb40 0x7f6b24133970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 >> 172.21.15.119:0/2076282293 conn(0x7f6b240a2ae0 msgr2=0x7f6b240a4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:34.972 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 shutdown_connections 2024-09-15T21:25:34.973 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:34.969+0000 7f6b28e2e640 1 -- 172.21.15.119:0/2076282293 wait complete. 2024-09-15T21:25:34.987 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:34.987 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h20 2024-09-15T21:25:35.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient: get_monmap_and_config 2024-09-15T21:25:35.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient: build_initial_monmap 2024-09-15T21:25:35.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient: monmap: 2024-09-15T21:25:35.097 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:35.097 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:35.097 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:35.098400+0000 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:35.098400+0000 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:35.098 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 Processor -- start 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 -- start start 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient: init 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff0600a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): picked mon.noname-b con 0x7ff0600a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff0600a5780 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0601383a0 0x7ff06013ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): picked mon.noname-c con 0x7ff0601383a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0600a1150 con 0x7ff0600a5780 2024-09-15T21:25:35.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff0600a3c80 con 0x7ff0601383a0 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff0600a0ef0 con 0x7ff0600a4e60 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff06586f640 10 monclient(hunting): authenticate will time out at 1453.895129s 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0601383a0 0x7ff06013ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0601383a0 0x7ff06013ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45634/0 (socket says 172.21.15.119:45634) 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 -- 172.21.15.119:0/1822531720 learned_addr learned my addr 172.21.15.119:0/1822531720 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): get_auth_request con 0x7ff0601383a0 auth_method 0 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:35.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_done global_id 84583 payload 274 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05effd640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff0600a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 -- 172.21.15.119:0/1822531720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 msgr2=0x7ff0600a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff0600a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.102 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 -- 172.21.15.119:0/1822531720 >> v1:172.21.15.119:6789/0 conn(0x7ff0600a5780 legacy=0x7ff060137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05f7fe640 1 -- 172.21.15.119:0/1822531720 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0600a6cf0 con 0x7ff0601383a0 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05effd640 10 monclient: get_auth_request con 0x7ff0600a4e60 auth_method 0 2024-09-15T21:25:35.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.093+0000 7ff05effd640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff0600a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:35.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05f7fe640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff06013ff60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff058009e50 tx=0x7ff05801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=182bb015d7bc82ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:35.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 1 -- 172.21.15.119:0/1822531720 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a55b0 con 0x7ff0601383a0 2024-09-15T21:25:35.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: dump: 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:35.111 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:35.111 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:35.111 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:35.111 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:35.111 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:35.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 5 monclient: authenticate success, global_id 84583 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 1 -- 172.21.15.119:0/1822531720 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580a5750 con 0x7ff0601383a0 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 4 set_mon_vals no callback set 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 1 -- 172.21.15.119:0/1822531720 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a5b20 con 0x7ff0601383a0 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff05dffb640 10 monclient: dump: 2024-09-15T21:25:35.112 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: shutdown 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 -- 172.21.15.119:0/1822531720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 msgr2=0x7ff06013ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff06013ff60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff058009e50 tx=0x7ff05801c8f0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 -- 172.21.15.119:0/1822531720 shutdown_connections 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff06013ff60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 --2- 172.21.15.119:0/1822531720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff0600a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 -- 172.21.15.119:0/1822531720 >> 172.21.15.119:0/1822531720 conn(0x7ff0600a6f60 msgr2=0x7ff0600a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 -- 172.21.15.119:0/1822531720 shutdown_connections 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 -- 172.21.15.119:0/1822531720 wait complete. 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: build_initial_monmap 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: monmap: 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:35.114 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 Processor -- start 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 1 -- start start 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: init 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.097+0000 7ff06586f640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff06013f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): picked mon.noname-c con 0x7ff0600a4e60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0600a5780 0x7ff06013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:35.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): picked mon.noname-a con 0x7ff0600a5780 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff060071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): picked mon.noname-b con 0x7ff0601383a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600a7e30 con 0x7ff0600a5780 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600a7cb0 con 0x7ff0601383a0 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff0600a2190 con 0x7ff0600a4e60 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient(hunting): authenticate will time out at 1453.900084s 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff060071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff060071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45648/0 (socket says 172.21.15.119:45648) 2024-09-15T21:25:35.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 -- 172.21.15.119:0/4225466466 learned_addr learned my addr 172.21.15.119:0/4225466466 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): get_auth_request con 0x7ff0601383a0 auth_method 0 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05e7fc640 1 -- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0600a5780 msgr2=0x7ff06013f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05e7fc640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0600a5780 0x7ff06013f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05effd640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff06013f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient(hunting): handle_auth_done global_id 84589 payload 824 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 -- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 msgr2=0x7ff06013f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:35.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff06013f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 -- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0600a5780 msgr2=0x7ff06013f8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0600a5780 0x7ff06013f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 -- 172.21.15.119:0/4225466466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0600a9530 con 0x7ff0601383a0 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05f7fe640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff060071660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0580af040 tx=0x7ff058019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580bc070 con 0x7ff0601383a0 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: dump: 2024-09-15T21:25:35.118 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:35.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 5 monclient: authenticate success, global_id 84589 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580b8640 con 0x7ff0601383a0 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580b8a30 con 0x7ff0601383a0 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 10 monclient: dump: 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:35.120 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05d7fa640 4 set_mon_vals no callback set 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient: _renew_subs 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0600a1650 con 0x7ff0601383a0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient: _renew_subs 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff060073120 con 0x7ff0601383a0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff0580b8bd0 con 0x7ff0601383a0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0600a5240 con 0x7ff0601383a0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff04c03b970 0x7ff04c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 10964+0+0 (secure 0 0 0) 0x7ff0580ebd00 con 0x7ff0601383a0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05effd640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff04c03b970 0x7ff04c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05effd640 10 monclient: get_auth_request con 0x7ff04c03b970 auth_method 0 2024-09-15T21:25:35.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.101+0000 7ff05effd640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff04c03b970 0x7ff04c03de30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff054006fd0 tx=0x7ff054008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:35.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.105+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff058016600 con 0x7ff0601383a0 2024-09-15T21:25:35.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.105+0000 7ff047fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:35.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.105+0000 7ff047fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:35.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.269+0000 7ff06586f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T21:25:35.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.269+0000 7ff06586f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T21:25:35.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.269+0000 7ff06586f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.269+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) -- 0x7ff060077100 con 0x7ff0601383a0 2024-09-15T21:25:35.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.865+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 removed item id -76 name 'h20' from crush map v239) ==== 120+0+0 (secure 0 0 0) 0x7ff0580a56f0 con 0x7ff0601383a0 2024-09-15T21:25:35.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.865+0000 7ff047fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T21:25:35.870 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.865+0000 7ff047fff640 10 monclient: _finish_command 2 = system:0 removed item id -76 name 'h20' from crush map 2024-09-15T21:25:35.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff06586f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T21:25:35.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff06586f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T21:25:35.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff06586f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:35.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) -- 0x7ff06006c390 con 0x7ff0601383a0 2024-09-15T21:25:35.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff047fff640 1 -- 172.21.15.119:0/4225466466 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 device 'h20' does not appear in the crush map v239) ==== 120+0+0 (secure 0 0 0) 0x7ff0580e8ee0 con 0x7ff0601383a0 2024-09-15T21:25:35.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff047fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T21:25:35.981 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.977+0000 7ff047fff640 10 monclient: _finish_command 3 = system:0 device 'h20' does not appear in the crush map 2024-09-15T21:25:35.981 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h20' does not appear in the crush map 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff04c03b970 msgr2=0x7ff04c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff04c03b970 0x7ff04c03de30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff054006fd0 tx=0x7ff054008040 comp rx=0 tx=0).stop 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 10 monclient: shutdown 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 msgr2=0x7ff060071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff060071660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0580af040 tx=0x7ff058019690 comp rx=0 tx=0).stop 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 shutdown_connections 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff04c03b970 0x7ff04c03de30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0601383a0 0x7ff060071660 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0600a5780 0x7ff06013f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.984 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 --2- 172.21.15.119:0/4225466466 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0600a4e60 0x7ff06013f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:35.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 >> 172.21.15.119:0/4225466466 conn(0x7ff0600a6f60 msgr2=0x7ff060133e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:35.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 shutdown_connections 2024-09-15T21:25:35.985 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:35.981+0000 7ff06586f640 1 -- 172.21.15.119:0/4225466466 wait complete. 2024-09-15T21:25:35.998 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:35.999 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h21 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient: get_monmap_and_config 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient: build_initial_monmap 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient: monmap: 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:36.106256+0000 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:36.106256+0000 2024-09-15T21:25:36.105 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:36.106 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 Processor -- start 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 -- start start 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient: init 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd228137bc0 0x7fd228137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): picked mon.noname-a con 0x7fd228137bc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2280a09e0 0x7fd228073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): picked mon.noname-b con 0x7fd2280a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:36.109 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2280a0ea0 0x7fd228073580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): picked mon.noname-c con 0x7fd2280a0ea0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd22809cfa0 con 0x7fd228137bc0 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd22809cd20 con 0x7fd2280a0ea0 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd2280a1820 con 0x7fd2280a09e0 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22f7d2640 10 monclient(hunting): authenticate will time out at 1454.903427s 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22d56e640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fd228137bc0 msgr2=0x7fd228137fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:36.110 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22d56e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd228137bc0 0x7fd228137fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2280a0ea0 0x7fd228073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2280a0ea0 0x7fd228073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45662/0 (socket says 172.21.15.119:45662) 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 1 -- 172.21.15.119:0/1702947623 learned_addr learned my addr 172.21.15.119:0/1702947623 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): get_auth_request con 0x7fd2280a0ea0 auth_method 0 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22cd6d640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2280a09e0 0x7fd228073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:36.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22cd6d640 10 monclient(hunting): get_auth_request con 0x7fd2280a09e0 auth_method 0 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22cd6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22cd6d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22cd6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 10 monclient(hunting): handle_auth_done global_id 84592 payload 274 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 1 -- 172.21.15.119:0/1702947623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2280a09e0 msgr2=0x7fd228073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.101+0000 7fd22dd6f640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2280a09e0 0x7fd228073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 1 -- 172.21.15.119:0/1702947623 >> v2:172.21.15.17:3300/0 conn(0x7fd228137bc0 msgr2=0x7fd228137fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 1 --2- 172.21.15.119:0/1702947623 >> v2:172.21.15.17:3300/0 conn(0x7fd228137bc0 0x7fd228137fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 10 monclient: _finish_hunting 0 2024-09-15T21:25:36.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 1 monclient: found mon.noname-c 2024-09-15T21:25:36.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:36.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 1 -- 172.21.15.119:0/1702947623 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2280a1b00 con 0x7fd2280a0ea0 2024-09-15T21:25:36.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22cd6d640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2280a09e0 0x7fd228073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:36.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22dd6f640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd228073580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd228069a90 tx=0x7fd22001c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fafb7ef53eae270c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:36.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 1 -- 172.21.15.119:0/1702947623 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200a56f0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:36.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:36.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: dump: 2024-09-15T21:25:36.119 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: _finish_auth 0 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:36.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 5 monclient: authenticate success, global_id 84592 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 1 -- 172.21.15.119:0/1702947623 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2200a5890 con 0x7fd2280a0ea0 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 1 -- 172.21.15.119:0/1702947623 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200a5c60 con 0x7fd2280a0ea0 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 4 set_mon_vals no callback set 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:36.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd21ffff640 10 monclient: dump: 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:36.122 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 10 monclient: shutdown 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1702947623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 msgr2=0x7fd228073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd228073580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd228069a90 tx=0x7fd22001c960 comp rx=0 tx=0).stop 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1702947623 shutdown_connections 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd228073580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1702947623 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd2280a09e0 0x7fd228073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1702947623 >> v2:172.21.15.17:3300/0 conn(0x7fd228137bc0 0x7fd228137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1702947623 >> 172.21.15.119:0/1702947623 conn(0x7fd2280a2ae0 msgr2=0x7fd2280a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1702947623 shutdown_connections 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1702947623 wait complete. 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 10 monclient: build_initial_monmap 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.105+0000 7fd22f7d2640 10 monclient: monmap: 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:36.124 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:36.125 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:36.125 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:36.125 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:36.126 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:36.126 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:36.126 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:36.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 Processor -- start 2024-09-15T21:25:36.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- start start 2024-09-15T21:25:36.126 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: init 2024-09-15T21:25:36.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:36.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:36.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2280a09e0 0x7fd22811eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): picked mon.noname-a con 0x7fd2280a09e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:36.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd22811caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): picked mon.noname-b con 0x7fd2280a0ea0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:36.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd228137bc0 0x7fd228134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): picked mon.noname-c con 0x7fd228137bc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:36.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:36.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd228141440 con 0x7fd2280a09e0 2024-09-15T21:25:36.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:36.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281412c0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:36.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd2281415c0 con 0x7fd228137bc0 2024-09-15T21:25:36.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:36.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient(hunting): authenticate will time out at 1454.909067s 2024-09-15T21:25:36.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd22811caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:36.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22d56e640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2280a09e0 msgr2=0x7fd22811eb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd22811caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45676/0 (socket says 172.21.15.119:45676) 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22d56e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2280a09e0 0x7fd22811eb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 -- 172.21.15.119:0/1674406896 learned_addr learned my addr 172.21.15.119:0/1674406896 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): get_auth_request con 0x7fd2280a0ea0 auth_method 0 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:36.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:36.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:36.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:36.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:36.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22dd6f640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd228137bc0 0x7fd228134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:36.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22dd6f640 10 monclient(hunting): get_auth_request con 0x7fd228137bc0 auth_method 0 2024-09-15T21:25:36.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22dd6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:36.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22dd6f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:36.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22dd6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:36.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient(hunting): handle_auth_done global_id 84595 payload 824 2024-09-15T21:25:36.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 -- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd228137bc0 msgr2=0x7fd228134970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd228137bc0 0x7fd228134970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 -- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2280a09e0 msgr2=0x7fd22811eb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2280a09e0 0x7fd22811eb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 monclient: found mon.noname-b 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:36.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 -- 172.21.15.119:0/1674406896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22811dfb0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22cd6d640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd22811caf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd21800e810 tx=0x7fd21800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218017070 con 0x7fd2280a0ea0 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: dump: 2024-09-15T21:25:36.135 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:36.136 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:36.136 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.136 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.136 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:36.136 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:36.136 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21800c8d0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 5 monclient: authenticate success, global_id 84595 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _renew_subs 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22809d1d0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21800f040 con 0x7fd2280a0ea0 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:36.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:36.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 10 monclient: dump: 2024-09-15T21:25:36.139 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:36.139 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:36.139 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.139 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _renew_subs 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21f7fe640 4 set_mon_vals no callback set 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd228120310 con 0x7fd2280a0ea0 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:36.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd228137fa0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd21801a070 con 0x7fd2280a0ea0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd20c03b970 0x7fd20c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(239..239 src has 1..239) ==== 10751+0+0 (secure 0 0 0) 0x7fd218051000 con 0x7fd2280a0ea0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22d56e640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd20c03b970 0x7fd20c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22d56e640 10 monclient: get_auth_request con 0x7fd20c03b970 auth_method 0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.109+0000 7fd22d56e640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd20c03b970 0x7fd20c03de30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd224003940 tx=0x7fd224006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:36.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.113+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd218024350 con 0x7fd2280a0ea0 2024-09-15T21:25:36.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.113+0000 7fd21dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:36.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.113+0000 7fd21dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:36.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.281+0000 7fd22f7d2640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T21:25:36.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.281+0000 7fd22f7d2640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T21:25:36.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.281+0000 7fd22f7d2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.281+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) -- 0x7fd228134eb0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.869+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 removed item id -79 name 'h21' from crush map v240) ==== 120+0+0 (secure 0 0 0) 0x7fd21801be20 con 0x7fd2280a0ea0 2024-09-15T21:25:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.869+0000 7fd21dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T21:25:36.873 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.869+0000 7fd21dffb640 10 monclient: _finish_command 2 = system:0 removed item id -79 name 'h21' from crush map 2024-09-15T21:25:36.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.973+0000 7fd22f7d2640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T21:25:36.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.973+0000 7fd22f7d2640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T21:25:36.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.973+0000 7fd22f7d2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:36.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.973+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) -- 0x7fd2281351a0 con 0x7fd2280a0ea0 2024-09-15T21:25:36.979 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd21dffb640 1 -- 172.21.15.119:0/1674406896 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 device 'h21' does not appear in the crush map v240) ==== 120+0+0 (secure 0 0 0) 0x7fd218043020 con 0x7fd2280a0ea0 2024-09-15T21:25:36.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd21dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T21:25:36.980 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd21dffb640 10 monclient: _finish_command 3 = system:0 device 'h21' does not appear in the crush map 2024-09-15T21:25:36.980 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h21' does not appear in the crush map 2024-09-15T21:25:36.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd20c03b970 msgr2=0x7fd20c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:36.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd20c03b970 0x7fd20c03de30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd224003940 tx=0x7fd224006d90 comp rx=0 tx=0).stop 2024-09-15T21:25:36.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 10 monclient: shutdown 2024-09-15T21:25:36.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:36.982 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 msgr2=0x7fd22811caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd22811caf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd21800e810 tx=0x7fd21800ecd0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 shutdown_connections 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd20c03b970 0x7fd20c03de30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd228137bc0 0x7fd228134970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2280a0ea0 0x7fd22811caf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 --2- 172.21.15.119:0/1674406896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2280a09e0 0x7fd22811eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 >> 172.21.15.119:0/1674406896 conn(0x7fd2280a2ae0 msgr2=0x7fd22809d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 shutdown_connections 2024-09-15T21:25:36.983 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:36.977+0000 7fd22f7d2640 1 -- 172.21.15.119:0/1674406896 wait complete. 2024-09-15T21:25:36.998 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:36.998 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h22 2024-09-15T21:25:37.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient: get_monmap_and_config 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient: build_initial_monmap 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient: monmap: 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:37.118220+0000 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:37.118220+0000 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:37.117 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:37.118 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:37.118 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:37.118 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 1 Processor -- start 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 1 -- start start 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient: init 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98080a4e60 0x7f98080a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient(hunting): picked mon.noname-c con 0x7f98080a4e60 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f98080a5780 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f98081383a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:37.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98080a1150 con 0x7f98081383a0 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98080a0ef0 con 0x7f98080a5780 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f98080a3c80 con 0x7f98080a4e60 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980f651640 10 monclient(hunting): authenticate will time out at 1455.915582s 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98080a4e60 0x7f98080a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980cbec640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f98080a5780 0x7f9808137c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40782/0 (socket says 172.21.15.119:40782) 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98080a4e60 0x7f98080a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45686/0 (socket says 172.21.15.119:45686) 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980cbec640 1 -- 172.21.15.119:0/141391203 learned_addr learned my addr 172.21.15.119:0/141391203 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): get_auth_request con 0x7f98080a4e60 auth_method 0 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:37.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980dbee640 1 -- 172.21.15.119:0/141391203 >> v1:172.21.15.17:6789/0 conn(0x7f98081383a0 legacy=0x7f980813ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.113+0000 7f980d3ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 10 monclient(hunting): handle_auth_done global_id 84598 payload 274 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 1 -- 172.21.15.119:0/141391203 >> v1:172.21.15.119:6789/0 conn(0x7f98080a5780 legacy=0x7f9808137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 1 -- 172.21.15.119:0/141391203 >> v1:172.21.15.17:6789/0 conn(0x7f98081383a0 legacy=0x7f980813ff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 10 monclient: _finish_hunting 0 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 1 monclient: found mon.noname-c 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 1 -- 172.21.15.119:0/141391203 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092558199 0 0) 0x7f98080a0ef0 con 0x7f98080a5780 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 1 -- 172.21.15.119:0/141391203 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98080a6cf0 con 0x7f98080a4e60 2024-09-15T21:25:37.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:37.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980d3ed640 1 --2- 172.21.15.119:0/141391203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f98080a5240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9804009d90 tx=0x7f980401c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace6a1457f33ed22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:37.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 1 -- 172.21.15.119:0/141391203 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040b2070 con 0x7f98080a4e60 2024-09-15T21:25:37.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:37.127 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:37.128 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: dump: 2024-09-15T21:25:37.128 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:37.128 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:37.128 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: _finish_auth 0 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:37.129 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 1 -- 172.21.15.119:0/141391203 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98040a5800 con 0x7f98080a4e60 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 5 monclient: authenticate success, global_id 84598 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 4 set_mon_vals no callback set 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 1 -- 172.21.15.119:0/141391203 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040a5bd0 con 0x7f98080a4e60 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f97fffff640 10 monclient: dump: 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.130 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 10 monclient: shutdown 2024-09-15T21:25:37.131 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 -- 172.21.15.119:0/141391203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 msgr2=0x7f98080a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 --2- 172.21.15.119:0/141391203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f98080a5240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9804009d90 tx=0x7f980401c830 comp rx=0 tx=0).stop 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 -- 172.21.15.119:0/141391203 shutdown_connections 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 --2- 172.21.15.119:0/141391203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f98080a5240 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 -- 172.21.15.119:0/141391203 >> 172.21.15.119:0/141391203 conn(0x7f98080a6f60 msgr2=0x7f98080a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 -- 172.21.15.119:0/141391203 shutdown_connections 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 -- 172.21.15.119:0/141391203 wait complete. 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 10 monclient: build_initial_monmap 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 10 monclient: monmap: 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:37.132 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 Processor -- start 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 1 -- start start 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.117+0000 7f980f651640 10 monclient: init 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:37.133 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f980812ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): picked mon.noname-b con 0x7f98080a4e60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98080a5780 0x7f9808130440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): picked mon.noname-a con 0x7f98080a5780 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98081383a0 0x7f9808134f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): picked mon.noname-c con 0x7f98081383a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f98080a7e30 con 0x7f98080a5780 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98080a7cb0 con 0x7f98080a4e60 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f98080a2190 con 0x7f98081383a0 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:37.134 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient(hunting): authenticate will time out at 1455.920441s 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f980812ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f980812ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45702/0 (socket says 172.21.15.119:45702) 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 -- 172.21.15.119:0/1450601927 learned_addr learned my addr 172.21.15.119:0/1450601927 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): get_auth_request con 0x7f98080a4e60 auth_method 0 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980cbec640 1 -- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98080a5780 msgr2=0x7f9808130440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980cbec640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98080a5780 0x7f9808130440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:37.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980dbee640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98081383a0 0x7f9808134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980dbee640 10 monclient(hunting): get_auth_request con 0x7f98081383a0 auth_method 0 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980dbee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980dbee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980dbee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient(hunting): handle_auth_done global_id 84604 payload 824 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 -- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98081383a0 msgr2=0x7f9808134f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98081383a0 0x7f9808134f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 -- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98080a5780 msgr2=0x7f9808130440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98080a5780 0x7f9808130440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient: _finish_hunting 0 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 monclient: found mon.noname-b 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:37.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 -- 172.21.15.119:0/1450601927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98080a1650 con 0x7f98080a4e60 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980d3ed640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f980812ff00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9804009d90 tx=0x7f98040195d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040bd030 con 0x7f98080a4e60 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: dump: 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:37.137 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98040b8470 con 0x7f98080a4e60 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040b2040 con 0x7f98080a4e60 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 5 monclient: authenticate success, global_id 84604 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _renew_subs 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9808139330 con 0x7f98080a4e60 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97ff7fe640 4 set_mon_vals no callback set 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 10 monclient: dump: 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:37.139 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _renew_subs 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98080a99c0 con 0x7f98080a4e60 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9808077180 con 0x7f98080a4e60 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f98040c0070 con 0x7f98080a4e60 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f97ec03b970 0x7f97ec03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 10538+0+0 (secure 0 0 0) 0x7f98040ebf10 con 0x7f98080a4e60 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980cbec640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f97ec03b970 0x7f97ec03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:37.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980cbec640 10 monclient: get_auth_request con 0x7f97ec03b970 auth_method 0 2024-09-15T21:25:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.121+0000 7f980cbec640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f97ec03b970 0x7f97ec03de30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9808138370 tx=0x7f97f8007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.125+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f98040c7070 con 0x7f98080a4e60 2024-09-15T21:25:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.125+0000 7f97fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:37.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.125+0000 7f97fdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:37.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.301+0000 7f980f651640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T21:25:37.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.301+0000 7f980f651640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T21:25:37.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.301+0000 7f980f651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.305 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.301+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) -- 0x7f98080a5240 con 0x7f98080a4e60 2024-09-15T21:25:37.800 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.797+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 removed item id -82 name 'h22' from crush map v241) ==== 120+0+0 (secure 0 0 0) 0x7f98080a1650 con 0x7f98080a4e60 2024-09-15T21:25:37.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.797+0000 7f97fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T21:25:37.801 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.797+0000 7f97fdffb640 10 monclient: _finish_command 2 = system:0 removed item id -82 name 'h22' from crush map 2024-09-15T21:25:37.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.905+0000 7f980f651640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T21:25:37.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.905+0000 7f980f651640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T21:25:37.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.905+0000 7f980f651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:37.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.905+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) -- 0x7f98080008e0 con 0x7f98080a4e60 2024-09-15T21:25:37.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.905+0000 7f97fdffb640 1 -- 172.21.15.119:0/1450601927 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 device 'h22' does not appear in the crush map v241) ==== 120+0+0 (secure 0 0 0) 0x7f9804016540 con 0x7f98080a4e60 2024-09-15T21:25:37.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f97fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T21:25:37.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f97fdffb640 10 monclient: _finish_command 3 = system:0 device 'h22' does not appear in the crush map 2024-09-15T21:25:37.912 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h22' does not appear in the crush map 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f97ec03b970 msgr2=0x7f97ec03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f97ec03b970 0x7f97ec03de30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9808138370 tx=0x7f97f8007a20 comp rx=0 tx=0).stop 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 10 monclient: shutdown 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 msgr2=0x7f980812ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f980812ff00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9804009d90 tx=0x7f98040195d0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 shutdown_connections 2024-09-15T21:25:37.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f97ec03b970 0x7f97ec03de30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98081383a0 0x7f9808134f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98080a5780 0x7f9808130440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 --2- 172.21.15.119:0/1450601927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98080a4e60 0x7f980812ff00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:37.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 >> 172.21.15.119:0/1450601927 conn(0x7f98080a6f60 msgr2=0x7f98080a8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:37.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 shutdown_connections 2024-09-15T21:25:37.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:37.909+0000 7f980f651640 1 -- 172.21.15.119:0/1450601927 wait complete. 2024-09-15T21:25:37.928 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:37.929 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h23 2024-09-15T21:25:38.032 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient: get_monmap_and_config 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient: build_initial_monmap 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient: monmap: 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:38.034081+0000 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:38.034081+0000 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:38.033 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:38.034 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:38.034 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:38.034 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:38.034 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:38.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 Processor -- start 2024-09-15T21:25:38.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 -- start start 2024-09-15T21:25:38.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient: init 2024-09-15T21:25:38.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient: _add_conns ranks=[2,3,0,4,1] 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 0x7f57c0136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): picked mon.noname-b con 0x7f57c0135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57c00a0160 0x7f57c00a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): picked mon.noname-a con 0x7f57c00a0160 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57c006ec30 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c01365c0 con 0x7f57c006ec30 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f57c009cf30 con 0x7f57c00a0160 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57c00a1ca0 con 0x7f57c0135ca0 2024-09-15T21:25:38.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c7ac3640 10 monclient(hunting): authenticate will time out at 1456.830646s 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c505e640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f57c00a0160 msgr2=0x7f57c00a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c6060640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f57c006ec30 legacy=0x7f57c00a0f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c505e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57c00a0160 0x7f57c00a09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 0x7f57c0136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 0x7f57c0136080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:50506/0 (socket says 172.21.15.119:50506) 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 -- 172.21.15.119:0/2171440821 learned_addr learned my addr 172.21.15.119:0/2171440821 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): get_auth_request con 0x7f57c0135ca0 auth_method 0 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:38.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient(hunting): handle_auth_done global_id 45215 payload 274 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 -- 172.21.15.119:0/2171440821 >> v2:172.21.15.17:3300/0 conn(0x7f57c00a0160 msgr2=0x7f57c00a09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 --2- 172.21.15.119:0/2171440821 >> v2:172.21.15.17:3300/0 conn(0x7f57c00a0160 0x7f57c00a09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 -- 172.21.15.119:0/2171440821 >> v1:172.21.15.17:6789/0 conn(0x7f57c006ec30 legacy=0x7f57c00a0f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient: _finish_hunting 0 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 monclient: found mon.noname-b 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:38.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.029+0000 7f57c585f640 1 -- 172.21.15.119:0/2171440821 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c00a1f80 con 0x7f57c0135ca0 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c585f640 1 --2- 172.21.15.119:0/2171440821 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 0x7f57c0136080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009e50 tx=0x7f57bc01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1bce6aaeaa7a678 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 1 -- 172.21.15.119:0/2171440821 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0a5550 con 0x7f57c0135ca0 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: dump: 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:38.045 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: _finish_auth 0 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: _check_auth_tickets 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 5 monclient: authenticate success, global_id 45215 2024-09-15T21:25:38.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 1 -- 172.21.15.119:0/2171440821 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc0a56f0 con 0x7f57c0135ca0 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 1 -- 172.21.15.119:0/2171440821 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0a5ac0 con 0x7f57c0135ca0 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 4 set_mon_vals no callback set 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c485d640 10 monclient: dump: 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.047 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 10 monclient: shutdown 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 -- 172.21.15.119:0/2171440821 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 msgr2=0x7f57c0136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/2171440821 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 0x7f57c0136080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009e50 tx=0x7f57bc01c910 comp rx=0 tx=0).stop 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 -- 172.21.15.119:0/2171440821 shutdown_connections 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/2171440821 >> v2:172.21.15.17:3300/0 conn(0x7f57c00a0160 0x7f57c00a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/2171440821 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c0135ca0 0x7f57c0136080 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:38.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 -- 172.21.15.119:0/2171440821 >> 172.21.15.119:0/2171440821 conn(0x7f57c00a2ae0 msgr2=0x7f57c00a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 -- 172.21.15.119:0/2171440821 shutdown_connections 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.033+0000 7f57c7ac3640 1 -- 172.21.15.119:0/2171440821 wait complete. 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: build_initial_monmap 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: monmap: 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:38.049 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 Processor -- start 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- start start 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: init 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c006ec30 0x7f57c0133480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): picked mon.noname-c con 0x7f57c006ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 0x7f57c01339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): picked mon.noname-b con 0x7f57c00a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c0135ca0 0x7f57c013b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): picked mon.noname-a con 0x7f57c0135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0140240 con 0x7f57c0135ca0 2024-09-15T21:25:38.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57c00a1840 con 0x7f57c00a0160 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57c01403c0 con 0x7f57c006ec30 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient(hunting): authenticate will time out at 1456.836980s 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c6060640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c0135ca0 msgr2=0x7f57c013b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c6060640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c0135ca0 0x7f57c013b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 0x7f57c01339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 0x7f57c01339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45704/0 (socket says 172.21.15.119:45704) 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 -- 172.21.15.119:0/1289946114 learned_addr learned my addr 172.21.15.119:0/1289946114 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): get_auth_request con 0x7f57c00a0160 auth_method 0 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:38.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c006ec30 0x7f57c0133480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 10 monclient(hunting): get_auth_request con 0x7f57c006ec30 auth_method 0 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient(hunting): handle_auth_done global_id 84607 payload 824 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 -- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c006ec30 msgr2=0x7f57c0133480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c006ec30 0x7f57c0133480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:38.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 -- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c0135ca0 msgr2=0x7f57c013b890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c0135ca0 0x7f57c013b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient: _finish_hunting 0 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 monclient: found mon.noname-b 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 -- 172.21.15.119:0/1289946114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c011cf30 con 0x7f57c00a0160 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c006ec30 0x7f57c0133480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c505e640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 0x7f57c01339c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57b000b530 tx=0x7f57b000b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0013020 con 0x7f57c00a0160 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: dump: 2024-09-15T21:25:38.053 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 5 monclient: authenticate success, global_id 84607 2024-09-15T21:25:38.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _renew_subs 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c0134e50 con 0x7f57c00a0160 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b0003df0 con 0x7f57c00a0160 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b00052b0 con 0x7f57c00a0160 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 10 monclient: dump: 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:38.055 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b7fff640 4 set_mon_vals no callback set 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _renew_subs 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c011cb60 con 0x7f57c00a0160 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c0136080 con 0x7f57c00a0160 2024-09-15T21:25:38.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f57b0016020 con 0x7f57c00a0160 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57a403b970 0x7f57a403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 10325+0+0 (secure 0 0 0) 0x7f57b0050dd0 con 0x7f57c00a0160 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57a403b970 0x7f57a403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 10 monclient: get_auth_request con 0x7f57a403b970 auth_method 0 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.037+0000 7f57c585f640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57a403b970 0x7f57a403de30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0ae000 tx=0x7f57bc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.041+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f57b001d020 con 0x7f57c00a0160 2024-09-15T21:25:38.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.041+0000 7f57b67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:38.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.041+0000 7f57b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:38.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.205+0000 7f57c7ac3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T21:25:38.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.205+0000 7f57c7ac3640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T21:25:38.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.205+0000 7f57c7ac3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:38.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.205+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) -- 0x7f57c009f8f0 con 0x7f57c00a0160 2024-09-15T21:25:38.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.897+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 removed item id -85 name 'h23' from crush map v242) ==== 120+0+0 (secure 0 0 0) 0x7f57b000fdc0 con 0x7f57c00a0160 2024-09-15T21:25:38.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.897+0000 7f57b67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T21:25:38.900 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.897+0000 7f57b67fc640 10 monclient: _finish_command 2 = system:0 removed item id -85 name 'h23' from crush map 2024-09-15T21:25:39.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.997+0000 7f57c7ac3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T21:25:39.003 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.997+0000 7f57c7ac3640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T21:25:39.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.997+0000 7f57c7ac3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:38.997+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) -- 0x7f57c0134820 con 0x7f57c00a0160 2024-09-15T21:25:39.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57b67fc640 1 -- 172.21.15.119:0/1289946114 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 device 'h23' does not appear in the crush map v242) ==== 120+0+0 (secure 0 0 0) 0x7f57b000d380 con 0x7f57c00a0160 2024-09-15T21:25:39.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57b67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T21:25:39.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57b67fc640 10 monclient: _finish_command 3 = system:0 device 'h23' does not appear in the crush map 2024-09-15T21:25:39.004 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h23' does not appear in the crush map 2024-09-15T21:25:39.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57a403b970 msgr2=0x7f57a403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:39.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57a403b970 0x7f57a403de30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0ae000 tx=0x7f57bc0a7070 comp rx=0 tx=0).stop 2024-09-15T21:25:39.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 10 monclient: shutdown 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 msgr2=0x7f57c01339c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 0x7f57c01339c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57b000b530 tx=0x7f57b000b9f0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 shutdown_connections 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57a403b970 0x7f57a403de30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c0135ca0 0x7f57c013b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c00a0160 0x7f57c01339c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 --2- 172.21.15.119:0/1289946114 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57c006ec30 0x7f57c0133480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 >> 172.21.15.119:0/1289946114 conn(0x7f57c00a2ae0 msgr2=0x7f57c00721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 shutdown_connections 2024-09-15T21:25:39.010 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.001+0000 7f57c7ac3640 1 -- 172.21.15.119:0/1289946114 wait complete. 2024-09-15T21:25:39.021 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:39.021 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h24 2024-09-15T21:25:39.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient: get_monmap_and_config 2024-09-15T21:25:39.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient: build_initial_monmap 2024-09-15T21:25:39.140 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient: monmap: 2024-09-15T21:25:39.140 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:39.140 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:39.140 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:39.141739+0000 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:39.141739+0000 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:39.141 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 Processor -- start 2024-09-15T21:25:39.147 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 -- start start 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient: init 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbdd409f640 0x7fbdd409d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): picked mon.noname-a con 0x7fbdd409f640 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbdd409fa20 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 0x7fbdd413bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): picked mon.noname-b con 0x7fbdd4139c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd409e880 con 0x7fbdd409fa20 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd409eb60 con 0x7fbdd409f640 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd409ee40 con 0x7fbdd4139c00 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdda4cb640 10 monclient(hunting): authenticate will time out at 1457.939209s 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdcbfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbdd409fa20 0x7fbdd409feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40792/0 (socket says 172.21.15.119:40792) 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd3fff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fbdd409f640 msgr2=0x7fbdd409d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdcbfff640 1 -- 172.21.15.119:0/2714365804 learned_addr learned my addr 172.21.15.119:0/2714365804 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/2714365804 >> v2:172.21.15.17:3300/0 conn(0x7fbdd409f640 0x7fbdd409d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 0x7fbdd413bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): get_auth_request con 0x7fbdd4139c00 auth_method 0 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869038636 0 0) 0x7fbdd409e880 con 0x7fbdd409fa20 2024-09-15T21:25:39.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 10 monclient(hunting): my global_id is 84610 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdc0003530 con 0x7fbdd409fa20 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.137+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3882281395 0 0) 0x7fbdc0003530 con 0x7fbdd409fa20 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 msgr2=0x7fbdd413bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 0x7fbdd413bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 >> v2:172.21.15.17:3300/0 conn(0x7fbdd409f640 msgr2=0x7fbdd409d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 --2- 172.21.15.119:0/2714365804 >> v2:172.21.15.17:3300/0 conn(0x7fbdd409f640 0x7fbdd409d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd40a2840 con 0x7fbdd409fa20 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fbdbc001700 con 0x7fbdd409fa20 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 5 monclient: authenticate success, global_id 84610 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: dump: 2024-09-15T21:25:39.151 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 >> v1:172.21.15.119:6789/0 conn(0x7fbdd409fa20 legacy=0x7fbdd409feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:39.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc00039c0 0x7fbdc0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): picked mon.b con 0x7fbdc00039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 0x7fbdc0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): picked mon.c con 0x7fbdd4139c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdc0004d50 0x7fbdc00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): picked mon.a con 0x7fbdc0004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd409eb60 con 0x7fbdc0004d50 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd409ee40 con 0x7fbdc00039c0 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc000bc30 con 0x7fbdd4139c00 2024-09-15T21:25:39.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc00039c0 0x7fbdc0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd8a68640 1 -- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdc0004d50 msgr2=0x7fbdc00051d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): get_auth_request con 0x7fbdc00039c0 auth_method 0 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd8a68640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdc0004d50 0x7fbdc00051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:39.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdcbfff640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 0x7fbdc0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdcbfff640 10 monclient(hunting): get_auth_request con 0x7fbdd4139c00 auth_method 0 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdcbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient(hunting): handle_auth_done global_id 84610 payload 293 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 -- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 msgr2=0x7fbdc0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdd4139c00 0x7fbdc0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 -- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdc0004d50 msgr2=0x7fbdc00051d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdc0004d50 0x7fbdc00051d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 monclient: found mon.b 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 -- 172.21.15.119:0/2714365804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc0003530 con 0x7fbdc00039c0 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc00039c0 0x7fbdc0003de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4003940 tx=0x7fbdc401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89869c0ba2ecd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4009520 con 0x7fbdc00039c0 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: dump: 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.156 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc4003bc0 con 0x7fbdc00039c0 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 4 set_mon_vals no callback set 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 1 -- 172.21.15.119:0/2714365804 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4005070 con 0x7fbdc00039c0 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:39.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd37fe640 10 monclient: dump: 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 10 monclient: shutdown 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:39.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 1 -- 172.21.15.119:0/2714365804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc00039c0 msgr2=0x7fbdc0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/2714365804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc00039c0 0x7fbdc0003de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4003940 tx=0x7fbdc401c420 comp rx=0 tx=0).stop 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdd8a68640 1 -- 172.21.15.119:0/2714365804 reap_dead start 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 1 -- 172.21.15.119:0/2714365804 shutdown_connections 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 1 -- 172.21.15.119:0/2714365804 >> 172.21.15.119:0/2714365804 conn(0x7fbdd40a2ae0 msgr2=0x7fbdd40a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 1 -- 172.21.15.119:0/2714365804 shutdown_connections 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.141+0000 7fbdda4cb640 1 -- 172.21.15.119:0/2714365804 wait complete. 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: build_initial_monmap 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: monmap: 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:39.159 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 Processor -- start 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- start start 2024-09-15T21:25:39.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: init 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdc00039c0 0x7fbdd4135be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): picked mon.noname-c con 0x7fbdc00039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd4136120 0x7fbdd41365a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): picked mon.noname-a con 0x7fbdd4136120 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 0x7fbdd41d5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): picked mon.noname-b con 0x7fbdd41d3a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd413c980 con 0x7fbdd4136120 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd413c800 con 0x7fbdd41d3a90 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd413cb00 con 0x7fbdc00039c0 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient(hunting): authenticate will time out at 1457.945146s 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdcbfff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd4136120 msgr2=0x7fbdd41365a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdcbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd4136120 0x7fbdd41365a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 0x7fbdd41d5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 0x7fbdd41d5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45734/0 (socket says 172.21.15.119:45734) 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 -- 172.21.15.119:0/708529454 learned_addr learned my addr 172.21.15.119:0/708529454 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): get_auth_request con 0x7fbdd41d3a90 auth_method 0 2024-09-15T21:25:39.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdc00039c0 0x7fbdd4135be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd3fff640 10 monclient(hunting): get_auth_request con 0x7fbdc00039c0 auth_method 0 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient(hunting): handle_auth_done global_id 84613 payload 824 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 -- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdc00039c0 msgr2=0x7fbdd4135be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:39.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdc00039c0 0x7fbdd4135be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 -- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd4136120 msgr2=0x7fbdd41365a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd4136120 0x7fbdd41365a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient: _finish_hunting 0 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 monclient: found mon.noname-b 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 -- 172.21.15.119:0/708529454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd411d7c0 con 0x7fbdd41d3a90 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdc00039c0 0x7fbdd4135be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd8a68640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 0x7fbdd41d5e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40a10c0 tx=0x7fbdcc00d560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc01a070 con 0x7fbdd41d3a90 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: dump: 2024-09-15T21:25:39.167 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:39.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc0039a0 con 0x7fbdd41d3a90 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc003d90 con 0x7fbdd41d3a90 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 10 monclient: dump: 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:39.169 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd2ffd640 4 set_mon_vals no callback set 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 5 monclient: authenticate success, global_id 84613 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _renew_subs 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd411daa0 con 0x7fbdd41d3a90 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _renew_subs 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd409d1d0 con 0x7fbdd41d3a90 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd409d9a0 con 0x7fbdd41d3a90 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.145+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbdcc004030 con 0x7fbdd41d3a90 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.149+0000 7fbdd17fa640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdb403b970 0x7fbdb403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.149+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(242..242 src has 1..242) ==== 10112+0+0 (secure 0 0 0) 0x7fbdcc051610 con 0x7fbdd41d3a90 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.149+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdb403b970 0x7fbdb403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.149+0000 7fbdd3fff640 10 monclient: get_auth_request con 0x7fbdb403b970 auth_method 0 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.149+0000 7fbdd3fff640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdb403b970 0x7fbdb403de30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40aa000 tx=0x7fbdc4005a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.153+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbdcc010070 con 0x7fbdd41d3a90 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.153+0000 7fbdd17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:39.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.153+0000 7fbdd17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:39.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.321+0000 7fbdda4cb640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T21:25:39.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.321+0000 7fbdda4cb640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T21:25:39.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.321+0000 7fbdda4cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:39.326 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.321+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) -- 0x7fbdd4137a10 con 0x7fbdd41d3a90 2024-09-15T21:25:39.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.905+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 removed item id -88 name 'h24' from crush map v243) ==== 120+0+0 (secure 0 0 0) 0x7fbdcc04eb50 con 0x7fbdd41d3a90 2024-09-15T21:25:39.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.905+0000 7fbdd17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T21:25:39.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:39.905+0000 7fbdd17fa640 10 monclient: _finish_command 2 = system:0 removed item id -88 name 'h24' from crush map 2024-09-15T21:25:40.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T21:25:40.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T21:25:40.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) -- 0x7fbdd40008e0 con 0x7fbdd41d3a90 2024-09-15T21:25:40.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdd17fa640 1 -- 172.21.15.119:0/708529454 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 device 'h24' does not appear in the crush map v243) ==== 120+0+0 (secure 0 0 0) 0x7fbdcc015780 con 0x7fbdd41d3a90 2024-09-15T21:25:40.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdd17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T21:25:40.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdd17fa640 10 monclient: _finish_command 3 = system:0 device 'h24' does not appear in the crush map 2024-09-15T21:25:40.017 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h24' does not appear in the crush map 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdb403b970 msgr2=0x7fbdb403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdb403b970 0x7fbdb403de30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40aa000 tx=0x7fbdc4005a90 comp rx=0 tx=0).stop 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 10 monclient: shutdown 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 msgr2=0x7fbdd41d5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 0x7fbdd41d5e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40a10c0 tx=0x7fbdcc00d560 comp rx=0 tx=0).stop 2024-09-15T21:25:40.019 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 shutdown_connections 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbdb403b970 0x7fbdb403de30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd41d3a90 0x7fbdd41d5e80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd4136120 0x7fbdd41365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 --2- 172.21.15.119:0/708529454 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdc00039c0 0x7fbdd4135be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 >> 172.21.15.119:0/708529454 conn(0x7fbdd40a2ae0 msgr2=0x7fbdd40a52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 shutdown_connections 2024-09-15T21:25:40.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.013+0000 7fbdda4cb640 1 -- 172.21.15.119:0/708529454 wait complete. 2024-09-15T21:25:40.033 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:40.033 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h25 2024-09-15T21:25:40.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.141+0000 7f8da19fc640 10 monclient: get_monmap_and_config 2024-09-15T21:25:40.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.141+0000 7f8da19fc640 10 monclient: build_initial_monmap 2024-09-15T21:25:40.146 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.141+0000 7f8da19fc640 10 monclient: monmap: 2024-09-15T21:25:40.146 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:40.147818+0000 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:40.147818+0000 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:40.147 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:40.148 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:40.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.141+0000 7f8da19fc640 1 Processor -- start 2024-09-15T21:25:40.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.141+0000 7f8da19fc640 1 -- start start 2024-09-15T21:25:40.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.141+0000 7f8da19fc640 10 monclient: init 2024-09-15T21:25:40.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8d9c135ca0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d9c0a0160 0x7f8d9c13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient(hunting): picked mon.noname-c con 0x7f8d9c0a0160 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8d9c06ec30 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d9c09cf30 con 0x7f8d9c06ec30 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d9c0a1ca0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8d9c0a1f80 con 0x7f8d9c0a0160 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient(hunting): authenticate will time out at 1458.944407s 2024-09-15T21:25:40.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d9c0a0160 0x7f8d9c13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9affd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8d9c135ca0 0x7f8d9c136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40794/0 (socket says 172.21.15.119:40794) 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9affd640 1 -- 172.21.15.119:0/2050071218 learned_addr learned my addr 172.21.15.119:0/2050071218 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): get_auth_request con 0x7f8d9c0a0160 auth_method 0 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9b7fe640 1 -- 172.21.15.119:0/2050071218 >> v1:172.21.15.17:6789/0 conn(0x7f8d9c06ec30 legacy=0x7f8d9c13cf50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:40.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient(hunting): handle_auth_done global_id 84616 payload 274 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 1 -- 172.21.15.119:0/2050071218 >> v1:172.21.15.119:6789/0 conn(0x7f8d9c135ca0 legacy=0x7f8d9c136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 1 -- 172.21.15.119:0/2050071218 >> v1:172.21.15.17:6789/0 conn(0x7f8d9c06ec30 legacy=0x7f8d9c13cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 1 monclient: found mon.noname-c 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 1 -- 172.21.15.119:0/2050071218 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969513344 0 0) 0x7f8d9c0a1ca0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 1 -- 172.21.15.119:0/2050071218 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c0a3860 con 0x7f8d9c0a0160 2024-09-15T21:25:40.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d93fff640 1 --2- 172.21.15.119:0/2050071218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c13aa00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d840097f0 tx=0x7f8d8401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483c1d07a43c8b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 1 -- 172.21.15.119:0/2050071218 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840b0070 con 0x7f8d9c0a0160 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: dump: 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.162 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 5 monclient: authenticate success, global_id 84616 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:40.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 1 -- 172.21.15.119:0/2050071218 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d840ab600 con 0x7f8d9c0a0160 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 4 set_mon_vals no callback set 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 1 -- 172.21.15.119:0/2050071218 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840ab9d0 con 0x7f8d9c0a0160 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8d9a7fc640 10 monclient: dump: 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.164 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 10 monclient: shutdown 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- 172.21.15.119:0/2050071218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c0a0160 msgr2=0x7f8d9c13aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 --2- 172.21.15.119:0/2050071218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c13aa00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d840097f0 tx=0x7f8d8401c360 comp rx=0 tx=0).stop 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- 172.21.15.119:0/2050071218 shutdown_connections 2024-09-15T21:25:40.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 --2- 172.21.15.119:0/2050071218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c13aa00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- 172.21.15.119:0/2050071218 >> 172.21.15.119:0/2050071218 conn(0x7f8d9c0a2ae0 msgr2=0x7f8d9c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- 172.21.15.119:0/2050071218 shutdown_connections 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.145+0000 7f8da19fc640 1 -- 172.21.15.119:0/2050071218 wait complete. 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: build_initial_monmap 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: monmap: 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:40.166 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 Processor -- start 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- start start 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: init 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d9c06ec30 0x7f8d9c1332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): picked mon.noname-c con 0x7f8d9c06ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:40.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c133800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): picked mon.noname-a con 0x7f8d9c0a0160 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 0x7f8d9c13b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): picked mon.noname-b con 0x7f8d9c135ca0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c141010 con 0x7f8d9c0a0160 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c140e90 con 0x7f8d9c135ca0 2024-09-15T21:25:40.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9c141190 con 0x7f8d9c06ec30 2024-09-15T21:25:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient(hunting): authenticate will time out at 1458.949238s 2024-09-15T21:25:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 0x7f8d9c13b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d93fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d9c0a0160 msgr2=0x7f8d9c133800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c133800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 0x7f8d9c13b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45754/0 (socket says 172.21.15.119:45754) 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 -- 172.21.15.119:0/1333973234 learned_addr learned my addr 172.21.15.119:0/1333973234 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): get_auth_request con 0x7f8d9c135ca0 auth_method 0 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9affd640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d9c06ec30 0x7f8d9c1332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_done global_id 84622 payload 824 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 -- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d9c06ec30 msgr2=0x7f8d9c1332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:40.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d9c06ec30 0x7f8d9c1332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 -- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d9c0a0160 msgr2=0x7f8d9c133800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c133800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 -- 172.21.15.119:0/1333973234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c11c8f0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9b7fe640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 0x7f8d9c13b300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400be50 tx=0x7f8d9400be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94016430 con 0x7f8d9c135ca0 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:40.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: dump: 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:40.173 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: _finish_auth 0 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d94010030 con 0x7f8d9c135ca0 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 5 monclient: authenticate success, global_id 84622 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _renew_subs 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c11cbd0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94016850 con 0x7f8d9c135ca0 2024-09-15T21:25:40.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 10 monclient: dump: 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:40.175 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d99ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _renew_subs 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c0a5740 con 0x7f8d9c135ca0 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c09f8f0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8d940060d0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d7c03b970 0x7f8d7c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:40.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 9899+0+0 (secure 0 0 0) 0x7f8d94051520 con 0x7f8d9c135ca0 2024-09-15T21:25:40.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9affd640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d7c03b970 0x7f8d7c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:40.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9affd640 10 monclient: get_auth_request con 0x7f8d7c03b970 auth_method 0 2024-09-15T21:25:40.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.149+0000 7f8d9affd640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d7c03b970 0x7f8d7c03de30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c007fd0 tx=0x7f8d8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:40.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.153+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8d9404eb30 con 0x7f8d9c135ca0 2024-09-15T21:25:40.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.153+0000 7f8d98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:40.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.153+0000 7f8d98ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.317+0000 7f8da19fc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T21:25:40.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.317+0000 7f8da19fc640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T21:25:40.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.317+0000 7f8da19fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:40.321 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.317+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) -- 0x7f8d9c06c460 con 0x7f8d9c135ca0 2024-09-15T21:25:40.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.909+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 removed item id -91 name 'h25' from crush map v244) ==== 120+0+0 (secure 0 0 0) 0x7f8d940153e0 con 0x7f8d9c135ca0 2024-09-15T21:25:40.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.909+0000 7f8d98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T21:25:40.913 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:40.909+0000 7f8d98ff9640 10 monclient: _finish_command 2 = system:0 removed item id -91 name 'h25' from crush map 2024-09-15T21:25:41.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8da19fc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T21:25:41.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8da19fc640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T21:25:41.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8da19fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) -- 0x7f8d9c133e60 con 0x7f8d9c135ca0 2024-09-15T21:25:41.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8d98ff9640 1 -- 172.21.15.119:0/1333973234 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 device 'h25' does not appear in the crush map v244) ==== 120+0+0 (secure 0 0 0) 0x7f8d94007a10 con 0x7f8d9c135ca0 2024-09-15T21:25:41.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8d98ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T21:25:41.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.013+0000 7f8d98ff9640 10 monclient: _finish_command 3 = system:0 device 'h25' does not appear in the crush map 2024-09-15T21:25:41.017 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h25' does not appear in the crush map 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d7c03b970 msgr2=0x7f8d7c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d7c03b970 0x7f8d7c03de30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c007fd0 tx=0x7f8d8c009040 comp rx=0 tx=0).stop 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 10 monclient: shutdown 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 msgr2=0x7f8d9c13b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 0x7f8d9c13b300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400be50 tx=0x7f8d9400be80 comp rx=0 tx=0).stop 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 shutdown_connections 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8d7c03b970 0x7f8d7c03de30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.020 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d9c135ca0 0x7f8d9c13b300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d9c0a0160 0x7f8d9c133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 --2- 172.21.15.119:0/1333973234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8d9c06ec30 0x7f8d9c1332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 >> 172.21.15.119:0/1333973234 conn(0x7f8d9c0a2ae0 msgr2=0x7f8d9c071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:41.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 shutdown_connections 2024-09-15T21:25:41.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.017+0000 7f8da19fc640 1 -- 172.21.15.119:0/1333973234 wait complete. 2024-09-15T21:25:41.034 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:41.034 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h26 2024-09-15T21:25:41.141 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient: get_monmap_and_config 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient: build_initial_monmap 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient: monmap: 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:41.143302+0000 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:41.143302+0000 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:41.142 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:41.143 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 1 Processor -- start 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 1 -- start start 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient: init 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 0x7f867c09eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient(hunting): picked mon.noname-b con 0x7f867c09f500 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f867c09f8e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f867c0a10f0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:41.148 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c0a02f0 con 0x7f867c0a10f0 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c074260 con 0x7f867c09f8e0 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f867c09cf30 con 0x7f867c09f500 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.137+0000 7f8683006640 10 monclient(hunting): authenticate will time out at 1459.939962s 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f867c0a10f0 legacy=0x7f867c144000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f867c09f8e0 0x7f867c0a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40796/0 (socket says 172.21.15.119:40796) 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867bfff640 1 -- 172.21.15.119:0/2032141100 learned_addr learned my addr 172.21.15.119:0/2032141100 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 0x7f867c09eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): get_auth_request con 0x7f867c09f500 auth_method 0 2024-09-15T21:25:41.149 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 238326759 0 0) 0x7f867c074260 con 0x7f867c09f8e0 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): my global_id is 84625 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8668003530 con 0x7f867c09f8e0 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8680da2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 771656769 0 0) 0x7f8668003530 con 0x7f867c09f8e0 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 msgr2=0x7f867c09eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:41.150 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 0x7f867c09eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 >> v1:172.21.15.17:6789/0 conn(0x7f867c0a10f0 legacy=0x7f867c144000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c09d6f0 con 0x7f867c09f8e0 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f8664001a80 con 0x7f867c09f8e0 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:41.151 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: dump: 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:41.152 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:41.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 >> v1:172.21.15.119:6789/0 conn(0x7f867c09f8e0 legacy=0x7f867c0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:41.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:41.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 0x7f86680039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): picked mon.c con 0x7f867c09f500 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:41.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668003f20 0x7f8668004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.153 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): picked mon.a con 0x7f8668003f20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8668004d50 0x7f86680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): picked mon.b con 0x7f8668004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f867c09cf30 con 0x7f8668003f20 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f866800bbc0 con 0x7f8668004d50 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.154 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f866800bdc0 con 0x7f867c09f500 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867bfff640 1 -- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668003f20 msgr2=0x7f8668004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8668004d50 0x7f86680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867bfff640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668003f20 0x7f8668004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): get_auth_request con 0x7f8668004d50 auth_method 0 2024-09-15T21:25:41.155 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient(hunting): handle_auth_done global_id 84625 payload 293 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 -- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 msgr2=0x7f86680039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:41.156 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c09f500 0x7f86680039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 -- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668003f20 msgr2=0x7f8668004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668003f20 0x7f8668004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient: _finish_hunting 0 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 monclient: found mon.b 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 -- 172.21.15.119:0/2032141100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8668003530 con 0x7f8668004d50 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient: _finish_auth 0 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 10 monclient: _check_auth_tickets 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 5 monclient: authenticate success, global_id 84625 2024-09-15T21:25:41.157 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8668004d50 0x7f86680051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f866c003940 tx=0x7f866c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbaa9e668e76468 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c009520 con 0x7f8668004d50 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: dump: 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.158 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:41.159 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c003bc0 con 0x7f8668004d50 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 4 set_mon_vals no callback set 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 1 -- 172.21.15.119:0/2032141100 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c005070 con 0x7f8668004d50 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f867b7fe640 10 monclient: dump: 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:41.160 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 10 monclient: shutdown 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 1 -- 172.21.15.119:0/2032141100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8668004d50 msgr2=0x7f86680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 1 --2- 172.21.15.119:0/2032141100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8668004d50 0x7f86680051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f866c003940 tx=0x7f866c01c420 comp rx=0 tx=0).stop 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f86815a3640 1 -- 172.21.15.119:0/2032141100 reap_dead start 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 1 -- 172.21.15.119:0/2032141100 shutdown_connections 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 1 -- 172.21.15.119:0/2032141100 >> 172.21.15.119:0/2032141100 conn(0x7f867c0a2ae0 msgr2=0x7f867c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:41.161 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 1 -- 172.21.15.119:0/2032141100 shutdown_connections 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.141+0000 7f8683006640 1 -- 172.21.15.119:0/2032141100 wait complete. 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: build_initial_monmap 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: monmap: 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.162 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 Processor -- start 2024-09-15T21:25:41.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- start start 2024-09-15T21:25:41.163 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: init 2024-09-15T21:25:41.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:41.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:41.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668004d50 0x7f867c124e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): picked mon.noname-a con 0x7f8668004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:41.167 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 0x7f867c125800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): picked mon.noname-b con 0x7f867c125380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c13b740 0x7f867c13bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): picked mon.noname-c con 0x7f867c13b740 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1460b0 con 0x7f8668004d50 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f867c145f30 con 0x7f867c125380 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f867c146230 con 0x7f867c13b740 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient(hunting): authenticate will time out at 1459.945368s 2024-09-15T21:25:41.168 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 0x7f867c125800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 0x7f867c125800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45768/0 (socket says 172.21.15.119:45768) 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 -- 172.21.15.119:0/2151885619 learned_addr learned my addr 172.21.15.119:0/2151885619 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): get_auth_request con 0x7f867c125380 auth_method 0 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8680da2640 1 -- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668004d50 msgr2=0x7f867c124e40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:41.169 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8680da2640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668004d50 0x7f867c124e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86815a3640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c13b740 0x7f867c13bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient(hunting): handle_auth_done global_id 84628 payload 824 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 -- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c13b740 msgr2=0x7f867c13bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c13b740 0x7f867c13bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 -- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668004d50 msgr2=0x7f867c124e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668004d50 0x7f867c124e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 monclient: found mon.noname-b 2024-09-15T21:25:41.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 -- 172.21.15.119:0/2151885619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c09c850 con 0x7f867c125380 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867bfff640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 0x7f867c125800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f866400c9f0 tx=0x7f866400ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664018070 con 0x7f867c125380 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: dump: 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.171 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866400a700 con 0x7f867c125380 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866400aaf0 con 0x7f867c125380 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 10 monclient: dump: 2024-09-15T21:25:41.172 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f867affd640 4 set_mon_vals no callback set 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 5 monclient: authenticate success, global_id 84628 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _renew_subs 2024-09-15T21:25:41.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c127500 con 0x7f867c125380 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _renew_subs 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c09c2d0 con 0x7f867c125380 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c0a09e0 con 0x7f867c125380 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f866400ac90 con 0x7f867c125380 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f865c03b970 0x7f865c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(244..244 src has 1..244) ==== 9686+0+0 (secure 0 0 0) 0x7f86640506a0 con 0x7f867c125380 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8680da2640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f865c03b970 0x7f865c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:41.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.145+0000 7f8680da2640 10 monclient: get_auth_request con 0x7f865c03b970 auth_method 0 2024-09-15T21:25:41.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.149+0000 7f8680da2640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f865c03b970 0x7f865c03de30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86740047c0 tx=0x7f8674009520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:41.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.149+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f867c09c850 con 0x7f867c125380 2024-09-15T21:25:41.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.149+0000 7f86797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:41.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.149+0000 7f86797fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:41.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.325+0000 7f8683006640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T21:25:41.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.325+0000 7f8683006640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T21:25:41.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.325+0000 7f8683006640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:41.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.325+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) -- 0x7f867c0a1bf0 con 0x7f867c125380 2024-09-15T21:25:41.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.925+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 removed item id -94 name 'h26' from crush map v245) ==== 120+0+0 (secure 0 0 0) 0x7f866404dd80 con 0x7f867c125380 2024-09-15T21:25:41.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.925+0000 7f86797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T21:25:41.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:41.925+0000 7f86797fa640 10 monclient: _finish_command 2 = system:0 removed item id -94 name 'h26' from crush map 2024-09-15T21:25:42.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f8683006640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T21:25:42.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f8683006640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T21:25:42.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f8683006640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.033 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) -- 0x7f867c1406e0 con 0x7f867c125380 2024-09-15T21:25:42.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f86797fa640 1 -- 172.21.15.119:0/2151885619 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 device 'h26' does not appear in the crush map v245) ==== 120+0+0 (secure 0 0 0) 0x7f86640199e0 con 0x7f867c125380 2024-09-15T21:25:42.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f86797fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T21:25:42.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.029+0000 7f86797fa640 10 monclient: _finish_command 3 = system:0 device 'h26' does not appear in the crush map 2024-09-15T21:25:42.034 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h26' does not appear in the crush map 2024-09-15T21:25:42.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f865c03b970 msgr2=0x7f865c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f865c03b970 0x7f865c03de30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86740047c0 tx=0x7f8674009520 comp rx=0 tx=0).stop 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 10 monclient: shutdown 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 msgr2=0x7f867c125800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 0x7f867c125800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f866400c9f0 tx=0x7f866400ceb0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 shutdown_connections 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f865c03b970 0x7f865c03de30 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86740047c0 tx=0x7f8674009520 comp rx=0 tx=0).stop 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f867c13b740 0x7f867c13bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867c125380 0x7f867c125800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 --2- 172.21.15.119:0/2151885619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8668004d50 0x7f867c124e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 >> 172.21.15.119:0/2151885619 conn(0x7f867c0a2ae0 msgr2=0x7f867c0a5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:42.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 shutdown_connections 2024-09-15T21:25:42.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.033+0000 7f8683006640 1 -- 172.21.15.119:0/2151885619 wait complete. 2024-09-15T21:25:42.051 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:42.051 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h27 2024-09-15T21:25:42.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient: get_monmap_and_config 2024-09-15T21:25:42.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient: build_initial_monmap 2024-09-15T21:25:42.165 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient: monmap: 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:42.166549+0000 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:42.166549+0000 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:42.166 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:42.167 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:42.167 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:42.167 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:42.167 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:42.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 1 Processor -- start 2024-09-15T21:25:42.170 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 1 -- start start 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient: init 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3e6c137bc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e6c0a09e0 0x7f3e6c13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient(hunting): picked mon.noname-c con 0x7f3e6c0a09e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3e6c0a0ea0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e6c09cfa0 con 0x7f3e6c0a0ea0 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e6c09cd20 con 0x7f3e6c137bc0 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:42.171 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6c1386b0 con 0x7f3e6c0a09e0 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e72b27640 10 monclient(hunting): authenticate will time out at 1460.963794s 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.161+0000 7f3e710c4640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f3e6c0a0ea0 legacy=0x7f3e6c13efe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e708c3640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3e6c137bc0 0x7f3e6c137fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40804/0 (socket says 172.21.15.119:40804) 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e708c3640 1 -- 172.21.15.119:0/2250214474 learned_addr learned my addr 172.21.15.119:0/2250214474 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 1 --2- 172.21.15.119:0/2250214474 >> v2:172.21.15.119:3300/0 conn(0x7f3e6c0a09e0 0x7f3e6c13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): get_auth_request con 0x7f3e6c0a09e0 auth_method 0 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:42.172 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 1 -- 172.21.15.119:0/2250214474 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200471445 0 0) 0x7f3e6c09cd20 con 0x7f3e6c137bc0 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient(hunting): _negotiate 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient(hunting): my global_id is 84634 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 1 -- 172.21.15.119:0/2250214474 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e54003530 con 0x7f3e6c137bc0 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient(hunting): handle_auth_done global_id 84631 payload 274 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 1 -- 172.21.15.119:0/2250214474 >> v1:172.21.15.119:6789/0 conn(0x7f3e6c137bc0 legacy=0x7f3e6c137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 1 -- 172.21.15.119:0/2250214474 >> v1:172.21.15.17:6789/0 conn(0x7f3e6c0a0ea0 legacy=0x7f3e6c13efe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 1 monclient: found mon.noname-c 2024-09-15T21:25:42.173 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 1 -- 172.21.15.119:0/2250214474 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c0a1820 con 0x7f3e6c0a09e0 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e63fff640 1 --2- 172.21.15.119:0/2250214474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c0a09e0 0x7f3e6c13ca90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e680097c0 tx=0x7f3e6801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b6704a1a360d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 1 -- 172.21.15.119:0/2250214474 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680b0070 con 0x7f3e6c0a09e0 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: dump: 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:42.174 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 5 monclient: authenticate success, global_id 84631 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:42.175 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 1 -- 172.21.15.119:0/2250214474 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680ab600 con 0x7f3e6c0a09e0 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 1 -- 172.21.15.119:0/2250214474 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680ab9b0 con 0x7f3e6c0a09e0 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e637fe640 10 monclient: dump: 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.176 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:42.177 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 4 set_mon_vals no callback set 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 10 monclient: shutdown 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 -- 172.21.15.119:0/2250214474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c0a09e0 msgr2=0x7f3e6c13ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2250214474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c0a09e0 0x7f3e6c13ca90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e680097c0 tx=0x7f3e6801c3f0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 -- 172.21.15.119:0/2250214474 shutdown_connections 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2250214474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c0a09e0 0x7f3e6c13ca90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 -- 172.21.15.119:0/2250214474 >> 172.21.15.119:0/2250214474 conn(0x7f3e6c0a2ae0 msgr2=0x7f3e6c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 -- 172.21.15.119:0/2250214474 shutdown_connections 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.165+0000 7f3e72b27640 1 -- 172.21.15.119:0/2250214474 wait complete. 2024-09-15T21:25:42.185 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: build_initial_monmap 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: monmap: 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:42.186 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 Processor -- start 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- start start 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: init 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e6c0a09e0 0x7f3e6c11ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): picked mon.noname-c con 0x7f3e6c0a09e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e6c0a0ea0 0x7f3e6c11d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): picked mon.noname-a con 0x7f3e6c0a0ea0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 0x7f3e6c1cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:42.187 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): picked mon.noname-b con 0x7f3e6c137bc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c143100 con 0x7f3e6c0a0ea0 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c142f80 con 0x7f3e6c137bc0 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6c143280 con 0x7f3e6c0a09e0 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient(hunting): authenticate will time out at 1460.969100s 2024-09-15T21:25:42.188 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 0x7f3e6c1cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 0x7f3e6c1cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45786/0 (socket says 172.21.15.119:45786) 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 -- 172.21.15.119:0/2889353280 learned_addr learned my addr 172.21.15.119:0/2889353280 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): get_auth_request con 0x7f3e6c137bc0 auth_method 0 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e63fff640 1 -- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e6c0a0ea0 msgr2=0x7f3e6c11d2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e63fff640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e6c0a0ea0 0x7f3e6c11d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:42.189 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e708c3640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e6c0a09e0 0x7f3e6c11ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient(hunting): handle_auth_done global_id 84637 payload 824 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 -- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e6c0a09e0 msgr2=0x7f3e6c11ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e6c0a09e0 0x7f3e6c11ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 -- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e6c0a0ea0 msgr2=0x7f3e6c11d2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e6c0a0ea0 0x7f3e6c11d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient: _finish_hunting 0 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 monclient: found mon.noname-b 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 -- 172.21.15.119:0/2889353280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c11f540 con 0x7f3e6c137bc0 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e708c3640 10 monclient: get_auth_request con 0x7f3e6c0a09e0 auth_method 0 2024-09-15T21:25:42.190 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e708c3640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e6c0a09e0 0x7f3e6c11ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e710c4640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 0x7f3e6c1cf510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400b760 tx=0x7f3e6400bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6400c960 con 0x7f3e6c137bc0 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: dump: 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:42.191 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 5 monclient: authenticate success, global_id 84637 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _renew_subs 2024-09-15T21:25:42.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e64010070 con 0x7f3e6c137bc0 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c11fb10 con 0x7f3e6c137bc0 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6400cdc0 con 0x7f3e6c137bc0 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 10 monclient: dump: 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:42.193 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _renew_subs 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e62ffd640 4 set_mon_vals no callback set 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c13f900 con 0x7f3e6c137bc0 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3e64015470 con 0x7f3e6c137bc0 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6c137110 con 0x7f3e6c137bc0 2024-09-15T21:25:42.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e5803b970 0x7f3e5803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 9473+0+0 (secure 0 0 0) 0x7f3e64050d20 con 0x7f3e6c137bc0 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e708c3640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e5803b970 0x7f3e5803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e708c3640 10 monclient: get_auth_request con 0x7f3e5803b970 auth_method 0 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.169+0000 7f3e708c3640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e5803b970 0x7f3e5803de30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c005890 tx=0x7f3e5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.173+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3e6404e4d0 con 0x7f3e6c137bc0 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.173+0000 7f3e617fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:42.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.173+0000 7f3e617fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:42.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.345+0000 7f3e72b27640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T21:25:42.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.345+0000 7f3e72b27640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T21:25:42.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.345+0000 7f3e72b27640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:42.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.345+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) -- 0x7f3e6c137fa0 con 0x7f3e6c137bc0 2024-09-15T21:25:42.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.933+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 removed item id -97 name 'h27' from crush map v246) ==== 120+0+0 (secure 0 0 0) 0x7f3e64014a30 con 0x7f3e6c137bc0 2024-09-15T21:25:42.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.933+0000 7f3e617fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T21:25:42.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:42.933+0000 7f3e617fa640 10 monclient: _finish_command 2 = system:0 removed item id -97 name 'h27' from crush map 2024-09-15T21:25:43.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e72b27640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T21:25:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e72b27640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T21:25:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e72b27640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) -- 0x7f3e6c0008e0 con 0x7f3e6c137bc0 2024-09-15T21:25:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e617fa640 1 -- 172.21.15.119:0/2889353280 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 device 'h27' does not appear in the crush map v246) ==== 120+0+0 (secure 0 0 0) 0x7f3e64007dc0 con 0x7f3e6c137bc0 2024-09-15T21:25:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e617fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T21:25:43.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.045+0000 7f3e617fa640 10 monclient: _finish_command 3 = system:0 device 'h27' does not appear in the crush map 2024-09-15T21:25:43.051 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h27' does not appear in the crush map 2024-09-15T21:25:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e5803b970 msgr2=0x7f3e5803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e5803b970 0x7f3e5803de30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c005890 tx=0x7f3e5c009040 comp rx=0 tx=0).stop 2024-09-15T21:25:43.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 10 monclient: shutdown 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 msgr2=0x7f3e6c1cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 0x7f3e6c1cf510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400b760 tx=0x7f3e6400bc20 comp rx=0 tx=0).stop 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 shutdown_connections 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e5803b970 0x7f3e5803de30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e6c137bc0 0x7f3e6c1cf510 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e6c0a0ea0 0x7f3e6c11d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 --2- 172.21.15.119:0/2889353280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e6c0a09e0 0x7f3e6c11ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 >> 172.21.15.119:0/2889353280 conn(0x7f3e6c0a2ae0 msgr2=0x7f3e6c09dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 shutdown_connections 2024-09-15T21:25:43.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.049+0000 7f3e72b27640 1 -- 172.21.15.119:0/2889353280 wait complete. 2024-09-15T21:25:43.067 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:43.068 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h28 2024-09-15T21:25:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.181+0000 7f077b193640 10 monclient: get_monmap_and_config 2024-09-15T21:25:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.181+0000 7f077b193640 10 monclient: build_initial_monmap 2024-09-15T21:25:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.181+0000 7f077b193640 10 monclient: monmap: 2024-09-15T21:25:43.186 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:43.187870+0000 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:43.187870+0000 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:43.187 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.181+0000 7f077b193640 1 Processor -- start 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.181+0000 7f077b193640 1 -- start start 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.181+0000 7f077b193640 10 monclient: init 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0774135ca0 0x7f0774136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient(hunting): picked mon.noname-a con 0x7f0774135ca0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f07740a0160 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f077406ec30 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07740a1800 con 0x7f07740a0160 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07741365c0 con 0x7f077406ec30 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f077409cf30 con 0x7f0774135ca0 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 10 monclient(hunting): authenticate will time out at 1461.984712s 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f0774135ca0 msgr2=0x7f0774136080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f077406ec30 0x7f07740a10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38114/0 (socket says 172.21.15.119:38114) 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 -- 172.21.15.119:0/3740661843 learned_addr learned my addr 172.21.15.119:0/3740661843 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 1 --2- 172.21.15.119:0/3740661843 >> v2:172.21.15.17:3300/0 conn(0x7f0774135ca0 0x7f0774136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0773fff640 1 -- 172.21.15.119:0/3740661843 >> v1:172.21.15.17:6789/0 conn(0x7f07740a0160 legacy=0x7f07740a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934042855 0 0) 0x7f07741365c0 con 0x7f077406ec30 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): _negotiate 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): my global_id is 84640 2024-09-15T21:25:43.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f075c003530 con 0x7f077406ec30 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 718899821 0 0) 0x7f075c003530 con 0x7f077406ec30 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 >> v2:172.21.15.17:3300/0 conn(0x7f0774135ca0 msgr2=0x7f0774136080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 --2- 172.21.15.119:0/3740661843 >> v2:172.21.15.17:3300/0 conn(0x7f0774135ca0 0x7f0774136080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 >> v1:172.21.15.17:6789/0 conn(0x7f07740a0160 legacy=0x7f07740a09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07740a1ca0 con 0x7f077406ec30 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:43.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 5 monclient: authenticate success, global_id 84640 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f076c001a80 con 0x7f077406ec30 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: dump: 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.202 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 >> v1:172.21.15.119:6789/0 conn(0x7f077406ec30 legacy=0x7f07740a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f075c0039c0 0x7f075c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): picked mon.c con 0x7f075c0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c004810 0x7f075c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): picked mon.a con 0x7f075c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075c005380 0x7f075c00b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): picked mon.b con 0x7f075c005380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f077409cf30 con 0x7f075c004810 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f075c00c2d0 con 0x7f075c005380 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f075c00c4d0 con 0x7f075c0039c0 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:43.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f07737fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f077b193640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0773fff640 1 -- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c004810 msgr2=0x7f075c004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075c005380 0x7f075c00b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0773fff640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c004810 0x7f075c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): get_auth_request con 0x7f075c005380 auth_method 0 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:43.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f075c0039c0 0x7f075c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 10 monclient(hunting): get_auth_request con 0x7f075c0039c0 auth_method 0 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0778f2f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient(hunting): handle_auth_done global_id 84640 payload 293 2024-09-15T21:25:43.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 -- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f075c0039c0 msgr2=0x7f075c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f075c0039c0 0x7f075c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 -- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c004810 msgr2=0x7f075c004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c004810 0x7f075c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient: _finish_hunting 0 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 monclient: found mon.b 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 1 -- 172.21.15.119:0/3740661843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c003530 con 0x7f075c005380 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient: _finish_auth 0 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 10 monclient: _check_auth_tickets 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.185+0000 7f0779730640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:43.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 1 -- 172.21.15.119:0/3740661843 reap_dead start 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075c005380 0x7f075c00b9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f076c00a810 tx=0x7f076c01d240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc75b2753495802 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00a460 con 0x7f075c005380 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: dump: 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.208 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076c0ac640 con 0x7f075c005380 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 1 -- 172.21.15.119:0/3740661843 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0aca30 con 0x7f075c005380 2024-09-15T21:25:43.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 4 set_mon_vals no callback set 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07737fe640 10 monclient: dump: 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:43.210 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: shutdown 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3740661843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075c005380 msgr2=0x7f075c00b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075c005380 0x7f075c00b9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f076c00a810 tx=0x7f076c01d240 comp rx=0 tx=0).stop 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3740661843 shutdown_connections 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 --2- 172.21.15.119:0/3740661843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075c005380 0x7f075c00b9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3740661843 >> 172.21.15.119:0/3740661843 conn(0x7f07740a2ae0 msgr2=0x7f07740a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3740661843 shutdown_connections 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3740661843 wait complete. 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: build_initial_monmap 2024-09-15T21:25:43.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: monmap: 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 Processor -- start 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- start start 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: init 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c005380 0x7f077413db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): picked mon.noname-a con 0x7f075c005380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 0x7f077413e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): picked mon.noname-b con 0x7f0774138f20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07741cda30 0x7f0774136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): picked mon.noname-c con 0x7f07741cda30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:43.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0774136200 con 0x7f075c005380 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0774136080 con 0x7f0774138f20 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0774136380 con 0x7f07741cda30 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient(hunting): authenticate will time out at 1461.990754s 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0778f2f640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c005380 msgr2=0x7f077413db70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0778f2f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c005380 0x7f077413db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 0x7f077413e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 0x7f077413e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37616/0 (socket says 172.21.15.119:37616) 2024-09-15T21:25:43.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 -- 172.21.15.119:0/3321359584 learned_addr learned my addr 172.21.15.119:0/3321359584 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): get_auth_request con 0x7f0774138f20 auth_method 0 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07741cda30 0x7f0774136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 10 monclient(hunting): get_auth_request con 0x7f07741cda30 auth_method 0 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0779730640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:43.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient(hunting): handle_auth_done global_id 84643 payload 824 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 -- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07741cda30 msgr2=0x7f0774136a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07741cda30 0x7f0774136a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 -- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c005380 msgr2=0x7f077413db70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c005380 0x7f077413db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 monclient: found mon.noname-b 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 -- 172.21.15.119:0/3321359584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0774135360 con 0x7f0774138f20 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0773fff640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 0x7f077413e0b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f076800b580 tx=0x7f076800ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:43.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768013020 con 0x7f0774138f20 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: dump: 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0768003df0 con 0x7f0774138f20 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680052b0 con 0x7f0774138f20 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f07717fa640 10 monclient: dump: 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.218 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 5 monclient: authenticate success, global_id 84643 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: _renew_subs 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f0772ffd640 4 set_mon_vals no callback set 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077409d1d0 con 0x7f0774138f20 2024-09-15T21:25:43.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: _renew_subs 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.189+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077413a720 con 0x7f0774138f20 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0768016020 con 0x7f0774138f20 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f077b193640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f077b193640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f077b193640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0774136560 con 0x7f0774138f20 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f07717fa640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f076003b9a0 0x7f076003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 9260+0+0 (secure 0 0 0) 0x7f07680509d0 con 0x7f0774138f20 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f0778f2f640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f076003b9a0 0x7f076003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f0778f2f640 10 monclient: get_auth_request con 0x7f076003b9a0 auth_method 0 2024-09-15T21:25:43.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.193+0000 7f0778f2f640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f076003b9a0 0x7f076003de60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0764003910 tx=0x7f0764008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:43.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.197+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f076801d020 con 0x7f0774138f20 2024-09-15T21:25:43.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.197+0000 7f07717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:43.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.197+0000 7f07717fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:43.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.369+0000 7f077b193640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T21:25:43.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.369+0000 7f077b193640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T21:25:43.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.369+0000 7f077b193640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:43.374 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.369+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) -- 0x7f077409f8f0 con 0x7f0774138f20 2024-09-15T21:25:43.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.937+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 removed item id -100 name 'h28' from crush map v247) ==== 121+0+0 (secure 0 0 0) 0x7f076800fe20 con 0x7f0774138f20 2024-09-15T21:25:43.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.937+0000 7f07717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T21:25:43.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:43.937+0000 7f07717fa640 10 monclient: _finish_command 2 = system:0 removed item id -100 name 'h28' from crush map 2024-09-15T21:25:44.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.041+0000 7f077b193640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T21:25:44.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.041+0000 7f077b193640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T21:25:44.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.041+0000 7f077b193640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.041+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) -- 0x7f07740008e0 con 0x7f0774138f20 2024-09-15T21:25:44.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f07717fa640 1 -- 172.21.15.119:0/3321359584 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 device 'h28' does not appear in the crush map v247) ==== 120+0+0 (secure 0 0 0) 0x7f076800d380 con 0x7f0774138f20 2024-09-15T21:25:44.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f07717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T21:25:44.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f07717fa640 10 monclient: _finish_command 3 = system:0 device 'h28' does not appear in the crush map 2024-09-15T21:25:44.048 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h28' does not appear in the crush map 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f076003b9a0 msgr2=0x7f076003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f076003b9a0 0x7f076003de60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0764003910 tx=0x7f0764008040 comp rx=0 tx=0).stop 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 10 monclient: shutdown 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 msgr2=0x7f077413e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 0x7f077413e0b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f076800b580 tx=0x7f076800ba40 comp rx=0 tx=0).stop 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 shutdown_connections 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f076003b9a0 0x7f076003de60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f07741cda30 0x7f0774136a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0774138f20 0x7f077413e0b0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f076800b580 tx=0x7f076800ba40 comp rx=0 tx=0).stop 2024-09-15T21:25:44.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 --2- 172.21.15.119:0/3321359584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f075c005380 0x7f077413db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 >> 172.21.15.119:0/3321359584 conn(0x7f07740a2ae0 msgr2=0x7f0774072620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:44.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 shutdown_connections 2024-09-15T21:25:44.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.045+0000 7f077b193640 1 -- 172.21.15.119:0/3321359584 wait complete. 2024-09-15T21:25:44.064 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:44.065 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h29 2024-09-15T21:25:44.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.177+0000 7fb93b281640 10 monclient: get_monmap_and_config 2024-09-15T21:25:44.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.177+0000 7fb93b281640 10 monclient: build_initial_monmap 2024-09-15T21:25:44.182 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.177+0000 7fb93b281640 10 monclient: monmap: 2024-09-15T21:25:44.182 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:44.183882+0000 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:44.183882+0000 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:44.183 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.177+0000 7fb93b281640 1 Processor -- start 2024-09-15T21:25:44.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.177+0000 7fb93b281640 1 -- start start 2024-09-15T21:25:44.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.177+0000 7fb93b281640 10 monclient: init 2024-09-15T21:25:44.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:44.192 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb934136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): picked mon.noname-b con 0x7fb934135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb9340a0160 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb93406ec30 0x7fb9340a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): picked mon.noname-a con 0x7fb93406ec30 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9341365c0 con 0x7fb9340a0160 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb93409cf30 con 0x7fb93406ec30 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb9340a1ca0 con 0x7fb934135ca0 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:44.193 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 10 monclient(hunting): authenticate will time out at 1462.980579s 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fb93406ec30 msgr2=0x7fb9340a10f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb9340a0160 0x7fb9340a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38124/0 (socket says 172.21.15.119:38124) 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 1 -- 172.21.15.119:0/1116801498 learned_addr learned my addr 172.21.15.119:0/1116801498 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 --2- 172.21.15.119:0/1116801498 >> v2:172.21.15.17:3300/0 conn(0x7fb93406ec30 0x7fb9340a10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb934136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): get_auth_request con 0x7fb934135ca0 auth_method 0 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036712397 0 0) 0x7fb9341365c0 con 0x7fb9340a0160 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): _negotiate 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:44.194 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): my global_id is 84646 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb920003530 con 0x7fb9340a0160 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 962177127 0 0) 0x7fb920003530 con 0x7fb9340a0160 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 msgr2=0x7fb934136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb934136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 >> v2:172.21.15.17:3300/0 conn(0x7fb93406ec30 msgr2=0x7fb9340a10f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 --2- 172.21.15.119:0/1116801498 >> v2:172.21.15.17:3300/0 conn(0x7fb93406ec30 0x7fb9340a10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:44.195 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9340a1f80 con 0x7fb9340a0160 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: _finish_auth 0 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb924001700 con 0x7fb9340a0160 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: dump: 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:44.196 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 >> v1:172.21.15.119:6789/0 conn(0x7fb9340a0160 legacy=0x7fb9340a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9200039c0 0x7fb920003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): picked mon.a con 0x7fb9200039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb920004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): picked mon.c con 0x7fb934135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb920004d50 0x7fb9200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): picked mon.b con 0x7fb920004d50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb93409cf30 con 0x7fb9200039c0 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9340a1ca0 con 0x7fb920004d50 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb920010040 con 0x7fb934135ca0 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb92bfff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:44.198 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 1 -- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9200039c0 msgr2=0x7fb920003de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93901d640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9200039c0 0x7fb920003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb920004d50 0x7fb9200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): get_auth_request con 0x7fb920004d50 auth_method 0 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb920004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 10 monclient(hunting): get_auth_request con 0x7fb934135ca0 auth_method 0 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:44.199 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient(hunting): handle_auth_done global_id 84646 payload 293 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 -- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 msgr2=0x7fb920004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb920004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 -- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9200039c0 msgr2=0x7fb920003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9200039c0 0x7fb920003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient: _finish_hunting 0 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 monclient: found mon.b 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 1 -- 172.21.15.119:0/1116801498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb920003530 con 0x7fb920004d50 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient: _finish_auth 0 2024-09-15T21:25:44.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 10 monclient: _check_auth_tickets 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93981e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 5 monclient: authenticate success, global_id 84646 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93b281640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.181+0000 7fb93881c640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934135ca0 0x7fb920004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb920004d50 0x7fb9200051d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb930003cc0 tx=0x7fb93001cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bbe2242107e6da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930009a50 con 0x7fb920004d50 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: dump: 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.201 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9300a9070 con 0x7fb920004d50 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 1 -- 172.21.15.119:0/1116801498 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300a5cd0 con 0x7fb920004d50 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:44.202 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92bfff640 10 monclient: dump: 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 4 set_mon_vals no callback set 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: shutdown 2024-09-15T21:25:44.203 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/1116801498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb920004d50 msgr2=0x7fb9200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 --2- 172.21.15.119:0/1116801498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb920004d50 0x7fb9200051d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb930003cc0 tx=0x7fb93001cdd0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 1 -- 172.21.15.119:0/1116801498 reap_dead start 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/1116801498 shutdown_connections 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/1116801498 >> 172.21.15.119:0/1116801498 conn(0x7fb9340a2ae0 msgr2=0x7fb9340a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/1116801498 shutdown_connections 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/1116801498 wait complete. 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: build_initial_monmap 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: monmap: 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:44.204 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 Processor -- start 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- start start 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: init 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 0x7fb934132c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): picked mon.noname-b con 0x7fb934132840 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:44.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9341cbab0 0x7fb93411f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): picked mon.noname-a con 0x7fb9341cbab0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934121a30 0x7fb93411f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): picked mon.noname-c con 0x7fb934121a30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb93409dcf0 con 0x7fb9341cbab0 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9340a1840 con 0x7fb934132840 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb93409de70 con 0x7fb934121a30 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient(hunting): authenticate will time out at 1462.986593s 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 0x7fb934132c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:44.206 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 0x7fb934132c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37636/0 (socket says 172.21.15.119:37636) 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93881c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9341cbab0 msgr2=0x7fb93411f410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 -- 172.21.15.119:0/4131195592 learned_addr learned my addr 172.21.15.119:0/4131195592 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): get_auth_request con 0x7fb934132840 auth_method 0 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93881c640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9341cbab0 0x7fb93411f410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934121a30 0x7fb93411f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 10 monclient(hunting): get_auth_request con 0x7fb934121a30 auth_method 0 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:44.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93981e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient(hunting): handle_auth_done global_id 84649 payload 824 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 -- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934121a30 msgr2=0x7fb93411f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934121a30 0x7fb93411f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 -- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9341cbab0 msgr2=0x7fb93411f410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9341cbab0 0x7fb93411f410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 monclient: found mon.noname-b 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 -- 172.21.15.119:0/4131195592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb934123230 con 0x7fb934132840 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93901d640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 0x7fb934132c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00e900 tx=0x7fb92c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:44.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c017070 con 0x7fb934132840 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: dump: 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c00ca00 con 0x7fb934132840 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 5 monclient: authenticate success, global_id 84649 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: _renew_subs 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93409d1d0 con 0x7fb934132840 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00cdd0 con 0x7fb934132840 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb929ffb640 10 monclient: dump: 2024-09-15T21:25:44.210 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: _renew_subs 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb92b7fe640 4 set_mon_vals no callback set 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.185+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb934122990 con 0x7fb934132840 2024-09-15T21:25:44.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93b281640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93b281640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93b281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb934136490 con 0x7fb934132840 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb92c01a070 con 0x7fb934132840 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb929ffb640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb91803b9d0 0x7fb91803de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 9047+0+0 (secure 0 0 0) 0x7fb92c050c30 con 0x7fb934132840 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93881c640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb91803b9d0 0x7fb91803de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93881c640 10 monclient: get_auth_request con 0x7fb91803b9d0 auth_method 0 2024-09-15T21:25:44.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.189+0000 7fb93881c640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb91803b9d0 0x7fb91803de90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb924001b80 tx=0x7fb924001af0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.193+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb92c04e590 con 0x7fb934132840 2024-09-15T21:25:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.193+0000 7fb929ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:44.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.193+0000 7fb929ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:44.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.369+0000 7fb93b281640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T21:25:44.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.369+0000 7fb93b281640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T21:25:44.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.369+0000 7fb93b281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:44.373 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.369+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) -- 0x7fb93409f8f0 con 0x7fb934132840 2024-09-15T21:25:44.958 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.953+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 removed item id -103 name 'h29' from crush map v248) ==== 121+0+0 (secure 0 0 0) 0x7fb92c013c70 con 0x7fb934132840 2024-09-15T21:25:44.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.953+0000 7fb929ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T21:25:44.959 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:44.953+0000 7fb929ffb640 10 monclient: _finish_command 2 = system:0 removed item id -103 name 'h29' from crush map 2024-09-15T21:25:45.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb93b281640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T21:25:45.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb93b281640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T21:25:45.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb93b281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) -- 0x7fb934133280 con 0x7fb934132840 2024-09-15T21:25:45.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb929ffb640 1 -- 172.21.15.119:0/4131195592 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 device 'h29' does not appear in the crush map v248) ==== 120+0+0 (secure 0 0 0) 0x7fb92c0232f0 con 0x7fb934132840 2024-09-15T21:25:45.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb929ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T21:25:45.071 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.065+0000 7fb929ffb640 10 monclient: _finish_command 3 = system:0 device 'h29' does not appear in the crush map 2024-09-15T21:25:45.071 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h29' does not appear in the crush map 2024-09-15T21:25:45.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb91803b9d0 msgr2=0x7fb91803de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:45.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb91803b9d0 0x7fb91803de90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb924001b80 tx=0x7fb924001af0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 10 monclient: shutdown 2024-09-15T21:25:45.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:45.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 msgr2=0x7fb934132c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 0x7fb934132c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00e900 tx=0x7fb92c00edc0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 shutdown_connections 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb91803b9d0 0x7fb91803de90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb934121a30 0x7fb93411f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9341cbab0 0x7fb93411f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 --2- 172.21.15.119:0/4131195592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934132840 0x7fb934132c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 >> 172.21.15.119:0/4131195592 conn(0x7fb9340a2ae0 msgr2=0x7fb9340a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 shutdown_connections 2024-09-15T21:25:45.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.069+0000 7fb93b281640 1 -- 172.21.15.119:0/4131195592 wait complete. 2024-09-15T21:25:45.088 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:45.088 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h30 2024-09-15T21:25:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient: get_monmap_and_config 2024-09-15T21:25:45.204 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient: build_initial_monmap 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient: monmap: 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:45.206191+0000 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:45.206191+0000 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:45.205 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:45.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 Processor -- start 2024-09-15T21:25:45.207 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 -- start start 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient: init 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f570c09f640 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f570c0a0130 0x7f570c09d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): picked mon.noname-c con 0x7f570c0a0130 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c139c00 0x7f570c13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): picked mon.noname-b con 0x7f570c139c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f570c09e880 con 0x7f570c09f640 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f570c09eb60 con 0x7f570c0a0130 2024-09-15T21:25:45.208 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f570c09ee40 con 0x7f570c139c00 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5712377640 10 monclient(hunting): authenticate will time out at 1464.003049s 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f570c0a0130 0x7f570c09d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570bfff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f570c09f640 legacy=0x7f570c09fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f570c0a0130 0x7f570c09d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37650/0 (socket says 172.21.15.119:37650) 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 -- 172.21.15.119:0/273234501 learned_addr learned my addr 172.21.15.119:0/273234501 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): get_auth_request con 0x7f570c0a0130 auth_method 0 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:45.209 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5710914640 1 --2- 172.21.15.119:0/273234501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c139c00 0x7f570c13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5710914640 10 monclient(hunting): get_auth_request con 0x7f570c139c00 auth_method 0 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5710914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5710914640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f5710914640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient(hunting): handle_auth_done global_id 84652 payload 274 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 -- 172.21.15.119:0/273234501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c139c00 msgr2=0x7f570c13bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 --2- 172.21.15.119:0/273234501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c139c00 0x7f570c13bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 -- 172.21.15.119:0/273234501 >> v1:172.21.15.17:6789/0 conn(0x7f570c09f640 legacy=0x7f570c09fa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:45.210 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.201+0000 7f570b7fe640 1 -- 172.21.15.119:0/273234501 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c0a2840 con 0x7f570c0a0130 2024-09-15T21:25:45.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570b7fe640 1 --2- 172.21.15.119:0/273234501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c0a0130 0x7f570c09d9a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5700009870 tx=0x7f570001c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc472112527fdbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:45.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 1 -- 172.21.15.119:0/273234501 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000b0070 con 0x7f570c0a0130 2024-09-15T21:25:45.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: dump: 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: _finish_auth 0 2024-09-15T21:25:45.215 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 1 -- 172.21.15.119:0/273234501 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57000ab600 con 0x7f570c0a0130 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 1 -- 172.21.15.119:0/273234501 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000ab9d0 con 0x7f570c0a0130 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f570affd640 10 monclient: dump: 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.216 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 5 monclient: authenticate success, global_id 84652 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 4 set_mon_vals no callback set 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 10 monclient: shutdown 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 -- 172.21.15.119:0/273234501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c0a0130 msgr2=0x7f570c09d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 --2- 172.21.15.119:0/273234501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c0a0130 0x7f570c09d9a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5700009870 tx=0x7f570001c420 comp rx=0 tx=0).stop 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 -- 172.21.15.119:0/273234501 shutdown_connections 2024-09-15T21:25:45.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 --2- 172.21.15.119:0/273234501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c139c00 0x7f570c13bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 --2- 172.21.15.119:0/273234501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c0a0130 0x7f570c09d9a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 -- 172.21.15.119:0/273234501 >> 172.21.15.119:0/273234501 conn(0x7f570c0a2ae0 msgr2=0x7f570c0a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 -- 172.21.15.119:0/273234501 shutdown_connections 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 -- 172.21.15.119:0/273234501 wait complete. 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 10 monclient: build_initial_monmap 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 10 monclient: monmap: 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:45.218 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 Processor -- start 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 1 -- start start 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.205+0000 7f5712377640 10 monclient: init 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f570c11caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): picked mon.noname-c con 0x7f570c09f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f570c072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): picked mon.noname-a con 0x7f570c0a0130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): picked mon.noname-b con 0x7f570c139c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.219 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f570c13eed0 con 0x7f570c0a0130 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f570c13ed50 con 0x7f570c139c00 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f570c13f050 con 0x7f570c09f640 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5712377640 10 monclient(hunting): authenticate will time out at 1464.008227s 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570b7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 msgr2=0x7f570c072190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37662/0 (socket says 172.21.15.119:37662) 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 learned_addr learned my addr 172.21.15.119:0/1255053058 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570b7fe640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f570c072190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): get_auth_request con 0x7f570c139c00 auth_method 0 2024-09-15T21:25:45.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570bfff640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f570c11caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570bfff640 10 monclient(hunting): get_auth_request con 0x7f570c09f640 auth_method 0 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f570bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient(hunting): handle_auth_done global_id 84655 payload 824 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 msgr2=0x7f570c11caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:45.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f570c11caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 msgr2=0x7f570c072190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f570c072190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient: _finish_hunting 0 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 monclient: found mon.noname-b 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c09d1d0 con 0x7f570c139c00 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 msgr2=0x7f570c132950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 msgr2=0x7f570c132950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f5704007e40 tx=0x7f570400e4b0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:45.222 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.209+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:45.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient: get_auth_request con 0x7f570c139c00 auth_method 0 2024-09-15T21:25:45.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 0 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:25:45.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.414 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: ms_handle_reset current mon [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 msgr2=0x7f570c132950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570c139c00 0x7f570c132950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f56f8005500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f570c0a0130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f56f8007fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f570c09f640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f8008a30 0x7f56f800aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f56f8008a30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f56f800b9c0 con 0x7f570c0a0130 2024-09-15T21:25:45.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56f800bb40 con 0x7f56f8008a30 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f56f800bd40 con 0x7f570c09f640 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570bfff640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 msgr2=0x7f56f8005500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f8008a30 0x7f56f800aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570bfff640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f56f8005500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): get_auth_request con 0x7f56f8008a30 auth_method 0 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f56f8007fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): get_auth_request con 0x7f570c09f640 auth_method 0 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient(hunting): handle_auth_done global_id 84655 payload 293 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 msgr2=0x7f56f8007fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f56f8007fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.417 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 msgr2=0x7f56f8005500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f56f8005500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient: _finish_hunting 0 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 monclient: found mon.noname-b 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f800bfa0 con 0x7f56f8008a30 2024-09-15T21:25:45.418 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5710914640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f8008a30 0x7f56f800aec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f570c073a40 tx=0x7f57040098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:45.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704023020 con 0x7f56f8008a30 2024-09-15T21:25:45.419 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: dump: 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:45.420 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: _finish_auth 0 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5704014950 con 0x7f56f8008a30 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 5 monclient: authenticate success, global_id 84655 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 monclient: _renew_subs 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570c0737a0 con 0x7f56f8008a30 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704014480 con 0x7f56f8008a30 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 10 monclient: dump: 2024-09-15T21:25:45.421 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570a7fc640 4 set_mon_vals no callback set 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 monclient: _renew_subs 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.422 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570c11caf0 con 0x7f56f8008a30 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5704026070 con 0x7f56f8008a30 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570c09d9a0 con 0x7f56f8008a30 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f570c139c00 0x7f56f8046160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 8834+0+0 (secure 0 0 0) 0x7f5704051be0 con 0x7f56f8008a30 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570bfff640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f570c139c00 0x7f56f8046160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570bfff640 10 monclient: get_auth_request con 0x7f570c139c00 auth_method 0 2024-09-15T21:25:45.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.413+0000 7f570bfff640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f570c139c00 0x7f56f8046160 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006fd0 tx=0x7f56fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:45.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.421+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f570401d3b0 con 0x7f56f8008a30 2024-09-15T21:25:45.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.421+0000 7f5708ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:45.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.421+0000 7f5708ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:45.625 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.621+0000 7f5712377640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T21:25:45.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.621+0000 7f5712377640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T21:25:45.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.621+0000 7f5712377640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:45.626 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.621+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) -- 0x7f570c11cde0 con 0x7f56f8008a30 2024-09-15T21:25:45.966 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.961+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 removed item id -106 name 'h30' from crush map v249) ==== 121+0+0 (secure 0 0 0) 0x7f570404f610 con 0x7f56f8008a30 2024-09-15T21:25:45.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.961+0000 7f5708ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T21:25:45.967 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:45.961+0000 7f5708ff9640 10 monclient: _finish_command 2 = system:0 removed item id -106 name 'h30' from crush map 2024-09-15T21:25:46.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.073+0000 7f5712377640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T21:25:46.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.073+0000 7f5712377640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T21:25:46.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.073+0000 7f5712377640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.073+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) -- 0x7f570c0008e0 con 0x7f56f8008a30 2024-09-15T21:25:46.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5708ff9640 1 -- 172.21.15.119:0/1255053058 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 device 'h30' does not appear in the crush map v249) ==== 120+0+0 (secure 0 0 0) 0x7f57040124a0 con 0x7f56f8008a30 2024-09-15T21:25:46.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5708ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T21:25:46.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5708ff9640 10 monclient: _finish_command 3 = system:0 device 'h30' does not appear in the crush map 2024-09-15T21:25:46.079 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h30' does not appear in the crush map 2024-09-15T21:25:46.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f570c139c00 msgr2=0x7f56f8046160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f570c139c00 0x7f56f8046160 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006fd0 tx=0x7f56fc008040 comp rx=0 tx=0).stop 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 10 monclient: shutdown 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f8008a30 msgr2=0x7f56f800aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f8008a30 0x7f56f800aec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f570c073a40 tx=0x7f57040098d0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 shutdown_connections 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f570c139c00 0x7f56f8046160 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f8008a30 0x7f56f800aec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f570c09f640 0x7f56f8007fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 --2- 172.21.15.119:0/1255053058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c0a0130 0x7f56f8005500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 >> 172.21.15.119:0/1255053058 conn(0x7f570c0a2ae0 msgr2=0x7f570c135be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:46.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 shutdown_connections 2024-09-15T21:25:46.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.077+0000 7f5712377640 1 -- 172.21.15.119:0/1255053058 wait complete. 2024-09-15T21:25:46.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:46.097 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h31 2024-09-15T21:25:46.176 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:25:46.176 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:25:46.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:25:46.179 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:25:46.179 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:25:46.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:46.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient: get_monmap_and_config 2024-09-15T21:25:46.216 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient: build_initial_monmap 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient: monmap: 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:46.218069+0000 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:46.218069+0000 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:46.217 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 Processor -- start 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 -- start start 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient: init 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b4135ca0 0x7fa4b4136080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): picked mon.noname-b con 0x7fa4b4135ca0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa4b40a0160 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa4b406ec30 0x7fa4b40a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.220 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): picked mon.noname-c con 0x7fa4b406ec30 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:46.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b41365c0 con 0x7fa4b40a0160 2024-09-15T21:25:46.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b409cf30 con 0x7fa4b406ec30 2024-09-15T21:25:46.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.221 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b40a1ca0 con 0x7fa4b4135ca0 2024-09-15T21:25:46.224 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b986f640 10 monclient(hunting): authenticate will time out at 1465.015288s 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4aa7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa4b40a0160 0x7fa4b40a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38134/0 (socket says 172.21.15.119:38134) 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4aa7fc640 1 -- 172.21.15.119:0/1480897405 learned_addr learned my addr 172.21.15.119:0/1480897405 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 1 --2- 172.21.15.119:0/1480897405 >> v2:172.21.15.119:3300/0 conn(0x7fa4b406ec30 0x7fa4b40a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): get_auth_request con 0x7fa4b406ec30 auth_method 0 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 1 --2- 172.21.15.119:0/1480897405 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b4135ca0 0x7fa4b4136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): get_auth_request con 0x7fa4b4135ca0 auth_method 0 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b27fc640 1 -- 172.21.15.119:0/1480897405 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907718229 0 0) 0x7fa4b41365c0 con 0x7fa4b40a0160 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b27fc640 10 monclient(hunting): _negotiate 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b27fc640 10 monclient(hunting): my global_id is 84661 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b27fc640 1 -- 172.21.15.119:0/1480897405 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a0003530 con 0x7fa4b40a0160 2024-09-15T21:25:46.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient(hunting): handle_auth_done global_id 84658 payload 274 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 1 -- 172.21.15.119:0/1480897405 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b4135ca0 msgr2=0x7fa4b4136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 1 --2- 172.21.15.119:0/1480897405 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b4135ca0 0x7fa4b4136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 1 -- 172.21.15.119:0/1480897405 >> v1:172.21.15.119:6789/0 conn(0x7fa4b40a0160 legacy=0x7fa4b40a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.213+0000 7fa4b37fe640 1 -- 172.21.15.119:0/1480897405 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b40a1f80 con 0x7fa4b406ec30 2024-09-15T21:25:46.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b37fe640 1 --2- 172.21.15.119:0/1480897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b406ec30 0x7fa4b40a10f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009d90 tx=0x7fa4ac01c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237d57c3e62510b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:46.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 1 -- 172.21.15.119:0/1480897405 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0a6700 con 0x7fa4b406ec30 2024-09-15T21:25:46.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:46.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: dump: 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:46.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 1 -- 172.21.15.119:0/1480897405 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac0a68a0 con 0x7fa4b406ec30 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 1 -- 172.21.15.119:0/1480897405 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0a6c70 con 0x7fa4b406ec30 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 5 monclient: authenticate success, global_id 84658 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 4 set_mon_vals no callback set 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b27fc640 10 monclient: dump: 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.235 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 10 monclient: shutdown 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 -- 172.21.15.119:0/1480897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b406ec30 msgr2=0x7fa4b40a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1480897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b406ec30 0x7fa4b40a10f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009d90 tx=0x7fa4ac01c830 comp rx=0 tx=0).stop 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 -- 172.21.15.119:0/1480897405 shutdown_connections 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1480897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b406ec30 0x7fa4b40a10f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1480897405 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b4135ca0 0x7fa4b4136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 -- 172.21.15.119:0/1480897405 >> 172.21.15.119:0/1480897405 conn(0x7fa4b40a2ae0 msgr2=0x7fa4b40a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 -- 172.21.15.119:0/1480897405 shutdown_connections 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 -- 172.21.15.119:0/1480897405 wait complete. 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 10 monclient: build_initial_monmap 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 10 monclient: monmap: 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:46.237 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.217+0000 7fa4b986f640 1 Processor -- start 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- start start 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: init 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b406ec30 0x7fa4b41307e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): picked mon.noname-c con 0x7fa4b406ec30 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 0x7fa4b4130d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): picked mon.noname-b con 0x7fa4b40a0160 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4b4135ca0 0x7fa4b409d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): picked mon.noname-a con 0x7fa4b4135ca0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41403c0 con 0x7fa4b4135ca0 2024-09-15T21:25:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4140240 con 0x7fa4b40a0160 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b4140540 con 0x7fa4b406ec30 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient(hunting): authenticate will time out at 1465.020577s 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b37fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4b4135ca0 msgr2=0x7fa4b409d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4b4135ca0 0x7fa4b409d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 0x7fa4b4130d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 0x7fa4b4130d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37702/0 (socket says 172.21.15.119:37702) 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 -- 172.21.15.119:0/1140646824 learned_addr learned my addr 172.21.15.119:0/1140646824 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): get_auth_request con 0x7fa4b40a0160 auth_method 0 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b406ec30 0x7fa4b41307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 10 monclient(hunting): get_auth_request con 0x7fa4b406ec30 auth_method 0 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient(hunting): handle_auth_done global_id 84664 payload 824 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 -- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b406ec30 msgr2=0x7fa4b41307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b406ec30 0x7fa4b41307e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 -- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4b4135ca0 msgr2=0x7fa4b409d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4b4135ca0 0x7fa4b409d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 monclient: found mon.noname-b 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 -- 172.21.15.119:0/1140646824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b411ceb0 con 0x7fa4b40a0160 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b406ec30 0x7fa4b41307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4aa7fc640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 0x7fa4b4130d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00c4d0 tx=0x7fa49c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c017070 con 0x7fa4b40a0160 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: dump: 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:46.241 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49c005cf0 con 0x7fa4b40a0160 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 5 monclient: authenticate success, global_id 84664 2024-09-15T21:25:46.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _renew_subs 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b411d190 con 0x7fa4b40a0160 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c007270 con 0x7fa4b40a0160 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 10 monclient: dump: 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.243 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b1ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _renew_subs 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b409d1d0 con 0x7fa4b40a0160 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b4136080 con 0x7fa4b40a0160 2024-09-15T21:25:46.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa49c01a070 con 0x7fa4b40a0160 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa49403b970 0x7fa49403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(249..249 src has 1..249) ==== 8621+0+0 (secure 0 0 0) 0x7fa49c04fd70 con 0x7fa4b40a0160 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa49403b970 0x7fa49403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 10 monclient: get_auth_request con 0x7fa49403b970 auth_method 0 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.221+0000 7fa4b2ffd640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa49403b970 0x7fa49403de30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009bb0 tx=0x7fa4a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.225+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa49c01b3b0 con 0x7fa4b40a0160 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.225+0000 7fa4b0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:46.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.225+0000 7fa4b0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:46.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient: get_monmap_and_config 2024-09-15T21:25:46.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient: build_initial_monmap 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient: monmap: 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:46.308260+0000 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:46.308260+0000 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:46.309 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:46.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 1 Processor -- start 2024-09-15T21:25:46.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 1 -- start start 2024-09-15T21:25:46.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient: init 2024-09-15T21:25:46.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:46.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:25:46.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff560137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient(hunting): picked mon.noname-b con 0x7ff5601378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5601381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff5600a5210 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5600a1a20 con 0x7ff5601381e0 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5600a1d00 con 0x7ff5600a5210 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff5600a1fe0 con 0x7ff5601378c0 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff5683b0640 10 monclient(hunting): authenticate will time out at 1469.703311s 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff560137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56594b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff5601381e0 0x7ff5600a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46328/0 (socket says 172.21.15.17:46328) 2024-09-15T21:25:46.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): get_auth_request con 0x7ff5601378c0 auth_method 0 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56594b640 1 -- 172.21.15.17:0/501835513 learned_addr learned my addr 172.21.15.17:0/501835513 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient(hunting): handle_auth_done global_id 45251 payload 274 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 1 -- 172.21.15.17:0/501835513 >> v1:172.21.15.119:6789/0 conn(0x7ff5600a5210 legacy=0x7ff5600a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:46.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 1 -- 172.21.15.17:0/501835513 >> v1:172.21.15.17:6789/0 conn(0x7ff5601381e0 legacy=0x7ff5600a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient: _finish_hunting 0 2024-09-15T21:25:46.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 1 monclient: found mon.noname-b 2024-09-15T21:25:46.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:46.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:46.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.303+0000 7ff56614c640 1 -- 172.21.15.17:0/501835513 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5600a22c0 con 0x7ff5601378c0 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56614c640 1 --2- 172.21.15.17:0/501835513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff560137ca0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097f0 tx=0x7ff55c01c080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd89055d275dd942 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 1 -- 172.21.15.17:0/501835513 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a5700 con 0x7ff5601378c0 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: dump: 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.319 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: _finish_auth 0 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 1 -- 172.21.15.17:0/501835513 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c0a58a0 con 0x7ff5601378c0 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 5 monclient: authenticate success, global_id 45251 2024-09-15T21:25:46.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 1 -- 172.21.15.17:0/501835513 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a5c70 con 0x7ff5601378c0 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 4 set_mon_vals no callback set 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff56514a640 10 monclient: dump: 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:46.321 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 10 monclient: shutdown 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 -- 172.21.15.17:0/501835513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 msgr2=0x7ff560137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 --2- 172.21.15.17:0/501835513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff560137ca0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097f0 tx=0x7ff55c01c080 comp rx=0 tx=0).stop 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 -- 172.21.15.17:0/501835513 shutdown_connections 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 --2- 172.21.15.17:0/501835513 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff560137ca0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 -- 172.21.15.17:0/501835513 >> 172.21.15.17:0/501835513 conn(0x7ff5600aac40 msgr2=0x7ff5600a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 -- 172.21.15.17:0/501835513 shutdown_connections 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 1 -- 172.21.15.17:0/501835513 wait complete. 2024-09-15T21:25:46.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 10 monclient: build_initial_monmap 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.307+0000 7ff5683b0640 10 monclient: monmap: 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 Processor -- start 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- start start 2024-09-15T21:25:46.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: init 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5600a5210 0x7ff56013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): picked mon.noname-b con 0x7ff5600a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff56013d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): picked mon.noname-c con 0x7ff5601378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 0x7ff560136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): picked mon.noname-a con 0x7ff5601381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601442c0 con 0x7ff5601381e0 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff560144140 con 0x7ff5600a5210 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:46.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff560144440 con 0x7ff5601378c0 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient(hunting): authenticate will time out at 1469.708831s 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff56013d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 0x7ff560136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff56013d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:41168/0 (socket says 172.21.15.17:41168) 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 -- 172.21.15.17:0/2052227401 learned_addr learned my addr 172.21.15.17:0/2052227401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): get_auth_request con 0x7ff5601378c0 auth_method 0 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 10 monclient(hunting): get_auth_request con 0x7ff5601381e0 auth_method 0 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 1 -- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 msgr2=0x7ff560136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 1 -- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 msgr2=0x7ff560136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 0x7ff560136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56614c640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5600a5210 0x7ff56013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56694d640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 0x7ff560136a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56614c640 10 monclient(hunting): get_auth_request con 0x7ff5600a5210 auth_method 0 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56614c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56614c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:46.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56614c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient(hunting): handle_auth_done global_id 45254 payload 824 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 -- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5600a5210 msgr2=0x7ff56013f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5600a5210 0x7ff56013f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 -- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 msgr2=0x7ff560136a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 0x7ff560136a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 monclient: found mon.noname-c 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 -- 172.21.15.17:0/2052227401 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5600a89d0 con 0x7ff5601378c0 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56594b640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff56013d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c400 tx=0x7ff55000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550017070 con 0x7ff5601378c0 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: dump: 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:46.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff550005d30 con 0x7ff5601378c0 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 5 monclient: authenticate success, global_id 45254 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550007310 con 0x7ff5601378c0 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 10 monclient: dump: 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:46.329 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff564949640 4 set_mon_vals no callback set 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _renew_subs 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5600a12f0 con 0x7ff5601378c0 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _renew_subs 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff560140770 con 0x7ff5601378c0 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:46.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff560137ca0 con 0x7ff5601378c0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff55001a070 con 0x7ff5601378c0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff54403b9c0 0x7ff54403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(249..249 src has 1..249) ==== 8621+0+0 (secure 0 0 0) 0x7ff55004fe10 con 0x7ff5601378c0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.311+0000 7ff56614c640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff54403b9c0 0x7ff54403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.315+0000 7ff56614c640 10 monclient: get_auth_request con 0x7ff54403b9c0 auth_method 0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.315+0000 7ff56614c640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff54403b9c0 0x7ff54403de80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ae000 tx=0x7ff55c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:46.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.319+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff550053030 con 0x7ff5601378c0 2024-09-15T21:25:46.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.319+0000 7ff556ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:46.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.319+0000 7ff556ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:46.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.401+0000 7fa4b986f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T21:25:46.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.401+0000 7fa4b986f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T21:25:46.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.401+0000 7fa4b986f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:46.406 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.401+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) -- 0x7fa4b409ea20 con 0x7fa4b40a0160 2024-09-15T21:25:46.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff5683b0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:46.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff5683b0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:46.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff5683b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:46.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff560069270 con 0x7ff5601378c0 2024-09-15T21:25:46.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff556ffd640 1 -- 172.21.15.17:0/2052227401 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7ff5600a89d0 con 0x7ff5601378c0 2024-09-15T21:25:46.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff556ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:46.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.563+0000 7ff556ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:46.570 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:46.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff54403b9c0 msgr2=0x7ff54403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff54403b9c0 0x7ff54403de80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ae000 tx=0x7ff55c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:25:46.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 10 monclient: shutdown 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 msgr2=0x7ff56013d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff56013d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c400 tx=0x7ff55000c8c0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 shutdown_connections 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff54403b9c0 0x7ff54403de80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5601381e0 0x7ff560136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5601378c0 0x7ff56013d190 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 --2- 172.21.15.17:0/2052227401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5600a5210 0x7ff56013f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 >> 172.21.15.17:0/2052227401 conn(0x7ff5600aac40 msgr2=0x7ff56009da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 shutdown_connections 2024-09-15T21:25:46.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:46.567+0000 7ff5683b0640 1 -- 172.21.15.17:0/2052227401 wait complete. 2024-09-15T21:25:46.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.965+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 removed item id -109 name 'h31' from crush map v250) ==== 121+0+0 (secure 0 0 0) 0x7fa49c04d820 con 0x7fa4b40a0160 2024-09-15T21:25:46.969 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.965+0000 7fa4b0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T21:25:46.970 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:46.965+0000 7fa4b0ff9640 10 monclient: _finish_command 2 = system:0 removed item id -109 name 'h31' from crush map 2024-09-15T21:25:47.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b986f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T21:25:47.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b986f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T21:25:47.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b986f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) -- 0x7fa4b40008e0 con 0x7fa4b40a0160 2024-09-15T21:25:47.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b0ff9640 1 -- 172.21.15.119:0/1140646824 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 device 'h31' does not appear in the crush map v250) ==== 120+0+0 (secure 0 0 0) 0x7fa49c019d60 con 0x7fa4b40a0160 2024-09-15T21:25:47.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T21:25:47.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.073+0000 7fa4b0ff9640 10 monclient: _finish_command 3 = system:0 device 'h31' does not appear in the crush map 2024-09-15T21:25:47.078 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h31' does not appear in the crush map 2024-09-15T21:25:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa49403b970 msgr2=0x7fa49403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa49403b970 0x7fa49403de30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009bb0 tx=0x7fa4a4008040 comp rx=0 tx=0).stop 2024-09-15T21:25:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 10 monclient: shutdown 2024-09-15T21:25:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:47.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 msgr2=0x7fa4b4130d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 0x7fa4b4130d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00c4d0 tx=0x7fa49c00c990 comp rx=0 tx=0).stop 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 shutdown_connections 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa49403b970 0x7fa49403de30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4b4135ca0 0x7fa4b409d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b40a0160 0x7fa4b4130d20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 --2- 172.21.15.119:0/1140646824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa4b406ec30 0x7fa4b41307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 >> 172.21.15.119:0/1140646824 conn(0x7fa4b40a2ae0 msgr2=0x7fa4b4072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 shutdown_connections 2024-09-15T21:25:47.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.077+0000 7fa4b986f640 1 -- 172.21.15.119:0/1140646824 wait complete. 2024-09-15T21:25:47.095 INFO:tasks.workunit.client.0.smithi119.stderr:+ for f in `seq 1 32` 2024-09-15T21:25:47.095 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm h32 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.209+0000 7f05f48ef640 10 monclient: get_monmap_and_config 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.209+0000 7f05f48ef640 10 monclient: build_initial_monmap 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.209+0000 7f05f48ef640 10 monclient: monmap: 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:47.215713+0000 2024-09-15T21:25:47.214 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:47.215713+0000 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:47.215 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.209+0000 7f05f48ef640 1 Processor -- start 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.209+0000 7f05f48ef640 1 -- start start 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.209+0000 7f05f48ef640 10 monclient: init 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f05f009f640 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05f00a0130 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 0x7f05f013d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient(hunting): picked mon.noname-a con 0x7f05f009d9a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:47.225 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05f009ddc0 con 0x7f05f00a0130 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05f009e880 con 0x7f05f009f640 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f05f009eb60 con 0x7f05f009d9a0 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient(hunting): authenticate will time out at 1466.012322s 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f05f009f640 0x7f05f009fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38150/0 (socket says 172.21.15.119:38150) 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 -- 172.21.15.119:0/1273196786 learned_addr learned my addr 172.21.15.119:0/1273196786 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 --2- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 0x7f05f013d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999894588 0 0) 0x7f05f009e880 con 0x7f05f009f640 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): _negotiate 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:47.226 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): my global_id is 84670 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05d8003530 con 0x7f05f009f640 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): get_auth_request con 0x7f05f009d9a0 auth_method 0 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2773640040 0 0) 0x7f05d8003530 con 0x7f05f009f640 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 -- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 msgr2=0x7f05f013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 msgr2=0x7f05f013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 -- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 msgr2=0x7f05f013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 --2- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 0x7f05f013d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 --2- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 0x7f05f013d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 --2- 172.21.15.119:0/1273196786 >> v2:172.21.15.17:3300/0 conn(0x7f05f009d9a0 0x7f05f013d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 >> v1:172.21.15.17:6789/0 conn(0x7f05f00a0130 legacy=0x7f05f0139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.227 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: _finish_hunting 0 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f009ee40 con 0x7f05f009f640 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: _finish_auth 0 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f05e0001700 con 0x7f05f009f640 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 5 monclient: authenticate success, global_id 84670 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:47.228 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: dump: 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:47.229 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 >> v1:172.21.15.119:6789/0 conn(0x7f05f009f640 legacy=0x7f05f009fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d8003a60 0x7f05d8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): picked mon.b con 0x7f05d8003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8004810 0x7f05d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): picked mon.a con 0x7f05d8004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d80053d0 0x7f05d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): picked mon.c con 0x7f05d80053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f05f009eb60 con 0x7f05d8004810 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05d800c260 con 0x7f05d8003a60 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f05d800c460 con 0x7f05d80053d0 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d8003a60 0x7f05d8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): get_auth_request con 0x7f05d8003a60 auth_method 0 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d80053d0 0x7f05d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05edd9b640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8004810 0x7f05d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.231 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): get_auth_request con 0x7f05d80053d0 auth_method 0 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient(hunting): handle_auth_done global_id 84670 payload 293 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 -- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d80053d0 msgr2=0x7f05d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d80053d0 0x7f05d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 -- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8004810 msgr2=0x7f05d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8004810 0x7f05d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient: _finish_hunting 0 2024-09-15T21:25:47.232 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 monclient: found mon.b 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05eed9d640 1 -- 172.21.15.119:0/1273196786 reap_dead start 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 -- 172.21.15.119:0/1273196786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d8003530 con 0x7f05d8003a60 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient: _finish_auth 0 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 10 monclient: _check_auth_tickets 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05edd9b640 10 monclient: get_auth_request con 0x7f05d8004810 auth_method 0 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05edd9b640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8004810 0x7f05d8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ee59c640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d8003a60 0x7f05d8003e80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f05e0001060 tx=0x7f05e001cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3223888eafed30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0007d20 con 0x7f05d8003a60 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:47.233 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: dump: 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e00a9020 con 0x7f05d8003a60 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:47.234 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 1 -- 172.21.15.119:0/1273196786 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0005b20 con 0x7f05d8003a60 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05f48ef640 4 set_mon_vals no callback set 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.213+0000 7f05ed59a640 10 monclient: dump: 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:47.235 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: shutdown 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/1273196786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d8003a60 msgr2=0x7f05d8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d8003a60 0x7f05d8003e80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f05e0001060 tx=0x7f05e001cb40 comp rx=0 tx=0).stop 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/1273196786 shutdown_connections 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 --2- 172.21.15.119:0/1273196786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d8003a60 0x7f05d8003e80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/1273196786 >> 172.21.15.119:0/1273196786 conn(0x7f05f00a2ae0 msgr2=0x7f05f00a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/1273196786 shutdown_connections 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/1273196786 wait complete. 2024-09-15T21:25:47.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: build_initial_monmap 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: monmap: 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 Processor -- start 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- start start 2024-09-15T21:25:47.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: init 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d8003a60 0x7f05f01cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): picked mon.noname-c con 0x7f05d8003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05f0132db0 0x7f05f0133230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): picked mon.noname-a con 0x7f05f0132db0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 0x7f05f0124e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): picked mon.noname-b con 0x7f05f01249d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f05f00a39e0 con 0x7f05f0132db0 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05f00a3860 con 0x7f05f01249d0 2024-09-15T21:25:47.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f05f0134fc0 con 0x7f05d8003a60 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient(hunting): authenticate will time out at 1466.018024s 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 0x7f05f0124e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 0x7f05f0124e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37720/0 (socket says 172.21.15.119:37720) 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 -- 172.21.15.119:0/2940186310 learned_addr learned my addr 172.21.15.119:0/2940186310 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): get_auth_request con 0x7f05f01249d0 auth_method 0 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:47.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05ee59c640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d8003a60 0x7f05f01cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05ee59c640 10 monclient(hunting): get_auth_request con 0x7f05d8003a60 auth_method 0 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05ee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05edd9b640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05f0132db0 0x7f05f0133230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient(hunting): handle_auth_done global_id 84673 payload 824 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 -- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d8003a60 msgr2=0x7f05f01cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d8003a60 0x7f05f01cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 -- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05f0132db0 msgr2=0x7f05f0133230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05f0132db0 0x7f05f0133230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:47.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 monclient: found mon.noname-b 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 -- 172.21.15.119:0/2940186310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f0133b40 con 0x7f05f01249d0 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05eed9d640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 0x7f05f0124e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009610 tx=0x7f05e8007700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8014410 con 0x7f05f01249d0 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: dump: 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.241 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 5 monclient: authenticate success, global_id 84673 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e8019070 con 0x7f05f01249d0 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _renew_subs 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01287f0 con 0x7f05f01249d0 2024-09-15T21:25:47.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e80147f0 con 0x7f05f01249d0 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 10 monclient: dump: 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:47.243 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _renew_subs 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05ecd99640 4 set_mon_vals no callback set 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01274b0 con 0x7f05f01249d0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f05e8014990 con 0x7f05f01249d0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.217+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f0078140 con 0x7f05f01249d0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.221+0000 7f05deffd640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05d003b9c0 0x7f05d003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.221+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 8408+0+0 (secure 0 0 0) 0x7f05e8050ee0 con 0x7f05f01249d0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.221+0000 7f05ee59c640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05d003b9c0 0x7f05d003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.221+0000 7f05ee59c640 10 monclient: get_auth_request con 0x7f05d003b9c0 auth_method 0 2024-09-15T21:25:47.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.221+0000 7f05ee59c640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05d003b9c0 0x7f05d003de80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f05e00aa000 tx=0x7f05e0000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:47.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.225+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f05f0133b40 con 0x7f05f01249d0 2024-09-15T21:25:47.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.225+0000 7f05deffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:47.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.225+0000 7f05deffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:47.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.397+0000 7f05f48ef640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T21:25:47.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.397+0000 7f05f48ef640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T21:25:47.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.397+0000 7f05f48ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.400 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.397+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) -- 0x7f05f01345c0 con 0x7f05f01249d0 2024-09-15T21:25:47.803 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.801+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 removed item id -112 name 'h32' from crush map v251) ==== 121+0+0 (secure 0 0 0) 0x7f05e801b310 con 0x7f05f01249d0 2024-09-15T21:25:47.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.801+0000 7f05deffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T21:25:47.804 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.801+0000 7f05deffd640 10 monclient: _finish_command 2 = system:0 removed item id -112 name 'h32' from crush map 2024-09-15T21:25:47.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05f48ef640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T21:25:47.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05f48ef640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T21:25:47.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05f48ef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:47.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) -- 0x7f05f01352f0 con 0x7f05f01249d0 2024-09-15T21:25:47.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05deffd640 1 -- 172.21.15.119:0/2940186310 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 device 'h32' does not appear in the crush map v251) ==== 120+0+0 (secure 0 0 0) 0x7f05e804eac0 con 0x7f05f01249d0 2024-09-15T21:25:47.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05deffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T21:25:47.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.913+0000 7f05deffd640 10 monclient: _finish_command 3 = system:0 device 'h32' does not appear in the crush map 2024-09-15T21:25:47.917 INFO:tasks.workunit.client.0.smithi119.stderr:device 'h32' does not appear in the crush map 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05d003b9c0 msgr2=0x7f05d003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05d003b9c0 0x7f05d003de80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f05e00aa000 tx=0x7f05e0000ff0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 10 monclient: shutdown 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 msgr2=0x7f05f0124e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 0x7f05f0124e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009610 tx=0x7f05e8007700 comp rx=0 tx=0).stop 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 shutdown_connections 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f05d003b9c0 0x7f05d003de80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f01249d0 0x7f05f0124e10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05f0132db0 0x7f05f0133230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 --2- 172.21.15.119:0/2940186310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f05d8003a60 0x7f05f01cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:47.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 >> 172.21.15.119:0/2940186310 conn(0x7f05f00a2ae0 msgr2=0x7f05f00a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:47.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 shutdown_connections 2024-09-15T21:25:47.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:47.917+0000 7f05f48ef640 1 -- 172.21.15.119:0/2940186310 wait complete. 2024-09-15T21:25:47.934 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush rm r1 2024-09-15T21:25:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient: get_monmap_and_config 2024-09-15T21:25:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient: build_initial_monmap 2024-09-15T21:25:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient: monmap: 2024-09-15T21:25:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:48.053916+0000 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:48.053916+0000 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:48.053 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 Processor -- start 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 -- start start 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient: init 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e700a4ed0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e700a59c0 0x7f5e7013ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): picked mon.noname-a con 0x7f5e700a59c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e700a5dd0 0x7f5e7013f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): picked mon.noname-c con 0x7f5e700a5dd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e700a6db0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5e700a1210 con 0x7f5e700a59c0 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e700a1b00 con 0x7f5e700a5dd0 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e7691b640 10 monclient(hunting): authenticate will time out at 1466.851102s 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e6ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5e700a4ed0 0x7f5e700a52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38152/0 (socket says 172.21.15.119:38152) 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e6ffff640 1 -- 172.21.15.119:0/3186106757 learned_addr learned my addr 172.21.15.119:0/3186106757 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 1 --2- 172.21.15.119:0/3186106757 >> v2:172.21.15.119:3300/0 conn(0x7f5e700a5dd0 0x7f5e7013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): get_auth_request con 0x7f5e700a5dd0 auth_method 0 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient(hunting): handle_auth_done global_id 84676 payload 274 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e67fff640 1 --2- 172.21.15.119:0/3186106757 >> v2:172.21.15.17:3300/0 conn(0x7f5e700a59c0 0x7f5e7013ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 1 -- 172.21.15.119:0/3186106757 >> v2:172.21.15.17:3300/0 conn(0x7f5e700a59c0 msgr2=0x7f5e7013ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 1 --2- 172.21.15.119:0/3186106757 >> v2:172.21.15.17:3300/0 conn(0x7f5e700a59c0 0x7f5e7013ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 1 -- 172.21.15.119:0/3186106757 >> v1:172.21.15.119:6789/0 conn(0x7f5e700a4ed0 legacy=0x7f5e700a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient: _finish_hunting 0 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 1 monclient: found mon.noname-c 2024-09-15T21:25:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e74eb8640 1 -- 172.21.15.119:0/3186106757 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e700a6ad0 con 0x7f5e700a5dd0 2024-09-15T21:25:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e6f7fe640 1 -- 172.21.15.119:0/3186106757 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223712493 0 0) 0x7f5e700a6db0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.049+0000 7f5e6f7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:48.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e74eb8640 1 --2- 172.21.15.119:0/3186106757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a5dd0 0x7f5e7013f420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e700a22b0 tx=0x7f5e6801c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47046e25cba656c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:48.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 1 -- 172.21.15.119:0/3186106757 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e680b2070 con 0x7f5e700a5dd0 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: dump: 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:48.067 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 1 -- 172.21.15.119:0/3186106757 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e680a5770 con 0x7f5e700a5dd0 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 5 monclient: authenticate success, global_id 84676 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 4 set_mon_vals no callback set 2024-09-15T21:25:48.068 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 1 -- 172.21.15.119:0/3186106757 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e680a5b40 con 0x7f5e700a5dd0 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e6f7fe640 10 monclient: dump: 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:48.069 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 10 monclient: shutdown 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 -- 172.21.15.119:0/3186106757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a5dd0 msgr2=0x7f5e7013f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 --2- 172.21.15.119:0/3186106757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a5dd0 0x7f5e7013f420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e700a22b0 tx=0x7f5e6801c790 comp rx=0 tx=0).stop 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 -- 172.21.15.119:0/3186106757 shutdown_connections 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 --2- 172.21.15.119:0/3186106757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a5dd0 0x7f5e7013f420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 --2- 172.21.15.119:0/3186106757 >> v2:172.21.15.17:3300/0 conn(0x7f5e700a59c0 0x7f5e7013ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 -- 172.21.15.119:0/3186106757 >> 172.21.15.119:0/3186106757 conn(0x7f5e700a6fd0 msgr2=0x7f5e700a73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 -- 172.21.15.119:0/3186106757 shutdown_connections 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 -- 172.21.15.119:0/3186106757 wait complete. 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 10 monclient: build_initial_monmap 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 10 monclient: monmap: 2024-09-15T21:25:48.070 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:48.071 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 Processor -- start 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 1 -- start start 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.053+0000 7f5e7691b640 10 monclient: init 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 0x7f5e70136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:48.072 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): picked mon.noname-b con 0x7f5e700a4ed0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e700a59c0 0x7f5e70136b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): picked mon.noname-a con 0x7f5e700a59c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e700a5dd0 0x7f5e7013c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): picked mon.noname-c con 0x7f5e700a5dd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70142180 con 0x7f5e700a59c0 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e700a7de0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:48.073 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e70142300 con 0x7f5e700a5dd0 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient(hunting): authenticate will time out at 1466.856149s 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 0x7f5e70136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 0x7f5e70136640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37744/0 (socket says 172.21.15.119:37744) 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 -- 172.21.15.119:0/4071772017 learned_addr learned my addr 172.21.15.119:0/4071772017 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): get_auth_request con 0x7f5e700a4ed0 auth_method 0 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:48.074 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:48.076 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e74eb8640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e700a5dd0 0x7f5e7013c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient(hunting): handle_auth_done global_id 84682 payload 824 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 -- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e700a5dd0 msgr2=0x7f5e7013c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e700a5dd0 0x7f5e7013c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 -- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e700a59c0 msgr2=0x7f5e70136b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e700a59c0 0x7f5e70136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 monclient: found mon.noname-b 2024-09-15T21:25:48.077 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 -- 172.21.15.119:0/4071772017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e700a9b30 con 0x7f5e700a4ed0 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e74eb8640 10 monclient: get_auth_request con 0x7f5e700a5dd0 auth_method 0 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e74eb8640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e700a5dd0 0x7f5e7013c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6ffff640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 0x7f5e70136640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000c430 tx=0x7f5e6000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e600074e0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: dump: 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.078 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e600047f0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.079 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 5 monclient: authenticate success, global_id 84682 2024-09-15T21:25:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _renew_subs 2024-09-15T21:25:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7013de00 con 0x7f5e700a4ed0 2024-09-15T21:25:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60004be0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.080 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 10 monclient: dump: 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:48.081 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _renew_subs 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6effd640 4 set_mon_vals no callback set 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7013e6a0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.082 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e700a52b0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e60011070 con 0x7f5e700a4ed0 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e5003b9f0 0x7f5e5003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(251..251 src has 1..251) ==== 8195+0+0 (secure 0 0 0) 0x7f5e600504b0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e67fff640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e5003b9f0 0x7f5e5003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e67fff640 10 monclient: get_auth_request con 0x7f5e5003b9f0 auth_method 0 2024-09-15T21:25:48.083 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.057+0000 7f5e67fff640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e5003b9f0 0x7f5e5003deb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5e700a42d0 tx=0x7f5e58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:48.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.061+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5e6004e160 con 0x7f5e700a4ed0 2024-09-15T21:25:48.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.061+0000 7f5e6d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:48.084 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.061+0000 7f5e6d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:48.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.237+0000 7f5e7691b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T21:25:48.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.237+0000 7f5e7691b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T21:25:48.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.237+0000 7f5e7691b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:48.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.237+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) -- 0x7f5e700008e0 con 0x7f5e700a4ed0 2024-09-15T21:25:48.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.989+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 removed item id -16 name 'r1' from crush map v252) ==== 118+0+0 (secure 0 0 0) 0x7f5e60004d80 con 0x7f5e700a4ed0 2024-09-15T21:25:48.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.989+0000 7f5e6d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T21:25:48.993 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:48.989+0000 7f5e6d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'r1' from crush map 2024-09-15T21:25:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e7691b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T21:25:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e7691b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T21:25:49.097 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e7691b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) -- 0x7f5e701370c0 con 0x7f5e700a4ed0 2024-09-15T21:25:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e6d7fa640 1 -- 172.21.15.119:0/4071772017 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 device 'r1' does not appear in the crush map v252) ==== 118+0+0 (secure 0 0 0) 0x7f5e6001a840 con 0x7f5e700a4ed0 2024-09-15T21:25:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e6d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T21:25:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.093+0000 7f5e6d7fa640 10 monclient: _finish_command 3 = system:0 device 'r1' does not appear in the crush map 2024-09-15T21:25:49.098 INFO:tasks.workunit.client.0.smithi119.stderr:device 'r1' does not appear in the crush map 2024-09-15T21:25:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e5003b9f0 msgr2=0x7f5e5003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.099 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e5003b9f0 0x7f5e5003deb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5e700a42d0 tx=0x7f5e58008040 comp rx=0 tx=0).stop 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 10 monclient: shutdown 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 msgr2=0x7f5e70136640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 0x7f5e70136640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000c430 tx=0x7f5e6000c8f0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 shutdown_connections 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e5003b9f0 0x7f5e5003deb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e700a5dd0 0x7f5e7013c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e700a59c0 0x7f5e70136b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 --2- 172.21.15.119:0/4071772017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e700a4ed0 0x7f5e70136640 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 >> 172.21.15.119:0/4071772017 conn(0x7f5e700a6fd0 msgr2=0x7f5e700a83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 shutdown_connections 2024-09-15T21:25:49.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.097+0000 7f5e7691b640 1 -- 172.21.15.119:0/4071772017 wait complete. 2024-09-15T21:25:49.114 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T21:25:49.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.225+0000 7fefddf0e640 10 monclient: get_monmap_and_config 2024-09-15T21:25:49.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.225+0000 7fefddf0e640 10 monclient: build_initial_monmap 2024-09-15T21:25:49.230 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.225+0000 7fefddf0e640 10 monclient: monmap: 2024-09-15T21:25:49.230 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:49.230 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.230 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:49.231835+0000 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:49.231835+0000 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:49.231 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 1 Processor -- start 2024-09-15T21:25:49.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 1 -- start start 2024-09-15T21:25:49.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient: init 2024-09-15T21:25:49.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fefd80a3600 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fefd80a1270 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefd813a700 0x7fefd813caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient(hunting): picked mon.noname-a con 0x7fefd813a700 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd80a1700 con 0x7fefd80a1270 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd80a2240 con 0x7fefd80a3600 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fefd80a39e0 con 0x7fefd813a700 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 10 monclient(hunting): authenticate will time out at 1468.029059s 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fefd80a3600 0x7fefd80a1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38162/0 (socket says 172.21.15.119:38162) 2024-09-15T21:25:49.236 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 -- 172.21.15.119:0/23374060 learned_addr learned my addr 172.21.15.119:0/23374060 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2656213598 0 0) 0x7fefd80a2240 con 0x7fefd80a3600 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): _negotiate 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): my global_id is 84685 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc0003530 con 0x7fefd80a3600 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 1 --2- 172.21.15.119:0/23374060 >> v2:172.21.15.17:3300/0 conn(0x7fefd813a700 0x7fefd813caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 10 monclient(hunting): get_auth_request con 0x7fefd813a700 auth_method 0 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3449080425 0 0) 0x7fefc0003530 con 0x7fefd80a3600 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 >> v2:172.21.15.17:3300/0 conn(0x7fefd813a700 msgr2=0x7fefd813caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:49.237 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 --2- 172.21.15.119:0/23374060 >> v2:172.21.15.17:3300/0 conn(0x7fefd813a700 0x7fefd813caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 >> v1:172.21.15.17:6789/0 conn(0x7fefd80a1270 legacy=0x7fefd8138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd80a27a0 con 0x7fefd80a3600 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fefc8001700 con 0x7fefd80a3600 2024-09-15T21:25:49.238 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 5 monclient: authenticate success, global_id 84685 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: dump: 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.239 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 >> v1:172.21.15.119:6789/0 conn(0x7fefd80a3600 legacy=0x7fefd80a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefc0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): picked mon.b con 0x7fefc0003ab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc0004810 0x7fefc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): picked mon.c con 0x7fefc0004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc0005400 0x7fefc000ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): picked mon.a con 0x7fefc0005400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fefd80a39e0 con 0x7fefc0005400 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc000c350 con 0x7fefc0003ab0 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefc000c550 con 0x7fefc0004810 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd67fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefddf0e640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:49.241 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefc0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): get_auth_request con 0x7fefc0003ab0 auth_method 0 2024-09-15T21:25:49.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:49.242 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc0005400 0x7fefc000ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient(hunting): handle_auth_done global_id 84685 payload 293 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 -- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc0004810 msgr2=0x7fefc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefc0004810 0x7fefc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.243 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 -- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc0005400 msgr2=0x7fefc000ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:49.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc0005400 0x7fefc000ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 monclient: found mon.b 2024-09-15T21:25:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd7fff640 1 -- 172.21.15.119:0/23374060 reap_dead start 2024-09-15T21:25:49.245 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 1 -- 172.21.15.119:0/23374060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc0003530 con 0x7fefc0003ab0 2024-09-15T21:25:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.229+0000 7fefd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefc0003ed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c190 tx=0x7fefc801cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c44d9be07597ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:49.246 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8007cd0 con 0x7fefc0003ab0 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: dump: 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.247 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.248 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.248 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.248 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.248 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.248 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.248 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc80a9020 con 0x7fefc0003ab0 2024-09-15T21:25:49.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:49.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 1 -- 172.21.15.119:0/23374060 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8005b00 con 0x7fefc0003ab0 2024-09-15T21:25:49.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.249 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd67fc640 10 monclient: dump: 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.250 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:49.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 4 set_mon_vals no callback set 2024-09-15T21:25:49.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: shutdown 2024-09-15T21:25:49.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:49.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/23374060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 msgr2=0x7fefc0003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.256 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefc0003ed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c190 tx=0x7fefc801cb10 comp rx=0 tx=0).stop 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/23374060 shutdown_connections 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 --2- 172.21.15.119:0/23374060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefc0003ed0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/23374060 >> 172.21.15.119:0/23374060 conn(0x7fefd80a2ae0 msgr2=0x7fefd80a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/23374060 shutdown_connections 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/23374060 wait complete. 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: build_initial_monmap 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: monmap: 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:49.257 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 Processor -- start 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- start start 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: init 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefd81cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): picked mon.noname-b con 0x7fefc0003ab0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd81d0010 0x7fefd81d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): picked mon.noname-a con 0x7fefd81d0010 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:49.258 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefd81cc8c0 0x7fefd81ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): picked mon.noname-c con 0x7fefd81cc8c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fefd80a1f40 con 0x7fefd81d0010 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefd809e5d0 con 0x7fefc0003ab0 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fefd80a20c0 con 0x7fefd81cc8c0 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient(hunting): authenticate will time out at 1468.034687s 2024-09-15T21:25:49.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefd81cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefd81cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37762/0 (socket says 172.21.15.119:37762) 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 -- 172.21.15.119:0/2500924236 learned_addr learned my addr 172.21.15.119:0/2500924236 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): get_auth_request con 0x7fefc0003ab0 auth_method 0 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:49.260 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd6ffd640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd81d0010 0x7fefd81d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient(hunting): handle_auth_done global_id 84688 payload 824 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd7fff640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefd81cc8c0 0x7fefd81ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 -- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefd81cc8c0 msgr2=0x7fefd81ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefd81cc8c0 0x7fefd81ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 -- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd81d0010 msgr2=0x7fefd81d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd81d0010 0x7fefd81d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 -- 172.21.15.119:0/2500924236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd8123100 con 0x7fefc0003ab0 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd6ffd640 10 monclient: get_auth_request con 0x7fefd81d0010 auth_method 0 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd6ffd640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd81d0010 0x7fefd81d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd77fe640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefd81cfad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fefc80ab040 tx=0x7fefc0005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8007cd0 con 0x7fefc0003ab0 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: dump: 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: _finish_auth 0 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 5 monclient: authenticate success, global_id 84688 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc80b2070 con 0x7fefc0003ab0 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:49.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: _renew_subs 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8001c10 con 0x7fefc0003ab0 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd81233e0 con 0x7fefc0003ab0 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefbffff640 10 monclient: dump: 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:49.264 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefd5ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: _renew_subs 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.233+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd8122e20 con 0x7fefc0003ab0 2024-09-15T21:25:49.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefddf0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefddf0e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefddf0e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd8133e70 con 0x7fefc0003ab0 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fefc8004a30 con 0x7fefc0003ab0 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefbffff640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7985+0+0 (secure 0 0 0) 0x7fefc80e9aa0 con 0x7fefc0003ab0 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefd6ffd640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefd6ffd640 10 monclient: get_auth_request con 0x7fefc403b9c0 auth_method 0 2024-09-15T21:25:49.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.237+0000 7fefd6ffd640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fefd81229a0 tx=0x7fefcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:49.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.241+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fefc80168f0 con 0x7fefc0003ab0 2024-09-15T21:25:49.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.241+0000 7fefbffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:49.267 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.241+0000 7fefbffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:49.423 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.421+0000 7fefddf0e640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:25:49.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.421+0000 7fefddf0e640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:25:49.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.421+0000 7fefddf0e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:49.424 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.421+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7fefd813a020 con 0x7fefc0003ab0 2024-09-15T21:25:49.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.715+0000 7f2ac1b31640 10 monclient: get_monmap_and_config 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.715+0000 7f2ac1b31640 10 monclient: build_initial_monmap 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.715+0000 7f2ac1b31640 10 monclient: monmap: 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:49.720661+0000 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:49.720661+0000 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.715+0000 7f2ac1b31640 1 Processor -- start 2024-09-15T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.715+0000 7f2ac1b31640 1 -- start start 2024-09-15T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.715+0000 7f2ac1b31640 10 monclient: init 2024-09-15T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc09cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): picked mon.noname-b con 0x7f2abc0a2820 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2abc0a2c00 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2abc13cd10 0x7f2abc13f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): picked mon.noname-c con 0x7f2abc13cd10 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2abc09d260 con 0x7f2abc0a2c00 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2abc09c060 con 0x7f2abc13cd10 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2abc09c340 con 0x7f2abc0a2820 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient(hunting): authenticate will time out at 1473.116255s 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2abc0a2c00 0x7f2abc0a3060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48386/0 (socket says 172.21.15.17:48386) 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abaffd640 1 -- 172.21.15.17:0/4257003025 learned_addr learned my addr 172.21.15.17:0/4257003025 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): get_auth_request con 0x7f2abc0a2820 auth_method 0 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abbfff640 1 --2- 172.21.15.17:0/4257003025 >> v2:172.21.15.119:3300/0 conn(0x7f2abc13cd10 0x7f2abc13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abbfff640 10 monclient(hunting): get_auth_request con 0x7f2abc13cd10 auth_method 0 2024-09-15T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_done global_id 45260 payload 274 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4257003025 >> v2:172.21.15.119:3300/0 conn(0x7f2abc13cd10 msgr2=0x7f2abc13f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 --2- 172.21.15.17:0/4257003025 >> v2:172.21.15.119:3300/0 conn(0x7f2abc13cd10 0x7f2abc13f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4257003025 >> v1:172.21.15.17:6789/0 conn(0x7f2abc0a2c00 legacy=0x7f2abc0a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4257003025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc09c620 con 0x7f2abc0a2820 2024-09-15T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2abb7fe640 1 --2- 172.21.15.17:0/4257003025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc09cd20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00097c0 tx=0x7f2ab001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8654e8ed47bef0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 1 -- 172.21.15.17:0/4257003025 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00b2070 con 0x7f2abc0a2820 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: dump: 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 5 monclient: authenticate success, global_id 45260 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 1 -- 172.21.15.17:0/4257003025 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab00ad680 con 0x7f2abc0a2820 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 4 set_mon_vals no callback set 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 1 -- 172.21.15.17:0/4257003025 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00ada50 con 0x7f2abc0a2820 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2aba7fc640 10 monclient: dump: 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 10 monclient: shutdown 2024-09-15T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4257003025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 msgr2=0x7f2abc09cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4257003025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc09cd20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00097c0 tx=0x7f2ab001c360 comp rx=0 tx=0).stop 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4257003025 shutdown_connections 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4257003025 >> v2:172.21.15.119:3300/0 conn(0x7f2abc13cd10 0x7f2abc13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4257003025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc09cd20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.719+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4257003025 >> 172.21.15.17:0/4257003025 conn(0x7f2abc0a74c0 msgr2=0x7f2abc0a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4257003025 shutdown_connections 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4257003025 wait complete. 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: build_initial_monmap 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: monmap: 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 Processor -- start 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- start start 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: init 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc120be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): picked mon.noname-c con 0x7f2abc0a2820 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2abc0a2c00 0x7f2abc137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): picked mon.noname-b con 0x7f2abc0a2c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 0x7f2abc1cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): picked mon.noname-a con 0x7f2abc13cd10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1422a0 con 0x7f2abc13cd10 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc142120 con 0x7f2abc0a2c00 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2abc142420 con 0x7f2abc0a2820 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient(hunting): authenticate will time out at 1473.121633s 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc120be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc120be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45538/0 (socket says 172.21.15.17:45538) 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4201071278 learned_addr learned my addr 172.21.15.17:0/4201071278 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): get_auth_request con 0x7f2abc0a2820 auth_method 0 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 0x7f2abc1cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 10 monclient(hunting): get_auth_request con 0x7f2abc13cd10 auth_method 0 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 1 -- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 msgr2=0x7f2abc1cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 1 -- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 msgr2=0x7f2abc1cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 0x7f2abc1cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abaffd640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2abc0a2c00 0x7f2abc137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abbfff640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 0x7f2abc1cf7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abaffd640 10 monclient(hunting): get_auth_request con 0x7f2abc0a2c00 auth_method 0 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient(hunting): handle_auth_done global_id 45263 payload 824 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2abc0a2c00 msgr2=0x7f2abc137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2abc0a2c00 0x7f2abc137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 msgr2=0x7f2abc1cf7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 0x7f2abc1cf7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 -- 172.21.15.17:0/4201071278 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc13f0f0 con 0x7f2abc0a2820 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2abb7fe640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc120be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00097c0 tx=0x7f2ab001aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00bc020 con 0x7f2abc0a2820 2024-09-15T21:25:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: dump: 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 5 monclient: authenticate success, global_id 45263 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab00adbb0 con 0x7f2abc0a2820 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _renew_subs 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc121b40 con 0x7f2abc0a2820 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00b2040 con 0x7f2abc0a2820 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 10 monclient: dump: 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _renew_subs 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc122100 con 0x7f2abc0a2820 2024-09-15T21:25:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ab9ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc09e160 con 0x7f2abc0a2820 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2ab00bf020 con 0x7f2abc0a2820 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.723+0000 7f2aa3fff640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2aa803b9f0 0x7f2aa803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.727+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(252..252 src has 1..252) ==== 7985+0+0 (secure 0 0 0) 0x7f2ab00ea6d0 con 0x7f2abc0a2820 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.727+0000 7f2abaffd640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2aa803b9f0 0x7f2aa803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.727+0000 7f2abaffd640 10 monclient: get_auth_request con 0x7f2aa803b9f0 auth_method 0 2024-09-15T21:25:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.727+0000 7f2abaffd640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2aa803b9f0 0x7f2aa803deb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0a1130 tx=0x7f2aac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.731+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2ab00c6070 con 0x7f2abc0a2820 2024-09-15T21:25:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.731+0000 7f2aa3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.731+0000 7f2aa3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:49.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2ac1b31640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:49.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2ac1b31640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:49.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2ac1b31640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:49.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2abc09cd20 con 0x7f2abc0a2820 2024-09-15T21:25:49.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2aa3fff640 1 -- 172.21.15.17:0/4201071278 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f2ab0016070 con 0x7f2abc0a2820 2024-09-15T21:25:49.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2aa3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:49.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.963+0000 7f2aa3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:49.969 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2aa803b9f0 msgr2=0x7f2aa803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2aa803b9f0 0x7f2aa803deb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0a1130 tx=0x7f2aac009040 comp rx=0 tx=0).stop 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 10 monclient: shutdown 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 msgr2=0x7f2abc120be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc120be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00097c0 tx=0x7f2ab001aba0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 shutdown_connections 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2aa803b9f0 0x7f2aa803deb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2abc13cd10 0x7f2abc1cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2abc0a2c00 0x7f2abc137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 --2- 172.21.15.17:0/4201071278 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2abc0a2820 0x7f2abc120be0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 >> 172.21.15.17:0/4201071278 conn(0x7f2abc0a74c0 msgr2=0x7f2abc0a0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 shutdown_connections 2024-09-15T21:25:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:49.967+0000 7f2ac1b31640 1 -- 172.21.15.17:0/4201071278 wait complete. 2024-09-15T21:25:50.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.997+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v253) ==== 78+0+0 (secure 0 0 0) 0x7fefc8004d20 con 0x7fefc0003ab0 2024-09-15T21:25:50.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.997+0000 7fefbffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:25:50.001 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:49.997+0000 7fefbffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:50.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:50.109+0000 7fefddf0e640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:25:50.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:50.109+0000 7fefddf0e640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:25:50.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:50.109+0000 7fefddf0e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:50.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:50.109+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7fefd813a020 con 0x7fefc0003ab0 2024-09-15T21:25:51.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefbffff640 1 -- 172.21.15.119:0/2500924236 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v254) ==== 78+0+0 (secure 0 0 0) 0x7fefc8007590 con 0x7fefc0003ab0 2024-09-15T21:25:51.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefbffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T21:25:51.004 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefbffff640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:25:51.007 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 msgr2=0x7fefc403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:51.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fefd81229a0 tx=0x7fefcc009040 comp rx=0 tx=0).stop 2024-09-15T21:25:51.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 10 monclient: shutdown 2024-09-15T21:25:51.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:51.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 msgr2=0x7fefd81cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:51.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefd81cfad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fefc80ab040 tx=0x7fefc0005840 comp rx=0 tx=0).stop 2024-09-15T21:25:51.008 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 shutdown_connections 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fefc403b9c0 0x7fefc403de80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fefd81cc8c0 0x7fefd81ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd81d0010 0x7fefd81d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 --2- 172.21.15.119:0/2500924236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc0003ab0 0x7fefd81cfad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 >> 172.21.15.119:0/2500924236 conn(0x7fefd80a2ae0 msgr2=0x7fefd80a3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 shutdown_connections 2024-09-15T21:25:51.009 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.001+0000 7fefddf0e640 1 -- 172.21.15.119:0/2500924236 wait complete. 2024-09-15T21:25:51.026 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo OK 2024-09-15T21:25:51.027 INFO:tasks.workunit.client.0.smithi119.stdout:OK 2024-09-15T21:25:51.028 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:25:51.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:25:51.084 INFO:tasks.workunit:Running workunits matching mon/osd.sh on client.0... 2024-09-15T21:25:51.087 INFO:tasks.workunit:Running workunit mon/osd.sh... 2024-09-15T21:25:51.087 DEBUG:teuthology.orchestra.run.smithi119:workunit test mon/osd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7498493279982dfe87f00616198a5967475f6169 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/osd.sh 2024-09-15T21:25:51.140 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -e 2024-09-15T21:25:51.140 INFO:tasks.workunit.client.0.smithi119.stderr:+ uuidgen 2024-09-15T21:25:51.140 INFO:tasks.workunit.client.0.smithi119.stderr:+ ua=79adef04-9b2e-463a-b4b9-a979dfe86845 2024-09-15T21:25:51.140 INFO:tasks.workunit.client.0.smithi119.stderr:+ uuidgen 2024-09-15T21:25:51.141 INFO:tasks.workunit.client.0.smithi119.stderr:+ ub=e2307bd0-cd40-470d-a650-9bfee3159ff3 2024-09-15T21:25:51.141 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd create 79adef04-9b2e-463a-b4b9-a979dfe86845 2024-09-15T21:25:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient: get_monmap_and_config 2024-09-15T21:25:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient: build_initial_monmap 2024-09-15T21:25:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient: monmap: 2024-09-15T21:25:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:51.259 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:51.261050+0000 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:51.261050+0000 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:51.260 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 Processor -- start 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 -- start start 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient: init 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8a0c0a4d80 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a0c0a5870 0x7f8a0c13cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): picked mon.noname-c con 0x7f8a0c0a5870 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c13d510 0x7f8a0c13f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): picked mon.noname-b con 0x7f8a0c13d510 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:51.263 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c0a2250 con 0x7f8a0c0a4d80 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0a4650 con 0x7f8a0c0a5870 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c06dab0 con 0x7f8a0c13d510 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a1464c640 10 monclient(hunting): authenticate will time out at 1470.057836s 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a0c0a5870 0x7f8a0c13cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a0c0a5870 0x7f8a0c13cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37778/0 (socket says 172.21.15.119:37778) 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 -- 172.21.15.119:0/2979188670 learned_addr learned my addr 172.21.15.119:0/2979188670 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): get_auth_request con 0x7f8a0c0a5870 auth_method 0 2024-09-15T21:25:51.264 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a12be9640 1 --2- 172.21.15.119:0/2979188670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c13d510 0x7f8a0c13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a12be9640 10 monclient(hunting): get_auth_request con 0x7f8a0c13d510 auth_method 0 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a12be9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a12be9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a12be9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient(hunting): handle_auth_done global_id 84697 payload 274 2024-09-15T21:25:51.265 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 -- 172.21.15.119:0/2979188670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c13d510 msgr2=0x7f8a0c13f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 --2- 172.21.15.119:0/2979188670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c13d510 0x7f8a0c13f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 -- 172.21.15.119:0/2979188670 >> v1:172.21.15.17:6789/0 conn(0x7f8a0c0a4d80 legacy=0x7f8a0c0a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient: _finish_hunting 0 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 monclient: found mon.noname-c 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:51.266 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.257+0000 7f8a11be7640 1 -- 172.21.15.119:0/2979188670 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c0a19b0 con 0x7f8a0c0a5870 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a11be7640 1 --2- 172.21.15.119:0/2979188670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c0a5870 0x7f8a0c13cfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098a0 tx=0x7f89fc01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea3a1f1972e1439 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 1 -- 172.21.15.119:0/2979188670 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0b1070 con 0x7f8a0c0a5870 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: dump: 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.272 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: _finish_auth 0 2024-09-15T21:25:51.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: _check_auth_tickets 2024-09-15T21:25:51.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:51.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 1 -- 172.21.15.119:0/2979188670 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc0ac640 con 0x7f8a0c0a5870 2024-09-15T21:25:51.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:51.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 1 -- 172.21.15.119:0/2979188670 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0aca10 con 0x7f8a0c0a5870 2024-09-15T21:25:51.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 5 monclient: authenticate success, global_id 84697 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 4 set_mon_vals no callback set 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a113e6640 10 monclient: dump: 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.275 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: shutdown 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- 172.21.15.119:0/2979188670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c0a5870 msgr2=0x7f8a0c13cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 --2- 172.21.15.119:0/2979188670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c0a5870 0x7f8a0c13cfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098a0 tx=0x7f89fc01c400 comp rx=0 tx=0).stop 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- 172.21.15.119:0/2979188670 shutdown_connections 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 --2- 172.21.15.119:0/2979188670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c13d510 0x7f8a0c13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 --2- 172.21.15.119:0/2979188670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c0a5870 0x7f8a0c13cfd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- 172.21.15.119:0/2979188670 >> 172.21.15.119:0/2979188670 conn(0x7f8a0c0a75c0 msgr2=0x7f8a0c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- 172.21.15.119:0/2979188670 shutdown_connections 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- 172.21.15.119:0/2979188670 wait complete. 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: build_initial_monmap 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: monmap: 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:51.277 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 Processor -- start 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- start start 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: init 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a4d80 0x7f8a0c132090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): picked mon.noname-a con 0x7f8a0c0a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c0a5870 0x7f8a0c139e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:51.278 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): picked mon.noname-c con 0x7f8a0c0a5870 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 0x7f8a0c13a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): picked mon.noname-b con 0x7f8a0c13d510 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0a2de0 con 0x7f8a0c0a4d80 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0a2c60 con 0x7f8a0c13d510 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c142660 con 0x7f8a0c0a5870 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a1464c640 10 monclient(hunting): authenticate will time out at 1470.063277s 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 0x7f8a0c13a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 0x7f8a0c13a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37784/0 (socket says 172.21.15.119:37784) 2024-09-15T21:25:51.279 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 -- 172.21.15.119:0/1210072635 learned_addr learned my addr 172.21.15.119:0/1210072635 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): get_auth_request con 0x7f8a0c13d510 auth_method 0 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:51.280 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a11be7640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c0a5870 0x7f8a0c139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a11be7640 10 monclient(hunting): get_auth_request con 0x7f8a0c0a5870 auth_method 0 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a11be7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a11be7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a11be7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a123e8640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a4d80 0x7f8a0c132090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a123e8640 10 monclient(hunting): get_auth_request con 0x7f8a0c0a4d80 auth_method 0 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a123e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a123e8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:51.281 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a123e8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient(hunting): handle_auth_done global_id 84700 payload 824 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 -- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c0a5870 msgr2=0x7f8a0c139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c0a5870 0x7f8a0c139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 -- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a4d80 msgr2=0x7f8a0c132090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a4d80 0x7f8a0c132090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient: _finish_hunting 0 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 monclient: found mon.noname-b 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.261+0000 7f8a12be9640 1 -- 172.21.15.119:0/1210072635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c06f190 con 0x7f8a0c13d510 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a12be9640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 0x7f8a0c13a3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d800 tx=0x7f8a0800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800b6c0 con 0x7f8a0c13d510 2024-09-15T21:25:51.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: dump: 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:51.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a08019070 con 0x7f8a0c13d510 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800baa0 con 0x7f8a0c13d510 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 10 monclient: dump: 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:51.284 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a10be5640 4 set_mon_vals no callback set 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 5 monclient: authenticate success, global_id 84700 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 monclient: _renew_subs 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c06ec10 con 0x7f8a0c13d510 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 monclient: _renew_subs 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c135d00 con 0x7f8a0c13d510 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:51.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c0a5160 con 0x7f8a0c13d510 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8a0800bc40 con 0x7f8a0c13d510 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89f003b9c0 0x7f89f003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(254..254 src has 1..254) ==== 7657+0+0 (secure 0 0 0) 0x7f8a08050b30 con 0x7f8a0c13d510 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a123e8640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89f003b9c0 0x7f89f003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a123e8640 10 monclient: get_auth_request con 0x7f89f003b9c0 auth_method 0 2024-09-15T21:25:51.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.265+0000 7f8a123e8640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89f003b9c0 0x7f89f003de80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8a040012f0 tx=0x7f8a04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.269+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8a0801cd70 con 0x7f8a0c13d510 2024-09-15T21:25:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.269+0000 7f8a02ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:51.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.269+0000 7f8a02ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:51.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.449+0000 7f8a1464c640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:51.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.449+0000 7f8a1464c640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:51.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.449+0000 7f8a1464c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:51.455 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:51.449+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"} v 0) -- 0x7f8a0c0a4850 con 0x7f8a0c13d510 2024-09-15T21:25:52.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.013+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}]=0 v255) ==== 106+0+3 (secure 0 0 0) 0x7f8a08013cb0 con 0x7f8a0c13d510 2024-09-15T21:25:52.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.013+0000 7f8a02ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.017 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.013+0000 7f8a02ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:52.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.135 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"} v 0) -- 0x7f8a0c13c0a0 con 0x7f8a0c13d510 2024-09-15T21:25:52.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a02ffd640 1 -- 172.21.15.119:0/1210072635 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}]=0 v255) ==== 106+0+3 (secure 0 0 0) 0x7f8a08041070 con 0x7f8a0c13d510 2024-09-15T21:25:52.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a02ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.136 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a02ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:25:52.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89f003b9c0 msgr2=0x7f89f003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89f003b9c0 0x7f89f003de80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8a040012f0 tx=0x7f8a04009040 comp rx=0 tx=0).stop 2024-09-15T21:25:52.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 10 monclient: shutdown 2024-09-15T21:25:52.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:52.138 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 msgr2=0x7f8a0c13a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 0x7f8a0c13a3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d800 tx=0x7f8a0800dcc0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 shutdown_connections 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f89f003b9c0 0x7f89f003de80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c13d510 0x7f8a0c13a3b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a0c0a5870 0x7f8a0c139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 --2- 172.21.15.119:0/1210072635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a4d80 0x7f8a0c132090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 >> 172.21.15.119:0/1210072635 conn(0x7f8a0c0a75c0 msgr2=0x7f8a0c137420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 shutdown_connections 2024-09-15T21:25:52.139 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.133+0000 7f8a1464c640 1 -- 172.21.15.119:0/1210072635 wait complete. 2024-09-15T21:25:52.153 INFO:tasks.workunit.client.0.smithi119.stderr:+ na=12 2024-09-15T21:25:52.153 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd create 79adef04-9b2e-463a-b4b9-a979dfe86845 2024-09-15T21:25:52.268 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient: get_monmap_and_config 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient: build_initial_monmap 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient: monmap: 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:52.270420+0000 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:52.270420+0000 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:52.269 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:52.270 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:52.270 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 Processor -- start 2024-09-15T21:25:52.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 -- start start 2024-09-15T21:25:52.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient: init 2024-09-15T21:25:52.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:52.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2024-09-15T21:25:52.272 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8f400a4d80 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f400a5870 0x7f8f4013cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): picked mon.noname-a con 0x7f8f400a5870 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f4013d510 0x7f8f4013f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): picked mon.noname-c con 0x7f8f4013d510 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f400a2250 con 0x7f8f400a4d80 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8f4006dab0 con 0x7f8f400a5870 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8f400a19b0 con 0x7f8f4013d510 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:52.273 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.265+0000 7f8f46d66640 10 monclient(hunting): authenticate will time out at 1471.067868s 2024-09-15T21:25:52.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f4013d510 0x7f8f4013f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f4013d510 0x7f8f4013f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58192/0 (socket says 172.21.15.119:58192) 2024-09-15T21:25:52.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 -- 172.21.15.119:0/528235378 learned_addr learned my addr 172.21.15.119:0/528235378 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:52.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): get_auth_request con 0x7f8f4013d510 auth_method 0 2024-09-15T21:25:52.274 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3ffff640 1 --2- 172.21.15.119:0/528235378 >> v2:172.21.15.17:3300/0 conn(0x7f8f400a5870 0x7f8f4013cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3ffff640 10 monclient(hunting): get_auth_request con 0x7f8f400a5870 auth_method 0 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.275 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient(hunting): handle_auth_done global_id 84703 payload 274 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 -- 172.21.15.119:0/528235378 >> v2:172.21.15.17:3300/0 conn(0x7f8f400a5870 msgr2=0x7f8f4013cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 --2- 172.21.15.119:0/528235378 >> v2:172.21.15.17:3300/0 conn(0x7f8f400a5870 0x7f8f4013cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 -- 172.21.15.119:0/528235378 >> v1:172.21.15.17:6789/0 conn(0x7f8f400a4d80 legacy=0x7f8f400a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient: _finish_hunting 0 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 monclient: found mon.noname-c 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.276 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 -- 172.21.15.119:0/528235378 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f400a1c90 con 0x7f8f4013d510 2024-09-15T21:25:52.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f45303640 1 --2- 172.21.15.119:0/528235378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f4013d510 0x7f8f4013f900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009e50 tx=0x7f8f3801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df227298209c9ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:52.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 1 -- 172.21.15.119:0/528235378 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380a5750 con 0x7f8f4013d510 2024-09-15T21:25:52.282 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: dump: 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.283 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 5 monclient: authenticate success, global_id 84703 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 1 -- 172.21.15.119:0/528235378 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f380a58f0 con 0x7f8f4013d510 2024-09-15T21:25:52.284 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:52.285 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:52.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 4 set_mon_vals no callback set 2024-09-15T21:25:52.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 1 -- 172.21.15.119:0/528235378 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380a5cc0 con 0x7f8f4013d510 2024-09-15T21:25:52.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:52.286 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f3f7fe640 10 monclient: dump: 2024-09-15T21:25:52.286 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 10 monclient: shutdown 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 -- 172.21.15.119:0/528235378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f4013d510 msgr2=0x7f8f4013f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 --2- 172.21.15.119:0/528235378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f4013d510 0x7f8f4013f900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009e50 tx=0x7f8f3801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.287 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 -- 172.21.15.119:0/528235378 shutdown_connections 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 --2- 172.21.15.119:0/528235378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f4013d510 0x7f8f4013f900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 --2- 172.21.15.119:0/528235378 >> v2:172.21.15.17:3300/0 conn(0x7f8f400a5870 0x7f8f4013cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 -- 172.21.15.119:0/528235378 >> 172.21.15.119:0/528235378 conn(0x7f8f400a75c0 msgr2=0x7f8f4006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 -- 172.21.15.119:0/528235378 shutdown_connections 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.269+0000 7f8f46d66640 1 -- 172.21.15.119:0/528235378 wait complete. 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: build_initial_monmap 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: monmap: 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:52.288 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 Processor -- start 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- start start 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: init 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f400a4d80 0x7f8f40134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): picked mon.noname-a con 0x7f8f400a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 0x7f8f40135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): picked mon.noname-b con 0x7f8f400a5870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:52.289 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f4013d510 0x7f8f401cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): picked mon.noname-c con 0x7f8f4013d510 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400a2de0 con 0x7f8f400a4d80 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400a2c60 con 0x7f8f400a5870 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8f40142660 con 0x7f8f4013d510 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient(hunting): authenticate will time out at 1471.073017s 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 0x7f8f40135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 0x7f8f40135430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58206/0 (socket says 172.21.15.119:58206) 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 -- 172.21.15.119:0/2599482909 learned_addr learned my addr 172.21.15.119:0/2599482909 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:52.290 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): get_auth_request con 0x7f8f400a5870 auth_method 0 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f45303640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f4013d510 0x7f8f401cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f44b02640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f400a4d80 0x7f8f40134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f45303640 10 monclient(hunting): get_auth_request con 0x7f8f4013d510 auth_method 0 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f45303640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f45303640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f45303640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient(hunting): handle_auth_done global_id 84706 payload 824 2024-09-15T21:25:52.291 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 -- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f4013d510 msgr2=0x7f8f401cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f4013d510 0x7f8f401cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 -- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f400a4d80 msgr2=0x7f8f40134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f400a4d80 0x7f8f40134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 monclient: found mon.noname-b 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 -- 172.21.15.119:0/2599482909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f40072bc0 con 0x7f8f400a5870 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f44b02640 10 monclient: get_auth_request con 0x7f8f400a4d80 auth_method 0 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f44b02640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f400a4d80 0x7f8f40134ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3ffff640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 0x7f8f40135430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d800 tx=0x7f8f2c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00b6c0 con 0x7f8f400a5870 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.292 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: dump: 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:52.293 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c019070 con 0x7f8f400a5870 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 5 monclient: authenticate success, global_id 84706 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _renew_subs 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4013fa30 con 0x7f8f400a5870 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00bae0 con 0x7f8f400a5870 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 10 monclient: dump: 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.294 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3effd640 4 set_mon_vals no callback set 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _renew_subs 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f40071880 con 0x7f8f400a5870 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8f2c00bc80 con 0x7f8f400a5870 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:52.295 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f400a5160 con 0x7f8f400a5870 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f2403b9c0 0x7f8f2403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 7657+0+0 (secure 0 0 0) 0x7f8f2c050b30 con 0x7f8f400a5870 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f44b02640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f2403b9c0 0x7f8f2403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f44b02640 10 monclient: get_auth_request con 0x7f8f2403b9c0 auth_method 0 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.273+0000 7f8f44b02640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f2403b9c0 0x7f8f2403de80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f34003e40 tx=0x7f8f34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.277+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8f2c011d50 con 0x7f8f400a5870 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.277+0000 7f8f3d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:52.296 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.277+0000 7f8f3d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:52.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f46d66640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f46d66640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.459 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f46d66640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"} v 0) -- 0x7f8f400a4850 con 0x7f8f400a5870 2024-09-15T21:25:52.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}]=0 v255) ==== 106+0+3 (secure 0 0 0) 0x7f8f2c013cb0 con 0x7f8f400a5870 2024-09-15T21:25:52.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f3d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.460 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.457+0000 7f8f3d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:52.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.569+0000 7f8f46d66640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.569+0000 7f8f46d66640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.572 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.569+0000 7f8f46d66640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.573 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.569+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"} v 0) -- 0x7f8f400008e0 con 0x7f8f400a5870 2024-09-15T21:25:52.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f3d7fa640 1 -- 172.21.15.119:0/2599482909 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}]=0 v255) ==== 106+0+3 (secure 0 0 0) 0x7f8f2c041070 con 0x7f8f400a5870 2024-09-15T21:25:52.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f3d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:52.593 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f3d7fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f2403b9c0 msgr2=0x7f8f2403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f2403b9c0 0x7f8f2403de80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f34003e40 tx=0x7f8f34009040 comp rx=0 tx=0).stop 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 10 monclient: shutdown 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 msgr2=0x7f8f40135430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 0x7f8f40135430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d800 tx=0x7f8f2c00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 shutdown_connections 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8f2403b9c0 0x7f8f2403de80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8f4013d510 0x7f8f401cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f400a5870 0x7f8f40135430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.595 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 --2- 172.21.15.119:0/2599482909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f400a4d80 0x7f8f40134ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 >> 172.21.15.119:0/2599482909 conn(0x7f8f400a75c0 msgr2=0x7f8f40137440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:52.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 shutdown_connections 2024-09-15T21:25:52.596 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.589+0000 7f8f46d66640 1 -- 172.21.15.119:0/2599482909 wait complete. 2024-09-15T21:25:52.610 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 12 -eq 12 2024-09-15T21:25:52.610 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd create e2307bd0-cd40-470d-a650-9bfee3159ff3 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.721+0000 7fe86f7c8640 10 monclient: get_monmap_and_config 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.721+0000 7fe86f7c8640 10 monclient: build_initial_monmap 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.721+0000 7fe86f7c8640 10 monclient: monmap: 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:52.728475+0000 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:52.728475+0000 2024-09-15T21:25:52.727 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:52.728 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 Processor -- start 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 -- start start 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient: init 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe868135f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): picked mon.noname-b con 0x7fe868135b80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe8681364a0 0x7fe8680a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): picked mon.noname-c con 0x7fe8681364a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe8680a0030 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:52.731 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86809fdb0 con 0x7fe8680a0030 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe8680a1b60 con 0x7fe8681364a0 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680a1e40 con 0x7fe868135b80 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86f7c8640 10 monclient(hunting): authenticate will time out at 1471.525247s 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86dd65640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe8680a0030 0x7fe8680a0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60166/0 (socket says 172.21.15.119:60166) 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe8681364a0 0x7fe8680a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86dd65640 1 -- 172.21.15.119:0/338116886 learned_addr learned my addr 172.21.15.119:0/338116886 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:52.732 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): get_auth_request con 0x7fe8681364a0 auth_method 0 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86d564640 1 --2- 172.21.15.119:0/338116886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe868135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86d564640 10 monclient(hunting): get_auth_request con 0x7fe868135b80 auth_method 0 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86d564640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.733 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86d564640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86d564640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe85ffff640 1 -- 172.21.15.119:0/338116886 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569699539 0 0) 0x7fe86809fdb0 con 0x7fe8680a0030 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe85ffff640 10 monclient(hunting): _negotiate 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe85ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe85ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe85ffff640 10 monclient(hunting): my global_id is 84712 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe85ffff640 1 -- 172.21.15.119:0/338116886 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe854003530 con 0x7fe8680a0030 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient(hunting): handle_auth_done global_id 84709 payload 274 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 1 -- 172.21.15.119:0/338116886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 msgr2=0x7fe868135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 1 --2- 172.21.15.119:0/338116886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe868135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.734 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 1 -- 172.21.15.119:0/338116886 >> v1:172.21.15.119:6789/0 conn(0x7fe8680a0030 legacy=0x7fe8680a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient: _finish_hunting 0 2024-09-15T21:25:52.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 1 monclient: found mon.noname-c 2024-09-15T21:25:52.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:52.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.735 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.725+0000 7fe86cd63640 1 -- 172.21.15.119:0/338116886 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8680a2120 con 0x7fe8681364a0 2024-09-15T21:25:52.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86cd63640 1 --2- 172.21.15.119:0/338116886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8681364a0 0x7fe8680a09e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe8580097c0 tx=0x7fe85801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d23c40599e3968 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:52.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 1 -- 172.21.15.119:0/338116886 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8580b0070 con 0x7fe8681364a0 2024-09-15T21:25:52.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.740 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: dump: 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: _finish_auth 0 2024-09-15T21:25:52.741 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 1 -- 172.21.15.119:0/338116886 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8580ab600 con 0x7fe8681364a0 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 1 -- 172.21.15.119:0/338116886 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8580ab9d0 con 0x7fe8681364a0 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85ffff640 10 monclient: dump: 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.742 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 5 monclient: authenticate success, global_id 84709 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 4 set_mon_vals no callback set 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: shutdown 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/338116886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8681364a0 msgr2=0x7fe8680a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/338116886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8681364a0 0x7fe8680a09e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe8580097c0 tx=0x7fe85801c3f0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.743 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/338116886 shutdown_connections 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/338116886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8681364a0 0x7fe8680a09e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/338116886 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe868135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/338116886 >> 172.21.15.119:0/338116886 conn(0x7fe86806cdc0 msgr2=0x7fe86806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/338116886 shutdown_connections 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/338116886 wait complete. 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: build_initial_monmap 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: monmap: 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:52.744 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 Processor -- start 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- start start 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: init 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:52.745 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 0x7fe86809d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): picked mon.noname-b con 0x7fe8680a0030 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe86809da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): picked mon.noname-c con 0x7fe868135b80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8681364a0 0x7fe86809df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): picked mon.noname-a con 0x7fe8681364a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe868140320 con 0x7fe8681364a0 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680a17d0 con 0x7fe8680a0030 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8681404a0 con 0x7fe868135b80 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:52.746 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient(hunting): authenticate will time out at 1471.530971s 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 0x7fe86809d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 0x7fe86809d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58230/0 (socket says 172.21.15.119:58230) 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 -- 172.21.15.119:0/3833609254 learned_addr learned my addr 172.21.15.119:0/3833609254 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): get_auth_request con 0x7fe8680a0030 auth_method 0 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient(hunting): handle_auth_done global_id 84715 payload 824 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86cd63640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe86809da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.747 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 -- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 msgr2=0x7fe86809da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe86809da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 -- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8681364a0 msgr2=0x7fe86809df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8681364a0 0x7fe86809df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient: _finish_hunting 0 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 monclient: found mon.noname-b 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 -- 172.21.15.119:0/3833609254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe868072330 con 0x7fe8680a0030 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86d564640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 0x7fe86809d4e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d800 tx=0x7fe86400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400b6c0 con 0x7fe8680a0030 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.748 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: dump: 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:52.749 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe864019070 con 0x7fe8680a0030 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400ba60 con 0x7fe8680a0030 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85dffb640 10 monclient: dump: 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:52.750 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 5 monclient: authenticate success, global_id 84715 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: _renew_subs 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe868120e80 con 0x7fe8680a0030 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe85f7fe640 4 set_mon_vals no callback set 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: _renew_subs 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.729+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86813d510 con 0x7fe8680a0030 2024-09-15T21:25:52.751 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86f7c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86f7c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86f7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe868135f60 con 0x7fe8680a0030 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe86400bc00 con 0x7fe8680a0030 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe85dffb640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe84c03b9c0 0x7fe84c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 7657+0+0 (secure 0 0 0) 0x7fe864050b20 con 0x7fe8680a0030 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86cd63640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe84c03b9c0 0x7fe84c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86cd63640 10 monclient: get_auth_request con 0x7fe84c03b9c0 auth_method 0 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.733+0000 7fe86cd63640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe84c03b9c0 0x7fe84c03de80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe8580ad000 tx=0x7fe8580a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.737+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe868072330 con 0x7fe8680a0030 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.737+0000 7fe85dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:52.752 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.737+0000 7fe85dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:52.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.905+0000 7fe86f7c8640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:52.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.905+0000 7fe86f7c8640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:52.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.905+0000 7fe86f7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:52.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:52.905+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"} v 0) -- 0x7fe86813d750 con 0x7fe8680a0030 2024-09-15T21:25:52.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:53.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient: get_monmap_and_config 2024-09-15T21:25:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient: monmap: 2024-09-15T21:25:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:53.116196+0000 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:53.116196+0000 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:53.186 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 1 Processor -- start 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 1 -- start start 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient: init 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ca00a2920 0x7f5ca00a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient(hunting): picked mon.noname-c con 0x7f5ca00a2920 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ca00a3240 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.111+0000 7f5ca7d04640 10 monclient(hunting): picked mon.noname-b con 0x7f5ca0135df0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca009ecf0 con 0x7f5ca00a3240 2024-09-15T21:25:53.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca009ea10 con 0x7f5ca00a2920 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca00a1230 con 0x7f5ca0135df0 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 10 monclient(hunting): authenticate will time out at 1476.512051s 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca529f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5ca00a3240 0x7f5ca01356e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48402/0 (socket says 172.21.15.17:48402) 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca529f640 1 -- 172.21.15.17:0/3039779008 learned_addr learned my addr 172.21.15.17:0/3039779008 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 --2- 172.21.15.17:0/3039779008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): get_auth_request con 0x7f5ca0135df0 auth_method 0 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 1 --2- 172.21.15.17:0/3039779008 >> v2:172.21.15.119:3300/0 conn(0x7f5ca00a2920 0x7f5ca00a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): get_auth_request con 0x7f5ca00a2920 auth_method 0 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 1 -- 172.21.15.17:0/3039779008 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89567067 0 0) 0x7f5ca009ecf0 con 0x7f5ca00a3240 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca5aa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient(hunting): _negotiate 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient(hunting): my global_id is 104097 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 1 -- 172.21.15.17:0/3039779008 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c8c003530 con 0x7f5ca00a3240 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_done global_id 45278 payload 274 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 -- 172.21.15.17:0/3039779008 >> v2:172.21.15.119:3300/0 conn(0x7f5ca00a2920 msgr2=0x7f5ca00a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 --2- 172.21.15.17:0/3039779008 >> v2:172.21.15.119:3300/0 conn(0x7f5ca00a2920 0x7f5ca00a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 -- 172.21.15.17:0/3039779008 >> v1:172.21.15.17:6789/0 conn(0x7f5ca00a3240 legacy=0x7f5ca01356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 monclient: found mon.noname-b 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 -- 172.21.15.17:0/3039779008 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca006d1b0 con 0x7f5ca0135df0 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca62a1640 1 --2- 172.21.15.17:0/3039779008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013db50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009dc0 tx=0x7f5c9801c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ded0b28733c5a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 1 -- 172.21.15.17:0/3039779008 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980b2070 con 0x7f5ca0135df0 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: dump: 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 5 monclient: authenticate success, global_id 45278 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 1 -- 172.21.15.17:0/3039779008 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c980a58f0 con 0x7f5ca0135df0 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 4 set_mon_vals no callback set 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 1 -- 172.21.15.17:0/3039779008 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980a5ca0 con 0x7f5ca0135df0 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca4a9e640 10 monclient: dump: 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 10 monclient: shutdown 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- 172.21.15.17:0/3039779008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 msgr2=0x7f5ca013db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/3039779008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013db50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009dc0 tx=0x7f5c9801c810 comp rx=0 tx=0).stop 2024-09-15T21:25:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- 172.21.15.17:0/3039779008 shutdown_connections 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/3039779008 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013db50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/3039779008 >> v2:172.21.15.119:3300/0 conn(0x7f5ca00a2920 0x7f5ca00a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- 172.21.15.17:0/3039779008 >> 172.21.15.17:0/3039779008 conn(0x7f5ca00a7510 msgr2=0x7f5ca00a3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- 172.21.15.17:0/3039779008 shutdown_connections 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.115+0000 7f5ca7d04640 1 -- 172.21.15.17:0/3039779008 wait complete. 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: monmap: 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 Processor -- start 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- start start 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: init 2024-09-15T21:25:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ca00a2920 0x7f5ca01cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): picked mon.noname-b con 0x7f5ca00a2920 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 0x7f5ca0134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): picked mon.noname-a con 0x7f5ca00a3240 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): picked mon.noname-c con 0x7f5ca0135df0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca006cf20 con 0x7f5ca00a3240 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca006cda0 con 0x7f5ca00a2920 2024-09-15T21:25:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca01408b0 con 0x7f5ca0135df0 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient(hunting): authenticate will time out at 1476.517430s 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 0x7f5ca0134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45568/0 (socket says 172.21.15.17:45568) 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 0x7f5ca0134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53864/0 (socket says 172.21.15.17:53864) 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 1 -- 172.21.15.17:0/2545810220 learned_addr learned my addr 172.21.15.17:0/2545810220 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): get_auth_request con 0x7f5ca0135df0 auth_method 0 2024-09-15T21:25:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): get_auth_request con 0x7f5ca00a3240 auth_method 0 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ca00a2920 0x7f5ca01cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): get_auth_request con 0x7f5ca00a2920 auth_method 0 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient(hunting): handle_auth_done global_id 104100 payload 824 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 -- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 msgr2=0x7f5ca013ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 -- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ca00a2920 msgr2=0x7f5ca01cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ca00a2920 0x7f5ca01cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 monclient: found mon.noname-a 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 -- 172.21.15.17:0/2545810220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca009d680 con 0x7f5ca00a3240 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca62a1640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca529f640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 0x7f5ca0134aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c400 tx=0x7f5c9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c017070 con 0x7f5ca00a3240 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: dump: 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.207 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c005cf0 con 0x7f5ca00a3240 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 5 monclient: authenticate success, global_id 104100 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _renew_subs 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca00a5d50 con 0x7f5ca00a3240 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0072d0 con 0x7f5ca00a3240 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 10 monclient: dump: 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c97fff640 4 set_mon_vals no callback set 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _renew_subs 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca009e100 con 0x7f5ca00a3240 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5c9c01a070 con 0x7f5ca00a3240 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca00a2d00 con 0x7f5ca00a3240 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c8403b9c0 0x7f5c8403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 7657+0+0 (secure 0 0 0) 0x7f5c9c04f8b0 con 0x7f5ca00a3240 2024-09-15T21:25:53.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.119+0000 7f5ca5aa0640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c8403b9c0 0x7f5c8403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.123+0000 7f5ca5aa0640 10 monclient: get_auth_request con 0x7f5c8403b9c0 auth_method 0 2024-09-15T21:25:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.123+0000 7f5ca5aa0640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c8403b9c0 0x7f5c8403de80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009b60 tx=0x7f5c90009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.123+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5c9c01b3b0 con 0x7f5ca00a3240 2024-09-15T21:25:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.127+0000 7f5c967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.127+0000 7f5c967fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:53.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.359+0000 7f5ca7d04640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.359+0000 7f5ca7d04640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.359+0000 7f5ca7d04640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.359+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5ca009d400 con 0x7f5ca00a3240 2024-09-15T21:25:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5c967fc640 1 -- 172.21.15.17:0/2545810220 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f5c9c01b730 con 0x7f5ca00a3240 2024-09-15T21:25:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5c967fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:53.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5c967fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:53.367 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c8403b9c0 msgr2=0x7f5c8403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c8403b9c0 0x7f5c8403de80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009b60 tx=0x7f5c90009190 comp rx=0 tx=0).stop 2024-09-15T21:25:53.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 10 monclient: shutdown 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 msgr2=0x7f5ca0134aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 0x7f5ca0134aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c400 tx=0x7f5c9c00c8c0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 shutdown_connections 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c8403b9c0 0x7f5c8403de80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ca0135df0 0x7f5ca013ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca00a3240 0x7f5ca0134aa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 --2- 172.21.15.17:0/2545810220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ca00a2920 0x7f5ca01cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 >> 172.21.15.17:0/2545810220 conn(0x7f5ca00a7510 msgr2=0x7f5ca0072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 shutdown_connections 2024-09-15T21:25:53.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.363+0000 7f5ca7d04640 1 -- 172.21.15.17:0/2545810220 wait complete. 2024-09-15T21:25:53.385 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:25:53.385 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:25:53.386 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient: get_monmap_and_config 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient: monmap: 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:53.506640+0000 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:53.506640+0000 2024-09-15T21:25:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:53.508 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 Processor -- start 2024-09-15T21:25:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 -- start start 2024-09-15T21:25:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient: init 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): picked mon.noname-b con 0x7f92cc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f92cc09ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92cc0a11b0 0x7f92cc135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): picked mon.noname-a con 0x7f92cc0a11b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0a3130 con 0x7f92cc09ffc0 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f92cc073df0 con 0x7f92cc0a11b0 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f92cc0a1bf0 con 0x7f92cc0a0890 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d3ad4640 10 monclient(hunting): authenticate will time out at 1476.901902s 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92cc0a11b0 0x7f92cc135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d106f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f92cc09ffc0 0x7f92cc135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48406/0 (socket says 172.21.15.17:48406) 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92cc0a11b0 0x7f92cc135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53888/0 (socket says 172.21.15.17:53888) 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d106f640 1 -- 172.21.15.17:0/2079610502 learned_addr learned my addr 172.21.15.17:0/2079610502 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): get_auth_request con 0x7f92cc0a11b0 auth_method 0 2024-09-15T21:25:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 1 --2- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): get_auth_request con 0x7f92cc0a0890 auth_method 0 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient(hunting): handle_auth_done global_id 104103 payload 274 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 -- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 msgr2=0x7f92cc0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 --2- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 -- 172.21.15.17:0/2079610502 >> v1:172.21.15.17:6789/0 conn(0x7f92cc09ffc0 legacy=0x7f92cc135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 monclient: found mon.noname-a 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d2071640 1 -- 172.21.15.17:0/2079610502 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc0a15e0 con 0x7f92cc0a11b0 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d086e640 1 -- 172.21.15.17:0/2079610502 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135156774 0 0) 0x7f92cc0a3130 con 0x7f92cc09ffc0 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.503+0000 7f92d086e640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d2071640 1 --2- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc0a11b0 0x7f92cc135d80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0a1870 tx=0x7f92c401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24838edde0234d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 1 -- 172.21.15.17:0/2079610502 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40b2070 con 0x7f92cc0a11b0 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: dump: 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.517 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 1 -- 172.21.15.17:0/2079610502 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c40a57e0 con 0x7f92cc0a11b0 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 5 monclient: authenticate success, global_id 104103 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 1 -- 172.21.15.17:0/2079610502 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40a5bb0 con 0x7f92cc0a11b0 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 4 set_mon_vals no callback set 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d086e640 10 monclient: dump: 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.519 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: shutdown 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc0a11b0 msgr2=0x7f92cc135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc0a11b0 0x7f92cc135d80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0a1870 tx=0x7f92c401c810 comp rx=0 tx=0).stop 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/2079610502 shutdown_connections 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc0a11b0 0x7f92cc135d80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/2079610502 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/2079610502 >> 172.21.15.17:0/2079610502 conn(0x7f92cc0a66a0 msgr2=0x7f92cc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/2079610502 shutdown_connections 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/2079610502 wait complete. 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: monmap: 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:53.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 Processor -- start 2024-09-15T21:25:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- start start 2024-09-15T21:25:53.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: init 2024-09-15T21:25:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc09ffc0 0x7f92cc12e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): picked mon.noname-a con 0x7f92cc09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc12ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): picked mon.noname-c con 0x7f92cc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92cc0a11b0 0x7f92cc13aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): picked mon.noname-b con 0x7f92cc0a11b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc140510 con 0x7f92cc09ffc0 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1364d0 con 0x7f92cc0a11b0 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f92cc140690 con 0x7f92cc0a0890 2024-09-15T21:25:53.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient(hunting): authenticate will time out at 1476.906829s 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc09ffc0 0x7f92cc12e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc12ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc09ffc0 0x7f92cc12e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53892/0 (socket says 172.21.15.17:53892) 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 1 -- 172.21.15.17:0/4248452675 learned_addr learned my addr 172.21.15.17:0/4248452675 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): get_auth_request con 0x7f92cc09ffc0 auth_method 0 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): get_auth_request con 0x7f92cc0a0890 auth_method 0 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d2071640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92cc0a11b0 0x7f92cc13aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient(hunting): handle_auth_done global_id 45287 payload 824 2024-09-15T21:25:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92cc0a11b0 msgr2=0x7f92cc13aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92cc0a11b0 0x7f92cc13aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc09ffc0 msgr2=0x7f92cc12e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc09ffc0 0x7f92cc12e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 monclient: found mon.noname-c 2024-09-15T21:25:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc09d420 con 0x7f92cc0a0890 2024-09-15T21:25:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d1870640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc09ffc0 0x7f92cc12e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d2071640 10 monclient: get_auth_request con 0x7f92cc0a11b0 auth_method 0 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d2071640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92cc0a11b0 0x7f92cc13aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d106f640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc12ef00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f92c800c3c0 tx=0x7f92c800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c80074e0 con 0x7f92cc0a0890 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: dump: 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c80047f0 con 0x7f92cc0a0890 2024-09-15T21:25:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 5 monclient: authenticate success, global_id 45287 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: _renew_subs 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc12f440 con 0x7f92cc0a0890 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004be0 con 0x7f92cc0a0890 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c27fc640 10 monclient: dump: 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92c3fff640 4 set_mon_vals no callback set 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: _renew_subs 2024-09-15T21:25:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.507+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc09e250 con 0x7f92cc0a0890 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d3ad4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 0x7f92cc13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc09da30 con 0x7f92cc13b3f0 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 0x7f92cc13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient: get_auth_request con 0x7f92cc13b3f0 auth_method 0 2024-09-15T21:25:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f92c8011070 con 0x7f92cc0a0890 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 10 monclient(hunting): handle_auth_done global_id 104112 payload 824 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92cc0a0c70 con 0x7f92cc13b3f0 2024-09-15T21:25:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f92b003b9c0 0x7f92b003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d1870640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 0x7f92cc13b7d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009ec0 tx=0x7f92bc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(255..255 src has 1..255) ==== 7657+0+0 (secure 0 0 0) 0x7f92c80502e0 con 0x7f92cc0a0890 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc004020 con 0x7f92cc13b3f0 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 10 monclient: dump: 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d2071640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f92b003b9c0 0x7f92b003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f92bc004020 con 0x7f92cc13b3f0 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 msgr2=0x7f92cc13b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 0x7f92cc13b7d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009ec0 tx=0x7f92bc00f610 comp rx=0 tx=0).stop 2024-09-15T21:25:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 msgr2=0x7f92cc13b7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92c27fc640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc13b3f0 0x7f92cc13b7d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d2071640 10 monclient: get_auth_request con 0x7f92b003b9c0 auth_method 0 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.511+0000 7f92d2071640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f92b003b9c0 0x7f92b003de80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f92c40af000 tx=0x7f92c40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d3ad4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 0x7f92cc070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d3ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc13bd10 con 0x7f92cc070970 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 0x7f92cc070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient: get_auth_request con 0x7f92cc070970 auth_method 0 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 10 monclient(hunting): handle_auth_done global_id 104115 payload 824 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 1 -- 172.21.15.17:0/4248452675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92cc09d6f0 con 0x7f92cc070970 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92d1870640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 0x7f92cc070d80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f92bc01b000 tx=0x7f92bc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0149f0 con 0x7f92cc070970 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92c27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92c27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.515+0000 7f92c27fc640 10 monclient: dump: 2024-09-15T21:25:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f92bc0273a0 con 0x7f92cc070970 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 msgr2=0x7f92cc070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 0x7f92cc070d80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f92bc01b000 tx=0x7f92bc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 msgr2=0x7f92cc070d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92c27fc640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92cc070970 0x7f92cc070d80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:53.545 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 82, 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7267, 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:53.549 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.550 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.551 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:53.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.557 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.558 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:53.559 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f92b003b9c0 msgr2=0x7f92b003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f92b003b9c0 0x7f92b003de80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f92c40af000 tx=0x7f92c40a7070 comp rx=0 tx=0).stop 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 10 monclient: shutdown 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 msgr2=0x7f92cc12ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 --2- 172.21.15.17:0/4248452675 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92cc0a0890 0x7f92cc12ef00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f92c800c3c0 tx=0x7f92c800c880 comp rx=0 tx=0).stop 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d2071640 1 -- 172.21.15.17:0/4248452675 reap_dead start 2024-09-15T21:25:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d2071640 1 -- 172.21.15.17:0/4248452675 reap_dead start 2024-09-15T21:25:53.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 shutdown_connections 2024-09-15T21:25:53.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 >> 172.21.15.17:0/4248452675 conn(0x7f92cc0a66a0 msgr2=0x7f92cc06fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 shutdown_connections 2024-09-15T21:25:53.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.519+0000 7f92d3ad4640 1 -- 172.21.15.17:0/4248452675 wait complete. 2024-09-15T21:25:53.562 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:53.562 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:25:53.562 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:25:53.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.645+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}]=0 v256) ==== 106+0+3 (secure 0 0 0) 0x7fe86401cd70 con 0x7fe8680a0030 2024-09-15T21:25:53.650 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.645+0000 7fe85dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:53.651 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.645+0000 7fe85dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:53.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient: get_monmap_and_config 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient: monmap: 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:53.678318+0000 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:53.678318+0000 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:53.679 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:53.680 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 Processor -- start 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 -- start start 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient: init 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 0x7f8cfc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): picked mon.noname-b con 0x7f8cfc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8cfc1381e0 0x7f8cfc0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): picked mon.noname-c con 0x7f8cfc1381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 0x7f8cfc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): picked mon.noname-a con 0x7f8cfc0a5040 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8cfc0a1d00 con 0x7f8cfc0a5040 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8cfc0a1fe0 con 0x7f8cfc1381e0 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc0a22c0 con 0x7f8cfc1378c0 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8d01f47640 10 monclient(hunting): authenticate will time out at 1477.073552s 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 0x7f8cfc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 0x7f8cfc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 0x7f8cfc137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45596/0 (socket says 172.21.15.17:45596) 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/4283712721 learned_addr learned my addr 172.21.15.17:0/4283712721 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 0x7f8cfc0a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53922/0 (socket says 172.21.15.17:53922) 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): get_auth_request con 0x7f8cfc1378c0 auth_method 0 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): get_auth_request con 0x7f8cfc0a5040 auth_method 0 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfaffd640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.119:3300/0 conn(0x7f8cfc1381e0 0x7f8cfc0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfaffd640 10 monclient(hunting): get_auth_request con 0x7f8cfc1381e0 auth_method 0 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_done global_id 45290 payload 274 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/4283712721 >> v2:172.21.15.119:3300/0 conn(0x7f8cfc1381e0 msgr2=0x7f8cfc0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.119:3300/0 conn(0x7f8cfc1381e0 0x7f8cfc0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/4283712721 >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 msgr2=0x7f8cfc0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 0x7f8cfc0a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/4283712721 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc0a4490 con 0x7f8cfc1378c0 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfbfff640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 0x7f8cfc0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:53.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.675+0000 7f8cfaffd640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.119:3300/0 conn(0x7f8cfc1381e0 0x7f8cfc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 --2- 172.21.15.17:0/4283712721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 0x7f8cfc137ca0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0a7b50 tx=0x7f8cec01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a234509fa865ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 1 -- 172.21.15.17:0/4283712721 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0b0070 con 0x7f8cfc1378c0 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: dump: 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 1 -- 172.21.15.17:0/4283712721 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec0ab600 con 0x7f8cfc1378c0 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 5 monclient: authenticate success, global_id 45290 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 1 -- 172.21.15.17:0/4283712721 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0ab9d0 con 0x7f8cfc1378c0 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 4 set_mon_vals no callback set 2024-09-15T21:25:53.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfa7fc640 10 monclient: dump: 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.699 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: shutdown 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/4283712721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 msgr2=0x7f8cfc137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- 172.21.15.17:0/4283712721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 0x7f8cfc137ca0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0a7b50 tx=0x7f8cec01c470 comp rx=0 tx=0).stop 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/4283712721 shutdown_connections 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.119:3300/0 conn(0x7f8cfc1381e0 0x7f8cfc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- 172.21.15.17:0/4283712721 >> v2:172.21.15.17:3300/0 conn(0x7f8cfc0a5040 0x7f8cfc0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- 172.21.15.17:0/4283712721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1378c0 0x7f8cfc137ca0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/4283712721 >> 172.21.15.17:0/4283712721 conn(0x7f8cfc0aac40 msgr2=0x7f8cfc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/4283712721 shutdown_connections 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/4283712721 wait complete. 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: monmap: 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:53.701 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 Processor -- start 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- start start 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: init 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc0a5040 0x7f8cfc1cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): picked mon.noname-c con 0x7f8cfc0a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cfc1378c0 0x7f8cfc136b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): picked mon.noname-a con 0x7f8cfc1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc1381e0 0x7f8cfc13ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): picked mon.noname-b con 0x7f8cfc1381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc141730 con 0x7f8cfc1378c0 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1415b0 con 0x7f8cfc1381e0 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc1418b0 con 0x7f8cfc0a5040 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient(hunting): authenticate will time out at 1477.078533s 2024-09-15T21:25:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc0a5040 0x7f8cfc1cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc0a5040 0x7f8cfc1cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45610/0 (socket says 172.21.15.17:45610) 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cfc1378c0 0x7f8cfc136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/3954225138 learned_addr learned my addr 172.21.15.17:0/3954225138 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): get_auth_request con 0x7f8cfc0a5040 auth_method 0 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): get_auth_request con 0x7f8cfc1378c0 auth_method 0 2024-09-15T21:25:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfbfff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc1381e0 0x7f8cfc13ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfbfff640 10 monclient(hunting): get_auth_request con 0x7f8cfc1381e0 auth_method 0 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient(hunting): handle_auth_done global_id 45293 payload 824 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc1381e0 msgr2=0x7f8cfc13ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc1381e0 0x7f8cfc13ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cfc1378c0 msgr2=0x7f8cfc136b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cfc1378c0 0x7f8cfc136b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc122240 con 0x7f8cfc0a5040 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cfb7fe640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc0a5040 0x7f8cfc1cfbe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0ae040 tx=0x7f8cec019210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0b0040 con 0x7f8cfc0a5040 2024-09-15T21:25:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: dump: 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec0b9070 con 0x7f8cfc0a5040 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0b4680 con 0x7f8cfc0a5040 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 5 monclient: authenticate success, global_id 45293 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: _renew_subs 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc122520 con 0x7f8cfc0a5040 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf9ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8cf3fff640 10 monclient: dump: 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: _renew_subs 2024-09-15T21:25:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.679+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc121c80 con 0x7f8cfc0a5040 2024-09-15T21:25:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8d01f47640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8d01f47640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 0x7f8cfc13d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc137ca0 con 0x7f8cfc120d20 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 0x7f8cfc13d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient: get_auth_request con 0x7f8cfc120d20 auth_method 0 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8cec0b4820 con 0x7f8cfc0a5040 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_done global_id 45296 payload 824 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cfc06e960 con 0x7f8cfc120d20 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8cdc03b9c0 0x7f8cdc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(256..256 src has 1..256) ==== 7657+0+0 (secure 0 0 0) 0x7f8cec0ea330 con 0x7f8cfc0a5040 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfaffd640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 0x7f8cfc13d360 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011520 tx=0x7f8ce40119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfbfff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8cdc03b9c0 0x7f8cdc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4019070 con 0x7f8cfc120d20 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 10 monclient: dump: 2024-09-15T21:25:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfbfff640 10 monclient: get_auth_request con 0x7f8cdc03b9c0 auth_method 0 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8ce4015930 con 0x7f8cfc120d20 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 msgr2=0x7f8cfc13d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 0x7f8cfc13d360 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011520 tx=0x7f8ce40119e0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 msgr2=0x7f8cfc13d360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cf3fff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc120d20 0x7f8cfc13d360 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011520 tx=0x7f8ce40119e0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.683+0000 7f8cfbfff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8cdc03b9c0 0x7f8cdc03de80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc120870 tx=0x7f8cf400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8d01f47640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8d01f47640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 0x7f8cfc13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8d01f47640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc069270 con 0x7f8cfc1370c0 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 0x7f8cfc13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient: get_auth_request con 0x7f8cfc1370c0 auth_method 0 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 10 monclient(hunting): handle_auth_done global_id 45299 payload 824 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 1 -- 172.21.15.17:0/3954225138 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8cfc06cba0 con 0x7f8cfc1370c0 2024-09-15T21:25:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cfaffd640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 0x7f8cfc13d8a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401b000 tx=0x7f8ce40046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400f040 con 0x7f8cfc1370c0 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cf3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.687+0000 7f8cf3fff640 10 monclient: dump: 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f8ce400f460 con 0x7f8cfc1370c0 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 msgr2=0x7f8cfc13d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 0x7f8cfc13d8a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401b000 tx=0x7f8ce40046d0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 msgr2=0x7f8cfc13d8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cf3fff640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc1370c0 0x7f8cfc13d8a0 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401b000 tx=0x7f8ce40046d0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 82, 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 322285, 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.718 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.719 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.720 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:53.721 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.722 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.725 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:53.728 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.729 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:53.731 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8cdc03b9c0 msgr2=0x7f8cdc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8cdc03b9c0 0x7f8cdc03de80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc120870 tx=0x7f8cf400b040 comp rx=0 tx=0).stop 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 10 monclient: shutdown 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc0a5040 msgr2=0x7f8cfc1cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 --2- 172.21.15.17:0/3954225138 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8cfc0a5040 0x7f8cfc1cfbe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0ae040 tx=0x7f8cec019210 comp rx=0 tx=0).stop 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cfbfff640 1 -- 172.21.15.17:0/3954225138 reap_dead start 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8cfbfff640 1 -- 172.21.15.17:0/3954225138 reap_dead start 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 shutdown_connections 2024-09-15T21:25:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 >> 172.21.15.17:0/3954225138 conn(0x7f8cfc0aac40 msgr2=0x7f8cfc0a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 shutdown_connections 2024-09-15T21:25:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.691+0000 7f8d01f47640 1 -- 172.21.15.17:0/3954225138 wait complete. 2024-09-15T21:25:53.734 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:53.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:25:53.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:25:53.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe86f7c8640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:53.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe86f7c8640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:53.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe86f7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:53.760 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"} v 0) -- 0x7fe8680008e0 con 0x7fe8680a0030 2024-09-15T21:25:53.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe85dffb640 1 -- 172.21.15.119:0/3833609254 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}]=0 v256) ==== 106+0+3 (secure 0 0 0) 0x7fe864013cb0 con 0x7fe8680a0030 2024-09-15T21:25:53.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe85dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:53.761 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.757+0000 7fe85dffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:25:53.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe84c03b9c0 msgr2=0x7fe84c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe84c03b9c0 0x7fe84c03de80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe8580ad000 tx=0x7fe8580a4f90 comp rx=0 tx=0).stop 2024-09-15T21:25:53.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 10 monclient: shutdown 2024-09-15T21:25:53.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 msgr2=0x7fe86809d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.764 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 0x7fe86809d4e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d800 tx=0x7fe86400dcc0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 shutdown_connections 2024-09-15T21:25:53.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe84c03b9c0 0x7fe84c03de80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8681364a0 0x7fe86809df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe868135b80 0x7fe86809da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 --2- 172.21.15.119:0/3833609254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8680a0030 0x7fe86809d4e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 >> 172.21.15.119:0/3833609254 conn(0x7fe86806cdc0 msgr2=0x7fe868072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 shutdown_connections 2024-09-15T21:25:53.766 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.761+0000 7fe86f7c8640 1 -- 172.21.15.119:0/3833609254 wait complete. 2024-09-15T21:25:53.779 INFO:tasks.workunit.client.0.smithi119.stderr:+ nb=14 2024-09-15T21:25:53.780 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd create e2307bd0-cd40-470d-a650-9bfee3159ff3 2024-09-15T21:25:53.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.851+0000 7f9c7d7c0640 10 monclient: get_monmap_and_config 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.851+0000 7f9c7d7c0640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.851+0000 7f9c7d7c0640 10 monclient: monmap: 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:53.857253+0000 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:53.857253+0000 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:53.859 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.851+0000 7f9c7d7c0640 1 Processor -- start 2024-09-15T21:25:53.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.851+0000 7f9c7d7c0640 1 -- start start 2024-09-15T21:25:53.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.851+0000 7f9c7d7c0640 10 monclient: init 2024-09-15T21:25:53.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9c781378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9c781383b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 0x7f9c78140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient(hunting): picked mon.noname-b con 0x7f9c7813e4a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c780a1a20 con 0x7f9c781378c0 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c780a1d00 con 0x7f9c781383b0 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9c780a1fe0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient(hunting): authenticate will time out at 1477.252634s 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 0x7f9c78140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 0x7f9c78140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45632/0 (socket says 172.21.15.17:45632) 2024-09-15T21:25:53.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 -- 172.21.15.17:0/2968757078 learned_addr learned my addr 172.21.15.17:0/2968757078 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): get_auth_request con 0x7f9c7813e4a0 auth_method 0 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 1 -- 172.21.15.17:0/2968757078 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 538714176 0 0) 0x7f9c70001120 con 0x7f9c781378c0 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient(hunting): handle_auth_done global_id 45302 payload 274 2024-09-15T21:25:53.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 -- 172.21.15.17:0/2968757078 >> v1:172.21.15.119:6789/0 conn(0x7f9c781383b0 legacy=0x7f9c7813cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 -- 172.21.15.17:0/2968757078 >> v1:172.21.15.17:6789/0 conn(0x7f9c781378c0 legacy=0x7f9c78137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 -- 172.21.15.17:0/2968757078 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c780a22c0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c777fe640 1 --2- 172.21.15.17:0/2968757078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 0x7f9c78140890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009eb0 tx=0x7f9c6801c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22cc3e789305b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 1 -- 172.21.15.17:0/2968757078 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680a4c90 con 0x7f9c7813e4a0 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: dump: 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.870 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 1 -- 172.21.15.17:0/2968757078 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c680a4e30 con 0x7f9c7813e4a0 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 1 -- 172.21.15.17:0/2968757078 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680ac5f0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 5 monclient: authenticate success, global_id 45302 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 4 set_mon_vals no callback set 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c75ffb640 10 monclient: dump: 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.855+0000 7f9c7d7c0640 10 monclient: shutdown 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2968757078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 msgr2=0x7f9c78140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 --2- 172.21.15.17:0/2968757078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 0x7f9c78140890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009eb0 tx=0x7f9c6801c5d0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2968757078 shutdown_connections 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 --2- 172.21.15.17:0/2968757078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c7813e4a0 0x7f9c78140890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2968757078 >> 172.21.15.17:0/2968757078 conn(0x7f9c780aac40 msgr2=0x7f9c780a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2968757078 shutdown_connections 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2968757078 wait complete. 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: monmap: 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:53.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 Processor -- start 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- start start 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: init 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c781378c0 0x7f9c78136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): picked mon.noname-c con 0x7f9c781378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781383b0 0x7f9c7809cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): picked mon.noname-b con 0x7f9c781383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c7813e4a0 0x7f9c7809cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): picked mon.noname-a con 0x7f9c7813e4a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781442e0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78144160 con 0x7f9c781383b0 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9c78144460 con 0x7f9c781378c0 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): authenticate will time out at 1477.257619s 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c781378c0 0x7f9c78136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c781378c0 0x7f9c78136b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45638/0 (socket says 172.21.15.17:45638) 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c7813e4a0 0x7f9c7809cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 1 -- 172.21.15.17:0/2901873687 learned_addr learned my addr 172.21.15.17:0/2901873687 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): get_auth_request con 0x7f9c781378c0 auth_method 0 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781383b0 0x7f9c7809cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): get_auth_request con 0x7f9c7813e4a0 auth_method 0 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): get_auth_request con 0x7f9c781383b0 auth_method 0 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient(hunting): handle_auth_done global_id 104127 payload 824 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c781378c0 msgr2=0x7f9c78136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c781378c0 0x7f9c78136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781383b0 msgr2=0x7f9c7809cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781383b0 0x7f9c7809cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 monclient: found mon.noname-a 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7809f2b0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9c781378c0 0x7f9c78136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c777fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c7813e4a0 0x7f9c7809cff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9c680aa000 tx=0x7f9c68019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680a4c90 con 0x7f9c7813e4a0 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: dump: 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c680090d0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 5 monclient: authenticate success, global_id 104127 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: _renew_subs 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7809e480 con 0x7f9c7813e4a0 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68009510 con 0x7f9c7813e4a0 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 10 monclient: dump: 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.881 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c757fa640 4 set_mon_vals no callback set 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: _renew_subs 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c780a07e0 con 0x7f9c7813e4a0 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 0x7f9c7809d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780a12f0 con 0x7f9c781370b0 2024-09-15T21:25:53.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9c680bf390 con 0x7f9c7813e4a0 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 0x7f9c7809d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient: get_auth_request con 0x7f9c781370b0 auth_method 0 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c5803b9c0 0x7f9c5803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(256..256 src has 1..256) ==== 7657+0+0 (secure 0 0 0) 0x7f9c680e9e30 con 0x7f9c7813e4a0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_done global_id 84733 payload 824 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.859+0000 7f9c76ffd640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c7813de60 con 0x7f9c781370b0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c767fc640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c5803b9c0 0x7f9c5803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c767fc640 10 monclient: get_auth_request con 0x7f9c5803b9c0 auth_method 0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c76ffd640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 0x7f9c7809d530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000faa0 tx=0x7f9c7000ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700053a0 con 0x7f9c781370b0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 10 monclient: dump: 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c767fc640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c5803b9c0 0x7f9c5803de80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9c60007fd0 tx=0x7f9c60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9c70018070 con 0x7f9c781370b0 2024-09-15T21:25:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 msgr2=0x7f9c7809d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 0x7f9c7809d530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000faa0 tx=0x7f9c7000ff60 comp rx=0 tx=0).stop 2024-09-15T21:25:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 msgr2=0x7f9c7809d530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.863+0000 7f9c6f7fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c781370b0 0x7f9c7809d530 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000faa0 tx=0x7f9c7000ff60 comp rx=0 tx=0).stop 2024-09-15T21:25:53.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c7d7c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c7d7c0640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 0x7f9c7813ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c7d7c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78137ca0 con 0x7f9c7809da70 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 0x7f9c7813ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient: get_auth_request con 0x7f9c7809da70 auth_method 0 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 10 monclient(hunting): handle_auth_done global_id 84736 payload 824 2024-09-15T21:25:53.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 1 -- 172.21.15.17:0/2901873687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9c7809de50 con 0x7f9c7809da70 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.867+0000 7f9c76ffd640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 0x7f9c7813ad70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9c70017000 tx=0x7f9c70000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7001ecb0 con 0x7f9c7809da70 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 10 monclient: dump: 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.889 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f9c7001ecb0 con 0x7f9c7809da70 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 msgr2=0x7f9c7813ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 0x7f9c7813ad70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9c70017000 tx=0x7f9c70000ee0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 msgr2=0x7f9c7813ad70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c6f7fe640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7809da70 0x7f9c7813ad70 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9c70017000 tx=0x7f9c70000ee0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:53.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 82, 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 106099, 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:53.891 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:53.894 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:53.895 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.897 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:53.898 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.899 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:53.903 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:53.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:53.904 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:53.904 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:53.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c5803b9c0 msgr2=0x7f9c5803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9c5803b9c0 0x7f9c5803de80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9c60007fd0 tx=0x7f9c60009040 comp rx=0 tx=0).stop 2024-09-15T21:25:53.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 10 monclient: shutdown 2024-09-15T21:25:53.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c7813e4a0 msgr2=0x7f9c7809cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 --2- 172.21.15.17:0/2901873687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c7813e4a0 0x7f9c7809cff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9c680aa000 tx=0x7f9c68019690 comp rx=0 tx=0).stop 2024-09-15T21:25:53.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c777fe640 1 -- 172.21.15.17:0/2901873687 reap_dead start 2024-09-15T21:25:53.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c777fe640 1 -- 172.21.15.17:0/2901873687 reap_dead start 2024-09-15T21:25:53.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 shutdown_connections 2024-09-15T21:25:53.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 >> 172.21.15.17:0/2901873687 conn(0x7f9c780aac40 msgr2=0x7f9c7806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 shutdown_connections 2024-09-15T21:25:53.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:53.871+0000 7f9c7d7c0640 1 -- 172.21.15.17:0/2901873687 wait complete. 2024-09-15T21:25:53.906 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:53.906 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:25:53.906 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:25:53.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient: get_monmap_and_config 2024-09-15T21:25:53.914 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient: monmap: 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:53.898118+0000 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:53.898118+0000 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:53.915 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 1 Processor -- start 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 1 -- start start 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient: init 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa6080a4d80 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa6080a5870 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa60813e6f0 0x7fa608140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient(hunting): picked mon.noname-c con 0x7fa60813e6f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6080a2250 con 0x7fa6080a4d80 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60806dab0 con 0x7fa6080a5870 2024-09-15T21:25:53.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa6080a19b0 con 0x7fa60813e6f0 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60ebce640 10 monclient(hunting): authenticate will time out at 1472.695053s 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa60813e6f0 0x7fa608140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa607fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa6080a5870 0x7fa60813cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60172/0 (socket says 172.21.15.119:60172) 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa607fff640 1 -- 172.21.15.119:0/1142540208 learned_addr learned my addr 172.21.15.119:0/1142540208 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): get_auth_request con 0x7fa60813e6f0 auth_method 0 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient(hunting): handle_auth_done global_id 84739 payload 274 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 1 -- 172.21.15.119:0/1142540208 >> v1:172.21.15.119:6789/0 conn(0x7fa6080a5870 legacy=0x7fa60813cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 1 -- 172.21.15.119:0/1142540208 >> v1:172.21.15.17:6789/0 conn(0x7fa6080a4d80 legacy=0x7fa6080a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 1 monclient: found mon.noname-c 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.893+0000 7fa60d16b640 1 -- 172.21.15.119:0/1142540208 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6080a1c90 con 0x7fa60813e6f0 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60d16b640 1 --2- 172.21.15.119:0/1142540208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60813e6f0 0x7fa608140ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa600009dc0 tx=0x7fa60001c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4bc960b1f4ebcf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 1 -- 172.21.15.119:0/1142540208 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b2070 con 0x7fa60813e6f0 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: dump: 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:53.918 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 5 monclient: authenticate success, global_id 84739 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:53.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 1 -- 172.21.15.119:0/1142540208 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6000a58f0 con 0x7fa60813e6f0 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 1 -- 172.21.15.119:0/1142540208 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000a5cc0 con 0x7fa60813e6f0 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 4 set_mon_vals no callback set 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa6077fe640 10 monclient: dump: 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:53.920 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: shutdown 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- 172.21.15.119:0/1142540208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60813e6f0 msgr2=0x7fa608140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 --2- 172.21.15.119:0/1142540208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60813e6f0 0x7fa608140ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa600009dc0 tx=0x7fa60001c810 comp rx=0 tx=0).stop 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- 172.21.15.119:0/1142540208 shutdown_connections 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 --2- 172.21.15.119:0/1142540208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60813e6f0 0x7fa608140ae0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- 172.21.15.119:0/1142540208 >> 172.21.15.119:0/1142540208 conn(0x7fa6080a75c0 msgr2=0x7fa60806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- 172.21.15.119:0/1142540208 shutdown_connections 2024-09-15T21:25:53.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- 172.21.15.119:0/1142540208 wait complete. 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: build_initial_monmap 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: monmap: 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 Processor -- start 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- start start 2024-09-15T21:25:53.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: init 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6080a4d80 0x7fa6081cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): picked mon.noname-a con 0x7fa6080a4d80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 0x7fa6081d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): picked mon.noname-b con 0x7fa6080a5870 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa60813e6f0 0x7fa6081d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): picked mon.noname-c con 0x7fa60813e6f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080a2de0 con 0x7fa6080a4d80 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080a2c60 con 0x7fa6080a5870 2024-09-15T21:25:53.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa608143840 con 0x7fa60813e6f0 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.897+0000 7fa60ebce640 10 monclient(hunting): authenticate will time out at 1472.699936s 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 0x7fa6081d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 0x7fa6081d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58262/0 (socket says 172.21.15.119:58262) 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 -- 172.21.15.119:0/3262273650 learned_addr learned my addr 172.21.15.119:0/3262273650 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): get_auth_request con 0x7fa6080a5870 auth_method 0 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:53.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60c96a640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6080a4d80 0x7fa6081cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60d16b640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa60813e6f0 0x7fa6081d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient(hunting): handle_auth_done global_id 84745 payload 824 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 -- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa60813e6f0 msgr2=0x7fa6081d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa60813e6f0 0x7fa6081d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 -- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6080a4d80 msgr2=0x7fa6081cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6080a4d80 0x7fa6081cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient: _finish_hunting 0 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 monclient: found mon.noname-b 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 -- 172.21.15.119:0/3262273650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa608135bc0 con 0x7fa6080a5870 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60c96a640 10 monclient: get_auth_request con 0x7fa6080a4d80 auth_method 0 2024-09-15T21:25:53.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60c96a640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6080a4d80 0x7fa6081cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa607fff640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 0x7fa6081d0040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c4d0 tx=0x7fa5f400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4017070 con 0x7fa6080a5870 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: dump: 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.926 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: _finish_auth 0 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: _check_auth_tickets 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 5 monclient: authenticate success, global_id 84745 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4005cf0 con 0x7fa6080a5870 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4007270 con 0x7fa6080a5870 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa606ffd640 4 set_mon_vals no callback set 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 10 monclient: dump: 2024-09-15T21:25:53.927 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 monclient: _renew_subs 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6080a1570 con 0x7fa6080a5870 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 monclient: _renew_subs 2024-09-15T21:25:53.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60813c560 con 0x7fa6080a5870 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60813c9b0 con 0x7fa6080a5870 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa5f401a070 con 0x7fa6080a5870 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa5ec03b9c0 0x7fa5ec03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(256..256 src has 1..256) ==== 7657+0+0 (secure 0 0 0) 0x7fa5f404f8f0 con 0x7fa6080a5870 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60c96a640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa5ec03b9c0 0x7fa5ec03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60c96a640 10 monclient: get_auth_request con 0x7fa5ec03b9c0 auth_method 0 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.901+0000 7fa60c96a640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa5ec03b9c0 0x7fa5ec03de80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc000f70 tx=0x7fa5fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.905+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa60813c9b0 con 0x7fa6080a5870 2024-09-15T21:25:53.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.905+0000 7fa6057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:53.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:53.905+0000 7fa6057fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:54.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.011+0000 7f6d87e26640 10 monclient: get_monmap_and_config 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.011+0000 7f6d87e26640 10 monclient: build_initial_monmap 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.011+0000 7f6d87e26640 10 monclient: monmap: 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:54.017549+0000 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:54.017549+0000 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:54.018 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:54.019 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:54.019 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:54.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.011+0000 7f6d87e26640 1 Processor -- start 2024-09-15T21:25:54.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- start start 2024-09-15T21:25:54.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient: init 2024-09-15T21:25:54.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2024-09-15T21:25:54.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d800a4a90 0x7f6d800a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): picked mon.noname-c con 0x7f6d800a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d800a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): picked mon.noname-b con 0x7f6d800a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6d80137850 0x7f6d80137cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): picked mon.noname-a con 0x7f6d80137850 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f6d800a2c30 con 0x7f6d80137850 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6d800a0f00 con 0x7f6d800a4a90 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d800a19b0 con 0x7f6d800a53b0 2024-09-15T21:25:54.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient(hunting): authenticate will time out at 1477.412634s 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6d80137850 0x7f6d80137cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d800a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6d80137850 0x7f6d80137cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53964/0 (socket says 172.21.15.17:53964) 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 -- 172.21.15.17:0/2931604814 learned_addr learned my addr 172.21.15.17:0/2931604814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): get_auth_request con 0x7f6d80137850 auth_method 0 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): get_auth_request con 0x7f6d800a53b0 auth_method 0 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 1 --2- 172.21.15.17:0/2931604814 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a4a90 0x7f6d800a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): get_auth_request con 0x7f6d800a4a90 auth_method 0 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d85bc2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient(hunting): handle_auth_done global_id 104133 payload 274 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 -- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 msgr2=0x7f6d800a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 --2- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d800a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 -- 172.21.15.17:0/2931604814 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a4a90 msgr2=0x7f6d800a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 --2- 172.21.15.17:0/2931604814 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a4a90 0x7f6d800a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 monclient: found mon.noname-a 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 -- 172.21.15.17:0/2931604814 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d800a1c90 con 0x7f6d80137850 2024-09-15T21:25:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d853c1640 1 --2- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d800a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d863c3640 1 --2- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d80137cc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d800a7900 tx=0x7f6d7801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b52e8fb334831d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 1 -- 172.21.15.17:0/2931604814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780a5600 con 0x7f6d80137850 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: dump: 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 5 monclient: authenticate success, global_id 104133 2024-09-15T21:25:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 1 -- 172.21.15.17:0/2931604814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d780a57a0 con 0x7f6d80137850 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 4 set_mon_vals no callback set 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 1 -- 172.21.15.17:0/2931604814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780a5b70 con 0x7f6d80137850 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d84bc0640 10 monclient: dump: 2024-09-15T21:25:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 10 monclient: shutdown 2024-09-15T21:25:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 msgr2=0x7f6d80137cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d80137cc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d800a7900 tx=0x7f6d7801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- 172.21.15.17:0/2931604814 shutdown_connections 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d80137cc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2931604814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d800a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2931604814 >> v2:172.21.15.119:3300/0 conn(0x7f6d800a4a90 0x7f6d800a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- 172.21.15.17:0/2931604814 >> 172.21.15.17:0/2931604814 conn(0x7f6d800aabb0 msgr2=0x7f6d800a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- 172.21.15.17:0/2931604814 shutdown_connections 2024-09-15T21:25:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.015+0000 7f6d87e26640 1 -- 172.21.15.17:0/2931604814 wait complete. 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: build_initial_monmap 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: monmap: 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 Processor -- start 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- start start 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: init 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a4a90 0x7f6d801cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): picked mon.noname-b con 0x7f6d800a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d80136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): picked mon.noname-c con 0x7f6d800a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d801cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): picked mon.noname-a con 0x7f6d80137850 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8006f2d0 con 0x7f6d80137850 2024-09-15T21:25:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8006f150 con 0x7f6d800a4a90 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6d80141540 con 0x7f6d800a53b0 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient(hunting): authenticate will time out at 1477.417351s 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d80136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d801cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d80136de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45656/0 (socket says 172.21.15.17:45656) 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 1 -- 172.21.15.17:0/2117714969 learned_addr learned my addr 172.21.15.17:0/2117714969 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d801cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53972/0 (socket says 172.21.15.17:53972) 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): get_auth_request con 0x7f6d800a53b0 auth_method 0 2024-09-15T21:25:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): get_auth_request con 0x7f6d80137850 auth_method 0 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d853c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d85bc2640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a4a90 0x7f6d801cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d85bc2640 10 monclient(hunting): get_auth_request con 0x7f6d800a4a90 auth_method 0 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d85bc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d85bc2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d85bc2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient(hunting): handle_auth_done global_id 104136 payload 824 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 -- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 msgr2=0x7f6d80136de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d80136de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 -- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a4a90 msgr2=0x7f6d801cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a4a90 0x7f6d801cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 monclient: found mon.noname-a 2024-09-15T21:25:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 -- 172.21.15.17:0/2117714969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d800a1280 con 0x7f6d80137850 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d863c3640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d801cfbe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d780ab000 tx=0x7f6d78019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d780bc030 con 0x7f6d80137850 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: dump: 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d780b8470 con 0x7f6d80137850 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78009510 con 0x7f6d80137850 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 10 monclient: dump: 2024-09-15T21:25:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d77fff640 4 set_mon_vals no callback set 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 5 monclient: authenticate success, global_id 104136 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _renew_subs 2024-09-15T21:25:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d80121780 con 0x7f6d80137850 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _renew_subs 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d80121ee0 con 0x7f6d80137850 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6d780bf070 con 0x7f6d80137850 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d800a57d0 con 0x7f6d80137850 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b9c0 0x7f6d6403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.019+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(256..256 src has 1..256) ==== 7657+0+0 (secure 0 0 0) 0x7f6d780eafc0 con 0x7f6d80137850 2024-09-15T21:25:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.023+0000 7f6d85bc2640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b9c0 0x7f6d6403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.023+0000 7f6d85bc2640 10 monclient: get_auth_request con 0x7f6d6403b9c0 auth_method 0 2024-09-15T21:25:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.023+0000 7f6d85bc2640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b9c0 0x7f6d6403de80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6d801cd760 tx=0x7f6d7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.027+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6d780c6070 con 0x7f6d80137850 2024-09-15T21:25:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.027+0000 7f6d767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.027+0000 7f6d767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:54.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa60ebce640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:54.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa60ebce640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:54.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa60ebce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.089 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"} v 0) -- 0x7fa6080a4850 con 0x7fa6080a5870 2024-09-15T21:25:54.090 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}]=0 v256) ==== 106+0+3 (secure 0 0 0) 0x7fa5f401a360 con 0x7fa6080a5870 2024-09-15T21:25:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa6057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:54.091 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.085+0000 7fa6057fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:54.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa60ebce640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:54.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa60ebce640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:54.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa60ebce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.196 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"} v 0) -- 0x7fa60813b350 con 0x7fa6080a5870 2024-09-15T21:25:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa6057fa640 1 -- 172.21.15.119:0/3262273650 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}]=0 v256) ==== 106+0+3 (secure 0 0 0) 0x7fa5f4012400 con 0x7fa6080a5870 2024-09-15T21:25:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa6057fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "e2307bd0-cd40-470d-a650-9bfee3159ff3"}] 2024-09-15T21:25:54.197 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.193+0000 7fa6057fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:25:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa5ec03b9c0 msgr2=0x7fa5ec03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa5ec03b9c0 0x7fa5ec03de80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc000f70 tx=0x7fa5fc009040 comp rx=0 tx=0).stop 2024-09-15T21:25:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 10 monclient: shutdown 2024-09-15T21:25:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 msgr2=0x7fa6081d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.200 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 0x7fa6081d0040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c4d0 tx=0x7fa5f400c990 comp rx=0 tx=0).stop 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 shutdown_connections 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa5ec03b9c0 0x7fa5ec03de80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa60813e6f0 0x7fa6081d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6080a5870 0x7fa6081d0040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 --2- 172.21.15.119:0/3262273650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6080a4d80 0x7fa6081cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 >> 172.21.15.119:0/3262273650 conn(0x7fa6080a75c0 msgr2=0x7fa608137420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 shutdown_connections 2024-09-15T21:25:54.201 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.197+0000 7fa60ebce640 1 -- 172.21.15.119:0/3262273650 wait complete. 2024-09-15T21:25:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 14 -eq 14 2024-09-15T21:25:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 14 -ne 12 2024-09-15T21:25:54.214 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm 12 2024-09-15T21:25:54.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d87e26640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:25:54.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d87e26640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:25:54.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d87e26640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:54.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f6d8013da80 con 0x7f6d80137850 2024-09-15T21:25:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d767fc640 1 -- 172.21.15.17:0/2117714969 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f6d780165c0 con 0x7f6d80137850 2024-09-15T21:25:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:25:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.255+0000 7f6d767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:54.263 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.263 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=10,logm=58} crc {auth=821600789,config=3541143443,health=765734218,kv=4294714171,logm=2259900865}) 2024-09-15T21:25:54.263 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=18,config=1,health=12,kv=11,logm=58} crc {auth=821600789,config=555582061,health=765734218,kv=3059833255,logm=327704498}) 2024-09-15T21:25:54.263 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.263 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=10,logm=58} crc {auth=821600789,config=3541143443,health=765734218,kv=4294714171,logm=2259900865}) 2024-09-15T21:25:54.263 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=821600789,config=3541143443,health=765734218,kv=3079713637,logm=3971862861}) 2024-09-15T21:25:54.265 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.265 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1119499225}) 2024-09-15T21:25:54.265 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=271377311}) 2024-09-15T21:25:54.265 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1119499225}) 2024-09-15T21:25:54.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.259+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=48869452}) 2024-09-15T21:25:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.259+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b9c0 msgr2=0x7f6d6403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.259+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b9c0 0x7f6d6403de80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6d801cd760 tx=0x7f6d7c008040 comp rx=0 tx=0).stop 2024-09-15T21:25:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.259+0000 7f6d87e26640 10 monclient: shutdown 2024-09-15T21:25:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.259+0000 7f6d87e26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.259+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 msgr2=0x7f6d801cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.259+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d801cfbe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d780ab000 tx=0x7f6d78019650 comp rx=0 tx=0).stop 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 shutdown_connections 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6d6403b9c0 0x7f6d6403de80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d80137850 0x7f6d801cfbe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6d800a53b0 0x7f6d80136de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 --2- 172.21.15.17:0/2117714969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d800a4a90 0x7f6d801cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 >> 172.21.15.17:0/2117714969 conn(0x7f6d800aabb0 msgr2=0x7f6d8013df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 shutdown_connections 2024-09-15T21:25:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:54.263+0000 7f6d87e26640 1 -- 172.21.15.17:0/2117714969 wait complete. 2024-09-15T21:25:54.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.263+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.263+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1042950658}) 2024-09-15T21:25:54.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.263+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2004703389}) 2024-09-15T21:25:54.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.263+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.263+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1042950658}) 2024-09-15T21:25:54.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.263+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215025636}) 2024-09-15T21:25:54.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2995061567}) 2024-09-15T21:25:54.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2214881619}) 2024-09-15T21:25:54.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2995061567}) 2024-09-15T21:25:54.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=548166390}) 2024-09-15T21:25:54.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167775599}) 2024-09-15T21:25:54.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2476625623}) 2024-09-15T21:25:54.273 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.273 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167775599}) 2024-09-15T21:25:54.273 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3116389379}) 2024-09-15T21:25:54.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.267+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100525168}) 2024-09-15T21:25:54.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3241072331}) 2024-09-15T21:25:54.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100525168}) 2024-09-15T21:25:54.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1062719387}) 2024-09-15T21:25:54.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2190349578}) 2024-09-15T21:25:54.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=337340276}) 2024-09-15T21:25:54.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2190349578}) 2024-09-15T21:25:54.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=318498958}) 2024-09-15T21:25:54.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874938439}) 2024-09-15T21:25:54.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2736170225}) 2024-09-15T21:25:54.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874938439}) 2024-09-15T21:25:54.279 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.271+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=520008144}) 2024-09-15T21:25:54.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=179471730}) 2024-09-15T21:25:54.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2318286732}) 2024-09-15T21:25:54.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=179471730}) 2024-09-15T21:25:54.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3186710026}) 2024-09-15T21:25:54.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=966417988}) 2024-09-15T21:25:54.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2701190100}) 2024-09-15T21:25:54.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=966417988}) 2024-09-15T21:25:54.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.275+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=427139693}) 2024-09-15T21:25:54.284 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:25:54.284 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.284 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=596513305}) 2024-09-15T21:25:54.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1954478185}) 2024-09-15T21:25:54.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=596513305}) 2024-09-15T21:25:54.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4052373483}) 2024-09-15T21:25:54.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1453626252}) 2024-09-15T21:25:54.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2112831996}) 2024-09-15T21:25:54.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1453626252}) 2024-09-15T21:25:54.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.279+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2196487284}) 2024-09-15T21:25:54.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=814068850}) 2024-09-15T21:25:54.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=501000767}) 2024-09-15T21:25:54.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=814068850}) 2024-09-15T21:25:54.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2929686131}) 2024-09-15T21:25:54.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4289320741}) 2024-09-15T21:25:54.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=501229525}) 2024-09-15T21:25:54.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4289320741}) 2024-09-15T21:25:54.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.283+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3865237831}) 2024-09-15T21:25:54.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=8944038}) 2024-09-15T21:25:54.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4100062733}) 2024-09-15T21:25:54.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=8944038}) 2024-09-15T21:25:54.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=359691393}) 2024-09-15T21:25:54.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161315713}) 2024-09-15T21:25:54.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4011597480}) 2024-09-15T21:25:54.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161315713}) 2024-09-15T21:25:54.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.287+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1444492627}) 2024-09-15T21:25:54.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2830542839}) 2024-09-15T21:25:54.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2030089088}) 2024-09-15T21:25:54.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2830542839}) 2024-09-15T21:25:54.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3389366839}) 2024-09-15T21:25:54.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=48411360}) 2024-09-15T21:25:54.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3068669750}) 2024-09-15T21:25:54.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=48411360}) 2024-09-15T21:25:54.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.291+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1704450565}) 2024-09-15T21:25:54.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3241855474}) 2024-09-15T21:25:54.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2591272801}) 2024-09-15T21:25:54.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3241855474}) 2024-09-15T21:25:54.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3897083707}) 2024-09-15T21:25:54.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2891469572}) 2024-09-15T21:25:54.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3786824683}) 2024-09-15T21:25:54.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2891469572}) 2024-09-15T21:25:54.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.295+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1966803305}) 2024-09-15T21:25:54.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.304 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3296757733}) 2024-09-15T21:25:54.304 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=134875993}) 2024-09-15T21:25:54.304 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.304 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3296757733}) 2024-09-15T21:25:54.304 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1266527763}) 2024-09-15T21:25:54.308 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.308 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470757328}) 2024-09-15T21:25:54.308 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3565178686}) 2024-09-15T21:25:54.308 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470757328}) 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.299+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3628150678}) 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=342544351}) 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1373511373}) 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=342544351}) 2024-09-15T21:25:54.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2523496642}) 2024-09-15T21:25:54.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886773263}) 2024-09-15T21:25:54.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=526359400}) 2024-09-15T21:25:54.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886773263}) 2024-09-15T21:25:54.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.303+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2037600840}) 2024-09-15T21:25:54.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3662446680}) 2024-09-15T21:25:54.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3048409953}) 2024-09-15T21:25:54.312 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.312 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3662446680}) 2024-09-15T21:25:54.312 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1391348126}) 2024-09-15T21:25:54.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889125833}) 2024-09-15T21:25:54.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3120890747}) 2024-09-15T21:25:54.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889125833}) 2024-09-15T21:25:54.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.307+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2803153088}) 2024-09-15T21:25:54.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180421337}) 2024-09-15T21:25:54.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2165928527}) 2024-09-15T21:25:54.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180421337}) 2024-09-15T21:25:54.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410720928}) 2024-09-15T21:25:54.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4055726179}) 2024-09-15T21:25:54.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=603620344}) 2024-09-15T21:25:54.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4055726179}) 2024-09-15T21:25:54.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.311+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3129696209}) 2024-09-15T21:25:54.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1798190771}) 2024-09-15T21:25:54.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1233746404}) 2024-09-15T21:25:54.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1798190771}) 2024-09-15T21:25:54.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1188760970}) 2024-09-15T21:25:54.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3711119255}) 2024-09-15T21:25:54.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3561602227}) 2024-09-15T21:25:54.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3711119255}) 2024-09-15T21:25:54.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=625495769}) 2024-09-15T21:25:54.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=602446366}) 2024-09-15T21:25:54.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3759073433}) 2024-09-15T21:25:54.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=602446366}) 2024-09-15T21:25:54.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.315+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3914042747}) 2024-09-15T21:25:54.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3894495780}) 2024-09-15T21:25:54.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790342579}) 2024-09-15T21:25:54.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3894495780}) 2024-09-15T21:25:54.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1216430116}) 2024-09-15T21:25:54.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3938385887}) 2024-09-15T21:25:54.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2859070390}) 2024-09-15T21:25:54.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3938385887}) 2024-09-15T21:25:54.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.319+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3046263851}) 2024-09-15T21:25:54.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1568827416}) 2024-09-15T21:25:54.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2861526164}) 2024-09-15T21:25:54.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1568827416}) 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3378272205}) 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399582621}) 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2660447763}) 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399582621}) 2024-09-15T21:25:54.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.323+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3684909370}) 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279602584}) 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2285926888}) 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279602584}) 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2743083049}) 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1443747400}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=642364185}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1443747400}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.327+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3704519372}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1594335654}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=842547226}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1594335654}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3508817621}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2159867133}) 2024-09-15T21:25:54.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2030284671}) 2024-09-15T21:25:54.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2159867133}) 2024-09-15T21:25:54.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.331+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3323235757}) 2024-09-15T21:25:54.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient: get_monmap_and_config 2024-09-15T21:25:54.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient: build_initial_monmap 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient: monmap: 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:54.328928+0000 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:54.328928+0000 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:54.343 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 Processor -- start 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 -- start start 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient: init 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f293406cbf0 0x7f293406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): picked mon.noname-a con 0x7f293406cbf0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f293406d510 0x7f29341347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): picked mon.noname-c con 0x7f293406d510 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2934134cf0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2934073b00 con 0x7f2934134cf0 2024-09-15T21:25:54.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f293409c890 con 0x7f293406cbf0 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f29340738c0 con 0x7f293406d510 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293c600640 10 monclient(hunting): authenticate will time out at 1473.125617s 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f293406d510 0x7f29341347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f293406d510 0x7f29341347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58278/0 (socket says 172.21.15.119:58278) 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 -- 172.21.15.119:0/1914331743 learned_addr learned my addr 172.21.15.119:0/1914331743 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): get_auth_request con 0x7f293406d510 auth_method 0 2024-09-15T21:25:54.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 1 --2- 172.21.15.119:0/1914331743 >> v2:172.21.15.17:3300/0 conn(0x7f293406cbf0 0x7f293406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): get_auth_request con 0x7f293406cbf0 auth_method 0 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f293a39c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient(hunting): handle_auth_done global_id 84754 payload 274 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 -- 172.21.15.119:0/1914331743 >> v2:172.21.15.17:3300/0 conn(0x7f293406cbf0 msgr2=0x7f293406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 --2- 172.21.15.119:0/1914331743 >> v2:172.21.15.17:3300/0 conn(0x7f293406cbf0 0x7f293406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 -- 172.21.15.119:0/1914331743 >> v1:172.21.15.119:6789/0 conn(0x7f2934134cf0 legacy=0x7f293413ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 monclient: found mon.noname-c 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.325+0000 7f2939b9b640 1 -- 172.21.15.119:0/1914331743 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293409bff0 con 0x7f293406d510 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2939b9b640 1 --2- 172.21.15.119:0/1914331743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406d510 0x7f29341347b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2934073aa0 tx=0x7f293001c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01665d0dc96233a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 1 -- 172.21.15.119:0/1914331743 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300b1070 con 0x7f293406d510 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:54.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: dump: 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 1 -- 172.21.15.119:0/1914331743 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29300a4840 con 0x7f293406d510 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 5 monclient: authenticate success, global_id 84754 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 1 -- 172.21.15.119:0/1914331743 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300a4c10 con 0x7f293406d510 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 4 set_mon_vals no callback set 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293939a640 10 monclient: dump: 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.349 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: shutdown 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/1914331743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406d510 msgr2=0x7f29341347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 --2- 172.21.15.119:0/1914331743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406d510 0x7f29341347b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2934073aa0 tx=0x7f293001c0c0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/1914331743 shutdown_connections 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 --2- 172.21.15.119:0/1914331743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406d510 0x7f29341347b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 --2- 172.21.15.119:0/1914331743 >> v2:172.21.15.17:3300/0 conn(0x7f293406cbf0 0x7f293406cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/1914331743 >> 172.21.15.119:0/1914331743 conn(0x7f29340a40f0 msgr2=0x7f29340a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/1914331743 shutdown_connections 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/1914331743 wait complete. 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: build_initial_monmap 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: monmap: 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.351 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 Processor -- start 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- start start 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: init 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 0x7f29341d0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): picked mon.noname-b con 0x7f293406cbf0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f293406d510 0x7f293413f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): picked mon.noname-c con 0x7f293406d510 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:54.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2934134cf0 0x7f293413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): picked mon.noname-a con 0x7f2934134cf0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f29340a3370 con 0x7f2934134cf0 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f29340a34f0 con 0x7f293406cbf0 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f29341435f0 con 0x7f293406d510 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient(hunting): authenticate will time out at 1473.130231s 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 0x7f29341d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 0x7f29341d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58280/0 (socket says 172.21.15.119:58280) 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 -- 172.21.15.119:0/4025190944 learned_addr learned my addr 172.21.15.119:0/4025190944 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:54.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): get_auth_request con 0x7f293406cbf0 auth_method 0 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2939b9b640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f293406d510 0x7f293413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2939b9b640 10 monclient(hunting): get_auth_request con 0x7f293406d510 auth_method 0 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2939b9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2939b9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2939b9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293ab9d640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2934134cf0 0x7f293413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293ab9d640 10 monclient(hunting): get_auth_request con 0x7f2934134cf0 auth_method 0 2024-09-15T21:25:54.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293ab9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293ab9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293ab9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient(hunting): handle_auth_done global_id 84757 payload 824 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 -- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f293406d510 msgr2=0x7f293413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f293406d510 0x7f293413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 -- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2934134cf0 msgr2=0x7f293413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2934134cf0 0x7f293413f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 monclient: found mon.noname-b 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 -- 172.21.15.119:0/4025190944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293413ab70 con 0x7f293406cbf0 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293ab9d640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2934134cf0 0x7f293413f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293a39c640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 0x7f29341d0920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f292800efc0 tx=0x7f292800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928018020 con 0x7f293406cbf0 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: dump: 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.356 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 5 monclient: authenticate success, global_id 84757 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2928014480 con 0x7f293406cbf0 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928014850 con 0x7f293406cbf0 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:54.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2938b99640 4 set_mon_vals no callback set 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f2926ffd640 10 monclient: dump: 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: _renew_subs 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29340a0d10 con 0x7f293406cbf0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: _renew_subs 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.329+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2934138600 con 0x7f293406cbf0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f292801b020 con 0x7f293406cbf0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f2926ffd640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f291803b9c0 0x7f291803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f293c600640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f293c600640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f293c600640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2934077180 con 0x7f293406cbf0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f2939b9b640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f291803b9c0 0x7f291803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f2939b9b640 10 monclient: get_auth_request con 0x7f291803b9c0 auth_method 0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(256..256 src has 1..256) ==== 7657+0+0 (secure 0 0 0) 0x7f292801d920 con 0x7f293406cbf0 2024-09-15T21:25:54.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.333+0000 7f2939b9b640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f291803b9c0 0x7f291803de80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f29300ae000 tx=0x7f2930008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.337+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f2928022030 con 0x7f293406cbf0 2024-09-15T21:25:54.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.337+0000 7f2926ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.337+0000 7f2926ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4263058748}) 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3022373232}) 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4263058748}) 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3520354885}) 2024-09-15T21:25:54.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1898120999}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1891603467}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1898120999}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.335+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1201582280}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400361234}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=322678282}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400361234}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1759355415}) 2024-09-15T21:25:54.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257451992}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=330871690}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257451992}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3589476445}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.339+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947034724}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1227535404}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947034724}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2491673816}) 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456379483}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2495557881}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1456379483}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1679519631}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=425345090}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4156012816}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=425345090}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=437313714}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3088730191}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=890543996}) 2024-09-15T21:25:54.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3088730191}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.343+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2828270325}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3345378000}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706598905}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3345378000}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3492471485}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=19,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_snap=2,osdmap=29} crc {logm=4213071634,mdsmap=3140514009,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=225831785,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_snap=841190635,osdmap=3819113854}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=18,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=31} crc {logm=1759746223,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1925989952,mon_config_key=2925152189,monmap=186996059,nvmeofgw=1596038039,osd_metadata=2194678291,osd_pg_creating=3530713735,osd_snap=841190635,osdmap=2968096523}) 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=19,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_snap=2,osdmap=29} crc {logm=4213071634,mdsmap=3140514009,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=225831785,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_snap=841190635,osdmap=3819113854}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.347+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=19,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=29} crc {logm=2691383002,mdsmap=173658045,mgr=2659938204,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=225831785,mon_config_key=3131655263,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=3530713735,osd_snap=841190635,osdmap=3819113854}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2727816398}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1143317863}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2727816398}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2705419548}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4055315330}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1605822861}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4055315330}) 2024-09-15T21:25:54.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.351+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=403174706}) 2024-09-15T21:25:54.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.355+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.355+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3177287164}) 2024-09-15T21:25:54.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.355+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1318296051}) 2024-09-15T21:25:54.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.355+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.355+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3177287164}) 2024-09-15T21:25:54.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.355+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2581857809}) 2024-09-15T21:25:54.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.363+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.363+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=55} crc {osdmap=2626794548}) 2024-09-15T21:25:54.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.363+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=56} crc {osdmap=1341362924}) 2024-09-15T21:25:54.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.363+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:25:54.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.363+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=55} crc {osdmap=2626794548}) 2024-09-15T21:25:54.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.363+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=54} crc {osdmap=3071079618}) 2024-09-15T21:25:54.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.517+0000 7f293c600640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:54.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.517+0000 7f293c600640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:54.520 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.517+0000 7f293c600640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.521 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.517+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7f293406c390 con 0x7f293406cbf0 2024-09-15T21:25:54.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.515+0000 7ff55acd2640 -1 mon.a@0(leader).osd e256 definitely_dead 0 2024-09-15T21:25:54.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.653+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 removed osd.12 v257) ==== 83+0+0 (secure 0 0 0) 0x7f2928010040 con 0x7f293406cbf0 2024-09-15T21:25:54.655 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.653+0000 7f2926ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:54.656 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.653+0000 7f2926ffd640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-09-15T21:25:54.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f293c600640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:54.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f293c600640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:54.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f293c600640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.768 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7f2934139370 con 0x7f293406cbf0 2024-09-15T21:25:54.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:54.763+0000 7ff55acd2640 -1 mon.a@0(leader).osd e257 definitely_dead 0 2024-09-15T21:25:54.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f2926ffd640 1 -- 172.21.15.119:0/4025190944 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v257) ==== 92+0+0 (secure 0 0 0) 0x7f2928013e20 con 0x7f293406cbf0 2024-09-15T21:25:54.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f2926ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:54.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.765+0000 7f2926ffd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-09-15T21:25:54.770 INFO:tasks.workunit.client.0.smithi119.stderr:osd.12 does not exist. 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f291803b9c0 msgr2=0x7f291803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f291803b9c0 0x7f291803de80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f29300ae000 tx=0x7f2930008040 comp rx=0 tx=0).stop 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 10 monclient: shutdown 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 msgr2=0x7f29341d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 0x7f29341d0920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f292800efc0 tx=0x7f292800c490 comp rx=0 tx=0).stop 2024-09-15T21:25:54.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 shutdown_connections 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f291803b9c0 0x7f291803de80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2934134cf0 0x7f293413f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f293406d510 0x7f293413f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 --2- 172.21.15.119:0/4025190944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f293406cbf0 0x7f29341d0920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 >> 172.21.15.119:0/4025190944 conn(0x7f29340a40f0 msgr2=0x7f293409ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 shutdown_connections 2024-09-15T21:25:54.774 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.769+0000 7f293c600640 1 -- 172.21.15.119:0/4025190944 wait complete. 2024-09-15T21:25:54.787 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm 12 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.897+0000 7f95448dd640 10 monclient: get_monmap_and_config 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.897+0000 7f95448dd640 10 monclient: build_initial_monmap 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.897+0000 7f95448dd640 10 monclient: monmap: 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:54.904680+0000 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:54.904680+0000 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.903 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:54.904 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:54.904 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:54.904 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:54.904 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:54.904 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 Processor -- start 2024-09-15T21:25:54.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 -- start start 2024-09-15T21:25:54.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient: init 2024-09-15T21:25:54.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.915 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 0x7f95400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): picked mon.noname-b con 0x7f95400a4d80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95400a56a0 0x7f9540137b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): picked mon.noname-a con 0x7f95400a56a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9540138080 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95400a26e0 con 0x7f9540138080 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f95400a6330 con 0x7f95400a56a0 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f95400a30a0 con 0x7f95400a4d80 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 10 monclient(hunting): authenticate will time out at 1473.701696s 2024-09-15T21:25:54.916 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9540138080 0x7f954013efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60204/0 (socket says 172.21.15.119:60204) 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 1 -- 172.21.15.119:0/3747875233 learned_addr learned my addr 172.21.15.119:0/3747875233 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 0x7f95400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 --2- 172.21.15.119:0/3747875233 >> v2:172.21.15.17:3300/0 conn(0x7f95400a56a0 0x7f9540137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): get_auth_request con 0x7f95400a4d80 auth_method 0 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): get_auth_request con 0x7f95400a56a0 auth_method 0 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763854974 0 0) 0x7f95400a26e0 con 0x7f9540138080 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.917 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): _negotiate 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): my global_id is 84760 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9528003530 con 0x7f9540138080 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859835798 0 0) 0x7f9528003530 con 0x7f9540138080 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 msgr2=0x7f95400a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.918 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 0x7f95400a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 >> v2:172.21.15.17:3300/0 conn(0x7f95400a56a0 msgr2=0x7f9540137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 --2- 172.21.15.119:0/3747875233 >> v2:172.21.15.17:3300/0 conn(0x7f95400a56a0 0x7f9540137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95400a6a30 con 0x7f9540138080 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 5 monclient: authenticate success, global_id 84760 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f9538001a80 con 0x7f9540138080 2024-09-15T21:25:54.919 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: dump: 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.920 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 >> v1:172.21.15.119:6789/0 conn(0x7f9540138080 legacy=0x7f954013efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 0x7f9528003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): picked mon.c con 0x7f95400a4d80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9528004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): picked mon.b con 0x7f9528003fe0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9528004d50 0x7f95280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): picked mon.a con 0x7f9528004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f95400a30a0 con 0x7f9528004d50 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f952800ba30 con 0x7f9528003fe0 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f952800bc30 con 0x7f95400a4d80 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953d59a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f95448dd640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9528004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): get_auth_request con 0x7f9528003fe0 auth_method 0 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 0x7f9528003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9528004d50 0x7f95280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): get_auth_request con 0x7f95400a4d80 auth_method 0 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 10 monclient(hunting): get_auth_request con 0x7f9528004d50 auth_method 0 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:25:54.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient(hunting): handle_auth_done global_id 84760 payload 293 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 -- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 msgr2=0x7f9528003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95400a4d80 0x7f9528003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 -- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9528004d50 msgr2=0x7f95280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9528004d50 0x7f95280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 monclient: found mon.b 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 1 -- 172.21.15.119:0/3747875233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9528003530 con 0x7f9528003fe0 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.901+0000 7f953ed9d640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9528004d50 0x7f95280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953dd9b640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9528004810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9534009ae0 tx=0x7f953401ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92cf7d2e9d356b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95340b2070 con 0x7f9528003fe0 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: dump: 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.925 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95340a5680 con 0x7f9528003fe0 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 4 set_mon_vals no callback set 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 1 -- 172.21.15.119:0/3747875233 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95340096b0 con 0x7f9528003fe0 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953d59a640 10 monclient: dump: 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.926 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 10 monclient: shutdown 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 1 -- 172.21.15.119:0/3747875233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 msgr2=0x7f9528004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 1 --2- 172.21.15.119:0/3747875233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9528004810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9534009ae0 tx=0x7f953401ca90 comp rx=0 tx=0).stop 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f953ed9d640 1 -- 172.21.15.119:0/3747875233 reap_dead start 2024-09-15T21:25:54.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 1 -- 172.21.15.119:0/3747875233 shutdown_connections 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 1 -- 172.21.15.119:0/3747875233 >> 172.21.15.119:0/3747875233 conn(0x7f95400a7a00 msgr2=0x7f954006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.905+0000 7f95448dd640 1 -- 172.21.15.119:0/3747875233 shutdown_connections 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- 172.21.15.119:0/3747875233 wait complete. 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: build_initial_monmap 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: monmap: 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:54.928 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 Processor -- start 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- start start 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: init 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9540123060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): picked mon.noname-b con 0x7f9528003fe0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:54.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95401235a0 0x7f9540123a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): picked mon.noname-c con 0x7f95401235a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95401d0d90 0x7f95401d3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): picked mon.noname-a con 0x7f95401d0d90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f95400a1880 con 0x7f95401d0d90 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95400a2400 con 0x7f9528003fe0 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f95400a5a80 con 0x7f95401235a0 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient(hunting): authenticate will time out at 1473.709640s 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9540123060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9540123060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58294/0 (socket says 172.21.15.119:58294) 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 -- 172.21.15.119:0/1430420999 learned_addr learned my addr 172.21.15.119:0/1430420999 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): get_auth_request con 0x7f9528003fe0 auth_method 0 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient(hunting): handle_auth_done global_id 84763 payload 824 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953ed9d640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95401d0d90 0x7f95401d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 -- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95401235a0 msgr2=0x7f9540123a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95401235a0 0x7f9540123a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 -- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95401d0d90 msgr2=0x7f95401d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95401d0d90 0x7f95401d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 monclient: found mon.noname-b 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 -- 172.21.15.119:0/1430420999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9540126010 con 0x7f9528003fe0 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f953e59c640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9540123060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9530011a00 tx=0x7f9530011ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953000fad0 con 0x7f9528003fe0 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: dump: 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.932 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: _finish_auth 0 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953000fc70 con 0x7f9528003fe0 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:54.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530014440 con 0x7f9528003fe0 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 10 monclient: dump: 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:54.934 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952ffff640 4 set_mon_vals no callback set 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 5 monclient: authenticate success, global_id 84763 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _renew_subs 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95400a1570 con 0x7f9528003fe0 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _renew_subs 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9540126e80 con 0x7f9528003fe0 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9530012040 con 0x7f9528003fe0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540126b00 con 0x7f9528003fe0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.909+0000 7f952effd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f952003b9f0 0x7f952003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.913+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7657+0+0 (secure 0 0 0) 0x7f9530050c20 con 0x7f9528003fe0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.913+0000 7f953dd9b640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f952003b9f0 0x7f952003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.913+0000 7f953dd9b640 10 monclient: get_auth_request con 0x7f952003b9f0 auth_method 0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.913+0000 7f953dd9b640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f952003b9f0 0x7f952003deb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95340af000 tx=0x7f9534004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:54.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.917+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9540126b00 con 0x7f9528003fe0 2024-09-15T21:25:54.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.917+0000 7f952effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:54.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:54.917+0000 7f952effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:55.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.097+0000 7f95448dd640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:55.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.097+0000 7f95448dd640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:55.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.097+0000 7f95448dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.101 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.097+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7f954006d560 con 0x7f9528003fe0 2024-09-15T21:25:55.102 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:55.099+0000 7ff55acd2640 -1 mon.a@0(leader).osd e257 definitely_dead 0 2024-09-15T21:25:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.101+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v257) ==== 92+0+0 (secure 0 0 0) 0x7f9530019030 con 0x7f9528003fe0 2024-09-15T21:25:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.101+0000 7f952effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:55.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.101+0000 7f952effd640 10 monclient: _finish_command 2 = system:0 osd.12 does not exist. 2024-09-15T21:25:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f95448dd640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f95448dd640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f95448dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7f9540137b90 con 0x7f9528003fe0 2024-09-15T21:25:55.213 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:55.211+0000 7ff55acd2640 -1 mon.a@0(leader).osd e257 definitely_dead 0 2024-09-15T21:25:55.213 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f952effd640 1 -- 172.21.15.119:0/1430420999 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v257) ==== 92+0+0 (secure 0 0 0) 0x7f9530014d40 con 0x7f9528003fe0 2024-09-15T21:25:55.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f952effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T21:25:55.214 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.209+0000 7f952effd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-09-15T21:25:55.214 INFO:tasks.workunit.client.0.smithi119.stderr:osd.12 does not exist. 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f952003b9f0 msgr2=0x7f952003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f952003b9f0 0x7f952003deb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95340af000 tx=0x7f9534004740 comp rx=0 tx=0).stop 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 10 monclient: shutdown 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 msgr2=0x7f9540123060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9540123060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9530011a00 tx=0x7f9530011ec0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 shutdown_connections 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f952003b9f0 0x7f952003deb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.217 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95401d0d90 0x7f95401d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f95401235a0 0x7f9540123a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 --2- 172.21.15.119:0/1430420999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9528003fe0 0x7f9540123060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 >> 172.21.15.119:0/1430420999 conn(0x7f95400a7a00 msgr2=0x7f9540137420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 shutdown_connections 2024-09-15T21:25:55.218 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.213+0000 7f95448dd640 1 -- 172.21.15.119:0/1430420999 wait complete. 2024-09-15T21:25:55.230 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm 14 2024-09-15T21:25:55.284 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:25:55.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:55.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient: get_monmap_and_config 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient: monmap: 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:55.346943+0000 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:55.346943+0000 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:55.346 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:55.347 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:55.347 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 1 Processor -- start 2024-09-15T21:25:55.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 1 -- start start 2024-09-15T21:25:55.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient: init 2024-09-15T21:25:55.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2024-09-15T21:25:55.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7d4135b80 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:55.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd7d40a0030 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7d413c200 0x7fd7d413e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient(hunting): picked mon.noname-c con 0x7fd7d413c200 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d409fdb0 con 0x7fd7d4135b80 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d40a1b60 con 0x7fd7d40a0030 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d40a1e40 con 0x7fd7d413c200 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d9067640 10 monclient(hunting): authenticate will time out at 1474.143561s 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7d413c200 0x7fd7d413e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd7d40a0030 0x7fd7d413aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60216/0 (socket says 172.21.15.119:60216) 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d259c640 1 -- 172.21.15.119:0/3835952535 learned_addr learned my addr 172.21.15.119:0/3835952535 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): get_auth_request con 0x7fd7d413c200 auth_method 0 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.341+0000 7fd7d359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 10 monclient(hunting): handle_auth_done global_id 84766 payload 274 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 1 -- 172.21.15.119:0/3835952535 >> v1:172.21.15.119:6789/0 conn(0x7fd7d40a0030 legacy=0x7fd7d413aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 1 -- 172.21.15.119:0/3835952535 >> v1:172.21.15.17:6789/0 conn(0x7fd7d4135b80 legacy=0x7fd7d4135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 1 monclient: found mon.noname-c 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 1 -- 172.21.15.119:0/3835952535 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d40a2120 con 0x7fd7d413c200 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d359e640 1 --2- 172.21.15.119:0/3835952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d413c200 0x7fd7d413e5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009d90 tx=0x7fd7cc01d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b23f1127ceb36d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 1 -- 172.21.15.119:0/3835952535 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b2070 con 0x7fd7d413c200 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: dump: 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.355 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 5 monclient: authenticate success, global_id 84766 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 1 -- 172.21.15.119:0/3835952535 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0a58e0 con 0x7fd7d413c200 2024-09-15T21:25:55.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 4 set_mon_vals no callback set 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 1 -- 172.21.15.119:0/3835952535 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0a5cb0 con 0x7fd7d413c200 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d1d9b640 10 monclient: dump: 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.357 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 10 monclient: shutdown 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 -- 172.21.15.119:0/3835952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d413c200 msgr2=0x7fd7d413e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 --2- 172.21.15.119:0/3835952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d413c200 0x7fd7d413e5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009d90 tx=0x7fd7cc01d260 comp rx=0 tx=0).stop 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 -- 172.21.15.119:0/3835952535 shutdown_connections 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 --2- 172.21.15.119:0/3835952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d413c200 0x7fd7d413e5f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 -- 172.21.15.119:0/3835952535 >> 172.21.15.119:0/3835952535 conn(0x7fd7d406cdc0 msgr2=0x7fd7d406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 -- 172.21.15.119:0/3835952535 shutdown_connections 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 -- 172.21.15.119:0/3835952535 wait complete. 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.358 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 10 monclient: monmap: 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.345+0000 7fd7d9067640 1 Processor -- start 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- start start 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: init 2024-09-15T21:25:55.359 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 0x7fd7d411c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): picked mon.noname-b con 0x7fd7d40a0030 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7d4135b80 0x7fd7d41347f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): picked mon.noname-c con 0x7fd7d4135b80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d413c200 0x7fd7d41396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): picked mon.noname-a con 0x7fd7d413c200 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4141760 con 0x7fd7d413c200 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d406dbc0 con 0x7fd7d40a0030 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d41418e0 con 0x7fd7d4135b80 2024-09-15T21:25:55.360 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient(hunting): authenticate will time out at 1474.148520s 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 0x7fd7d411c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 0x7fd7d411c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58314/0 (socket says 172.21.15.119:58314) 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 -- 172.21.15.119:0/190453662 learned_addr learned my addr 172.21.15.119:0/190453662 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): get_auth_request con 0x7fd7d40a0030 auth_method 0 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d359e640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d413c200 0x7fd7d41396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.361 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d359e640 10 monclient(hunting): get_auth_request con 0x7fd7d413c200 auth_method 0 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7d4135b80 0x7fd7d41347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 10 monclient(hunting): get_auth_request con 0x7fd7d4135b80 auth_method 0 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient(hunting): handle_auth_done global_id 84772 payload 824 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 -- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7d4135b80 msgr2=0x7fd7d41347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7d4135b80 0x7fd7d41347f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 -- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d413c200 msgr2=0x7fd7d41396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d413c200 0x7fd7d41396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.362 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 monclient: found mon.noname-b 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 -- 172.21.15.119:0/190453662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d409d1d0 con 0x7fd7d40a0030 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d2d9d640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 0x7fd7d411c990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c530 tx=0x7fd7c400c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4017070 con 0x7fd7d40a0030 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: dump: 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.363 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c40052d0 con 0x7fd7d40a0030 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 5 monclient: authenticate success, global_id 84772 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _renew_subs 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d411deb0 con 0x7fd7d40a0030 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.364 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012870 con 0x7fd7d40a0030 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 10 monclient: dump: 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.365 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _renew_subs 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d159a640 4 set_mon_vals no callback set 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d411d320 con 0x7fd7d40a0030 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d4135f60 con 0x7fd7d40a0030 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd7c401a070 con 0x7fd7d40a0030 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd7c003b9c0 0x7fd7c003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7657+0+0 (secure 0 0 0) 0x7fd7c404f860 con 0x7fd7d40a0030 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd7c003b9c0 0x7fd7c003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 10 monclient: get_auth_request con 0x7fd7c003b9c0 auth_method 0 2024-09-15T21:25:55.366 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.349+0000 7fd7d259c640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd7c003b9c0 0x7fd7c003de80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd7d413b940 tx=0x7fd7c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.353+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd7d409d1d0 con 0x7fd7d40a0030 2024-09-15T21:25:55.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.353+0000 7fd7bb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.367 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.353+0000 7fd7bb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:55.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient: get_monmap_and_config 2024-09-15T21:25:55.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient: monmap: 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:55.416189+0000 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:55.416189+0000 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:55.417 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:55.418 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 Processor -- start 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 -- start start 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient: init 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc07809d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc07809def0 0x7fc078139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): picked mon.noname-b con 0x7fc07809def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc078139ff0 0x7fc07813c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): picked mon.noname-c con 0x7fc078139ff0 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0780a3130 con 0x7fc07809d400 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc0780a1bc0 con 0x7fc078139ff0 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc0780a3490 con 0x7fc07809def0 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc07d999640 10 monclient(hunting): authenticate will time out at 1478.811578s 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc07809def0 0x7fc078139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc076ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc07809d400 0x7fc07809d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48418/0 (socket says 172.21.15.17:48418) 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc076ffd640 1 -- 172.21.15.17:0/3942214396 learned_addr learned my addr 172.21.15.17:0/3942214396 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:55.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 10 monclient(hunting): get_auth_request con 0x7fc07809def0 auth_method 0 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.411+0000 7fc0777fe640 1 --2- 172.21.15.17:0/3942214396 >> v2:172.21.15.119:3300/0 conn(0x7fc078139ff0 0x7fc07813c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0777fe640 10 monclient(hunting): get_auth_request con 0x7fc078139ff0 auth_method 0 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 10 monclient(hunting): handle_auth_done global_id 45323 payload 274 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 1 -- 172.21.15.17:0/3942214396 >> v2:172.21.15.119:3300/0 conn(0x7fc078139ff0 msgr2=0x7fc07813c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 1 --2- 172.21.15.17:0/3942214396 >> v2:172.21.15.119:3300/0 conn(0x7fc078139ff0 0x7fc07813c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 1 -- 172.21.15.17:0/3942214396 >> v1:172.21.15.17:6789/0 conn(0x7fc07809d400 legacy=0x7fc07809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 1 monclient: found mon.noname-b 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 1 -- 172.21.15.17:0/3942214396 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07809e7a0 con 0x7fc07809def0 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc075ffb640 1 -- 172.21.15.17:0/3942214396 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464571694 0 0) 0x7fc0780a3130 con 0x7fc07809d400 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc075ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0777fe640 1 --2- 172.21.15.17:0/3942214396 >> v2:172.21.15.119:3300/0 conn(0x7fc078139ff0 0x7fc07813c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.415+0000 7fc0767fc640 1 --2- 172.21.15.17:0/3942214396 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc07809def0 0x7fc078139ab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097c0 tx=0x7fc06001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ab69397068a43a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 1 -- 172.21.15.17:0/3942214396 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0600b0070 con 0x7fc07809def0 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: dump: 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 1 -- 172.21.15.17:0/3942214396 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0600ab600 con 0x7fc07809def0 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 1 -- 172.21.15.17:0/3942214396 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0600ab9d0 con 0x7fc07809def0 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc075ffb640 10 monclient: dump: 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 5 monclient: authenticate success, global_id 45323 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 4 set_mon_vals no callback set 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 10 monclient: shutdown 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 -- 172.21.15.17:0/3942214396 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc07809def0 msgr2=0x7fc078139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 --2- 172.21.15.17:0/3942214396 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc07809def0 0x7fc078139ab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097c0 tx=0x7fc06001c360 comp rx=0 tx=0).stop 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 -- 172.21.15.17:0/3942214396 shutdown_connections 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 --2- 172.21.15.17:0/3942214396 >> v2:172.21.15.119:3300/0 conn(0x7fc078139ff0 0x7fc07813c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 --2- 172.21.15.17:0/3942214396 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc07809def0 0x7fc078139ab0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 -- 172.21.15.17:0/3942214396 >> 172.21.15.17:0/3942214396 conn(0x7fc0780a66a0 msgr2=0x7fc0780a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 -- 172.21.15.17:0/3942214396 shutdown_connections 2024-09-15T21:25:55.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 -- 172.21.15.17:0/3942214396 wait complete. 2024-09-15T21:25:55.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 10 monclient: monmap: 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 Processor -- start 2024-09-15T21:25:55.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 1 -- start start 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.419+0000 7fc07d999640 10 monclient: init 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07809d400 0x7fc0781c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): picked mon.noname-a con 0x7fc07809d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc07809def0 0x7fc0780738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): picked mon.noname-b con 0x7fc07809def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 0x7fc078073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): picked mon.noname-c con 0x7fc078139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780a0890 con 0x7fc07809d400 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780a0a10 con 0x7fc07809def0 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc0780a0b90 con 0x7fc078139ff0 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient(hunting): authenticate will time out at 1478.820253s 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 0x7fc078073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07809d400 0x7fc0781c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 0x7fc078073e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45666/0 (socket says 172.21.15.17:45666) 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 -- 172.21.15.17:0/1114817802 learned_addr learned my addr 172.21.15.17:0/1114817802 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): get_auth_request con 0x7fc078139ff0 auth_method 0 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): get_auth_request con 0x7fc07809d400 auth_method 0 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc07809def0 0x7fc0780738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): get_auth_request con 0x7fc07809def0 auth_method 0 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient(hunting): handle_auth_done global_id 45326 payload 824 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 -- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc07809def0 msgr2=0x7fc0780738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc07809def0 0x7fc0780738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 -- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07809d400 msgr2=0x7fc0781c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07809d400 0x7fc0781c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 monclient: found mon.noname-c 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 -- 172.21.15.17:0/1114817802 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0781c8960 con 0x7fc078139ff0 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0777fe640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 0x7fc078073e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e840 tx=0x7fc06800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068017070 con 0x7fc078139ff0 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: dump: 2024-09-15T21:25:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 5 monclient: authenticate success, global_id 45326 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _renew_subs 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06800c8d0 con 0x7fc078139ff0 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07809d0f0 con 0x7fc078139ff0 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800f040 con 0x7fc078139ff0 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 10 monclient: dump: 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _renew_subs 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc0757fa640 4 set_mon_vals no callback set 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0780a5950 con 0x7fc078139ff0 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc06801a070 con 0x7fc078139ff0 2024-09-15T21:25:55.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:55.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0780a53e0 con 0x7fc078139ff0 2024-09-15T21:25:55.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc05803b9c0 0x7fc05803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(257..257 src has 1..257) ==== 7657+0+0 (secure 0 0 0) 0x7fc068050400 con 0x7fc078139ff0 2024-09-15T21:25:55.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc05803b9c0 0x7fc05803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 10 monclient: get_auth_request con 0x7fc05803b9c0 auth_method 0 2024-09-15T21:25:55.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.423+0000 7fc076ffd640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc05803b9c0 0x7fc05803de80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc070001940 tx=0x7fc0700018d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.431+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc07809d0f0 con 0x7fc078139ff0 2024-09-15T21:25:55.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.431+0000 7fc06ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.431+0000 7fc06ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.533+0000 7fd7d9067640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T21:25:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.533+0000 7fd7d9067640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T21:25:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.533+0000 7fd7d9067640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.533+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) -- 0x7fd7d411dc30 con 0x7fd7d40a0030 2024-09-15T21:25:55.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:55.535+0000 7ff55acd2640 -1 mon.a@0(leader).osd e257 definitely_dead 0 2024-09-15T21:25:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc07d999640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc07d999640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc07d999640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc07809d7e0 con 0x7fc078139ff0 2024-09-15T21:25:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc06ffff640 1 -- 172.21.15.17:0/1114817802 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fc0781c8960 con 0x7fc078139ff0 2024-09-15T21:25:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc06ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:55.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.667+0000 7fc06ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:55.674 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:55.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.669+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 removed osd.14 v258) ==== 83+0+0 (secure 0 0 0) 0x7fd7c4005480 con 0x7fd7d40a0030 2024-09-15T21:25:55.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.669+0000 7fd7bb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T21:25:55.675 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.669+0000 7fd7bb7fe640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-09-15T21:25:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc05803b9c0 msgr2=0x7fc05803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc05803b9c0 0x7fc05803de80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc070001940 tx=0x7fc0700018d0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 10 monclient: shutdown 2024-09-15T21:25:55.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 msgr2=0x7fc078073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 0x7fc078073e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e840 tx=0x7fc06800ed00 comp rx=0 tx=0).stop 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 shutdown_connections 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc05803b9c0 0x7fc05803de80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc078139ff0 0x7fc078073e30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc07809def0 0x7fc0780738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 --2- 172.21.15.17:0/1114817802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07809d400 0x7fc0781c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 >> 172.21.15.17:0/1114817802 conn(0x7fc0780a66a0 msgr2=0x7fc07813c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 shutdown_connections 2024-09-15T21:25:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.671+0000 7fc07d999640 1 -- 172.21.15.17:0/1114817802 wait complete. 2024-09-15T21:25:55.693 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:25:55.693 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:25:55.693 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:25:55.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:25:55.778 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.773+0000 7fd7d9067640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T21:25:55.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.773+0000 7fd7d9067640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T21:25:55.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.773+0000 7fd7d9067640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.779 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.773+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) -- 0x7fd7d41c6930 con 0x7fd7d40a0030 2024-09-15T21:25:55.779 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:55.775+0000 7ff55acd2640 -1 mon.a@0(leader).osd e258 definitely_dead 0 2024-09-15T21:25:55.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7bb7fe640 1 -- 172.21.15.119:0/190453662 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 osd.14 does not exist. v258) ==== 92+0+0 (secure 0 0 0) 0x7fd7c4003d60 con 0x7fd7d40a0030 2024-09-15T21:25:55.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7bb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T21:25:55.780 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7bb7fe640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-09-15T21:25:55.780 INFO:tasks.workunit.client.0.smithi119.stderr:osd.14 does not exist. 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd7c003b9c0 msgr2=0x7fd7c003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd7c003b9c0 0x7fd7c003de80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd7d413b940 tx=0x7fd7c8009040 comp rx=0 tx=0).stop 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 10 monclient: shutdown 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 msgr2=0x7fd7d411c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 0x7fd7d411c990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c530 tx=0x7fd7c400c9f0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 shutdown_connections 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd7c003b9c0 0x7fd7c003de80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d413c200 0x7fd7d41396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.783 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7d4135b80 0x7fd7d41347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 --2- 172.21.15.119:0/190453662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7d40a0030 0x7fd7d411c990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 >> 172.21.15.119:0/190453662 conn(0x7fd7d406cdc0 msgr2=0x7fd7d413cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 shutdown_connections 2024-09-15T21:25:55.784 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.777+0000 7fd7d9067640 1 -- 172.21.15.119:0/190453662 wait complete. 2024-09-15T21:25:55.797 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd rm 1000 2024-09-15T21:25:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient: get_monmap_and_config 2024-09-15T21:25:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient: monmap: 2024-09-15T21:25:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:55.818979+0000 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:55.818979+0000 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:55.820 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 Processor -- start 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 -- start start 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient: init 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f406409d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f406409def0 0x7f4064139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): picked mon.noname-a con 0x7f406409def0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): picked mon.noname-b con 0x7f4064139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40640a3130 con 0x7f406409d400 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f40640a1bc0 con 0x7f406409def0 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f40640a3490 con 0x7f4064139ff0 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4069d8b640 10 monclient(hunting): authenticate will time out at 1479.215083s 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406413c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45674/0 (socket says 172.21.15.17:45674) 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f40637fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f406409d400 0x7f406409d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48434/0 (socket says 172.21.15.17:48434) 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 1 -- 172.21.15.17:0/1598429089 learned_addr learned my addr 172.21.15.17:0/1598429089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): get_auth_request con 0x7f4064139ff0 auth_method 0 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 1 --2- 172.21.15.17:0/1598429089 >> v2:172.21.15.17:3300/0 conn(0x7f406409def0 0x7f4064139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): get_auth_request con 0x7f406409def0 auth_method 0 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient(hunting): handle_auth_done global_id 104160 payload 274 2024-09-15T21:25:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 1 -- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 msgr2=0x7f406413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 1 --2- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406413c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 1 -- 172.21.15.17:0/1598429089 >> v1:172.21.15.17:6789/0 conn(0x7f406409d400 legacy=0x7f406409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 1 monclient: found mon.noname-a 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4062ffd640 1 -- 172.21.15.17:0/1598429089 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406409e7a0 con 0x7f406409def0 2024-09-15T21:25:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.815+0000 7f4063fff640 1 --2- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406413c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4062ffd640 1 --2- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409def0 0x7f4064139ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40580097c0 tx=0x7f405801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a785f63079b6b48c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 1 -- 172.21.15.17:0/1598429089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580b0070 con 0x7f406409def0 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: dump: 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 5 monclient: authenticate success, global_id 104160 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 1 -- 172.21.15.17:0/1598429089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40580ab600 con 0x7f406409def0 2024-09-15T21:25:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 4 set_mon_vals no callback set 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 1 -- 172.21.15.17:0/1598429089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580ab9d0 con 0x7f406409def0 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f40627fc640 10 monclient: dump: 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 10 monclient: shutdown 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 -- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409def0 msgr2=0x7f4064139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 --2- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409def0 0x7f4064139ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40580097c0 tx=0x7f405801c300 comp rx=0 tx=0).stop 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 -- 172.21.15.17:0/1598429089 shutdown_connections 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 --2- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406413c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 --2- 172.21.15.17:0/1598429089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409def0 0x7f4064139ab0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 -- 172.21.15.17:0/1598429089 >> 172.21.15.17:0/1598429089 conn(0x7f40640a66a0 msgr2=0x7f40640a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 -- 172.21.15.17:0/1598429089 shutdown_connections 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 -- 172.21.15.17:0/1598429089 wait complete. 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 10 monclient: monmap: 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 Processor -- start 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 1 -- start start 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.819+0000 7f4069d8b640 10 monclient: init 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409d400 0x7f406411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): picked mon.noname-a con 0x7f406409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406409def0 0x7f406411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): picked mon.noname-b con 0x7f406409def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406411cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): picked mon.noname-c con 0x7f4064139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f40640a0890 con 0x7f406409d400 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f40640a0a10 con 0x7f406409def0 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f40640a0b90 con 0x7f4064139ff0 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): authenticate will time out at 1479.220316s 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409d400 0x7f406411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409d400 0x7f406411c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54000/0 (socket says 172.21.15.17:54000) 2024-09-15T21:25:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 -- 172.21.15.17:0/2718985979 learned_addr learned my addr 172.21.15.17:0/2718985979 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): get_auth_request con 0x7f406409d400 auth_method 0 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406411cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): get_auth_request con 0x7f4064139ff0 auth_method 0 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406409def0 0x7f406411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): get_auth_request con 0x7f406409def0 auth_method 0 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient(hunting): handle_auth_done global_id 104166 payload 824 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 msgr2=0x7f406411cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4064139ff0 0x7f406411cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406409def0 msgr2=0x7f406411ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f406409def0 0x7f406411ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 monclient: found mon.noname-a 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406411fae0 con 0x7f406409d400 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f40637fe640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409d400 0x7f406411c4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f405400c870 tx=0x7f405400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054017070 con 0x7f406409d400 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: dump: 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405400a5e0 con 0x7f406409d400 2024-09-15T21:25:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 5 monclient: authenticate success, global_id 104166 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient: _renew_subs 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40640a58f0 con 0x7f406409d400 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405400a990 con 0x7f406409d400 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: dump: 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.843 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4061ffb640 4 set_mon_vals no callback set 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient: _renew_subs 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406411d780 con 0x7f406409d400 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 0x7f406406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f406409fc10 con 0x7f406406f230 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f405401a070 con 0x7f406409d400 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 0x7f406406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient: get_auth_request con 0x7f406406f230 auth_method 0 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405003ba40 0x7f405003df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 10 monclient(hunting): handle_auth_done global_id 104169 payload 824 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f406409d7e0 con 0x7f406406f230 2024-09-15T21:25:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7f405404fcc0 con 0x7f406409d400 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4062ffd640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 0x7f406406f610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f406409d0a0 tx=0x7f405801aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405003ba40 0x7f405003df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 10 monclient: get_auth_request con 0x7f405003ba40 auth_method 0 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580bb070 con 0x7f406406f230 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: dump: 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f40580b1040 con 0x7f406406f230 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 msgr2=0x7f406406f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 0x7f406406f610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f406409d0a0 tx=0x7f405801aba0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 msgr2=0x7f406406f610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f404bfff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406f230 0x7f406406f610 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f406409d0a0 tx=0x7f405801aba0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.823+0000 7f4063fff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405003ba40 0x7f405003df00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f405c006cd0 tx=0x7f405c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:55.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 0x7f40641d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f406406d020 con 0x7f406406fb50 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 0x7f40641d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient: get_auth_request con 0x7f406406fb50 auth_method 0 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 10 monclient(hunting): handle_auth_done global_id 104172 payload 824 2024-09-15T21:25:55.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 1 -- 172.21.15.17:0/2718985979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4064132450 con 0x7f406406fb50 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4062ffd640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 0x7f40641d9f60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40580ae040 tx=0x7f4058019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580c3070 con 0x7f406406fb50 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 10 monclient: dump: 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f40580bd2c0 con 0x7f406406fb50 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 msgr2=0x7f40641d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 0x7f40641d9f60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40580ae040 tx=0x7f4058019100 comp rx=0 tx=0).stop 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 msgr2=0x7f40641d9f60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f404bfff640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406406fb50 0x7f40641d9f60 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40580ae040 tx=0x7f4058019100 comp rx=0 tx=0).stop 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 82, 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9579, 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:25:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:55.866 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:55.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405003ba40 msgr2=0x7f405003df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f405003ba40 0x7f405003df00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f405c006cd0 tx=0x7f405c009240 comp rx=0 tx=0).stop 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 10 monclient: shutdown 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409d400 msgr2=0x7f406411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 --2- 172.21.15.17:0/2718985979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406409d400 0x7f406411c4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f405400c870 tx=0x7f405400cd30 comp rx=0 tx=0).stop 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4063fff640 1 -- 172.21.15.17:0/2718985979 reap_dead start 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4063fff640 1 -- 172.21.15.17:0/2718985979 reap_dead start 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 shutdown_connections 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 >> 172.21.15.17:0/2718985979 conn(0x7f40640a66a0 msgr2=0x7f4064134ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 shutdown_connections 2024-09-15T21:25:55.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.831+0000 7f4069d8b640 1 -- 172.21.15.17:0/2718985979 wait complete. 2024-09-15T21:25:55.869 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:55.869 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:25:55.869 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient: get_monmap_and_config 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient: monmap: 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:55.910290+0000 2024-09-15T21:25:55.909 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:55.910290+0000 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 Processor -- start 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 -- start start 2024-09-15T21:25:55.910 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient: init 2024-09-15T21:25:55.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2024-09-15T21:25:55.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa44c135b80 0x7fa44c135f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): picked mon.noname-c con 0x7fa44c135b80 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:55.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa44c1364a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:55.921 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa44c0a0030 0x7fa44c0a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): picked mon.noname-a con 0x7fa44c0a0030 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:55.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c09fdb0 con 0x7fa44c1364a0 2024-09-15T21:25:55.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c0a1b60 con 0x7fa44c0a0030 2024-09-15T21:25:55.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.922 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c0a1e40 con 0x7fa44c135b80 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa451913640 10 monclient(hunting): authenticate will time out at 1474.707057s 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa44c135b80 0x7fa44c135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44a7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa44c1364a0 0x7fa44c0a09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60220/0 (socket says 172.21.15.119:60220) 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa44c135b80 0x7fa44c135f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58320/0 (socket says 172.21.15.119:58320) 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44a7fc640 1 -- 172.21.15.119:0/2220998814 learned_addr learned my addr 172.21.15.119:0/2220998814 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:55.923 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): get_auth_request con 0x7fa44c135b80 auth_method 0 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 1 --2- 172.21.15.119:0/2220998814 >> v2:172.21.15.17:3300/0 conn(0x7fa44c0a0030 0x7fa44c0a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): get_auth_request con 0x7fa44c0a0030 auth_method 0 2024-09-15T21:25:55.924 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient(hunting): handle_auth_done global_id 84784 payload 274 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 -- 172.21.15.119:0/2220998814 >> v2:172.21.15.17:3300/0 conn(0x7fa44c0a0030 msgr2=0x7fa44c0a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 --2- 172.21.15.119:0/2220998814 >> v2:172.21.15.17:3300/0 conn(0x7fa44c0a0030 0x7fa44c0a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 -- 172.21.15.119:0/2220998814 >> v1:172.21.15.119:6789/0 conn(0x7fa44c1364a0 legacy=0x7fa44c0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.925 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa449ffb640 1 -- 172.21.15.119:0/2220998814 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718518281 0 0) 0x7fa44c09fdb0 con 0x7fa44c1364a0 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 monclient: found mon.noname-c 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa44affd640 1 -- 172.21.15.119:0/2220998814 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0a2120 con 0x7fa44c135b80 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.905+0000 7fa449ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa44affd640 1 --2- 172.21.15.119:0/2220998814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c135b80 0x7fa44c135f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa44c074b10 tx=0x7fa43c01c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964aec9848bcd34a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 1 -- 172.21.15.119:0/2220998814 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0b2070 con 0x7fa44c135b80 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: dump: 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.926 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.927 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 1 -- 172.21.15.119:0/2220998814 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c0a9070 con 0x7fa44c135b80 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 5 monclient: authenticate success, global_id 84784 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 4 set_mon_vals no callback set 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 1 -- 172.21.15.119:0/2220998814 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0ad880 con 0x7fa44c135b80 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.928 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa449ffb640 10 monclient: dump: 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.929 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: shutdown 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- 172.21.15.119:0/2220998814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c135b80 msgr2=0x7fa44c135f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 --2- 172.21.15.119:0/2220998814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c135b80 0x7fa44c135f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa44c074b10 tx=0x7fa43c01c340 comp rx=0 tx=0).stop 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- 172.21.15.119:0/2220998814 shutdown_connections 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 --2- 172.21.15.119:0/2220998814 >> v2:172.21.15.17:3300/0 conn(0x7fa44c0a0030 0x7fa44c0a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.930 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 --2- 172.21.15.119:0/2220998814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c135b80 0x7fa44c135f60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- 172.21.15.119:0/2220998814 >> 172.21.15.119:0/2220998814 conn(0x7fa44c06cdc0 msgr2=0x7fa44c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:55.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- 172.21.15.119:0/2220998814 shutdown_connections 2024-09-15T21:25:55.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- 172.21.15.119:0/2220998814 wait complete. 2024-09-15T21:25:55.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.931 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: monmap: 2024-09-15T21:25:55.932 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:55.932 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:55.932 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:55.932 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:55.932 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.932 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 Processor -- start 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- start start 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: init 2024-09-15T21:25:55.933 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa44c0a0030 0x7fa44c11c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient(hunting): picked mon.noname-c con 0x7fa44c0a0030 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c135b80 0x7fa44c11cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient(hunting): picked mon.noname-a con 0x7fa44c135b80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 0x7fa44c11d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient(hunting): picked mon.noname-b con 0x7fa44c1364a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.934 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0a16e0 con 0x7fa44c135b80 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.909+0000 7fa451913640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c140320 con 0x7fa44c1364a0 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c1355c0 con 0x7fa44c0a0030 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient(hunting): authenticate will time out at 1474.712019s 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 0x7fa44c11d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 0x7fa44c11d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58334/0 (socket says 172.21.15.119:58334) 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 -- 172.21.15.119:0/2791660198 learned_addr learned my addr 172.21.15.119:0/2791660198 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): get_auth_request con 0x7fa44c1364a0 auth_method 0 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c135b80 0x7fa44c11cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa44c0a0030 0x7fa44c11c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): get_auth_request con 0x7fa44c0a0030 auth_method 0 2024-09-15T21:25:55.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): get_auth_request con 0x7fa44c135b80 auth_method 0 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.937 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient(hunting): handle_auth_done global_id 84790 payload 824 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 -- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa44c0a0030 msgr2=0x7fa44c11c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa44c0a0030 0x7fa44c11c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 -- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c135b80 msgr2=0x7fa44c11cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.938 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c135b80 0x7fa44c11cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:25:55.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 -- 172.21.15.119:0/2791660198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c11ecf0 con 0x7fa44c1364a0 2024-09-15T21:25:55.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44b7fe640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 0x7fa44c11d0b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa444009d20 tx=0x7fa44400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444018020 con 0x7fa44c1364a0 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: dump: 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.940 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 5 monclient: authenticate success, global_id 84790 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444009b10 con 0x7fa44c1364a0 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.941 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440146c0 con 0x7fa44c1364a0 2024-09-15T21:25:55.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:55.942 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 10 monclient: dump: 2024-09-15T21:25:55.942 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:55.942 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.942 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa4497fa640 4 set_mon_vals no callback set 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient: _renew_subs 2024-09-15T21:25:55.943 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c11efd0 con 0x7fa44c1364a0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient: _renew_subs 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c11ea10 con 0x7fa44c1364a0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c135f60 con 0x7fa44c1364a0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa44401b020 con 0x7fa44c1364a0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43803b9c0 0x7fa43803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7fa444051150 con 0x7fa44c1364a0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43803b9c0 0x7fa43803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.944 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 10 monclient: get_auth_request con 0x7fa43803b9c0 auth_method 0 2024-09-15T21:25:55.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.913+0000 7fa44affd640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43803b9c0 0x7fa43803de80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0af000 tx=0x7fa43c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.917+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa444022070 con 0x7fa44c1364a0 2024-09-15T21:25:55.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.917+0000 7fa433fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:55.945 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:55.917+0000 7fa433fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:55.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.979+0000 7fc99c620640 10 monclient: get_monmap_and_config 2024-09-15T21:25:55.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient: build_initial_monmap 2024-09-15T21:25:55.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient: monmap: 2024-09-15T21:25:55.986 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:55.986216+0000 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:55.986216+0000 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:55.987 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 Processor -- start 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 -- start start 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient: init 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc994073640 0x7fc994073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): picked mon.noname-a con 0x7fc994073640 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073f60 0x7fc994137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): picked mon.noname-b con 0x7fc994073f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc994137bd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9940a0890 con 0x7fc994137bd0 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc9940746e0 con 0x7fc994073640 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940a1640 con 0x7fc994073f60 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99c620640 10 monclient(hunting): authenticate will time out at 1479.381268s 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc994073640 0x7fc994073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073f60 0x7fc994137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99abbd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc994137bd0 0x7fc99413ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48450/0 (socket says 172.21.15.17:48450) 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): get_auth_request con 0x7fc994073640 auth_method 0 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99abbd640 1 -- 172.21.15.17:0/4059126722 learned_addr learned my addr 172.21.15.17:0/4059126722 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:55.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): get_auth_request con 0x7fc994073f60 auth_method 0 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:55.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_done global_id 104181 payload 274 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc9993ba640 1 -- 172.21.15.17:0/4059126722 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625044464 0 0) 0x7fc9940a0890 con 0x7fc994137bd0 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 1 -- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073f60 msgr2=0x7fc994137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 1 --2- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073f60 0x7fc994137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 1 -- 172.21.15.17:0/4059126722 >> v1:172.21.15.17:6789/0 conn(0x7fc994137bd0 legacy=0x7fc99413ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient: _finish_hunting 0 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 1 monclient: found mon.noname-a 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc99a3bc640 1 -- 172.21.15.17:0/4059126722 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940a1920 con 0x7fc994073640 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc999bbb640 1 --2- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073f60 0x7fc994137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.983+0000 7fc9993ba640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 --2- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073640 0x7fc994073a20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9840097c0 tx=0x7fc98401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f19558ad441b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:55.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 1 -- 172.21.15.17:0/4059126722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9840b0070 con 0x7fc994073640 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: dump: 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:55.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: _finish_auth 0 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: _check_auth_tickets 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 5 monclient: authenticate success, global_id 104181 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 1 -- 172.21.15.17:0/4059126722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9840ab600 con 0x7fc994073640 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 1 -- 172.21.15.17:0/4059126722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9840ab9d0 con 0x7fc994073640 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 4 set_mon_vals no callback set 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc9993ba640 10 monclient: dump: 2024-09-15T21:25:55.996 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:55.997 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: shutdown 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073640 msgr2=0x7fc994073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 --2- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073640 0x7fc994073a20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9840097c0 tx=0x7fc98401c3f0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/4059126722 shutdown_connections 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 --2- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073f60 0x7fc994137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 --2- 172.21.15.17:0/4059126722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073640 0x7fc994073a20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/4059126722 >> 172.21.15.17:0/4059126722 conn(0x7fc9940a66a0 msgr2=0x7fc9940a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/4059126722 shutdown_connections 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/4059126722 wait complete. 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: monmap: 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:56.005 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 Processor -- start 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- start start 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: init 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073640 0x7fc9941cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): picked mon.noname-c con 0x7fc994073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073f60 0x7fc99413ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): picked mon.noname-a con 0x7fc994073f60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994137bd0 0x7fc99413f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): picked mon.noname-b con 0x7fc994137bd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc99409ce50 con 0x7fc994073f60 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc99409ccd0 con 0x7fc994137bd0 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940a0d00 con 0x7fc994073640 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient(hunting): authenticate will time out at 1479.386557s 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073640 0x7fc9941cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073f60 0x7fc99413ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073640 0x7fc9941cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45694/0 (socket says 172.21.15.17:45694) 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 -- 172.21.15.17:0/3259008125 learned_addr learned my addr 172.21.15.17:0/3259008125 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073f60 0x7fc99413ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54044/0 (socket says 172.21.15.17:54044) 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): get_auth_request con 0x7fc994073f60 auth_method 0 2024-09-15T21:25:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): get_auth_request con 0x7fc994073640 auth_method 0 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99abbd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994137bd0 0x7fc99413f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient(hunting): handle_auth_done global_id 45341 payload 824 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994137bd0 msgr2=0x7fc99413f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994137bd0 0x7fc99413f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073f60 msgr2=0x7fc99413ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc994073f60 0x7fc99413ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 monclient: found mon.noname-c 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99409c5b0 con 0x7fc994073640 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99abbd640 10 monclient: get_auth_request con 0x7fc994137bd0 auth_method 0 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99abbd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994137bd0 0x7fc99413f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:25:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99a3bc640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073640 0x7fc9941cfcb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc9840097c0 tx=0x7fc98401ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9840b0050 con 0x7fc994073640 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: dump: 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9840b1040 con 0x7fc994073640 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984006e20 con 0x7fc994073640 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 5 monclient: authenticate success, global_id 45341 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: _renew_subs 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99413d570 con 0x7fc994073640 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc998bb9640 4 set_mon_vals no callback set 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc98affd640 10 monclient: dump: 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: _renew_subs 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.987+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99409d0f0 con 0x7fc994073640 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99c620640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99c620640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 0x7fc994139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc99413f800 con 0x7fc994139ab0 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 0x7fc994139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient: get_auth_request con 0x7fc994139ab0 auth_method 0 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc9840be060 con 0x7fc994073640 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 10 monclient(hunting): handle_auth_done global_id 45344 payload 824 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc994073a20 con 0x7fc994139ab0 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc97803b9f0 0x7fc97803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7fc9840ea010 con 0x7fc994073640 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc999bbb640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 0x7fc994139e90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc99413b0a0 tx=0x7fc990011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990019070 con 0x7fc994139ab0 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 10 monclient: dump: 2024-09-15T21:25:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99abbd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc97803b9f0 0x7fc97803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99abbd640 10 monclient: get_auth_request con 0x7fc97803b9f0 auth_method 0 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc990004240 con 0x7fc994139ab0 2024-09-15T21:25:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 msgr2=0x7fc994139e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 0x7fc994139e90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc99413b0a0 tx=0x7fc990011e30 comp rx=0 tx=0).stop 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 msgr2=0x7fc994139e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc98affd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994139ab0 0x7fc994139e90 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc99413b0a0 tx=0x7fc990011e30 comp rx=0 tx=0).stop 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.991+0000 7fc99abbd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc97803b9f0 0x7fc97803deb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0012f0 tx=0x7fc98c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc99c620640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc99c620640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 0x7fc994133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc99c620640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc99413c400 con 0x7fc99413a3d0 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 0x7fc994133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient: get_auth_request con 0x7fc99413a3d0 auth_method 0 2024-09-15T21:25:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.995+0000 7fc999bbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc999bbb640 10 monclient(hunting): handle_auth_done global_id 45347 payload 824 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc999bbb640 1 -- 172.21.15.17:0/3259008125 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc99413b4f0 con 0x7fc99413a3d0 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc999bbb640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 0x7fc994133ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc99413b0a0 tx=0x7fc9900046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990019040 con 0x7fc99413a3d0 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 10 monclient: dump: 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7fc990003c40 con 0x7fc99413a3d0 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 msgr2=0x7fc994133ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 0x7fc994133ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc99413b0a0 tx=0x7fc9900046d0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 msgr2=0x7fc994133ee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc98affd640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc99413a3d0 0x7fc994133ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 82, 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 324597, 2024-09-15T21:25:56.019 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:56.020 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:56.020 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:56.020 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:56.020 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.020 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:25:56.020 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:56.021 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:56.022 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:56.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:56.022 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:56.023 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:56.024 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:56.025 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:56.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:56.026 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:56.026 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:56.027 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:56.027 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:56.027 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:56.027 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:56.027 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.027 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:56.030 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:56.031 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.032 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:56.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:56.035 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:56.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc97803b9f0 msgr2=0x7fc97803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc97803b9f0 0x7fc97803deb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0012f0 tx=0x7fc98c009040 comp rx=0 tx=0).stop 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 10 monclient: shutdown 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073640 msgr2=0x7fc9941cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 --2- 172.21.15.17:0/3259008125 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc994073640 0x7fc9941cfcb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc9840097c0 tx=0x7fc98401ac30 comp rx=0 tx=0).stop 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99abbd640 1 -- 172.21.15.17:0/3259008125 reap_dead start 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99abbd640 1 -- 172.21.15.17:0/3259008125 reap_dead start 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 shutdown_connections 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 >> 172.21.15.17:0/3259008125 conn(0x7fc9940a66a0 msgr2=0x7fc9940a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 shutdown_connections 2024-09-15T21:25:56.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:55.999+0000 7fc99c620640 1 -- 172.21.15.17:0/3259008125 wait complete. 2024-09-15T21:25:56.038 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:56.038 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:25:56.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:25:56.086 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.081+0000 7fa451913640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T21:25:56.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.081+0000 7fa451913640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T21:25:56.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.081+0000 7fa451913640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:56.087 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.081+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) -- 0x7fa44c06c440 con 0x7fa44c1364a0 2024-09-15T21:25:56.087 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:56.083+0000 7ff55acd2640 -1 mon.a@0(leader).osd e258 definitely_dead 0 2024-09-15T21:25:56.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.085+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v258) ==== 96+0+0 (secure 0 0 0) 0x7fa444010040 con 0x7fa44c1364a0 2024-09-15T21:25:56.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.085+0000 7fa433fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T21:25:56.088 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.085+0000 7fa433fff640 10 monclient: _finish_command 2 = system:0 osd.1000 does not exist. 2024-09-15T21:25:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient: get_monmap_and_config 2024-09-15T21:25:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient: monmap: 2024-09-15T21:25:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:56.154890+0000 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:56.154890+0000 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:56.157 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 Processor -- start 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 -- start start 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient: init 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 0x7f3ca409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): picked mon.noname-b con 0x7f3ca409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ca409dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ca4134d70 0x7f3ca413c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): picked mon.noname-c con 0x7f3ca4134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca40a3490 con 0x7f3ca409dd20 2024-09-15T21:25:56.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca40a1bc0 con 0x7f3ca4134d70 2024-09-15T21:25:56.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca40a0320 con 0x7f3ca409d400 2024-09-15T21:25:56.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3cab9f3640 10 monclient(hunting): authenticate will time out at 1479.550200s 2024-09-15T21:25:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 0x7f3ca409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 0x7f3ca409d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45714/0 (socket says 172.21.15.17:45714) 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 -- 172.21.15.17:0/1033887253 learned_addr learned my addr 172.21.15.17:0/1033887253 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): get_auth_request con 0x7f3ca409d400 auth_method 0 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca9f90640 1 --2- 172.21.15.17:0/1033887253 >> v2:172.21.15.119:3300/0 conn(0x7f3ca4134d70 0x7f3ca413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca9f90640 10 monclient(hunting): get_auth_request con 0x7f3ca4134d70 auth_method 0 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca9f90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca9f90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca9f90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient(hunting): handle_auth_done global_id 45350 payload 274 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 -- 172.21.15.17:0/1033887253 >> v2:172.21.15.119:3300/0 conn(0x7f3ca4134d70 msgr2=0x7f3ca413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 --2- 172.21.15.17:0/1033887253 >> v2:172.21.15.119:3300/0 conn(0x7f3ca4134d70 0x7f3ca413c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 -- 172.21.15.17:0/1033887253 >> v1:172.21.15.119:6789/0 conn(0x7f3ca409dd20 legacy=0x7f3ca4134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 monclient: found mon.noname-b 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.151+0000 7f3ca978f640 1 -- 172.21.15.17:0/1033887253 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca409e7a0 con 0x7f3ca409d400 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 1 --2- 172.21.15.17:0/1033887253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 0x7f3ca409d7e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009870 tx=0x7f3ca001ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71362eb973c87ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 1 -- 172.21.15.17:0/1033887253 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00b2070 con 0x7f3ca409d400 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:56.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: dump: 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 1 -- 172.21.15.17:0/1033887253 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00ad640 con 0x7f3ca409d400 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 1 -- 172.21.15.17:0/1033887253 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00ada10 con 0x7f3ca409d400 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 5 monclient: authenticate success, global_id 45350 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 4 set_mon_vals no callback set 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3c9bfff640 10 monclient: dump: 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.173 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: shutdown 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- 172.21.15.17:0/1033887253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 msgr2=0x7f3ca409d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/1033887253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 0x7f3ca409d7e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009870 tx=0x7f3ca001ce50 comp rx=0 tx=0).stop 2024-09-15T21:25:56.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- 172.21.15.17:0/1033887253 shutdown_connections 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/1033887253 >> v2:172.21.15.119:3300/0 conn(0x7f3ca4134d70 0x7f3ca413c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/1033887253 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409d400 0x7f3ca409d7e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- 172.21.15.17:0/1033887253 >> 172.21.15.17:0/1033887253 conn(0x7f3ca40a66a0 msgr2=0x7f3ca40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- 172.21.15.17:0/1033887253 shutdown_connections 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- 172.21.15.17:0/1033887253 wait complete. 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: monmap: 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 Processor -- start 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- start start 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: init 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca409d400 0x7f3ca41cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): picked mon.noname-b con 0x7f3ca409d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409dd20 0x7f3ca411d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): picked mon.noname-c con 0x7f3ca409dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca4134d70 0x7f3ca4133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): picked mon.noname-a con 0x7f3ca4134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca40a32b0 con 0x7f3ca4134d70 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca40a3130 con 0x7f3ca409d400 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca409ccd0 con 0x7f3ca409dd20 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3cab9f3640 10 monclient(hunting): authenticate will time out at 1479.555302s 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409dd20 0x7f3ca411d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409dd20 0x7f3ca411d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45722/0 (socket says 172.21.15.17:45722) 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 -- 172.21.15.17:0/2969477900 learned_addr learned my addr 172.21.15.17:0/2969477900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): get_auth_request con 0x7f3ca409dd20 auth_method 0 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca4134d70 0x7f3ca4133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca409d400 0x7f3ca41cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): get_auth_request con 0x7f3ca4134d70 auth_method 0 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): get_auth_request con 0x7f3ca409d400 auth_method 0 2024-09-15T21:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca9f90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient(hunting): handle_auth_done global_id 45353 payload 824 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca409d400 msgr2=0x7f3ca41cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca409d400 0x7f3ca41cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca4134d70 msgr2=0x7f3ca4133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca4134d70 0x7f3ca4133810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 monclient: found mon.noname-c 2024-09-15T21:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.155+0000 7f3ca8f8e640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca40725e0 con 0x7f3ca409dd20 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca8f8e640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409dd20 0x7f3ca411d9b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400c450 tx=0x7f3c9400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94012830 con 0x7f3ca409dd20 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: dump: 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c94017070 con 0x7f3ca409dd20 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94012c30 con 0x7f3ca409dd20 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: dump: 2024-09-15T21:25:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c9b7fe640 4 set_mon_vals no callback set 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 5 monclient: authenticate success, global_id 45353 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 10 monclient: _renew_subs 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca411f1b0 con 0x7f3ca409dd20 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 10 monclient: _renew_subs 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca4133d50 con 0x7f3ca409dd20 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 0x7f3ca411c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca409d0f0 con 0x7f3ca411c4e0 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3c9401c460 con 0x7f3ca409dd20 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 0x7f3ca411c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient: get_auth_request con 0x7f3ca411c4e0 auth_method 0 2024-09-15T21:25:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803b9c0 0x7f3c8803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7f3c940502f0 con 0x7f3ca409dd20 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 10 monclient(hunting): handle_auth_done global_id 84799 payload 824 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ca4134070 con 0x7f3ca411c4e0 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca9f90640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803b9c0 0x7f3c8803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca9f90640 10 monclient: get_auth_request con 0x7f3c8803b9c0 auth_method 0 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca978f640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 0x7f3ca411c8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3ca406e7b0 tx=0x7f3ca001ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00bd030 con 0x7f3ca411c4e0 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: dump: 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.186 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3ca9f90640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803b9c0 0x7f3c8803de80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3ca411d860 tx=0x7f3c9c009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3ca00adc10 con 0x7f3ca411c4e0 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 msgr2=0x7f3ca411c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 0x7f3ca411c8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3ca406e7b0 tx=0x7f3ca001ac30 comp rx=0 tx=0).stop 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 msgr2=0x7f3ca411c8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.159+0000 7f3c99ffb640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411c4e0 0x7f3ca411c8c0 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3ca406e7b0 tx=0x7f3ca001ac30 comp rx=0 tx=0).stop 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 0x7f3ca413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca409d7e0 con 0x7f3ca411ce00 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 0x7f3ca413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient: get_auth_request con 0x7f3ca411ce00 auth_method 0 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 10 monclient(hunting): handle_auth_done global_id 84802 payload 824 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 1 -- 172.21.15.17:0/2969477900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3ca40a4fa0 con 0x7f3ca411ce00 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca978f640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 0x7f3ca413aa90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001cc90 tx=0x7f3ca0019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00c7070 con 0x7f3ca411ce00 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 10 monclient: dump: 2024-09-15T21:25:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f3ca00c3ce0 con 0x7f3ca411ce00 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 msgr2=0x7f3ca413aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 0x7f3ca413aa90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001cc90 tx=0x7f3ca0019190 comp rx=0 tx=0).stop 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 msgr2=0x7f3ca413aa90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3c99ffb640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca411ce00 0x7f3ca413aa90 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001cc90 tx=0x7f3ca0019190 comp rx=0 tx=0).stop 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 82, 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 108399, 2024-09-15T21:25:56.192 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:56.193 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:56.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:25:56.195 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:25:56.196 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:25:56.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:56.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:56.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:56.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:25:56.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:25:56.202 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:25:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:25:56.204 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:25:56.205 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803b9c0 msgr2=0x7f3c8803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803b9c0 0x7f3c8803de80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3ca411d860 tx=0x7f3c9c009820 comp rx=0 tx=0).stop 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 10 monclient: shutdown 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409dd20 msgr2=0x7f3ca411d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 --2- 172.21.15.17:0/2969477900 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca409dd20 0x7f3ca411d9b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400c450 tx=0x7f3c9400c910 comp rx=0 tx=0).stop 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca9f90640 1 -- 172.21.15.17:0/2969477900 reap_dead start 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3ca9f90640 1 -- 172.21.15.17:0/2969477900 reap_dead start 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 shutdown_connections 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 >> 172.21.15.17:0/2969477900 conn(0x7f3ca40a66a0 msgr2=0x7f3ca40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 shutdown_connections 2024-09-15T21:25:56.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.167+0000 7f3cab9f3640 1 -- 172.21.15.17:0/2969477900 wait complete. 2024-09-15T21:25:56.209 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:25:56.209 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:25:56.209 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:25:56.209 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:25:56.209 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:25:56.210 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:25:56.210 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:25:56.210 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:25:56.187+0000 7ff55acd2640 -1 mon.a@0(leader).osd e258 definitely_dead 0 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa451913640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa451913640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa451913640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) -- 0x7fa44c13c940 con 0x7fa44c1364a0 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa433fff640 1 -- 172.21.15.119:0/2791660198 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v258) ==== 96+0+0 (secure 0 0 0) 0x7fa4440133e0 con 0x7fa44c1364a0 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa433fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.189+0000 7fa433fff640 10 monclient: _finish_command 3 = system:0 osd.1000 does not exist. 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:osd.1000 does not exist. 2024-09-15T21:25:56.211 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43803b9c0 msgr2=0x7fa43803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43803b9c0 0x7fa43803de80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0af000 tx=0x7fa43c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 10 monclient: shutdown 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 msgr2=0x7fa44c11d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 0x7fa44c11d0b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa444009d20 tx=0x7fa44400c490 comp rx=0 tx=0).stop 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 shutdown_connections 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43803b9c0 0x7fa43803de80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c1364a0 0x7fa44c11d0b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c135b80 0x7fa44c11cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 --2- 172.21.15.119:0/2791660198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa44c0a0030 0x7fa44c11c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 >> 172.21.15.119:0/2791660198 conn(0x7fa44c06cdc0 msgr2=0x7fa44c0a47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 shutdown_connections 2024-09-15T21:25:56.212 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.193+0000 7fa451913640 1 -- 172.21.15.119:0/2791660198 wait complete. 2024-09-15T21:25:56.213 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph osd create 79adef04-9b2e-463a-b4b9-a979dfe86845 2024-09-15T21:25:56.221 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:25:56.221 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:25:56.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:56.323 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient: get_monmap_and_config 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient: monmap: 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:25:56.325416+0000 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:25:56.325416+0000 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:56.324 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:56.325 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:56.325 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:56.327 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 Processor -- start 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 -- start start 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient: init 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0930135b80 0x7f0930135f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): picked mon.noname-c con 0x7f0930135b80 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09301364a0 0x7f09300a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): picked mon.noname-a con 0x7f09301364a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f09300a0030 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093009fdb0 con 0x7f09300a0030 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.328 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f09300a1b60 con 0x7f09301364a0 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f09300a1e40 con 0x7f0930135b80 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0937a60640 10 monclient(hunting): authenticate will time out at 1475.122089s 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f09357fc640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f0930135b80 msgr2=0x7f0930135f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f09357fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0930135b80 0x7f0930135f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0935ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f09300a0030 0x7f09300a0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:46910/0 (socket says 172.21.15.119:46910) 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 1 --2- 172.21.15.119:0/2834712460 >> v2:172.21.15.17:3300/0 conn(0x7f09301364a0 0x7f09300a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0935ffd640 1 -- 172.21.15.119:0/2834712460 learned_addr learned my addr 172.21.15.119:0/2834712460 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): get_auth_request con 0x7f09301364a0 auth_method 0 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.329 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient(hunting): handle_auth_done global_id 104193 payload 274 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 1 -- 172.21.15.119:0/2834712460 >> v2:172.21.15.119:3300/0 conn(0x7f0930135b80 msgr2=0x7f0930135f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 1 --2- 172.21.15.119:0/2834712460 >> v2:172.21.15.119:3300/0 conn(0x7f0930135b80 0x7f0930135f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 1 -- 172.21.15.119:0/2834712460 >> v1:172.21.15.17:6789/0 conn(0x7f09300a0030 legacy=0x7f09300a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 1 monclient: found mon.noname-a 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.321+0000 7f0934ffb640 1 -- 172.21.15.119:0/2834712460 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09300a2120 con 0x7f09301364a0 2024-09-15T21:25:56.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0934ffb640 1 --2- 172.21.15.119:0/2834712460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f09300a09e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0930074b10 tx=0x7f092001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e2a5c324575c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 1 -- 172.21.15.119:0/2834712460 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200b0070 con 0x7f09301364a0 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: dump: 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.336 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 5 monclient: authenticate success, global_id 104193 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 1 -- 172.21.15.119:0/2834712460 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09200ab600 con 0x7f09301364a0 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:56.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 4 set_mon_vals no callback set 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 1 -- 172.21.15.119:0/2834712460 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200ab9d0 con 0x7f09301364a0 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0927fff640 10 monclient: dump: 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.338 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: shutdown 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- 172.21.15.119:0/2834712460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 msgr2=0x7f09300a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 --2- 172.21.15.119:0/2834712460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f09300a09e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0930074b10 tx=0x7f092001c3f0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- 172.21.15.119:0/2834712460 shutdown_connections 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 --2- 172.21.15.119:0/2834712460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f09300a09e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 --2- 172.21.15.119:0/2834712460 >> v2:172.21.15.119:3300/0 conn(0x7f0930135b80 0x7f0930135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.339 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- 172.21.15.119:0/2834712460 >> 172.21.15.119:0/2834712460 conn(0x7f093006cdc0 msgr2=0x7f093006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- 172.21.15.119:0/2834712460 shutdown_connections 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- 172.21.15.119:0/2834712460 wait complete. 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: monmap: 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.340 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 Processor -- start 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- start start 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: init 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09300a0030 0x7f093011db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): picked mon.noname-c con 0x7f09300a0030 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0930135b80 0x7f09301349c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): picked mon.noname-b con 0x7f0930135b80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f093011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): picked mon.noname-a con 0x7f09301364a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f09300a16e0 con 0x7f09301364a0 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0930140320 con 0x7f0930135b80 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f09301ce610 con 0x7f09300a0030 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0937a60640 10 monclient(hunting): authenticate will time out at 1475.127365s 2024-09-15T21:25:56.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0934ffb640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0930135b80 msgr2=0x7f09301349c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0934ffb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0930135b80 0x7f09301349c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f093011c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f093011c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:37190/0 (socket says 172.21.15.119:37190) 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 1 -- 172.21.15.119:0/862771478 learned_addr learned my addr 172.21.15.119:0/862771478 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): get_auth_request con 0x7f09301364a0 auth_method 0 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09300a0030 0x7f093011db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): get_auth_request con 0x7f09300a0030 auth_method 0 2024-09-15T21:25:56.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.344 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f0935ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.345 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.325+0000 7f09357fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 10 monclient(hunting): handle_auth_done global_id 104199 payload 824 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 -- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09300a0030 msgr2=0x7f093011db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09300a0030 0x7f093011db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 -- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0930135b80 msgr2=0x7f09301349c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0930135b80 0x7f09301349c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 monclient: found mon.noname-a 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 -- 172.21.15.119:0/862771478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093011f070 con 0x7f09301364a0 2024-09-15T21:25:56.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0935ffd640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f093011c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f092800caa0 tx=0x7f092800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928017070 con 0x7f09301364a0 2024-09-15T21:25:56.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:56.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: dump: 2024-09-15T21:25:56.347 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:56.347 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092800aba0 con 0x7f09301364a0 2024-09-15T21:25:56.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928012400 con 0x7f09301364a0 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 10 monclient: dump: 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.349 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 5 monclient: authenticate success, global_id 104199 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 monclient: _renew_subs 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093011f350 con 0x7f09301364a0 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f09277fe640 4 set_mon_vals no callback set 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 monclient: _renew_subs 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093011eab0 con 0x7f09301364a0 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930135f60 con 0x7f09301364a0 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f092801a070 con 0x7f09301364a0 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f091403b970 0x7f091403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7f0928050570 con 0x7f09301364a0 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f09357fc640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f091403b970 0x7f091403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f09357fc640 10 monclient: get_auth_request con 0x7f091403b970 auth_method 0 2024-09-15T21:25:56.351 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.329+0000 7f09357fc640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f091403b970 0x7f091403de30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f092c0093d0 tx=0x7f092c009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.341+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f092804d440 con 0x7f09301364a0 2024-09-15T21:25:56.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.341+0000 7f0925ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.341+0000 7f0925ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient: get_monmap_and_config 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient: monmap: 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:56.347594+0000 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:56.347594+0000 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 1 Processor -- start 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 1 -- start start 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient: init 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5bd406f080 addr v1:172.21.15.119:6789/0 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5bd40a4050 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd413e370 0x7f5bd4140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient(hunting): picked mon.noname-b con 0x7f5bd413e370 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd4137f50 con 0x7f5bd40a4050 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd40a4590 con 0x7f5bd406f080 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd40a5e50 con 0x7f5bd413e370 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd8f3b640 10 monclient(hunting): authenticate will time out at 1479.742968s 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd259c640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f5bd406f080 legacy=0x7f5bd40a3940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd1d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5bd40a4050 0x7f5bd413cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48458/0 (socket says 172.21.15.17:48458) 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd1d9b640 1 -- 172.21.15.17:0/3418143342 learned_addr learned my addr 172.21.15.17:0/3418143342 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 1 --2- 172.21.15.17:0/3418143342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd413e370 0x7f5bd4140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): get_auth_request con 0x7f5bd413e370 auth_method 0 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd159a640 1 -- 172.21.15.17:0/3418143342 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1156670675 0 0) 0x7f5bbc001120 con 0x7f5bd40a4050 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd159a640 10 monclient(hunting): _negotiate 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd159a640 10 monclient(hunting): my global_id is 104202 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd159a640 1 -- 172.21.15.17:0/3418143342 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc0003530 con 0x7f5bd40a4050 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_done global_id 45359 payload 274 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 1 -- 172.21.15.17:0/3418143342 >> v1:172.21.15.119:6789/0 conn(0x7f5bd406f080 legacy=0x7f5bd40a3940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 1 -- 172.21.15.17:0/3418143342 >> v1:172.21.15.17:6789/0 conn(0x7f5bd40a4050 legacy=0x7f5bd413cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 1 monclient: found mon.noname-b 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.343+0000 7f5bd2d9d640 1 -- 172.21.15.17:0/3418143342 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd40a1e70 con 0x7f5bd413e370 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd2d9d640 1 --2- 172.21.15.17:0/3418143342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd413e370 0x7f5bd4140760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009dc0 tx=0x7f5bcc01c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58b1aec69003965 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 1 -- 172.21.15.17:0/3418143342 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0a66e0 con 0x7f5bd413e370 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: dump: 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 5 monclient: authenticate success, global_id 45359 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 1 -- 172.21.15.17:0/3418143342 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0a6880 con 0x7f5bd413e370 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 4 set_mon_vals no callback set 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 1 -- 172.21.15.17:0/3418143342 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0a6c50 con 0x7f5bd413e370 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd159a640 10 monclient: dump: 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.366 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: shutdown 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/3418143342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd413e370 msgr2=0x7f5bd4140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/3418143342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd413e370 0x7f5bd4140760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009dc0 tx=0x7f5bcc01c850 comp rx=0 tx=0).stop 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/3418143342 shutdown_connections 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/3418143342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd413e370 0x7f5bd4140760 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/3418143342 >> 172.21.15.17:0/3418143342 conn(0x7f5bd40a6c00 msgr2=0x7f5bd40a6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/3418143342 shutdown_connections 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/3418143342 wait complete. 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: build_initial_monmap 2024-09-15T21:25:56.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: monmap: 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 Processor -- start 2024-09-15T21:25:56.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- start start 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: init 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 0x7f5bd413a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): picked mon.noname-c con 0x7f5bd406f080 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd40a4050 0x7f5bd413af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): picked mon.noname-b con 0x7f5bd40a4050 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd413e370 0x7f5bd413b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): picked mon.noname-a con 0x7f5bd413e370 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41436d0 con 0x7f5bd413e370 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd406f460 con 0x7f5bd40a4050 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd4143850 con 0x7f5bd406f080 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.347+0000 7f5bd8f3b640 10 monclient(hunting): authenticate will time out at 1479.747982s 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 0x7f5bd413a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd1d9b640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd40a4050 msgr2=0x7f5bd413af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd413e370 0x7f5bd413b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 0x7f5bd413a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45732/0 (socket says 172.21.15.17:45732) 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 -- 172.21.15.17:0/1995139254 learned_addr learned my addr 172.21.15.17:0/1995139254 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): get_auth_request con 0x7f5bd406f080 auth_method 0 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): get_auth_request con 0x7f5bd413e370 auth_method 0 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd1d9b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd40a4050 0x7f5bd413af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient(hunting): handle_auth_done global_id 45362 payload 824 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 -- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd40a4050 msgr2=0x7f5bd413af00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd40a4050 0x7f5bd413af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 -- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd413e370 msgr2=0x7f5bd413b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd413e370 0x7f5bd413b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient: _finish_hunting 0 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 monclient: found mon.noname-c 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 -- 172.21.15.17:0/1995139254 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd409c6e0 con 0x7f5bd406f080 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd2d9d640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd413e370 0x7f5bd413b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd259c640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 0x7f5bd413a9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400b560 tx=0x7f5bc400ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4013020 con 0x7f5bd406f080 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:25:56.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: dump: 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.373 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: _finish_auth 0 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4003df0 con 0x7f5bd406f080 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 5 monclient: authenticate success, global_id 45362 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 monclient: _renew_subs 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd40a12f0 con 0x7f5bd406f080 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40052b0 con 0x7f5bd406f080 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:25:56.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 10 monclient: dump: 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd0d99640 4 set_mon_vals no callback set 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 monclient: _renew_subs 2024-09-15T21:25:56.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd413c3e0 con 0x7f5bd406f080 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5bc4016020 con 0x7f5bd406f080 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd40a3940 con 0x7f5bd406f080 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5bb403b970 0x7f5bb403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7f5bc40502d0 con 0x7f5bd406f080 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd1d9b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5bb403b970 0x7f5bb403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd1d9b640 10 monclient: get_auth_request con 0x7f5bb403b970 auth_method 0 2024-09-15T21:25:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.351+0000 7f5bd1d9b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5bb403b970 0x7f5bb403de30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc007fd0 tx=0x7f5bbc007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:56.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.355+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5bc401d020 con 0x7f5bd406f080 2024-09-15T21:25:56.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.355+0000 7f5bcaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:56.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.355+0000 7f5bcaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:56.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.509+0000 7f0937a60640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:56.512 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.509+0000 7f0937a60640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:25:56.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.509+0000 7f0937a60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:56.513 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:56.509+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"} v 0) -- 0x7f093006c440 con 0x7f09301364a0 2024-09-15T21:25:56.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bd8f3b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:56.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bd8f3b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:56.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bd8f3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:25:56.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5bd409bf40 con 0x7f5bd406f080 2024-09-15T21:25:56.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bcaffd640 1 -- 172.21.15.17:0/1995139254 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f5bc400d380 con 0x7f5bd406f080 2024-09-15T21:25:56.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bcaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:56.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.591+0000 7f5bcaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:56.596 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5bb403b970 msgr2=0x7f5bb403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5bb403b970 0x7f5bb403de30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc007fd0 tx=0x7f5bbc007900 comp rx=0 tx=0).stop 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 10 monclient: shutdown 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 msgr2=0x7f5bd413a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 0x7f5bd413a9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400b560 tx=0x7f5bc400ba20 comp rx=0 tx=0).stop 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 shutdown_connections 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5bb403b970 0x7f5bb403de30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd413e370 0x7f5bd413b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd40a4050 0x7f5bd413af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 --2- 172.21.15.17:0/1995139254 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5bd406f080 0x7f5bd413a9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:56.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 >> 172.21.15.17:0/1995139254 conn(0x7f5bd40a6c00 msgr2=0x7f5bd406dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:56.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 shutdown_connections 2024-09-15T21:25:56.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:56.595+0000 7f5bd8f3b640 1 -- 172.21.15.17:0/1995139254 wait complete. 2024-09-15T21:25:59.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:59.325+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:25:59.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:59.325+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:25:59.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:59.325+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:59.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:25:59.325+0000 7f0924ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:59.616 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient: get_monmap_and_config 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient: build_initial_monmap 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient: monmap: 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:25:59.747847+0000 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:25:59.747847+0000 2024-09-15T21:25:59.748 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:25:59.749 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 Processor -- start 2024-09-15T21:25:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 -- start start 2024-09-15T21:25:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient: init 2024-09-15T21:25:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd69c0a28b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd69c0a33a0 0x7fd69c13ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): picked mon.noname-a con 0x7fd69c0a33a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd69c13b070 0x7fd69c13d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): picked mon.noname-c con 0x7fd69c13b070 addr v2:172.21.15.119:3300/0 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd69c09f1d0 con 0x7fd69c0a28b0 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c06d0d0 con 0x7fd69c0a33a0 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c09ec10 con 0x7fd69c13b070 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.743+0000 7fd6a416d640 10 monclient(hunting): authenticate will time out at 1483.143871s 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a270a640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fd69c13b070 msgr2=0x7fd69c13d460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a270a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd69c13b070 0x7fd69c13d460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd69c0a33a0 0x7fd69c13ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1f09640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd69c0a28b0 0x7fd69c0a2c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43786/0 (socket says 172.21.15.17:43786) 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1f09640 1 -- 172.21.15.17:0/2260355343 learned_addr learned my addr 172.21.15.17:0/2260355343 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): get_auth_request con 0x7fd69c0a33a0 auth_method 0 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient(hunting): handle_auth_done global_id 104208 payload 274 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 -- 172.21.15.17:0/2260355343 >> v2:172.21.15.119:3300/0 conn(0x7fd69c13b070 msgr2=0x7fd69c13d460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 --2- 172.21.15.17:0/2260355343 >> v2:172.21.15.119:3300/0 conn(0x7fd69c13b070 0x7fd69c13d460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 -- 172.21.15.17:0/2260355343 >> v1:172.21.15.17:6789/0 conn(0x7fd69c0a28b0 legacy=0x7fd69c0a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient: _finish_hunting 0 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 monclient: found mon.noname-a 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 -- 172.21.15.17:0/2260355343 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c09e410 con 0x7fd69c0a33a0 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 1 -- 172.21.15.17:0/2260355343 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2051661588 0 0) 0x7fd69c09f1d0 con 0x7fd69c0a28b0 2024-09-15T21:25:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a1708640 1 --2- 172.21.15.17:0/2260355343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a33a0 0x7fd69c13ab30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0097c0 tx=0x7fd68c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d109aa765f1d0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 1 -- 172.21.15.17:0/2260355343 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0b1070 con 0x7fd69c0a33a0 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: dump: 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:59.759 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: _finish_auth 0 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: _check_auth_tickets 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 1 -- 172.21.15.17:0/2260355343 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c0ac640 con 0x7fd69c0a33a0 2024-09-15T21:25:59.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 1 -- 172.21.15.17:0/2260355343 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0aca10 con 0x7fd69c0a33a0 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a0f07640 10 monclient: dump: 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:59.761 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 5 monclient: authenticate success, global_id 104208 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 10 monclient: get_monmap_and_config success 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 4 set_mon_vals no callback set 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 10 monclient: shutdown 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 -- 172.21.15.17:0/2260355343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a33a0 msgr2=0x7fd69c13ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 --2- 172.21.15.17:0/2260355343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a33a0 0x7fd69c13ab30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0097c0 tx=0x7fd68c01c310 comp rx=0 tx=0).stop 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 -- 172.21.15.17:0/2260355343 shutdown_connections 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 --2- 172.21.15.17:0/2260355343 >> v2:172.21.15.119:3300/0 conn(0x7fd69c13b070 0x7fd69c13d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 --2- 172.21.15.17:0/2260355343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a33a0 0x7fd69c13ab30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 -- 172.21.15.17:0/2260355343 >> 172.21.15.17:0/2260355343 conn(0x7fd69c0a74a0 msgr2=0x7fd69c0a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:25:59.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 -- 172.21.15.17:0/2260355343 shutdown_connections 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.747+0000 7fd6a416d640 1 -- 172.21.15.17:0/2260355343 wait complete. 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: build_initial_monmap 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: monmap: 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:25:59.763 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 Processor -- start 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- start start 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: init 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _reopen_session rank -1 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 0x7fd69c1cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): picked mon.noname-a con 0x7fd69c0a28b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd69c0a33a0 0x7fd69c132820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): picked mon.noname-c con 0x7fd69c0a33a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69c13b070 0x7fd69c136b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): picked mon.noname-b con 0x7fd69c13b070 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:59.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c09fd30 con 0x7fd69c0a28b0 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c09fbb0 con 0x7fd69c13b070 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c1401c0 con 0x7fd69c0a33a0 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): _renew_subs 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient(hunting): authenticate will time out at 1483.149061s 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 0x7fd69c1cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 0x7fd69c1cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55924/0 (socket says 172.21.15.17:55924) 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 -- 172.21.15.17:0/432084175 learned_addr learned my addr 172.21.15.17:0/432084175 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): get_auth_request con 0x7fd69c0a28b0 auth_method 0 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:59.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd69c0a33a0 0x7fd69c132820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a270a640 1 -- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69c13b070 msgr2=0x7fd69c136b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): get_auth_request con 0x7fd69c0a33a0 auth_method 0 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a270a640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69c13b070 0x7fd69c136b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:25:59.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient(hunting): handle_auth_done global_id 104214 payload 824 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 -- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd69c0a33a0 msgr2=0x7fd69c132820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd69c0a33a0 0x7fd69c132820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 -- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69c13b070 msgr2=0x7fd69c136b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69c13b070 0x7fd69c136b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient: _finish_hunting 0 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 monclient: found mon.noname-a 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 -- 172.21.15.17:0/432084175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c13a5e0 con 0x7fd69c0a28b0 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1f09640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 0x7fd69c1cd510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c870 tx=0x7fd69800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698018070 con 0x7fd69c0a28b0 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:25:59.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: dump: 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: _finish_auth 0 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: _check_auth_tickets 2024-09-15T21:25:59.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69800a5e0 con 0x7fd69c0a28b0 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 5 monclient: authenticate success, global_id 104214 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _renew_subs 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c09e100 con 0x7fd69c0a28b0 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69800a9d0 con 0x7fd69c0a28b0 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 10 monclient: dump: 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.769 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd693fff640 4 set_mon_vals no callback set 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _renew_subs 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c138560 con 0x7fd69c0a28b0 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:59.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd69c0a2c90 con 0x7fd69c0a28b0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd69801b070 con 0x7fd69c0a28b0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd68003b970 0x7fd68003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7fd69804fc10 con 0x7fd69c0a28b0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd68003b970 0x7fd68003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.751+0000 7fd6a1708640 10 monclient: get_auth_request con 0x7fd68003b970 auth_method 0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.755+0000 7fd6a1708640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd68003b970 0x7fd68003de30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0ae000 tx=0x7fd68c0a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:25:59.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.755+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd698010300 con 0x7fd69c0a28b0 2024-09-15T21:25:59.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.755+0000 7fd6927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:25:59.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.755+0000 7fd6927fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:25:59.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6a416d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:25:59.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6a416d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:59.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6a416d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:25:59.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd69c13ae50 con 0x7fd69c0a28b0 2024-09-15T21:25:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6927fc640 1 -- 172.21.15.17:0/432084175 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fd698007980 con 0x7fd69c0a28b0 2024-09-15T21:25:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:25:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.991+0000 7fd6927fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:25:59.997 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:25:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd68003b970 msgr2=0x7fd68003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:25:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd68003b970 0x7fd68003de30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0ae000 tx=0x7fd68c0a6070 comp rx=0 tx=0).stop 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 10 monclient: shutdown 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 msgr2=0x7fd69c1cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 0x7fd69c1cd510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c870 tx=0x7fd69800cd30 comp rx=0 tx=0).stop 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 shutdown_connections 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd68003b970 0x7fd68003de30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69c13b070 0x7fd69c136b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd69c0a33a0 0x7fd69c132820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 --2- 172.21.15.17:0/432084175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd69c0a28b0 0x7fd69c1cd510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 >> 172.21.15.17:0/432084175 conn(0x7fd69c0a74a0 msgr2=0x7fd69c135870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 shutdown_connections 2024-09-15T21:26:00.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:25:59.995+0000 7fd6a416d640 1 -- 172.21.15.17:0/432084175 wait complete. 2024-09-15T21:26:00.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:00.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:00.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:00.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:00.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:00.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:01.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:01.325+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:01.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:01.325+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:01.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:01.325+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:02.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:02.325+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:02.330 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:02.325+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:02.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:02.325+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:03.015 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient: get_monmap_and_config 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient: build_initial_monmap 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient: monmap: 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:03.147770+0000 2024-09-15T21:26:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:03.147770+0000 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:03.149 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 Processor -- start 2024-09-15T21:26:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 -- start start 2024-09-15T21:26:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient: init 2024-09-15T21:26:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-09-15T21:26:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7041340c0 0x7fe7041344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): picked mon.noname-c con 0x7fe7041340c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe70409d6c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7040a0fa0 0x7fe70413be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): picked mon.noname-a con 0x7fe7040a0fa0 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7041349e0 con 0x7fe70409d6c0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe70409d520 con 0x7fe7040a0fa0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe70409e1b0 con 0x7fe7041340c0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe709d29640 10 monclient(hunting): authenticate will time out at 1486.543406s 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe7037fe640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fe7041340c0 msgr2=0x7fe7041344a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe7037fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7041340c0 0x7fe7041344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7040a0fa0 0x7fe70413be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe702ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe70409d6c0 0x7fe7040a0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43800/0 (socket says 172.21.15.17:43800) 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe702ffd640 1 -- 172.21.15.17:0/1514071194 learned_addr learned my addr 172.21.15.17:0/1514071194 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): get_auth_request con 0x7fe7040a0fa0 auth_method 0 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 10 monclient(hunting): handle_auth_done global_id 104217 payload 274 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.143+0000 7fe703fff640 1 -- 172.21.15.17:0/1514071194 >> v2:172.21.15.119:3300/0 conn(0x7fe7041340c0 msgr2=0x7fe7041344a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 1 --2- 172.21.15.17:0/1514071194 >> v2:172.21.15.119:3300/0 conn(0x7fe7041340c0 0x7fe7041344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 1 -- 172.21.15.17:0/1514071194 >> v1:172.21.15.17:6789/0 conn(0x7fe70409d6c0 legacy=0x7fe7040a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 1 monclient: found mon.noname-a 2024-09-15T21:26:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 1 -- 172.21.15.17:0/1514071194 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70409e490 con 0x7fe7040a0fa0 2024-09-15T21:26:03.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe703fff640 1 --2- 172.21.15.17:0/1514071194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7040a0fa0 0x7fe70413be70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe70409d0a0 tx=0x7fe6f401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f4acd7bb68a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 1 -- 172.21.15.17:0/1514071194 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40b2070 con 0x7fe7040a0fa0 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: dump: 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 1 -- 172.21.15.17:0/1514071194 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f40a5760 con 0x7fe7040a0fa0 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe709d29640 5 monclient: authenticate success, global_id 104217 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe709d29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe709d29640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 1 -- 172.21.15.17:0/1514071194 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40a5b30 con 0x7fe7040a0fa0 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe709d29640 4 set_mon_vals no callback set 2024-09-15T21:26:03.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.147+0000 7fe7027fc640 10 monclient: dump: 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:03.161 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:03.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: shutdown 2024-09-15T21:26:03.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:03.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/1514071194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7040a0fa0 msgr2=0x7fe70413be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 --2- 172.21.15.17:0/1514071194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7040a0fa0 0x7fe70413be70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe70409d0a0 tx=0x7fe6f401c810 comp rx=0 tx=0).stop 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/1514071194 shutdown_connections 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 --2- 172.21.15.17:0/1514071194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7040a0fa0 0x7fe70413be70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 --2- 172.21.15.17:0/1514071194 >> v2:172.21.15.119:3300/0 conn(0x7fe7041340c0 0x7fe7041344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/1514071194 >> 172.21.15.17:0/1514071194 conn(0x7fe7040a66a0 msgr2=0x7fe7040a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/1514071194 shutdown_connections 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/1514071194 wait complete. 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: build_initial_monmap 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: monmap: 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:03.163 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 Processor -- start 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- start start 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: init 2024-09-15T21:26:03.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70409d6c0 0x7fe7041d0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): picked mon.noname-b con 0x7fe70409d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7040a0fa0 0x7fe7041d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): picked mon.noname-c con 0x7fe7040a0fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 0x7fe70411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): picked mon.noname-a con 0x7fe7041340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe70413c3b0 con 0x7fe7041340c0 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7040a27a0 con 0x7fe70409d6c0 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:03.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe70413c530 con 0x7fe7040a0fa0 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient(hunting): authenticate will time out at 1486.550338s 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 0x7fe70411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe7037fe640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70409d6c0 msgr2=0x7fe7041d0500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe7037fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70409d6c0 0x7fe7041d0500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 0x7fe70411c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55944/0 (socket says 172.21.15.17:55944) 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 -- 172.21.15.17:0/218555198 learned_addr learned my addr 172.21.15.17:0/218555198 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): get_auth_request con 0x7fe7041340c0 auth_method 0 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7040a0fa0 0x7fe7041d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:03.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): get_auth_request con 0x7fe7040a0fa0 auth_method 0 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe702ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient(hunting): handle_auth_done global_id 104220 payload 824 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 -- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7040a0fa0 msgr2=0x7fe7041d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7040a0fa0 0x7fe7041d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 -- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70409d6c0 msgr2=0x7fe7041d0500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70409d6c0 0x7fe7041d0500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 monclient: found mon.noname-a 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 -- 172.21.15.17:0/218555198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70411eb00 con 0x7fe7041340c0 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe703fff640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 0x7fe70411c4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40b0040 tx=0x7fe6f40195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40b2050 con 0x7fe7041340c0 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: dump: 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:03.168 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: _finish_auth 0 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f40be070 con 0x7fe7041340c0 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 5 monclient: authenticate success, global_id 104220 2024-09-15T21:26:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: _renew_subs 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe704073c00 con 0x7fe7041340c0 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40ada70 con 0x7fe7041340c0 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe6fbfff640 10 monclient: dump: 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:03.171 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe701ffb640 4 set_mon_vals no callback set 2024-09-15T21:26:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: _renew_subs 2024-09-15T21:26:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.151+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70411fbe0 con 0x7fe7041340c0 2024-09-15T21:26:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe709d29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe709d29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:03.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe709d29640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:03.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7041344a0 con 0x7fe7041340c0 2024-09-15T21:26:03.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe6f40a55e0 con 0x7fe7041340c0 2024-09-15T21:26:03.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe6fbfff640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6e403b970 0x7fe6e403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:03.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(258..258 src has 1..258) ==== 7657+0+0 (secure 0 0 0) 0x7fe6f40eb790 con 0x7fe7041340c0 2024-09-15T21:26:03.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe7037fe640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6e403b970 0x7fe6e403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe7037fe640 10 monclient: get_auth_request con 0x7fe6e403b970 auth_method 0 2024-09-15T21:26:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.155+0000 7fe7037fe640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6e403b970 0x7fe6e403de30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc003940 tx=0x7fe6fc006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.159+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe6f4016520 con 0x7fe7041340c0 2024-09-15T21:26:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.159+0000 7fe6fbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.159+0000 7fe6fbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:03.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:03.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:03.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:03.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:03.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:03.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.391+0000 7fe709d29640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.391+0000 7fe709d29640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.391+0000 7fe709d29640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.391+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe70411f290 con 0x7fe7041340c0 2024-09-15T21:26:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe6fbfff640 1 -- 172.21.15.17:0/218555198 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fe6f40c3460 con 0x7fe7041340c0 2024-09-15T21:26:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe6fbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe6fbfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:03.399 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6e403b970 msgr2=0x7fe6e403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6e403b970 0x7fe6e403de30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc003940 tx=0x7fe6fc006d90 comp rx=0 tx=0).stop 2024-09-15T21:26:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 10 monclient: shutdown 2024-09-15T21:26:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 msgr2=0x7fe70411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 0x7fe70411c4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40b0040 tx=0x7fe6f40195b0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 shutdown_connections 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe6e403b970 0x7fe6e403de30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7041340c0 0x7fe70411c4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7040a0fa0 0x7fe7041d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 --2- 172.21.15.17:0/218555198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70409d6c0 0x7fe7041d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 >> 172.21.15.17:0/218555198 conn(0x7fe7040a66a0 msgr2=0x7fe7040a3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 shutdown_connections 2024-09-15T21:26:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:03.395+0000 7fe709d29640 1 -- 172.21.15.17:0/218555198 wait complete. 2024-09-15T21:26:04.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:04.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:04.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:04.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:04.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:04.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:05.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:05.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:05.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:05.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:05.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:05.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:06.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:06.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:06.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:06.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:06.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:06.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:06.418 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient: get_monmap_and_config 2024-09-15T21:26:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient: build_initial_monmap 2024-09-15T21:26:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient: monmap: 2024-09-15T21:26:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:06.548073+0000 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:06.548073+0000 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:06.549 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 Processor -- start 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 -- start start 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient: init 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): picked mon.noname-b con 0x7f6dd40a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6dd409ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 0x7f6dd4135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): picked mon.noname-c con 0x7f6dd40a11b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd40a3130 con 0x7f6dd409ffc0 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd4073df0 con 0x7f6dd40a11b0 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd40a1bf0 con 0x7f6dd40a0890 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd9ac6640 10 monclient(hunting): authenticate will time out at 1489.943448s 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd2ffd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f6dd409ffc0 legacy=0x7f6dd4135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:06.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd3fff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 msgr2=0x7f6dd4135d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45890/0 (socket says 172.21.15.17:45890) 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 learned_addr learned my addr 172.21.15.17:0/1489155889 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 0x7f6dd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request con 0x7f6dd40a0890 auth_method 0 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.543+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:06.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v1:172.21.15.119:6789/0 conn(0x7f6dd409ffc0 legacy=0x7f6dd4135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd3fff640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 msgr2=0x7f6dd4135d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 0x7f6dd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request con 0x7f6dd40a0890 auth_method 0 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:06.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:06.747+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:26:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v1:172.21.15.119:6789/0 conn(0x7f6dd409ffc0 legacy=0x7f6dd4135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:26:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd3fff640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 msgr2=0x7f6dd4135d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 0x7f6dd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request con 0x7f6dd40a0890 auth_method 0 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:07.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.151+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:26:07.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:07.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:07.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:07.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:07.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:07.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:07.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v1:172.21.15.119:6789/0 conn(0x7f6dd409ffc0 legacy=0x7f6dd4135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd3fff640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 msgr2=0x7f6dd4135d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 0x7f6dd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request con 0x7f6dd40a0890 auth_method 0 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:07.955+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:26:08.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:08.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:08.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:08.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:08.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:08.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:09.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:09.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:09.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:09.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:09.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:09.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:09.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:09.329+0000 7f0924ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:09.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): tick 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 msgr2=0x7f6dd40a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dd40a0890 0x7f6dd40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 msgr2=0x7f6dd4135d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.119:3300/0 conn(0x7f6dd40a11b0 0x7f6dd4135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 -- 172.21.15.17:0/1489155889 >> v1:172.21.15.119:6789/0 conn(0x7f6dd409ffc0 legacy=0x7f6dd4135670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:09.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient: _add_conns ranks=[0,3] 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6db40036b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): picked mon.noname-a con 0x7f6db4007400 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 -- 172.21.15.17:0/1489155889 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd40a15e0 con 0x7f6db40036b0 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 1 -- 172.21.15.17:0/1489155889 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd4073df0 con 0x7f6db4007400 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.543+0000 7f6dd17fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request con 0x7f6db4007400 auth_method 0 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:09.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:09.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:09.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:26:09.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:09.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:09.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.547+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request con 0x7f6db4007400 auth_method 0 2024-09-15T21:26:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:09.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:26:09.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:09.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:09.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:09.751+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:26:10.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request con 0x7f6db4007400 auth_method 0 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:10.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.155+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:26:10.331 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:10.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:10.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:10.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:10.332 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:10.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.703+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356864646 0 0) 0x7f6dd40a15e0 con 0x7f6db40036b0 2024-09-15T21:26:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.703+0000 7f6dd27fc640 10 monclient(hunting): _negotiate 2024-09-15T21:26:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.703+0000 7f6dd27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.703+0000 7f6dd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.703+0000 7f6dd27fc640 10 monclient(hunting): my global_id is 104223 2024-09-15T21:26:10.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.703+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dc0003530 con 0x7f6db40036b0 2024-09-15T21:26:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3993900110 0 0) 0x7f6dc0003530 con 0x7f6db40036b0 2024-09-15T21:26:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 msgr2=0x7f6db40098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 --2- 172.21.15.17:0/1489155889 >> v2:172.21.15.17:3300/0 conn(0x7f6db4007400 0x7f6db40098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: _finish_hunting 0 2024-09-15T21:26:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd40a3130 con 0x7f6db40036b0 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f6dc400f040 con 0x7f6db40036b0 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd9ac6640 5 monclient: authenticate success, global_id 104223 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd9ac6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: dump: 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:10.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 >> v1:172.21.15.17:6789/0 conn(0x7f6db40036b0 legacy=0x7f6db4005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd3fff640 1 -- 172.21.15.17:0/1489155889 reap_dead start 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc00039c0 0x7f6dc0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): picked mon.b con 0x7f6dc00039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): picked mon.a con 0x7f6dc0004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00052a0 0x7f6dc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): picked mon.c con 0x7f6dc00052a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4073df0 con 0x7f6dc0004810 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc000c1f0 con 0x7f6dc00039c0 2024-09-15T21:26:10.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc000c3f0 con 0x7f6dc00052a0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd27fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd9ac6640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.707+0000 7f6dd37fe640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc00039c0 msgr2=0x7f6dc0003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc00039c0 0x7f6dc0003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00052a0 0x7f6dc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): get_auth_request con 0x7f6dc00052a0 auth_method 0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request con 0x7f6dc0004810 auth_method 0 2024-09-15T21:26:10.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient(hunting): handle_auth_done global_id 104223 payload 293 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00052a0 msgr2=0x7f6dc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00052a0 0x7f6dc000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc00039c0 msgr2=0x7f6dc0003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc00039c0 0x7f6dc0003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 monclient: found mon.a 2024-09-15T21:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 1 -- 172.21.15.17:0/1489155889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc0003530 con 0x7f6dc0004810 2024-09-15T21:26:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:26:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.711+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00052a0 0x7f6dc000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0114f0 tx=0x7f6dbc01dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e20e0b2687da1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0ac070 con 0x7f6dc0004810 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: dump: 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0a6790 con 0x7f6dc0004810 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 1 -- 172.21.15.17:0/1489155889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc007d30 con 0x7f6dc0004810 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 4 set_mon_vals no callback set 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd27fc640 10 monclient: dump: 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 10 monclient: shutdown 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc0004810 msgr2=0x7f6dc0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0114f0 tx=0x7f6dbc01dc50 comp rx=0 tx=0).stop 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/1489155889 shutdown_connections 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00052a0 0x7f6dc000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/1489155889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc00039c0 0x7f6dc0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/1489155889 >> 172.21.15.17:0/1489155889 conn(0x7f6dd40a66a0 msgr2=0x7f6dd40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/1489155889 shutdown_connections 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/1489155889 wait complete. 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 10 monclient: build_initial_monmap 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.719+0000 7f6dd9ac6640 10 monclient: monmap: 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 Processor -- start 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- start start 2024-09-15T21:26:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: init 2024-09-15T21:26:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00039c0 0x7f6dd41cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): picked mon.noname-c con 0x7f6dc00039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc0004810 0x7f6dd41cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): picked mon.noname-b con 0x7f6dc0004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 0x7f6dd409d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): picked mon.noname-a con 0x7f6dc00052a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4136210 con 0x7f6dc00052a0 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4136090 con 0x7f6dc0004810 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd4136390 con 0x7f6dc00039c0 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient(hunting): authenticate will time out at 1494.121297s 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 0x7f6dd409d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 0x7f6dd409d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49306/0 (socket says 172.21.15.17:49306) 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd2ffd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc0004810 msgr2=0x7f6dd41cfe10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 -- 172.21.15.17:0/4235688823 learned_addr learned my addr 172.21.15.17:0/4235688823 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): get_auth_request con 0x7f6dc00052a0 auth_method 0 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd2ffd640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc0004810 0x7f6dd41cfe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00039c0 0x7f6dd41cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request con 0x7f6dc00039c0 auth_method 0 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient(hunting): handle_auth_done global_id 104226 payload 824 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 -- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00039c0 msgr2=0x7f6dd41cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00039c0 0x7f6dd41cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 -- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc0004810 msgr2=0x7f6dd41cfe10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc0004810 0x7f6dd41cfe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 monclient: found mon.noname-a 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 -- 172.21.15.17:0/4235688823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd41256d0 con 0x7f6dc00052a0 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00039c0 0x7f6dd41cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd3fff640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 0x7f6dd409d430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6db4007370 tx=0x7f6dcc009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc016020 con 0x7f6dc00052a0 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: dump: 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:10.736 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: _finish_auth 0 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc012600 con 0x7f6dc00052a0 2024-09-15T21:26:10.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:10.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc012a30 con 0x7f6dc00052a0 2024-09-15T21:26:10.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 5 monclient: authenticate success, global_id 104226 2024-09-15T21:26:10.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _renew_subs 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd41259b0 con 0x7f6dc00052a0 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd1ffb640 4 set_mon_vals no callback set 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 10 monclient: dump: 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _renew_subs 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd41253f0 con 0x7f6dc00052a0 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd4124240 con 0x7f6dc00052a0 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.723+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6dcc019020 con 0x7f6dc00052a0 2024-09-15T21:26:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.727+0000 7f6dcbfff640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6db403c090 0x7f6db403e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.727+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(259..259 src has 1..259) ==== 7657+0+0 (secure 0 0 0) 0x7f6dcc04ffe0 con 0x7f6dc00052a0 2024-09-15T21:26:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.727+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6db403c090 0x7f6db403e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.727+0000 7f6dd37fe640 10 monclient: get_auth_request con 0x7f6db403c090 auth_method 0 2024-09-15T21:26:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.727+0000 7f6dd37fe640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6db403c090 0x7f6db403e550 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4001ec0 tx=0x7f6dc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.731+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6dcc020030 con 0x7f6dc00052a0 2024-09-15T21:26:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.731+0000 7f6dcbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:10.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.731+0000 7f6dcbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:10.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dd9ac6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:10.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dd9ac6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:10.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dd9ac6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:10.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6dd4138ab0 con 0x7f6dc00052a0 2024-09-15T21:26:10.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dcbfff640 1 -- 172.21.15.17:0/4235688823 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7f6dcc022300 con 0x7f6dc00052a0 2024-09-15T21:26:10.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dcbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:10.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.971+0000 7f6dcbfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:10.977 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6db403c090 msgr2=0x7f6db403e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6db403c090 0x7f6db403e550 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4001ec0 tx=0x7f6dc4008040 comp rx=0 tx=0).stop 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 10 monclient: shutdown 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 msgr2=0x7f6dd409d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 0x7f6dd409d430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6db4007370 tx=0x7f6dcc009980 comp rx=0 tx=0).stop 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 shutdown_connections 2024-09-15T21:26:10.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6db403c090 0x7f6db403e550 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dc00052a0 0x7f6dd409d430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6dc0004810 0x7f6dd41cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 --2- 172.21.15.17:0/4235688823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6dc00039c0 0x7f6dd41cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 >> 172.21.15.17:0/4235688823 conn(0x7f6dd40a66a0 msgr2=0x7f6dd40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 shutdown_connections 2024-09-15T21:26:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:10.975+0000 7f6dd9ac6640 1 -- 172.21.15.17:0/4235688823 wait complete. 2024-09-15T21:26:10.995 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:26:10.996 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:26:10.996 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:26:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.111+0000 7f5c2a1b8640 10 monclient: get_monmap_and_config 2024-09-15T21:26:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.111+0000 7f5c2a1b8640 10 monclient: build_initial_monmap 2024-09-15T21:26:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.111+0000 7f5c2a1b8640 10 monclient: monmap: 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:11.118111+0000 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:11.118111+0000 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:11.119 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 1 Processor -- start 2024-09-15T21:26:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 1 -- start start 2024-09-15T21:26:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient: init 2024-09-15T21:26:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:26:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c24073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient(hunting): picked mon.noname-b con 0x7f5c24073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c24073f60 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5c24137da0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c240a0890 con 0x7f5c24073f60 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c240746e0 con 0x7f5c24137da0 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c240a1640 con 0x7f5c24073640 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c2a1b8640 10 monclient(hunting): authenticate will time out at 1494.513467s 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c22ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5c24073f60 0x7f5c24137690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47164/0 (socket says 172.21.15.17:47164) 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c22ffd640 1 -- 172.21.15.17:0/1655436523 learned_addr learned my addr 172.21.15.17:0/1655436523 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c23fff640 1 -- 172.21.15.17:0/1655436523 >> v1:172.21.15.119:6789/0 conn(0x7f5c24137da0 legacy=0x7f5c2413fc20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 1 --2- 172.21.15.17:0/1655436523 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c24073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): get_auth_request con 0x7f5c24073640 auth_method 0 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c227fc640 1 -- 172.21.15.17:0/1655436523 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547742916 0 0) 0x7f5c240a0890 con 0x7f5c24073f60 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c227fc640 10 monclient(hunting): _negotiate 2024-09-15T21:26:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c227fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c227fc640 10 monclient(hunting): my global_id is 104229 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c227fc640 1 -- 172.21.15.17:0/1655436523 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c10003530 con 0x7f5c24073f60 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient(hunting): handle_auth_done global_id 45374 payload 274 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 1 -- 172.21.15.17:0/1655436523 >> v1:172.21.15.119:6789/0 conn(0x7f5c24137da0 legacy=0x7f5c2413fc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 1 -- 172.21.15.17:0/1655436523 >> v1:172.21.15.17:6789/0 conn(0x7f5c24073f60 legacy=0x7f5c24137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient: _finish_hunting 0 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 1 monclient: found mon.noname-b 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:11.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.115+0000 7f5c237fe640 1 -- 172.21.15.17:0/1655436523 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c240a1920 con 0x7f5c24073640 2024-09-15T21:26:11.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c237fe640 1 --2- 172.21.15.17:0/1655436523 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c24073a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0097f0 tx=0x7f5c0c01c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4adfd81b304b9c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 1 -- 172.21.15.17:0/1655436523 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0b0070 con 0x7f5c24073640 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: dump: 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 1 -- 172.21.15.17:0/1655436523 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0ab600 con 0x7f5c24073640 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 5 monclient: authenticate success, global_id 45374 2024-09-15T21:26:11.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 1 -- 172.21.15.17:0/1655436523 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0ab9b0 con 0x7f5c24073640 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 4 set_mon_vals no callback set 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c227fc640 10 monclient: dump: 2024-09-15T21:26:11.135 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: shutdown 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/1655436523 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 msgr2=0x7f5c24073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 --2- 172.21.15.17:0/1655436523 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c24073a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0097f0 tx=0x7f5c0c01c3a0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/1655436523 shutdown_connections 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 --2- 172.21.15.17:0/1655436523 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c24073a20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/1655436523 >> 172.21.15.17:0/1655436523 conn(0x7f5c240a66a0 msgr2=0x7f5c240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/1655436523 shutdown_connections 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/1655436523 wait complete. 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: build_initial_monmap 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: monmap: 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 Processor -- start 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- start start 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: init 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c2411e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): picked mon.noname-c con 0x7f5c24073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c24073f60 0x7f5c24136750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): picked mon.noname-a con 0x7f5c24073f60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c24137da0 0x7f5c2411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): picked mon.noname-b con 0x7f5c24137da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c240a0e80 con 0x7f5c24073f60 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c240a0d00 con 0x7f5c24137da0 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5c24143b60 con 0x7f5c24073640 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.119+0000 7f5c2a1b8640 10 monclient(hunting): authenticate will time out at 1494.519992s 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c24073f60 0x7f5c24136750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c2411e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c24073f60 0x7f5c24136750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49322/0 (socket says 172.21.15.17:49322) 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 -- 172.21.15.17:0/2156894914 learned_addr learned my addr 172.21.15.17:0/2156894914 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): get_auth_request con 0x7f5c24073f60 auth_method 0 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): get_auth_request con 0x7f5c24073640 auth_method 0 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c23fff640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c24137da0 msgr2=0x7f5c2411c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c23fff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c24137da0 0x7f5c2411c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient(hunting): handle_auth_done global_id 104232 payload 824 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 msgr2=0x7f5c2411e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5c24073640 0x7f5c2411e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c24137da0 msgr2=0x7f5c2411c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:11.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c24137da0 0x7f5c2411c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 monclient: found mon.noname-a 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2411fb10 con 0x7f5c24073f60 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c22ffd640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c24073f60 0x7f5c24136750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c500 tx=0x7f5c1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14005cf0 con 0x7f5c24073f60 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: dump: 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.142 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: _finish_auth 0 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c14007070 con 0x7f5c24073f60 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14007420 con 0x7f5c24073f60 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 5 monclient: authenticate success, global_id 104232 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 10 monclient: _renew_subs 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2409d0f0 con 0x7f5c24073f60 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c21ffb640 4 set_mon_vals no callback set 2024-09-15T21:26:11.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: dump: 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 10 monclient: _renew_subs 2024-09-15T21:26:11.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2411dd70 con 0x7f5c24073f60 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 0x7f5c2411ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2411d340 con 0x7f5c2411ca20 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 0x7f5c2411ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient: get_auth_request con 0x7f5c2411ca20 auth_method 0 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5c140075c0 con 0x7f5c24073f60 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 10 monclient(hunting): handle_auth_done global_id 104235 payload 824 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c24073a20 con 0x7f5c2411ca20 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c0403b970 0x7f5c0403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(259..259 src has 1..259) ==== 7657+0+0 (secure 0 0 0) 0x7f5c14050240 con 0x7f5c24073f60 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c237fe640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 0x7f5c2411ce00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c2406e7b0 tx=0x7f5c0c019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0bb070 con 0x7f5c2411ca20 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: dump: 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.147 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c23fff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c0403b970 0x7f5c0403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c23fff640 10 monclient: get_auth_request con 0x7f5c0403b970 auth_method 0 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5c0c003820 con 0x7f5c2411ca20 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 msgr2=0x7f5c2411ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 0x7f5c2411ce00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c2406e7b0 tx=0x7f5c0c019140 comp rx=0 tx=0).stop 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 msgr2=0x7f5c2411ce00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c1bfff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2411ca20 0x7f5c2411ce00 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c2406e7b0 tx=0x7f5c0c019140 comp rx=0 tx=0).stop 2024-09-15T21:26:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.123+0000 7f5c23fff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c0403b970 0x7f5c0403de30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5c2411e660 tx=0x7f5c1c0091f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 0x7f5c2413c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24120190 con 0x7f5c2413bc00 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 0x7f5c2413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient: get_auth_request con 0x7f5c2413bc00 auth_method 0 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 10 monclient(hunting): handle_auth_done global_id 104238 payload 824 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 1 -- 172.21.15.17:0/2156894914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5c241c8ba0 con 0x7f5c2413bc00 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c237fe640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 0x7f5c2413c010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c01c3e0 tx=0x7f5c0c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0b0040 con 0x7f5c2413bc00 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 10 monclient: dump: 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7f5c0c0b7660 con 0x7f5c2413bc00 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 msgr2=0x7f5c2413c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 0x7f5c2413c010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c01c3e0 tx=0x7f5c0c009190 comp rx=0 tx=0).stop 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 msgr2=0x7f5c2413c010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c1bfff640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c2413bc00 0x7f5c2413c010 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c01c3e0 tx=0x7f5c0c009190 comp rx=0 tx=0).stop 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:11.154 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 24879, 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:11.155 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:11.156 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:11.157 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:11.158 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:11.159 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:11.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:11.161 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:11.162 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:11.163 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.164 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:11.165 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:11.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.168 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:11.168 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:11.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c0403b970 msgr2=0x7f5c0403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5c0403b970 0x7f5c0403de30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5c2411e660 tx=0x7f5c1c0091f0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 10 monclient: shutdown 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c24073f60 msgr2=0x7f5c24136750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 --2- 172.21.15.17:0/2156894914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c24073f60 0x7f5c24136750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c500 tx=0x7f5c1400c9c0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c23fff640 1 -- 172.21.15.17:0/2156894914 reap_dead start 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c23fff640 1 -- 172.21.15.17:0/2156894914 reap_dead start 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 shutdown_connections 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 >> 172.21.15.17:0/2156894914 conn(0x7f5c240a66a0 msgr2=0x7f5c2409dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 shutdown_connections 2024-09-15T21:26:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.131+0000 7f5c2a1b8640 1 -- 172.21.15.17:0/2156894914 wait complete. 2024-09-15T21:26:11.172 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:26:11.172 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:26:11.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:26:11.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient: get_monmap_and_config 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient: build_initial_monmap 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient: monmap: 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:11.290459+0000 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:11.290459+0000 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:11.292 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 Processor -- start 2024-09-15T21:26:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 -- start start 2024-09-15T21:26:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient: init 2024-09-15T21:26:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3fc1378c0 0x7fc3fc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): picked mon.noname-a con 0x7fc3fc1378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc3fc1381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): picked mon.noname-b con 0x7fc3fc0a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3fc0a1d00 con 0x7fc3fc1381e0 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc3fc0a1fe0 con 0x7fc3fc1378c0 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc3fc0a22c0 con 0x7fc3fc0a5210 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:11.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc402141640 10 monclient(hunting): authenticate will time out at 1494.685880s 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3faffd640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fc3fc1381e0 legacy=0x7fc3fc0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc0a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56862/0 (socket says 172.21.15.17:56862) 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 -- 172.21.15.17:0/1932413046 learned_addr learned my addr 172.21.15.17:0/1932413046 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): get_auth_request con 0x7fc3fc0a5210 auth_method 0 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 1 --2- 172.21.15.17:0/1932413046 >> v2:172.21.15.17:3300/0 conn(0x7fc3fc1378c0 0x7fc3fc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): get_auth_request con 0x7fc3fc1378c0 auth_method 0 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient(hunting): handle_auth_done global_id 45380 payload 274 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 -- 172.21.15.17:0/1932413046 >> v2:172.21.15.17:3300/0 conn(0x7fc3fc1378c0 msgr2=0x7fc3fc137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 --2- 172.21.15.17:0/1932413046 >> v2:172.21.15.17:3300/0 conn(0x7fc3fc1378c0 0x7fc3fc137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 -- 172.21.15.17:0/1932413046 >> v1:172.21.15.119:6789/0 conn(0x7fc3fc1381e0 legacy=0x7fc3fc0a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 monclient: found mon.noname-b 2024-09-15T21:26:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.287+0000 7fc3fbfff640 1 -- 172.21.15.17:0/1932413046 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc0a4490 con 0x7fc3fc0a5210 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fbfff640 1 --2- 172.21.15.17:0/1932413046 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc0a56a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009e80 tx=0x7fc3f401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ddc7c7375aa89b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 1 -- 172.21.15.17:0/1932413046 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40a5730 con 0x7fc3fc0a5210 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: dump: 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 5 monclient: authenticate success, global_id 45380 2024-09-15T21:26:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 1 -- 172.21.15.17:0/1932413046 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f40a58d0 con 0x7fc3fc0a5210 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 4 set_mon_vals no callback set 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 1 -- 172.21.15.17:0/1932413046 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40a5ca0 con 0x7fc3fc0a5210 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fa7fc640 10 monclient: dump: 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: shutdown 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- 172.21.15.17:0/1932413046 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 msgr2=0x7fc3fc0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 --2- 172.21.15.17:0/1932413046 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc0a56a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009e80 tx=0x7fc3f401c8d0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- 172.21.15.17:0/1932413046 shutdown_connections 2024-09-15T21:26:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 --2- 172.21.15.17:0/1932413046 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc0a56a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 --2- 172.21.15.17:0/1932413046 >> v2:172.21.15.17:3300/0 conn(0x7fc3fc1378c0 0x7fc3fc137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- 172.21.15.17:0/1932413046 >> 172.21.15.17:0/1932413046 conn(0x7fc3fc0aac40 msgr2=0x7fc3fc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- 172.21.15.17:0/1932413046 shutdown_connections 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- 172.21.15.17:0/1932413046 wait complete. 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: build_initial_monmap 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: monmap: 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 Processor -- start 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- start start 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: init 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc1cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): picked mon.noname-c con 0x7fc3fc0a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc1378c0 0x7fc3fc1d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): picked mon.noname-a con 0x7fc3fc1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc1381e0 0x7fc3fc13cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): picked mon.noname-b con 0x7fc3fc1381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1425c0 con 0x7fc3fc1378c0 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc138700 con 0x7fc3fc1381e0 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc3fc142740 con 0x7fc3fc0a5210 2024-09-15T21:26:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc402141640 10 monclient(hunting): authenticate will time out at 1494.691459s 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc1378c0 0x7fc3fc1d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fbfff640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc1381e0 msgr2=0x7fc3fc13cfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc1cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc1381e0 0x7fc3fc13cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc1378c0 0x7fc3fc1d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49364/0 (socket says 172.21.15.17:49364) 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 1 -- 172.21.15.17:0/4182177834 learned_addr learned my addr 172.21.15.17:0/4182177834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): get_auth_request con 0x7fc3fc1378c0 auth_method 0 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): get_auth_request con 0x7fc3fc0a5210 auth_method 0 2024-09-15T21:26:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.291+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 10 monclient(hunting): handle_auth_done global_id 45383 payload 824 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc1381e0 msgr2=0x7fc3fc13cfc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc1381e0 0x7fc3fc13cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc1378c0 msgr2=0x7fc3fc1d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc1378c0 0x7fc3fc1d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 monclient: found mon.noname-c 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc0a12f0 con 0x7fc3fc0a5210 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc1378c0 0x7fc3fc1d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fb7fe640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc1cfa40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00e870 tx=0x7fc3ec00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec017070 con 0x7fc3fc0a5210 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: dump: 2024-09-15T21:26:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.314 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.314 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.316 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: _finish_auth 0 2024-09-15T21:26:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec00c940 con 0x7fc3fc0a5210 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 5 monclient: authenticate success, global_id 45383 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 10 monclient: _renew_subs 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc120f60 con 0x7fc3fc0a5210 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00ccf0 con 0x7fc3fc0a5210 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: dump: 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.319 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 10 monclient: _renew_subs 2024-09-15T21:26:11.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:11.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc13e840 con 0x7fc3fc0a5210 2024-09-15T21:26:11.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3f9ffb640 4 set_mon_vals no callback set 2024-09-15T21:26:11.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 0x7fc3fc13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc3fc0a56a0 con 0x7fc3fc1d0750 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 0x7fc3fc13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient: get_auth_request con 0x7fc3fc1d0750 auth_method 0 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc3ec01a070 con 0x7fc3fc0a5210 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 10 monclient(hunting): handle_auth_done global_id 45386 payload 824 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3fc0008e0 con 0x7fc3fc1d0750 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc3e803b970 0x7fc3e803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(259..259 src has 1..259) ==== 7657+0+0 (secure 0 0 0) 0x7fc3ec0505e0 con 0x7fc3fc0a5210 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3faffd640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 0x7fc3fc13d500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009fd0 tx=0x7fc3f0005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0016070 con 0x7fc3fc1d0750 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: dump: 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fbfff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc3e803b970 0x7fc3e803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3fbfff640 10 monclient: get_auth_request con 0x7fc3e803b970 auth_method 0 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc3f000eaf0 con 0x7fc3fc1d0750 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 msgr2=0x7fc3fc13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 0x7fc3fc13d500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009fd0 tx=0x7fc3f0005780 comp rx=0 tx=0).stop 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 msgr2=0x7fc3fc13d500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.295+0000 7fc3e3fff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc1d0750 0x7fc3fc13d500 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009fd0 tx=0x7fc3f0005780 comp rx=0 tx=0).stop 2024-09-15T21:26:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.299+0000 7fc3fbfff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc3e803b970 0x7fc3e803de30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40ae000 tx=0x7fc3f40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 0x7fc3fc09ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc3fc13de50 con 0x7fc3fc13da40 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 0x7fc3fc09ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient: get_auth_request con 0x7fc3fc13da40 auth_method 0 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 10 monclient(hunting): handle_auth_done global_id 45389 payload 824 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 1 -- 172.21.15.17:0/4182177834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc3fc121480 con 0x7fc3fc13da40 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3faffd640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 0x7fc3fc09ea80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0018000 tx=0x7fc3f0005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0016040 con 0x7fc3fc13da40 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 10 monclient: dump: 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4868 (secure 0 0 0) 0x7fc3f0005a00 con 0x7fc3fc13da40 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 msgr2=0x7fc3fc09ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 0x7fc3fc09ea80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0018000 tx=0x7fc3f0005990 comp rx=0 tx=0).stop 2024-09-15T21:26:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 msgr2=0x7fc3fc09ea80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:11.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3e3fff640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc13da40 0x7fc3fc09ea80 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0018000 tx=0x7fc3f0005990 comp rx=0 tx=0).stop 2024-09-15T21:26:11.331 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:26:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-15T21:26:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 339901, 2024-09-15T21:26:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:26:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:11.348 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:11.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.329+0000 7f0924ff9640 10 monclient: tick 2024-09-15T21:26:11.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.329+0000 7f0924ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:26:11.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.329+0000 7f0924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:11.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc3e803b970 msgr2=0x7fc3e803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc3e803b970 0x7fc3e803de30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40ae000 tx=0x7fc3f40a7070 comp rx=0 tx=0).stop 2024-09-15T21:26:11.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 10 monclient: shutdown 2024-09-15T21:26:11.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:11.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 msgr2=0x7fc3fc1cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 --2- 172.21.15.17:0/4182177834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3fc0a5210 0x7fc3fc1cfa40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00e870 tx=0x7fc3ec00ed30 comp rx=0 tx=0).stop 2024-09-15T21:26:11.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3fbfff640 1 -- 172.21.15.17:0/4182177834 reap_dead start 2024-09-15T21:26:11.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc3fbfff640 1 -- 172.21.15.17:0/4182177834 reap_dead start 2024-09-15T21:26:11.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 shutdown_connections 2024-09-15T21:26:11.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 >> 172.21.15.17:0/4182177834 conn(0x7fc3fc0aac40 msgr2=0x7fc3fc0a0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:11.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 shutdown_connections 2024-09-15T21:26:11.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:11.303+0000 7fc402141640 1 -- 172.21.15.17:0/4182177834 wait complete. 2024-09-15T21:26:11.351 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:26:11.351 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:26:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.693+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}]=0 v260) ==== 106+0+3 (secure 0 0 0) 0x7f092804d7c0 con 0x7f09301364a0 2024-09-15T21:26:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.697+0000 7f0925ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:26:11.699 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.697+0000 7f0925ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:11.816 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0937a60640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:26:11.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0937a60640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:26:11.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0937a60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:11.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"} v 0) -- 0x7f0930134f00 con 0x7f09301364a0 2024-09-15T21:26:11.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0925ffb640 1 -- 172.21.15.119:0/862771478 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}]=0 v260) ==== 106+0+3 (secure 0 0 0) 0x7f0928007820 con 0x7f09301364a0 2024-09-15T21:26:11.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0925ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "79adef04-9b2e-463a-b4b9-a979dfe86845"}] 2024-09-15T21:26:11.817 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.813+0000 7f0925ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f091403b970 msgr2=0x7f091403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f091403b970 0x7f091403de30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f092c0093d0 tx=0x7f092c009360 comp rx=0 tx=0).stop 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 10 monclient: shutdown 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 msgr2=0x7f093011c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f093011c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f092800caa0 tx=0x7f092800cf60 comp rx=0 tx=0).stop 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 shutdown_connections 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f091403b970 0x7f091403de30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.821 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09301364a0 0x7f093011c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0930135b80 0x7f09301349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 --2- 172.21.15.119:0/862771478 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f09300a0030 0x7f093011db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:11.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 >> 172.21.15.119:0/862771478 conn(0x7f093006cdc0 msgr2=0x7f09300a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:11.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 shutdown_connections 2024-09-15T21:26:11.822 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:11.817+0000 7f0937a60640 1 -- 172.21.15.119:0/862771478 wait complete. 2024-09-15T21:26:11.836 INFO:tasks.workunit.client.0.smithi119.stdout:OK 2024-09-15T21:26:11.837 INFO:tasks.workunit.client.0.smithi119.stderr:+ na2=12 2024-09-15T21:26:11.837 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo OK 2024-09-15T21:26:11.837 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:26:11.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:26:11.850 INFO:tasks.workunit:Running workunits matching mon/caps.sh on client.0... 2024-09-15T21:26:11.851 INFO:tasks.workunit:Running workunit mon/caps.sh... 2024-09-15T21:26:11.851 DEBUG:teuthology.orchestra.run.smithi119:workunit test mon/caps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7498493279982dfe87f00616198a5967475f6169 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh 2024-09-15T21:26:11.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ tmp=/tmp/cephtest-mon-caps-madness 2024-09-15T21:26:11.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ exit_on_error=1 2024-09-15T21:26:11.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ ! -z '' ]] 2024-09-15T21:26:11.900 INFO:tasks.workunit.client.0.smithi119.stderr:++ uname 2024-09-15T21:26:11.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' Linux = FreeBSD ']' 2024-09-15T21:26:11.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ ETIMEDOUT=110 2024-09-15T21:26:11.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 0 2024-09-15T21:26:11.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 2024-09-15T21:26:11.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=0 2024-09-15T21:26:11.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-09-15T21:26:11.902 INFO:tasks.workunit.client.0.smithi119.stdout:ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-09-15T21:26:11.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-09-15T21:26:12.433 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=0 2024-09-15T21:26:12.433 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T21:26:12.433 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:12.433 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 13 2024-09-15T21:26:12.434 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 2024-09-15T21:26:12.434 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:26:12.434 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-09-15T21:26:12.435 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-09-15T21:26:12.435 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-09-15T21:26:15.549 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:26:15.549 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:26:15.549 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:15.549 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph auth del client.bazar 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient: get_monmap_and_config 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient: build_initial_monmap 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient: monmap: 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:26:15.669028+0000 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:26:15.669028+0000 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:26:15.667 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:15.668 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:15.668 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:15.668 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:15.668 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:15.668 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:26:15.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 Processor -- start 2024-09-15T21:26:15.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 -- start start 2024-09-15T21:26:15.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient: init 2024-09-15T21:26:15.670 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f864c09d4e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f864c09f960 0x7f864c139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): picked mon.noname-c con 0x7f864c09f960 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 0x7f864c13c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): picked mon.noname-b con 0x7f864c09dfd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f864c09ef80 con 0x7f864c09d4e0 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f864c06da70 con 0x7f864c09f960 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f864c0746f0 con 0x7f864c09dfd0 2024-09-15T21:26:15.671 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8650aa7640 10 monclient(hunting): authenticate will time out at 1494.465577s 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8649d9b640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f864c09f960 msgr2=0x7f864c139c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f8649d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f864c09f960 0x7f864c139c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 0x7f864c13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f864c09d4e0 0x7f864c09d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.119:46108/0 (socket says 172.21.15.119:46108) 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 0x7f864c13c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33676/0 (socket says 172.21.15.119:33676) 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864a59c640 1 -- 172.21.15.119:0/488114234 learned_addr learned my addr 172.21.15.119:0/488114234 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): get_auth_request con 0x7f864c09dfd0 auth_method 0 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:15.672 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864959a640 1 -- 172.21.15.119:0/488114234 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730777 0 0) 0x7f864c09ef80 con 0x7f864c09d4e0 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864959a640 10 monclient(hunting): _negotiate 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864959a640 10 monclient(hunting): my global_id is 104259 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864959a640 1 -- 172.21.15.119:0/488114234 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8634005740 con 0x7f864c09d4e0 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient(hunting): handle_auth_done global_id 45398 payload 274 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 -- 172.21.15.119:0/488114234 >> v2:172.21.15.119:3300/0 conn(0x7f864c09f960 msgr2=0x7f864c139c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 --2- 172.21.15.119:0/488114234 >> v2:172.21.15.119:3300/0 conn(0x7f864c09f960 0x7f864c139c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:15.673 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 -- 172.21.15.119:0/488114234 >> v1:172.21.15.17:6789/0 conn(0x7f864c09d4e0 legacy=0x7f864c09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:15.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:26:15.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:26:15.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:15.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:15.674 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.665+0000 7f864ad9d640 1 -- 172.21.15.119:0/488114234 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c09e6e0 con 0x7f864c09dfd0 2024-09-15T21:26:15.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 1 --2- 172.21.15.119:0/488114234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 0x7f864c13c150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8644009d90 tx=0x7f864401c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bac4cf56b309ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:15.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 1 -- 172.21.15.119:0/488114234 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440b2070 con 0x7f864c09dfd0 2024-09-15T21:26:15.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:15.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:15.680 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: dump: 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:15.681 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: _finish_auth 0 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: _check_auth_tickets 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 5 monclient: authenticate success, global_id 45398 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 1 -- 172.21.15.119:0/488114234 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86440a5800 con 0x7f864c09dfd0 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 4 set_mon_vals no callback set 2024-09-15T21:26:15.682 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 1 -- 172.21.15.119:0/488114234 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440a5bd0 con 0x7f864c09dfd0 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864959a640 10 monclient: dump: 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:15.683 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:15.684 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:15.684 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:26:15.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: shutdown 2024-09-15T21:26:15.684 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- 172.21.15.119:0/488114234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 msgr2=0x7f864c13c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 --2- 172.21.15.119:0/488114234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 0x7f864c13c150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8644009d90 tx=0x7f864401c820 comp rx=0 tx=0).stop 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- 172.21.15.119:0/488114234 shutdown_connections 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 --2- 172.21.15.119:0/488114234 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09dfd0 0x7f864c13c150 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 --2- 172.21.15.119:0/488114234 >> v2:172.21.15.119:3300/0 conn(0x7f864c09f960 0x7f864c139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- 172.21.15.119:0/488114234 >> 172.21.15.119:0/488114234 conn(0x7f864c06cdc0 msgr2=0x7f864c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- 172.21.15.119:0/488114234 shutdown_connections 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- 172.21.15.119:0/488114234 wait complete. 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: build_initial_monmap 2024-09-15T21:26:15.685 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: monmap: 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 Processor -- start 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- start start 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: init 2024-09-15T21:26:15.686 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09d4e0 0x7f864c11c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): picked mon.noname-c con 0x7f864c09d4e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c09dfd0 0x7f864c11cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): picked mon.noname-b con 0x7f864c09dfd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f864c09f960 0x7f864c11d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): picked mon.noname-a con 0x7f864c09f960 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f864c0a1a80 con 0x7f864c09f960 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f864c0a2750 con 0x7f864c09dfd0 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f864c0a1c00 con 0x7f864c09d4e0 2024-09-15T21:26:15.687 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8650aa7640 10 monclient(hunting): authenticate will time out at 1494.471186s 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8649d9b640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c09dfd0 msgr2=0x7f864c11cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f8649d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c09dfd0 0x7f864c11cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f864c09f960 0x7f864c11d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09d4e0 0x7f864c11c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09d4e0 0x7f864c11c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.119:33682/0 (socket says 172.21.15.119:33682) 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 1 -- 172.21.15.119:0/1011630877 learned_addr learned my addr 172.21.15.119:0/1011630877 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): get_auth_request con 0x7f864c09d4e0 auth_method 0 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): get_auth_request con 0x7f864c09f960 auth_method 0 2024-09-15T21:26:15.688 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.669+0000 7f864ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 10 monclient(hunting): handle_auth_done global_id 104262 payload 824 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 -- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09d4e0 msgr2=0x7f864c11c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09d4e0 0x7f864c11c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 -- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c09dfd0 msgr2=0x7f864c11cb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:15.689 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c09dfd0 0x7f864c11cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 monclient: found mon.noname-a 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 -- 172.21.15.119:0/1011630877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c13ba10 con 0x7f864c09f960 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864ad9d640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f864c09f960 0x7f864c11d0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86440ac000 tx=0x7f86440195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440b2050 con 0x7f864c09f960 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: dump: 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.690 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: _finish_auth 0 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 5 monclient: authenticate success, global_id 104262 2024-09-15T21:26:15.691 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 monclient: _renew_subs 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c11e980 con 0x7f864c09f960 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86440b3040 con 0x7f864c09f960 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440ad650 con 0x7f864c09f960 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 10 monclient: dump: 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:15.692 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 monclient: _renew_subs 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c06efc0 con 0x7f864c09f960 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863bfff640 4 set_mon_vals no callback set 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864c0a2010 con 0x7f864c09f960 2024-09-15T21:26:15.693 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f86440c0020 con 0x7f864c09f960 2024-09-15T21:26:15.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f862c03b9a0 0x7f862c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:15.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(260..260 src has 1..260) ==== 7657+0+0 (secure 0 0 0) 0x7f86440eb4e0 con 0x7f864c09f960 2024-09-15T21:26:15.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864a59c640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f862c03b9a0 0x7f862c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:15.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864a59c640 10 monclient: get_auth_request con 0x7f862c03b9a0 auth_method 0 2024-09-15T21:26:15.694 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.673+0000 7f864a59c640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f862c03b9a0 0x7f862c03de60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f863c007fd0 tx=0x7f863c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:15.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.685+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8644016560 con 0x7f864c09f960 2024-09-15T21:26:15.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.685+0000 7f863affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:15.695 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.685+0000 7f863affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:15.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.905+0000 7f8650aa7640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T21:26:15.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.905+0000 7f8650aa7640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T21:26:15.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.905+0000 7f8650aa7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:15.909 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.905+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) -- 0x7f864c06c440 con 0x7f864c09f960 2024-09-15T21:26:15.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.909+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v15) ==== 82+0+0 (secure 0 0 0) 0x7f86440e83b0 con 0x7f864c09f960 2024-09-15T21:26:15.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.909+0000 7f863affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T21:26:15.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:15.909+0000 7f863affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:16.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f8650aa7640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T21:26:16.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f8650aa7640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T21:26:16.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f8650aa7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:16.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) -- 0x7f864c13ab60 con 0x7f864c09f960 2024-09-15T21:26:16.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f863affd640 1 -- 172.21.15.119:0/1011630877 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v15) ==== 82+0+0 (secure 0 0 0) 0x7f86440b8cd0 con 0x7f864c09f960 2024-09-15T21:26:16.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f863affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T21:26:16.062 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.057+0000 7f863affd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:26:16.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f862c03b9a0 msgr2=0x7f862c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:16.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f862c03b9a0 0x7f862c03de60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f863c007fd0 tx=0x7f863c009040 comp rx=0 tx=0).stop 2024-09-15T21:26:16.064 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 10 monclient: shutdown 2024-09-15T21:26:16.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:16.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f864c09f960 msgr2=0x7f864c11d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:16.065 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f864c09f960 0x7f864c11d0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86440ac000 tx=0x7f86440195c0 comp rx=0 tx=0).stop 2024-09-15T21:26:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 shutdown_connections 2024-09-15T21:26:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f862c03b9a0 0x7f862c03de60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f864c09f960 0x7f864c11d0b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c09dfd0 0x7f864c11cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 --2- 172.21.15.119:0/1011630877 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f864c09d4e0 0x7f864c11c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:16.066 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 >> 172.21.15.119:0/1011630877 conn(0x7f864c06cdc0 msgr2=0x7f864c0a5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 shutdown_connections 2024-09-15T21:26:16.067 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:26:16.061+0000 7f8650aa7640 1 -- 172.21.15.119:0/1011630877 wait complete. 2024-09-15T21:26:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=''\''allow command "auth ls", allow command quorum_status'\''' 2024-09-15T21:26:16.079 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 0 2024-09-15T21:26:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 2024-09-15T21:26:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=0 2024-09-15T21:26:16.080 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-09-15T21:26:16.080 INFO:tasks.workunit.client.0.smithi119.stdout:ceph auth get-or-create client.foo mon 'allow command "auth ls", allow command quorum_status' > /tmp/cephtest-mon-caps-madness.foo.keyring 2024-09-15T21:26:16.081 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-09-15T21:26:16.609 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=0 2024-09-15T21:26:16.609 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T21:26:16.609 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:16.609 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 0 2024-09-15T21:26:16.609 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-09-15T21:26:16.609 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=0 2024-09-15T21:26:16.610 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T21:26:16.610 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T21:26:16.610 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T21:26:17.146 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=0 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 0 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=0 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-09-15T21:26:17.147 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-09-15T21:26:17.148 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-09-15T21:26:17.685 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=0 2024-09-15T21:26:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T21:26:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 13 2024-09-15T21:26:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 2024-09-15T21:26:17.686 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:26:17.686 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-09-15T21:26:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-09-15T21:26:17.687 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-09-15T21:26:18.036 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 13 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-09-15T21:26:18.037 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-09-15T21:26:18.038 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-09-15T21:26:18.402 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 13 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-09-15T21:26:18.403 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-09-15T21:26:18.404 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-09-15T21:26:18.702 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:26:18.702 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:26:18.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:26:18.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T21:26:18.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T21:26:18.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 110 2024-09-15T21:26:18.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 2024-09-15T21:26:18.704 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=110 2024-09-15T21:26:18.704 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-09-15T21:26:18.704 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-09-15T21:26:18.705 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-09-15T21:26:31.353 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:26:31.354 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:26:31.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:26:31.358 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:26:31.358 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:26:31.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.486+0000 7f69fbe54640 10 monclient: get_monmap_and_config 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.486+0000 7f69fbe54640 10 monclient: build_initial_monmap 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.486+0000 7f69fbe54640 10 monclient: monmap: 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:31.492731+0000 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:31.492731+0000 2024-09-15T21:26:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.486+0000 7f69fbe54640 1 Processor -- start 2024-09-15T21:26:31.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 1 -- start start 2024-09-15T21:26:31.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient: init 2024-09-15T21:26:31.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:31.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69f41378c0 0x7f69f4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): picked mon.noname-c con 0x7f69f41378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 0x7f69f40a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): picked mon.noname-b con 0x7f69f41381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f69f40a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69f40a1d00 con 0x7f69f40a5040 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f69f40a1fe0 con 0x7f69f41378c0 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f69f40a22c0 con 0x7f69f41381e0 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69fbe54640 10 monclient(hunting): authenticate will time out at 1514.889089s 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 0x7f69f40a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 0x7f69f40a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40786/0 (socket says 172.21.15.17:40786) 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 -- 172.21.15.17:0/2180223552 learned_addr learned my addr 172.21.15.17:0/2180223552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): get_auth_request con 0x7f69f41381e0 auth_method 0 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f9bf0640 1 --2- 172.21.15.17:0/2180223552 >> v2:172.21.15.119:3300/0 conn(0x7f69f41378c0 0x7f69f4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f9bf0640 10 monclient(hunting): get_auth_request con 0x7f69f41378c0 auth_method 0 2024-09-15T21:26:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f9bf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f9bf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f9bf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient(hunting): handle_auth_done global_id 45437 payload 274 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 -- 172.21.15.17:0/2180223552 >> v2:172.21.15.119:3300/0 conn(0x7f69f41378c0 msgr2=0x7f69f4137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 --2- 172.21.15.17:0/2180223552 >> v2:172.21.15.119:3300/0 conn(0x7f69f41378c0 0x7f69f4137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 -- 172.21.15.17:0/2180223552 >> v1:172.21.15.119:6789/0 conn(0x7f69f40a5040 legacy=0x7f69f40a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient: _finish_hunting 0 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 monclient: found mon.noname-b 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:31.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 -- 172.21.15.17:0/2180223552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f40a5c00 con 0x7f69f41381e0 2024-09-15T21:26:31.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.490+0000 7f69f93ef640 1 --2- 172.21.15.17:0/2180223552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 0x7f69f40a4b00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009870 tx=0x7f69e401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b592764c6ee21f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:31.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 1 -- 172.21.15.17:0/2180223552 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40b0070 con 0x7f69f41381e0 2024-09-15T21:26:31.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:31.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:31.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: dump: 2024-09-15T21:26:31.506 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: _finish_auth 0 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: _check_auth_tickets 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:31.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 1 -- 172.21.15.17:0/2180223552 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e40ab600 con 0x7f69f41381e0 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 1 -- 172.21.15.17:0/2180223552 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40ab9d0 con 0x7f69f41381e0 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 5 monclient: authenticate success, global_id 45437 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 4 set_mon_vals no callback set 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f8bee640 10 monclient: dump: 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.508 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: shutdown 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/2180223552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 msgr2=0x7f69f40a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 --2- 172.21.15.17:0/2180223552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 0x7f69f40a4b00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009870 tx=0x7f69e401c420 comp rx=0 tx=0).stop 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/2180223552 shutdown_connections 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 --2- 172.21.15.17:0/2180223552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f41381e0 0x7f69f40a4b00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 --2- 172.21.15.17:0/2180223552 >> v2:172.21.15.119:3300/0 conn(0x7f69f41378c0 0x7f69f4137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/2180223552 >> 172.21.15.17:0/2180223552 conn(0x7f69f40aac40 msgr2=0x7f69f40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/2180223552 shutdown_connections 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/2180223552 wait complete. 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: build_initial_monmap 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: monmap: 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:31.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 Processor -- start 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- start start 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: init 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 0x7f69f41cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): picked mon.noname-c con 0x7f69f40a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69f41378c0 0x7f69f4136a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): picked mon.noname-a con 0x7f69f41378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69f41381e0 0x7f69f413cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): picked mon.noname-b con 0x7f69f41381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69f41425c0 con 0x7f69f41378c0 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:31.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4138700 con 0x7f69f41381e0 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f69f4142740 con 0x7f69f40a5040 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient(hunting): authenticate will time out at 1514.894455s 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 0x7f69f41cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69f41378c0 0x7f69f4136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 0x7f69f41cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40788/0 (socket says 172.21.15.17:40788) 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 -- 172.21.15.17:0/331788890 learned_addr learned my addr 172.21.15.17:0/331788890 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): get_auth_request con 0x7f69f40a5040 auth_method 0 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): get_auth_request con 0x7f69f41378c0 auth_method 0 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fa3f1640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69f41381e0 0x7f69f413cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fa3f1640 10 monclient(hunting): get_auth_request con 0x7f69f41381e0 auth_method 0 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fa3f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fa3f1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fa3f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient(hunting): handle_auth_done global_id 45440 payload 824 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 -- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69f41381e0 msgr2=0x7f69f413cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69f41381e0 0x7f69f413cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 -- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69f41378c0 msgr2=0x7f69f4136a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69f41378c0 0x7f69f4136a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient: _finish_hunting 0 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 monclient: found mon.noname-c 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 -- 172.21.15.17:0/331788890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f41210f0 con 0x7f69f40a5040 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f93ef640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69f41378c0 0x7f69f4136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69f9bf0640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 0x7f69f41cf9f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69f000d800 tx=0x7f69f000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f000b6c0 con 0x7f69f40a5040 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: dump: 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 5 monclient: authenticate success, global_id 45440 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: _renew_subs 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f0019070 con 0x7f69f40a5040 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f40a8a80 con 0x7f69f40a5040 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f000bae0 con 0x7f69f40a5040 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ea7fc640 10 monclient: dump: 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69ebfff640 4 set_mon_vals no callback set 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: _renew_subs 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.494+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f4120750 con 0x7f69f40a5040 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69fbe54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69fbe54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69fbe54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f4137ca0 con 0x7f69f40a5040 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f69f000bc80 con 0x7f69f40a5040 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69ea7fc640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69d803b9f0 0x7f69d803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f69f0050a90 con 0x7f69f40a5040 2024-09-15T21:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69f93ef640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69d803b9f0 0x7f69d803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69f93ef640 10 monclient: get_auth_request con 0x7f69d803b9f0 auth_method 0 2024-09-15T21:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.498+0000 7f69f93ef640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69d803b9f0 0x7f69d803deb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f69e40ad000 tx=0x7f69e40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.502+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f69f0011d50 con 0x7f69f40a5040 2024-09-15T21:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.502+0000 7f69ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.502+0000 7f69ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:31.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69fbe54640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:31.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69fbe54640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:31.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69fbe54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:31.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f69f41d0870 con 0x7f69f40a5040 2024-09-15T21:26:31.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69ea7fc640 1 -- 172.21.15.17:0/331788890 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f69f0013390 con 0x7f69f40a5040 2024-09-15T21:26:31.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:31.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.742+0000 7f69ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:31.750 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:31.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69d803b9f0 msgr2=0x7f69d803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:31.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69d803b9f0 0x7f69d803deb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f69e40ad000 tx=0x7f69e40a6000 comp rx=0 tx=0).stop 2024-09-15T21:26:31.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 10 monclient: shutdown 2024-09-15T21:26:31.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:31.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 msgr2=0x7f69f41cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:31.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 0x7f69f41cf9f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69f000d800 tx=0x7f69f000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 shutdown_connections 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69d803b9f0 0x7f69d803deb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69f41381e0 0x7f69f413cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69f41378c0 0x7f69f4136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 --2- 172.21.15.17:0/331788890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69f40a5040 0x7f69f41cf9f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 >> 172.21.15.17:0/331788890 conn(0x7f69f40aac40 msgr2=0x7f69f40a8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 shutdown_connections 2024-09-15T21:26:31.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:31.746+0000 7f69fbe54640 1 -- 172.21.15.17:0/331788890 wait complete. 2024-09-15T21:26:34.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient: get_monmap_and_config 2024-09-15T21:26:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient: build_initial_monmap 2024-09-15T21:26:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient: monmap: 2024-09-15T21:26:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:34.903934+0000 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:34.903934+0000 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:34.905 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 Processor -- start 2024-09-15T21:26:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 -- start start 2024-09-15T21:26:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient: init 2024-09-15T21:26:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-09-15T21:26:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa5500a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa55009ffc0 0x7fa55013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fa55009ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5500a1380 0x7fa55013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): picked mon.noname-b con 0x7fa5500a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5500a3130 con 0x7fa5500a0890 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa550073df0 con 0x7fa55009ffc0 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa5500a1bf0 con 0x7fa5500a1380 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa555f9d640 10 monclient(hunting): authenticate will time out at 1518.299325s 2024-09-15T21:26:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa5500a0890 0x7fa5500a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50354/0 (socket says 172.21.15.17:50354) 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5500a1380 0x7fa55013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54f7fe640 1 -- 172.21.15.17:0/2090794629 learned_addr learned my addr 172.21.15.17:0/2090794629 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): get_auth_request con 0x7fa5500a1380 auth_method 0 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54effd640 1 --2- 172.21.15.17:0/2090794629 >> v2:172.21.15.119:3300/0 conn(0x7fa55009ffc0 0x7fa55013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54effd640 10 monclient(hunting): get_auth_request con 0x7fa55009ffc0 auth_method 0 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient(hunting): handle_auth_done global_id 45443 payload 274 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54e7fc640 1 -- 172.21.15.17:0/2090794629 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346085031 0 0) 0x7fa5500a3130 con 0x7fa5500a0890 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 1 -- 172.21.15.17:0/2090794629 >> v2:172.21.15.119:3300/0 conn(0x7fa55009ffc0 msgr2=0x7fa55013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 1 --2- 172.21.15.17:0/2090794629 >> v2:172.21.15.119:3300/0 conn(0x7fa55009ffc0 0x7fa55013aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 1 -- 172.21.15.17:0/2090794629 >> v1:172.21.15.17:6789/0 conn(0x7fa5500a0890 legacy=0x7fa5500a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 1 monclient: found mon.noname-b 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54ffff640 1 -- 172.21.15.17:0/2090794629 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55009ccd0 con 0x7fa5500a1380 2024-09-15T21:26:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.898+0000 7fa54e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:26:34.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54ffff640 1 --2- 172.21.15.17:0/2090794629 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5500a1380 0x7fa55013d010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa548009dc0 tx=0x7fa54801c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefe6542435dd25c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 1 -- 172.21.15.17:0/2090794629 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480b2070 con 0x7fa5500a1380 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: dump: 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 1 -- 172.21.15.17:0/2090794629 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5480a57e0 con 0x7fa5500a1380 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 5 monclient: authenticate success, global_id 45443 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 4 set_mon_vals no callback set 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 1 -- 172.21.15.17:0/2090794629 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480a5bb0 con 0x7fa5500a1380 2024-09-15T21:26:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa54e7fc640 10 monclient: dump: 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 10 monclient: shutdown 2024-09-15T21:26:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 -- 172.21.15.17:0/2090794629 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5500a1380 msgr2=0x7fa55013d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 --2- 172.21.15.17:0/2090794629 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5500a1380 0x7fa55013d010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa548009dc0 tx=0x7fa54801c810 comp rx=0 tx=0).stop 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 -- 172.21.15.17:0/2090794629 shutdown_connections 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 --2- 172.21.15.17:0/2090794629 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5500a1380 0x7fa55013d010 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 --2- 172.21.15.17:0/2090794629 >> v2:172.21.15.119:3300/0 conn(0x7fa55009ffc0 0x7fa55013aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 -- 172.21.15.17:0/2090794629 >> 172.21.15.17:0/2090794629 conn(0x7fa5500a66a0 msgr2=0x7fa5500a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 -- 172.21.15.17:0/2090794629 shutdown_connections 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 1 -- 172.21.15.17:0/2090794629 wait complete. 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 10 monclient: build_initial_monmap 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.902+0000 7fa555f9d640 10 monclient: monmap: 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:34.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 Processor -- start 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- start start 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: init 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:34.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 0x7fa550135190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fa55009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5500a0890 0x7fa5501356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): picked mon.noname-a con 0x7fa5500a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5500a1380 0x7fa55011e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): picked mon.noname-b con 0x7fa5500a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa550140100 con 0x7fa5500a0890 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa55013ff80 con 0x7fa5500a1380 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa550140280 con 0x7fa55009ffc0 2024-09-15T21:26:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient(hunting): authenticate will time out at 1518.305074s 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 0x7fa550135190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 0x7fa550135190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40812/0 (socket says 172.21.15.17:40812) 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 -- 172.21.15.17:0/4246716397 learned_addr learned my addr 172.21.15.17:0/4246716397 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): get_auth_request con 0x7fa55009ffc0 auth_method 0 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5500a0890 0x7fa5501356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): get_auth_request con 0x7fa5500a0890 auth_method 0 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:34.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient(hunting): handle_auth_done global_id 45446 payload 824 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54ffff640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5500a1380 0x7fa55011e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 -- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5500a1380 msgr2=0x7fa55011e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5500a1380 0x7fa55011e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 -- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5500a0890 msgr2=0x7fa5501356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5500a0890 0x7fa5501356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:34.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 -- 172.21.15.17:0/4246716397 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa550136440 con 0x7fa55009ffc0 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5500a0890 0x7fa5501356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54f7fe640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 0x7fa550135190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c490 tx=0x7fa54400c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5440074e0 con 0x7fa55009ffc0 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: dump: 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:34.924 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: _finish_auth 0 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5440047f0 con 0x7fa55009ffc0 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:34.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544004be0 con 0x7fa55009ffc0 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 10 monclient: dump: 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:34.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 5 monclient: authenticate success, global_id 45446 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _renew_subs 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa550121fe0 con 0x7fa55009ffc0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54dffb640 4 set_mon_vals no callback set 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _renew_subs 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55011fc80 con 0x7fa55009ffc0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5500a0c70 con 0x7fa55009ffc0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa544011070 con 0x7fa55009ffc0 2024-09-15T21:26:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa53c03b9c0 0x7fa53c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa5440502c0 con 0x7fa55009ffc0 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa53c03b9c0 0x7fa53c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.906+0000 7fa54effd640 10 monclient: get_auth_request con 0x7fa53c03b9c0 auth_method 0 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.910+0000 7fa54effd640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa53c03b9c0 0x7fa53c03de80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa550120ef0 tx=0x7fa540008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.914+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa544004d80 con 0x7fa55009ffc0 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.914+0000 7fa537fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:34.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:34.914+0000 7fa537fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:35.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa555f9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa555f9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa555f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa550121d60 con 0x7fa55009ffc0 2024-09-15T21:26:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa537fff640 1 -- 172.21.15.17:0/4246716397 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fa544013c60 con 0x7fa55009ffc0 2024-09-15T21:26:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa537fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.142+0000 7fa537fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa53c03b9c0 msgr2=0x7fa53c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa53c03b9c0 0x7fa53c03de80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa550120ef0 tx=0x7fa540008040 comp rx=0 tx=0).stop 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 10 monclient: shutdown 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 msgr2=0x7fa550135190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 0x7fa550135190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c490 tx=0x7fa54400c950 comp rx=0 tx=0).stop 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 shutdown_connections 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa53c03b9c0 0x7fa53c03de80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5500a1380 0x7fa55011e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5500a0890 0x7fa5501356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 --2- 172.21.15.17:0/4246716397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa55009ffc0 0x7fa550135190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 >> 172.21.15.17:0/4246716397 conn(0x7fa5500a66a0 msgr2=0x7fa5500a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 shutdown_connections 2024-09-15T21:26:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:35.146+0000 7fa555f9d640 1 -- 172.21.15.17:0/4246716397 wait complete. 2024-09-15T21:26:38.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:38.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient: get_monmap_and_config 2024-09-15T21:26:38.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient: build_initial_monmap 2024-09-15T21:26:38.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient: monmap: 2024-09-15T21:26:38.301 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:38.301 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.301 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:38.300450+0000 2024-09-15T21:26:38.302 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:38.300450+0000 2024-09-15T21:26:38.302 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:38.302 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.302 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:38.302 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:38.303 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:38.303 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:38.303 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:38.303 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 Processor -- start 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 -- start start 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient: init 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f95780a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:38.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f957809ffc0 0x7f957813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): picked mon.noname-c con 0x7f957809ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95780a1380 0x7f957813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): picked mon.noname-a con 0x7f95780a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95780a3130 con 0x7f95780a0890 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9578073df0 con 0x7f95780a1380 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f95780a1bf0 con 0x7f957809ffc0 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.294+0000 7f957fe11640 10 monclient(hunting): authenticate will time out at 1521.695815s 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95780a1380 0x7f957813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95780a1380 0x7f957813d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54038/0 (socket says 172.21.15.17:54038) 2024-09-15T21:26:38.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 -- 172.21.15.17:0/2774403574 learned_addr learned my addr 172.21.15.17:0/2774403574 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): get_auth_request con 0x7f95780a1380 auth_method 0 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957d3ac640 1 --2- 172.21.15.17:0/2774403574 >> v2:172.21.15.119:3300/0 conn(0x7f957809ffc0 0x7f957813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957d3ac640 10 monclient(hunting): get_auth_request con 0x7f957809ffc0 auth_method 0 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957d3ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957d3ac640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957d3ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient(hunting): handle_auth_done global_id 104310 payload 274 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 -- 172.21.15.17:0/2774403574 >> v2:172.21.15.119:3300/0 conn(0x7f957809ffc0 msgr2=0x7f957813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 --2- 172.21.15.17:0/2774403574 >> v2:172.21.15.119:3300/0 conn(0x7f957809ffc0 0x7f957813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 -- 172.21.15.17:0/2774403574 >> v1:172.21.15.119:6789/0 conn(0x7f95780a0890 legacy=0x7f95780a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient: _finish_hunting 0 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 monclient: found mon.noname-a 2024-09-15T21:26:38.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:38.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 -- 172.21.15.17:0/2774403574 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957809ccd0 con 0x7f95780a1380 2024-09-15T21:26:38.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957e3ae640 1 --2- 172.21.15.17:0/2774403574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f957813d010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9570009870 tx=0x7f957001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ef03cb5a0ab320 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 1 -- 172.21.15.17:0/2774403574 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700b0070 con 0x7f95780a1380 2024-09-15T21:26:38.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:38.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: dump: 2024-09-15T21:26:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: _finish_auth 0 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: _check_auth_tickets 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 5 monclient: authenticate success, global_id 104310 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 1 -- 172.21.15.17:0/2774403574 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95700ab600 con 0x7f95780a1380 2024-09-15T21:26:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 4 set_mon_vals no callback set 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 1 -- 172.21.15.17:0/2774403574 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700ab9d0 con 0x7f95780a1380 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957cbab640 10 monclient: dump: 2024-09-15T21:26:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 10 monclient: shutdown 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 -- 172.21.15.17:0/2774403574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 msgr2=0x7f957813d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 --2- 172.21.15.17:0/2774403574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f957813d010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9570009870 tx=0x7f957001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 -- 172.21.15.17:0/2774403574 shutdown_connections 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 --2- 172.21.15.17:0/2774403574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f957813d010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 --2- 172.21.15.17:0/2774403574 >> v2:172.21.15.119:3300/0 conn(0x7f957809ffc0 0x7f957813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 -- 172.21.15.17:0/2774403574 >> 172.21.15.17:0/2774403574 conn(0x7f95780a66a0 msgr2=0x7f95780a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 -- 172.21.15.17:0/2774403574 shutdown_connections 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.298+0000 7f957fe11640 1 -- 172.21.15.17:0/2774403574 wait complete. 2024-09-15T21:26:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: build_initial_monmap 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: monmap: 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 Processor -- start 2024-09-15T21:26:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- start start 2024-09-15T21:26:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: init 2024-09-15T21:26:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f957809ffc0 0x7f957811ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): picked mon.noname-c con 0x7f957809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95780a0890 0x7f9578135900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): picked mon.noname-b con 0x7f95780a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f95781cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): picked mon.noname-a con 0x7f95780a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9578140100 con 0x7f95780a1380 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f957813ff80 con 0x7f95780a0890 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9578140280 con 0x7f957809ffc0 2024-09-15T21:26:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient(hunting): authenticate will time out at 1521.701038s 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f95781cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f957809ffc0 0x7f957811ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f95781cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54052/0 (socket says 172.21.15.17:54052) 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 -- 172.21.15.17:0/2500757044 learned_addr learned my addr 172.21.15.17:0/2500757044 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): get_auth_request con 0x7f95780a1380 auth_method 0 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): get_auth_request con 0x7f957809ffc0 auth_method 0 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957d3ac640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95780a0890 0x7f9578135900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957d3ac640 10 monclient(hunting): get_auth_request con 0x7f95780a0890 auth_method 0 2024-09-15T21:26:38.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957d3ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957d3ac640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957d3ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient(hunting): handle_auth_done global_id 104313 payload 824 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 -- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f957809ffc0 msgr2=0x7f957811ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f957809ffc0 0x7f957811ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 -- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95780a0890 msgr2=0x7f9578135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95780a0890 0x7f9578135900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient: _finish_hunting 0 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 monclient: found mon.noname-a 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 -- 172.21.15.17:0/2500757044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9578120290 con 0x7f95780a1380 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957e3ae640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f95781cd270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95700ae040 tx=0x7f9570019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700b0040 con 0x7f95780a1380 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: dump: 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.334 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95700aa730 con 0x7f95780a1380 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700aab20 con 0x7f95780a1380 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 10 monclient: dump: 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 5 monclient: authenticate success, global_id 104313 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _renew_subs 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f9577fff640 4 set_mon_vals no callback set 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957811df30 con 0x7f95780a1380 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _renew_subs 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781365a0 con 0x7f95780a1380 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95780a0c70 con 0x7f95780a1380 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f95700aacc0 con 0x7f95780a1380 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f956403b9c0 0x7f956403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f95700eac30 con 0x7f95780a1380 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f956403b9c0 0x7f956403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.302+0000 7f957dbad640 10 monclient: get_auth_request con 0x7f956403b9c0 auth_method 0 2024-09-15T21:26:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.306+0000 7f957dbad640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f956403b9c0 0x7f956403de80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f95781204a0 tx=0x7f956c007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.306+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f95700f1080 con 0x7f95780a1380 2024-09-15T21:26:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.306+0000 7f95767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.306+0000 7f95767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f957811ff10 con 0x7f95780a1380 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f95767fc640 1 -- 172.21.15.17:0/2500757044 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f95700160f0 con 0x7f95780a1380 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f95767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f95767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:38.556 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f956403b9c0 msgr2=0x7f956403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f956403b9c0 0x7f956403de80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f95781204a0 tx=0x7f956c007e40 comp rx=0 tx=0).stop 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 10 monclient: shutdown 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 msgr2=0x7f95781cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f95781cd270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95700ae040 tx=0x7f9570019180 comp rx=0 tx=0).stop 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 shutdown_connections 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f956403b9c0 0x7f956403de80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95780a1380 0x7f95781cd270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95780a0890 0x7f9578135900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 --2- 172.21.15.17:0/2500757044 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f957809ffc0 0x7f957811ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 >> 172.21.15.17:0/2500757044 conn(0x7f95780a66a0 msgr2=0x7f95780714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:38.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 shutdown_connections 2024-09-15T21:26:38.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.550+0000 7f957fe11640 1 -- 172.21.15.17:0/2500757044 wait complete. 2024-09-15T21:26:38.575 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:26:38.575 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:26:38.575 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:26:38.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient: get_monmap_and_config 2024-09-15T21:26:38.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient: build_initial_monmap 2024-09-15T21:26:38.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient: monmap: 2024-09-15T21:26:38.692 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:38.692 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.692 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:38.691483+0000 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:38.691483+0000 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:38.693 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 Processor -- start 2024-09-15T21:26:38.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 -- start start 2024-09-15T21:26:38.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient: init 2024-09-15T21:26:38.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff69009d400 0x7ff69009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): picked mon.noname-c con 0x7ff69009d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 0x7ff690134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): picked mon.noname-b con 0x7ff69009dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff690134ba0 0x7ff690134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): picked mon.noname-a con 0x7ff690134ba0 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff6900a1bc0 con 0x7ff690134ba0 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff6900a3490 con 0x7ff69009d400 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff6900a3130 con 0x7ff69009dd20 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff695dc5640 10 monclient(hunting): authenticate will time out at 1522.087342s 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 0x7ff690134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff690134ba0 0x7ff690134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 0x7ff690134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56164/0 (socket says 172.21.15.17:56164) 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 1 -- 172.21.15.17:0/1215534428 learned_addr learned my addr 172.21.15.17:0/1215534428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): get_auth_request con 0x7ff69009dd20 auth_method 0 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): get_auth_request con 0x7ff690134ba0 auth_method 0 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68f7fe640 1 --2- 172.21.15.17:0/1215534428 >> v2:172.21.15.119:3300/0 conn(0x7ff69009d400 0x7ff69009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68f7fe640 10 monclient(hunting): get_auth_request con 0x7ff69009d400 auth_method 0 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient(hunting): handle_auth_done global_id 104316 payload 274 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 -- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 msgr2=0x7ff690134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 --2- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 0x7ff690134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 -- 172.21.15.17:0/1215534428 >> v2:172.21.15.119:3300/0 conn(0x7ff69009d400 msgr2=0x7ff69009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 --2- 172.21.15.17:0/1215534428 >> v2:172.21.15.119:3300/0 conn(0x7ff69009d400 0x7ff69009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 monclient: found mon.noname-a 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68ffff640 1 -- 172.21.15.17:0/1215534428 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69009e7a0 con 0x7ff690134ba0 2024-09-15T21:26:38.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.686+0000 7ff68effd640 1 --2- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 0x7ff690134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68ffff640 1 --2- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690134ba0 0x7ff690134fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff69009d0a0 tx=0x7ff68801c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63aa25c26082de8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 1 -- 172.21.15.17:0/1215534428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6880ac070 con 0x7ff690134ba0 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: dump: 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.705 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 1 -- 172.21.15.17:0/1215534428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6880a6680 con 0x7ff690134ba0 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 1 -- 172.21.15.17:0/1215534428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6880a6a50 con 0x7ff690134ba0 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:38.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff68e7fc640 10 monclient: dump: 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 5 monclient: authenticate success, global_id 104316 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 4 set_mon_vals no callback set 2024-09-15T21:26:38.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 10 monclient: shutdown 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 -- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690134ba0 msgr2=0x7ff690134fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 --2- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690134ba0 0x7ff690134fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff69009d0a0 tx=0x7ff68801c7f0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 -- 172.21.15.17:0/1215534428 shutdown_connections 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 --2- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690134ba0 0x7ff690134fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 --2- 172.21.15.17:0/1215534428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff69009dd20 0x7ff690134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 --2- 172.21.15.17:0/1215534428 >> v2:172.21.15.119:3300/0 conn(0x7ff69009d400 0x7ff69009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 -- 172.21.15.17:0/1215534428 >> 172.21.15.17:0/1215534428 conn(0x7ff6900a66a0 msgr2=0x7ff6900a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 -- 172.21.15.17:0/1215534428 shutdown_connections 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 -- 172.21.15.17:0/1215534428 wait complete. 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 10 monclient: build_initial_monmap 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 10 monclient: monmap: 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:38.708 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 Processor -- start 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 1 -- start start 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.690+0000 7ff695dc5640 10 monclient: init 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:26:38.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d400 0x7ff69011c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): picked mon.noname-a con 0x7ff69009d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009dd20 0x7ff69013a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): picked mon.noname-b con 0x7ff69009dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff690134ba0 0x7ff6901cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): picked mon.noname-c con 0x7ff690134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff69009fb80 con 0x7ff69009d400 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff69009fa00 con 0x7ff69009dd20 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff6900a0890 con 0x7ff690134ba0 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): authenticate will time out at 1522.092477s 2024-09-15T21:26:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff690134ba0 0x7ff6901cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d400 0x7ff69011c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff690134ba0 0x7ff6901cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56180/0 (socket says 172.21.15.17:56180) 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 1 -- 172.21.15.17:0/3429283950 learned_addr learned my addr 172.21.15.17:0/3429283950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): get_auth_request con 0x7ff690134ba0 auth_method 0 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): get_auth_request con 0x7ff69009d400 auth_method 0 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009dd20 0x7ff69013a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): get_auth_request con 0x7ff69009dd20 auth_method 0 2024-09-15T21:26:38.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient(hunting): handle_auth_done global_id 104319 payload 824 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff690134ba0 msgr2=0x7ff6901cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff690134ba0 0x7ff6901cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009dd20 msgr2=0x7ff69013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69009dd20 0x7ff69013a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69013b2b0 con 0x7ff69009d400 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff690134ba0 0x7ff6901cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68f7fe640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d400 0x7ff69011c840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d5f0 tx=0x7ff68000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6800125f0 con 0x7ff69009d400 2024-09-15T21:26:38.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: dump: 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.715 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: _finish_auth 0 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 5 monclient: authenticate success, global_id 104319 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff680019070 con 0x7ff69009d400 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6800129b0 con 0x7ff69009d400 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient: _renew_subs 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6900a1600 con 0x7ff69009d400 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:38.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: dump: 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68dffb640 4 set_mon_vals no callback set 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient: _renew_subs 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:38.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff690133cd0 con 0x7ff69009d400 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 0x7ff6901cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff680012b50 con 0x7ff69009d400 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff69011dca0 con 0x7ff69013aa80 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 0x7ff6901cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient: get_auth_request con 0x7ff69013aa80 auth_method 0 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67c03b9c0 0x7ff67c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff680050950 con 0x7ff69009d400 2024-09-15T21:26:38.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 10 monclient(hunting): handle_auth_done global_id 104322 payload 824 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6900008e0 con 0x7ff69013aa80 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67c03b9c0 0x7ff67c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 10 monclient: get_auth_request con 0x7ff67c03b9c0 auth_method 0 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68effd640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 0x7ff6901cba30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff684011600 tx=0x7ff684011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684015870 con 0x7ff69013aa80 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: dump: 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.720 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.721 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.721 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.721 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff684015870 con 0x7ff69013aa80 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 msgr2=0x7ff6901cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 0x7ff6901cba30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff684011600 tx=0x7ff684011ac0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 msgr2=0x7ff6901cba30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff677fff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69013aa80 0x7ff6901cba30 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff684011600 tx=0x7ff684011ac0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.694+0000 7ff68ffff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67c03b9c0 0x7ff67c03de80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff688009bd0 tx=0x7ff6880a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:38.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 0x7ff6901cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff690134670 con 0x7ff690139d20 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 0x7ff6901cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient: get_auth_request con 0x7ff690139d20 auth_method 0 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 10 monclient(hunting): handle_auth_done global_id 104325 payload 824 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 1 -- 172.21.15.17:0/3429283950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff6901d6860 con 0x7ff690139d20 2024-09-15T21:26:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68effd640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 0x7ff6901cbf70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff684011600 tx=0x7ff6840046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684004230 con 0x7ff690139d20 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 10 monclient: dump: 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7ff684004230 con 0x7ff690139d20 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 msgr2=0x7ff6901cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 0x7ff6901cbf70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff684011600 tx=0x7ff6840046d0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 msgr2=0x7ff6901cbf70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff677fff640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff690139d20 0x7ff6901cbf70 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff684011600 tx=0x7ff6840046d0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 88, 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 52451, 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:38.733 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:38.740 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:38.741 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.743 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67c03b9c0 msgr2=0x7ff67c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff67c03b9c0 0x7ff67c03de80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff688009bd0 tx=0x7ff6880a7000 comp rx=0 tx=0).stop 2024-09-15T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 10 monclient: shutdown 2024-09-15T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d400 msgr2=0x7ff69011c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 --2- 172.21.15.17:0/3429283950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff69009d400 0x7ff69011c840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d5f0 tx=0x7ff68000dab0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68ffff640 1 -- 172.21.15.17:0/3429283950 reap_dead start 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff68ffff640 1 -- 172.21.15.17:0/3429283950 reap_dead start 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 shutdown_connections 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 >> 172.21.15.17:0/3429283950 conn(0x7ff6900a66a0 msgr2=0x7ff690071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 shutdown_connections 2024-09-15T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.702+0000 7ff695dc5640 1 -- 172.21.15.17:0/3429283950 wait complete. 2024-09-15T21:26:38.747 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:26:38.748 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:26:38.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient: get_monmap_and_config 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient: build_initial_monmap 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient: monmap: 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:38.858979+0000 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:38.858979+0000 2024-09-15T21:26:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:38.860 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 1 Processor -- start 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 1 -- start start 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient: init 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f547409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 0x7f5474139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient(hunting): picked mon.noname-b con 0x7f547409def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5474139ff0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54740a3130 con 0x7f5474139ff0 2024-09-15T21:26:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547409e7a0 con 0x7f547409d400 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f54740a1bc0 con 0x7f547409def0 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f547b479640 10 monclient(hunting): authenticate will time out at 1522.254286s 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 0x7f5474139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 0x7f5474139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56190/0 (socket says 172.21.15.17:56190) 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 -- 172.21.15.17:0/1015539159 learned_addr learned my addr 172.21.15.17:0/1015539159 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): get_auth_request con 0x7f547409def0 auth_method 0 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5479a16640 1 --1- 172.21.15.17:0/1015539159 >> v1:172.21.15.17:6789/0 conn(0x7f5474139ff0 0x7f547413c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40480/0 (socket says 172.21.15.17:40480) 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient(hunting): handle_auth_done global_id 45458 payload 274 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 -- 172.21.15.17:0/1015539159 >> v1:172.21.15.119:6789/0 conn(0x7f547409d400 legacy=0x7f547409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 -- 172.21.15.17:0/1015539159 >> v1:172.21.15.17:6789/0 conn(0x7f5474139ff0 legacy=0x7f547413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient: _finish_hunting 0 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 monclient: found mon.noname-b 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.854+0000 7f5478a14640 1 -- 172.21.15.17:0/1015539159 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54740a0320 con 0x7f547409def0 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5478a14640 1 --2- 172.21.15.17:0/1015539159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 0x7f5474139ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54640098a0 tx=0x7f546401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817a8ffaa51d1e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 1 -- 172.21.15.17:0/1015539159 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640b0070 con 0x7f547409def0 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: dump: 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: _finish_auth 0 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 1 -- 172.21.15.17:0/1015539159 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54640ab600 con 0x7f547409def0 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 5 monclient: authenticate success, global_id 45458 2024-09-15T21:26:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 4 set_mon_vals no callback set 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 1 -- 172.21.15.17:0/1015539159 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640ab9d0 con 0x7f547409def0 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546bfff640 10 monclient: dump: 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: shutdown 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1015539159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 msgr2=0x7f5474139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 --2- 172.21.15.17:0/1015539159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 0x7f5474139ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54640098a0 tx=0x7f546401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1015539159 shutdown_connections 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 --2- 172.21.15.17:0/1015539159 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409def0 0x7f5474139ab0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1015539159 >> 172.21.15.17:0/1015539159 conn(0x7f54740a66a0 msgr2=0x7f54740a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1015539159 shutdown_connections 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1015539159 wait complete. 2024-09-15T21:26:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: build_initial_monmap 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: monmap: 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 Processor -- start 2024-09-15T21:26:38.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- start start 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: init 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409d400 0x7f54741cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): picked mon.noname-c con 0x7f547409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f547409def0 0x7f5474134af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): picked mon.noname-b con 0x7f547409def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5474139ff0 0x7f5474135030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): picked mon.noname-a con 0x7f5474139ff0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5474140940 con 0x7f5474139ff0 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54740a3490 con 0x7f547409def0 2024-09-15T21:26:38.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5474140ac0 con 0x7f547409d400 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient(hunting): authenticate will time out at 1522.258848s 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409d400 0x7f54741cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5474139ff0 0x7f5474135030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409d400 0x7f54741cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56196/0 (socket says 172.21.15.17:56196) 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 -- 172.21.15.17:0/1465148112 learned_addr learned my addr 172.21.15.17:0/1465148112 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): get_auth_request con 0x7f547409d400 auth_method 0 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5474139ff0 0x7f5474135030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54090/0 (socket says 172.21.15.17:54090) 2024-09-15T21:26:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): get_auth_request con 0x7f5474139ff0 auth_method 0 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5478a14640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f547409def0 0x7f5474134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5478a14640 10 monclient(hunting): get_auth_request con 0x7f547409def0 auth_method 0 2024-09-15T21:26:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5478a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5478a14640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5478a14640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient(hunting): handle_auth_done global_id 45461 payload 824 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f547409def0 msgr2=0x7f5474134af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f547409def0 0x7f5474134af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5474139ff0 msgr2=0x7f5474135030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5474139ff0 0x7f5474135030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient: _finish_hunting 0 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 monclient: found mon.noname-c 2024-09-15T21:26:38.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741211c0 con 0x7f547409d400 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479a16640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5474139ff0 0x7f5474135030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5479215640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409d400 0x7f54741cfe90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f547000c450 tx=0x7f547000c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470011790 con 0x7f547409d400 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: dump: 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.879 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: _finish_auth 0 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5470017070 con 0x7f547409d400 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 5 monclient: authenticate success, global_id 45461 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: _renew_subs 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547411f140 con 0x7f547409d400 2024-09-15T21:26:38.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470011b90 con 0x7f547409d400 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f5469ffb640 10 monclient: dump: 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.881 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f546b7fe640 4 set_mon_vals no callback set 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: _renew_subs 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.858+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5474120310 con 0x7f547409d400 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f547b479640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f547b479640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 0x7f547411d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f547b479640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5474120d70 con 0x7f547411fca0 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 0x7f547411d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient: get_auth_request con 0x7f547411fca0 auth_method 0 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5470011d30 con 0x7f547409d400 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 10 monclient(hunting): handle_auth_done global_id 45464 payload 824 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54740a3d70 con 0x7f547411fca0 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545803b9f0 0x7f545803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5470050500 con 0x7f547409d400 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5478a14640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 0x7f547411d810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54740a2730 tx=0x7f546401ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640ba030 con 0x7f547411fca0 2024-09-15T21:26:38.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 10 monclient: dump: 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5479a16640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545803b9f0 0x7f545803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5479a16640 10 monclient: get_auth_request con 0x7f545803b9f0 auth_method 0 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f54640abbe0 con 0x7f547411fca0 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 msgr2=0x7f547411d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 0x7f547411d810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54740a2730 tx=0x7f546401ac80 comp rx=0 tx=0).stop 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 msgr2=0x7f547411d810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5469ffb640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411fca0 0x7f547411d810 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54740a2730 tx=0x7f546401ac80 comp rx=0 tx=0).stop 2024-09-15T21:26:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.862+0000 7f5479a16640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545803b9f0 0x7f545803deb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f547411f910 tx=0x7f546c0016b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f547b479640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f547b479640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 0x7f547411e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f547b479640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f547411e670 con 0x7f547411dd50 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 0x7f547411e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient: get_auth_request con 0x7f547411dd50 auth_method 0 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:38.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 10 monclient(hunting): handle_auth_done global_id 45467 payload 824 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 1 -- 172.21.15.17:0/1465148112 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f547409d7e0 con 0x7f547411dd50 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5478a14640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 0x7f547411e130 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54640097f0 tx=0x7f54640191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640ba040 con 0x7f547411dd50 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5469ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5469ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.866+0000 7f5469ffb640 10 monclient: dump: 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:38.888 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7f5464003d30 con 0x7f547411dd50 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 msgr2=0x7f547411e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 0x7f547411e130 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54640097f0 tx=0x7f54640191e0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 msgr2=0x7f547411e130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5469ffb640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547411dd50 0x7f547411e130 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54640097f0 tx=0x7f54640191e0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 88, 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 367469, 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:38.901 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:38.902 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:38.903 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545803b9f0 msgr2=0x7f545803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545803b9f0 0x7f545803deb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f547411f910 tx=0x7f546c0016b0 comp rx=0 tx=0).stop 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 10 monclient: shutdown 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409d400 msgr2=0x7f54741cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 --2- 172.21.15.17:0/1465148112 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f547409d400 0x7f54741cfe90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f547000c450 tx=0x7f547000c910 comp rx=0 tx=0).stop 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5479a16640 1 -- 172.21.15.17:0/1465148112 reap_dead start 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f5479a16640 1 -- 172.21.15.17:0/1465148112 reap_dead start 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 shutdown_connections 2024-09-15T21:26:38.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 >> 172.21.15.17:0/1465148112 conn(0x7f54740a66a0 msgr2=0x7f547413bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 shutdown_connections 2024-09-15T21:26:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:38.870+0000 7f547b479640 1 -- 172.21.15.17:0/1465148112 wait complete. 2024-09-15T21:26:38.906 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:26:38.906 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:26:38.906 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:26:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.010+0000 7f4a15d86640 10 monclient: get_monmap_and_config 2024-09-15T21:26:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.010+0000 7f4a15d86640 10 monclient: build_initial_monmap 2024-09-15T21:26:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.010+0000 7f4a15d86640 10 monclient: monmap: 2024-09-15T21:26:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:39.018163+0000 2024-09-15T21:26:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:39.018163+0000 2024-09-15T21:26:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:39.020 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 1 Processor -- start 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 1 -- start start 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient: init 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4a100a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 0x7f4a1013cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient(hunting): picked mon.noname-c con 0x7f4a100a5580 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:39.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4a100a59d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a100a2c30 con 0x7f4a100a4a90 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a100a19b0 con 0x7f4a100a59d0 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4a100a0f40 con 0x7f4a100a5580 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 10 monclient(hunting): authenticate will time out at 1522.413526s 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4a100a4a90 0x7f4a100a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40482/0 (socket says 172.21.15.17:40482) 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 1 -- 172.21.15.17:0/1906471413 learned_addr learned my addr 172.21.15.17:0/1906471413 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 1 --2- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 0x7f4a1013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): get_auth_request con 0x7f4a100a5580 auth_method 0 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 1 -- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 msgr2=0x7f4a1013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 1 -- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 msgr2=0x7f4a1013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 1 --2- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 0x7f4a1013cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 1 --2- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 0x7f4a1013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246133999 0 0) 0x7f4a00001120 con 0x7f4a100a4a90 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): my global_id is 104334 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f8003530 con 0x7f4a100a4a90 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3452058423 0 0) 0x7f49f8003530 con 0x7f4a100a4a90 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 msgr2=0x7f4a1013cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 --2- 172.21.15.17:0/1906471413 >> v2:172.21.15.119:3300/0 conn(0x7f4a100a5580 0x7f4a1013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 >> v1:172.21.15.119:6789/0 conn(0x7f4a100a59d0 legacy=0x7f4a1013f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a100a1ce0 con 0x7f4a100a4a90 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f4a000018d0 con 0x7f4a100a4a90 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: dump: 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.034 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 >> v1:172.21.15.17:6789/0 conn(0x7f4a100a4a90 legacy=0x7f4a100a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:26:39.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f8003ae0 0x7f49f8003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): picked mon.a con 0x7f49f8003ae0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): picked mon.b con 0x7f49f8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f49f80053d0 0x7f49f800b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): picked mon.c con 0x7f49f80053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100a0f40 con 0x7f49f8003ae0 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f49f800c2d0 con 0x7f49f8004810 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f49f800c4d0 con 0x7f49f80053d0 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f8003ae0 0x7f49f8003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f49f80053d0 0x7f49f800b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): get_auth_request con 0x7f49f8003ae0 auth_method 0 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request con 0x7f49f80053d0 auth_method 0 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): get_auth_request con 0x7f49f8004810 auth_method 0 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:39.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_done global_id 104334 payload 293 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 -- 172.21.15.17:0/1906471413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f8004810 msgr2=0x7f49f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 -- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f8003ae0 msgr2=0x7f49f8003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f8003ae0 0x7f49f8003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 monclient: found mon.c 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 1 -- 172.21.15.17:0/1906471413 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f8003530 con 0x7f49f80053d0 2024-09-15T21:26:39.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient: _finish_auth 0 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 5 monclient: authenticate success, global_id 104334 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.014+0000 7f4a15d86640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0ffff640 1 -- 172.21.15.17:0/1906471413 reap_dead start 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f49f80053d0 0x7f49f800b9f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4a08001e90 tx=0x7f4a0801c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d454681b11af5317 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08001ae0 con 0x7f49f80053d0 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: dump: 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08003b80 con 0x7f49f80053d0 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 1 -- 172.21.15.17:0/1906471413 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080050d0 con 0x7f49f80053d0 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 4 set_mon_vals no callback set 2024-09-15T21:26:39.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a0e7fc640 10 monclient: dump: 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 10 monclient: shutdown 2024-09-15T21:26:39.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 -- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f49f80053d0 msgr2=0x7f49f800b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f49f80053d0 0x7f49f800b9f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4a08001e90 tx=0x7f4a0801c720 comp rx=0 tx=0).stop 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 -- 172.21.15.17:0/1906471413 shutdown_connections 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 --2- 172.21.15.17:0/1906471413 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f49f80053d0 0x7f49f800b9f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 -- 172.21.15.17:0/1906471413 >> 172.21.15.17:0/1906471413 conn(0x7f4a100aabb0 msgr2=0x7f4a100a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 -- 172.21.15.17:0/1906471413 shutdown_connections 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 -- 172.21.15.17:0/1906471413 wait complete. 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 10 monclient: build_initial_monmap 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 10 monclient: monmap: 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:39.043 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.018+0000 7f4a15d86640 1 Processor -- start 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- start start 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: init 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80053d0 0x7f4a101cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): picked mon.noname-a con 0x7f49f80053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10136d90 0x7f4a10137190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): picked mon.noname-b con 0x7f4a10136d90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a101cc370 0x7f4a101cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): picked mon.noname-c con 0x7f4a101cc370 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100a4ff0 con 0x7f49f80053d0 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100a4e70 con 0x7f4a10136d90 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a100a5170 con 0x7f4a101cc370 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): authenticate will time out at 1522.420538s 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80053d0 0x7f4a101cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80053d0 0x7f4a101cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54118/0 (socket says 172.21.15.17:54118) 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 -- 172.21.15.17:0/2166100824 learned_addr learned my addr 172.21.15.17:0/2166100824 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): get_auth_request con 0x7f49f80053d0 auth_method 0 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a101cc370 0x7f4a101cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request con 0x7f4a101cc370 auth_method 0 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10136d90 0x7f4a10137190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): get_auth_request con 0x7f4a10136d90 auth_method 0 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient(hunting): handle_auth_done global_id 104337 payload 824 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a101cc370 msgr2=0x7f4a101cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a101cc370 0x7f4a101cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10136d90 msgr2=0x7f4a10137190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10136d90 0x7f4a10137190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1013f870 con 0x7f49f80053d0 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10136d90 0x7f4a10137190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0f7fe640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80053d0 0x7f4a101cf8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000fc0 tx=0x7f4a00007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007760 con 0x7f49f80053d0 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: dump: 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.048 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: _finish_auth 0 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 5 monclient: authenticate success, global_id 104337 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00018070 con 0x7f49f80053d0 2024-09-15T21:26:39.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: _renew_subs 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1013e630 con 0x7f49f80053d0 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000fb70 con 0x7f49f80053d0 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 10 monclient: dump: 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0dffb640 4 set_mon_vals no callback set 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: _renew_subs 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a100a1280 con 0x7f49f80053d0 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 0x7f4a10138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100a0900 con 0x7f4a101ccd30 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4a0000fd10 con 0x7f49f80053d0 2024-09-15T21:26:39.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f49fc03b9c0 0x7f49fc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 0x7f4a10138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4a00050520 con 0x7f49f80053d0 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient: get_auth_request con 0x7f4a101ccd30 auth_method 0 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f49fc03b9c0 0x7f49fc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 10 monclient: get_auth_request con 0x7f49fc03b9c0 auth_method 0 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 10 monclient(hunting): handle_auth_done global_id 114103 payload 824 2024-09-15T21:26:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a101cd110 con 0x7f4a101ccd30 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0ffff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f49fc03b9c0 0x7f49fc03de80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f49f800c6a0 tx=0x7f4a080ab040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.022+0000 7f4a0effd640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 0x7f4a10138cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000ee0 tx=0x7f4a04012ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400f040 con 0x7f4a101ccd30 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 10 monclient: dump: 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4a04010980 con 0x7f4a101ccd30 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 msgr2=0x7f4a10138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 0x7f4a10138cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000ee0 tx=0x7f4a04012ad0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 msgr2=0x7f4a10138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.026+0000 7f49f7fff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a101ccd30 0x7f4a10138cf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a15d86640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a15d86640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 0x7f4a10139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a15d86640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a10069200 con 0x7f4a10139230 2024-09-15T21:26:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 0x7f4a10139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient: get_auth_request con 0x7f4a10139230 auth_method 0 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 10 monclient(hunting): handle_auth_done global_id 114106 payload 824 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.030+0000 7f4a0effd640 1 -- 172.21.15.17:0/2166100824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a10137790 con 0x7f4a10139230 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a0effd640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 0x7f4a10139640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000ee0 tx=0x7f4a04004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04023570 con 0x7f4a10139230 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 10 monclient: dump: 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f4a04023570 con 0x7f4a10139230 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 msgr2=0x7f4a10139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 0x7f4a10139640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000ee0 tx=0x7f4a04004740 comp rx=0 tx=0).stop 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 msgr2=0x7f4a10139640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f49f7fff640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10139230 0x7f4a10139640 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000ee0 tx=0x7f4a04004740 comp rx=0 tx=0).stop 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:39.059 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 88, 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7599, 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:39.060 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:39.061 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:39.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:39.063 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:39.064 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:39.065 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:39.066 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:39.068 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:39.069 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:39.070 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:39.071 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f49fc03b9c0 msgr2=0x7f49fc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f49fc03b9c0 0x7f49fc03de80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f49f800c6a0 tx=0x7f4a080ab040 comp rx=0 tx=0).stop 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 10 monclient: shutdown 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80053d0 msgr2=0x7f4a101cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 --2- 172.21.15.17:0/2166100824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80053d0 0x7f4a101cf8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000fc0 tx=0x7f4a00007570 comp rx=0 tx=0).stop 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a0ffff640 1 -- 172.21.15.17:0/2166100824 reap_dead start 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a0ffff640 1 -- 172.21.15.17:0/2166100824 reap_dead start 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 shutdown_connections 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 >> 172.21.15.17:0/2166100824 conn(0x7f4a100aabb0 msgr2=0x7f4a1013ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:39.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 shutdown_connections 2024-09-15T21:26:39.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.034+0000 7f4a15d86640 1 -- 172.21.15.17:0/2166100824 wait complete. 2024-09-15T21:26:39.074 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:26:39.074 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:26:39.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:26:39.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.178+0000 7f6e2c48f640 10 monclient: get_monmap_and_config 2024-09-15T21:26:39.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.178+0000 7f6e2c48f640 10 monclient: build_initial_monmap 2024-09-15T21:26:39.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.178+0000 7f6e2c48f640 10 monclient: monmap: 2024-09-15T21:26:39.232 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:39.232 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.232 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:39.185256+0000 2024-09-15T21:26:39.232 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:39.185256+0000 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:39.233 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 1 Processor -- start 2024-09-15T21:26:39.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 1 -- start start 2024-09-15T21:26:39.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient: init 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6e241378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e241383b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 0x7f6e24140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient(hunting): picked mon.noname-b con 0x7f6e2413e4a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e240a1a20 con 0x7f6e241378c0 2024-09-15T21:26:39.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e240a1d00 con 0x7f6e241383b0 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6e240a1fe0 con 0x7f6e2413e4a0 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2c48f640 10 monclient(hunting): authenticate will time out at 1522.581132s 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 0x7f6e24140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2a22b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6e241378c0 0x7f6e24137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40488/0 (socket says 172.21.15.17:40488) 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 0x7f6e24140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56234/0 (socket says 172.21.15.17:56234) 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2a22b640 1 -- 172.21.15.17:0/908633126 learned_addr learned my addr 172.21.15.17:0/908633126 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): get_auth_request con 0x7f6e2413e4a0 auth_method 0 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 1 -- 172.21.15.17:0/908633126 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111295778 0 0) 0x7f6e14001120 con 0x7f6e241378c0 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient(hunting): _negotiate 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient(hunting): my global_id is 104340 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 1 -- 172.21.15.17:0/908633126 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e0c003530 con 0x7f6e241378c0 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient(hunting): handle_auth_done global_id 45473 payload 274 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 -- 172.21.15.17:0/908633126 >> v1:172.21.15.119:6789/0 conn(0x7f6e241383b0 legacy=0x7f6e2413cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 -- 172.21.15.17:0/908633126 >> v1:172.21.15.17:6789/0 conn(0x7f6e241378c0 legacy=0x7f6e24137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient: _finish_hunting 0 2024-09-15T21:26:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 monclient: found mon.noname-b 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 -- 172.21.15.17:0/908633126 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e240a22c0 con 0x7f6e2413e4a0 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e2aa2c640 1 --2- 172.21.15.17:0/908633126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 0x7f6e24140890 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009820 tx=0x7f6e1c01c060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc7daba14b52943 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 1 -- 172.21.15.17:0/908633126 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0b1070 con 0x7f6e2413e4a0 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.182+0000 7f6e29229640 10 monclient: dump: 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 10 monclient: _finish_auth 0 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 10 monclient: _check_auth_tickets 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 5 monclient: authenticate success, global_id 45473 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 1 -- 172.21.15.17:0/908633126 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c0a4810 con 0x7f6e2413e4a0 2024-09-15T21:26:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 4 set_mon_vals no callback set 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 1 -- 172.21.15.17:0/908633126 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0a4bc0 con 0x7f6e2413e4a0 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29229640 10 monclient: dump: 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: shutdown 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/908633126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 msgr2=0x7f6e24140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/908633126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 0x7f6e24140890 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009820 tx=0x7f6e1c01c060 comp rx=0 tx=0).stop 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/908633126 shutdown_connections 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/908633126 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e2413e4a0 0x7f6e24140890 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/908633126 >> 172.21.15.17:0/908633126 conn(0x7f6e240aac40 msgr2=0x7f6e240a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/908633126 shutdown_connections 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/908633126 wait complete. 2024-09-15T21:26:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: build_initial_monmap 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: monmap: 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 Processor -- start 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- start start 2024-09-15T21:26:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: init 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 0x7f6e2409f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): picked mon.noname-c con 0x7f6e241378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e241383b0 0x7f6e24136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): picked mon.noname-a con 0x7f6e241383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e2413e4a0 0x7f6e241370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): picked mon.noname-b con 0x7f6e2413e4a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6e24144290 con 0x7f6e241383b0 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6e24144110 con 0x7f6e2413e4a0 2024-09-15T21:26:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6e24144410 con 0x7f6e241378c0 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient(hunting): authenticate will time out at 1522.586202s 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 0x7f6e2409f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e241383b0 0x7f6e24136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 0x7f6e2409f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56244/0 (socket says 172.21.15.17:56244) 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 -- 172.21.15.17:0/370880952 learned_addr learned my addr 172.21.15.17:0/370880952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): get_auth_request con 0x7f6e241378c0 auth_method 0 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): get_auth_request con 0x7f6e241383b0 auth_method 0 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient(hunting): handle_auth_done global_id 45476 payload 824 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2aa2c640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e2413e4a0 0x7f6e241370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 -- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e2413e4a0 msgr2=0x7f6e241370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e2413e4a0 0x7f6e241370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 -- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e241383b0 msgr2=0x7f6e24136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e241383b0 0x7f6e24136b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient: _finish_hunting 0 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 monclient: found mon.noname-c 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 -- 172.21.15.17:0/370880952 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e240a0640 con 0x7f6e241378c0 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e29a2a640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e241383b0 0x7f6e24136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2a22b640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 0x7f6e2409f120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400c730 tx=0x7f6e1400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14017070 con 0x7f6e241378c0 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: dump: 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: _finish_auth 0 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e14005ac0 con 0x7f6e241378c0 2024-09-15T21:26:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 5 monclient: authenticate success, global_id 45476 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _renew_subs 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2409d4a0 con 0x7f6e241378c0 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14007070 con 0x7f6e241378c0 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 10 monclient: dump: 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e28a28640 4 set_mon_vals no callback set 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _renew_subs 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e240a0080 con 0x7f6e241378c0 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e24137ca0 con 0x7f6e241378c0 2024-09-15T21:26:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.186+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6e1401a070 con 0x7f6e241378c0 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.190+0000 7f6e22ffd640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e1003b9c0 0x7f6e1003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.190+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f6e1404f9c0 con 0x7f6e241378c0 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.190+0000 7f6e29a2a640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e1003b9c0 0x7f6e1003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.190+0000 7f6e29a2a640 10 monclient: get_auth_request con 0x7f6e1003b9c0 auth_method 0 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.190+0000 7f6e29a2a640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e1003b9c0 0x7f6e1003de80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6e240a0ed0 tx=0x7f6e18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.194+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f6e240a1a20 con 0x7f6e241378c0 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.194+0000 7f6e22ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.194+0000 7f6e22ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:39.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.422+0000 7f6e2c48f640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:26:39.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.422+0000 7f6e2c48f640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:26:39.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.422+0000 7f6e2c48f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:39.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.422+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f6e240a0250 con 0x7f6e241378c0 2024-09-15T21:26:39.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.426+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.426+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=10,kv=12,logm=54} crc {auth=3508013303,config=3541143443,health=4112062886,kv=3079713637,logm=234466131}) 2024-09-15T21:26:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.426+0000 7f6e22ffd640 1 -- 172.21.15.17:0/370880952 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f6e1400a5e0 con 0x7f6e241378c0 2024-09-15T21:26:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.426+0000 7f6e22ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:26:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.426+0000 7f6e22ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:39.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.426+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=2,health=12,kv=12,logm=55} crc {auth=302063939,config=3541143443,health=4041312242,kv=3079713637,logm=4158711596}) 2024-09-15T21:26:39.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.426+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.426+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=10,kv=12,logm=54} crc {auth=3508013303,config=3541143443,health=4112062886,kv=3079713637,logm=234466131}) 2024-09-15T21:26:39.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.426+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=21,config=2,health=11,kv=12,logm=54} crc {auth=961236513,config=3541143443,health=1535110898,kv=3079713637,logm=1158035308}) 2024-09-15T21:26:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e1003b9c0 msgr2=0x7f6e1003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e1003b9c0 0x7f6e1003de80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6e240a0ed0 tx=0x7f6e18009040 comp rx=0 tx=0).stop 2024-09-15T21:26:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 10 monclient: shutdown 2024-09-15T21:26:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 msgr2=0x7f6e2409f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 0x7f6e2409f120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400c730 tx=0x7f6e1400cbf0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 shutdown_connections 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6e1003b9c0 0x7f6e1003de80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e2413e4a0 0x7f6e241370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e241383b0 0x7f6e24136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 --2- 172.21.15.17:0/370880952 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6e241378c0 0x7f6e2409f120 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 >> 172.21.15.17:0/370880952 conn(0x7f6e240aac40 msgr2=0x7f6e2406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 shutdown_connections 2024-09-15T21:26:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:39.430+0000 7f6e2c48f640 1 -- 172.21.15.17:0/370880952 wait complete. 2024-09-15T21:26:39.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.430+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.430+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=567500861}) 2024-09-15T21:26:39.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.430+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=207380581}) 2024-09-15T21:26:39.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.430+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.430+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=567500861}) 2024-09-15T21:26:39.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.430+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1189815167}) 2024-09-15T21:26:39.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1045700985}) 2024-09-15T21:26:39.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3639561753}) 2024-09-15T21:26:39.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1045700985}) 2024-09-15T21:26:39.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1616606695}) 2024-09-15T21:26:39.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.434+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=61105202}) 2024-09-15T21:26:39.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817286515}) 2024-09-15T21:26:39.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=61105202}) 2024-09-15T21:26:39.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2183779360}) 2024-09-15T21:26:39.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1498396444}) 2024-09-15T21:26:39.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1905580839}) 2024-09-15T21:26:39.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1498396444}) 2024-09-15T21:26:39.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2497114363}) 2024-09-15T21:26:39.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1809709454}) 2024-09-15T21:26:39.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3284370448}) 2024-09-15T21:26:39.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1809709454}) 2024-09-15T21:26:39.447 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.438+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=367932181}) 2024-09-15T21:26:39.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220611466}) 2024-09-15T21:26:39.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3503461641}) 2024-09-15T21:26:39.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220611466}) 2024-09-15T21:26:39.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2903725291}) 2024-09-15T21:26:39.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1960410008}) 2024-09-15T21:26:39.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1316441957}) 2024-09-15T21:26:39.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1960410008}) 2024-09-15T21:26:39.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.442+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3458765684}) 2024-09-15T21:26:39.452 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:26:39.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232244632}) 2024-09-15T21:26:39.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1957746692}) 2024-09-15T21:26:39.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232244632}) 2024-09-15T21:26:39.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1481933831}) 2024-09-15T21:26:39.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142324203}) 2024-09-15T21:26:39.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414789798}) 2024-09-15T21:26:39.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142324203}) 2024-09-15T21:26:39.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.446+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858355920}) 2024-09-15T21:26:39.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1879669844}) 2024-09-15T21:26:39.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3229234685}) 2024-09-15T21:26:39.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1879669844}) 2024-09-15T21:26:39.455 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1520869545}) 2024-09-15T21:26:39.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3166489415}) 2024-09-15T21:26:39.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1844223930}) 2024-09-15T21:26:39.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3166489415}) 2024-09-15T21:26:39.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.450+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=648396761}) 2024-09-15T21:26:39.459 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.459 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3836080800}) 2024-09-15T21:26:39.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1545015239}) 2024-09-15T21:26:39.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3836080800}) 2024-09-15T21:26:39.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2564380842}) 2024-09-15T21:26:39.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2449579934}) 2024-09-15T21:26:39.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2505725586}) 2024-09-15T21:26:39.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2449579934}) 2024-09-15T21:26:39.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.454+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1508221840}) 2024-09-15T21:26:39.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911182106}) 2024-09-15T21:26:39.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4266450978}) 2024-09-15T21:26:39.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=911182106}) 2024-09-15T21:26:39.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=121951563}) 2024-09-15T21:26:39.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2406191036}) 2024-09-15T21:26:39.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1282582969}) 2024-09-15T21:26:39.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2406191036}) 2024-09-15T21:26:39.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.458+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842886788}) 2024-09-15T21:26:39.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2770923439}) 2024-09-15T21:26:39.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2215986904}) 2024-09-15T21:26:39.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2770923439}) 2024-09-15T21:26:39.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3480097986}) 2024-09-15T21:26:39.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360329974}) 2024-09-15T21:26:39.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3587691556}) 2024-09-15T21:26:39.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360329974}) 2024-09-15T21:26:39.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=395461210}) 2024-09-15T21:26:39.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.462+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918842175}) 2024-09-15T21:26:39.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082389201}) 2024-09-15T21:26:39.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918842175}) 2024-09-15T21:26:39.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4192988884}) 2024-09-15T21:26:39.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2201258364}) 2024-09-15T21:26:39.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3440863294}) 2024-09-15T21:26:39.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2201258364}) 2024-09-15T21:26:39.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2055709164}) 2024-09-15T21:26:39.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4255713194}) 2024-09-15T21:26:39.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=627074399}) 2024-09-15T21:26:39.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4255713194}) 2024-09-15T21:26:39.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.466+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3971717233}) 2024-09-15T21:26:39.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564235871}) 2024-09-15T21:26:39.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1528934696}) 2024-09-15T21:26:39.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564235871}) 2024-09-15T21:26:39.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4257529941}) 2024-09-15T21:26:39.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2704916100}) 2024-09-15T21:26:39.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90126316}) 2024-09-15T21:26:39.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2704916100}) 2024-09-15T21:26:39.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.470+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1927866638}) 2024-09-15T21:26:39.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021069198}) 2024-09-15T21:26:39.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=931398506}) 2024-09-15T21:26:39.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021069198}) 2024-09-15T21:26:39.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593958025}) 2024-09-15T21:26:39.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2886224405}) 2024-09-15T21:26:39.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3293857137}) 2024-09-15T21:26:39.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2886224405}) 2024-09-15T21:26:39.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.474+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=150712482}) 2024-09-15T21:26:39.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1201173392}) 2024-09-15T21:26:39.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2456779277}) 2024-09-15T21:26:39.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1201173392}) 2024-09-15T21:26:39.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4022612823}) 2024-09-15T21:26:39.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322818575}) 2024-09-15T21:26:39.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=392862626}) 2024-09-15T21:26:39.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322818575}) 2024-09-15T21:26:39.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.478+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244810544}) 2024-09-15T21:26:39.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=456836667}) 2024-09-15T21:26:39.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1181565708}) 2024-09-15T21:26:39.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=456836667}) 2024-09-15T21:26:39.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510701562}) 2024-09-15T21:26:39.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1113318095}) 2024-09-15T21:26:39.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2993559972}) 2024-09-15T21:26:39.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1113318095}) 2024-09-15T21:26:39.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=92193539}) 2024-09-15T21:26:39.494 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.494 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3251923531}) 2024-09-15T21:26:39.494 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187678365}) 2024-09-15T21:26:39.494 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3251923531}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.482+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=46969846}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=622307893}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=251821143}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=622307893}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1688203685}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=234145628}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1764284347}) 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=234145628}) 2024-09-15T21:26:39.496 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.486+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2232475012}) 2024-09-15T21:26:39.496 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524824624}) 2024-09-15T21:26:39.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2875702321}) 2024-09-15T21:26:39.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524824624}) 2024-09-15T21:26:39.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208423524}) 2024-09-15T21:26:39.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=609349391}) 2024-09-15T21:26:39.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=967109164}) 2024-09-15T21:26:39.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=609349391}) 2024-09-15T21:26:39.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.490+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=654279160}) 2024-09-15T21:26:39.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2886236046}) 2024-09-15T21:26:39.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1549337120}) 2024-09-15T21:26:39.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2886236046}) 2024-09-15T21:26:39.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229638106}) 2024-09-15T21:26:39.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=957521634}) 2024-09-15T21:26:39.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3023386194}) 2024-09-15T21:26:39.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=957521634}) 2024-09-15T21:26:39.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.494+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=690056172}) 2024-09-15T21:26:39.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467202577}) 2024-09-15T21:26:39.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=963368}) 2024-09-15T21:26:39.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467202577}) 2024-09-15T21:26:39.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=972861991}) 2024-09-15T21:26:39.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.498+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4196094337}) 2024-09-15T21:26:39.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3778373038}) 2024-09-15T21:26:39.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4196094337}) 2024-09-15T21:26:39.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3497960150}) 2024-09-15T21:26:39.508 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3571942277}) 2024-09-15T21:26:39.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3733443455}) 2024-09-15T21:26:39.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3571942277}) 2024-09-15T21:26:39.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.502+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4211376189}) 2024-09-15T21:26:39.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=431987182}) 2024-09-15T21:26:39.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3505342900}) 2024-09-15T21:26:39.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=431987182}) 2024-09-15T21:26:39.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122103150}) 2024-09-15T21:26:39.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1831903361}) 2024-09-15T21:26:39.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2478697229}) 2024-09-15T21:26:39.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1831903361}) 2024-09-15T21:26:39.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.506+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1617717185}) 2024-09-15T21:26:39.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1496015756}) 2024-09-15T21:26:39.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1569464585}) 2024-09-15T21:26:39.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1496015756}) 2024-09-15T21:26:39.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2828543079}) 2024-09-15T21:26:39.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2450190685}) 2024-09-15T21:26:39.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3701008697}) 2024-09-15T21:26:39.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2450190685}) 2024-09-15T21:26:39.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.510+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=834094969}) 2024-09-15T21:26:39.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1580049045}) 2024-09-15T21:26:39.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=265178806}) 2024-09-15T21:26:39.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1580049045}) 2024-09-15T21:26:39.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3821590197}) 2024-09-15T21:26:39.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=305343702}) 2024-09-15T21:26:39.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2726214}) 2024-09-15T21:26:39.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=305343702}) 2024-09-15T21:26:39.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.514+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3143488008}) 2024-09-15T21:26:39.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280611476}) 2024-09-15T21:26:39.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1076791025}) 2024-09-15T21:26:39.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280611476}) 2024-09-15T21:26:39.524 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2187733292}) 2024-09-15T21:26:39.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4073363374}) 2024-09-15T21:26:39.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1576634430}) 2024-09-15T21:26:39.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4073363374}) 2024-09-15T21:26:39.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.518+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2669398263}) 2024-09-15T21:26:39.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1996948705}) 2024-09-15T21:26:39.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2707279526}) 2024-09-15T21:26:39.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1996948705}) 2024-09-15T21:26:39.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229144285}) 2024-09-15T21:26:39.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2951125273}) 2024-09-15T21:26:39.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3584980484}) 2024-09-15T21:26:39.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2951125273}) 2024-09-15T21:26:39.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.522+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4089867998}) 2024-09-15T21:26:39.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1800497638}) 2024-09-15T21:26:39.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3047576925}) 2024-09-15T21:26:39.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1800497638}) 2024-09-15T21:26:39.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169914888}) 2024-09-15T21:26:39.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2810784995}) 2024-09-15T21:26:39.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4195756240}) 2024-09-15T21:26:39.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2810784995}) 2024-09-15T21:26:39.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.526+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2999056860}) 2024-09-15T21:26:39.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.530+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.530+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=701395678}) 2024-09-15T21:26:39.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.530+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3746799715}) 2024-09-15T21:26:39.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.530+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.530+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=701395678}) 2024-09-15T21:26:39.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.530+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2512949026}) 2024-09-15T21:26:39.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=1960885382,mdsmap=3508357332,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875}) 2024-09-15T21:26:39.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=88,mdsmap=3,mgr=8,mgr_metadata=1} crc {logm=2494264232,mdsmap=173658045,mgr=3916736423,mgr_metadata=240635875}) 2024-09-15T21:26:39.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=1960885382,mdsmap=3508357332,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875}) 2024-09-15T21:26:39.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=88,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1} crc {logm=923490188,mdsmap=173658045,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875}) 2024-09-15T21:26:39.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=5,osd_snap=1,osdmap=65} crc {mgrstat=3574048060,mon_config_key=2183229183,monmap=186996059,nvmeofgw=1596038039,osd_metadata=166226148,osd_snap=3823785925,osdmap=2118473519}) 2024-09-15T21:26:39.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=12,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=60} crc {mgrstat=1699642055,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=3068027012,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1567347665}) 2024-09-15T21:26:39.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=5,osd_snap=1,osdmap=65} crc {mgrstat=3574048060,mon_config_key=2183229183,monmap=186996059,nvmeofgw=1596038039,osd_metadata=166226148,osd_snap=3823785925,osdmap=2118473519}) 2024-09-15T21:26:39.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.534+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_snap=2,osdmap=62} crc {mgrstat=173197589,mon_config_key=1866194834,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_snap=841190635,osdmap=2936429989}) 2024-09-15T21:26:39.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.538+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.538+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1632563062}) 2024-09-15T21:26:39.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.538+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2081882113}) 2024-09-15T21:26:39.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.538+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.538+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1632563062}) 2024-09-15T21:26:39.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.538+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2109045396}) 2024-09-15T21:26:39.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.542+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.542+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=235884577}) 2024-09-15T21:26:39.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.542+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=71990868}) 2024-09-15T21:26:39.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.542+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.542+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=235884577}) 2024-09-15T21:26:39.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.542+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2294992192}) 2024-09-15T21:26:39.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.546+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.546+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=218688813}) 2024-09-15T21:26:39.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.546+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3308193258}) 2024-09-15T21:26:39.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.546+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.546+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=218688813}) 2024-09-15T21:26:39.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.546+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3622394692}) 2024-09-15T21:26:39.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.562+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.562+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=25} crc {osdmap=3196278398}) 2024-09-15T21:26:39.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.562+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=24} crc {osdmap=1997539722}) 2024-09-15T21:26:39.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.562+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:26:39.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.562+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=25} crc {osdmap=3196278398}) 2024-09-15T21:26:39.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:26:39.562+0000 7ff55acd2640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=27} crc {osdmap=1375952498}) 2024-09-15T21:26:40.453 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:26:40.453 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:40.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient: get_monmap_and_config 2024-09-15T21:26:40.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient: build_initial_monmap 2024-09-15T21:26:40.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient: monmap: 2024-09-15T21:26:40.584 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:40.584 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:40.583888+0000 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:40.583888+0000 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:40.585 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 1 Processor -- start 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 1 -- start start 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient: init 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96c409d400 0x7f96c409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient(hunting): picked mon.noname-c con 0x7f96c409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f96c409dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f96c4134d70 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c40a3490 con 0x7f96c409dd20 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c40a0320 con 0x7f96c4134d70 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f96c40a1bc0 con 0x7f96c409d400 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96cc18f640 10 monclient(hunting): authenticate will time out at 1523.979744s 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96c972a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96c409dd20 0x7f96c4134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40494/0 (socket says 172.21.15.17:40494) 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.578+0000 7f96c972a640 1 -- 172.21.15.17:0/2456408590 learned_addr learned my addr 172.21.15.17:0/2456408590 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 1 --2- 172.21.15.17:0/2456408590 >> v2:172.21.15.119:3300/0 conn(0x7f96c409d400 0x7f96c409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): get_auth_request con 0x7f96c409d400 auth_method 0 2024-09-15T21:26:40.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 355364566 0 0) 0x7f96c40a3490 con 0x7f96c409dd20 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): _negotiate 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): my global_id is 104346 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96b0003530 con 0x7f96c409dd20 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2433671433 0 0) 0x7f96b0003530 con 0x7f96c409dd20 2024-09-15T21:26:40.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 >> v2:172.21.15.119:3300/0 conn(0x7f96c409d400 msgr2=0x7f96c409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 --2- 172.21.15.17:0/2456408590 >> v2:172.21.15.119:3300/0 conn(0x7f96c409d400 0x7f96c409d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 >> v1:172.21.15.119:6789/0 conn(0x7f96c4134d70 legacy=0x7f96c413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: _finish_hunting 0 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c409e7a0 con 0x7f96c409dd20 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: _finish_auth 0 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: _check_auth_tickets 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f96b4001720 con 0x7f96c409dd20 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 5 monclient: authenticate success, global_id 104346 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:40.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: dump: 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:40.595 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 >> v1:172.21.15.17:6789/0 conn(0x7f96c409dd20 legacy=0x7f96c4134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b0003b00 0x7f96b0003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): picked mon.a con 0x7f96b0003b00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b0004810 0x7f96b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): picked mon.b con 0x7f96b0004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:40.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b00053d0 0x7f96b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): picked mon.c con 0x7f96b00053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40a1bc0 con 0x7f96b0003b00 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f96b000c2f0 con 0x7f96b0004810 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f96b000c4f0 con 0x7f96b00053d0 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b00053d0 0x7f96b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b0003b00 0x7f96b0003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): get_auth_request con 0x7f96b00053d0 auth_method 0 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): get_auth_request con 0x7f96b0003b00 auth_method 0 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c972a640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b0004810 0x7f96b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c972a640 10 monclient(hunting): get_auth_request con 0x7f96b0004810 auth_method 0 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c972a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c972a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c972a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient(hunting): handle_auth_done global_id 104346 payload 293 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 -- 172.21.15.17:0/2456408590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b0004810 msgr2=0x7f96b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b0004810 0x7f96b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 -- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b0003b00 msgr2=0x7f96b0003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b0003b00 0x7f96b0003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient: _finish_hunting 0 2024-09-15T21:26:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 monclient: found mon.c 2024-09-15T21:26:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 -- 172.21.15.17:0/2456408590 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0003530 con 0x7f96b00053d0 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient: _finish_auth 0 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 10 monclient: _check_auth_tickets 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 -- 172.21.15.17:0/2456408590 reap_dead start 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96ca72c640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b00053d0 0x7f96b000ba10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96bc001e90 tx=0x7f96bc01d180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7a501a72596815 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc001ae0 con 0x7f96b00053d0 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: dump: 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc003b80 con 0x7f96b00053d0 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 4 set_mon_vals no callback set 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 1 -- 172.21.15.17:0/2456408590 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0050d0 con 0x7f96b00053d0 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96c8f29640 10 monclient: dump: 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 10 monclient: shutdown 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 1 -- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b00053d0 msgr2=0x7f96b000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.582+0000 7f96cc18f640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b00053d0 0x7f96b000ba10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96bc001e90 tx=0x7f96bc01d180 comp rx=0 tx=0).stop 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/2456408590 shutdown_connections 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 --2- 172.21.15.17:0/2456408590 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96b00053d0 0x7f96b000ba10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/2456408590 >> 172.21.15.17:0/2456408590 conn(0x7f96c40a66a0 msgr2=0x7f96c40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/2456408590 shutdown_connections 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/2456408590 wait complete. 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: build_initial_monmap 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: monmap: 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 Processor -- start 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- start start 2024-09-15T21:26:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: init 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 0x7f96c41cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): picked mon.noname-a con 0x7f96b00053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 0x7f96c413cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): picked mon.noname-c con 0x7f96c413a190 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c413d400 0x7f96c4137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): picked mon.noname-b con 0x7f96c413d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40a32b0 con 0x7f96b00053d0 2024-09-15T21:26:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40a3130 con 0x7f96c413d400 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f96c409ccd0 con 0x7f96c413a190 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient(hunting): authenticate will time out at 1523.985697s 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 0x7f96c41cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 0x7f96c413cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 0x7f96c413cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56266/0 (socket says 172.21.15.17:56266) 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 0x7f96c41cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54154/0 (socket says 172.21.15.17:54154) 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 -- 172.21.15.17:0/3914393067 learned_addr learned my addr 172.21.15.17:0/3914393067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): get_auth_request con 0x7f96c413a190 auth_method 0 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): get_auth_request con 0x7f96b00053d0 auth_method 0 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ca72c640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c413d400 0x7f96c4137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ca72c640 10 monclient(hunting): get_auth_request con 0x7f96c413d400 auth_method 0 2024-09-15T21:26:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ca72c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ca72c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ca72c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient(hunting): handle_auth_done global_id 45479 payload 824 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 -- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c413d400 msgr2=0x7f96c4137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c413d400 0x7f96c4137ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 -- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 msgr2=0x7f96c41cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 0x7f96c41cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient: _finish_hunting 0 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 monclient: found mon.noname-c 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 -- 172.21.15.17:0/3914393067 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c40a1410 con 0x7f96c413a190 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c9f2b640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 0x7f96c41cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96c972a640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 0x7f96c413cec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96b400e5d0 tx=0x7f96b400e9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4013b10 con 0x7f96c413a190 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: dump: 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: _finish_auth 0 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 5 monclient: authenticate success, global_id 45479 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b4017070 con 0x7f96c413a190 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _renew_subs 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c40a4da0 con 0x7f96c413a190 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b401c470 con 0x7f96c413a190 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 10 monclient: dump: 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96bbfff640 4 set_mon_vals no callback set 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _renew_subs 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c413b0f0 con 0x7f96c413a190 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c409d7e0 con 0x7f96c413a190 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f96b40253f0 con 0x7f96c413a190 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f96a803b9c0 0x7f96a803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.586+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f96b4054510 con 0x7f96c413a190 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.590+0000 7f96c9f2b640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f96a803b9c0 0x7f96a803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.590+0000 7f96c9f2b640 10 monclient: get_auth_request con 0x7f96a803b9c0 auth_method 0 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.590+0000 7f96c9f2b640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f96a803b9c0 0x7f96a803de80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009f50 tx=0x7f96c0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.594+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f96b4019c40 con 0x7f96c413a190 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.594+0000 7f96ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.594+0000 7f96ba7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:40.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96cc18f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:40.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96cc18f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:40.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96cc18f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f96c406d020 con 0x7f96c413a190 2024-09-15T21:26:40.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96ba7fc640 1 -- 172.21.15.17:0/3914393067 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f96b40243a0 con 0x7f96c413a190 2024-09-15T21:26:40.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:40.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.830+0000 7f96ba7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:40.838 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:40.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f96a803b9c0 msgr2=0x7f96a803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f96a803b9c0 0x7f96a803de80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009f50 tx=0x7f96c0009190 comp rx=0 tx=0).stop 2024-09-15T21:26:40.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 10 monclient: shutdown 2024-09-15T21:26:40.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:40.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 msgr2=0x7f96c413cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:40.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 0x7f96c413cec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96b400e5d0 tx=0x7f96b400e9d0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 shutdown_connections 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f96a803b9c0 0x7f96a803de80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c413d400 0x7f96c4137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f96c413a190 0x7f96c413cec0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 --2- 172.21.15.17:0/3914393067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96b00053d0 0x7f96c41cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 >> 172.21.15.17:0/3914393067 conn(0x7f96c40a66a0 msgr2=0x7f96c4130c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 shutdown_connections 2024-09-15T21:26:40.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.834+0000 7f96cc18f640 1 -- 172.21.15.17:0/3914393067 wait complete. 2024-09-15T21:26:40.856 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:26:40.856 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:26:40.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:26:40.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.978+0000 7f4e88b59640 10 monclient: get_monmap_and_config 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.978+0000 7f4e88b59640 10 monclient: build_initial_monmap 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.978+0000 7f4e88b59640 10 monclient: monmap: 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:40.984968+0000 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:40.984968+0000 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:40.989 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.978+0000 7f4e88b59640 1 Processor -- start 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.978+0000 7f4e88b59640 1 -- start start 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.978+0000 7f4e88b59640 10 monclient: init 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841378c0 0x7f4e84137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): picked mon.noname-b con 0x7f4e841378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e841381e0 0x7f4e840a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): picked mon.noname-c con 0x7f4e841381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:40.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 0x7f4e840a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): picked mon.noname-a con 0x7f4e840a5040 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4e840a1d00 con 0x7f4e840a5040 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4e840a1fe0 con 0x7f4e841381e0 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4e840a22c0 con 0x7f4e841378c0 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient(hunting): authenticate will time out at 1524.380314s 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 0x7f4e840a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841378c0 0x7f4e84137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 0x7f4e840a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54168/0 (socket says 172.21.15.17:54168) 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 1 -- 172.21.15.17:0/4148331205 learned_addr learned my addr 172.21.15.17:0/4148331205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:40.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): get_auth_request con 0x7f4e840a5040 auth_method 0 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): get_auth_request con 0x7f4e841378c0 auth_method 0 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/4148331205 >> v2:172.21.15.119:3300/0 conn(0x7f4e841381e0 0x7f4e840a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): get_auth_request con 0x7f4e841381e0 auth_method 0 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient(hunting): handle_auth_done global_id 45482 payload 274 2024-09-15T21:26:40.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 -- 172.21.15.17:0/4148331205 >> v2:172.21.15.119:3300/0 conn(0x7f4e841381e0 msgr2=0x7f4e840a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 --2- 172.21.15.17:0/4148331205 >> v2:172.21.15.119:3300/0 conn(0x7f4e841381e0 0x7f4e840a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 -- 172.21.15.17:0/4148331205 >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 msgr2=0x7f4e840a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 --2- 172.21.15.17:0/4148331205 >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 0x7f4e840a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient: _finish_hunting 0 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 monclient: found mon.noname-b 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 -- 172.21.15.17:0/4148331205 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e840a4490 con 0x7f4e841378c0 2024-09-15T21:26:40.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e82d9d640 1 --2- 172.21.15.17:0/4148331205 >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 0x7f4e840a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:41.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8259c640 1 --2- 172.21.15.17:0/4148331205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841378c0 0x7f4e84137ca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4e840a7b50 tx=0x7f4e7401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c930eb8230aaf870 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 1 -- 172.21.15.17:0/4148331205 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e740b0070 con 0x7f4e841378c0 2024-09-15T21:26:41.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:41.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: dump: 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 5 monclient: authenticate success, global_id 45482 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 1 -- 172.21.15.17:0/4148331205 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e740ab600 con 0x7f4e841378c0 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 1 -- 172.21.15.17:0/4148331205 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e740ab9d0 con 0x7f4e841378c0 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e88b59640 4 set_mon_vals no callback set 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.982+0000 7f4e8159a640 10 monclient: dump: 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.005 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: shutdown 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/4148331205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841378c0 msgr2=0x7f4e84137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- 172.21.15.17:0/4148331205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841378c0 0x7f4e84137ca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4e840a7b50 tx=0x7f4e7401c3d0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/4148331205 shutdown_connections 2024-09-15T21:26:41.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- 172.21.15.17:0/4148331205 >> v2:172.21.15.119:3300/0 conn(0x7f4e841381e0 0x7f4e840a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- 172.21.15.17:0/4148331205 >> v2:172.21.15.17:3300/0 conn(0x7f4e840a5040 0x7f4e840a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- 172.21.15.17:0/4148331205 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841378c0 0x7f4e84137ca0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/4148331205 >> 172.21.15.17:0/4148331205 conn(0x7f4e840aac40 msgr2=0x7f4e840a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/4148331205 shutdown_connections 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/4148331205 wait complete. 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: monmap: 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.007 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 Processor -- start 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- start start 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: init 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e840a5040 0x7f4e841cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): picked mon.noname-a con 0x7f4e840a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e841378c0 0x7f4e84136b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): picked mon.noname-b con 0x7f4e841378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:41.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841381e0 0x7f4e8413ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): picked mon.noname-c con 0x7f4e841381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84141730 con 0x7f4e840a5040 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841415b0 con 0x7f4e841378c0 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4e841418b0 con 0x7f4e841381e0 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): authenticate will time out at 1524.385774s 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e840a5040 0x7f4e841cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e840a5040 0x7f4e841cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54178/0 (socket says 172.21.15.17:54178) 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 -- 172.21.15.17:0/460688480 learned_addr learned my addr 172.21.15.17:0/460688480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): get_auth_request con 0x7f4e840a5040 auth_method 0 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841381e0 0x7f4e8413ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): get_auth_request con 0x7f4e841381e0 auth_method 0 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e841378c0 0x7f4e84136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient(hunting): handle_auth_done global_id 104355 payload 824 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841381e0 msgr2=0x7f4e8413ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841381e0 0x7f4e8413ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e841378c0 msgr2=0x7f4e84136b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e841378c0 0x7f4e84136b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 monclient: found mon.noname-a 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:26:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e84122240 con 0x7f4e840a5040 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e81d9b640 10 monclient: get_auth_request con 0x7f4e841378c0 auth_method 0 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e841378c0 0x7f4e84136b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e82d9d640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4e841381e0 0x7f4e8413ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e8259c640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e840a5040 0x7f4e841cfbe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e740ae040 tx=0x7f4e74019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e740b8020 con 0x7f4e840a5040 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:26:41.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: dump: 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e740b4410 con 0x7f4e840a5040 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e740b0040 con 0x7f4e840a5040 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e72ffd640 10 monclient: dump: 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e80d99640 4 set_mon_vals no callback set 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 5 monclient: authenticate success, global_id 104355 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: _renew_subs 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e84122520 con 0x7f4e840a5040 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: _renew_subs 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e84121c80 con 0x7f4e840a5040 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 0x7f4e8413d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.986+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84137ca0 con 0x7f4e84120d20 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 0x7f4e8413d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient: get_auth_request con 0x7f4e84120d20 auth_method 0 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4e740bb020 con 0x7f4e840a5040 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_done global_id 104358 payload 824 2024-09-15T21:26:41.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e84121ec0 con 0x7f4e84120d20 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e6403b9c0 0x7f4e6403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4e740ea1c0 con 0x7f4e840a5040 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 0x7f4e8413d360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e780107a0 tx=0x7f4e78010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800f040 con 0x7f4e84120d20 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 10 monclient: dump: 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e82d9d640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e6403b9c0 0x7f4e6403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e82d9d640 10 monclient: get_auth_request con 0x7f4e6403b9c0 auth_method 0 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4e78004220 con 0x7f4e84120d20 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 msgr2=0x7f4e8413d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 0x7f4e8413d360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e780107a0 tx=0x7f4e78010c60 comp rx=0 tx=0).stop 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 msgr2=0x7f4e8413d360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e72ffd640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84120d20 0x7f4e8413d360 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.990+0000 7f4e82d9d640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e6403b9c0 0x7f4e6403de80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4e84120870 tx=0x7f4e7c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e88b59640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e88b59640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 0x7f4e8413d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e88b59640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84069270 con 0x7f4e841370c0 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 0x7f4e8413d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient: get_auth_request con 0x7f4e841370c0 auth_method 0 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 10 monclient(hunting): handle_auth_done global_id 104361 payload 824 2024-09-15T21:26:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 1 -- 172.21.15.17:0/460688480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e8406cba0 con 0x7f4e841370c0 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e81d9b640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 0x7f4e8413d8a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e780107a0 tx=0x7f4e780046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78003e60 con 0x7f4e841370c0 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 10 monclient: dump: 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7f4e78003e60 con 0x7f4e841370c0 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 msgr2=0x7f4e8413d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 0x7f4e8413d8a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e780107a0 tx=0x7f4e780046d0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 msgr2=0x7f4e8413d8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.994+0000 7f4e72ffd640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e841370c0 0x7f4e8413d8a0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e780107a0 tx=0x7f4e780046d0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 88, 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 54743, 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.023 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.024 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:41.025 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.026 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:41.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:41.031 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:41.032 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.034 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:41.034 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e6403b9c0 msgr2=0x7f4e6403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4e6403b9c0 0x7f4e6403de80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4e84120870 tx=0x7f4e7c00b040 comp rx=0 tx=0).stop 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 10 monclient: shutdown 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e840a5040 msgr2=0x7f4e841cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 --2- 172.21.15.17:0/460688480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e840a5040 0x7f4e841cfbe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e740ae040 tx=0x7f4e74019170 comp rx=0 tx=0).stop 2024-09-15T21:26:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e82d9d640 1 -- 172.21.15.17:0/460688480 reap_dead start 2024-09-15T21:26:41.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e82d9d640 1 -- 172.21.15.17:0/460688480 reap_dead start 2024-09-15T21:26:41.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 shutdown_connections 2024-09-15T21:26:41.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 >> 172.21.15.17:0/460688480 conn(0x7f4e840aac40 msgr2=0x7f4e840a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 shutdown_connections 2024-09-15T21:26:41.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:40.998+0000 7f4e88b59640 1 -- 172.21.15.17:0/460688480 wait complete. 2024-09-15T21:26:41.036 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:26:41.036 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:26:41.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:26:41.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient: get_monmap_and_config 2024-09-15T21:26:41.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient: monmap: 2024-09-15T21:26:41.162 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:41.162266+0000 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:41.162266+0000 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:41.163 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 Processor -- start 2024-09-15T21:26:41.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 -- start start 2024-09-15T21:26:41.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient: init 2024-09-15T21:26:41.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70181378c0 0x7f7018137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): picked mon.noname-a con 0x7f70181378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70181381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f70180a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): picked mon.noname-b con 0x7f70180a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70180a1d00 con 0x7f70181381e0 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f70180a1fe0 con 0x7f70181378c0 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f70180a22c0 con 0x7f70180a5210 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701f305640 10 monclient(hunting): authenticate will time out at 1524.557643s 2024-09-15T21:26:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f70180a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70181378c0 0x7f7018137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f70180a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56290/0 (socket says 172.21.15.17:56290) 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 -- 172.21.15.17:0/3585894200 learned_addr learned my addr 172.21.15.17:0/3585894200 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): get_auth_request con 0x7f70180a5210 auth_method 0 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): get_auth_request con 0x7f70181378c0 auth_method 0 2024-09-15T21:26:41.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d0a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient(hunting): handle_auth_done global_id 45488 payload 274 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 -- 172.21.15.17:0/3585894200 >> v2:172.21.15.17:3300/0 conn(0x7f70181378c0 msgr2=0x7f7018137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 --2- 172.21.15.17:0/3585894200 >> v2:172.21.15.17:3300/0 conn(0x7f70181378c0 0x7f7018137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 -- 172.21.15.17:0/3585894200 >> v1:172.21.15.17:6789/0 conn(0x7f70181381e0 legacy=0x7f70180a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 monclient: found mon.noname-b 2024-09-15T21:26:41.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.158+0000 7f701d8a2640 1 -- 172.21.15.17:0/3585894200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70180a4490 con 0x7f70180a5210 2024-09-15T21:26:41.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 1 --2- 172.21.15.17:0/3585894200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f70180a56a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f70080098a0 tx=0x7f700801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec5b5fe314b5c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 1 -- 172.21.15.17:0/3585894200 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70080ab620 con 0x7f70180a5210 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: dump: 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.175 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 1 -- 172.21.15.17:0/3585894200 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70080b0070 con 0x7f70180a5210 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 1 -- 172.21.15.17:0/3585894200 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70080ab9e0 con 0x7f70180a5210 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 5 monclient: authenticate success, global_id 45488 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 4 set_mon_vals no callback set 2024-09-15T21:26:41.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7017fff640 10 monclient: dump: 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.177 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: shutdown 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/3585894200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 msgr2=0x7f70180a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 --2- 172.21.15.17:0/3585894200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f70180a56a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f70080098a0 tx=0x7f700801c400 comp rx=0 tx=0).stop 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/3585894200 shutdown_connections 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 --2- 172.21.15.17:0/3585894200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f70180a56a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 --2- 172.21.15.17:0/3585894200 >> v2:172.21.15.17:3300/0 conn(0x7f70181378c0 0x7f7018137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/3585894200 >> 172.21.15.17:0/3585894200 conn(0x7f70180aac40 msgr2=0x7f70180a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/3585894200 shutdown_connections 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/3585894200 wait complete. 2024-09-15T21:26:41.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: monmap: 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 Processor -- start 2024-09-15T21:26:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- start start 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: init 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f7018136700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): picked mon.noname-c con 0x7f70180a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70181378c0 0x7f7018136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): picked mon.noname-b con 0x7f70181378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70181381e0 0x7f701813cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): picked mon.noname-a con 0x7f70181381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70181425c0 con 0x7f70181381e0 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7018138700 con 0x7f70181378c0 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7018142740 con 0x7f70180a5210 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient(hunting): authenticate will time out at 1524.562492s 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f7018136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70181381e0 0x7f701813cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f7018136700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56298/0 (socket says 172.21.15.17:56298) 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 -- 172.21.15.17:0/1936195428 learned_addr learned my addr 172.21.15.17:0/1936195428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70181381e0 0x7f701813cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54208/0 (socket says 172.21.15.17:54208) 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): get_auth_request con 0x7f70180a5210 auth_method 0 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): get_auth_request con 0x7f70181381e0 auth_method 0 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701c8a0640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70181378c0 0x7f7018136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701c8a0640 10 monclient(hunting): get_auth_request con 0x7f70181378c0 auth_method 0 2024-09-15T21:26:41.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701c8a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701c8a0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701c8a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient(hunting): handle_auth_done global_id 45491 payload 824 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70181378c0 msgr2=0x7f7018136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70181378c0 0x7f7018136c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70181381e0 msgr2=0x7f701813cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70181381e0 0x7f701813cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 monclient: found mon.noname-c 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7018120fb0 con 0x7f70180a5210 2024-09-15T21:26:41.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d8a2640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70181381e0 0x7f701813cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701d0a1640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f7018136700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f701000e930 tx=0x7f701000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010017070 con 0x7f70180a5210 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: dump: 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.184 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 5 monclient: authenticate success, global_id 45491 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: _renew_subs 2024-09-15T21:26:41.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701000ca00 con 0x7f70180a5210 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7018121290 con 0x7f70180a5210 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701000cdb0 con 0x7f70180a5210 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f7015ffb640 10 monclient: dump: 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.186 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: _renew_subs 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f70180a8910 con 0x7f70180a5210 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.162+0000 7f70177fe640 4 set_mon_vals no callback set 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701f305640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701f305640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 0x7f701813d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7018120df0 con 0x7f701813d0f0 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f701001a070 con 0x7f70180a5210 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 0x7f701813d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient: get_auth_request con 0x7f701813d0f0 auth_method 0 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f700403b9f0 0x7f700403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 10 monclient(hunting): handle_auth_done global_id 45494 payload 824 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70180a56a0 con 0x7f701813d0f0 2024-09-15T21:26:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7010050750 con 0x7f70180a5210 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701c8a0640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 0x7f701813d4d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f701813ebf0 tx=0x7f700c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c003b90 con 0x7f701813d0f0 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 10 monclient: dump: 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.189 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701d8a2640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f700403b9f0 0x7f700403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701d8a2640 10 monclient: get_auth_request con 0x7f700403b9f0 auth_method 0 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f700c003b90 con 0x7f701813d0f0 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 msgr2=0x7f701813d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 0x7f701813d4d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f701813ebf0 tx=0x7f700c00ecd0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 msgr2=0x7f701813d4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f7015ffb640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813d0f0 0x7f701813d4d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.166+0000 7f701d8a2640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f700403b9f0 0x7f700403deb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f70080af000 tx=0x7f70080a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701f305640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701f305640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 0x7f7018138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701f305640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7018069270 con 0x7f701813da10 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 0x7f7018138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient: get_auth_request con 0x7f701813da10 auth_method 0 2024-09-15T21:26:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.170+0000 7f701c8a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701c8a0640 10 monclient(hunting): handle_auth_done global_id 45497 payload 824 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701c8a0640 1 -- 172.21.15.17:0/1936195428 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f701813ede0 con 0x7f701813da10 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701c8a0640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 0x7f7018138d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f701813ebf0 tx=0x7f700c001260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c01ccc0 con 0x7f701813da10 2024-09-15T21:26:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 10 monclient: dump: 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7f700c0050d0 con 0x7f701813da10 2024-09-15T21:26:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:41.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:41.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 msgr2=0x7f7018138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 0x7f7018138d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f701813ebf0 tx=0x7f700c001260 comp rx=0 tx=0).stop 2024-09-15T21:26:41.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 msgr2=0x7f7018138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:41.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f7015ffb640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f701813da10 0x7f7018138d60 secure :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f701813ebf0 tx=0x7f700c001260 comp rx=0 tx=0).stop 2024-09-15T21:26:41.196 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:41.196 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:41.196 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:41.196 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 88, 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 369773, 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.197 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:41.198 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:41.199 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:41.200 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.201 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:41.204 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:41.205 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:41.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.208 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:41.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.210 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.210 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:41.210 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f700403b9f0 msgr2=0x7f700403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f700403b9f0 0x7f700403deb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f70080af000 tx=0x7f70080a7000 comp rx=0 tx=0).stop 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 10 monclient: shutdown 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 msgr2=0x7f7018136700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 --2- 172.21.15.17:0/1936195428 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70180a5210 0x7f7018136700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f701000e930 tx=0x7f701000edf0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701d8a2640 1 -- 172.21.15.17:0/1936195428 reap_dead start 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701d8a2640 1 -- 172.21.15.17:0/1936195428 reap_dead start 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 shutdown_connections 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 >> 172.21.15.17:0/1936195428 conn(0x7f70180aac40 msgr2=0x7f701809da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 shutdown_connections 2024-09-15T21:26:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.174+0000 7f701f305640 1 -- 172.21.15.17:0/1936195428 wait complete. 2024-09-15T21:26:41.213 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:26:41.213 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:26:41.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:26:41.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.322+0000 7fb9904ca640 10 monclient: get_monmap_and_config 2024-09-15T21:26:41.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.322+0000 7fb9904ca640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.322+0000 7fb9904ca640 10 monclient: monmap: 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:41.329532+0000 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:41.329532+0000 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:41.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 Processor -- start 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 -- start start 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient: init 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9881378c0 0x7fb988137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): picked mon.noname-b con 0x7fb9881378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9881381e0 0x7fb9880a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): picked mon.noname-c con 0x7fb9881381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb9880a5040 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9880a1d00 con 0x7fb9880a5040 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb9880a1fe0 con 0x7fb9881381e0 2024-09-15T21:26:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb9880a22c0 con 0x7fb9881378c0 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient(hunting): authenticate will time out at 1524.724907s 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98ea67640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb9880a5040 0x7fb9880a54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40508/0 (socket says 172.21.15.17:40508) 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98ea67640 1 -- 172.21.15.17:0/1160892445 learned_addr learned my addr 172.21.15.17:0/1160892445 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 --2- 172.21.15.17:0/1160892445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9881378c0 0x7fb988137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): get_auth_request con 0x7fb9881378c0 auth_method 0 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98da65640 1 --2- 172.21.15.17:0/1160892445 >> v2:172.21.15.119:3300/0 conn(0x7fb9881381e0 0x7fb9880a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient(hunting): handle_auth_done global_id 45500 payload 274 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 -- 172.21.15.17:0/1160892445 >> v2:172.21.15.119:3300/0 conn(0x7fb9881381e0 msgr2=0x7fb9880a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 --2- 172.21.15.17:0/1160892445 >> v2:172.21.15.119:3300/0 conn(0x7fb9881381e0 0x7fb9880a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 -- 172.21.15.17:0/1160892445 >> v1:172.21.15.17:6789/0 conn(0x7fb9880a5040 legacy=0x7fb9880a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 monclient: found mon.noname-b 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 -- 172.21.15.17:0/1160892445 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880a5c00 con 0x7fb9881378c0 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 1 -- 172.21.15.17:0/1160892445 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014949809 0 0) 0x7fb9880a1d00 con 0x7fb9880a5040 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98da65640 10 monclient: get_auth_request con 0x7fb9881381e0 auth_method 0 2024-09-15T21:26:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98da65640 1 --2- 172.21.15.17:0/1160892445 >> v2:172.21.15.119:3300/0 conn(0x7fb9881381e0 0x7fb9880a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:26:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98e266640 1 --2- 172.21.15.17:0/1160892445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9881378c0 0x7fb988137ca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb9840097c0 tx=0x7fb98401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c209e87028d1d7e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 1 -- 172.21.15.17:0/1160892445 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840b0070 con 0x7fb9881378c0 2024-09-15T21:26:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: dump: 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 5 monclient: authenticate success, global_id 45500 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 1 -- 172.21.15.17:0/1160892445 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9840ab600 con 0x7fb9881378c0 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb9904ca640 4 set_mon_vals no callback set 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 1 -- 172.21.15.17:0/1160892445 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840ab9d0 con 0x7fb9881378c0 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.326+0000 7fb98d264640 10 monclient: dump: 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: shutdown 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/1160892445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9881378c0 msgr2=0x7fb988137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- 172.21.15.17:0/1160892445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9881378c0 0x7fb988137ca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb9840097c0 tx=0x7fb98401c3f0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/1160892445 shutdown_connections 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- 172.21.15.17:0/1160892445 >> v2:172.21.15.119:3300/0 conn(0x7fb9881381e0 0x7fb9880a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- 172.21.15.17:0/1160892445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9881378c0 0x7fb988137ca0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/1160892445 >> 172.21.15.17:0/1160892445 conn(0x7fb9880aac40 msgr2=0x7fb9880a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/1160892445 shutdown_connections 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/1160892445 wait complete. 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: monmap: 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 Processor -- start 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- start start 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: init 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9880a5040 0x7fb9881367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): picked mon.noname-c con 0x7fb9880a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9881378c0 0x7fb98813bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): picked mon.noname-b con 0x7fb9881378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9881381e0 0x7fb98813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): picked mon.noname-a con 0x7fb9881381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881425c0 con 0x7fb9881381e0 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881386d0 con 0x7fb9881378c0 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb988142740 con 0x7fb9880a5040 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): authenticate will time out at 1524.729759s 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9880a5040 0x7fb9881367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9880a5040 0x7fb9881367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56328/0 (socket says 172.21.15.17:56328) 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 -- 172.21.15.17:0/295401388 learned_addr learned my addr 172.21.15.17:0/295401388 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): get_auth_request con 0x7fb9880a5040 auth_method 0 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9881381e0 0x7fb98813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): get_auth_request con 0x7fb9881381e0 auth_method 0 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98ea67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98da65640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9881378c0 0x7fb98813bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient(hunting): handle_auth_done global_id 45503 payload 824 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9881378c0 msgr2=0x7fb98813bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9881378c0 0x7fb98813bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9881381e0 msgr2=0x7fb98813c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9881381e0 0x7fb98813c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 monclient: found mon.noname-c 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98809fdf0 con 0x7fb9880a5040 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98da65640 10 monclient: get_auth_request con 0x7fb9881378c0 auth_method 0 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98da65640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9881378c0 0x7fb98813bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb98e266640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9880a5040 0x7fb9881367b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9840097c0 tx=0x7fb984019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840b8030 con 0x7fb9880a5040 2024-09-15T21:26:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: dump: 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9840b4410 con 0x7fb9880a5040 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840b0040 con 0x7fb9880a5040 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97effd640 10 monclient: dump: 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb97ffff640 4 set_mon_vals no callback set 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 5 monclient: authenticate success, global_id 45503 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: _renew_subs 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb988120fd0 con 0x7fb9880a5040 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: _renew_subs 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9880a12f0 con 0x7fb9880a5040 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 0x7fb98802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.330+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb988137ca0 con 0x7fb988120bc0 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb9840bb070 con 0x7fb9880a5040 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 0x7fb98802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient: get_auth_request con 0x7fb988120bc0 auth_method 0 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb96c03b9f0 0x7fb96c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb9840ea2a0 con 0x7fb9880a5040 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98ea67640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb96c03b9f0 0x7fb96c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98ea67640 10 monclient: get_auth_request con 0x7fb96c03b9f0 auth_method 0 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 10 monclient(hunting): handle_auth_done global_id 114124 payload 824 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb988120750 con 0x7fb988120bc0 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98da65640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 0x7fb98802d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d790 tx=0x7fb97800dc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978016070 con 0x7fb988120bc0 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 10 monclient: dump: 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb98ea67640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb96c03b9f0 0x7fb96c03deb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb980004fd0 tx=0x7fb980001850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb978008910 con 0x7fb988120bc0 2024-09-15T21:26:41.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:41.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 msgr2=0x7fb98802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 0x7fb98802d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d790 tx=0x7fb97800dc50 comp rx=0 tx=0).stop 2024-09-15T21:26:41.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 msgr2=0x7fb98802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:41.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.334+0000 7fb97effd640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988120bc0 0x7fb98802d450 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d790 tx=0x7fb97800dc50 comp rx=0 tx=0).stop 2024-09-15T21:26:41.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:41.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 0x7fb98802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb98809f180 con 0x7fb988136cf0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 0x7fb98802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient: get_auth_request con 0x7fb988136cf0 auth_method 0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 10 monclient(hunting): handle_auth_done global_id 114127 payload 824 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 1 -- 172.21.15.17:0/295401388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb988069270 con 0x7fb988136cf0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98da65640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 0x7fb98802d990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d790 tx=0x7fb978004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97801b650 con 0x7fb988136cf0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 10 monclient: dump: 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7fb9780113c0 con 0x7fb988136cf0 2024-09-15T21:26:41.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 msgr2=0x7fb98802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 0x7fb98802d990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d790 tx=0x7fb978004740 comp rx=0 tx=0).stop 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 msgr2=0x7fb98802d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb97effd640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb988136cf0 0x7fb98802d990 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d790 tx=0x7fb978004740 comp rx=0 tx=0).stop 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:26:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 88, 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9907, 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:41.373 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:41.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb96c03b9f0 msgr2=0x7fb96c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb96c03b9f0 0x7fb96c03deb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb980004fd0 tx=0x7fb980001850 comp rx=0 tx=0).stop 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 10 monclient: shutdown 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9880a5040 msgr2=0x7fb9881367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 --2- 172.21.15.17:0/295401388 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb9880a5040 0x7fb9881367b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9840097c0 tx=0x7fb984019190 comp rx=0 tx=0).stop 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98ea67640 1 -- 172.21.15.17:0/295401388 reap_dead start 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb98ea67640 1 -- 172.21.15.17:0/295401388 reap_dead start 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 shutdown_connections 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 >> 172.21.15.17:0/295401388 conn(0x7fb9880aac40 msgr2=0x7fb98813f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 shutdown_connections 2024-09-15T21:26:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.342+0000 7fb9904ca640 1 -- 172.21.15.17:0/295401388 wait complete. 2024-09-15T21:26:41.375 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:26:41.376 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:26:41.376 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:26:41.376 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:26:41.376 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:26:41.376 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:26:41.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:26:41.388 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:26:41.388 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:26:41.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.538+0000 7f7b3079c640 10 monclient: get_monmap_and_config 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.538+0000 7f7b3079c640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.538+0000 7f7b3079c640 10 monclient: monmap: 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:41.545673+0000 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:41.545673+0000 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:41.547 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 1 Processor -- start 2024-09-15T21:26:41.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 1 -- start start 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient: init 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b281378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b281383b0 0x7f7b2813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient(hunting): picked mon.noname-a con 0x7f7b281383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7b2813d2c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f7b281378c0 legacy=0x7f7b28137ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b280a1a20 con 0x7f7b281378c0 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2dd37640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f7b281383b0 msgr2=0x7f7b2813cd80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b280a1d00 con 0x7f7b2813d2c0 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7b280a1fe0 con 0x7f7b281383b0 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2dd37640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b281383b0 0x7f7b2813cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 10 monclient(hunting): authenticate will time out at 1524.941285s 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7b2813d2c0 0x7f7b2813f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:41596/0 (socket says 172.21.15.17:41596) 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 1 -- 172.21.15.17:0/1487531017 learned_addr learned my addr 172.21.15.17:0/1487531017 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539239001 0 0) 0x7f7b280a1d00 con 0x7f7b2813d2c0 2024-09-15T21:26:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): _negotiate 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): my global_id is 114130 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b14003530 con 0x7f7b2813d2c0 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3711953278 0 0) 0x7f7b14003530 con 0x7f7b2813d2c0 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 >> v2:172.21.15.17:3300/0 conn(0x7f7b281383b0 msgr2=0x7f7b2813cd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 --2- 172.21.15.17:0/1487531017 >> v2:172.21.15.17:3300/0 conn(0x7f7b281383b0 0x7f7b2813cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 >> v1:172.21.15.17:6789/0 conn(0x7f7b281378c0 legacy=0x7f7b28137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b280a22c0 con 0x7f7b2813d2c0 2024-09-15T21:26:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7b20001a80 con 0x7f7b2813d2c0 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: dump: 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 >> v1:172.21.15.119:6789/0 conn(0x7f7b2813d2c0 legacy=0x7f7b2813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): picked mon.c con 0x7f7b140039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b14004810 0x7f7b14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): picked mon.a con 0x7f7b14004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b140053d0 0x7f7b1400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): picked mon.b con 0x7f7b140053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280a1fe0 con 0x7f7b14004810 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1400c310 con 0x7f7b140053d0 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1400c510 con 0x7f7b140039c0 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2dd37640 1 -- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b14004810 msgr2=0x7f7b14004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2d536640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2dd37640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b14004810 0x7f7b14004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): get_auth_request con 0x7f7b140039c0 auth_method 0 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b140053d0 0x7f7b1400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): get_auth_request con 0x7f7b140053d0 auth_method 0 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient(hunting): handle_auth_done global_id 114130 payload 293 2024-09-15T21:26:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 -- 172.21.15.17:0/1487531017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b140053d0 msgr2=0x7f7b1400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b140053d0 0x7f7b1400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 -- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b14004810 msgr2=0x7f7b14004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b14004810 0x7f7b14004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 monclient: found mon.c 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 1 -- 172.21.15.17:0/1487531017 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b14003530 con 0x7f7b140039c0 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2e538640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b2ed39640 1 -- 172.21.15.17:0/1487531017 reap_dead start 2024-09-15T21:26:41.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 5 monclient: authenticate success, global_id 114130 2024-09-15T21:26:41.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.542+0000 7f7b3079c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2e538640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b14003dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7b24003990 tx=0x7f7b2401c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7a378545bfc924 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24009520 con 0x7f7b140039c0 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: dump: 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.570 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b24003bc0 con 0x7f7b140039c0 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 1 -- 172.21.15.17:0/1487531017 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24005070 con 0x7f7b140039c0 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b2d536640 10 monclient: dump: 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.571 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 4 set_mon_vals no callback set 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 10 monclient: shutdown 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 -- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 msgr2=0x7f7b14003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b14003dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7b24003990 tx=0x7f7b2401c100 comp rx=0 tx=0).stop 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 -- 172.21.15.17:0/1487531017 shutdown_connections 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 --2- 172.21.15.17:0/1487531017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b14003dc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 -- 172.21.15.17:0/1487531017 >> 172.21.15.17:0/1487531017 conn(0x7f7b280aac40 msgr2=0x7f7b280a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 -- 172.21.15.17:0/1487531017 shutdown_connections 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.546+0000 7f7b3079c640 1 -- 172.21.15.17:0/1487531017 wait complete. 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: build_initial_monmap 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: monmap: 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:41.573 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 Processor -- start 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- start start 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: init 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b281d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): picked mon.noname-c con 0x7f7b140039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b281d1c40 0x7f7b281230b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): picked mon.noname-b con 0x7f7b281d1c40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b281d20c0 0x7f7b281235f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): picked mon.noname-a con 0x7f7b281d20c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2813b580 con 0x7f7b281d20c0 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280a5940 con 0x7f7b281d1c40 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2813b700 con 0x7f7b140039c0 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient(hunting): authenticate will time out at 1524.949449s 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2ed39640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b281d20c0 msgr2=0x7f7b281235f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2ed39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b281d20c0 0x7f7b281235f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b281d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b281d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56338/0 (socket says 172.21.15.17:56338) 2024-09-15T21:26:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 -- 172.21.15.17:0/3955146306 learned_addr learned my addr 172.21.15.17:0/3955146306 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): get_auth_request con 0x7f7b140039c0 auth_method 0 2024-09-15T21:26:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2dd37640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b281d1c40 0x7f7b281230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2dd37640 10 monclient(hunting): get_auth_request con 0x7f7b281d1c40 auth_method 0 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2dd37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2dd37640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2dd37640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient(hunting): handle_auth_done global_id 45506 payload 824 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 -- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b281d1c40 msgr2=0x7f7b281230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b281d1c40 0x7f7b281230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 -- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b281d20c0 msgr2=0x7f7b281235f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b281d20c0 0x7f7b281235f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient: _finish_hunting 0 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 monclient: found mon.noname-c 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 -- 172.21.15.17:0/3955146306 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b28126bf0 con 0x7f7b140039c0 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2e538640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b281d1700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7b24000c10 tx=0x7f7b14005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240bc070 con 0x7f7b140039c0 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: dump: 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.579 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: _finish_auth 0 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b24009500 con 0x7f7b140039c0 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 5 monclient: authenticate success, global_id 45506 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _renew_subs 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240056d0 con 0x7f7b140039c0 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b28124b70 con 0x7f7b140039c0 2024-09-15T21:26:41.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:41.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b2cd35640 4 set_mon_vals no callback set 2024-09-15T21:26:41.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b1effd640 10 monclient: dump: 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _renew_subs 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b28136890 con 0x7f7b140039c0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.550+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2813ba90 con 0x7f7b140039c0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.554+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7b240bf070 con 0x7f7b140039c0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.554+0000 7f7b1effd640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b0c03b9a0 0x7f7b0c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.554+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7b240e9ba0 con 0x7f7b140039c0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.554+0000 7f7b2dd37640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b0c03b9a0 0x7f7b0c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.554+0000 7f7b2dd37640 10 monclient: get_auth_request con 0x7f7b0c03b9a0 auth_method 0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.554+0000 7f7b2dd37640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b0c03b9a0 0x7f7b0c03de60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7b18006fd0 tx=0x7f7b18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.558+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7b240c6070 con 0x7f7b140039c0 2024-09-15T21:26:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.558+0000 7f7b1effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.558+0000 7f7b1effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:41.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.790+0000 7f7b3079c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.790+0000 7f7b3079c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.790+0000 7f7b3079c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.790+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7b281256d0 con 0x7f7b140039c0 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b1effd640 1 -- 172.21.15.17:0/3955146306 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f7b240a9020 con 0x7f7b140039c0 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b1effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b1effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:41.798 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b0c03b9a0 msgr2=0x7f7b0c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b0c03b9a0 0x7f7b0c03de60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7b18006fd0 tx=0x7f7b18008040 comp rx=0 tx=0).stop 2024-09-15T21:26:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 10 monclient: shutdown 2024-09-15T21:26:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 msgr2=0x7f7b281d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b281d1700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7b24000c10 tx=0x7f7b14005840 comp rx=0 tx=0).stop 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 shutdown_connections 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b0c03b9a0 0x7f7b0c03de60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b281d20c0 0x7f7b281235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b281d1c40 0x7f7b281230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 --2- 172.21.15.17:0/3955146306 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b140039c0 0x7f7b281d1700 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 >> 172.21.15.17:0/3955146306 conn(0x7f7b280aac40 msgr2=0x7f7b280a02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 shutdown_connections 2024-09-15T21:26:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:41.794+0000 7f7b3079c640 1 -- 172.21.15.17:0/3955146306 wait complete. 2024-09-15T21:26:44.818 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient: get_monmap_and_config 2024-09-15T21:26:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient: build_initial_monmap 2024-09-15T21:26:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient: monmap: 2024-09-15T21:26:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:44.951539+0000 2024-09-15T21:26:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:44.951539+0000 2024-09-15T21:26:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:44.955 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:44.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 Processor -- start 2024-09-15T21:26:44.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 -- start start 2024-09-15T21:26:44.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient: init 2024-09-15T21:26:44.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:44.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:26:44.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f294c0a0890 0x7f294c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:44.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): picked mon.noname-a con 0x7f294c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:44.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:44.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): picked mon.noname-b con 0x7f294c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:44.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f294c135670 0x7f294c135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:44.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): picked mon.noname-c con 0x7f294c135670 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:44.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:44.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f294c0a16f0 con 0x7f294c0a0890 2024-09-15T21:26:44.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:44.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f294c0a3130 con 0x7f294c135670 2024-09-15T21:26:44.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:44.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f294c073df0 con 0x7f294c09ffc0 2024-09-15T21:26:44.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:44.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f295172d640 10 monclient(hunting): authenticate will time out at 1528.347660s 2024-09-15T21:26:44.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294affd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f294c0a0890 msgr2=0x7f294c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:44.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f294c0a0890 0x7f294c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:44.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:44.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c0a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56350/0 (socket says 172.21.15.17:56350) 2024-09-15T21:26:44.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 1 -- 172.21.15.17:0/1481107735 learned_addr learned my addr 172.21.15.17:0/1481107735 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:44.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 10 monclient(hunting): get_auth_request con 0x7f294c09ffc0 auth_method 0 2024-09-15T21:26:44.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:44.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:44.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.946+0000 7f294a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294b7fe640 1 --2- 172.21.15.17:0/1481107735 >> v2:172.21.15.119:3300/0 conn(0x7f294c135670 0x7f294c135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294b7fe640 10 monclient(hunting): get_auth_request con 0x7f294c135670 auth_method 0 2024-09-15T21:26:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:44.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:44.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:44.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 10 monclient(hunting): handle_auth_done global_id 45509 payload 274 2024-09-15T21:26:44.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 -- 172.21.15.17:0/1481107735 >> v2:172.21.15.119:3300/0 conn(0x7f294c135670 msgr2=0x7f294c135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:44.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1481107735 >> v2:172.21.15.119:3300/0 conn(0x7f294c135670 0x7f294c135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 -- 172.21.15.17:0/1481107735 >> v2:172.21.15.17:3300/0 conn(0x7f294c0a0890 msgr2=0x7f294c0a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:44.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1481107735 >> v2:172.21.15.17:3300/0 conn(0x7f294c0a0890 0x7f294c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:26:44.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 monclient: found mon.noname-b 2024-09-15T21:26:44.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 -- 172.21.15.17:0/1481107735 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c0a1bf0 con 0x7f294c09ffc0 2024-09-15T21:26:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1481107735 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c0a11b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f294c06f1c0 tx=0x7f294001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a012446f9e51d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 1 -- 172.21.15.17:0/1481107735 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400b0070 con 0x7f294c09ffc0 2024-09-15T21:26:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: dump: 2024-09-15T21:26:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:44.987 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: _finish_auth 0 2024-09-15T21:26:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:26:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 5 monclient: authenticate success, global_id 45509 2024-09-15T21:26:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 1 -- 172.21.15.17:0/1481107735 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29400ab600 con 0x7f294c09ffc0 2024-09-15T21:26:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 4 set_mon_vals no callback set 2024-09-15T21:26:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 1 -- 172.21.15.17:0/1481107735 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400ab9d0 con 0x7f294c09ffc0 2024-09-15T21:26:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f2949ffb640 10 monclient: dump: 2024-09-15T21:26:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:44.990 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 10 monclient: shutdown 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 -- 172.21.15.17:0/1481107735 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 msgr2=0x7f294c0a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 --2- 172.21.15.17:0/1481107735 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c0a11b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f294c06f1c0 tx=0x7f294001c400 comp rx=0 tx=0).stop 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 -- 172.21.15.17:0/1481107735 shutdown_connections 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 --2- 172.21.15.17:0/1481107735 >> v2:172.21.15.119:3300/0 conn(0x7f294c135670 0x7f294c135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 --2- 172.21.15.17:0/1481107735 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c0a11b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 --2- 172.21.15.17:0/1481107735 >> v2:172.21.15.17:3300/0 conn(0x7f294c0a0890 0x7f294c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 -- 172.21.15.17:0/1481107735 >> 172.21.15.17:0/1481107735 conn(0x7f294c0a66a0 msgr2=0x7f294c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 -- 172.21.15.17:0/1481107735 shutdown_connections 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 1 -- 172.21.15.17:0/1481107735 wait complete. 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 10 monclient: build_initial_monmap 2024-09-15T21:26:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.950+0000 7f295172d640 10 monclient: monmap: 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:44.992 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 Processor -- start 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- start start 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: init 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c1cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): picked mon.noname-c con 0x7f294c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f294c0a0890 0x7f294c1349d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): picked mon.noname-b con 0x7f294c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f294c135670 0x7f294c11c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): picked mon.noname-a con 0x7f294c135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f294c13f680 con 0x7f294c135670 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f294c13f500 con 0x7f294c0a0890 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f294c13f800 con 0x7f294c09ffc0 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient(hunting): authenticate will time out at 1528.352775s 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294b7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f294c135670 msgr2=0x7f294c11c550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f294c135670 0x7f294c11c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c1cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c1cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56354/0 (socket says 172.21.15.17:56354) 2024-09-15T21:26:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 -- 172.21.15.17:0/1191049354 learned_addr learned my addr 172.21.15.17:0/1191049354 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): get_auth_request con 0x7f294c09ffc0 auth_method 0 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f294c0a0890 0x7f294c1349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 10 monclient(hunting): get_auth_request con 0x7f294c0a0890 auth_method 0 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient(hunting): handle_auth_done global_id 45512 payload 824 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 -- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f294c0a0890 msgr2=0x7f294c1349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f294c0a0890 0x7f294c1349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 -- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f294c135670 msgr2=0x7f294c11c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f294c135670 0x7f294c11c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient: _finish_hunting 0 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 monclient: found mon.noname-c 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 -- 172.21.15.17:0/1191049354 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c0a12f0 con 0x7f294c09ffc0 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f294c0a0890 0x7f294c1349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294affd640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c1cdaa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f293c00b9e0 tx=0x7f293c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c00cb40 con 0x7f294c09ffc0 2024-09-15T21:26:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: dump: 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: _finish_auth 0 2024-09-15T21:26:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: _check_auth_tickets 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293c010070 con 0x7f294c09ffc0 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c015470 con 0x7f294c09ffc0 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 10 monclient: dump: 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29497fa640 4 set_mon_vals no callback set 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 5 monclient: authenticate success, global_id 45512 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _renew_subs 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c120090 con 0x7f294c09ffc0 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _renew_subs 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c09d0f0 con 0x7f294c09ffc0 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c135aa0 con 0x7f294c09ffc0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f293c01f3f0 con 0x7f294c09ffc0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f293803b9a0 0x7f293803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f293c0507a0 con 0x7f294c09ffc0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.954+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f293803b9a0 0x7f293803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.958+0000 7f294a7fc640 10 monclient: get_auth_request con 0x7f293803b9a0 auth_method 0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.958+0000 7f294a7fc640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f293803b9a0 0x7f293803de60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f29400ad000 tx=0x7f2940008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.958+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f293c052d10 con 0x7f294c09ffc0 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.958+0000 7f29337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:44.958+0000 7f29337fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f295172d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:45.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f295172d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:45.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f295172d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:45.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f294c11e010 con 0x7f294c09ffc0 2024-09-15T21:26:45.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f29337fe640 1 -- 172.21.15.17:0/1191049354 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f293c01e3a0 con 0x7f294c09ffc0 2024-09-15T21:26:45.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f29337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:45.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.202+0000 7f29337fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:45.208 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:45.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f293803b9a0 msgr2=0x7f293803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f293803b9a0 0x7f293803de60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f29400ad000 tx=0x7f2940008040 comp rx=0 tx=0).stop 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 10 monclient: shutdown 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 msgr2=0x7f294c1cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c1cdaa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f293c00b9e0 tx=0x7f293c00bea0 comp rx=0 tx=0).stop 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 shutdown_connections 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f293803b9a0 0x7f293803de60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f294c135670 0x7f294c11c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f294c0a0890 0x7f294c1349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 --2- 172.21.15.17:0/1191049354 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f294c09ffc0 0x7f294c1cdaa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 >> 172.21.15.17:0/1191049354 conn(0x7f294c0a66a0 msgr2=0x7f294c0a4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 shutdown_connections 2024-09-15T21:26:45.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:45.206+0000 7f295172d640 1 -- 172.21.15.17:0/1191049354 wait complete. 2024-09-15T21:26:48.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 10 monclient: get_monmap_and_config 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 10 monclient: build_initial_monmap 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 10 monclient: monmap: 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:48.360102+0000 2024-09-15T21:26:48.360 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:48.360102+0000 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:48.361 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 1 Processor -- start 2024-09-15T21:26:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 1 -- start start 2024-09-15T21:26:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 10 monclient: init 2024-09-15T21:26:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.354+0000 7f148e877640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-09-15T21:26:48.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14881378c0 0x7f1488137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): picked mon.noname-c con 0x7f14881378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f14881381e0 0x7f14880a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): picked mon.noname-a con 0x7f14881381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f14880a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14880a1d00 con 0x7f14880a5040 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f14880a1fe0 con 0x7f14881381e0 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f14880a22c0 con 0x7f14881378c0 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient(hunting): authenticate will time out at 1531.756171s 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f147f7fe640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f14881381e0 msgr2=0x7f14880a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f147f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f14881381e0 0x7f14880a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:48.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14881378c0 0x7f1488137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14881378c0 0x7f1488137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:48652/0 (socket says 172.21.15.17:48652) 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 -- 172.21.15.17:0/3763437450 learned_addr learned my addr 172.21.15.17:0/3763437450 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): get_auth_request con 0x7f14881378c0 auth_method 0 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient(hunting): handle_auth_done global_id 114142 payload 274 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 -- 172.21.15.17:0/3763437450 >> v2:172.21.15.17:3300/0 conn(0x7f14881381e0 msgr2=0x7f14880a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 --2- 172.21.15.17:0/3763437450 >> v2:172.21.15.17:3300/0 conn(0x7f14881381e0 0x7f14880a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 -- 172.21.15.17:0/3763437450 >> v1:172.21.15.119:6789/0 conn(0x7f14880a5040 legacy=0x7f14880a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:48.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 monclient: found mon.noname-c 2024-09-15T21:26:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:26:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 -- 172.21.15.17:0/3763437450 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14880a4490 con 0x7f14881378c0 2024-09-15T21:26:48.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f1487fff640 1 --2- 172.21.15.17:0/3763437450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881378c0 0x7f1488137ca0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1478009870 tx=0x7f147801c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11f89185648f40d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:48.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 1 -- 172.21.15.17:0/3763437450 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780ab620 con 0x7f14881378c0 2024-09-15T21:26:48.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:48.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:48.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: dump: 2024-09-15T21:26:48.373 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:48.373 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: _finish_auth 0 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: _check_auth_tickets 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 1 -- 172.21.15.17:0/3763437450 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14780b0070 con 0x7f14881378c0 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:48.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 1 -- 172.21.15.17:0/3763437450 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780ab9e0 con 0x7f14881378c0 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 5 monclient: authenticate success, global_id 114142 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f148e877640 4 set_mon_vals no callback set 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.358+0000 7f14877fe640 10 monclient: dump: 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:48.375 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: shutdown 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/3763437450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881378c0 msgr2=0x7f1488137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 --2- 172.21.15.17:0/3763437450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881378c0 0x7f1488137ca0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1478009870 tx=0x7f147801c400 comp rx=0 tx=0).stop 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/3763437450 shutdown_connections 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 --2- 172.21.15.17:0/3763437450 >> v2:172.21.15.17:3300/0 conn(0x7f14881381e0 0x7f14880a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 --2- 172.21.15.17:0/3763437450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881378c0 0x7f1488137ca0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/3763437450 >> 172.21.15.17:0/3763437450 conn(0x7f14880aac40 msgr2=0x7f14880a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/3763437450 shutdown_connections 2024-09-15T21:26:48.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/3763437450 wait complete. 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: build_initial_monmap 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: monmap: 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 Processor -- start 2024-09-15T21:26:48.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- start start 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: init 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 0x7f14881cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): picked mon.noname-c con 0x7f14880a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14881378c0 0x7f14881d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): picked mon.noname-a con 0x7f14881378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881381e0 0x7f148813cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): picked mon.noname-b con 0x7f14881381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f148806f340 con 0x7f14881378c0 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f148806f1c0 con 0x7f14881381e0 2024-09-15T21:26:48.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f14881425c0 con 0x7f14880a5040 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient(hunting): authenticate will time out at 1531.762106s 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f147f7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14881378c0 msgr2=0x7f14881d0100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14881378c0 0x7f14881d0100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 0x7f14881cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 0x7f14881cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52932/0 (socket says 172.21.15.17:52932) 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 -- 172.21.15.17:0/2030838707 learned_addr learned my addr 172.21.15.17:0/2030838707 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): get_auth_request con 0x7f14880a5040 auth_method 0 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148ce14640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881381e0 0x7f148813cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient(hunting): handle_auth_done global_id 45515 payload 824 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 -- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881381e0 msgr2=0x7f148813cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881381e0 0x7f148813cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 -- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14881378c0 msgr2=0x7f14881d0100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14881378c0 0x7f14881d0100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient: _finish_hunting 0 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 monclient: found mon.noname-c 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 -- 172.21.15.17:0/2030838707 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14880a8a30 con 0x7f14880a5040 2024-09-15T21:26:48.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148ce14640 10 monclient: get_auth_request con 0x7f14881381e0 auth_method 0 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148ce14640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881381e0 0x7f148813cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f1487fff640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 0x7f14881cf930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14780ad040 tx=0x7f147801ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780abdb0 con 0x7f14880a5040 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: dump: 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:48.381 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: _finish_auth 0 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: _check_auth_tickets 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14780be3f0 con 0x7f14880a5040 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 5 monclient: authenticate success, global_id 45515 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: _renew_subs 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148813f350 con 0x7f14880a5040 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780be7e0 con 0x7f14880a5040 2024-09-15T21:26:48.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14857fa640 10 monclient: dump: 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f14867fc640 4 set_mon_vals no callback set 2024-09-15T21:26:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: _renew_subs 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.362+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f148813e7f0 con 0x7f14880a5040 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f148e877640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f148e877640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f148e877640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1488137ca0 con 0x7f14880a5040 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f14780c7440 con 0x7f14880a5040 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f14857fa640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f146803b970 0x7f146803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f14780ead60 con 0x7f14880a5040 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f147f7fe640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f146803b970 0x7f146803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f147f7fe640 10 monclient: get_auth_request con 0x7f146803b970 auth_method 0 2024-09-15T21:26:48.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.366+0000 7f147f7fe640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f146803b970 0x7f146803de30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1470003990 tx=0x7f1470006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:48.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.370+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1478016110 con 0x7f14880a5040 2024-09-15T21:26:48.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.370+0000 7f14857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:48.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.370+0000 7f14857fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:48.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f148e877640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f148e877640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f148e877640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1488121560 con 0x7f14880a5040 2024-09-15T21:26:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f14857fa640 1 -- 172.21.15.17:0/2030838707 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f14780b43f0 con 0x7f14880a5040 2024-09-15T21:26:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f14857fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.606+0000 7f14857fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:48.614 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f146803b970 msgr2=0x7f146803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f146803b970 0x7f146803de30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1470003990 tx=0x7f1470006d90 comp rx=0 tx=0).stop 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 10 monclient: shutdown 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 msgr2=0x7f14881cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 0x7f14881cf930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14780ad040 tx=0x7f147801ac40 comp rx=0 tx=0).stop 2024-09-15T21:26:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 shutdown_connections 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f146803b970 0x7f146803de30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881381e0 0x7f148813cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14881378c0 0x7f14881d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 --2- 172.21.15.17:0/2030838707 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f14880a5040 0x7f14881cf930 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 >> 172.21.15.17:0/2030838707 conn(0x7f14880aac40 msgr2=0x7f148809da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 shutdown_connections 2024-09-15T21:26:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:48.610+0000 7f148e877640 1 -- 172.21.15.17:0/2030838707 wait complete. 2024-09-15T21:26:51.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:26:51.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient: get_monmap_and_config 2024-09-15T21:26:51.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient: build_initial_monmap 2024-09-15T21:26:51.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient: monmap: 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:51.767464+0000 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:51.767464+0000 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:51.768 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 Processor -- start 2024-09-15T21:26:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 -- start start 2024-09-15T21:26:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient: init 2024-09-15T21:26:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 0x7f332c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): picked mon.noname-a con 0x7f332c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): picked mon.noname-b con 0x7f332c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f332c135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332c0a3130 con 0x7f332c135670 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f332c0a16f0 con 0x7f332c0a0890 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f332c073df0 con 0x7f332c09ffc0 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f33317f8640 10 monclient(hunting): authenticate will time out at 1535.162985s 2024-09-15T21:26:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332affd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 msgr2=0x7f332c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 0x7f332c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332b7fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f332c135670 legacy=0x7f332c135aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:52948/0 (socket says 172.21.15.17:52948) 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 learned_addr learned my addr 172.21.15.17:0/2329912384 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 10 monclient(hunting): get_auth_request con 0x7f332c09ffc0 auth_method 0 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:51.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.762+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 msgr2=0x7f332c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332b7fe640 1 -- 172.21.15.17:0/2329912384 >> v1:172.21.15.17:6789/0 conn(0x7f332c135670 legacy=0x7f332c135aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:26:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 0x7f332c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:26:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 10 monclient(hunting): get_auth_request con 0x7f332c09ffc0 auth_method 0 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:51.966+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:26:52.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:52.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 10 monclient(hunting): get_auth_request con 0x7f332c09ffc0 auth_method 0 2024-09-15T21:26:52.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:52.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:52.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:52.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332b7fe640 1 -- 172.21.15.17:0/2329912384 >> v1:172.21.15.17:6789/0 conn(0x7f332c135670 legacy=0x7f332c135aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:26:52.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 msgr2=0x7f332c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:52.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 0x7f332c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:26:52.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:52.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:52.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:52.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:52.370+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332b7fe640 1 -- 172.21.15.17:0/2329912384 >> v1:172.21.15.17:6789/0 conn(0x7f332c135670 legacy=0x7f332c135aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 msgr2=0x7f332c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 0x7f332c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 10 monclient(hunting): get_auth_request con 0x7f332c09ffc0 auth_method 0 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:26:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:53.174+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:26:54.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): tick 2024-09-15T21:26:54.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:26:54.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 monclient(hunting): continuing hunt 2024-09-15T21:26:54.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 msgr2=0x7f332c0a11b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f332c09ffc0 0x7f332c0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 msgr2=0x7f332c0a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.17:3300/0 conn(0x7f332c0a0890 0x7f332c0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 -- 172.21.15.17:0/2329912384 >> v1:172.21.15.17:6789/0 conn(0x7f332c135670 legacy=0x7f332c135aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient: _add_conns ranks=[4,1] 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f33180036d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3318006220 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 -- 172.21.15.17:0/2329912384 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332c0a1bf0 con 0x7f3318006220 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 1 -- 172.21.15.17:0/2329912384 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f332c0a16f0 con 0x7f33180036d0 2024-09-15T21:26:54.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f3328ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 10 monclient(hunting): get_auth_request con 0x7f33180036d0 auth_method 0 2024-09-15T21:26:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.766+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 10 monclient(hunting): get_auth_request con 0x7f33180036d0 auth_method 0 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:54.970+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:26:55.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:55.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 10 monclient(hunting): get_auth_request con 0x7f33180036d0 auth_method 0 2024-09-15T21:26:55.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:55.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:55.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:55.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:55.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:55.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:55.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:55.374+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:26:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 10 monclient(hunting): get_auth_request con 0x7f33180036d0 auth_method 0 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:26:56.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.178+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:26:56.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.322+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3130228115 0 0) 0x7f332c0a1bf0 con 0x7f3318006220 2024-09-15T21:26:56.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.322+0000 7f3329ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:26:56.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.322+0000 7f3329ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.322+0000 7f3329ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.322+0000 7f3329ffb640 10 monclient(hunting): my global_id is 114145 2024-09-15T21:26:56.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.322+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3314003530 con 0x7f3318006220 2024-09-15T21:26:56.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1717635298 0 0) 0x7f3314003530 con 0x7f3318006220 2024-09-15T21:26:56.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 msgr2=0x7f3318005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:56.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 --2- 172.21.15.17:0/2329912384 >> v2:172.21.15.119:3300/0 conn(0x7f33180036d0 0x7f3318005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c0a3130 con 0x7f3318006220 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f331c00f040 con 0x7f3318006220 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f33317f8640 5 monclient: authenticate success, global_id 114145 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f33317f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: dump: 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.343 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 >> v1:172.21.15.119:6789/0 conn(0x7f3318006220 legacy=0x7f33180086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 1 -- 172.21.15.17:0/2329912384 reap_dead start 2024-09-15T21:26:56.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f3314003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): picked mon.b con 0x7f33140039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f3314004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): picked mon.a con 0x7f3314004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f331400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): picked mon.c con 0x7f33140052a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f332c0a16f0 con 0x7f3314004810 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f331400c1f0 con 0x7f33140039c0 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f331400c3f0 con 0x7f33140052a0 2024-09-15T21:26:56.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332a7fc640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 msgr2=0x7f3314004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332a7fc640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f3314004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f3329ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f33317f8640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f3314003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): get_auth_request con 0x7f33140039c0 auth_method 0 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f331400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): get_auth_request con 0x7f33140052a0 auth_method 0 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient(hunting): handle_auth_done global_id 114145 payload 293 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 msgr2=0x7f331400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f331400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 msgr2=0x7f3314004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f3314004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 monclient: found mon.b 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 1 -- 172.21.15.17:0/2329912384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3314003530 con 0x7f33140039c0 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.330+0000 7f332affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f332affd640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f3314003dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3320005550 tx=0x7f332001d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f615a41e35817b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320005150 con 0x7f33140039c0 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: dump: 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.348 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33200a9030 con 0x7f33140039c0 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 4 set_mon_vals no callback set 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 1 -- 172.21.15.17:0/2329912384 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200af660 con 0x7f33140039c0 2024-09-15T21:26:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f3329ffb640 10 monclient: dump: 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 10 monclient: shutdown 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 1 -- 172.21.15.17:0/2329912384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 msgr2=0x7f3314003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.334+0000 7f33317f8640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f3314003dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3320005550 tx=0x7f332001d590 comp rx=0 tx=0).stop 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- 172.21.15.17:0/2329912384 shutdown_connections 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f331400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f3314004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 --2- 172.21.15.17:0/2329912384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f3314003dc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- 172.21.15.17:0/2329912384 >> 172.21.15.17:0/2329912384 conn(0x7f332c0a66a0 msgr2=0x7f332c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- 172.21.15.17:0/2329912384 shutdown_connections 2024-09-15T21:26:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- 172.21.15.17:0/2329912384 wait complete. 2024-09-15T21:26:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient: build_initial_monmap 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient: monmap: 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 Processor -- start 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- start start 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient: init 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f332c122020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): picked mon.noname-b con 0x7f33140039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f332c11fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): picked mon.noname-a con 0x7f3314004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f332c11ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): picked mon.noname-c con 0x7f33140052a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f332c138ab0 con 0x7f3314004810 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1364a0 con 0x7f33140039c0 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f332c138c30 con 0x7f33140052a0 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33317f8640 10 monclient(hunting): authenticate will time out at 1539.738721s 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332a7fc640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 msgr2=0x7f332c11fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f332c11fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f332c11ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f332c11ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53002/0 (socket says 172.21.15.17:53002) 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 -- 172.21.15.17:0/2763509603 learned_addr learned my addr 172.21.15.17:0/2763509603 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f332c122020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): get_auth_request con 0x7f33140052a0 auth_method 0 2024-09-15T21:26:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): get_auth_request con 0x7f33140039c0 auth_method 0 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient(hunting): handle_auth_done global_id 45518 payload 824 2024-09-15T21:26:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 -- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 msgr2=0x7f332c122020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f332c122020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 -- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 msgr2=0x7f332c11fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f332c11fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 -- 172.21.15.17:0/2763509603 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c06f820 con 0x7f33140052a0 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f332b7fe640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f332c11ff40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3318009f20 tx=0x7f332400fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324016020 con 0x7f33140052a0 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.338+0000 7f33137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: dump: 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332400da00 con 0x7f33140052a0 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 5 monclient: authenticate success, global_id 45518 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 monclient: _renew_subs 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c09d0f0 con 0x7f33140052a0 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332400ddf0 con 0x7f33140052a0 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 10 monclient: dump: 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.362 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 monclient: _renew_subs 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33297fa640 4 set_mon_vals no callback set 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c123540 con 0x7f33140052a0 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3324019020 con 0x7f33140052a0 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332c138db0 con 0x7f33140052a0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f331803c110 0x7f331803e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3324050b50 con 0x7f33140052a0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f332affd640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f331803c110 0x7f331803e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f332affd640 10 monclient: get_auth_request con 0x7f331803c110 auth_method 0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.342+0000 7f332affd640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f331803c110 0x7f331803e5d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3320003990 tx=0x7f3320005be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.346+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3324020030 con 0x7f33140052a0 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.346+0000 7f33137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.346+0000 7f33137fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:56.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33317f8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:26:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33317f8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33317f8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f332c073150 con 0x7f33140052a0 2024-09-15T21:26:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33137fe640 1 -- 172.21.15.17:0/2763509603 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7f33240113a0 con 0x7f33140052a0 2024-09-15T21:26:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:26:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.602+0000 7f33137fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:56.609 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:26:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f331803c110 msgr2=0x7f331803e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f331803c110 0x7f331803e5d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3320003990 tx=0x7f3320005be0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 10 monclient: shutdown 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 msgr2=0x7f332c11ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f332c11ff40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3318009f20 tx=0x7f332400fd40 comp rx=0 tx=0).stop 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 shutdown_connections 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f331803c110 0x7f331803e5d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f33140052a0 0x7f332c11ff40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3314004810 0x7f332c11fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 --2- 172.21.15.17:0/2763509603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140039c0 0x7f332c122020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 >> 172.21.15.17:0/2763509603 conn(0x7f332c0a66a0 msgr2=0x7f332c0a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 shutdown_connections 2024-09-15T21:26:56.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.606+0000 7f33317f8640 1 -- 172.21.15.17:0/2763509603 wait complete. 2024-09-15T21:26:56.628 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:26:56.628 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:26:56.628 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:26:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient: get_monmap_and_config 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient: build_initial_monmap 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient: monmap: 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:56.747560+0000 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:56.747560+0000 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:56.749 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 Processor -- start 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 -- start start 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient: init 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f783409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f783409def0 0x7f7834139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): picked mon.noname-c con 0x7f783409def0 addr v2:172.21.15.119:3300/0 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 0x7f783413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): picked mon.noname-b con 0x7f7834139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78340a3130 con 0x7f783409d400 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f78340a1bc0 con 0x7f783409def0 2024-09-15T21:26:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f78340a3490 con 0x7f7834139ff0 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783c429640 10 monclient(hunting): authenticate will time out at 1540.142979s 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 0x7f783413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 0x7f783413c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53016/0 (socket says 172.21.15.17:53016) 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 -- 172.21.15.17:0/764234199 learned_addr learned my addr 172.21.15.17:0/764234199 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): get_auth_request con 0x7f7834139ff0 auth_method 0 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f78399c4640 1 --2- 172.21.15.17:0/764234199 >> v2:172.21.15.119:3300/0 conn(0x7f783409def0 0x7f7834139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f78399c4640 10 monclient(hunting): get_auth_request con 0x7f783409def0 auth_method 0 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f78399c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f78399c4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f78399c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient(hunting): handle_auth_done global_id 45521 payload 274 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 -- 172.21.15.17:0/764234199 >> v2:172.21.15.119:3300/0 conn(0x7f783409def0 msgr2=0x7f7834139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 --2- 172.21.15.17:0/764234199 >> v2:172.21.15.119:3300/0 conn(0x7f783409def0 0x7f7834139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 -- 172.21.15.17:0/764234199 >> v1:172.21.15.119:6789/0 conn(0x7f783409d400 legacy=0x7f783409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 monclient: found mon.noname-b 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.742+0000 7f783a9c6640 1 -- 172.21.15.17:0/764234199 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783409e7a0 con 0x7f7834139ff0 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783a9c6640 1 --2- 172.21.15.17:0/764234199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 0x7f783413c3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f782c009870 tx=0x7f782c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d897ad28b9777783 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 1 -- 172.21.15.17:0/764234199 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0b0070 con 0x7f7834139ff0 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: dump: 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 5 monclient: authenticate success, global_id 45521 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 1 -- 172.21.15.17:0/764234199 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c0ab600 con 0x7f7834139ff0 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 1 -- 172.21.15.17:0/764234199 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0ab9d0 con 0x7f7834139ff0 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f78391c3640 10 monclient: dump: 2024-09-15T21:26:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 4 set_mon_vals no callback set 2024-09-15T21:26:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 10 monclient: shutdown 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 -- 172.21.15.17:0/764234199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 msgr2=0x7f783413c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 --2- 172.21.15.17:0/764234199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 0x7f783413c3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f782c009870 tx=0x7f782c01c420 comp rx=0 tx=0).stop 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 -- 172.21.15.17:0/764234199 shutdown_connections 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 --2- 172.21.15.17:0/764234199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834139ff0 0x7f783413c3e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 --2- 172.21.15.17:0/764234199 >> v2:172.21.15.119:3300/0 conn(0x7f783409def0 0x7f7834139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 -- 172.21.15.17:0/764234199 >> 172.21.15.17:0/764234199 conn(0x7f78340a66a0 msgr2=0x7f78340a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 -- 172.21.15.17:0/764234199 shutdown_connections 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 -- 172.21.15.17:0/764234199 wait complete. 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 10 monclient: build_initial_monmap 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 10 monclient: monmap: 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:56.768 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 Processor -- start 2024-09-15T21:26:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 1 -- start start 2024-09-15T21:26:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.746+0000 7f783c429640 10 monclient: init 2024-09-15T21:26:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:26:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f783409d400 0x7f7834133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): picked mon.noname-c con 0x7f783409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f783409def0 0x7f78341342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): picked mon.noname-b con 0x7f783409def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7834139ff0 0x7f78341347f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): picked mon.noname-a con 0x7f7834139ff0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f783413d2b0 con 0x7f7834139ff0 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f783413d130 con 0x7f783409def0 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f783413d430 con 0x7f783409d400 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): authenticate will time out at 1540.148430s 2024-09-15T21:26:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a9c6640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7834139ff0 msgr2=0x7f78341347f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a9c6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7834139ff0 0x7f78341347f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f783409d400 0x7f7834133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f783409d400 0x7f7834133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53032/0 (socket says 172.21.15.17:53032) 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 -- 172.21.15.17:0/1265691720 learned_addr learned my addr 172.21.15.17:0/1265691720 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): get_auth_request con 0x7f783409d400 auth_method 0 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f783409def0 0x7f78341342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): get_auth_request con 0x7f783409def0 auth_method 0 2024-09-15T21:26:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient(hunting): handle_auth_done global_id 45524 payload 824 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f783409def0 msgr2=0x7f78341342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f783409def0 0x7f78341342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7834139ff0 msgr2=0x7f78341347f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7834139ff0 0x7f78341347f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 monclient: found mon.noname-c 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78340739e0 con 0x7f783409d400 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a1c5640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f783409d400 0x7f7834133d70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f783000c7c0 tx=0x7f783000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830016020 con 0x7f783409d400 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: dump: 2024-09-15T21:26:56.773 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f783000aa50 con 0x7f783409d400 2024-09-15T21:26:56.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783000ae40 con 0x7f783409d400 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 5 monclient: authenticate success, global_id 45524 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient: _renew_subs 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7834139cd0 con 0x7f783409d400 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: dump: 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78389c2640 4 set_mon_vals no callback set 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient: _renew_subs 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f783413f420 con 0x7f783409d400 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 0x7f7834137aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f783409d7e0 con 0x7f7834073380 2024-09-15T21:26:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 0x7f7834137aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient: get_auth_request con 0x7f7834073380 auth_method 0 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7830019020 con 0x7f783409d400 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 10 monclient(hunting): handle_auth_done global_id 45527 payload 824 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7834073760 con 0x7f7834073380 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f782003b9a0 0x7f782003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f78300505f0 con 0x7f783409d400 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f78399c4640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 0x7f7834137aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f783413fa90 tx=0x7f7824010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824018020 con 0x7f7834073380 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: dump: 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a9c6640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f782003b9a0 0x7f782003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f783a9c6640 10 monclient: get_auth_request con 0x7f782003b9a0 auth_method 0 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7824004240 con 0x7f7834073380 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 msgr2=0x7f7834137aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 0x7f7834137aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f783413fa90 tx=0x7f7824010c80 comp rx=0 tx=0).stop 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 msgr2=0x7f7834137aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.750+0000 7f782affd640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834073380 0x7f7834137aa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.754+0000 7f783a9c6640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f782003b9a0 0x7f782003de60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f782c0ad000 tx=0x7f782c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 0x7f78341383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f783406d020 con 0x7f7834137fe0 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 0x7f78341383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient: get_auth_request con 0x7f7834137fe0 auth_method 0 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 10 monclient(hunting): handle_auth_done global_id 45530 payload 824 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 1 -- 172.21.15.17:0/1265691720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7834138930 con 0x7f7834137fe0 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f78399c4640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 0x7f78341383f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f783413fa90 tx=0x7f78240046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824018070 con 0x7f7834137fe0 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 10 monclient: dump: 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.782 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4868 (secure 0 0 0) 0x7f782400eb50 con 0x7f7834137fe0 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 msgr2=0x7f78341383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 0x7f78341383f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f783413fa90 tx=0x7f78240046d0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 msgr2=0x7f78341383f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f782affd640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7834137fe0 0x7f78341383f0 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f783413fa90 tx=0x7f78240046d0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.784 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:56.784 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:56.784 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 90, 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 385356, 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:56.785 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:56.786 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:56.787 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:56.787 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:56.787 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:56.788 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:56.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:56.790 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:56.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.792 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:56.793 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.794 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:56.796 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:56.797 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.798 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:56.799 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.800 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.800 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:56.800 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:56.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f782003b9a0 msgr2=0x7f782003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f782003b9a0 0x7f782003de60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f782c0ad000 tx=0x7f782c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 10 monclient: shutdown 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f783409d400 msgr2=0x7f7834133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 --2- 172.21.15.17:0/1265691720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f783409d400 0x7f7834133d70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f783000c7c0 tx=0x7f783000cc80 comp rx=0 tx=0).stop 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783a9c6640 1 -- 172.21.15.17:0/1265691720 reap_dead start 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783a9c6640 1 -- 172.21.15.17:0/1265691720 reap_dead start 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 shutdown_connections 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 >> 172.21.15.17:0/1265691720 conn(0x7f78340a66a0 msgr2=0x7f783413c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 shutdown_connections 2024-09-15T21:26:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.758+0000 7f783c429640 1 -- 172.21.15.17:0/1265691720 wait complete. 2024-09-15T21:26:56.803 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:26:56.803 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:26:56.803 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:26:56.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient: get_monmap_and_config 2024-09-15T21:26:56.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient: build_initial_monmap 2024-09-15T21:26:56.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient: monmap: 2024-09-15T21:26:56.918 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:26:56.918332+0000 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:26:56.918332+0000 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:26:56.919 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 1 Processor -- start 2024-09-15T21:26:56.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 1 -- start start 2024-09-15T21:26:56.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient: init 2024-09-15T21:26:56.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:56.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2024-09-15T21:26:56.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6081378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb60813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient(hunting): picked mon.noname-b con 0x7fb6081383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb60813d2c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6080a1a20 con 0x7fb6081378c0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6080a1d00 con 0x7fb60813d2c0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6080a1fe0 con 0x7fb6081383b0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb606d9d640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fb6081378c0 legacy=0x7fb608137ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60d080640 10 monclient(hunting): authenticate will time out at 1540.313804s 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb60813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb60813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53060/0 (socket says 172.21.15.17:53060) 2024-09-15T21:26:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 -- 172.21.15.17:0/3295076222 learned_addr learned my addr 172.21.15.17:0/3295076222 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): get_auth_request con 0x7fb6081383b0 auth_method 0 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient(hunting): handle_auth_done global_id 45533 payload 274 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 -- 172.21.15.17:0/3295076222 >> v1:172.21.15.119:6789/0 conn(0x7fb60813d2c0 legacy=0x7fb60813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 -- 172.21.15.17:0/3295076222 >> v1:172.21.15.17:6789/0 conn(0x7fb6081378c0 legacy=0x7fb608137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 monclient: found mon.noname-b 2024-09-15T21:26:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.914+0000 7fb60659c640 1 -- 172.21.15.17:0/3295076222 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6080a22c0 con 0x7fb6081383b0 2024-09-15T21:26:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 --2- 172.21.15.17:0/3295076222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb60813cd80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0098a0 tx=0x7fb5fc01bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff0414c4cdec0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 1 -- 172.21.15.17:0/3295076222 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ab070 con 0x7fb6081383b0 2024-09-15T21:26:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:26:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: dump: 2024-09-15T21:26:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 1 -- 172.21.15.17:0/3295076222 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0a5620 con 0x7fb6081383b0 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 1 -- 172.21.15.17:0/3295076222 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0a59f0 con 0x7fb6081383b0 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 5 monclient: authenticate success, global_id 45533 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: get_monmap_and_config success 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 4 set_mon_vals no callback set 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb605d9b640 10 monclient: dump: 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: shutdown 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/3295076222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 msgr2=0x7fb60813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 --2- 172.21.15.17:0/3295076222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb60813cd80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0098a0 tx=0x7fb5fc01bf40 comp rx=0 tx=0).stop 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/3295076222 shutdown_connections 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 --2- 172.21.15.17:0/3295076222 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb60813cd80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/3295076222 >> 172.21.15.17:0/3295076222 conn(0x7fb6080aac40 msgr2=0x7fb6080a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/3295076222 shutdown_connections 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/3295076222 wait complete. 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: build_initial_monmap 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: monmap: 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:26:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 Processor -- start 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- start start 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: init 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: _reopen_session rank -1 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6081378c0 0x7fb6081d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): picked mon.noname-b con 0x7fb6081378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb608136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): picked mon.noname-c con 0x7fb6081383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60813d2c0 0x7fb60809cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): picked mon.noname-a con 0x7fb60813d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb608144340 con 0x7fb60813d2c0 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081441c0 con 0x7fb6081378c0 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6081444c0 con 0x7fb6081383b0 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): _renew_subs 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient(hunting): authenticate will time out at 1540.318781s 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60759e640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60813d2c0 msgr2=0x7fb60809cd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60813d2c0 0x7fb60809cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb608136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb608136d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53066/0 (socket says 172.21.15.17:53066) 2024-09-15T21:26:56.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 -- 172.21.15.17:0/2774157341 learned_addr learned my addr 172.21.15.17:0/2774157341 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): get_auth_request con 0x7fb6081383b0 auth_method 0 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb606d9d640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6081378c0 0x7fb6081d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb606d9d640 10 monclient(hunting): get_auth_request con 0x7fb6081378c0 auth_method 0 2024-09-15T21:26:56.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb606d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb606d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb606d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient(hunting): handle_auth_done global_id 45536 payload 824 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6081378c0 msgr2=0x7fb6081d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6081378c0 0x7fb6081d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60813d2c0 msgr2=0x7fb60809cd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60813d2c0 0x7fb60809cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient: _finish_hunting 0 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 monclient: found mon.noname-c 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6080a0770 con 0x7fb6081383b0 2024-09-15T21:26:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60659c640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb608136d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc01bd70 tx=0x7fb5fc019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ab050 con 0x7fb6081383b0 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: dump: 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.940 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: _finish_auth 0 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 5 monclient: authenticate success, global_id 45536 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: _renew_subs 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0b0650 con 0x7fb6081383b0 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6081407b0 con 0x7fb6081383b0 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0b0a00 con 0x7fb6081383b0 2024-09-15T21:26:56.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb5ef7fe640 10 monclient: dump: 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: _renew_subs 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb608137260 con 0x7fb6081383b0 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.918+0000 7fb60559a640 4 set_mon_vals no callback set 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60d080640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60d080640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 0x7fb60802d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60d080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6080a12f0 con 0x7fb60802d450 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb5fc0ae070 con 0x7fb6081383b0 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 0x7fb60802d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient: get_auth_request con 0x7fb60802d450 auth_method 0 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b970 0x7fb5f403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb5fc0e9a50 con 0x7fb6081383b0 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 10 monclient(hunting): handle_auth_done global_id 114160 payload 824 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6080a8020 con 0x7fb60802d450 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60759e640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b970 0x7fb5f403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60759e640 10 monclient: get_auth_request con 0x7fb5f403b970 auth_method 0 2024-09-15T21:26:56.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb606d9d640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 0x7fb60802d830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb600009fd0 tx=0x7fb600005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600019070 con 0x7fb60802d450 2024-09-15T21:26:56.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:56.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 10 monclient: dump: 2024-09-15T21:26:56.946 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.946 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.946 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.946 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb60759e640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b970 0x7fb5f403de30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8007fd0 tx=0x7fb5f80079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb600014410 con 0x7fb60802d450 2024-09-15T21:26:56.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:26:56.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:26:56.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 msgr2=0x7fb60802d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 0x7fb60802d830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb600009fd0 tx=0x7fb600005420 comp rx=0 tx=0).stop 2024-09-15T21:26:56.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 msgr2=0x7fb60802d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:56.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.922+0000 7fb5ef7fe640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802d450 0x7fb60802d830 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb600009fd0 tx=0x7fb600005420 comp rx=0 tx=0).stop 2024-09-15T21:26:56.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 0x7fb60802e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 10 monclient(hunting): start opening mon connection 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6080a0300 con 0x7fb60802dd70 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 0x7fb60802e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient: get_auth_request con 0x7fb60802dd70 auth_method 0 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:26:56.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 10 monclient(hunting): handle_auth_done global_id 114163 payload 824 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 1 -- 172.21.15.17:0/2774157341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb60809e6f0 con 0x7fb60802dd70 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb606d9d640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 0x7fb60802e150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb600005d30 tx=0x7fb600005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600019040 con 0x7fb60802dd70 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 10 monclient: dump: 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:26:56.951 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7fb6000233b0 con 0x7fb60802dd70 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 msgr2=0x7fb60802e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 0x7fb60802e150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb600005d30 tx=0x7fb600005d60 comp rx=0 tx=0).stop 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 msgr2=0x7fb60802e150 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:26:56.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb5ef7fe640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb60802dd70 0x7fb60802e150 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb600005d30 tx=0x7fb600005d60 comp rx=0 tx=0).stop 2024-09-15T21:26:56.953 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:26:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:26:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 90, 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25495, 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:26:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:26:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:26:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:26:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:26:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:26:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:26:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:26:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:26:56.969 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b970 msgr2=0x7fb5f403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b970 0x7fb5f403de30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8007fd0 tx=0x7fb5f80079a0 comp rx=0 tx=0).stop 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 10 monclient: shutdown 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 msgr2=0x7fb608136d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 --2- 172.21.15.17:0/2774157341 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6081383b0 0x7fb608136d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc01bd70 tx=0x7fb5fc019000 comp rx=0 tx=0).stop 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60759e640 1 -- 172.21.15.17:0/2774157341 reap_dead start 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60759e640 1 -- 172.21.15.17:0/2774157341 reap_dead start 2024-09-15T21:26:56.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 shutdown_connections 2024-09-15T21:26:56.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 >> 172.21.15.17:0/2774157341 conn(0x7fb6080aac40 msgr2=0x7fb60813f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:26:56.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 shutdown_connections 2024-09-15T21:26:56.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:26:56.930+0000 7fb60d080640 1 -- 172.21.15.17:0/2774157341 wait complete. 2024-09-15T21:26:56.972 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:26:56.972 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:27:16.973 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:27:16.974 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:27:16.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:27:16.978 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:27:16.978 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:27:16.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient: get_monmap_and_config 2024-09-15T21:27:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient: build_initial_monmap 2024-09-15T21:27:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient: monmap: 2024-09-15T21:27:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:17.112432+0000 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:17.112432+0000 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:17.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 1 Processor -- start 2024-09-15T21:27:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 1 -- start start 2024-09-15T21:27:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient: init 2024-09-15T21:27:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2024-09-15T21:27:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb18c099600 0x7fb18c09ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient(hunting): picked mon.noname-c con 0x7fb18c099600 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb18c0999e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:27:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb18c099e10 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18c0a2110 con 0x7fb18c0999e0 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18c075200 con 0x7fb18c099e10 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb18c0748c0 con 0x7fb18c099600 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb1936b8640 10 monclient(hunting): authenticate will time out at 1560.507644s 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb190c53640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb18c0999e0 0x7fb18c0a12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51494/0 (socket says 172.21.15.17:51494) 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.106+0000 7fb190c53640 1 -- 172.21.15.17:0/2786578891 learned_addr learned my addr 172.21.15.17:0/2786578891 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 1 --2- 172.21.15.17:0/2786578891 >> v2:172.21.15.119:3300/0 conn(0x7fb18c099600 0x7fb18c09ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): get_auth_request con 0x7fb18c099600 auth_method 0 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient(hunting): handle_auth_done global_id 114166 payload 274 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 1 -- 172.21.15.17:0/2786578891 >> v1:172.21.15.119:6789/0 conn(0x7fb18c099e10 legacy=0x7fb18c0a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 1 -- 172.21.15.17:0/2786578891 >> v1:172.21.15.17:6789/0 conn(0x7fb18c0999e0 legacy=0x7fb18c0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient: _finish_hunting 0 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 1 monclient: found mon.noname-c 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:27:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 1 -- 172.21.15.17:0/2786578891 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c074d80 con 0x7fb18c099600 2024-09-15T21:27:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb191454640 1 --2- 172.21.15.17:0/2786578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c09ee60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0097c0 tx=0x7fb17c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d887a0d37e84263b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 1 -- 172.21.15.17:0/2786578891 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0b0070 con 0x7fb18c099600 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: dump: 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: _finish_auth 0 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb1936b8640 5 monclient: authenticate success, global_id 114166 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb1936b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 1 -- 172.21.15.17:0/2786578891 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb17c0ab600 con 0x7fb18c099600 2024-09-15T21:27:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb1936b8640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb1936b8640 4 set_mon_vals no callback set 2024-09-15T21:27:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 1 -- 172.21.15.17:0/2786578891 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0ab9d0 con 0x7fb18c099600 2024-09-15T21:27:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.110+0000 7fb183fff640 10 monclient: dump: 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: shutdown 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2786578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 msgr2=0x7fb18c09ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2786578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c09ee60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0097c0 tx=0x7fb17c01c330 comp rx=0 tx=0).stop 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2786578891 shutdown_connections 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2786578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c09ee60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2786578891 >> 172.21.15.17:0/2786578891 conn(0x7fb18c0a6820 msgr2=0x7fb18c0a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2786578891 shutdown_connections 2024-09-15T21:27:17.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2786578891 wait complete. 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: build_initial_monmap 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: monmap: 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 Processor -- start 2024-09-15T21:27:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- start start 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: init 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): picked mon.noname-b con 0x7fb18c099600 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 0x7fb18c140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): picked mon.noname-c con 0x7fb18c0999e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 0x7fb18c090fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): picked mon.noname-a con 0x7fb18c099e10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1438a0 con 0x7fb18c099e10 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c143720 con 0x7fb18c099600 2024-09-15T21:27:17.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb18c143a20 con 0x7fb18c0999e0 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient(hunting): authenticate will time out at 1560.513717s 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 0x7fb18c140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 0x7fb18c090fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 0x7fb18c140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:46462/0 (socket says 172.21.15.17:46462) 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 -- 172.21.15.17:0/2576723025 learned_addr learned my addr 172.21.15.17:0/2576723025 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): get_auth_request con 0x7fb18c0999e0 auth_method 0 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 10 monclient(hunting): get_auth_request con 0x7fb18c099e10 auth_method 0 2024-09-15T21:27:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 1 -- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 msgr2=0x7fb18c090fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 1 -- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 msgr2=0x7fb18c090fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 0x7fb18c090fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191c55640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 0x7fb18c090fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): get_auth_request con 0x7fb18c099600 auth_method 0 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb191454640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient(hunting): handle_auth_done global_id 45539 payload 824 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 -- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 msgr2=0x7fb18c13fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c13fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 -- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 msgr2=0x7fb18c090fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 0x7fb18c090fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient: _finish_hunting 0 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 monclient: found mon.noname-c 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 -- 172.21.15.17:0/2576723025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c092a80 con 0x7fb18c0999e0 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb190c53640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 0x7fb18c140150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c4d0 tx=0x7fb18800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188017070 con 0x7fb18c0999e0 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: dump: 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: _finish_auth 0 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb188005cf0 con 0x7fb18c0999e0 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188007270 con 0x7fb18c0999e0 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 5 monclient: authenticate success, global_id 45539 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 10 monclient: dump: 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1837fe640 4 set_mon_vals no callback set 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _renew_subs 2024-09-15T21:27:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c098fe0 con 0x7fb18c0999e0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _renew_subs 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c0935e0 con 0x7fb18c0999e0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18c09ee60 con 0x7fb18c0999e0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb18801a070 con 0x7fb18c0999e0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.114+0000 7fb181ffb640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb17003b9c0 0x7fb17003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.118+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb18804f8d0 con 0x7fb18c0999e0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.118+0000 7fb191454640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb17003b9c0 0x7fb17003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.118+0000 7fb191454640 10 monclient: get_auth_request con 0x7fb17003b9c0 auth_method 0 2024-09-15T21:27:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.118+0000 7fb191454640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb17003b9c0 0x7fb17003de80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0ad000 tx=0x7fb17c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.122+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb188012400 con 0x7fb18c0999e0 2024-09-15T21:27:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.122+0000 7fb181ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.122+0000 7fb181ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:17.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb1936b8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb1936b8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb1936b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb18c069db0 con 0x7fb18c0999e0 2024-09-15T21:27:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb181ffb640 1 -- 172.21.15.17:0/2576723025 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fb188012780 con 0x7fb18c0999e0 2024-09-15T21:27:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb181ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.354+0000 7fb181ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:17.361 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:17.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb17003b9c0 msgr2=0x7fb17003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb17003b9c0 0x7fb17003de80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0ad000 tx=0x7fb17c008040 comp rx=0 tx=0).stop 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 10 monclient: shutdown 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 msgr2=0x7fb18c140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 0x7fb18c140150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c4d0 tx=0x7fb18800c990 comp rx=0 tx=0).stop 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 shutdown_connections 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb17003b9c0 0x7fb17003de80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb18c099e10 0x7fb18c090fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb18c0999e0 0x7fb18c140150 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 --2- 172.21.15.17:0/2576723025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb18c099600 0x7fb18c13fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 >> 172.21.15.17:0/2576723025 conn(0x7fb18c0a6820 msgr2=0x7fb18c097df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 shutdown_connections 2024-09-15T21:27:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:17.358+0000 7fb1936b8640 1 -- 172.21.15.17:0/2576723025 wait complete. 2024-09-15T21:27:20.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.506+0000 7f7659a2e640 10 monclient: get_monmap_and_config 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.506+0000 7f7659a2e640 10 monclient: build_initial_monmap 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.506+0000 7f7659a2e640 10 monclient: monmap: 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:20.512899+0000 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:20.512899+0000 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:20.514 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:20.514 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:20.514 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:20.514 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.506+0000 7f7659a2e640 1 Processor -- start 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.506+0000 7f7659a2e640 1 -- start start 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.506+0000 7f7659a2e640 10 monclient: init 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f76541378c0 0x7f7654137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): picked mon.noname-c con 0x7f76541378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f76540a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): picked mon.noname-b con 0x7f76541381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:20.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f76540a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76540a1d00 con 0x7f76540a5040 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f76540a1fe0 con 0x7f76541378c0 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f76540a22c0 con 0x7f76541381e0 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient(hunting): authenticate will time out at 1563.908317s 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f76540a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f76540a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49392/0 (socket says 172.21.15.17:49392) 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 -- 172.21.15.17:0/2432560091 learned_addr learned my addr 172.21.15.17:0/2432560091 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): get_auth_request con 0x7f76541381e0 auth_method 0 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:20.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7652ffd640 1 --2- 172.21.15.17:0/2432560091 >> v2:172.21.15.119:3300/0 conn(0x7f76541378c0 0x7f7654137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient(hunting): handle_auth_done global_id 45542 payload 274 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 -- 172.21.15.17:0/2432560091 >> v2:172.21.15.119:3300/0 conn(0x7f76541378c0 msgr2=0x7f7654137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 --2- 172.21.15.17:0/2432560091 >> v2:172.21.15.119:3300/0 conn(0x7f76541378c0 0x7f7654137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 -- 172.21.15.17:0/2432560091 >> v1:172.21.15.119:6789/0 conn(0x7f76540a5040 legacy=0x7f76540a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient: _finish_hunting 0 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 monclient: found mon.noname-b 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 -- 172.21.15.17:0/2432560091 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76540a5c00 con 0x7f76541381e0 2024-09-15T21:27:20.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7652ffd640 10 monclient: get_auth_request con 0x7f76541378c0 auth_method 0 2024-09-15T21:27:20.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7652ffd640 1 --2- 172.21.15.17:0/2432560091 >> v2:172.21.15.119:3300/0 conn(0x7f76541378c0 0x7f7654137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:27:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f76527fc640 1 --2- 172.21.15.17:0/2432560091 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f76540a4b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7648009870 tx=0x7f764801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f95740a1b9759c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 1 -- 172.21.15.17:0/2432560091 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76480b0070 con 0x7f76541381e0 2024-09-15T21:27:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: dump: 2024-09-15T21:27:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: _finish_auth 0 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:20.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 1 -- 172.21.15.17:0/2432560091 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76480ab600 con 0x7f76541381e0 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 1 -- 172.21.15.17:0/2432560091 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76480ab9d0 con 0x7f76541381e0 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 5 monclient: authenticate success, global_id 45542 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 4 set_mon_vals no callback set 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7651ffb640 10 monclient: dump: 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:20.526 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 10 monclient: shutdown 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- 172.21.15.17:0/2432560091 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 msgr2=0x7f76540a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2432560091 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f76540a4b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7648009870 tx=0x7f764801c420 comp rx=0 tx=0).stop 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- 172.21.15.17:0/2432560091 shutdown_connections 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2432560091 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f76540a4b00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2432560091 >> v2:172.21.15.119:3300/0 conn(0x7f76541378c0 0x7f7654137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- 172.21.15.17:0/2432560091 >> 172.21.15.17:0/2432560091 conn(0x7f76540aac40 msgr2=0x7f76540a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.510+0000 7f7659a2e640 1 -- 172.21.15.17:0/2432560091 shutdown_connections 2024-09-15T21:27:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- 172.21.15.17:0/2432560091 wait complete. 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: build_initial_monmap 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: monmap: 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 Processor -- start 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- start start 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: init 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76540a5040 0x7f76541cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): picked mon.noname-b con 0x7f76540a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 0x7f7654136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): picked mon.noname-a con 0x7f76541378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f765413cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): picked mon.noname-c con 0x7f76541381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f76541425c0 con 0x7f76541378c0 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7654138700 con 0x7f76540a5040 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7654142740 con 0x7f76541381e0 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient(hunting): authenticate will time out at 1563.913553s 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f765413cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 0x7f7654136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f765413cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49398/0 (socket says 172.21.15.17:49398) 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 -- 172.21.15.17:0/2162893581 learned_addr learned my addr 172.21.15.17:0/2162893581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): get_auth_request con 0x7f76541381e0 auth_method 0 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 10 monclient(hunting): get_auth_request con 0x7f76541378c0 auth_method 0 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 1 -- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 msgr2=0x7f7654136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 1 -- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 msgr2=0x7f7654136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 0x7f7654136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7652ffd640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76540a5040 0x7f76541cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76527fc640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 0x7f7654136960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7652ffd640 10 monclient(hunting): get_auth_request con 0x7f76540a5040 auth_method 0 2024-09-15T21:27:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7652ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7652ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7652ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient(hunting): handle_auth_done global_id 45545 payload 824 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 -- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76540a5040 msgr2=0x7f76541cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76540a5040 0x7f76541cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 -- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 msgr2=0x7f7654136960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 0x7f7654136960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 monclient: found mon.noname-c 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 -- 172.21.15.17:0/2162893581 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76541212f0 con 0x7f76541381e0 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7652ffd640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76540a5040 0x7f76541cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:27:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76537fe640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f765413cba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f764c00c440 tx=0x7f764c00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c017070 con 0x7f76541381e0 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: dump: 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: _finish_auth 0 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c003600 con 0x7f76541381e0 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 5 monclient: authenticate success, global_id 45545 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0039f0 con 0x7f76541381e0 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 10 monclient: dump: 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _renew_subs 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f76517fa640 4 set_mon_vals no callback set 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765409fed0 con 0x7f76541381e0 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _renew_subs 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76540a12f0 con 0x7f76541381e0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f764c003b90 con 0x7f76541381e0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7654137ca0 con 0x7f76541381e0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764003b9f0 0x7f764003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.514+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f764c04f7a0 con 0x7f76541381e0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.518+0000 7f7652ffd640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764003b9f0 0x7f764003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.518+0000 7f7652ffd640 10 monclient: get_auth_request con 0x7f764003b9f0 auth_method 0 2024-09-15T21:27:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.518+0000 7f7652ffd640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764003b9f0 0x7f764003deb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f764400e550 tx=0x7f7644007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.522+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f764c051d10 con 0x7f76541381e0 2024-09-15T21:27:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.522+0000 7f763bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.522+0000 7f763bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:20.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.758+0000 7f7659a2e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:20.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.758+0000 7f7659a2e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:20.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.758+0000 7f7659a2e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:20.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.758+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7654069270 con 0x7f76541381e0 2024-09-15T21:27:20.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f763bfff640 1 -- 172.21.15.17:0/2162893581 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f764c01acf0 con 0x7f76541381e0 2024-09-15T21:27:20.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f763bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:20.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f763bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:20.766 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:20.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764003b9f0 msgr2=0x7f764003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:20.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764003b9f0 0x7f764003deb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f764400e550 tx=0x7f7644007ab0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 10 monclient: shutdown 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 msgr2=0x7f765413cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f765413cba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f764c00c440 tx=0x7f764c00c900 comp rx=0 tx=0).stop 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 shutdown_connections 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764003b9f0 0x7f764003deb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f76541381e0 0x7f765413cba0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76541378c0 0x7f7654136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 --2- 172.21.15.17:0/2162893581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76540a5040 0x7f76541cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 >> 172.21.15.17:0/2162893581 conn(0x7f76540aac40 msgr2=0x7f76540a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 shutdown_connections 2024-09-15T21:27:20.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:20.762+0000 7f7659a2e640 1 -- 172.21.15.17:0/2162893581 wait complete. 2024-09-15T21:27:23.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient: get_monmap_and_config 2024-09-15T21:27:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient: build_initial_monmap 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient: monmap: 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:23.919596+0000 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:23.919596+0000 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:23.921 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 Processor -- start 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 -- start start 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient: init 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a780a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): picked mon.noname-b con 0x7f0a780a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0a7809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a780a11b0 0x7f0a78135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): picked mon.noname-a con 0x7f0a780a11b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a780a3130 con 0x7f0a7809ffc0 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0a78073df0 con 0x7f0a780a11b0 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0a780a1bf0 con 0x7f0a780a0890 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a80403640 10 monclient(hunting): authenticate will time out at 1567.314999s 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a780a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a780a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49426/0 (socket says 172.21.15.17:49426) 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 -- 172.21.15.17:0/1638248999 learned_addr learned my addr 172.21.15.17:0/1638248999 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 1 --2- 172.21.15.17:0/1638248999 >> v2:172.21.15.17:3300/0 conn(0x7f0a780a11b0 0x7f0a78135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): get_auth_request con 0x7f0a780a0890 auth_method 0 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): get_auth_request con 0x7f0a780a11b0 auth_method 0 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient(hunting): handle_auth_done global_id 45548 payload 274 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 -- 172.21.15.17:0/1638248999 >> v2:172.21.15.17:3300/0 conn(0x7f0a780a11b0 msgr2=0x7f0a78135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 --2- 172.21.15.17:0/1638248999 >> v2:172.21.15.17:3300/0 conn(0x7f0a780a11b0 0x7f0a78135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 -- 172.21.15.17:0/1638248999 >> v1:172.21.15.119:6789/0 conn(0x7f0a7809ffc0 legacy=0x7f0a78135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient: _finish_hunting 0 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 monclient: found mon.noname-b 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.914+0000 7f0a7e19f640 1 -- 172.21.15.17:0/1638248999 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a780a15e0 con 0x7f0a780a0890 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7e19f640 1 --2- 172.21.15.17:0/1638248999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a780a0c70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a780a1870 tx=0x7f0a6c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2180b59c8fd849 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 1 -- 172.21.15.17:0/1638248999 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0b1070 con 0x7f0a780a0890 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: dump: 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: _finish_auth 0 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: _check_auth_tickets 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 5 monclient: authenticate success, global_id 45548 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 1 -- 172.21.15.17:0/1638248999 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0ac640 con 0x7f0a780a0890 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 4 set_mon_vals no callback set 2024-09-15T21:27:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 1 -- 172.21.15.17:0/1638248999 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0aca10 con 0x7f0a780a0890 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a7d19d640 10 monclient: dump: 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:23.938 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 10 monclient: shutdown 2024-09-15T21:27:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 -- 172.21.15.17:0/1638248999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 msgr2=0x7f0a780a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 --2- 172.21.15.17:0/1638248999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a780a0c70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a780a1870 tx=0x7f0a6c01c400 comp rx=0 tx=0).stop 2024-09-15T21:27:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 -- 172.21.15.17:0/1638248999 shutdown_connections 2024-09-15T21:27:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 --2- 172.21.15.17:0/1638248999 >> v2:172.21.15.17:3300/0 conn(0x7f0a780a11b0 0x7f0a78135d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 --2- 172.21.15.17:0/1638248999 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a780a0c70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 -- 172.21.15.17:0/1638248999 >> 172.21.15.17:0/1638248999 conn(0x7f0a780a66a0 msgr2=0x7f0a780a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 -- 172.21.15.17:0/1638248999 shutdown_connections 2024-09-15T21:27:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 -- 172.21.15.17:0/1638248999 wait complete. 2024-09-15T21:27:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 10 monclient: build_initial_monmap 2024-09-15T21:27:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 10 monclient: monmap: 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 Processor -- start 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 1 -- start start 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.918+0000 7f0a80403640 10 monclient: init 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7809ffc0 0x7f0a78137180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): picked mon.noname-b con 0x7f0a7809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a781376c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): picked mon.noname-c con 0x7f0a780a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 0x7f0a78137c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): picked mon.noname-a con 0x7f0a780a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7813e680 con 0x7f0a780a11b0 2024-09-15T21:27:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7813e500 con 0x7f0a7809ffc0 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7813e800 con 0x7f0a780a0890 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient(hunting): authenticate will time out at 1567.320464s 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 0x7f0a78137c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a781376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 0x7f0a78137c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37074/0 (socket says 172.21.15.17:37074) 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 -- 172.21.15.17:0/3131610345 learned_addr learned my addr 172.21.15.17:0/3131610345 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): get_auth_request con 0x7f0a780a11b0 auth_method 0 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): get_auth_request con 0x7f0a780a0890 auth_method 0 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7d99e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7809ffc0 0x7f0a78137180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 10 monclient(hunting): get_auth_request con 0x7f0a7809ffc0 auth_method 0 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient(hunting): handle_auth_done global_id 124101 payload 824 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 -- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 msgr2=0x7f0a781376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a781376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 -- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7809ffc0 msgr2=0x7f0a78137180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7809ffc0 0x7f0a78137180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient: _finish_hunting 0 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 monclient: found mon.noname-a 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 -- 172.21.15.17:0/3131610345 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7806f3c0 con 0x7f0a780a11b0 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e9a0640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 0x7f0a78137c00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000efc0 tx=0x7f0a7000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70018020 con 0x7f0a780a11b0 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: dump: 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: _finish_auth 0 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: _check_auth_tickets 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a70014480 con 0x7f0a780a11b0 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70014870 con 0x7f0a780a11b0 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 10 monclient: dump: 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7c99c640 4 set_mon_vals no callback set 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 5 monclient: authenticate success, global_id 124101 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _renew_subs 2024-09-15T21:27:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a780a54d0 con 0x7f0a780a11b0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _renew_subs 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7813afa0 con 0x7f0a780a11b0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a780a0c70 con 0x7f0a780a11b0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0a7001b020 con 0x7f0a780a11b0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a5c03b9c0 0x7f0a5c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0a70051110 con 0x7f0a780a11b0 2024-09-15T21:27:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a5c03b9c0 0x7f0a5c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.922+0000 7f0a7e19f640 10 monclient: get_auth_request con 0x7f0a5c03b9c0 auth_method 0 2024-09-15T21:27:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.926+0000 7f0a7e19f640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a5c03b9c0 0x7f0a5c03de80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0ae000 tx=0x7f0a6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.926+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0a70022070 con 0x7f0a780a11b0 2024-09-15T21:27:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.926+0000 7f0a6affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:23.926+0000 7f0a6affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0a78139b70 con 0x7f0a780a11b0 2024-09-15T21:27:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a6affd640 1 -- 172.21.15.17:0/3131610345 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7f0a70010040 con 0x7f0a780a11b0 2024-09-15T21:27:24.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a6affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:24.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a6affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:24.172 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a5c03b9c0 msgr2=0x7f0a5c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a5c03b9c0 0x7f0a5c03de80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0ae000 tx=0x7f0a6c008040 comp rx=0 tx=0).stop 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 10 monclient: shutdown 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 msgr2=0x7f0a78137c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 0x7f0a78137c00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000efc0 tx=0x7f0a7000c490 comp rx=0 tx=0).stop 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 shutdown_connections 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a5c03b9c0 0x7f0a5c03de80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780a11b0 0x7f0a78137c00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a780a0890 0x7f0a781376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 --2- 172.21.15.17:0/3131610345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7809ffc0 0x7f0a78137180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 >> 172.21.15.17:0/3131610345 conn(0x7f0a780a66a0 msgr2=0x7f0a780727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 shutdown_connections 2024-09-15T21:27:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.166+0000 7f0a80403640 1 -- 172.21.15.17:0/3131610345 wait complete. 2024-09-15T21:27:24.191 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:27:24.191 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:27:24.192 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:27:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient: get_monmap_and_config 2024-09-15T21:27:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient: build_initial_monmap 2024-09-15T21:27:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient: monmap: 2024-09-15T21:27:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:24.319179+0000 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:24.319179+0000 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:24.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 Processor -- start 2024-09-15T21:27:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 -- start start 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient: init 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): picked mon.noname-b con 0x7f8fd41378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8fd41381e0 0x7f8fd40a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): picked mon.noname-c con 0x7f8fd41381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8fd40a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fd40a1d00 con 0x7f8fd40a5040 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8fd40a1fe0 con 0x7f8fd41381e0 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd40a22c0 con 0x7f8fd41378c0 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd9aae640 10 monclient(hunting): authenticate will time out at 1567.714547s 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49452/0 (socket says 172.21.15.17:49452) 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 -- 172.21.15.17:0/2570136310 learned_addr learned my addr 172.21.15.17:0/2570136310 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): get_auth_request con 0x7f8fd41378c0 auth_method 0 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/2570136310 >> v2:172.21.15.119:3300/0 conn(0x7f8fd41381e0 0x7f8fd40a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd2ffd640 10 monclient(hunting): get_auth_request con 0x7f8fd41381e0 auth_method 0 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_done global_id 45554 payload 274 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 -- 172.21.15.17:0/2570136310 >> v2:172.21.15.119:3300/0 conn(0x7f8fd41381e0 msgr2=0x7f8fd40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 --2- 172.21.15.17:0/2570136310 >> v2:172.21.15.119:3300/0 conn(0x7f8fd41381e0 0x7f8fd40a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 -- 172.21.15.17:0/2570136310 >> v1:172.21.15.119:6789/0 conn(0x7f8fd40a5040 legacy=0x7f8fd40a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 monclient: found mon.noname-b 2024-09-15T21:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.314+0000 7f8fd37fe640 1 -- 172.21.15.17:0/2570136310 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd40a5c00 con 0x7f8fd41378c0 2024-09-15T21:27:24.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 1 --2- 172.21.15.17:0/2570136310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4137ca0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009870 tx=0x7f8fc401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a5161bbc089d0fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 1 -- 172.21.15.17:0/2570136310 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40b0070 con 0x7f8fd41378c0 2024-09-15T21:27:24.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: dump: 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: _finish_auth 0 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 1 -- 172.21.15.17:0/2570136310 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc40ab600 con 0x7f8fd41378c0 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 5 monclient: authenticate success, global_id 45554 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 1 -- 172.21.15.17:0/2570136310 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40ab9d0 con 0x7f8fd41378c0 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:24.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd27fc640 10 monclient: dump: 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 4 set_mon_vals no callback set 2024-09-15T21:27:24.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: shutdown 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- 172.21.15.17:0/2570136310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 msgr2=0x7f8fd4137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/2570136310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4137ca0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009870 tx=0x7f8fc401c4b0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- 172.21.15.17:0/2570136310 shutdown_connections 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/2570136310 >> v2:172.21.15.119:3300/0 conn(0x7f8fd41381e0 0x7f8fd40a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/2570136310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4137ca0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- 172.21.15.17:0/2570136310 >> 172.21.15.17:0/2570136310 conn(0x7f8fd40aac40 msgr2=0x7f8fd40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- 172.21.15.17:0/2570136310 shutdown_connections 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- 172.21.15.17:0/2570136310 wait complete. 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: build_initial_monmap 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: monmap: 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:24.336 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 Processor -- start 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- start start 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: init 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:27:24.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40a5040 0x7f8fd41cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): picked mon.noname-a con 0x7f8fd40a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): picked mon.noname-c con 0x7f8fd41378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd41381e0 0x7f8fd413cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): picked mon.noname-b con 0x7f8fd41381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41425c0 con 0x7f8fd40a5040 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41386d0 con 0x7f8fd41381e0 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd4142740 con 0x7f8fd41378c0 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd9aae640 10 monclient(hunting): authenticate will time out at 1567.719799s 2024-09-15T21:27:24.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40a5040 0x7f8fd41cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40a5040 0x7f8fd41cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37080/0 (socket says 172.21.15.17:37080) 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 1 -- 172.21.15.17:0/52032010 learned_addr learned my addr 172.21.15.17:0/52032010 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 10 monclient(hunting): get_auth_request con 0x7f8fd40a5040 auth_method 0 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.318+0000 7f8fd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): get_auth_request con 0x7f8fd41378c0 auth_method 0 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd41381e0 0x7f8fd413cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 10 monclient(hunting): get_auth_request con 0x7f8fd41381e0 auth_method 0 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 10 monclient(hunting): handle_auth_done global_id 124104 payload 824 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 msgr2=0x7f8fd4136ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8fd41378c0 0x7f8fd4136ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd41381e0 msgr2=0x7f8fd413cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd41381e0 0x7f8fd413cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 monclient: found mon.noname-a 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd413e730 con 0x7f8fd40a5040 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd37fe640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40a5040 0x7f8fd41cfa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40ae040 tx=0x7f8fc4019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40b0040 con 0x7f8fd40a5040 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: dump: 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.341 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: _finish_auth 0 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 5 monclient: authenticate success, global_id 124104 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 10 monclient: _renew_subs 2024-09-15T21:27:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc40b9070 con 0x7f8fd40a5040 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd413f290 con 0x7f8fd40a5040 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40b4680 con 0x7f8fd40a5040 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: dump: 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.343 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 10 monclient: _renew_subs 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd1ffb640 4 set_mon_vals no callback set 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd4121440 con 0x7f8fd40a5040 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 0x7f8fd413d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd4137ca0 con 0x7f8fd4136ff0 2024-09-15T21:27:24.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 0x7f8fd413d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient: get_auth_request con 0x7f8fd4136ff0 auth_method 0 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8fc40b4820 con 0x7f8fd40a5040 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_done global_id 124107 payload 824 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fbc03b9f0 0x7f8fbc03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fd41d08c0 con 0x7f8fd4136ff0 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f8fc40ea330 con 0x7f8fd40a5040 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 0x7f8fd413d1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00107f0 tx=0x7f8fc0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0018020 con 0x7f8fd4136ff0 2024-09-15T21:27:24.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:24.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: dump: 2024-09-15T21:27:24.346 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.346 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.346 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fbc03b9f0 0x7f8fbc03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 10 monclient: get_auth_request con 0x7f8fbc03b9f0 auth_method 0 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8fc0004240 con 0x7f8fd4136ff0 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 msgr2=0x7f8fd413d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 0x7f8fd413d1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00107f0 tx=0x7f8fc0010cb0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 msgr2=0x7f8fd413d1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fcbfff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd4136ff0 0x7f8fd413d1b0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00107f0 tx=0x7f8fc0010cb0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.322+0000 7f8fd3fff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fbc03b9f0 0x7f8fbc03deb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8fd413f020 tx=0x7f8fcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd9aae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd9aae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd413db00 con 0x7f8fd413d6f0 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient: get_auth_request con 0x7f8fd413d6f0 auth_method 0 2024-09-15T21:27:24.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 10 monclient(hunting): handle_auth_done global_id 124110 payload 824 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 1 -- 172.21.15.17:0/52032010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8fd4120f60 con 0x7f8fd413d6f0 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 msgr2=0x7f8fd4138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 msgr2=0x7f8fd4138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00107f0 tx=0x7f8fc00046d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:24.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.330+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: get_auth_request con 0x7f8fd413d6f0 auth_method 0 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: _init_auth method 2 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: _init_auth already have auth, reseting 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: handle_auth_reply_more payload 9 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 10 monclient: handle_auth_done global_id 124110 payload 293 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd2ffd640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0010cf0 tx=0x7f8fc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc002e020 con 0x7f8fd413d6f0 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 10 monclient: dump: 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.540 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.541 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f8fc000ec20 con 0x7f8fd413d6f0 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 msgr2=0x7f8fd4138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0010cf0 tx=0x7f8fc0014000 comp rx=0 tx=0).stop 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 msgr2=0x7f8fd4138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fcbfff640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd413d6f0 0x7f8fd4138d60 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0010cf0 tx=0x7f8fc0014000 comp rx=0 tx=0).stop 2024-09-15T21:27:24.542 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 94, 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7483, 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:27:24.543 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.544 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.545 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:24.546 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.547 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.548 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.550 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.551 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.551 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:24.551 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:24.551 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:24.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:24.555 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.556 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.556 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.556 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:27:24.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.558 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:24.558 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fbc03b9f0 msgr2=0x7f8fbc03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8fbc03b9f0 0x7f8fbc03deb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8fd413f020 tx=0x7f8fcc009040 comp rx=0 tx=0).stop 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 10 monclient: shutdown 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40a5040 msgr2=0x7f8fd41cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 1 --2- 172.21.15.17:0/52032010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40a5040 0x7f8fd41cfa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40ae040 tx=0x7f8fc4019250 comp rx=0 tx=0).stop 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd3fff640 1 -- 172.21.15.17:0/52032010 reap_dead start 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd3fff640 1 -- 172.21.15.17:0/52032010 reap_dead start 2024-09-15T21:27:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 shutdown_connections 2024-09-15T21:27:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.534+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 >> 172.21.15.17:0/52032010 conn(0x7f8fd40aac40 msgr2=0x7f8fd40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.538+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 shutdown_connections 2024-09-15T21:27:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.538+0000 7f8fd9aae640 1 -- 172.21.15.17:0/52032010 wait complete. 2024-09-15T21:27:24.565 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:27:24.565 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:27:24.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:27:24.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient: get_monmap_and_config 2024-09-15T21:27:24.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient: build_initial_monmap 2024-09-15T21:27:24.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient: monmap: 2024-09-15T21:27:24.695 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:24.695 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:24.695142+0000 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:24.695142+0000 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:24.696 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 Processor -- start 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 -- start start 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient: init 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bd80a0890 0x7f1bd80a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): picked mon.noname-a con 0x7f1bd80a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd809ffc0 0x7f1bd80a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): picked mon.noname-b con 0x7f1bd809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:24.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1bd8135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:27:24.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd80a3130 con 0x7f1bd8135670 2024-09-15T21:27:24.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd80a16f0 con 0x7f1bd80a0890 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd8073df0 con 0x7f1bd809ffc0 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bddf43640 10 monclient(hunting): authenticate will time out at 1568.090555s 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bd80a0890 0x7f1bd80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd809ffc0 0x7f1bd80a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1bd8135670 0x7f1bd8135aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39920/0 (socket says 172.21.15.17:39920) 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd7fff640 1 -- 172.21.15.17:0/976053449 learned_addr learned my addr 172.21.15.17:0/976053449 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): get_auth_request con 0x7f1bd809ffc0 auth_method 0 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): get_auth_request con 0x7f1bd80a0890 auth_method 0 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_done global_id 124113 payload 274 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 1 -- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd809ffc0 msgr2=0x7f1bd80a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd809ffc0 0x7f1bd80a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 1 -- 172.21.15.17:0/976053449 >> v1:172.21.15.17:6789/0 conn(0x7f1bd8135670 legacy=0x7f1bd8135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:24.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 1 monclient: found mon.noname-a 2024-09-15T21:27:24.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:24.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd77fe640 1 -- 172.21.15.17:0/976053449 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd80a1bf0 con 0x7f1bd80a0890 2024-09-15T21:27:24.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.690+0000 7f1bd6ffd640 1 --2- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd809ffc0 0x7f1bd80a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:24.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd80a0890 0x7f1bd80a0c70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80a1870 tx=0x7f1bc801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982beef877fea4f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 1 -- 172.21.15.17:0/976053449 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80b0070 con 0x7f1bd80a0890 2024-09-15T21:27:24.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: dump: 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.716 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: _finish_auth 0 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 1 -- 172.21.15.17:0/976053449 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc80ab600 con 0x7f1bd80a0890 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 1 -- 172.21.15.17:0/976053449 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80ab9d0 con 0x7f1bd80a0890 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 5 monclient: authenticate success, global_id 124113 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 4 set_mon_vals no callback set 2024-09-15T21:27:24.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:24.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd67fc640 10 monclient: dump: 2024-09-15T21:27:24.718 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.719 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: shutdown 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd80a0890 msgr2=0x7f1bd80a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 --2- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd80a0890 0x7f1bd80a0c70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80a1870 tx=0x7f1bc801c360 comp rx=0 tx=0).stop 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- 172.21.15.17:0/976053449 shutdown_connections 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 --2- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd809ffc0 0x7f1bd80a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 --2- 172.21.15.17:0/976053449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd80a0890 0x7f1bd80a0c70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- 172.21.15.17:0/976053449 >> 172.21.15.17:0/976053449 conn(0x7f1bd80a66a0 msgr2=0x7f1bd80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- 172.21.15.17:0/976053449 shutdown_connections 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- 172.21.15.17:0/976053449 wait complete. 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: build_initial_monmap 2024-09-15T21:27:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: monmap: 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 Processor -- start 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- start start 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: init 2024-09-15T21:27:24.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bd809ffc0 0x7f1bd811c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): picked mon.noname-b con 0x7f1bd809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd80a0890 0x7f1bd811ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): picked mon.noname-c con 0x7f1bd80a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd8135670 0x7f1bd811cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): picked mon.noname-a con 0x7f1bd8135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8140690 con 0x7f1bd8135670 2024-09-15T21:27:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8140510 con 0x7f1bd809ffc0 2024-09-15T21:27:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd8140810 con 0x7f1bd80a0890 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bddf43640 10 monclient(hunting): authenticate will time out at 1568.095762s 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd80a0890 0x7f1bd811ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd8135670 0x7f1bd811cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd80a0890 0x7f1bd811ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49490/0 (socket says 172.21.15.17:49490) 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 1 -- 172.21.15.17:0/3268429695 learned_addr learned my addr 172.21.15.17:0/3268429695 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 10 monclient(hunting): get_auth_request con 0x7f1bd80a0890 auth_method 0 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 10 monclient(hunting): get_auth_request con 0x7f1bd8135670 auth_method 0 2024-09-15T21:27:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.694+0000 7f1bd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bd809ffc0 0x7f1bd811c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 10 monclient(hunting): handle_auth_done global_id 124119 payload 824 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd80a0890 msgr2=0x7f1bd811ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd80a0890 0x7f1bd811ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bd809ffc0 msgr2=0x7f1bd811c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bd809ffc0 0x7f1bd811c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 monclient: found mon.noname-a 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd811e850 con 0x7f1bd8135670 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient: get_auth_request con 0x7f1bd809ffc0 auth_method 0 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bd809ffc0 0x7f1bd811c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd7fff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd8135670 0x7f1bd811cf60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000cae0 tx=0x7f1bd000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0017070 con 0x7f1bd8135670 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: dump: 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.727 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: _finish_auth 0 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd000ac40 con 0x7f1bd8135670 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0012600 con 0x7f1bd8135670 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 5 monclient: authenticate success, global_id 124119 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 10 monclient: _renew_subs 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd811eb30 con 0x7f1bd8135670 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd5ffb640 4 set_mon_vals no callback set 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: dump: 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 10 monclient: _renew_subs 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd811e570 con 0x7f1bd8135670 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 0x7f1bd8133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd813d840 con 0x7f1bd813c850 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 0x7f1bd8133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1bd001a070 con 0x7f1bd8135670 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient: get_auth_request con 0x7f1bd813c850 auth_method 0 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_done global_id 45566 payload 824 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1bc403b9c0 0x7f1bc403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bd8135aa0 con 0x7f1bd813c850 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f1bd0050380 con 0x7f1bd8135670 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 0x7f1bd8133cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1bd806f1c0 tx=0x7f1bc801aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80bb070 con 0x7f1bd813c850 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: dump: 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd6ffd640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1bc403b9c0 0x7f1bc403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd6ffd640 10 monclient: get_auth_request con 0x7f1bc403b9c0 auth_method 0 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1bc80b1040 con 0x7f1bd813c850 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 msgr2=0x7f1bd8133cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 0x7f1bd8133cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1bd806f1c0 tx=0x7f1bc801aba0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 msgr2=0x7f1bd8133cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bbffff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd813c850 0x7f1bd8133cf0 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1bd806f1c0 tx=0x7f1bc801aba0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.698+0000 7f1bd6ffd640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1bc403b9c0 0x7f1bc403de80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1bd811d5a0 tx=0x7f1bcc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 0x7f1bd8134640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd8134b80 con 0x7f1bd8134230 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 0x7f1bd8134640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient: get_auth_request con 0x7f1bd8134230 auth_method 0 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 10 monclient(hunting): handle_auth_done global_id 45569 payload 824 2024-09-15T21:27:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 1 -- 172.21.15.17:0/3268429695 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1bd81d8c10 con 0x7f1bd8134230 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd77fe640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 0x7f1bd8134640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000c10 tx=0x7f1bc8019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80bb040 con 0x7f1bd8134230 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 10 monclient: dump: 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.736 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7f1bc80bd2d0 con 0x7f1bd8134230 2024-09-15T21:27:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 msgr2=0x7f1bd8134640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 0x7f1bd8134640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000c10 tx=0x7f1bc8019100 comp rx=0 tx=0).stop 2024-09-15T21:27:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 msgr2=0x7f1bd8134640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bbffff640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1bd8134230 0x7f1bd8134640 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000c10 tx=0x7f1bc8019100 comp rx=0 tx=0).stop 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 94, 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 413304, 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.738 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.739 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.740 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.740 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.740 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.740 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.741 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.742 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.742 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.742 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:24.742 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.743 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:24.744 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.745 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.746 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:24.750 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.752 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:27:24.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:27:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:24.755 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:24.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1bc403b9c0 msgr2=0x7f1bc403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1bc403b9c0 0x7f1bc403de80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1bd811d5a0 tx=0x7f1bcc009190 comp rx=0 tx=0).stop 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 10 monclient: shutdown 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd8135670 msgr2=0x7f1bd811cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 --2- 172.21.15.17:0/3268429695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd8135670 0x7f1bd811cf60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000cae0 tx=0x7f1bd000a060 comp rx=0 tx=0).stop 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd7fff640 1 -- 172.21.15.17:0/3268429695 reap_dead start 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bd7fff640 1 -- 172.21.15.17:0/3268429695 reap_dead start 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 shutdown_connections 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 >> 172.21.15.17:0/3268429695 conn(0x7f1bd80a66a0 msgr2=0x7f1bd80a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 shutdown_connections 2024-09-15T21:27:24.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.706+0000 7f1bddf43640 1 -- 172.21.15.17:0/3268429695 wait complete. 2024-09-15T21:27:24.758 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:27:24.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:27:24.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:27:24.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient: get_monmap_and_config 2024-09-15T21:27:24.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient: build_initial_monmap 2024-09-15T21:27:24.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient: monmap: 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:24.878541+0000 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:24.878541+0000 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:24.879 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:24.880 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:24.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 Processor -- start 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 -- start start 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient: init 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 0x7f7274073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): picked mon.noname-b con 0x7f7274073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7274073f60 0x7f7274137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): picked mon.noname-c con 0x7f7274073f60 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7274137bd0 0x7f7274138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): picked mon.noname-a con 0x7f7274137bd0 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f72740a3640 con 0x7f7274137bd0 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f72740a0890 con 0x7f7274073f60 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f72740746e0 con 0x7f7274073640 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7279483640 10 monclient(hunting): authenticate will time out at 1568.273953s 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 0x7f7274073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7274137bd0 0x7f7274138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 0x7f7274073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49528/0 (socket says 172.21.15.17:49528) 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 -- 172.21.15.17:0/63924684 learned_addr learned my addr 172.21.15.17:0/63924684 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): get_auth_request con 0x7f7274073640 auth_method 0 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): get_auth_request con 0x7f7274137bd0 auth_method 0 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72727fc640 1 --2- 172.21.15.17:0/63924684 >> v2:172.21.15.119:3300/0 conn(0x7f7274073f60 0x7f7274137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72727fc640 10 monclient(hunting): get_auth_request con 0x7f7274073f60 auth_method 0 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72727fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient(hunting): handle_auth_done global_id 45572 payload 274 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 -- 172.21.15.17:0/63924684 >> v2:172.21.15.119:3300/0 conn(0x7f7274073f60 msgr2=0x7f7274137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 --2- 172.21.15.17:0/63924684 >> v2:172.21.15.119:3300/0 conn(0x7f7274073f60 0x7f7274137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 -- 172.21.15.17:0/63924684 >> v2:172.21.15.17:3300/0 conn(0x7f7274137bd0 msgr2=0x7f7274138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 --2- 172.21.15.17:0/63924684 >> v2:172.21.15.17:3300/0 conn(0x7f7274137bd0 0x7f7274138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 monclient: found mon.noname-b 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f7272ffd640 1 -- 172.21.15.17:0/63924684 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72740a1640 con 0x7f7274073640 2024-09-15T21:27:24.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.874+0000 7f72737fe640 1 --2- 172.21.15.17:0/63924684 >> v2:172.21.15.17:3300/0 conn(0x7f7274137bd0 0x7f7274138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:24.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7272ffd640 1 --2- 172.21.15.17:0/63924684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 0x7f7274073a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f727406e7b0 tx=0x7f726401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a102d76500d9b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 1 -- 172.21.15.17:0/63924684 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640b0070 con 0x7f7274073640 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: dump: 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: _finish_auth 0 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 1 -- 172.21.15.17:0/63924684 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72640ab600 con 0x7f7274073640 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 1 -- 172.21.15.17:0/63924684 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640ab9d0 con 0x7f7274073640 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 5 monclient: authenticate success, global_id 45572 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 4 set_mon_vals no callback set 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7271ffb640 10 monclient: dump: 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: shutdown 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- 172.21.15.17:0/63924684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 msgr2=0x7f7274073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- 172.21.15.17:0/63924684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 0x7f7274073a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f727406e7b0 tx=0x7f726401c470 comp rx=0 tx=0).stop 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- 172.21.15.17:0/63924684 shutdown_connections 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- 172.21.15.17:0/63924684 >> v2:172.21.15.119:3300/0 conn(0x7f7274073f60 0x7f7274137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- 172.21.15.17:0/63924684 >> v2:172.21.15.17:3300/0 conn(0x7f7274137bd0 0x7f7274138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- 172.21.15.17:0/63924684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274073640 0x7f7274073a20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- 172.21.15.17:0/63924684 >> 172.21.15.17:0/63924684 conn(0x7f72740a66a0 msgr2=0x7f72740a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- 172.21.15.17:0/63924684 shutdown_connections 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- 172.21.15.17:0/63924684 wait complete. 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: build_initial_monmap 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: monmap: 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:24.896 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:24.896 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:24.896 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 Processor -- start 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- start start 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: init 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7274073640 0x7f72741cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): picked mon.noname-b con 0x7f7274073640 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7274073f60 0x7f7274134850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): picked mon.noname-a con 0x7f7274073f60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274137bd0 0x7f7274134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): picked mon.noname-c con 0x7f7274137bd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f727409ce50 con 0x7f7274073f60 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f727409ccd0 con 0x7f7274073640 2024-09-15T21:27:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f72740a0d00 con 0x7f7274137bd0 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7279483640 10 monclient(hunting): authenticate will time out at 1568.279487s 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7274073f60 0x7f7274134850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7274073f60 0x7f7274134850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37146/0 (socket says 172.21.15.17:37146) 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 -- 172.21.15.17:0/3494313457 learned_addr learned my addr 172.21.15.17:0/3494313457 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): get_auth_request con 0x7f7274073f60 auth_method 0 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274137bd0 0x7f7274134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): get_auth_request con 0x7f7274137bd0 auth_method 0 2024-09-15T21:27:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7272ffd640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7274073640 0x7f72741cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7272ffd640 10 monclient(hunting): get_auth_request con 0x7f7274073640 auth_method 0 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7272ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7272ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f7272ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient(hunting): handle_auth_done global_id 124125 payload 824 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274137bd0 msgr2=0x7f7274134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7274137bd0 0x7f7274134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7274073640 msgr2=0x7f72741cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7274073640 0x7f72741cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 10 monclient: _finish_hunting 0 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.878+0000 7f72727fc640 1 monclient: found mon.noname-a 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72727fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72727fc640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727409f090 con 0x7f7274073f60 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72727fc640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7274073f60 0x7f7274134850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f726800d800 tx=0x7f726800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726800b6c0 con 0x7f7274073f60 2024-09-15T21:27:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: dump: 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:27:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7268019070 con 0x7f7274073f60 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726800bae0 con 0x7f7274073f60 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 5 monclient: authenticate success, global_id 124125 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 10 monclient: _renew_subs 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741202f0 con 0x7f7274073f60 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72717fa640 4 set_mon_vals no callback set 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: dump: 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 10 monclient: _renew_subs 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727411df90 con 0x7f7274073f60 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 0x7f727411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7274137690 con 0x7f727411eaf0 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f726800bc80 con 0x7f7274073f60 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 0x7f727411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient: get_auth_request con 0x7f727411eaf0 auth_method 0 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f726003b9c0 0x7f726003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7268050a90 con 0x7f7274073f60 2024-09-15T21:27:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 10 monclient(hunting): handle_auth_done global_id 114190 payload 824 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72737fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f726003b9c0 0x7f726003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f727411fc90 con 0x7f727411eaf0 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72737fe640 10 monclient: get_auth_request con 0x7f726003b9c0 auth_method 0 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f7272ffd640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 0x7f727411c4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f727406ccf0 tx=0x7f726401acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640ba030 con 0x7f727411eaf0 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: dump: 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f72737fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f726003b9c0 0x7f726003de80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f727411ff10 tx=0x7f726c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7264009490 con 0x7f727411eaf0 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 msgr2=0x7f727411c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 0x7f727411c4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f727406ccf0 tx=0x7f726401acb0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 msgr2=0x7f727411c4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:24.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.882+0000 7f725b7fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411eaf0 0x7f727411c4e0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f727406ccf0 tx=0x7f726401acb0 comp rx=0 tx=0).stop 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7279483640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7279483640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 0x7f727411ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7279483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f727411d370 con 0x7f727411ca20 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 0x7f727411ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient: get_auth_request con 0x7f727411ca20 auth_method 0 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 10 monclient(hunting): handle_auth_done global_id 114193 payload 824 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 1 -- 172.21.15.17:0/3494313457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7274120070 con 0x7f727411ca20 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f7272ffd640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 0x7f727411ce30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f726401c2a0 tx=0x7f7264019210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640ba040 con 0x7f727411ca20 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f725b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f725b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.890+0000 7f725b7fe640 10 monclient: dump: 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f72640c0ce0 con 0x7f727411ca20 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 msgr2=0x7f727411ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 0x7f727411ce30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f726401c2a0 tx=0x7f7264019210 comp rx=0 tx=0).stop 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 msgr2=0x7f727411ce30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f725b7fe640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727411ca20 0x7f727411ce30 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f726401c2a0 tx=0x7f7264019210 comp rx=0 tx=0).stop 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 94, 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 53459, 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:27:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:24.925 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f726003b9c0 msgr2=0x7f726003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f726003b9c0 0x7f726003de80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f727411ff10 tx=0x7f726c00b040 comp rx=0 tx=0).stop 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 10 monclient: shutdown 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7274073f60 msgr2=0x7f7274134850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 --2- 172.21.15.17:0/3494313457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7274073f60 0x7f7274134850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f726800d800 tx=0x7f726800dc00 comp rx=0 tx=0).stop 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f72737fe640 1 -- 172.21.15.17:0/3494313457 reap_dead start 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f72737fe640 1 -- 172.21.15.17:0/3494313457 reap_dead start 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 shutdown_connections 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 >> 172.21.15.17:0/3494313457 conn(0x7f72740a66a0 msgr2=0x7f72740a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 shutdown_connections 2024-09-15T21:27:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:24.894+0000 7f7279483640 1 -- 172.21.15.17:0/3494313457 wait complete. 2024-09-15T21:27:24.928 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:27:24.928 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:27:24.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:27:25.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 10 monclient: get_monmap_and_config 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 10 monclient: build_initial_monmap 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 10 monclient: monmap: 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:25.052704+0000 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:25.052704+0000 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:25.054 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:25.054 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 1 Processor -- start 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 1 -- start start 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 10 monclient: init 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.046+0000 7f5e5d3b1640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e58073640 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 0x7f5e5813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): picked mon.noname-b con 0x7f5e58074130 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:25.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e5813d140 0x7f5e5813f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): picked mon.noname-c con 0x7f5e5813d140 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e580a0890 con 0x7f5e58073640 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e580a3640 con 0x7f5e5813d140 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e580746e0 con 0x7f5e58074130 2024-09-15T21:27:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient(hunting): authenticate will time out at 1568.448099s 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 0x7f5e5813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 0x7f5e5813cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49550/0 (socket says 172.21.15.17:49550) 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 -- 172.21.15.17:0/1293423834 learned_addr learned my addr 172.21.15.17:0/1293423834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): get_auth_request con 0x7f5e58074130 auth_method 0 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 1 --2- 172.21.15.17:0/1293423834 >> v2:172.21.15.119:3300/0 conn(0x7f5e5813d140 0x7f5e5813f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): get_auth_request con 0x7f5e5813d140 auth_method 0 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:25.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient(hunting): handle_auth_done global_id 45578 payload 274 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 -- 172.21.15.17:0/1293423834 >> v2:172.21.15.119:3300/0 conn(0x7f5e5813d140 msgr2=0x7f5e5813f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 --2- 172.21.15.17:0/1293423834 >> v2:172.21.15.119:3300/0 conn(0x7f5e5813d140 0x7f5e5813f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 -- 172.21.15.17:0/1293423834 >> v1:172.21.15.119:6789/0 conn(0x7f5e58073640 legacy=0x7f5e58073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient: _finish_hunting 0 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 monclient: found mon.noname-b 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 -- 172.21.15.17:0/1293423834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e580a1640 con 0x7f5e58074130 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e567fc640 1 --2- 172.21.15.17:0/1293423834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 0x7f5e5813cc00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009870 tx=0x7f5e4c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc299527ab5d710 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 1 -- 172.21.15.17:0/1293423834 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0ab5f0 con 0x7f5e58074130 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: dump: 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: _finish_auth 0 2024-09-15T21:27:25.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 5 monclient: authenticate success, global_id 45578 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 1 -- 172.21.15.17:0/1293423834 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4c0b0070 con 0x7f5e58074130 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 1 -- 172.21.15.17:0/1293423834 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0ab9b0 con 0x7f5e58074130 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e5d3b1640 4 set_mon_vals no callback set 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.050+0000 7f5e55ffb640 10 monclient: dump: 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:25.064 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: shutdown 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/1293423834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 msgr2=0x7f5e5813cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/1293423834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 0x7f5e5813cc00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009870 tx=0x7f5e4c01c3f0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/1293423834 shutdown_connections 2024-09-15T21:27:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/1293423834 >> v2:172.21.15.119:3300/0 conn(0x7f5e5813d140 0x7f5e5813f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/1293423834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58074130 0x7f5e5813cc00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/1293423834 >> 172.21.15.17:0/1293423834 conn(0x7f5e580a66a0 msgr2=0x7f5e580a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/1293423834 shutdown_connections 2024-09-15T21:27:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/1293423834 wait complete. 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: build_initial_monmap 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: monmap: 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:25.070 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 Processor -- start 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- start start 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: init 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58073640 0x7f5e58130a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): picked mon.noname-c con 0x7f5e58073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:25.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 0x7f5e58130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): picked mon.noname-a con 0x7f5e58074130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5813d140 0x7f5e58134680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): picked mon.noname-b con 0x7f5e5813d140 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e580a0e80 con 0x7f5e58074130 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e580a0d00 con 0x7f5e5813d140 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e58142290 con 0x7f5e58073640 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient(hunting): authenticate will time out at 1568.454310s 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 0x7f5e58130fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:25.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58073640 0x7f5e58130a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 0x7f5e58130fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37156/0 (socket says 172.21.15.17:37156) 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 -- 172.21.15.17:0/3254648444 learned_addr learned my addr 172.21.15.17:0/3254648444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): get_auth_request con 0x7f5e58074130 auth_method 0 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): get_auth_request con 0x7f5e58073640 auth_method 0 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:25.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e577fe640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5813d140 0x7f5e58134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient(hunting): handle_auth_done global_id 124128 payload 824 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 -- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58073640 msgr2=0x7f5e58130a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58073640 0x7f5e58130a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 -- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5813d140 msgr2=0x7f5e58134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5813d140 0x7f5e58134680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient: _finish_hunting 0 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 monclient: found mon.noname-a 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:25.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 -- 172.21.15.17:0/3254648444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e580a41b0 con 0x7f5e58074130 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e577fe640 10 monclient: get_auth_request con 0x7f5e5813d140 auth_method 0 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e577fe640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5813d140 0x7f5e58134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e567fc640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 0x7f5e58130fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0ad040 tx=0x7f5e4c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0be3f0 con 0x7f5e58074130 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: dump: 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:25.075 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4c0be5b0 con 0x7f5e58074130 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0be9a0 con 0x7f5e58074130 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:25.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e3f7fe640 10 monclient: dump: 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 5 monclient: authenticate success, global_id 124128 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e557fa640 4 set_mon_vals no callback set 2024-09-15T21:27:25.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: _renew_subs 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5809d0f0 con 0x7f5e58074130 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: _renew_subs 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.054+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e58136d60 con 0x7f5e58074130 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e5d3b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e5d3b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e5d3b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e4c0c7440 con 0x7f5e58074130 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e58073a20 con 0x7f5e58074130 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e3f7fe640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e4403b9c0 0x7f5e4403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5e4c0eadd0 con 0x7f5e58074130 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e56ffd640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e4403b9c0 0x7f5e4403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:25.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e56ffd640 10 monclient: get_auth_request con 0x7f5e4403b9c0 auth_method 0 2024-09-15T21:27:25.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.058+0000 7f5e56ffd640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e4403b9c0 0x7f5e4403de80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007fd0 tx=0x7f5e48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:25.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.062+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5e4c016130 con 0x7f5e58074130 2024-09-15T21:27:25.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.062+0000 7f5e3f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:25.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.062+0000 7f5e3f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:25.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e5d3b1640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:27:25.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e5d3b1640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:27:25.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e5d3b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:25.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f5e58138790 con 0x7f5e58074130 2024-09-15T21:27:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e3f7fe640 1 -- 172.21.15.17:0/3254648444 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f5e4c0bdc90 con 0x7f5e58074130 2024-09-15T21:27:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e3f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:27:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.294+0000 7f5e3f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:25.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.294+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.294+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=4169748611,config=3541143443,health=765734218,kv=3795840987,logm=1417791978}) 2024-09-15T21:27:25.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.294+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=11,logm=54} crc {auth=3651887210,config=3541143443,health=765734218,kv=1736545667,logm=3315183480}) 2024-09-15T21:27:25.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.294+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.294+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=4169748611,config=3541143443,health=765734218,kv=3795840987,logm=1417791978}) 2024-09-15T21:27:25.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.294+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=12,kv=11,logm=52} crc {auth=3325651776,config=3541143443,health=765734218,kv=2435832620,logm=833813208}) 2024-09-15T21:27:25.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e4403b9c0 msgr2=0x7f5e4403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:25.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e4403b9c0 0x7f5e4403de80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007fd0 tx=0x7f5e48009040 comp rx=0 tx=0).stop 2024-09-15T21:27:25.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 10 monclient: shutdown 2024-09-15T21:27:25.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 msgr2=0x7f5e58130fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 0x7f5e58130fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0ad040 tx=0x7f5e4c01ac30 comp rx=0 tx=0).stop 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 shutdown_connections 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e4403b9c0 0x7f5e4403de80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5813d140 0x7f5e58134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e58074130 0x7f5e58130fb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 --2- 172.21.15.17:0/3254648444 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e58073640 0x7f5e58130a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 >> 172.21.15.17:0/3254648444 conn(0x7f5e580a66a0 msgr2=0x7f5e58131840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 shutdown_connections 2024-09-15T21:27:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:25.298+0000 7f5e5d3b1640 1 -- 172.21.15.17:0/3254648444 wait complete. 2024-09-15T21:27:25.306 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.298+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.306 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.298+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1459654319}) 2024-09-15T21:27:25.306 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.298+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3774823962}) 2024-09-15T21:27:25.306 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.298+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.306 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.298+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1459654319}) 2024-09-15T21:27:25.306 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.298+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3367506054}) 2024-09-15T21:27:25.308 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.302+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.302+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3095253654}) 2024-09-15T21:27:25.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.302+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540713643}) 2024-09-15T21:27:25.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.302+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.302+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3095253654}) 2024-09-15T21:27:25.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.302+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1454276493}) 2024-09-15T21:27:25.310 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2238673760}) 2024-09-15T21:27:25.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1317602155}) 2024-09-15T21:27:25.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2238673760}) 2024-09-15T21:27:25.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2193309840}) 2024-09-15T21:27:25.312 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1983677439}) 2024-09-15T21:27:25.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2497439868}) 2024-09-15T21:27:25.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1983677439}) 2024-09-15T21:27:25.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.306+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=945935206}) 2024-09-15T21:27:25.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613796257}) 2024-09-15T21:27:25.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3015047381}) 2024-09-15T21:27:25.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613796257}) 2024-09-15T21:27:25.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=243489726}) 2024-09-15T21:27:25.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1205241205}) 2024-09-15T21:27:25.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1190305076}) 2024-09-15T21:27:25.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1205241205}) 2024-09-15T21:27:25.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.310+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3114486440}) 2024-09-15T21:27:25.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=971405097}) 2024-09-15T21:27:25.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=412052143}) 2024-09-15T21:27:25.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=971405097}) 2024-09-15T21:27:25.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2282080447}) 2024-09-15T21:27:25.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308616042}) 2024-09-15T21:27:25.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172919661}) 2024-09-15T21:27:25.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308616042}) 2024-09-15T21:27:25.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.314+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2784213566}) 2024-09-15T21:27:25.322 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:27:25.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=265448664}) 2024-09-15T21:27:25.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1765020806}) 2024-09-15T21:27:25.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=265448664}) 2024-09-15T21:27:25.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3303815988}) 2024-09-15T21:27:25.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.324 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3644892278}) 2024-09-15T21:27:25.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045012273}) 2024-09-15T21:27:25.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3644892278}) 2024-09-15T21:27:25.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.318+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=186117285}) 2024-09-15T21:27:25.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698187483}) 2024-09-15T21:27:25.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=620559936}) 2024-09-15T21:27:25.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698187483}) 2024-09-15T21:27:25.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2730275612}) 2024-09-15T21:27:25.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2091648906}) 2024-09-15T21:27:25.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2410125936}) 2024-09-15T21:27:25.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2091648906}) 2024-09-15T21:27:25.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1649632359}) 2024-09-15T21:27:25.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3264251775}) 2024-09-15T21:27:25.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3334413572}) 2024-09-15T21:27:25.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3264251775}) 2024-09-15T21:27:25.330 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.322+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3866885072}) 2024-09-15T21:27:25.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971526930}) 2024-09-15T21:27:25.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=985514788}) 2024-09-15T21:27:25.332 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.332 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971526930}) 2024-09-15T21:27:25.332 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2039303458}) 2024-09-15T21:27:25.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363044405}) 2024-09-15T21:27:25.334 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2013048119}) 2024-09-15T21:27:25.334 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363044405}) 2024-09-15T21:27:25.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.326+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2511810512}) 2024-09-15T21:27:25.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=552859005}) 2024-09-15T21:27:25.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3332072585}) 2024-09-15T21:27:25.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=552859005}) 2024-09-15T21:27:25.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1088414547}) 2024-09-15T21:27:25.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2195774416}) 2024-09-15T21:27:25.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=747299445}) 2024-09-15T21:27:25.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2195774416}) 2024-09-15T21:27:25.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.330+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=867539199}) 2024-09-15T21:27:25.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3637286145}) 2024-09-15T21:27:25.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3171211588}) 2024-09-15T21:27:25.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3637286145}) 2024-09-15T21:27:25.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2829330778}) 2024-09-15T21:27:25.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2732092499}) 2024-09-15T21:27:25.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2250293240}) 2024-09-15T21:27:25.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2732092499}) 2024-09-15T21:27:25.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.334+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3712117114}) 2024-09-15T21:27:25.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854389742}) 2024-09-15T21:27:25.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1023506597}) 2024-09-15T21:27:25.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854389742}) 2024-09-15T21:27:25.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=277909504}) 2024-09-15T21:27:25.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2297955245}) 2024-09-15T21:27:25.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2353916924}) 2024-09-15T21:27:25.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2297955245}) 2024-09-15T21:27:25.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.338+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3364991501}) 2024-09-15T21:27:25.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=546986259}) 2024-09-15T21:27:25.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737341200}) 2024-09-15T21:27:25.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=546986259}) 2024-09-15T21:27:25.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2499713144}) 2024-09-15T21:27:25.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1819700140}) 2024-09-15T21:27:25.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3224169891}) 2024-09-15T21:27:25.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1819700140}) 2024-09-15T21:27:25.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.342+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1963664415}) 2024-09-15T21:27:25.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2116535269}) 2024-09-15T21:27:25.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1345465748}) 2024-09-15T21:27:25.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2116535269}) 2024-09-15T21:27:25.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1928398148}) 2024-09-15T21:27:25.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2951881762}) 2024-09-15T21:27:25.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1882544790}) 2024-09-15T21:27:25.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2951881762}) 2024-09-15T21:27:25.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.346+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=368233032}) 2024-09-15T21:27:25.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=732192864}) 2024-09-15T21:27:25.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=876754575}) 2024-09-15T21:27:25.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=732192864}) 2024-09-15T21:27:25.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1444652828}) 2024-09-15T21:27:25.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033864660}) 2024-09-15T21:27:25.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2509523802}) 2024-09-15T21:27:25.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033864660}) 2024-09-15T21:27:25.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=795812651}) 2024-09-15T21:27:25.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3141285725}) 2024-09-15T21:27:25.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154282697}) 2024-09-15T21:27:25.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3141285725}) 2024-09-15T21:27:25.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.350+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=227337455}) 2024-09-15T21:27:25.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448853808}) 2024-09-15T21:27:25.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3208373067}) 2024-09-15T21:27:25.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448853808}) 2024-09-15T21:27:25.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1016772444}) 2024-09-15T21:27:25.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3799469400}) 2024-09-15T21:27:25.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=679043358}) 2024-09-15T21:27:25.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3799469400}) 2024-09-15T21:27:25.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.354+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2732037116}) 2024-09-15T21:27:25.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882779334}) 2024-09-15T21:27:25.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266677234}) 2024-09-15T21:27:25.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882779334}) 2024-09-15T21:27:25.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3374576379}) 2024-09-15T21:27:25.364 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2349447630}) 2024-09-15T21:27:25.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=126027876}) 2024-09-15T21:27:25.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2349447630}) 2024-09-15T21:27:25.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.358+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2159688536}) 2024-09-15T21:27:25.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421953341}) 2024-09-15T21:27:25.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2431188748}) 2024-09-15T21:27:25.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421953341}) 2024-09-15T21:27:25.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3649328369}) 2024-09-15T21:27:25.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595349559}) 2024-09-15T21:27:25.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3185110743}) 2024-09-15T21:27:25.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3595349559}) 2024-09-15T21:27:25.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.362+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558507267}) 2024-09-15T21:27:25.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3450741082}) 2024-09-15T21:27:25.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050826328}) 2024-09-15T21:27:25.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3450741082}) 2024-09-15T21:27:25.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1287233313}) 2024-09-15T21:27:25.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2354699022}) 2024-09-15T21:27:25.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3156542779}) 2024-09-15T21:27:25.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2354699022}) 2024-09-15T21:27:25.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.366+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=764996954}) 2024-09-15T21:27:25.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2632613753}) 2024-09-15T21:27:25.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3934873105}) 2024-09-15T21:27:25.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2632613753}) 2024-09-15T21:27:25.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2790074287}) 2024-09-15T21:27:25.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046715074}) 2024-09-15T21:27:25.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4221832307}) 2024-09-15T21:27:25.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046715074}) 2024-09-15T21:27:25.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3658869141}) 2024-09-15T21:27:25.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1207837520}) 2024-09-15T21:27:25.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=30795935}) 2024-09-15T21:27:25.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1207837520}) 2024-09-15T21:27:25.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.370+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3319550986}) 2024-09-15T21:27:25.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619892480}) 2024-09-15T21:27:25.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=599479979}) 2024-09-15T21:27:25.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619892480}) 2024-09-15T21:27:25.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2154467590}) 2024-09-15T21:27:25.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255865088}) 2024-09-15T21:27:25.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=215283646}) 2024-09-15T21:27:25.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255865088}) 2024-09-15T21:27:25.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.374+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3568402189}) 2024-09-15T21:27:25.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985752601}) 2024-09-15T21:27:25.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=514684232}) 2024-09-15T21:27:25.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985752601}) 2024-09-15T21:27:25.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4100665540}) 2024-09-15T21:27:25.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437171270}) 2024-09-15T21:27:25.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1397783715}) 2024-09-15T21:27:25.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437171270}) 2024-09-15T21:27:25.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.378+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3428393768}) 2024-09-15T21:27:25.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3363296384}) 2024-09-15T21:27:25.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=798647008}) 2024-09-15T21:27:25.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3363296384}) 2024-09-15T21:27:25.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3571123656}) 2024-09-15T21:27:25.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3812979679}) 2024-09-15T21:27:25.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48393606}) 2024-09-15T21:27:25.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3812979679}) 2024-09-15T21:27:25.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1916314968}) 2024-09-15T21:27:25.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985791871}) 2024-09-15T21:27:25.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2752195684}) 2024-09-15T21:27:25.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985791871}) 2024-09-15T21:27:25.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.382+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2962487612}) 2024-09-15T21:27:25.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960734397}) 2024-09-15T21:27:25.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3584678684}) 2024-09-15T21:27:25.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960734397}) 2024-09-15T21:27:25.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652422057}) 2024-09-15T21:27:25.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1115022221}) 2024-09-15T21:27:25.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=175522337}) 2024-09-15T21:27:25.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1115022221}) 2024-09-15T21:27:25.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.386+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3107472565}) 2024-09-15T21:27:25.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1174196627}) 2024-09-15T21:27:25.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=34107309}) 2024-09-15T21:27:25.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1174196627}) 2024-09-15T21:27:25.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635582266}) 2024-09-15T21:27:25.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=228851545}) 2024-09-15T21:27:25.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3103571437}) 2024-09-15T21:27:25.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=228851545}) 2024-09-15T21:27:25.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.390+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=761967173}) 2024-09-15T21:27:25.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=882894226}) 2024-09-15T21:27:25.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212365641}) 2024-09-15T21:27:25.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=882894226}) 2024-09-15T21:27:25.400 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2842812911}) 2024-09-15T21:27:25.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3886149876}) 2024-09-15T21:27:25.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1223393142}) 2024-09-15T21:27:25.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3886149876}) 2024-09-15T21:27:25.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.394+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2303398458}) 2024-09-15T21:27:25.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3817479681}) 2024-09-15T21:27:25.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3635742589}) 2024-09-15T21:27:25.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3817479681}) 2024-09-15T21:27:25.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2708376237}) 2024-09-15T21:27:25.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=278250113}) 2024-09-15T21:27:25.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3471318105}) 2024-09-15T21:27:25.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=278250113}) 2024-09-15T21:27:25.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.398+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=815733371}) 2024-09-15T21:27:25.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2887244153}) 2024-09-15T21:27:25.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2909775037}) 2024-09-15T21:27:25.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2887244153}) 2024-09-15T21:27:25.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2191176896}) 2024-09-15T21:27:25.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1210837512}) 2024-09-15T21:27:25.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2020121095}) 2024-09-15T21:27:25.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1210837512}) 2024-09-15T21:27:25.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.402+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=64690679}) 2024-09-15T21:27:25.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.406+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.406+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=21,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=29} crc {logm=3350474710,mdsmap=3009587245,mgr=205827678,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=847205798,mon_config_key=3131655263,monmap=186996059,nvmeofgw=1596038039,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3819113854}) 2024-09-15T21:27:25.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.406+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=25,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=23} crc {logm=3020721999,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2067998849,mon_config_key=3562240218,monmap=2617021892,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1185457765}) 2024-09-15T21:27:25.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.406+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.406+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=21,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=29} crc {logm=3350474710,mdsmap=3009587245,mgr=205827678,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=847205798,mon_config_key=3131655263,monmap=186996059,nvmeofgw=1596038039,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3819113854}) 2024-09-15T21:27:25.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.406+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=25} crc {logm=2330862458,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=200440474,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=3068027012,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1665231473}) 2024-09-15T21:27:25.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.410+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.410+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3326909887}) 2024-09-15T21:27:25.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.410+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1140134668}) 2024-09-15T21:27:25.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.410+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.410+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3326909887}) 2024-09-15T21:27:25.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.410+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=107534553}) 2024-09-15T21:27:25.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.414+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.414+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1311662305}) 2024-09-15T21:27:25.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.414+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1308152811}) 2024-09-15T21:27:25.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.414+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.414+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1311662305}) 2024-09-15T21:27:25.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.414+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1726763112}) 2024-09-15T21:27:25.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.426+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.426+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2173037253}) 2024-09-15T21:27:25.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.426+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=987333994}) 2024-09-15T21:27:25.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.426+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.426+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2173037253}) 2024-09-15T21:27:25.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.426+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3520834323}) 2024-09-15T21:27:25.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.454+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.454+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=65} crc {osdmap=3400824642}) 2024-09-15T21:27:25.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.454+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=61} crc {osdmap=824258024}) 2024-09-15T21:27:25.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.454+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:27:25.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.454+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=65} crc {osdmap=3400824642}) 2024-09-15T21:27:25.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:27:25.454+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=63} crc {osdmap=3538079094}) 2024-09-15T21:27:26.323 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:27:26.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:26.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient: get_monmap_and_config 2024-09-15T21:27:26.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient: build_initial_monmap 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient: monmap: 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:26.455739+0000 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:26.455739+0000 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:26.456 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:26.457 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 Processor -- start 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 -- start start 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient: init 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd40c1340c0 0x7fd40c1344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): picked mon.noname-a con 0x7fd40c1340c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd40c09d6c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd40c0a0fa0 0x7fd40c13be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): picked mon.noname-c con 0x7fd40c0a0fa0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40c1349e0 con 0x7fd40c09d6c0 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd40c09d520 con 0x7fd40c1340c0 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd40c09e1b0 con 0x7fd40c0a0fa0 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd413b76640 10 monclient(hunting): authenticate will time out at 1569.851579s 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411111640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd40c09d6c0 0x7fd40c0a0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39930/0 (socket says 172.21.15.17:39930) 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411111640 1 -- 172.21.15.17:0/2710410067 learned_addr learned my addr 172.21.15.17:0/2710410067 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 1 --2- 172.21.15.17:0/2710410067 >> v2:172.21.15.17:3300/0 conn(0x7fd40c1340c0 0x7fd40c1344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 10 monclient(hunting): get_auth_request con 0x7fd40c1340c0 auth_method 0 2024-09-15T21:27:26.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.450+0000 7fd411912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd412113640 1 --2- 172.21.15.17:0/2710410067 >> v2:172.21.15.119:3300/0 conn(0x7fd40c0a0fa0 0x7fd40c13be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd412113640 10 monclient(hunting): get_auth_request con 0x7fd40c0a0fa0 auth_method 0 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd412113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd412113640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd412113640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 1 -- 172.21.15.17:0/2710410067 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2539766041 0 0) 0x7fd40c1349e0 con 0x7fd40c09d6c0 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient(hunting): _negotiate 2024-09-15T21:27:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient(hunting): my global_id is 124134 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 1 -- 172.21.15.17:0/2710410067 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f8005740 con 0x7fd40c09d6c0 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 10 monclient(hunting): handle_auth_done global_id 124131 payload 274 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 1 -- 172.21.15.17:0/2710410067 >> v2:172.21.15.119:3300/0 conn(0x7fd40c0a0fa0 msgr2=0x7fd40c13be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 1 --2- 172.21.15.17:0/2710410067 >> v2:172.21.15.119:3300/0 conn(0x7fd40c0a0fa0 0x7fd40c13be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 1 -- 172.21.15.17:0/2710410067 >> v1:172.21.15.17:6789/0 conn(0x7fd40c09d6c0 legacy=0x7fd40c0a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 10 monclient: _finish_hunting 0 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 1 monclient: found mon.noname-a 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:26.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 1 -- 172.21.15.17:0/2710410067 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c09e490 con 0x7fd40c1340c0 2024-09-15T21:27:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd411912640 1 --2- 172.21.15.17:0/2710410067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c1344a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0a1fb0 tx=0x7fd40801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8d1a318b344817 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 1 -- 172.21.15.17:0/2710410067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b0070 con 0x7fd40c1340c0 2024-09-15T21:27:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: dump: 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: _finish_auth 0 2024-09-15T21:27:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: _check_auth_tickets 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 5 monclient: authenticate success, global_id 124131 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 1 -- 172.21.15.17:0/2710410067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4080ab600 con 0x7fd40c1340c0 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 4 set_mon_vals no callback set 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 1 -- 172.21.15.17:0/2710410067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080a4e60 con 0x7fd40c1340c0 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd410910640 10 monclient: dump: 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 10 monclient: shutdown 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 -- 172.21.15.17:0/2710410067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 msgr2=0x7fd40c1344a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 --2- 172.21.15.17:0/2710410067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c1344a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0a1fb0 tx=0x7fd40801c340 comp rx=0 tx=0).stop 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 -- 172.21.15.17:0/2710410067 shutdown_connections 2024-09-15T21:27:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 --2- 172.21.15.17:0/2710410067 >> v2:172.21.15.119:3300/0 conn(0x7fd40c0a0fa0 0x7fd40c13be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 --2- 172.21.15.17:0/2710410067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c1344a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 -- 172.21.15.17:0/2710410067 >> 172.21.15.17:0/2710410067 conn(0x7fd40c0a66a0 msgr2=0x7fd40c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 -- 172.21.15.17:0/2710410067 shutdown_connections 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 1 -- 172.21.15.17:0/2710410067 wait complete. 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 10 monclient: build_initial_monmap 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.454+0000 7fd413b76640 10 monclient: monmap: 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:26.471 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 Processor -- start 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- start start 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: init 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd40c09d6c0 0x7fd40c12da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): picked mon.noname-c con 0x7fd40c09d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd40c0a0fa0 0x7fd40c12dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): picked mon.noname-b con 0x7fd40c0a0fa0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:26.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): picked mon.noname-a con 0x7fd40c1340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c13e540 con 0x7fd40c1340c0 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c13e3c0 con 0x7fd40c0a0fa0 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c13e6c0 con 0x7fd40c09d6c0 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient(hunting): authenticate will time out at 1569.856608s 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd40c09d6c0 0x7fd40c12da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c138ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37174/0 (socket says 172.21.15.17:37174) 2024-09-15T21:27:26.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): get_auth_request con 0x7fd40c09d6c0 auth_method 0 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 1 -- 172.21.15.17:0/3842455723 learned_addr learned my addr 172.21.15.17:0/3842455723 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): get_auth_request con 0x7fd40c1340c0 auth_method 0 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411111640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd40c0a0fa0 0x7fd40c12dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient(hunting): handle_auth_done global_id 45584 payload 824 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 -- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd40c0a0fa0 msgr2=0x7fd40c12dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd40c0a0fa0 0x7fd40c12dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 -- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 msgr2=0x7fd40c138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient: _finish_hunting 0 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 monclient: found mon.noname-c 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 -- 172.21.15.17:0/3842455723 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c073cc0 con 0x7fd40c09d6c0 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411111640 10 monclient: get_auth_request con 0x7fd40c0a0fa0 auth_method 0 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411111640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd40c0a0fa0 0x7fd40c12dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd412113640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411912640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd40c09d6c0 0x7fd40c12da60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd4080ab000 tx=0x7fd4080190e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b8070 con 0x7fd40c09d6c0 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: dump: 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.476 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: _finish_auth 0 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: _check_auth_tickets 2024-09-15T21:27:26.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 5 monclient: authenticate success, global_id 45584 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _renew_subs 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4080a4e60 con 0x7fd40c09d6c0 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c09d0f0 con 0x7fd40c09d6c0 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b0040 con 0x7fd40c09d6c0 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd403fff640 4 set_mon_vals no callback set 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 10 monclient: dump: 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _renew_subs 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c13b500 con 0x7fd40c09d6c0 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd4080bb070 con 0x7fd40c09d6c0 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c1d0040 con 0x7fd40c09d6c0 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd3f003b9c0 0x7fd3f003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd4080ea270 con 0x7fd40c09d6c0 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411111640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd3f003b9c0 0x7fd3f003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411111640 10 monclient: get_auth_request con 0x7fd3f003b9c0 auth_method 0 2024-09-15T21:27:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.458+0000 7fd411111640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd3f003b9c0 0x7fd3f003de80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc001230 tx=0x7fd3fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.462+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd408016050 con 0x7fd40c09d6c0 2024-09-15T21:27:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.462+0000 7fd4027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.462+0000 7fd4027fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:26.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd413b76640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:26.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd413b76640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:26.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd413b76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd40c1344a0 con 0x7fd40c09d6c0 2024-09-15T21:27:26.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd4027fc640 1 -- 172.21.15.17:0/3842455723 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fd4080b4b50 con 0x7fd40c09d6c0 2024-09-15T21:27:26.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd4027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:26.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.690+0000 7fd4027fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:26.697 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd3f003b9c0 msgr2=0x7fd3f003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd3f003b9c0 0x7fd3f003de80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc001230 tx=0x7fd3fc009040 comp rx=0 tx=0).stop 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 10 monclient: shutdown 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd40c09d6c0 msgr2=0x7fd40c12da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd40c09d6c0 0x7fd40c12da60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd4080ab000 tx=0x7fd4080190e0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 shutdown_connections 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd3f003b9c0 0x7fd3f003de80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40c1340c0 0x7fd40c138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd40c0a0fa0 0x7fd40c12dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 --2- 172.21.15.17:0/3842455723 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd40c09d6c0 0x7fd40c12da60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 >> 172.21.15.17:0/3842455723 conn(0x7fd40c0a66a0 msgr2=0x7fd40c132200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 shutdown_connections 2024-09-15T21:27:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.694+0000 7fd413b76640 1 -- 172.21.15.17:0/3842455723 wait complete. 2024-09-15T21:27:26.713 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:27:26.713 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:27:26.713 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:27:26.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient: get_monmap_and_config 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient: build_initial_monmap 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient: monmap: 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:26.823121+0000 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:26.823121+0000 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.823 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:26.824 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 Processor -- start 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 -- start start 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient: init 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8ac0a0890 0x7fd8ac0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): picked mon.noname-a con 0x7fd8ac0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac09ffc0 0x7fd8ac0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): picked mon.noname-b con 0x7fd8ac09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd8ac135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ac0a3130 con 0x7fd8ac135670 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ac0a16f0 con 0x7fd8ac0a0890 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac073df0 con 0x7fd8ac09ffc0 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b35cc640 10 monclient(hunting): authenticate will time out at 1570.218606s 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8ac0a0890 0x7fd8ac0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8ac0a0890 0x7fd8ac0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37182/0 (socket says 172.21.15.17:37182) 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 1 -- 172.21.15.17:0/1046754935 learned_addr learned my addr 172.21.15.17:0/1046754935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac09ffc0 0x7fd8ac0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): get_auth_request con 0x7fd8ac0a0890 auth_method 0 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): get_auth_request con 0x7fd8ac09ffc0 auth_method 0 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient(hunting): handle_auth_done global_id 45587 payload 274 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 1 -- 172.21.15.17:0/1046754935 >> v2:172.21.15.17:3300/0 conn(0x7fd8ac0a0890 msgr2=0x7fd8ac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:26.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 1 --2- 172.21.15.17:0/1046754935 >> v2:172.21.15.17:3300/0 conn(0x7fd8ac0a0890 0x7fd8ac0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 1 -- 172.21.15.17:0/1046754935 >> v1:172.21.15.119:6789/0 conn(0x7fd8ac135670 legacy=0x7fd8ac135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient: _finish_hunting 0 2024-09-15T21:27:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 1 monclient: found mon.noname-b 2024-09-15T21:27:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.818+0000 7fd8b0b67640 1 -- 172.21.15.17:0/1046754935 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac0a1bf0 con 0x7fd8ac09ffc0 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b0b67640 1 --2- 172.21.15.17:0/1046754935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac09ffc0 0x7fd8ac0a11b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0a1870 tx=0x7fd8a001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6a30dd2bb43ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 1 -- 172.21.15.17:0/1046754935 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00b0070 con 0x7fd8ac09ffc0 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: dump: 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: _finish_auth 0 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 1 -- 172.21.15.17:0/1046754935 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a00ab600 con 0x7fd8ac09ffc0 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 5 monclient: authenticate success, global_id 45587 2024-09-15T21:27:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 1 -- 172.21.15.17:0/1046754935 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00ab9d0 con 0x7fd8ac09ffc0 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd89ffff640 10 monclient: dump: 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.842 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 4 set_mon_vals no callback set 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: shutdown 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- 172.21.15.17:0/1046754935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac09ffc0 msgr2=0x7fd8ac0a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/1046754935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac09ffc0 0x7fd8ac0a11b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0a1870 tx=0x7fd8a001c360 comp rx=0 tx=0).stop 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- 172.21.15.17:0/1046754935 shutdown_connections 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/1046754935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac09ffc0 0x7fd8ac0a11b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/1046754935 >> v2:172.21.15.17:3300/0 conn(0x7fd8ac0a0890 0x7fd8ac0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- 172.21.15.17:0/1046754935 >> 172.21.15.17:0/1046754935 conn(0x7fd8ac0a66a0 msgr2=0x7fd8ac0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- 172.21.15.17:0/1046754935 shutdown_connections 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- 172.21.15.17:0/1046754935 wait complete. 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: build_initial_monmap 2024-09-15T21:27:26.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: monmap: 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 Processor -- start 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- start start 2024-09-15T21:27:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: init 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac09ffc0 0x7fd8ac11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): picked mon.noname-a con 0x7fd8ac09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0a0890 0x7fd8ac11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): picked mon.noname-b con 0x7fd8ac0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac135670 0x7fd8ac11cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): picked mon.noname-c con 0x7fd8ac135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac140690 con 0x7fd8ac09ffc0 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac140510 con 0x7fd8ac0a0890 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac140810 con 0x7fd8ac135670 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b35cc640 10 monclient(hunting): authenticate will time out at 1570.223822s 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac09ffc0 0x7fd8ac11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac09ffc0 0x7fd8ac11c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37184/0 (socket says 172.21.15.17:37184) 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 1 -- 172.21.15.17:0/2977022506 learned_addr learned my addr 172.21.15.17:0/2977022506 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 10 monclient(hunting): get_auth_request con 0x7fd8ac09ffc0 auth_method 0 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1368640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.822+0000 7fd8b1b69640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac135670 0x7fd8ac11cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): get_auth_request con 0x7fd8ac135670 auth_method 0 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b0b67640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0a0890 0x7fd8ac11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient(hunting): handle_auth_done global_id 45590 payload 824 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0a0890 msgr2=0x7fd8ac11ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0a0890 0x7fd8ac11ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac09ffc0 msgr2=0x7fd8ac11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac09ffc0 0x7fd8ac11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient: _finish_hunting 0 2024-09-15T21:27:26.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 monclient: found mon.noname-c 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac13d150 con 0x7fd8ac135670 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b0b67640 10 monclient: get_auth_request con 0x7fd8ac0a0890 auth_method 0 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b0b67640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0a0890 0x7fd8ac11ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac09ffc0 0x7fd8ac11c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1b69640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac135670 0x7fd8ac11cf60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c780 tx=0x7fd8a400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4016020 con 0x7fd8ac135670 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: dump: 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.848 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: _finish_auth 0 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 5 monclient: authenticate success, global_id 45590 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 10 monclient: _renew_subs 2024-09-15T21:27:26.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a400a9f0 con 0x7fd8ac135670 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac11eef0 con 0x7fd8ac135670 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400ae20 con 0x7fd8ac135670 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: dump: 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89f7fe640 4 set_mon_vals no callback set 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 10 monclient: _renew_subs 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac0a5830 con 0x7fd8ac135670 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 0x7fd8ac02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac135aa0 con 0x7fd8ac1344b0 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 0x7fd8ac02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient: get_auth_request con 0x7fd8ac1344b0 auth_method 0 2024-09-15T21:27:26.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd8a4019020 con 0x7fd8ac135670 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 10 monclient(hunting): handle_auth_done global_id 124146 payload 824 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8ac1391c0 con 0x7fd8ac1344b0 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd89003b9c0 0x7fd89003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd8a4050500 con 0x7fd8ac135670 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b1368640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 0x7fd8ac02d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009ef0 tx=0x7fd8a800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b0b67640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd89003b9c0 0x7fd89003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b0b67640 10 monclient: get_auth_request con 0x7fd89003b9c0 auth_method 0 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800fe50 con 0x7fd8ac1344b0 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: dump: 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd8a800fe50 con 0x7fd8ac1344b0 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 msgr2=0x7fd8ac02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 0x7fd8ac02d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009ef0 tx=0x7fd8a800f610 comp rx=0 tx=0).stop 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 msgr2=0x7fd8ac02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd89dffb640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac1344b0 0x7fd8ac02d450 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009ef0 tx=0x7fd8a800f610 comp rx=0 tx=0).stop 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.826+0000 7fd8b0b67640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd89003b9c0 0x7fd89003de80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00ad000 tx=0x7fd8a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 0x7fd8ac02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac06d020 con 0x7fd8ac134890 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 0x7fd8ac02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient: get_auth_request con 0x7fd8ac134890 auth_method 0 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 10 monclient(hunting): handle_auth_done global_id 124149 payload 824 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 1 -- 172.21.15.17:0/2977022506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd8ac13c9e0 con 0x7fd8ac134890 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1368640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 0x7fd8ac02d990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801b000 tx=0x7fd8a80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003820 con 0x7fd8ac134890 2024-09-15T21:27:26.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 10 monclient: dump: 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7fd8a8003820 con 0x7fd8ac134890 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 msgr2=0x7fd8ac02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 0x7fd8ac02d990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801b000 tx=0x7fd8a80046d0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 msgr2=0x7fd8ac02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd89dffb640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8ac134890 0x7fd8ac02d990 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801b000 tx=0x7fd8a80046d0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:26.857 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 94, 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9783, 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:26.858 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:26.869 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:26.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd89003b9c0 msgr2=0x7fd89003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd89003b9c0 0x7fd89003de80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00ad000 tx=0x7fd8a0008040 comp rx=0 tx=0).stop 2024-09-15T21:27:26.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 10 monclient: shutdown 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac135670 msgr2=0x7fd8ac11cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b1b69640 1 -- 172.21.15.17:0/2977022506 reap_dead start 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac135670 0x7fd8ac11cf60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c780 tx=0x7fd8a400cc40 comp rx=0 tx=0).stop 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 shutdown_connections 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 --2- 172.21.15.17:0/2977022506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8ac135670 0x7fd8ac11cf60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 >> 172.21.15.17:0/2977022506 conn(0x7fd8ac0a66a0 msgr2=0x7fd8ac0a4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 shutdown_connections 2024-09-15T21:27:26.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.834+0000 7fd8b35cc640 1 -- 172.21.15.17:0/2977022506 wait complete. 2024-09-15T21:27:26.871 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:27:26.871 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:27:26.872 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:27:26.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.978+0000 7f90d6fdf640 10 monclient: get_monmap_and_config 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.978+0000 7f90d6fdf640 10 monclient: build_initial_monmap 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.978+0000 7f90d6fdf640 10 monclient: monmap: 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:26.984776+0000 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:26.984776+0000 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:26.986 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.978+0000 7f90d6fdf640 1 Processor -- start 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.978+0000 7f90d6fdf640 1 -- start start 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.978+0000 7f90d6fdf640 10 monclient: init 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 0x7f90d00a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): picked mon.noname-b con 0x7f90d00a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90d009ffc0 0x7f90d00a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): picked mon.noname-c con 0x7f90d009ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90d0135670 0x7f90d0135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): picked mon.noname-a con 0x7f90d0135670 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f90d00a16f0 con 0x7f90d0135670 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f90d00a3130 con 0x7f90d009ffc0 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90d0073df0 con 0x7f90d00a0890 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient(hunting): authenticate will time out at 1570.380329s 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 0x7f90d00a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 0x7f90d00a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:49594/0 (socket says 172.21.15.17:49594) 2024-09-15T21:27:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90d0135670 0x7f90d0135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1790166674 learned_addr learned my addr 172.21.15.17:0/1790166674 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): get_auth_request con 0x7f90d00a0890 auth_method 0 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): get_auth_request con 0x7f90d0135670 auth_method 0 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cffff640 1 --2- 172.21.15.17:0/1790166674 >> v2:172.21.15.119:3300/0 conn(0x7f90d009ffc0 0x7f90d00a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cffff640 10 monclient(hunting): get_auth_request con 0x7f90d009ffc0 auth_method 0 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_done global_id 45593 payload 274 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1790166674 >> v2:172.21.15.119:3300/0 conn(0x7f90d009ffc0 msgr2=0x7f90d00a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 --2- 172.21.15.17:0/1790166674 >> v2:172.21.15.119:3300/0 conn(0x7f90d009ffc0 0x7f90d00a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1790166674 >> v2:172.21.15.17:3300/0 conn(0x7f90d0135670 msgr2=0x7f90d0135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 --2- 172.21.15.17:0/1790166674 >> v2:172.21.15.17:3300/0 conn(0x7f90d0135670 0x7f90d0135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient: _finish_hunting 0 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 monclient: found mon.noname-b 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1790166674 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d00a1bf0 con 0x7f90d00a0890 2024-09-15T21:27:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d557c640 1 --2- 172.21.15.17:0/1790166674 >> v2:172.21.15.17:3300/0 conn(0x7f90d0135670 0x7f90d0135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:26.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d4d7b640 1 --2- 172.21.15.17:0/1790166674 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 0x7f90d00a0c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90d006e7b0 tx=0x7f90c001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e2c6cb334780976 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:26.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 1 -- 172.21.15.17:0/1790166674 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00a5640 con 0x7f90d00a0890 2024-09-15T21:27:26.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:26.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:27:26.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: dump: 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:27:26.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 1 -- 172.21.15.17:0/1790166674 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c00a57e0 con 0x7f90d00a0890 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 5 monclient: authenticate success, global_id 45593 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90d6fdf640 4 set_mon_vals no callback set 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 1 -- 172.21.15.17:0/1790166674 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00a5bb0 con 0x7f90d00a0890 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.982+0000 7f90cf7fe640 10 monclient: dump: 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.000 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: shutdown 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1790166674 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 msgr2=0x7f90d00a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1790166674 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 0x7f90d00a0c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90d006e7b0 tx=0x7f90c001c8b0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1790166674 shutdown_connections 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1790166674 >> v2:172.21.15.119:3300/0 conn(0x7f90d009ffc0 0x7f90d00a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1790166674 >> v2:172.21.15.17:3300/0 conn(0x7f90d0135670 0x7f90d0135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1790166674 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d00a0890 0x7f90d00a0c70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1790166674 >> 172.21.15.17:0/1790166674 conn(0x7f90d00a66a0 msgr2=0x7f90d00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1790166674 shutdown_connections 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1790166674 wait complete. 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: build_initial_monmap 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: monmap: 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:27.002 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 Processor -- start 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- start start 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: init 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 0x7f90d0134640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): picked mon.noname-a con 0x7f90d009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d00a0890 0x7f90d011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): picked mon.noname-b con 0x7f90d00a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0135670 0x7f90d011ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): picked mon.noname-c con 0x7f90d0135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:27.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f90d013f680 con 0x7f90d009ffc0 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f90d013f500 con 0x7f90d00a0890 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90d013f800 con 0x7f90d0135670 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient(hunting): authenticate will time out at 1570.385951s 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 0x7f90d0134640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0135670 0x7f90d011ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 0x7f90d0134640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37210/0 (socket says 172.21.15.17:37210) 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1958733809 learned_addr learned my addr 172.21.15.17:0/1958733809 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): get_auth_request con 0x7f90d009ffc0 auth_method 0 2024-09-15T21:27:27.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): get_auth_request con 0x7f90d0135670 auth_method 0 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cffff640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d00a0890 0x7f90d011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cffff640 10 monclient(hunting): get_auth_request con 0x7f90d00a0890 auth_method 0 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient(hunting): handle_auth_done global_id 124155 payload 824 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0135670 msgr2=0x7f90d011ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0135670 0x7f90d011ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d00a0890 msgr2=0x7f90d011c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d00a0890 0x7f90d011c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient: _finish_hunting 0 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 monclient: found mon.noname-a 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d011dc80 con 0x7f90d009ffc0 2024-09-15T21:27:27.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d557c640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0135670 0x7f90d011ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d4d7b640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 0x7f90d0134640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90c001af60 tx=0x7f90c0019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00bc020 con 0x7f90d009ffc0 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: dump: 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.007 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: _finish_auth 0 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c00b86b0 con 0x7f90d009ffc0 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0009510 con 0x7f90d009ffc0 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90cd7fa640 10 monclient: dump: 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.008 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90ceffd640 4 set_mon_vals no callback set 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 5 monclient: authenticate success, global_id 124155 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: _renew_subs 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d01ce5f0 con 0x7f90d009ffc0 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: _renew_subs 2024-09-15T21:27:27.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.986+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d01353e0 con 0x7f90d009ffc0 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d6fdf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 0x7f90d002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90d0135aa0 con 0x7f90d011e7e0 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f90c00bf020 con 0x7f90d009ffc0 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 0x7f90d002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient: get_auth_request con 0x7f90d011e7e0 auth_method 0 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f90bc03b9c0 0x7f90bc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f90c00eb0b0 con 0x7f90d009ffc0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 10 monclient(hunting): handle_auth_done global_id 45599 payload 824 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90d01391c0 con 0x7f90d011e7e0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cffff640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 0x7f90d002d450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90c400f4b0 tx=0x7f90c400f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d557c640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f90bc03b9c0 0x7f90bc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d557c640 10 monclient: get_auth_request con 0x7f90bc03b9c0 auth_method 0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4019020 con 0x7f90d011e7e0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 10 monclient: dump: 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.011 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f90c4014870 con 0x7f90d011e7e0 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 msgr2=0x7f90d002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 0x7f90d002d450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90c400f4b0 tx=0x7f90c400f970 comp rx=0 tx=0).stop 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 msgr2=0x7f90d002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:27.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90cd7fa640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d011e7e0 0x7f90d002d450 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90c400f4b0 tx=0x7f90c400f970 comp rx=0 tx=0).stop 2024-09-15T21:27:27.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.990+0000 7f90d557c640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f90bc03b9c0 0x7f90bc03de80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009580 tx=0x7f90c8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90d6fdf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 0x7f90d002d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90d6fdf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90d006d020 con 0x7f90d0134b80 2024-09-15T21:27:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 0x7f90d002d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient: get_auth_request con 0x7f90d0134b80 auth_method 0 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 10 monclient(hunting): handle_auth_done global_id 45602 payload 824 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 1 -- 172.21.15.17:0/1958733809 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f90d011fda0 con 0x7f90d0134b80 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cffff640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 0x7f90d002d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f90c401a000 tx=0x7f90c40046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4019070 con 0x7f90d0134b80 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cd7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:27:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.994+0000 7f90cd7fa640 10 monclient: dump: 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7f90c4004230 con 0x7f90d0134b80 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 msgr2=0x7f90d002d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 0x7f90d002d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f90c401a000 tx=0x7f90c40046d0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 msgr2=0x7f90d002d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90cd7fa640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90d0134b80 0x7f90d002d990 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f90c401a000 tx=0x7f90c40046d0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 94, 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 415596, 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:27.017 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:27.018 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:27.019 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.021 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:27.021 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:27.021 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:27.021 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:27.022 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:27.023 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.024 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:27.025 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:27.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.029 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.030 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:27:27.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:27.032 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:27.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f90bc03b9c0 msgr2=0x7f90bc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f90bc03b9c0 0x7f90bc03de80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009580 tx=0x7f90c8009190 comp rx=0 tx=0).stop 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 10 monclient: shutdown 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 msgr2=0x7f90d0134640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 0x7f90d0134640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90c001af60 tx=0x7f90c0019650 comp rx=0 tx=0).stop 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d557c640 1 -- 172.21.15.17:0/1958733809 reap_dead start 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 shutdown_connections 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 --2- 172.21.15.17:0/1958733809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d009ffc0 0x7f90d0134640 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 >> 172.21.15.17:0/1958733809 conn(0x7f90d00a66a0 msgr2=0x7f90d00a4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 shutdown_connections 2024-09-15T21:27:27.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:26.998+0000 7f90d6fdf640 1 -- 172.21.15.17:0/1958733809 wait complete. 2024-09-15T21:27:27.035 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:27:27.035 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:27:27.035 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:27:27.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient: get_monmap_and_config 2024-09-15T21:27:27.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient: build_initial_monmap 2024-09-15T21:27:27.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient: monmap: 2024-09-15T21:27:27.147 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:27.147 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:27.147455+0000 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:27.147455+0000 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:27.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 Processor -- start 2024-09-15T21:27:27.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 -- start start 2024-09-15T21:27:27.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient: init 2024-09-15T21:27:27.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51700a0890 0x7f51700a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): picked mon.noname-c con 0x7f51700a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f517009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51700a11b0 0x7f5170135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): picked mon.noname-a con 0x7f51700a11b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51700a3130 con 0x7f517009ffc0 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5170073df0 con 0x7f51700a11b0 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f51700a1bf0 con 0x7f51700a0890 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5177885640 10 monclient(hunting): authenticate will time out at 1570.542471s 2024-09-15T21:27:27.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51700a11b0 0x7f5170135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51700a11b0 0x7f5170135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37220/0 (socket says 172.21.15.17:37220) 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 -- 172.21.15.17:0/2904424909 learned_addr learned my addr 172.21.15.17:0/2904424909 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): get_auth_request con 0x7f51700a11b0 auth_method 0 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175621640 1 --2- 172.21.15.17:0/2904424909 >> v2:172.21.15.119:3300/0 conn(0x7f51700a0890 0x7f51700a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient(hunting): handle_auth_done global_id 124158 payload 274 2024-09-15T21:27:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 -- 172.21.15.17:0/2904424909 >> v2:172.21.15.119:3300/0 conn(0x7f51700a0890 msgr2=0x7f51700a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 --2- 172.21.15.17:0/2904424909 >> v2:172.21.15.119:3300/0 conn(0x7f51700a0890 0x7f51700a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 -- 172.21.15.17:0/2904424909 >> v1:172.21.15.119:6789/0 conn(0x7f517009ffc0 legacy=0x7f5170135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient: _finish_hunting 0 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 monclient: found mon.noname-a 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.142+0000 7f5175e22640 1 -- 172.21.15.17:0/2904424909 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51700a15e0 con 0x7f51700a11b0 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 --2- 172.21.15.17:0/2904424909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f5170135d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5168009e50 tx=0x7f516801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a88396860c16e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 1 -- 172.21.15.17:0/2904424909 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680a5640 con 0x7f51700a11b0 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: dump: 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: _finish_auth 0 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 1 -- 172.21.15.17:0/2904424909 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51680a57e0 con 0x7f51700a11b0 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 5 monclient: authenticate success, global_id 124158 2024-09-15T21:27:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 4 set_mon_vals no callback set 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 1 -- 172.21.15.17:0/2904424909 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680a5bb0 con 0x7f51700a11b0 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5167fff640 10 monclient: dump: 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: shutdown 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- 172.21.15.17:0/2904424909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 msgr2=0x7f5170135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 --2- 172.21.15.17:0/2904424909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f5170135d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5168009e50 tx=0x7f516801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- 172.21.15.17:0/2904424909 shutdown_connections 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 --2- 172.21.15.17:0/2904424909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f5170135d80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 --2- 172.21.15.17:0/2904424909 >> v2:172.21.15.119:3300/0 conn(0x7f51700a0890 0x7f51700a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- 172.21.15.17:0/2904424909 >> 172.21.15.17:0/2904424909 conn(0x7f51700a66a0 msgr2=0x7f51700a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- 172.21.15.17:0/2904424909 shutdown_connections 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- 172.21.15.17:0/2904424909 wait complete. 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: build_initial_monmap 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: monmap: 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:27.163 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 Processor -- start 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- start start 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: init 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517009ffc0 0x7f517011e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): picked mon.noname-b con 0x7f517009ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51700a0890 0x7f5170135250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): picked mon.noname-c con 0x7f51700a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f517011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): picked mon.noname-a con 0x7f51700a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:27.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5170140510 con 0x7f51700a11b0 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51701364d0 con 0x7f517009ffc0 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5170140690 con 0x7f51700a0890 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5177885640 10 monclient(hunting): authenticate will time out at 1570.547505s 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f517011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51700a0890 0x7f5170135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f517011c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37234/0 (socket says 172.21.15.17:37234) 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 -- 172.21.15.17:0/3093921084 learned_addr learned my addr 172.21.15.17:0/3093921084 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): get_auth_request con 0x7f51700a11b0 auth_method 0 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): get_auth_request con 0x7f51700a0890 auth_method 0 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5174e20640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175621640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517009ffc0 0x7f517011e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175621640 10 monclient(hunting): get_auth_request con 0x7f517009ffc0 auth_method 0 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175621640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175621640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient(hunting): handle_auth_done global_id 124161 payload 824 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51700a0890 msgr2=0x7f5170135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f51700a0890 0x7f5170135250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517009ffc0 msgr2=0x7f517011e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517009ffc0 0x7f517011e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient: _finish_hunting 0 2024-09-15T21:27:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 monclient: found mon.noname-a 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.146+0000 7f5175e22640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517011fbe0 con 0x7f51700a11b0 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175e22640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f517011c4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f516801af60 tx=0x7f5168019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680bc020 con 0x7f51700a11b0 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: dump: 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: _finish_auth 0 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51680b8670 con 0x7f51700a11b0 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 5 monclient: authenticate success, global_id 124161 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 10 monclient: _renew_subs 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5170135ef0 con 0x7f51700a11b0 2024-09-15T21:27:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680b8a60 con 0x7f51700a11b0 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: dump: 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.170 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f51677fe640 4 set_mon_vals no callback set 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 10 monclient: _renew_subs 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517011f900 con 0x7f51700a11b0 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 0x7f517011ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f517009d0f0 con 0x7f5170135790 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f51680bf020 con 0x7f51700a11b0 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 0x7f517011ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient: get_auth_request con 0x7f5170135790 auth_method 0 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f515403b9f0 0x7f515403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f51680eb560 con 0x7f51700a11b0 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5174e20640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f515403b9f0 0x7f515403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5174e20640 10 monclient: get_auth_request con 0x7f515403b9f0 auth_method 0 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 10 monclient(hunting): handle_auth_done global_id 114211 payload 824 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51701ce440 con 0x7f5170135790 2024-09-15T21:27:27.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5175621640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 0x7f517011ca20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f516000e7a0 tx=0x7f516000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5174e20640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f515403b9f0 0x7f515403deb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f517013cdb0 tx=0x7f516c009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160019020 con 0x7f5170135790 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: dump: 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.173 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f516000ee50 con 0x7f5170135790 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 msgr2=0x7f517011ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 0x7f517011ca20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f516000e7a0 tx=0x7f516000ec60 comp rx=0 tx=0).stop 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 msgr2=0x7f517011ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:27.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.150+0000 7f5165ffb640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5170135790 0x7f517011ca20 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f516000e7a0 tx=0x7f516000ec60 comp rx=0 tx=0).stop 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5177885640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5177885640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 0x7f5170137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5177885640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f517011d370 con 0x7f517011cf60 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 0x7f5170137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient: get_auth_request con 0x7f517011cf60 auth_method 0 2024-09-15T21:27:27.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 10 monclient(hunting): handle_auth_done global_id 114214 payload 824 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 1 -- 172.21.15.17:0/3093921084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f51700a0c70 con 0x7f517011cf60 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5175621640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 0x7f5170137ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f516000e7a0 tx=0x7f51600046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160019030 con 0x7f517011cf60 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 10 monclient: dump: 2024-09-15T21:27:27.177 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f5160020220 con 0x7f517011cf60 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 msgr2=0x7f5170137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 0x7f5170137ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f516000e7a0 tx=0x7f51600046d0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 msgr2=0x7f5170137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.158+0000 7f5165ffb640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517011cf60 0x7f5170137ab0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f516000e7a0 tx=0x7f51600046d0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:27.179 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 94, 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 55731, 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:27.180 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:27.181 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:27.183 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:27.184 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:27.184 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:27.184 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:27.184 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:27.184 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:27.184 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.186 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:27.186 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:27.186 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:27.186 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:27.186 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:27.186 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:27.187 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:27.188 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:27.188 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:27.189 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:27.190 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:27.191 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:27.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.193 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:27.195 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:27.195 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:27.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.196 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:27.197 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:27.198 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:27.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:27.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:27.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:27.198 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:27.198 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:27.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f515403b9f0 msgr2=0x7f515403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f515403b9f0 0x7f515403deb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f517013cdb0 tx=0x7f516c009060 comp rx=0 tx=0).stop 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 10 monclient: shutdown 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 msgr2=0x7f517011c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5175e22640 1 -- 172.21.15.17:0/3093921084 reap_dead start 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f517011c4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f516801af60 tx=0x7f5168019650 comp rx=0 tx=0).stop 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 shutdown_connections 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 --2- 172.21.15.17:0/3093921084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51700a11b0 0x7f517011c4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 >> 172.21.15.17:0/3093921084 conn(0x7f51700a66a0 msgr2=0x7f517006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 shutdown_connections 2024-09-15T21:27:27.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.162+0000 7f5177885640 1 -- 172.21.15.17:0/3093921084 wait complete. 2024-09-15T21:27:27.201 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:27:27.201 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T21:27:27.201 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:27:27.201 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T21:27:27.201 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T21:27:27.201 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:27:27.202 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:27:27.217 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:27:27.218 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:27:27.218 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:27.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 10 monclient: get_monmap_and_config 2024-09-15T21:27:27.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 10 monclient: build_initial_monmap 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 10 monclient: monmap: 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:27.380660+0000 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:27.380660+0000 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:27.382 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 1 Processor -- start 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 1 -- start start 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 10 monclient: init 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.374+0000 7fe366230640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe3600a4c00 0x7fe36009cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): picked mon.noname-a con 0x7fe3600a4c00 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3600a4fe0 0x7fe3600a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): picked mon.noname-c con 0x7fe3600a4fe0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe3601379c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe36009c700 con 0x7fe3601379c0 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe36006f820 con 0x7fe3600a4c00 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe3600a5fc0 con 0x7fe3600a4fe0 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient(hunting): authenticate will time out at 1570.776120s 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe3600a4c00 0x7fe36009cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe3600a4c00 0x7fe36009cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37258/0 (socket says 172.21.15.17:37258) 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 -- 172.21.15.17:0/3952988392 learned_addr learned my addr 172.21.15.17:0/3952988392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): get_auth_request con 0x7fe3600a4c00 auth_method 0 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35effd640 1 --2- 172.21.15.17:0/3952988392 >> v2:172.21.15.119:3300/0 conn(0x7fe3600a4fe0 0x7fe3600a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35effd640 10 monclient(hunting): get_auth_request con 0x7fe3600a4fe0 auth_method 0 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_done global_id 124164 payload 274 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 -- 172.21.15.17:0/3952988392 >> v2:172.21.15.119:3300/0 conn(0x7fe3600a4fe0 msgr2=0x7fe3600a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 --2- 172.21.15.17:0/3952988392 >> v2:172.21.15.119:3300/0 conn(0x7fe3600a4fe0 0x7fe3600a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 -- 172.21.15.17:0/3952988392 >> v1:172.21.15.119:6789/0 conn(0x7fe3601379c0 legacy=0x7fe360137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:27:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 -- 172.21.15.17:0/3952988392 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3600a5d10 con 0x7fe3600a4c00 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35f7fe640 1 --2- 172.21.15.17:0/3952988392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe36009cb20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe36009bd60 tx=0x7fe35801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c9c06be1cc4ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 1 -- 172.21.15.17:0/3952988392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580a55c0 con 0x7fe3600a4c00 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: dump: 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 1 -- 172.21.15.17:0/3952988392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3580a5760 con 0x7fe3600a4c00 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 1 -- 172.21.15.17:0/3952988392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580a5b30 con 0x7fe3600a4c00 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 5 monclient: authenticate success, global_id 124164 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 4 set_mon_vals no callback set 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe35e7fc640 10 monclient: dump: 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.395 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 10 monclient: shutdown 2024-09-15T21:27:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 -- 172.21.15.17:0/3952988392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 msgr2=0x7fe36009cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 --2- 172.21.15.17:0/3952988392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe36009cb20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe36009bd60 tx=0x7fe35801c940 comp rx=0 tx=0).stop 2024-09-15T21:27:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 -- 172.21.15.17:0/3952988392 shutdown_connections 2024-09-15T21:27:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 --2- 172.21.15.17:0/3952988392 >> v2:172.21.15.119:3300/0 conn(0x7fe3600a4fe0 0x7fe3600a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 --2- 172.21.15.17:0/3952988392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe36009cb20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.378+0000 7fe366230640 1 -- 172.21.15.17:0/3952988392 >> 172.21.15.17:0/3952988392 conn(0x7fe3600aaeb0 msgr2=0x7fe3600a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- 172.21.15.17:0/3952988392 shutdown_connections 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- 172.21.15.17:0/3952988392 wait complete. 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: build_initial_monmap 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: monmap: 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 Processor -- start 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- start start 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: init 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe3601cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): picked mon.noname-a con 0x7fe3600a4c00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3600a4fe0 0x7fe360136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): picked mon.noname-b con 0x7fe3600a4fe0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3601379c0 0x7fe36013ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): picked mon.noname-c con 0x7fe3601379c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe3601429e0 con 0x7fe3600a4c00 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe360142860 con 0x7fe3600a4fe0 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe360142b60 con 0x7fe3601379c0 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient(hunting): authenticate will time out at 1570.781932s 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35ffff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3601379c0 msgr2=0x7fe36013ce50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3601379c0 0x7fe36013ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe3601cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe3601cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37262/0 (socket says 172.21.15.17:37262) 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 -- 172.21.15.17:0/555613641 learned_addr learned my addr 172.21.15.17:0/555613641 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:27.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): get_auth_request con 0x7fe3600a4c00 auth_method 0 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35effd640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3600a4fe0 0x7fe360136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient(hunting): handle_auth_done global_id 124167 payload 824 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 -- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3601379c0 msgr2=0x7fe36013ce50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3601379c0 0x7fe36013ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 -- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3600a4fe0 msgr2=0x7fe360136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3600a4fe0 0x7fe360136d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 -- 172.21.15.17:0/555613641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe360120870 con 0x7fe3600a4c00 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35f7fe640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe3601cfdf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe35801aff0 tx=0x7fe3580196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580bc020 con 0x7fe3600a4c00 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: dump: 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.401 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: _finish_auth 0 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: _check_auth_tickets 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3580b8690 con 0x7fe3600a4c00 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358009510 con 0x7fe3600a4c00 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:27.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 10 monclient: dump: 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 5 monclient: authenticate success, global_id 124167 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _renew_subs 2024-09-15T21:27:27.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3601216a0 con 0x7fe3600a4c00 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe35d7fa640 4 set_mon_vals no callback set 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _renew_subs 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3601213c0 con 0x7fe3600a4c00 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe3580bf020 con 0x7fe3600a4c00 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe347fff640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe34c03b970 0x7fe34c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.382+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36009cb20 con 0x7fe3600a4c00 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.386+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe3580eb530 con 0x7fe3600a4c00 2024-09-15T21:27:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.386+0000 7fe35effd640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe34c03b970 0x7fe34c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:27.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.386+0000 7fe35effd640 10 monclient: get_auth_request con 0x7fe34c03b970 auth_method 0 2024-09-15T21:27:27.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.386+0000 7fe35effd640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe34c03b970 0x7fe34c03de30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe36013fe00 tx=0x7fe354008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:27.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.390+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe3580c6030 con 0x7fe3600a4c00 2024-09-15T21:27:27.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.390+0000 7fe347fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:27.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.390+0000 7fe347fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe366230640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe366230640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe366230640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe360120b40 con 0x7fe3600a4c00 2024-09-15T21:27:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe347fff640 1 -- 172.21.15.17:0/555613641 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fe3580165c0 con 0x7fe3600a4c00 2024-09-15T21:27:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe347fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.626+0000 7fe347fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:27.633 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe34c03b970 msgr2=0x7fe34c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe34c03b970 0x7fe34c03de30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe36013fe00 tx=0x7fe354008040 comp rx=0 tx=0).stop 2024-09-15T21:27:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 10 monclient: shutdown 2024-09-15T21:27:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 msgr2=0x7fe3601cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe3601cfdf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe35801aff0 tx=0x7fe3580196e0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 shutdown_connections 2024-09-15T21:27:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe34c03b970 0x7fe34c03de30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3601379c0 0x7fe36013ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3600a4fe0 0x7fe360136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 --2- 172.21.15.17:0/555613641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3600a4c00 0x7fe3601cfdf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 >> 172.21.15.17:0/555613641 conn(0x7fe3600aaeb0 msgr2=0x7fe36009fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 shutdown_connections 2024-09-15T21:27:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:27.630+0000 7fe366230640 1 -- 172.21.15.17:0/555613641 wait complete. 2024-09-15T21:27:30.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:30.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient: get_monmap_and_config 2024-09-15T21:27:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient: build_initial_monmap 2024-09-15T21:27:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient: monmap: 2024-09-15T21:27:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:30.787644+0000 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:30.787644+0000 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:30.790 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:30.790 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 Processor -- start 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 -- start start 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient: init 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a40a0890 0x7fe0a40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): picked mon.noname-b con 0x7fe0a40a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:30.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe0a409ffc0 0x7fe0a40a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): picked mon.noname-c con 0x7fe0a409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0a4135670 0x7fe0a4135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): picked mon.noname-a con 0x7fe0a4135670 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a40a16f0 con 0x7fe0a4135670 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a40a3130 con 0x7fe0a409ffc0 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a4073df0 con 0x7fe0a40a0890 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0aa6b9640 10 monclient(hunting): authenticate will time out at 1574.183087s 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a3fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a40a0890 msgr2=0x7fe0a40a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a40a0890 0x7fe0a40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0a4135670 0x7fe0a4135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0a4135670 0x7fe0a4135aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45568/0 (socket says 172.21.15.17:45568) 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 -- 172.21.15.17:0/182424695 learned_addr learned my addr 172.21.15.17:0/182424695 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): get_auth_request con 0x7fe0a4135670 auth_method 0 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a37fe640 1 --2- 172.21.15.17:0/182424695 >> v2:172.21.15.119:3300/0 conn(0x7fe0a409ffc0 0x7fe0a40a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a37fe640 10 monclient(hunting): get_auth_request con 0x7fe0a409ffc0 auth_method 0 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient(hunting): handle_auth_done global_id 124170 payload 274 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 -- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a40a0890 msgr2=0x7fe0a40a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 --2- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a40a0890 0x7fe0a40a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 -- 172.21.15.17:0/182424695 >> v2:172.21.15.119:3300/0 conn(0x7fe0a409ffc0 msgr2=0x7fe0a40a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 --2- 172.21.15.17:0/182424695 >> v2:172.21.15.119:3300/0 conn(0x7fe0a409ffc0 0x7fe0a40a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient: _finish_hunting 0 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 monclient: found mon.noname-a 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.782+0000 7fe0a8c56640 1 -- 172.21.15.17:0/182424695 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a40a1bf0 con 0x7fe0a4135670 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a8c56640 1 --2- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a4135670 0x7fe0a4135aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0a406e7b0 tx=0x7fe09c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628af62d98261efa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 1 -- 172.21.15.17:0/182424695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0b0070 con 0x7fe0a4135670 2024-09-15T21:27:30.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: dump: 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 1 -- 172.21.15.17:0/182424695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09c0ab600 con 0x7fe0a4135670 2024-09-15T21:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 5 monclient: authenticate success, global_id 124170 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 4 set_mon_vals no callback set 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 1 -- 172.21.15.17:0/182424695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0ab9d0 con 0x7fe0a4135670 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0a2ffd640 10 monclient: dump: 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 10 monclient: shutdown 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a4135670 msgr2=0x7fe0a4135aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a4135670 0x7fe0a4135aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0a406e7b0 tx=0x7fe09c01c400 comp rx=0 tx=0).stop 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/182424695 shutdown_connections 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/182424695 >> v2:172.21.15.119:3300/0 conn(0x7fe0a409ffc0 0x7fe0a40a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a4135670 0x7fe0a4135aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/182424695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a40a0890 0x7fe0a40a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/182424695 >> 172.21.15.17:0/182424695 conn(0x7fe0a40a66a0 msgr2=0x7fe0a40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/182424695 shutdown_connections 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/182424695 wait complete. 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 10 monclient: build_initial_monmap 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.786+0000 7fe0aa6b9640 10 monclient: monmap: 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 Processor -- start 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- start start 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: init 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a409ffc0 0x7fe0a41cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): picked mon.noname-c con 0x7fe0a409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 0x7fe0a4134750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): picked mon.noname-a con 0x7fe0a40a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a4135670 0x7fe0a4134c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): picked mon.noname-b con 0x7fe0a4135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a413f680 con 0x7fe0a40a0890 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a413f500 con 0x7fe0a4135670 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a413f800 con 0x7fe0a409ffc0 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a3fff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a409ffc0 msgr2=0x7fe0a41cf820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient(hunting): authenticate will time out at 1574.188736s 2024-09-15T21:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a409ffc0 0x7fe0a41cf820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 0x7fe0a4134750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 0x7fe0a4134750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45576/0 (socket says 172.21.15.17:45576) 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 -- 172.21.15.17:0/1377305323 learned_addr learned my addr 172.21.15.17:0/1377305323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): get_auth_request con 0x7fe0a40a0890 auth_method 0 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a8c56640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a4135670 0x7fe0a4134c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a8c56640 10 monclient(hunting): get_auth_request con 0x7fe0a4135670 auth_method 0 2024-09-15T21:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a8c56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a8c56640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a8c56640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient(hunting): handle_auth_done global_id 124173 payload 824 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 -- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a409ffc0 msgr2=0x7fe0a41cf820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a409ffc0 0x7fe0a41cf820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 -- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a4135670 msgr2=0x7fe0a4134c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a4135670 0x7fe0a4134c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 monclient: found mon.noname-a 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 -- 172.21.15.17:0/1377305323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a409eaa0 con 0x7fe0a40a0890 2024-09-15T21:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a37fe640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 0x7fe0a4134750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe09400d800 tx=0x7fe09400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09400b660 con 0x7fe0a40a0890 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: dump: 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 5 monclient: authenticate success, global_id 124173 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe094019070 con 0x7fe0a40a0890 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09400ba00 con 0x7fe0a40a0890 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a1ffb640 4 set_mon_vals no callback set 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 10 monclient: dump: 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _renew_subs 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a4122130 con 0x7fe0a40a0890 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _renew_subs 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a4120a50 con 0x7fe0a40a0890 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a4135aa0 con 0x7fe0a40a0890 2024-09-15T21:27:30.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe09400bba0 con 0x7fe0a40a0890 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08403b970 0x7fe08403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe094050ae0 con 0x7fe0a40a0890 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a3fff640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08403b970 0x7fe08403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.790+0000 7fe0a3fff640 10 monclient: get_auth_request con 0x7fe08403b970 auth_method 0 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.794+0000 7fe0a3fff640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08403b970 0x7fe08403de30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe08c003d10 tx=0x7fe08c009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.794+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe09401cd70 con 0x7fe0a40a0890 2024-09-15T21:27:30.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.794+0000 7fe0a0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:30.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:30.794+0000 7fe0a0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:31.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.030+0000 7fe0aa6b9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:31.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.030+0000 7fe0aa6b9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:31.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.030+0000 7fe0aa6b9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:31.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.030+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe0a406d020 con 0x7fe0a40a0890 2024-09-15T21:27:31.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.030+0000 7fe0a0ff9640 1 -- 172.21.15.17:0/1377305323 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1560 (secure 0 0 0) 0x7fe094013390 con 0x7fe0a40a0890 2024-09-15T21:27:31.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0a0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:31.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0a0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:31.038 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:31.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08403b970 msgr2=0x7fe08403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:31.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08403b970 0x7fe08403de30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe08c003d10 tx=0x7fe08c009270 comp rx=0 tx=0).stop 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 10 monclient: shutdown 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 msgr2=0x7fe0a4134750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 0x7fe0a4134750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe09400d800 tx=0x7fe09400dcc0 comp rx=0 tx=0).stop 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 shutdown_connections 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe08403b970 0x7fe08403de30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a4135670 0x7fe0a4134c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a40a0890 0x7fe0a4134750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 --2- 172.21.15.17:0/1377305323 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe0a409ffc0 0x7fe0a41cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 >> 172.21.15.17:0/1377305323 conn(0x7fe0a40a66a0 msgr2=0x7fe0a40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 shutdown_connections 2024-09-15T21:27:31.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:31.034+0000 7fe0aa6b9640 1 -- 172.21.15.17:0/1377305323 wait complete. 2024-09-15T21:27:34.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:34.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.186+0000 7f52366e5640 10 monclient: get_monmap_and_config 2024-09-15T21:27:34.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.186+0000 7f52366e5640 10 monclient: build_initial_monmap 2024-09-15T21:27:34.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.186+0000 7f52366e5640 10 monclient: monmap: 2024-09-15T21:27:34.192 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:34.192609+0000 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:34.192609+0000 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:34.193 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:34.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.186+0000 7f52366e5640 1 Processor -- start 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.186+0000 7f52366e5640 1 -- start start 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.186+0000 7f52366e5640 10 monclient: init 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52300a4a90 0x7f52300a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): picked mon.noname-c con 0x7f52300a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f52300a53b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a5800 0x7f5230137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): picked mon.noname-b con 0x7f52300a5800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52300a2c30 con 0x7f52300a53b0 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f52300a0f00 con 0x7f52300a4a90 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f52300a19b0 con 0x7f52300a5800 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f52366e5640 10 monclient(hunting): authenticate will time out at 1577.588537s 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f5234c82640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a5800 msgr2=0x7f5230137f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f5234c82640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a5800 0x7f5230137f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f52300a53b0 0x7f5230137850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59130/0 (socket says 172.21.15.17:59130) 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522f7fe640 1 -- 172.21.15.17:0/3288124018 learned_addr learned my addr 172.21.15.17:0/3288124018 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 1 --2- 172.21.15.17:0/3288124018 >> v2:172.21.15.119:3300/0 conn(0x7f52300a4a90 0x7f52300a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): get_auth_request con 0x7f52300a4a90 auth_method 0 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:34.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient(hunting): handle_auth_done global_id 114226 payload 274 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 1 -- 172.21.15.17:0/3288124018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a5800 msgr2=0x7f5230137f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 1 --2- 172.21.15.17:0/3288124018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a5800 0x7f5230137f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 1 -- 172.21.15.17:0/3288124018 >> v1:172.21.15.17:6789/0 conn(0x7f52300a53b0 legacy=0x7f5230137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 1 monclient: found mon.noname-c 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:27:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.190+0000 7f522ffff640 1 -- 172.21.15.17:0/3288124018 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52300a1c90 con 0x7f52300a4a90 2024-09-15T21:27:34.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522ffff640 1 --2- 172.21.15.17:0/3288124018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f52300a4e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52200097f0 tx=0x7f522001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87bbe21461336ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 1 -- 172.21.15.17:0/3288124018 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52200b0070 con 0x7f52300a4a90 2024-09-15T21:27:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: dump: 2024-09-15T21:27:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: _finish_auth 0 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: _check_auth_tickets 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 1 -- 172.21.15.17:0/3288124018 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52200ab600 con 0x7f52300a4a90 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 1 -- 172.21.15.17:0/3288124018 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52200ab9d0 con 0x7f52300a4a90 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f522effd640 10 monclient: dump: 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f52366e5640 5 monclient: authenticate success, global_id 114226 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f52366e5640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.194+0000 7f52366e5640 4 set_mon_vals no callback set 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: shutdown 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/3288124018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 msgr2=0x7f52300a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 --2- 172.21.15.17:0/3288124018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f52300a4e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52200097f0 tx=0x7f522001c380 comp rx=0 tx=0).stop 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/3288124018 shutdown_connections 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 --2- 172.21.15.17:0/3288124018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a5800 0x7f5230137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 --2- 172.21.15.17:0/3288124018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f52300a4e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/3288124018 >> 172.21.15.17:0/3288124018 conn(0x7f52300aabb0 msgr2=0x7f52300a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/3288124018 shutdown_connections 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/3288124018 wait complete. 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: build_initial_monmap 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: monmap: 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:34.211 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 Processor -- start 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- start start 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: init 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f5230073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): picked mon.noname-b con 0x7f52300a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a53b0 0x7f52301d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): picked mon.noname-c con 0x7f52300a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 0x7f52301d1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): picked mon.noname-a con 0x7f52300a5800 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5230138620 con 0x7f52300a5800 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52301384a0 con 0x7f52300a4a90 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5230142550 con 0x7f52300a53b0 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient(hunting): authenticate will time out at 1577.598385s 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522f7fe640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a53b0 msgr2=0x7f52301d0bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a53b0 0x7f52301d0bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:34.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 0x7f52301d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 0x7f52301d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45590/0 (socket says 172.21.15.17:45590) 2024-09-15T21:27:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 -- 172.21.15.17:0/1035224594 learned_addr learned my addr 172.21.15.17:0/1035224594 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): get_auth_request con 0x7f52300a5800 auth_method 0 2024-09-15T21:27:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522ffff640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f5230073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522ffff640 10 monclient(hunting): get_auth_request con 0x7f52300a4a90 auth_method 0 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient(hunting): handle_auth_done global_id 124176 payload 824 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 -- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a53b0 msgr2=0x7f52301d0bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a53b0 0x7f52301d0bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 -- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 msgr2=0x7f5230073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f5230073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient: _finish_hunting 0 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 monclient: found mon.noname-a 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 -- 172.21.15.17:0/1035224594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523009fdb0 con 0x7f52300a5800 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f5234c82640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 0x7f52301d1130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f522800be80 tx=0x7f522800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228016430 con 0x7f52300a5800 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: dump: 2024-09-15T21:27:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: _finish_auth 0 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5228010030 con 0x7f52300a5800 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228016810 con 0x7f52300a5800 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522cff9640 10 monclient: dump: 2024-09-15T21:27:34.219 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:34.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f522e7fc640 4 set_mon_vals no callback set 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 5 monclient: authenticate success, global_id 124176 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: _renew_subs 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52300a97e0 con 0x7f52300a5800 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: _renew_subs 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.198+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523013c500 con 0x7f52300a5800 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f52366e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f52366e5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f52366e5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f52280153e0 con 0x7f52300a5800 2024-09-15T21:27:34.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52300a4e70 con 0x7f52300a5800 2024-09-15T21:27:34.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f522cff9640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f521c03b970 0x7f521c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:34.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5228050bb0 con 0x7f52300a5800 2024-09-15T21:27:34.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f522ffff640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f521c03b970 0x7f521c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:34.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f522ffff640 10 monclient: get_auth_request con 0x7f521c03b970 auth_method 0 2024-09-15T21:27:34.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.202+0000 7f522ffff640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f521c03b970 0x7f521c03de30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f52200ad000 tx=0x7f52200a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:34.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.206+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5228007cf0 con 0x7f52300a5800 2024-09-15T21:27:34.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.206+0000 7f522cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:34.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.206+0000 7f522cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:34.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:34.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:34.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:34.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f523013b750 con 0x7f52300a5800 2024-09-15T21:27:34.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f522cff9640 1 -- 172.21.15.17:0/1035224594 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f52280060d0 con 0x7f52300a5800 2024-09-15T21:27:34.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f522cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:34.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f522cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:34.444 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f521c03b970 msgr2=0x7f521c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f521c03b970 0x7f521c03de30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f52200ad000 tx=0x7f52200a6000 comp rx=0 tx=0).stop 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 10 monclient: shutdown 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 msgr2=0x7f52301d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 0x7f52301d1130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f522800be80 tx=0x7f522800beb0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 shutdown_connections 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f521c03b970 0x7f521c03de30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300a5800 0x7f52301d1130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f52300a53b0 0x7f52301d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 --2- 172.21.15.17:0/1035224594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52300a4a90 0x7f5230073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.438+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 >> 172.21.15.17:0/1035224594 conn(0x7f52300aabb0 msgr2=0x7f52300a02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.442+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 shutdown_connections 2024-09-15T21:27:34.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:34.442+0000 7f52366e5640 1 -- 172.21.15.17:0/1035224594 wait complete. 2024-09-15T21:27:37.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient: get_monmap_and_config 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient: build_initial_monmap 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient: monmap: 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:37.596020+0000 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:37.596020+0000 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:37.597 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:37.597 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:37.597 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:37.597 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:37.597 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:37.597 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:37.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 Processor -- start 2024-09-15T21:27:37.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 -- start start 2024-09-15T21:27:37.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient: init 2024-09-15T21:27:37.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:37.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:27:37.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): picked mon.noname-a con 0x7f0f901340c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0f9009d6c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 0x7f0f9013be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): picked mon.noname-b con 0x7f0f900a0fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f901349e0 con 0x7f0f9009d6c0 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9009d520 con 0x7f0f901340c0 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9009e1b0 con 0x7f0f900a0fa0 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f96ad6640 10 monclient(hunting): authenticate will time out at 1580.991413s 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f95073640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 msgr2=0x7f0f9013be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f95073640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 0x7f0f9013be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:37.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45602/0 (socket says 172.21.15.17:45602) 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 learned_addr learned my addr 172.21.15.17:0/1268600201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 10 monclient(hunting): get_auth_request con 0x7f0f901340c0 auth_method 0 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:37.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:37.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.590+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 10 monclient(hunting): get_auth_request con 0x7f0f901340c0 auth_method 0 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:27:37.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 msgr2=0x7f0f9013be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:37.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:37.794+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 0x7f0f9013be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:27:38.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 msgr2=0x7f0f9013be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:38.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 0x7f0f9013be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:27:38.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:38.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 10 monclient(hunting): get_auth_request con 0x7f0f901340c0 auth_method 0 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:38.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:38.198+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 msgr2=0x7f0f9013be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 0x7f0f9013be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 10 monclient(hunting): get_auth_request con 0x7f0f901340c0 auth_method 0 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:39.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:39.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:39.002+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:27:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): tick 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 msgr2=0x7f0f9013be70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f900a0fa0 0x7f0f9013be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 msgr2=0x7f0f901344a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.17:3300/0 conn(0x7f0f901340c0 0x7f0f901344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 -- 172.21.15.17:0/1268600201 >> v1:172.21.15.119:6789/0 conn(0x7f0f9009d6c0 legacy=0x7f0f900a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient: _add_conns ranks=[0,4] 2024-09-15T21:27:40.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0f7c005900 addr v1:172.21.15.17:6789/0 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7f0f7c009630 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 -- 172.21.15.17:0/1268600201 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9009e490 con 0x7f0f7c005900 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 1 -- 172.21.15.17:0/1268600201 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9009d520 con 0x7f0f7c009630 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f7c009630 auth_method 0 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:27:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.594+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f7c009630 auth_method 0 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:40.798+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f7c009630 auth_method 0 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:41.202+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:27:42.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f7c009630 auth_method 0 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:27:42.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.006+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:27:42.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.122+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1662624423 0 0) 0x7f0f9009e490 con 0x7f0f7c005900 2024-09-15T21:27:42.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.122+0000 7f0f8f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:27:42.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.122+0000 7f0f8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.122+0000 7f0f8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.122+0000 7f0f8f7fe640 10 monclient(hunting): my global_id is 124179 2024-09-15T21:27:42.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f78003530 con 0x7f0f7c005900 2024-09-15T21:27:42.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3333234884 0 0) 0x7f0f78003530 con 0x7f0f7c005900 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 msgr2=0x7f0f7c00baf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 --2- 172.21.15.17:0/1268600201 >> v2:172.21.15.119:3300/0 conn(0x7f0f7c009630 0x7f0f7c00baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c00d280 con 0x7f0f7c005900 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0f8000f040 con 0x7f0f7c005900 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: dump: 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.135 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 >> v1:172.21.15.17:6789/0 conn(0x7f0f7c005900 legacy=0x7f0f7c007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 reap_dead start 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780039c0 0x7f0f78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): picked mon.c con 0x7f0f780039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): picked mon.b con 0x7f0f78004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780052a0 0x7f0f7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): picked mon.a con 0x7f0f780052a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9009d520 con 0x7f0f780052a0 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7800c1f0 con 0x7f0f78004810 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7800c3f0 con 0x7f0f780039c0 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:42.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f94872640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780039c0 msgr2=0x7f0f78003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f94872640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780039c0 0x7f0f78003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780052a0 0x7f0f7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): get_auth_request con 0x7f0f780052a0 auth_method 0 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f78004810 auth_method 0 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient(hunting): handle_auth_done global_id 124179 payload 293 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780039c0 msgr2=0x7f0f78003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780039c0 0x7f0f78003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 msgr2=0x7f0f78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 monclient: found mon.a 2024-09-15T21:27:42.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 1 -- 172.21.15.17:0/1268600201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f78003530 con 0x7f0f780052a0 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f95073640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f96ad6640 5 monclient: authenticate success, global_id 124179 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f96ad6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:42.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.126+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f78004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:27:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f95073640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780052a0 0x7f0f7800b9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f900a2960 tx=0x7f0f8801cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a62b1b60178d382 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88007a60 con 0x7f0f780052a0 2024-09-15T21:27:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: dump: 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f880a9070 con 0x7f0f780052a0 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f96ad6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f96ad6640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 1 -- 172.21.15.17:0/1268600201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f880ae470 con 0x7f0f780052a0 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f96ad6640 4 set_mon_vals no callback set 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.130+0000 7f0f8f7fe640 10 monclient: dump: 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.147 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient: shutdown 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780052a0 msgr2=0x7f0f7800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780052a0 0x7f0f7800b9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f900a2960 tx=0x7f0f8801cce0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- 172.21.15.17:0/1268600201 shutdown_connections 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780052a0 0x7f0f7800b9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f78004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/1268600201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780039c0 0x7f0f78003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- 172.21.15.17:0/1268600201 >> 172.21.15.17:0/1268600201 conn(0x7f0f900a66a0 msgr2=0x7f0f900a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- 172.21.15.17:0/1268600201 shutdown_connections 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- 172.21.15.17:0/1268600201 wait complete. 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient: build_initial_monmap 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient: monmap: 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 Processor -- start 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- start start 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient: init 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 0x7f0f90121df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): picked mon.noname-a con 0x7f0f780039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f90122330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): picked mon.noname-b con 0x7f0f78004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780052a0 0x7f0f90122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): picked mon.noname-c con 0x7f0f780052a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9009ce50 con 0x7f0f780039c0 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9009ccd0 con 0x7f0f78004810 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0f900a2590 con 0x7f0f780052a0 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f96ad6640 10 monclient(hunting): authenticate will time out at 1585.535531s 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 0x7f0f90121df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f95073640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780052a0 msgr2=0x7f0f90122870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 0x7f0f90121df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34380/0 (socket says 172.21.15.17:34380) 2024-09-15T21:27:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 1 -- 172.21.15.17:0/2728784224 learned_addr learned my addr 172.21.15.17:0/2728784224 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): get_auth_request con 0x7f0f780039c0 auth_method 0 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f95073640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780052a0 0x7f0f90122870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f94872640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.134+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f90122330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f78004810 auth_method 0 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 10 monclient(hunting): handle_auth_done global_id 124182 payload 824 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 -- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780052a0 msgr2=0x7f0f90122870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780052a0 0x7f0f90122870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 -- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 msgr2=0x7f0f90122330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f90122330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 monclient: found mon.noname-a 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 -- 172.21.15.17:0/2728784224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f90124580 con 0x7f0f780039c0 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f90122330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f94872640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 0x7f0f90121df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f80003990 tx=0x7f0f80003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8001c070 con 0x7f0f780039c0 2024-09-15T21:27:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: dump: 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80010040 con 0x7f0f780039c0 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80017400 con 0x7f0f780039c0 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 10 monclient: dump: 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 5 monclient: authenticate success, global_id 124182 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 monclient: _renew_subs 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8e7fc640 4 set_mon_vals no callback set 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f901256d0 con 0x7f0f780039c0 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 monclient: _renew_subs 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f900a4450 con 0x7f0f780039c0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0f800175a0 con 0x7f0f780039c0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f900a3f00 con 0x7f0f780039c0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f7c03c1f0 0x7f0f7c03e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0f800540d0 con 0x7f0f780039c0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f7c03c1f0 0x7f0f7c03e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 10 monclient: get_auth_request con 0x7f0f7c03c1f0 auth_method 0 2024-09-15T21:27:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.138+0000 7f0f8ffff640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f7c03c1f0 0x7f0f7c03e6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f84007c80 tx=0x7f0f84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.142+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0f80025020 con 0x7f0f780039c0 2024-09-15T21:27:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.142+0000 7f0f8d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.142+0000 7f0f8d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f96ad6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:27:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f96ad6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f96ad6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0f900a14c0 con 0x7f0f780039c0 2024-09-15T21:27:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f8d7fa640 1 -- 172.21.15.17:0/2728784224 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1554 (secure 0 0 0) 0x7f0f80027300 con 0x7f0f780039c0 2024-09-15T21:27:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f8d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:27:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.382+0000 7f0f8d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:42.390 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f7c03c1f0 msgr2=0x7f0f7c03e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f7c03c1f0 0x7f0f7c03e6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f84007c80 tx=0x7f0f84009040 comp rx=0 tx=0).stop 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 10 monclient: shutdown 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 msgr2=0x7f0f90121df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 0x7f0f90121df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f80003990 tx=0x7f0f80003e50 comp rx=0 tx=0).stop 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 shutdown_connections 2024-09-15T21:27:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0f7c03c1f0 0x7f0f7c03e6b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0f780052a0 0x7f0f90122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78004810 0x7f0f90122330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 --2- 172.21.15.17:0/2728784224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f780039c0 0x7f0f90121df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 >> 172.21.15.17:0/2728784224 conn(0x7f0f900a66a0 msgr2=0x7f0f9009ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 shutdown_connections 2024-09-15T21:27:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.386+0000 7f0f96ad6640 1 -- 172.21.15.17:0/2728784224 wait complete. 2024-09-15T21:27:42.410 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:27:42.411 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:27:42.411 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.526+0000 7f70aec37640 10 monclient: get_monmap_and_config 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.526+0000 7f70aec37640 10 monclient: build_initial_monmap 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.526+0000 7f70aec37640 10 monclient: monmap: 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:42.534040+0000 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:42.534040+0000 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:42.535 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 1 Processor -- start 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 1 -- start start 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient: init 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70a80a0890 0x7f70a80a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient(hunting): picked mon.noname-a con 0x7f70a80a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f70a809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70a80a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a8136490 con 0x7f70a80a11b0 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a80a3130 con 0x7f70a809ffc0 2024-09-15T21:27:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f70a8073df0 con 0x7f70a80a0890 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70aec37640 10 monclient(hunting): authenticate will time out at 1585.929467s 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70a80a0890 0x7f70a80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70a80a0890 0x7f70a80a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34388/0 (socket says 172.21.15.17:34388) 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 -- 172.21.15.17:0/1548029332 learned_addr learned my addr 172.21.15.17:0/1548029332 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): get_auth_request con 0x7f70a80a0890 auth_method 0 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_done global_id 124185 payload 274 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 -- 172.21.15.17:0/1548029332 >> v1:172.21.15.119:6789/0 conn(0x7f70a809ffc0 legacy=0x7f70a8135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 -- 172.21.15.17:0/1548029332 >> v1:172.21.15.17:6789/0 conn(0x7f70a80a11b0 legacy=0x7f70a8135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 monclient: found mon.noname-a 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.530+0000 7f70ac9d3640 1 -- 172.21.15.17:0/1548029332 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a80a1bf0 con 0x7f70a80a0890 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70ac9d3640 1 --2- 172.21.15.17:0/1548029332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a80a0c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7098009e80 tx=0x7f709801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c92d7e4c714220 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 1 -- 172.21.15.17:0/1548029332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70980a56a0 con 0x7f70a80a0890 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: dump: 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.546 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 1 -- 172.21.15.17:0/1548029332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70980a5840 con 0x7f70a80a0890 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 5 monclient: authenticate success, global_id 124185 2024-09-15T21:27:42.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 4 set_mon_vals no callback set 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 1 -- 172.21.15.17:0/1548029332 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70980a5c10 con 0x7f70a80a0890 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a77fe640 10 monclient: dump: 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.548 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.549 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.549 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.549 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.549 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.549 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: shutdown 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/1548029332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 msgr2=0x7f70a80a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 --2- 172.21.15.17:0/1548029332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a80a0c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7098009e80 tx=0x7f709801c8d0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/1548029332 shutdown_connections 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 --2- 172.21.15.17:0/1548029332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a80a0c70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/1548029332 >> 172.21.15.17:0/1548029332 conn(0x7f70a80a66a0 msgr2=0x7f70a80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/1548029332 shutdown_connections 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/1548029332 wait complete. 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: build_initial_monmap 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: monmap: 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:42.551 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 Processor -- start 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- start start 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: init 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a809ffc0 0x7f70a809e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): picked mon.noname-b con 0x7f70a809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a8135590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): picked mon.noname-a con 0x7f70a80a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70a80a11b0 0x7f70a809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): picked mon.noname-c con 0x7f70a80a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:42.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70a8141ab0 con 0x7f70a80a0890 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70a8141930 con 0x7f70a809ffc0 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f70a8141c30 con 0x7f70a80a11b0 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient(hunting): authenticate will time out at 1585.934833s 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70ad1d4640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70a80a11b0 msgr2=0x7f70a809d400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70ad1d4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70a80a11b0 0x7f70a809d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:42.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a8135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a8135590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34402/0 (socket says 172.21.15.17:34402) 2024-09-15T21:27:42.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 -- 172.21.15.17:0/752377066 learned_addr learned my addr 172.21.15.17:0/752377066 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:42.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): get_auth_request con 0x7f70a80a0890 auth_method 0 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient(hunting): handle_auth_done global_id 124188 payload 824 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70a80a11b0 msgr2=0x7f70a809d400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70a80a11b0 0x7f70a809d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70ac9d3640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a809ffc0 0x7f70a809e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a809ffc0 msgr2=0x7f70a809e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a809ffc0 0x7f70a809e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 monclient: found mon.noname-a 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a809d0f0 con 0x7f70a80a0890 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a7fff640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a8135590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f709000c450 tx=0x7f709000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090012830 con 0x7f70a80a0890 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: dump: 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.556 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7090017070 con 0x7f70a80a0890 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090012c30 con 0x7f70a80a0890 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 5 monclient: authenticate success, global_id 124188 2024-09-15T21:27:42.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: _renew_subs 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a81d0070 con 0x7f70a80a0890 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a6ffd640 4 set_mon_vals no callback set 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70a57fa640 10 monclient: dump: 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.558 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: _renew_subs 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.534+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a813dfd0 con 0x7f70a80a0890 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70aec37640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70aec37640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 0x7f70a809d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70aec37640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70a809f410 con 0x7f70a8135ad0 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f709001c460 con 0x7f70a80a0890 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 0x7f70a809d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient: get_auth_request con 0x7f70a8135ad0 auth_method 0 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f708803b9a0 0x7f708803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7090050440 con 0x7f70a80a0890 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_done global_id 124191 payload 824 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70a80a0c70 con 0x7f70a8135ad0 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ac9d3640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 0x7f70a809d940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70a8073b00 tx=0x7f7098019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ad1d4640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f708803b9a0 0x7f708803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ad1d4640 10 monclient: get_auth_request con 0x7f708803b9a0 auth_method 0 2024-09-15T21:27:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70980bd030 con 0x7f70a8135ad0 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 10 monclient: dump: 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f70980b9470 con 0x7f70a8135ad0 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 msgr2=0x7f70a809d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 0x7f70a809d940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70a8073b00 tx=0x7f7098019670 comp rx=0 tx=0).stop 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 msgr2=0x7f70a809d940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70a57fa640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a8135ad0 0x7f70a809d940 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70a8073b00 tx=0x7f7098019670 comp rx=0 tx=0).stop 2024-09-15T21:27:42.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.538+0000 7f70ad1d4640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f708803b9a0 0x7f708803de60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f70a0007fd0 tx=0x7f70a0007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:42.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 0x7f70a8138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70a809e260 con 0x7f70a809de80 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 0x7f70a8138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient: get_auth_request con 0x7f70a809de80 auth_method 0 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 10 monclient(hunting): handle_auth_done global_id 124194 payload 824 2024-09-15T21:27:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 1 -- 172.21.15.17:0/752377066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f70a806d020 con 0x7f70a809de80 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ac9d3640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 0x7f70a8138ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f709801af80 tx=0x7f70980a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70980bd030 con 0x7f70a809de80 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 10 monclient: dump: 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.566 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7f70980c6070 con 0x7f70a809de80 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 msgr2=0x7f70a8138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 0x7f70a8138ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f709801af80 tx=0x7f70980a7000 comp rx=0 tx=0).stop 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 msgr2=0x7f70a8138ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70a57fa640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a809de80 0x7f70a8138ab0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f709801af80 tx=0x7f70980a7000 comp rx=0 tx=0).stop 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25495, 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.568 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:42.569 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:42.570 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:42.571 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:42.571 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:42.572 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:42.573 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:42.574 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:42.574 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:42.574 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:42.575 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:42.575 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:42.576 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:42.576 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:42.576 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:42.576 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:42.576 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:42.576 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:42.577 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:42.577 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:42.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:42.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:42.582 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:42.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.584 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.585 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.586 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.588 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:42.589 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f708803b9a0 msgr2=0x7f708803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f708803b9a0 0x7f708803de60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f70a0007fd0 tx=0x7f70a0007980 comp rx=0 tx=0).stop 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 10 monclient: shutdown 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 msgr2=0x7f70a8135590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 --2- 172.21.15.17:0/752377066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80a0890 0x7f70a8135590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f709000c450 tx=0x7f709000c910 comp rx=0 tx=0).stop 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ad1d4640 1 -- 172.21.15.17:0/752377066 reap_dead start 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70ad1d4640 1 -- 172.21.15.17:0/752377066 reap_dead start 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 shutdown_connections 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 >> 172.21.15.17:0/752377066 conn(0x7f70a80a66a0 msgr2=0x7f70a8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 shutdown_connections 2024-09-15T21:27:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.546+0000 7f70aec37640 1 -- 172.21.15.17:0/752377066 wait complete. 2024-09-15T21:27:42.591 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:27:42.592 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:27:42.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.702+0000 7f922ac5e640 10 monclient: get_monmap_and_config 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.702+0000 7f922ac5e640 10 monclient: build_initial_monmap 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.702+0000 7f922ac5e640 10 monclient: monmap: 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:27:42.710160+0000 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:27:42.710160+0000 2024-09-15T21:27:42.710 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:27:42.711 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 Processor -- start 2024-09-15T21:27:42.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 -- start start 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient: init 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92240a0890 0x7f92240a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): picked mon.noname-b con 0x7f92240a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f922409ffc0 0x7f92240a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): picked mon.noname-c con 0x7f922409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9224135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92240a3130 con 0x7f9224135670 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f92240a16f0 con 0x7f922409ffc0 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9224073df0 con 0x7f92240a0890 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f922ac5e640 10 monclient(hunting): authenticate will time out at 1586.105554s 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f92289fa640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92240a0890 msgr2=0x7f92240a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f92289fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92240a0890 0x7f92240a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f922409ffc0 0x7f92240a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f922409ffc0 0x7f92240a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:46226/0 (socket says 172.21.15.17:46226) 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 -- 172.21.15.17:0/1792228418 learned_addr learned my addr 172.21.15.17:0/1792228418 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): get_auth_request con 0x7f922409ffc0 auth_method 0 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient(hunting): handle_auth_done global_id 114235 payload 274 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 -- 172.21.15.17:0/1792228418 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92240a0890 msgr2=0x7f92240a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 --2- 172.21.15.17:0/1792228418 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92240a0890 0x7f92240a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 -- 172.21.15.17:0/1792228418 >> v1:172.21.15.119:6789/0 conn(0x7f9224135670 legacy=0x7f9224135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 monclient: found mon.noname-c 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:27:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.706+0000 7f9223fff640 1 -- 172.21.15.17:0/1792228418 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240a1bf0 con 0x7f922409ffc0 2024-09-15T21:27:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f9223fff640 1 --2- 172.21.15.17:0/1792228418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922409ffc0 0x7f92240a11b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92240a1870 tx=0x7f920c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eabfd90d078a27b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 1 -- 172.21.15.17:0/1792228418 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b0070 con 0x7f922409ffc0 2024-09-15T21:27:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:27:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: dump: 2024-09-15T21:27:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 1 -- 172.21.15.17:0/1792228418 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c0ab600 con 0x7f922409ffc0 2024-09-15T21:27:42.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 5 monclient: authenticate success, global_id 114235 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: get_monmap_and_config success 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 4 set_mon_vals no callback set 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 1 -- 172.21.15.17:0/1792228418 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0ab9d0 con 0x7f922409ffc0 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92237fe640 10 monclient: dump: 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.725 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: shutdown 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- 172.21.15.17:0/1792228418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922409ffc0 msgr2=0x7f92240a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1792228418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922409ffc0 0x7f92240a11b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92240a1870 tx=0x7f920c01c420 comp rx=0 tx=0).stop 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- 172.21.15.17:0/1792228418 shutdown_connections 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1792228418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922409ffc0 0x7f92240a11b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1792228418 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f92240a0890 0x7f92240a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- 172.21.15.17:0/1792228418 >> 172.21.15.17:0/1792228418 conn(0x7f92240a66a0 msgr2=0x7f92240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- 172.21.15.17:0/1792228418 shutdown_connections 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- 172.21.15.17:0/1792228418 wait complete. 2024-09-15T21:27:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: build_initial_monmap 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: monmap: 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:27:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 Processor -- start 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- start start 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: init 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: _reopen_session rank -1 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f922409ffc0 0x7f922411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): picked mon.noname-c con 0x7f922409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240a0890 0x7f922411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): picked mon.noname-b con 0x7f92240a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 0x7f922411cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): picked mon.noname-a con 0x7f9224135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:27:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9224140690 con 0x7f9224135670 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9224140510 con 0x7f92240a0890 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9224140810 con 0x7f922409ffc0 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): _renew_subs 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f922ac5e640 10 monclient(hunting): authenticate will time out at 1586.111063s 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92289fa640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f922409ffc0 msgr2=0x7f922411c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92289fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f922409ffc0 0x7f922411c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:27:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 0x7f922411cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 0x7f922411cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34440/0 (socket says 172.21.15.17:34440) 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 -- 172.21.15.17:0/1972090380 learned_addr learned my addr 172.21.15.17:0/1972090380 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): get_auth_request con 0x7f9224135670 auth_method 0 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f9223fff640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240a0890 0x7f922411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f9223fff640 10 monclient(hunting): get_auth_request con 0x7f92240a0890 auth_method 0 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f9223fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f9223fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f9223fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient(hunting): handle_auth_done global_id 124197 payload 824 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f922409ffc0 msgr2=0x7f922411c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f922409ffc0 0x7f922411c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240a0890 msgr2=0x7f922411ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240a0890 0x7f922411ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient: _finish_hunting 0 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 monclient: found mon.noname-a 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922411eba0 con 0x7f9224135670 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.710+0000 7f92291fb640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 0x7f922411cf60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f921c00c4d0 tx=0x7f921c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0052d0 con 0x7f9224135670 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: dump: 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: _finish_auth 0 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: _check_auth_tickets 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c005470 con 0x7f9224135670 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c012850 con 0x7f9224135670 2024-09-15T21:27:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: dump: 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 5 monclient: authenticate success, global_id 124197 2024-09-15T21:27:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f9222ffd640 4 set_mon_vals no callback set 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 10 monclient: _renew_subs 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922409d0f0 con 0x7f9224135670 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 10 monclient: _renew_subs 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922411ee80 con 0x7f9224135670 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 0x7f9224133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f922411d9b0 con 0x7f922413c8d0 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f921c0129f0 con 0x7f9224135670 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 0x7f9224133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f920403b9a0 0x7f920403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient: get_auth_request con 0x7f922413c8d0 auth_method 0 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 7657+0+0 (secure 0 0 0) 0x7f921c0502f0 con 0x7f9224135670 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f9223fff640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f920403b9a0 0x7f920403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f9223fff640 10 monclient: get_auth_request con 0x7f920403b9a0 auth_method 0 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 10 monclient(hunting): handle_auth_done global_id 114244 payload 824 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9224135aa0 con 0x7f922413c8d0 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92289fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 0x7f9224133ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f921400bcc0 tx=0x7f9214009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214014630 con 0x7f922413c8d0 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: dump: 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f9223fff640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f920403b9a0 0x7f920403de60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ad000 tx=0x7f920c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9214014630 con 0x7f922413c8d0 2024-09-15T21:27:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:27:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:27:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 msgr2=0x7f9224133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 0x7f9224133ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f921400bcc0 tx=0x7f9214009f30 comp rx=0 tx=0).stop 2024-09-15T21:27:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 msgr2=0x7f9224133ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.714+0000 7f92217fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922413c8d0 0x7f9224133ca0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f921400bcc0 tx=0x7f9214009f30 comp rx=0 tx=0).stop 2024-09-15T21:27:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 0x7f92241345f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9224134b30 con 0x7f92241341e0 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 0x7f92241345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient: get_auth_request con 0x7f92241341e0 auth_method 0 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 10 monclient(hunting): handle_auth_done global_id 114247 payload 824 2024-09-15T21:27:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 1 -- 172.21.15.17:0/1972090380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f922411e820 con 0x7f92241341e0 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92289fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 0x7f92241345f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f921400bcc0 tx=0x7f9214005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214019040 con 0x7f92241341e0 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 10 monclient: dump: 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:27:42.741 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:27:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4533 (secure 0 0 0) 0x7f9214023400 con 0x7f92241341e0 2024-09-15T21:27:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:27:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:27:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 msgr2=0x7f92241345f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 0x7f92241345f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f921400bcc0 tx=0x7f9214005e20 comp rx=0 tx=0).stop 2024-09-15T21:27:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 msgr2=0x7f92241345f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:27:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92217fa640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92241341e0 0x7f92241345f0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f921400bcc0 tx=0x7f9214005e20 comp rx=0 tx=0).stop 2024-09-15T21:27:42.744 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 71291, 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.745 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:42.746 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:42.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:27:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:27:42.749 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:27:42.750 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:27:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:27:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:27:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:27:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:27:42.757 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f920403b9a0 msgr2=0x7f920403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f920403b9a0 0x7f920403de60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ad000 tx=0x7f920c008040 comp rx=0 tx=0).stop 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 10 monclient: shutdown 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 msgr2=0x7f922411cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f92291fb640 1 -- 172.21.15.17:0/1972090380 reap_dead start 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 0x7f922411cf60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f921c00c4d0 tx=0x7f921c00c990 comp rx=0 tx=0).stop 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 shutdown_connections 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 --2- 172.21.15.17:0/1972090380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9224135670 0x7f922411cf60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 >> 172.21.15.17:0/1972090380 conn(0x7f92240a66a0 msgr2=0x7f9224072c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 shutdown_connections 2024-09-15T21:27:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:27:42.722+0000 7f922ac5e640 1 -- 172.21.15.17:0/1972090380 wait complete. 2024-09-15T21:27:42.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:27:42.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:28:02.760 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T21:28:02.761 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:28:02.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:28:02.765 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:28:02.765 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:28:02.765 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:02.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient: get_monmap_and_config 2024-09-15T21:28:02.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient: build_initial_monmap 2024-09-15T21:28:02.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient: monmap: 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:02.887634+0000 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:02.887634+0000 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:02.888 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:02.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 1 Processor -- start 2024-09-15T21:28:02.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 1 -- start start 2024-09-15T21:28:02.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient: init 2024-09-15T21:28:02.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:02.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2024-09-15T21:28:02.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa0a409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:02.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0a409def0 0x7fa0a4139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:02.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient(hunting): picked mon.noname-a con 0x7fa0a409def0 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:02.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa0a4139ff0 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:02.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a40a3130 con 0x7fa0a4139ff0 2024-09-15T21:28:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a409e7a0 con 0x7fa0a409d400 2024-09-15T21:28:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40a1bc0 con 0x7fa0a409def0 2024-09-15T21:28:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a9f34640 10 monclient(hunting): authenticate will time out at 1606.282739s 2024-09-15T21:28:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0a409def0 0x7fa0a4139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0a409def0 0x7fa0a4139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49298/0 (socket says 172.21.15.17:49298) 2024-09-15T21:28:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 -- 172.21.15.17:0/3198167279 learned_addr learned my addr 172.21.15.17:0/3198167279 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request con 0x7fa0a409def0 auth_method 0 2024-09-15T21:28:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_done global_id 124200 payload 274 2024-09-15T21:28:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 -- 172.21.15.17:0/3198167279 >> v1:172.21.15.119:6789/0 conn(0x7fa0a409d400 legacy=0x7fa0a409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 -- 172.21.15.17:0/3198167279 >> v1:172.21.15.17:6789/0 conn(0x7fa0a4139ff0 legacy=0x7fa0a413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 monclient: found mon.noname-a 2024-09-15T21:28:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.886+0000 7fa0a2ffd640 1 -- 172.21.15.17:0/3198167279 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40a0320 con 0x7fa0a409def0 2024-09-15T21:28:02.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a2ffd640 1 --2- 172.21.15.17:0/3198167279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409def0 0x7fa0a4139ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444dca0b0ce154d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:02.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 1 -- 172.21.15.17:0/3198167279 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0b0070 con 0x7fa0a409def0 2024-09-15T21:28:02.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:02.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:02.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: dump: 2024-09-15T21:28:02.912 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:02.912 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:02.912 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.912 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.912 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:02.913 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: _finish_auth 0 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: _check_auth_tickets 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 1 -- 172.21.15.17:0/3198167279 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa08c0ab600 con 0x7fa0a409def0 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 1 -- 172.21.15.17:0/3198167279 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0ab9d0 con 0x7fa0a409def0 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 5 monclient: authenticate success, global_id 124200 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 4 set_mon_vals no callback set 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a27fc640 10 monclient: dump: 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: shutdown 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- 172.21.15.17:0/3198167279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409def0 msgr2=0x7fa0a4139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/3198167279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409def0 0x7fa0a4139ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c01c420 comp rx=0 tx=0).stop 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- 172.21.15.17:0/3198167279 shutdown_connections 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/3198167279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409def0 0x7fa0a4139ab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- 172.21.15.17:0/3198167279 >> 172.21.15.17:0/3198167279 conn(0x7fa0a40a66a0 msgr2=0x7fa0a40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- 172.21.15.17:0/3198167279 shutdown_connections 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- 172.21.15.17:0/3198167279 wait complete. 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: build_initial_monmap 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: monmap: 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 Processor -- start 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- start start 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: init 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 0x7fa0a411f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): picked mon.noname-a con 0x7fa0a409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a409def0 0x7fa0a411cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): picked mon.noname-b con 0x7fa0a409def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 0x7fa0a4133f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): picked mon.noname-c con 0x7fa0a4139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40a3490 con 0x7fa0a409d400 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4140940 con 0x7fa0a409def0 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a4140ac0 con 0x7fa0a4139ff0 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.890+0000 7fa0a9f34640 10 monclient(hunting): authenticate will time out at 1606.287945s 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 0x7fa0a411f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 0x7fa0a411f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49308/0 (socket says 172.21.15.17:49308) 2024-09-15T21:28:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 0x7fa0a4133f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 -- 172.21.15.17:0/1559834420 learned_addr learned my addr 172.21.15.17:0/1559834420 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): get_auth_request con 0x7fa0a409d400 auth_method 0 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 10 monclient(hunting): get_auth_request con 0x7fa0a4139ff0 auth_method 0 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 1 -- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 msgr2=0x7fa0a4133f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 1 -- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 msgr2=0x7fa0a4133f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 0x7fa0a4133f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a409def0 0x7fa0a411cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a3fff640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 0x7fa0a4133f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request con 0x7fa0a409def0 auth_method 0 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_done global_id 124203 payload 824 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 -- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 msgr2=0x7fa0a4133f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 0x7fa0a4133f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 -- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a409def0 msgr2=0x7fa0a411cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a409def0 0x7fa0a411cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 monclient: found mon.noname-a 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 -- 172.21.15.17:0/1559834420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a411e5a0 con 0x7fa0a409d400 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a409def0 0x7fa0a411cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a37fe640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 0x7fa0a411f100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c530 tx=0x7fa09400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094017070 con 0x7fa0a409d400 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: dump: 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: _finish_auth 0 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0940052d0 con 0x7fa0a409d400 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 5 monclient: authenticate success, global_id 124203 2024-09-15T21:28:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 monclient: _renew_subs 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a4120060 con 0x7fa0a409d400 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094012870 con 0x7fa0a409d400 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 10 monclient: dump: 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 monclient: _renew_subs 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a4134490 con 0x7fa0a409d400 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a1ffb640 4 set_mon_vals no callback set 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a4120450 con 0x7fa0a409d400 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa09401a070 con 0x7fa0a409d400 2024-09-15T21:28:02.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa08403b9f0 0x7fa08403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa09404f840 con 0x7fa0a409d400 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa08403b9f0 0x7fa08403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 10 monclient: get_auth_request con 0x7fa08403b9f0 auth_method 0 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.894+0000 7fa0a2ffd640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa08403b9f0 0x7fa08403deb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ad000 tx=0x7fa08c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.898+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa094003d10 con 0x7fa0a409d400 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.898+0000 7fa09bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:02.898+0000 7fa09bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.130+0000 7fa0a9f34640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.130+0000 7fa0a9f34640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.130+0000 7fa0a9f34640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.130+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa0a4120230 con 0x7fa0a409d400 2024-09-15T21:28:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa09bfff640 1 -- 172.21.15.17:0/1559834420 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fa094019cd0 con 0x7fa0a409d400 2024-09-15T21:28:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa09bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa09bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:03.134 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa08403b9f0 msgr2=0x7fa08403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa08403b9f0 0x7fa08403deb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ad000 tx=0x7fa08c008040 comp rx=0 tx=0).stop 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 10 monclient: shutdown 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 msgr2=0x7fa0a411f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 0x7fa0a411f100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c530 tx=0x7fa09400c9f0 comp rx=0 tx=0).stop 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 shutdown_connections 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa08403b9f0 0x7fa08403deb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa0a4139ff0 0x7fa0a4133f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a409def0 0x7fa0a411cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 --2- 172.21.15.17:0/1559834420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0a409d400 0x7fa0a411f100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 >> 172.21.15.17:0/1559834420 conn(0x7fa0a40a66a0 msgr2=0x7fa0a413cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 shutdown_connections 2024-09-15T21:28:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:03.134+0000 7fa0a9f34640 1 -- 172.21.15.17:0/1559834420 wait complete. 2024-09-15T21:28:06.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:06.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient: get_monmap_and_config 2024-09-15T21:28:06.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient: build_initial_monmap 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient: monmap: 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:06.284189+0000 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:06.284189+0000 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:06.285 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:06.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 Processor -- start 2024-09-15T21:28:06.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 -- start start 2024-09-15T21:28:06.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient: init 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): picked mon.noname-b con 0x7f59d8073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59d8073f60 0x7f59d8137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): picked mon.noname-c con 0x7f59d8073f60 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f59d8137bd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d80a0890 con 0x7f59d8137bd0 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f59d80746e0 con 0x7f59d8073f60 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f59d80a1640 con 0x7f59d8073640 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59dd775640 10 monclient(hunting): authenticate will time out at 1609.679593s 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37524/0 (socket says 172.21.15.17:37524) 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 1 -- 172.21.15.17:0/2953079864 learned_addr learned my addr 172.21.15.17:0/2953079864 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 10 monclient(hunting): get_auth_request con 0x7f59d8073640 auth_method 0 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 1 -- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 msgr2=0x7f59d8073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 1 -- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 msgr2=0x7f59d8073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d6ffd640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.282+0000 7f59d67fc640 1 --2- 172.21.15.17:0/2953079864 >> v2:172.21.15.119:3300/0 conn(0x7f59d8073f60 0x7f59d8137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d6ffd640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): get_auth_request con 0x7f59d8073f60 auth_method 0 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient(hunting): handle_auth_done global_id 114256 payload 274 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 1 -- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 msgr2=0x7f59d8073a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 1 -- 172.21.15.17:0/2953079864 >> v1:172.21.15.119:6789/0 conn(0x7f59d8137bd0 legacy=0x7f59d813ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 1 monclient: found mon.noname-c 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:28:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 1 -- 172.21.15.17:0/2953079864 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d80a1920 con 0x7f59d8073f60 2024-09-15T21:28:06.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d67fc640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d8137690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0097c0 tx=0x7f59cc01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28085e4ddaeae14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:06.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 1 -- 172.21.15.17:0/2953079864 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0b0070 con 0x7f59d8073f60 2024-09-15T21:28:06.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: dump: 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:06.296 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 5 monclient: authenticate success, global_id 114256 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 1 -- 172.21.15.17:0/2953079864 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59cc0ab600 con 0x7f59d8073f60 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 1 -- 172.21.15.17:0/2953079864 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0ab9d0 con 0x7f59d8073f60 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 4 set_mon_vals no callback set 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59d5ffb640 10 monclient: dump: 2024-09-15T21:28:06.297 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 10 monclient: shutdown 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 -- 172.21.15.17:0/2953079864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 msgr2=0x7f59d8137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:06.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d8137690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0097c0 tx=0x7f59cc01c3f0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 -- 172.21.15.17:0/2953079864 shutdown_connections 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d8137690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 --2- 172.21.15.17:0/2953079864 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8073640 0x7f59d8073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 -- 172.21.15.17:0/2953079864 >> 172.21.15.17:0/2953079864 conn(0x7f59d80a66a0 msgr2=0x7f59d80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 -- 172.21.15.17:0/2953079864 shutdown_connections 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.286+0000 7f59dd775640 1 -- 172.21.15.17:0/2953079864 wait complete. 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: build_initial_monmap 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: monmap: 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 Processor -- start 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- start start 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: init 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 0x7f59d812e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): picked mon.noname-a con 0x7f59d8073640 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d812eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:06.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): picked mon.noname-b con 0x7f59d8073f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): picked mon.noname-c con 0x7f59d8137bd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f59d809ce50 con 0x7f59d8073640 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59d809ccd0 con 0x7f59d8073f60 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f59d80a0d00 con 0x7f59d8137bd0 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient(hunting): authenticate will time out at 1609.685332s 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 0x7f59d812e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:06.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37532/0 (socket says 172.21.15.17:37532) 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 -- 172.21.15.17:0/1800813082 learned_addr learned my addr 172.21.15.17:0/1800813082 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 0x7f59d812e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49370/0 (socket says 172.21.15.17:49370) 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 10 monclient(hunting): get_auth_request con 0x7f59d8137bd0 auth_method 0 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): get_auth_request con 0x7f59d8073640 auth_method 0 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 -- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 msgr2=0x7f59d8138be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:06.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 -- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 msgr2=0x7f59d8138be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d77fe640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d67fc640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d812eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient(hunting): handle_auth_done global_id 124206 payload 824 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 -- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 msgr2=0x7f59d8138be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 -- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 msgr2=0x7f59d812eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d812eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 monclient: found mon.noname-a 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:06.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 -- 172.21.15.17:0/1800813082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d813c660 con 0x7f59d8073640 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d67fc640 10 monclient: get_auth_request con 0x7f59d8073f60 auth_method 0 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d67fc640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d812eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d6ffd640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 0x7f59d812e860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59d0007af0 tx=0x7f59d000d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d001a070 con 0x7f59d8073640 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: dump: 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.304 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 5 monclient: authenticate success, global_id 124206 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _renew_subs 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d000dd30 con 0x7f59d8073640 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d813c940 con 0x7f59d8073640 2024-09-15T21:28:06.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0015640 con 0x7f59d8073640 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 10 monclient: dump: 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:06.306 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _renew_subs 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d813a5e0 con 0x7f59d8073640 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59d4ff9640 4 set_mon_vals no callback set 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d813d360 con 0x7f59d8073640 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f59d001d070 con 0x7f59d8073640 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59c403b9c0 0x7f59c403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.290+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f59d0050640 con 0x7f59d8073640 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.294+0000 7f59d67fc640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59c403b9c0 0x7f59c403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.294+0000 7f59d67fc640 10 monclient: get_auth_request con 0x7f59c403b9c0 auth_method 0 2024-09-15T21:28:06.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.294+0000 7f59d67fc640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59c403b9c0 0x7f59c403de80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0ad000 tx=0x7f59cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:06.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.298+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f59d0010070 con 0x7f59d8073640 2024-09-15T21:28:06.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.298+0000 7f59bf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:06.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.298+0000 7f59bf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:06.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59dd775640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59dd775640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59dd775640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f59d8073a20 con 0x7f59d8073640 2024-09-15T21:28:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59bf7fe640 1 -- 172.21.15.17:0/1800813082 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f59d001ed50 con 0x7f59d8073640 2024-09-15T21:28:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59bf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.534+0000 7f59bf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:06.536 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59c403b9c0 msgr2=0x7f59c403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59c403b9c0 0x7f59c403de80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0ad000 tx=0x7f59cc008040 comp rx=0 tx=0).stop 2024-09-15T21:28:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 10 monclient: shutdown 2024-09-15T21:28:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 msgr2=0x7f59d812e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 0x7f59d812e860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59d0007af0 tx=0x7f59d000d0d0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 shutdown_connections 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f59c403b9c0 0x7f59c403de80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f59d8137bd0 0x7f59d8138be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d8073f60 0x7f59d812eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 --2- 172.21.15.17:0/1800813082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59d8073640 0x7f59d812e860 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 >> 172.21.15.17:0/1800813082 conn(0x7f59d80a66a0 msgr2=0x7f59d8131830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 shutdown_connections 2024-09-15T21:28:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:06.538+0000 7f59dd775640 1 -- 172.21.15.17:0/1800813082 wait complete. 2024-09-15T21:28:09.557 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient: get_monmap_and_config 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient: build_initial_monmap 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient: monmap: 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:09.691350+0000 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:09.691350+0000 2024-09-15T21:28:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:09.692 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 1 Processor -- start 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 1 -- start start 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient: init 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0b48073640 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0b48074130 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b4813e320 0x7f0b48140710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient(hunting): picked mon.noname-c con 0x7f0b4813e320 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480a0890 con 0x7f0b48073640 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480746e0 con 0x7f0b48074130 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0b480a1640 con 0x7f0b4813e320 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 10 monclient(hunting): authenticate will time out at 1613.086824s 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b47fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0b48073640 0x7f0b48073a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41382/0 (socket says 172.21.15.17:41382) 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b47fff640 1 -- 172.21.15.17:0/2940342435 learned_addr learned my addr 172.21.15.17:0/2940342435 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 1 --2- 172.21.15.17:0/2940342435 >> v2:172.21.15.119:3300/0 conn(0x7f0b4813e320 0x7f0b48140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): get_auth_request con 0x7f0b4813e320 auth_method 0 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551070089 0 0) 0x7f0b480a0890 con 0x7f0b48073640 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): my global_id is 124209 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b34003530 con 0x7f0b48073640 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 758441770 0 0) 0x7f0b34003530 con 0x7f0b48073640 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 >> v2:172.21.15.119:3300/0 conn(0x7f0b4813e320 msgr2=0x7f0b48140710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 --2- 172.21.15.17:0/2940342435 >> v2:172.21.15.119:3300/0 conn(0x7f0b4813e320 0x7f0b48140710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 >> v1:172.21.15.119:6789/0 conn(0x7f0b48074130 legacy=0x7f0b4813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b480a1920 con 0x7f0b48073640 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: _finish_auth 0 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f0b30001700 con 0x7f0b48073640 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 5 monclient: authenticate success, global_id 124209 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b4e85f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: dump: 2024-09-15T21:28:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 >> v1:172.21.15.17:6789/0 conn(0x7f0b48073640 legacy=0x7f0b48073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34003a60 0x7f0b34003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): picked mon.b con 0x7f0b34003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b34004810 0x7f0b34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): picked mon.c con 0x7f0b34004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b340053d0 0x7f0b3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.690+0000 7f0b46ffd640 10 monclient(hunting): picked mon.a con 0x7f0b340053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0b480a1640 con 0x7f0b340053d0 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3400c240 con 0x7f0b34003a60 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3400c440 con 0x7f0b34004810 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b34004810 0x7f0b34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b340053d0 0x7f0b3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): get_auth_request con 0x7f0b34004810 auth_method 0 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): get_auth_request con 0x7f0b340053d0 auth_method 0 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34003a60 0x7f0b34003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): get_auth_request con 0x7f0b34003a60 auth_method 0 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b47fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient(hunting): handle_auth_done global_id 124209 payload 293 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 -- 172.21.15.17:0/2940342435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34003a60 msgr2=0x7f0b34003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34003a60 0x7f0b34003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 -- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b340053d0 msgr2=0x7f0b3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b340053d0 0x7f0b3400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 monclient: found mon.c 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 -- 172.21.15.17:0/2940342435 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b34003530 con 0x7f0b34004810 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient: _finish_auth 0 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 10 monclient: _check_auth_tickets 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b340053d0 0x7f0b3400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4cdfc640 1 -- 172.21.15.17:0/2940342435 reap_dead start 2024-09-15T21:28:09.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b477fe640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b34004810 0x7f0b34004c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b38001790 tx=0x7f0b3801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd12f6e71bd748ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:09.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b380013e0 con 0x7f0b34004810 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: dump: 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b380052d0 con 0x7f0b34004810 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 1 -- 172.21.15.17:0/2940342435 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38003840 con 0x7f0b34004810 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 4 set_mon_vals no callback set 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b46ffd640 10 monclient: dump: 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 10 monclient: shutdown 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 -- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b34004810 msgr2=0x7f0b34004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b34004810 0x7f0b34004c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b38001790 tx=0x7f0b3801c400 comp rx=0 tx=0).stop 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 -- 172.21.15.17:0/2940342435 shutdown_connections 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/2940342435 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b34004810 0x7f0b34004c90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 -- 172.21.15.17:0/2940342435 >> 172.21.15.17:0/2940342435 conn(0x7f0b480a66a0 msgr2=0x7f0b480a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 -- 172.21.15.17:0/2940342435 shutdown_connections 2024-09-15T21:28:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.694+0000 7f0b4e85f640 1 -- 172.21.15.17:0/2940342435 wait complete. 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: build_initial_monmap 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: monmap: 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 Processor -- start 2024-09-15T21:28:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- start start 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: init 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34004810 0x7f0b48122940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): picked mon.noname-b con 0x7f0b34004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b48120830 0x7f0b48136780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): picked mon.noname-a con 0x7f0b48120830 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 0x7f0b481cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): picked mon.noname-c con 0x7f0b48136cc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0b480a0e80 con 0x7f0b48120830 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b480a0d00 con 0x7f0b34004810 2024-09-15T21:28:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0b48137ea0 con 0x7f0b48136cc0 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient(hunting): authenticate will time out at 1613.093092s 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 0x7f0b481cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b48120830 0x7f0b48136780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 0x7f0b481cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44462/0 (socket says 172.21.15.17:44462) 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 -- 172.21.15.17:0/1348241155 learned_addr learned my addr 172.21.15.17:0/1348241155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): get_auth_request con 0x7f0b48136cc0 auth_method 0 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): get_auth_request con 0x7f0b48120830 auth_method 0 2024-09-15T21:28:09.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34004810 0x7f0b48122940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 10 monclient(hunting): get_auth_request con 0x7f0b34004810 auth_method 0 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:09.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient(hunting): handle_auth_done global_id 134099 payload 824 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 -- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34004810 msgr2=0x7f0b48122940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34004810 0x7f0b48122940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 -- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b48120830 msgr2=0x7f0b48136780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b48120830 0x7f0b48136780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient: _finish_hunting 0 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 monclient: found mon.noname-c 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 -- 172.21.15.17:0/1348241155 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b48121de0 con 0x7f0b48136cc0 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b477fe640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b48120830 0x7f0b48136780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4cdfc640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 0x7f0b481cfbd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b480a3560 tx=0x7f0b4000f580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40009c10 con 0x7f0b48136cc0 2024-09-15T21:28:09.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: dump: 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: _finish_auth 0 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:28:09.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b400039a0 con 0x7f0b48136cc0 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 5 monclient: authenticate success, global_id 134099 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _renew_subs 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b48137130 con 0x7f0b48136cc0 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40003d70 con 0x7f0b48136cc0 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 10 monclient: dump: 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _renew_subs 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b467fc640 4 set_mon_vals no callback set 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b481238a0 con 0x7f0b48136cc0 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b481381d0 con 0x7f0b48136cc0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0b40003f10 con 0x7f0b48136cc0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2803b9c0 0x7f0b2803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0b40051400 con 0x7f0b48136cc0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2803b9c0 0x7f0b2803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.698+0000 7f0b47fff640 10 monclient: get_auth_request con 0x7f0b2803b9c0 auth_method 0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.702+0000 7f0b47fff640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2803b9c0 0x7f0b2803de80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0b30000fc0 tx=0x7f0b30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.702+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0b48137130 con 0x7f0b48136cc0 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.702+0000 7f0b44ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:09.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.702+0000 7f0b44ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:09.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:09.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:09.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:09.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0b48120dd0 con 0x7f0b48136cc0 2024-09-15T21:28:09.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b44ff9640 1 -- 172.21.15.17:0/1348241155 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f0b40019030 con 0x7f0b48136cc0 2024-09-15T21:28:09.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b44ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:09.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b44ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:09.943 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2803b9c0 msgr2=0x7f0b2803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2803b9c0 0x7f0b2803de80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0b30000fc0 tx=0x7f0b30009040 comp rx=0 tx=0).stop 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 10 monclient: shutdown 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 msgr2=0x7f0b481cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 0x7f0b481cfbd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b480a3560 tx=0x7f0b4000f580 comp rx=0 tx=0).stop 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 shutdown_connections 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0b2803b9c0 0x7f0b2803de80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0b48136cc0 0x7f0b481cfbd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b48120830 0x7f0b48136780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.942+0000 7f0b4e85f640 1 --2- 172.21.15.17:0/1348241155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b34004810 0x7f0b48122940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:09.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.946+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 >> 172.21.15.17:0/1348241155 conn(0x7f0b480a66a0 msgr2=0x7f0b4809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:09.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.946+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 shutdown_connections 2024-09-15T21:28:09.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:09.946+0000 7f0b4e85f640 1 -- 172.21.15.17:0/1348241155 wait complete. 2024-09-15T21:28:09.963 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:28:09.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:28:09.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient: get_monmap_and_config 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient: monmap: 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:10.091036+0000 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:10.091036+0000 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:10.092 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 Processor -- start 2024-09-15T21:28:10.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 -- start start 2024-09-15T21:28:10.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient: init 2024-09-15T21:28:10.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:28:10.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a440a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): picked mon.noname-b con 0x7f0a440a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a4409ffc0 0x7f0a440a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): picked mon.noname-a con 0x7f0a4409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0a44135670 0x7f0a44135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): picked mon.noname-c con 0x7f0a44135670 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0a440a16f0 con 0x7f0a4409ffc0 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0a440a3130 con 0x7f0a44135670 2024-09-15T21:28:10.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0a44073df0 con 0x7f0a440a0890 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a4a268640 10 monclient(hunting): authenticate will time out at 1613.486417s 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a4409ffc0 0x7f0a440a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a440a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a4409ffc0 0x7f0a440a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37018/0 (socket says 172.21.15.17:37018) 2024-09-15T21:28:10.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 -- 172.21.15.17:0/823082866 learned_addr learned my addr 172.21.15.17:0/823082866 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a440a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44472/0 (socket says 172.21.15.17:44472) 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): get_auth_request con 0x7f0a4409ffc0 auth_method 0 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): get_auth_request con 0x7f0a440a0890 auth_method 0 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a48805640 1 --2- 172.21.15.17:0/823082866 >> v2:172.21.15.119:3300/0 conn(0x7f0a44135670 0x7f0a44135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a48805640 10 monclient(hunting): get_auth_request con 0x7f0a44135670 auth_method 0 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a48805640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a48805640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a48805640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient(hunting): handle_auth_done global_id 124215 payload 274 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 -- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 msgr2=0x7f0a440a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:10.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 --2- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a440a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 -- 172.21.15.17:0/823082866 >> v2:172.21.15.119:3300/0 conn(0x7f0a44135670 msgr2=0x7f0a44135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 --2- 172.21.15.17:0/823082866 >> v2:172.21.15.119:3300/0 conn(0x7f0a44135670 0x7f0a44135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a437fe640 1 -- 172.21.15.17:0/823082866 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a440a1bf0 con 0x7f0a4409ffc0 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.089+0000 7f0a43fff640 1 --2- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a440a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a437fe640 1 --2- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a440a11b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a4406f1c0 tx=0x7f0a2c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becd7e32cefd64a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 1 -- 172.21.15.17:0/823082866 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0a5640 con 0x7f0a4409ffc0 2024-09-15T21:28:10.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: dump: 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.101 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 1 -- 172.21.15.17:0/823082866 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c0a57e0 con 0x7f0a4409ffc0 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 1 -- 172.21.15.17:0/823082866 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0a5bb0 con 0x7f0a4409ffc0 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a42ffd640 10 monclient: dump: 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 5 monclient: authenticate success, global_id 124215 2024-09-15T21:28:10.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:10.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 4 set_mon_vals no callback set 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: shutdown 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 msgr2=0x7f0a440a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a440a11b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a4406f1c0 tx=0x7f0a2c01c8b0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- 172.21.15.17:0/823082866 shutdown_connections 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- 172.21.15.17:0/823082866 >> v2:172.21.15.119:3300/0 conn(0x7f0a44135670 0x7f0a44135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a440a11b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- 172.21.15.17:0/823082866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a440a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- 172.21.15.17:0/823082866 >> 172.21.15.17:0/823082866 conn(0x7f0a440a66a0 msgr2=0x7f0a440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- 172.21.15.17:0/823082866 shutdown_connections 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- 172.21.15.17:0/823082866 wait complete. 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: monmap: 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 Processor -- start 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- start start 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: init 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a441345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): picked mon.noname-a con 0x7f0a4409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:10.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a4411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): picked mon.noname-c con 0x7f0a440a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a44135670 0x7f0a4411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): picked mon.noname-b con 0x7f0a44135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4413f680 con 0x7f0a4409ffc0 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4413f500 con 0x7f0a44135670 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4413f800 con 0x7f0a440a0890 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a4a268640 10 monclient(hunting): authenticate will time out at 1613.491835s 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a441345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a437fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a4411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a441345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37022/0 (socket says 172.21.15.17:37022) 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 1 -- 172.21.15.17:0/2961626385 learned_addr learned my addr 172.21.15.17:0/2961626385 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 10 monclient(hunting): get_auth_request con 0x7f0a4409ffc0 auth_method 0 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a437fe640 10 monclient(hunting): get_auth_request con 0x7f0a440a0890 auth_method 0 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.093+0000 7f0a437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a44135670 0x7f0a4411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 10 monclient(hunting): get_auth_request con 0x7f0a44135670 auth_method 0 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 10 monclient(hunting): handle_auth_done global_id 124218 payload 824 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 msgr2=0x7f0a4411c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a4411c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a44135670 msgr2=0x7f0a4411ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a44135670 0x7f0a4411ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4409d0f0 con 0x7f0a4409ffc0 2024-09-15T21:28:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0a440a0890 0x7f0a4411c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a43fff640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a441345a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00ebe0 tx=0x7f0a3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c017070 con 0x7f0a4409ffc0 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: dump: 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a3c00cc30 con 0x7f0a4409ffc0 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 5 monclient: authenticate success, global_id 124218 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 10 monclient: _renew_subs 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a44135340 con 0x7f0a4409ffc0 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c013470 con 0x7f0a4409ffc0 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: dump: 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a427fc640 4 set_mon_vals no callback set 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 10 monclient: _renew_subs 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4411e740 con 0x7f0a4409ffc0 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 0x7f0a4402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0a440a5350 con 0x7f0a44134ae0 2024-09-15T21:28:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 0x7f0a4402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient: get_auth_request con 0x7f0a44134ae0 auth_method 0 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0a3c01a070 con 0x7f0a4409ffc0 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 10 monclient(hunting): handle_auth_done global_id 124221 payload 824 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a44072fe0 con 0x7f0a44134ae0 2024-09-15T21:28:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a2403b9c0 0x7f0a2403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0a3c0505c0 con 0x7f0a4409ffc0 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a437fe640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 0x7f0a4402d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a4406e7b0 tx=0x7f0a2c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009510 con 0x7f0a44134ae0 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: dump: 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a2403b9c0 0x7f0a2403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 10 monclient: get_auth_request con 0x7f0a2403b9c0 auth_method 0 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0a2c0a5640 con 0x7f0a44134ae0 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 msgr2=0x7f0a4402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 0x7f0a4402d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a4406e7b0 tx=0x7f0a2c019650 comp rx=0 tx=0).stop 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 msgr2=0x7f0a4402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a40ff9640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a44134ae0 0x7f0a4402d450 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a4406e7b0 tx=0x7f0a2c019650 comp rx=0 tx=0).stop 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.097+0000 7f0a48805640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a2403b9c0 0x7f0a2403de80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400d8d0 tx=0x7f0a34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 0x7f0a4402dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44135aa0 con 0x7f0a4402d990 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 0x7f0a4402dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient: get_auth_request con 0x7f0a4402d990 auth_method 0 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 10 monclient(hunting): handle_auth_done global_id 124224 payload 824 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 1 -- 172.21.15.17:0/2961626385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0a441391c0 con 0x7f0a4402d990 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a437fe640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 0x7f0a4402dd70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000f90 tx=0x7f0a2c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0ca640 con 0x7f0a4402d990 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 10 monclient: dump: 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4881 (secure 0 0 0) 0x7f0a2c009af0 con 0x7f0a4402d990 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 msgr2=0x7f0a4402dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 0x7f0a4402dd70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000f90 tx=0x7f0a2c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 msgr2=0x7f0a4402dd70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a40ff9640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4402d990 0x7f0a4402dd70 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000f90 tx=0x7f0a2c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 53055, 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:28:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:10.144 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:10.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a2403b9c0 msgr2=0x7f0a2403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0a2403b9c0 0x7f0a2403de80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400d8d0 tx=0x7f0a34008040 comp rx=0 tx=0).stop 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 10 monclient: shutdown 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 msgr2=0x7f0a441345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 --2- 172.21.15.17:0/2961626385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a4409ffc0 0x7f0a441345a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00ebe0 tx=0x7f0a3c00c520 comp rx=0 tx=0).stop 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a48805640 1 -- 172.21.15.17:0/2961626385 reap_dead start 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a48805640 1 -- 172.21.15.17:0/2961626385 reap_dead start 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 shutdown_connections 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 >> 172.21.15.17:0/2961626385 conn(0x7f0a440a66a0 msgr2=0x7f0a440a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 shutdown_connections 2024-09-15T21:28:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.105+0000 7f0a4a268640 1 -- 172.21.15.17:0/2961626385 wait complete. 2024-09-15T21:28:10.147 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:28:10.147 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:28:10.147 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient: get_monmap_and_config 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient: monmap: 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:10.271346+0000 2024-09-15T21:28:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:10.271346+0000 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:10.272 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 Processor -- start 2024-09-15T21:28:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 -- start start 2024-09-15T21:28:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient: init 2024-09-15T21:28:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2024-09-15T21:28:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe8ec09d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ec09def0 0x7fe8ec139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): picked mon.noname-a con 0x7fe8ec09def0 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec13c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): picked mon.noname-b con 0x7fe8ec139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8ec0a3130 con 0x7fe8ec09d400 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe8ec0a1bc0 con 0x7fe8ec09def0 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec0a3490 con 0x7fe8ec139ff0 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8f20f1640 10 monclient(hunting): authenticate will time out at 1613.667236s 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ec09def0 0x7fe8ec139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec13c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ec09def0 0x7fe8ec139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37052/0 (socket says 172.21.15.17:37052) 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4191993811 learned_addr learned my addr 172.21.15.17:0/4191993811 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): get_auth_request con 0x7fe8ec09def0 auth_method 0 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): get_auth_request con 0x7fe8ec139ff0 auth_method 0 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_done global_id 124227 payload 274 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 msgr2=0x7fe8ec13c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec13c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4191993811 >> v1:172.21.15.119:6789/0 conn(0x7fe8ec09d400 legacy=0x7fe8ec09d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4191993811 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec09e7a0 con 0x7fe8ec09def0 2024-09-15T21:28:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.269+0000 7fe8ebfff640 1 --2- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec13c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:10.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09def0 0x7fe8ec139ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e10c92b47cdd6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/4191993811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00b0070 con 0x7fe8ec09def0 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: dump: 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/4191993811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e00a4e50 con 0x7fe8ec09def0 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/4191993811 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00ab820 con 0x7fe8ec09def0 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 5 monclient: authenticate success, global_id 124227 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 4 set_mon_vals no callback set 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8ea7fc640 10 monclient: dump: 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 10 monclient: shutdown 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09def0 msgr2=0x7fe8ec139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09def0 0x7fe8ec139ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e001c3c0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4191993811 shutdown_connections 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec13c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4191993811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09def0 0x7fe8ec139ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4191993811 >> 172.21.15.17:0/4191993811 conn(0x7fe8ec0a66a0 msgr2=0x7fe8ec0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4191993811 shutdown_connections 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4191993811 wait complete. 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 10 monclient: monmap: 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 Processor -- start 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 1 -- start start 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.273+0000 7fe8f20f1640 10 monclient: init 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09d400 0x7fe8ec11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): picked mon.noname-a con 0x7fe8ec09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ec09def0 0x7fe8ec11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): picked mon.noname-b con 0x7fe8ec09def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec11cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): picked mon.noname-c con 0x7fe8ec139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec0a0890 con 0x7fe8ec09d400 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec0a0a10 con 0x7fe8ec09def0 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec0a0b90 con 0x7fe8ec139ff0 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): authenticate will time out at 1613.672406s 2024-09-15T21:28:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09d400 0x7fe8ec11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09d400 0x7fe8ec11c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37054/0 (socket says 172.21.15.17:37054) 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 -- 172.21.15.17:0/4055698276 learned_addr learned my addr 172.21.15.17:0/4055698276 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): get_auth_request con 0x7fe8ec09d400 auth_method 0 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec11cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): get_auth_request con 0x7fe8ec139ff0 auth_method 0 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ec09def0 0x7fe8ec11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): get_auth_request con 0x7fe8ec09def0 auth_method 0 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient(hunting): handle_auth_done global_id 124230 payload 824 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 msgr2=0x7fe8ec11cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec139ff0 0x7fe8ec11cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ec09def0 msgr2=0x7fe8ec11ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ec09def0 0x7fe8ec11ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec11f9c0 con 0x7fe8ec09d400 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eb7fe640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09d400 0x7fe8ec11c4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c4e0 tx=0x7fe8dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc017070 con 0x7fe8ec09d400 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: dump: 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc0052d0 con 0x7fe8ec09d400 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003820 con 0x7fe8ec09d400 2024-09-15T21:28:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: dump: 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 5 monclient: authenticate success, global_id 124230 2024-09-15T21:28:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient: _renew_subs 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec0a5860 con 0x7fe8ec09d400 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8e9ffb640 4 set_mon_vals no callback set 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient: _renew_subs 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec11eb50 con 0x7fe8ec09d400 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 0x7fe8ec06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec09fc10 con 0x7fe8ec06f230 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 0x7fe8ec06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient: get_auth_request con 0x7fe8ec06f230 auth_method 0 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe8dc01a070 con 0x7fe8ec09d400 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_done global_id 134114 payload 824 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8ec09d7e0 con 0x7fe8ec06f230 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8d803b9f0 0x7fe8d803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe8dc04f8b0 con 0x7fe8ec09d400 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 0x7fe8ec06f610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09d0a0 tx=0x7fe8e001ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00ba030 con 0x7fe8ec06f230 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: dump: 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8d803b9f0 0x7fe8d803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 10 monclient: get_auth_request con 0x7fe8d803b9f0 auth_method 0 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe8e00a4e50 con 0x7fe8ec06f230 2024-09-15T21:28:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 msgr2=0x7fe8ec06f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 0x7fe8ec06f610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09d0a0 tx=0x7fe8e001ac60 comp rx=0 tx=0).stop 2024-09-15T21:28:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 msgr2=0x7fe8ec06f610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8d3fff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06f230 0x7fe8ec06f610 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09d0a0 tx=0x7fe8e001ac60 comp rx=0 tx=0).stop 2024-09-15T21:28:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.277+0000 7fe8ebfff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8d803b9f0 0x7fe8d803deb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec11e030 tx=0x7fe8e400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8f20f1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:10.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8f20f1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec06d020 con 0x7fe8ec06fb50 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient: get_auth_request con 0x7fe8ec06fb50 auth_method 0 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 10 monclient(hunting): handle_auth_done global_id 134117 payload 824 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4055698276 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe8ec132330 con 0x7fe8ec06fb50 2024-09-15T21:28:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 msgr2=0x7fe8ec1d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:28:10.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 msgr2=0x7fe8ec1d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:10.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fe8e001c400 tx=0x7fe8e00191c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:10.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.285+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:10.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: get_auth_request con 0x7fe8ec06fb50 auth_method 0 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: _init_auth method 2 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: _init_auth already have auth, reseting 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: handle_auth_reply_more payload 9 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 10 monclient: handle_auth_done global_id 134117 payload 293 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8eaffd640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0019030 tx=0x7fe8e00b6000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00ba040 con 0x7fe8ec06fb50 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 10 monclient: dump: 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7fe8e00c0d00 con 0x7fe8ec06fb50 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 msgr2=0x7fe8ec1d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0019030 tx=0x7fe8e00b6000 comp rx=0 tx=0).stop 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 msgr2=0x7fe8ec1d9f60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8d3fff640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8ec06fb50 0x7fe8ec1d9f60 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0019030 tx=0x7fe8e00b6000 comp rx=0 tx=0).stop 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7643, 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:10.509 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8d803b9f0 msgr2=0x7fe8d803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe8d803b9f0 0x7fe8d803deb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec11e030 tx=0x7fe8e400b040 comp rx=0 tx=0).stop 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 10 monclient: shutdown 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09d400 msgr2=0x7fe8ec11c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 --2- 172.21.15.17:0/4055698276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ec09d400 0x7fe8ec11c4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c4e0 tx=0x7fe8dc00c8e0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8ebfff640 1 -- 172.21.15.17:0/4055698276 reap_dead start 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8ebfff640 1 -- 172.21.15.17:0/4055698276 reap_dead start 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 shutdown_connections 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 >> 172.21.15.17:0/4055698276 conn(0x7fe8ec0a66a0 msgr2=0x7fe8ec134d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 shutdown_connections 2024-09-15T21:28:10.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.489+0000 7fe8f20f1640 1 -- 172.21.15.17:0/4055698276 wait complete. 2024-09-15T21:28:10.516 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:28:10.516 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:28:10.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:28:10.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient: get_monmap_and_config 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient: monmap: 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:10.646974+0000 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:10.646974+0000 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:10.647 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:10.648 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:10.648 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 Processor -- start 2024-09-15T21:28:10.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 -- start start 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient: init 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdac40a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdac409ffc0 0x7fdac413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): picked mon.noname-a con 0x7fdac409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdac40a1380 0x7fdac413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): picked mon.noname-c con 0x7fdac40a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdac40a3130 con 0x7fdac40a0890 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fdac4073df0 con 0x7fdac409ffc0 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdac40a1bf0 con 0x7fdac40a1380 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacc549640 10 monclient(hunting): authenticate will time out at 1614.042392s 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdac409ffc0 0x7fdac413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdac409ffc0 0x7fdac413aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37070/0 (socket says 172.21.15.17:37070) 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 -- 172.21.15.17:0/4205302462 learned_addr learned my addr 172.21.15.17:0/4205302462 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): get_auth_request con 0x7fdac409ffc0 auth_method 0 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdacaae6640 1 --2- 172.21.15.17:0/4205302462 >> v2:172.21.15.119:3300/0 conn(0x7fdac40a1380 0x7fdac413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_done global_id 124233 payload 274 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 -- 172.21.15.17:0/4205302462 >> v2:172.21.15.119:3300/0 conn(0x7fdac40a1380 msgr2=0x7fdac413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 --2- 172.21.15.17:0/4205302462 >> v2:172.21.15.119:3300/0 conn(0x7fdac40a1380 0x7fdac413d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 -- 172.21.15.17:0/4205302462 >> v1:172.21.15.17:6789/0 conn(0x7fdac40a0890 legacy=0x7fdac40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.645+0000 7fdac9ae4640 1 -- 172.21.15.17:0/4205302462 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac409ccd0 con 0x7fdac409ffc0 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac9ae4640 1 --2- 172.21.15.17:0/4205302462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac409ffc0 0x7fdac413aac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009870 tx=0x7fdab401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57846ae28efa2d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 1 -- 172.21.15.17:0/4205302462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40b0070 con 0x7fdac409ffc0 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: dump: 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.659 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.660 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.660 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.660 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.660 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.660 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 1 -- 172.21.15.17:0/4205302462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab40ab600 con 0x7fdac409ffc0 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 1 -- 172.21.15.17:0/4205302462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40ab9d0 con 0x7fdac409ffc0 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 5 monclient: authenticate success, global_id 124233 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 4 set_mon_vals no callback set 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdac92e3640 10 monclient: dump: 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 10 monclient: shutdown 2024-09-15T21:28:10.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 -- 172.21.15.17:0/4205302462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac409ffc0 msgr2=0x7fdac413aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 --2- 172.21.15.17:0/4205302462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac409ffc0 0x7fdac413aac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009870 tx=0x7fdab401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 -- 172.21.15.17:0/4205302462 shutdown_connections 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 --2- 172.21.15.17:0/4205302462 >> v2:172.21.15.119:3300/0 conn(0x7fdac40a1380 0x7fdac413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 --2- 172.21.15.17:0/4205302462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac409ffc0 0x7fdac413aac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 -- 172.21.15.17:0/4205302462 >> 172.21.15.17:0/4205302462 conn(0x7fdac40a66a0 msgr2=0x7fdac40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 -- 172.21.15.17:0/4205302462 shutdown_connections 2024-09-15T21:28:10.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 -- 172.21.15.17:0/4205302462 wait complete. 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 10 monclient: monmap: 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 Processor -- start 2024-09-15T21:28:10.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 1 -- start start 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.649+0000 7fdacc549640 10 monclient: init 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac409ffc0 0x7fdac413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): picked mon.noname-b con 0x7fdac409ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40a0890 0x7fdac4132560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): picked mon.noname-a con 0x7fdac40a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdac40a1380 0x7fdac4139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): picked mon.noname-c con 0x7fdac40a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:10.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4140100 con 0x7fdac40a0890 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac413ff80 con 0x7fdac409ffc0 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fdac4140280 con 0x7fdac40a1380 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): authenticate will time out at 1614.048197s 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40a0890 0x7fdac4132560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdac40a1380 0x7fdac4139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40a0890 0x7fdac4132560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37078/0 (socket says 172.21.15.17:37078) 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdac40a1380 0x7fdac4139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44552/0 (socket says 172.21.15.17:44552) 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 1 -- 172.21.15.17:0/3087736225 learned_addr learned my addr 172.21.15.17:0/3087736225 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): get_auth_request con 0x7fdac40a0890 auth_method 0 2024-09-15T21:28:10.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): get_auth_request con 0x7fdac40a1380 auth_method 0 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac409ffc0 0x7fdac413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): get_auth_request con 0x7fdac409ffc0 auth_method 0 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient(hunting): handle_auth_done global_id 134120 payload 824 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac409ffc0 msgr2=0x7fdac413bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac409ffc0 0x7fdac413bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40a0890 msgr2=0x7fdac4132560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40a0890 0x7fdac4132560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 monclient: found mon.noname-c 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:10.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac411cef0 con 0x7fdac40a1380 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40a0890 0x7fdac4132560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacaae6640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdac40a1380 0x7fdac4139920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ccb0 tx=0x7fdabc00a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc003ba0 con 0x7fdac40a1380 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: dump: 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.669 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc003d40 con 0x7fdac40a1380 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc004240 con 0x7fdac40a1380 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 5 monclient: authenticate success, global_id 134120 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient: _renew_subs 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac4133200 con 0x7fdac40a1380 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac8ae2640 4 set_mon_vals no callback set 2024-09-15T21:28:10.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: dump: 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient: _renew_subs 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac411c650 con 0x7fdac40a1380 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fdabc0043e0 con 0x7fdac40a1380 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 0x7fdac4139e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac411c820 con 0x7fdac4132aa0 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdaa803b9c0 0x7fdaa803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fdabc051100 con 0x7fdac40a1380 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 0x7fdac4139e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient: get_auth_request con 0x7fdac4132aa0 auth_method 0 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdaa803b9c0 0x7fdaa803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 10 monclient: get_auth_request con 0x7fdaa803b9c0 auth_method 0 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_done global_id 114286 payload 824 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdac40a0c70 con 0x7fdac4132aa0 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdaca2e5640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 0x7fdac4139e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e500 tx=0x7fdac000e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac00052e0 con 0x7fdac4132aa0 2024-09-15T21:28:10.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdabaffd640 10 monclient: dump: 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.674 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.653+0000 7fdac9ae4640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdaa803b9c0 0x7fdaa803de80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdab401c420 tx=0x7fdab4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdac0018070 con 0x7fdac4132aa0 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 msgr2=0x7fdac4139e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 0x7fdac4139e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e500 tx=0x7fdac000e9c0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 msgr2=0x7fdac4139e60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:10.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.657+0000 7fdabaffd640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac4132aa0 0x7fdac4139e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdacc549640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:10.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdacc549640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 0x7fdac413a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdacc549640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac406d020 con 0x7fdac41c5110 2024-09-15T21:28:10.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 0x7fdac413a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient: get_auth_request con 0x7fdac41c5110 auth_method 0 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 10 monclient(hunting): handle_auth_done global_id 114289 payload 824 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.661+0000 7fdaca2e5640 1 -- 172.21.15.17:0/3087736225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdac41d8470 con 0x7fdac41c5110 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdaca2e5640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 0x7fdac413a3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e500 tx=0x7fdac0000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac00038d0 con 0x7fdac41c5110 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 10 monclient: dump: 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7fdac00038d0 con 0x7fdac41c5110 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:10.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 msgr2=0x7fdac413a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 0x7fdac413a3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e500 tx=0x7fdac0000ee0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 msgr2=0x7fdac413a3a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdabaffd640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41c5110 0x7fdac413a3a0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e500 tx=0x7fdac0000ee0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-15T21:28:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 99231, 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:10.692 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdaa803b9c0 msgr2=0x7fdaa803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fdaa803b9c0 0x7fdaa803de80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdab401c420 tx=0x7fdab4008040 comp rx=0 tx=0).stop 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 10 monclient: shutdown 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdac40a1380 msgr2=0x7fdac4139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 --2- 172.21.15.17:0/3087736225 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fdac40a1380 0x7fdac4139920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ccb0 tx=0x7fdabc00a660 comp rx=0 tx=0).stop 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacaae6640 1 -- 172.21.15.17:0/3087736225 reap_dead start 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacaae6640 1 -- 172.21.15.17:0/3087736225 reap_dead start 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 shutdown_connections 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 >> 172.21.15.17:0/3087736225 conn(0x7fdac40a66a0 msgr2=0x7fdac4070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 shutdown_connections 2024-09-15T21:28:10.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.665+0000 7fdacc549640 1 -- 172.21.15.17:0/3087736225 wait complete. 2024-09-15T21:28:10.695 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:28:10.695 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:28:10.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:28:10.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.805+0000 7f4a5022a640 10 monclient: get_monmap_and_config 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.805+0000 7f4a5022a640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.805+0000 7f4a5022a640 10 monclient: monmap: 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:10.809994+0000 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:10.809994+0000 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:10.810 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:10.811 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:10.811 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:10.811 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 1 Processor -- start 2024-09-15T21:28:10.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 1 -- start start 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient: init 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4a480a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4a480a5580 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a480a59d0 0x7f4a48140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient(hunting): picked mon.noname-a con 0x7f4a480a59d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a480a2c30 con 0x7f4a480a5580 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a480a19b0 con 0x7f4a480a4a90 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4a480a0f40 con 0x7f4a480a59d0 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a5022a640 10 monclient(hunting): authenticate will time out at 1614.205277s 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a480a59d0 0x7f4a48140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a480a59d0 0x7f4a48140440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37082/0 (socket says 172.21.15.17:37082) 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 -- 172.21.15.17:0/2527033530 learned_addr learned my addr 172.21.15.17:0/2527033530 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): get_auth_request con 0x7f4a480a59d0 auth_method 0 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_done global_id 124242 payload 274 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 -- 172.21.15.17:0/2527033530 >> v1:172.21.15.119:6789/0 conn(0x7f4a480a4a90 legacy=0x7f4a480a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 -- 172.21.15.17:0/2527033530 >> v1:172.21.15.17:6789/0 conn(0x7f4a480a5580 legacy=0x7f4a4813cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.809+0000 7f4a4e7c7640 1 -- 172.21.15.17:0/2527033530 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480a1ce0 con 0x7f4a480a59d0 2024-09-15T21:28:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 1 --2- 172.21.15.17:0/2527033530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a59d0 0x7f4a48140440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4a400098a0 tx=0x7f4a4001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d90202a6c48425 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 1 -- 172.21.15.17:0/2527033530 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400b2070 con 0x7f4a480a59d0 2024-09-15T21:28:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: dump: 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 5 monclient: authenticate success, global_id 124242 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 1 -- 172.21.15.17:0/2527033530 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a400ad680 con 0x7f4a480a59d0 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 1 -- 172.21.15.17:0/2527033530 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400ada50 con 0x7f4a480a59d0 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 4 set_mon_vals no callback set 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4cfc4640 10 monclient: dump: 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: shutdown 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- 172.21.15.17:0/2527033530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a59d0 msgr2=0x7f4a48140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2527033530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a59d0 0x7f4a48140440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4a400098a0 tx=0x7f4a4001c460 comp rx=0 tx=0).stop 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- 172.21.15.17:0/2527033530 shutdown_connections 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2527033530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a59d0 0x7f4a48140440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- 172.21.15.17:0/2527033530 >> 172.21.15.17:0/2527033530 conn(0x7f4a480aabb0 msgr2=0x7f4a480a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- 172.21.15.17:0/2527033530 shutdown_connections 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- 172.21.15.17:0/2527033530 wait complete. 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: build_initial_monmap 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: monmap: 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:10.824 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 Processor -- start 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- start start 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: init 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a480a4a90 0x7f4a48137710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): picked mon.noname-b con 0x7f4a480a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 0x7f4a48137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): picked mon.noname-a con 0x7f4a480a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 0x7f4a4813cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): picked mon.noname-c con 0x7f4a480a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481437c0 con 0x7f4a480a5580 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4806f2a0 con 0x7f4a480a4a90 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a48143940 con 0x7f4a480a59d0 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a5022a640 10 monclient(hunting): authenticate will time out at 1614.211083s 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 0x7f4a48137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 0x7f4a4813cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 0x7f4a48137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37096/0 (socket says 172.21.15.17:37096) 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 -- 172.21.15.17:0/2506716275 learned_addr learned my addr 172.21.15.17:0/2506716275 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): get_auth_request con 0x7f4a480a5580 auth_method 0 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): get_auth_request con 0x7f4a480a59d0 auth_method 0 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4dfc6640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a480a4a90 0x7f4a48137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4dfc6640 10 monclient(hunting): get_auth_request con 0x7f4a480a4a90 auth_method 0 2024-09-15T21:28:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4dfc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4dfc6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4dfc6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient(hunting): handle_auth_done global_id 124245 payload 824 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 1 -- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 msgr2=0x7f4a4813cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 1 -- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 msgr2=0x7f4a4813cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 0x7f4a4813cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 -- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 msgr2=0x7f4a4813cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4e7c7640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 0x7f4a4813cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 0x7f4a4813cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 -- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a480a4a90 msgr2=0x7f4a48137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a480a4a90 0x7f4a48137710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient: _finish_hunting 0 2024-09-15T21:28:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 monclient: found mon.noname-a 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 -- 172.21.15.17:0/2506716275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480a1280 con 0x7f4a480a5580 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a4d7c5640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 0x7f4a48137c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c8a0 tx=0x7f4a3c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.813+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c003b80 con 0x7f4a480a5580 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: dump: 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.829 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3c003d20 con 0x7f4a480a5580 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 5 monclient: authenticate success, global_id 124245 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 monclient: _renew_subs 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a48120c80 con 0x7f4a480a5580 2024-09-15T21:28:10.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00aa90 con 0x7f4a480a5580 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 10 monclient: dump: 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:10.831 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3bfff640 4 set_mon_vals no callback set 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 monclient: _renew_subs 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a48121240 con 0x7f4a480a5580 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a480a4e70 con 0x7f4a480a5580 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4a3c00ac30 con 0x7f4a480a5580 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2c03b9f0 0x7f4a2c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4a3c0504d0 con 0x7f4a480a5580 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a4dfc6640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2c03b9f0 0x7f4a2c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:10.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a4dfc6640 10 monclient: get_auth_request con 0x7f4a2c03b9f0 auth_method 0 2024-09-15T21:28:10.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.817+0000 7f4a4dfc6640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2c03b9f0 0x7f4a2c03deb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000f70 tx=0x7f4a44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:10.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.821+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4a480a1280 con 0x7f4a480a5580 2024-09-15T21:28:10.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.821+0000 7f4a3a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:10.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:10.821+0000 7f4a3a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.057+0000 7f4a5022a640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:28:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.057+0000 7f4a5022a640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:28:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.061+0000 7f4a5022a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.061+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f4a48069200 con 0x7f4a480a5580 2024-09-15T21:28:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.061+0000 7f4a3a7fc640 1 -- 172.21.15.17:0/2506716275 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f4a480a2c30 con 0x7f4a480a5580 2024-09-15T21:28:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.061+0000 7f4a3a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:28:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.061+0000 7f4a3a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:11.065 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.061+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.065 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.061+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=3081495999,config=3541143443,health=765734218,kv=3079713637,logm=1711209184}) 2024-09-15T21:28:11.065 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.061+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=2,health=12,kv=12,logm=51} crc {auth=3763581874,config=3541143443,health=765734218,kv=3079713637,logm=371505192}) 2024-09-15T21:28:11.065 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.061+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.065 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.061+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=3081495999,config=3541143443,health=765734218,kv=3079713637,logm=1711209184}) 2024-09-15T21:28:11.065 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.061+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=12,kv=11,logm=52} crc {auth=1815266089,config=3541143443,health=765734218,kv=2435832620,logm=3450606437}) 2024-09-15T21:28:11.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2c03b9f0 msgr2=0x7f4a2c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:11.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2c03b9f0 0x7f4a2c03deb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000f70 tx=0x7f4a44009040 comp rx=0 tx=0).stop 2024-09-15T21:28:11.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 10 monclient: shutdown 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 msgr2=0x7f4a48137c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 0x7f4a48137c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c8a0 tx=0x7f4a3c00cd60 comp rx=0 tx=0).stop 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 shutdown_connections 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2c03b9f0 0x7f4a2c03deb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a480a59d0 0x7f4a4813cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a480a5580 0x7f4a48137c50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 --2- 172.21.15.17:0/2506716275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a480a4a90 0x7f4a48137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 >> 172.21.15.17:0/2506716275 conn(0x7f4a480aabb0 msgr2=0x7f4a480a8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 shutdown_connections 2024-09-15T21:28:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:11.065+0000 7f4a5022a640 1 -- 172.21.15.17:0/2506716275 wait complete. 2024-09-15T21:28:11.068 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.065+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.068 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.065+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606842953}) 2024-09-15T21:28:11.068 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.065+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=543348290}) 2024-09-15T21:28:11.068 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.065+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.069 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.065+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606842953}) 2024-09-15T21:28:11.069 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.065+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107628098}) 2024-09-15T21:28:11.071 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.069+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.072 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.069+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615427526}) 2024-09-15T21:28:11.072 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.069+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=522132934}) 2024-09-15T21:28:11.072 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.069+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.072 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.069+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615427526}) 2024-09-15T21:28:11.072 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.069+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1077615225}) 2024-09-15T21:28:11.075 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.075 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3662222235}) 2024-09-15T21:28:11.075 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=855270788}) 2024-09-15T21:28:11.075 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.075 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3662222235}) 2024-09-15T21:28:11.075 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4225651567}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3922191859}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2092486309}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3922191859}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.073+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=634469597}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.077+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.077+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738144414}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.077+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=741339957}) 2024-09-15T21:28:11.080 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.077+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.081 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.077+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738144414}) 2024-09-15T21:28:11.081 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.077+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1886476935}) 2024-09-15T21:28:11.082 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.082 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1130093310}) 2024-09-15T21:28:11.082 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1511904351}) 2024-09-15T21:28:11.082 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.082 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1130093310}) 2024-09-15T21:28:11.082 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3457503260}) 2024-09-15T21:28:11.084 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.084 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2333397228}) 2024-09-15T21:28:11.084 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2921436582}) 2024-09-15T21:28:11.084 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.084 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2333397228}) 2024-09-15T21:28:11.085 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.081+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2703989957}) 2024-09-15T21:28:11.086 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:28:11.088 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.085+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.088 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.085+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=840868084}) 2024-09-15T21:28:11.088 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.085+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211680427}) 2024-09-15T21:28:11.088 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.085+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.088 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.085+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=840868084}) 2024-09-15T21:28:11.088 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.085+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=71182717}) 2024-09-15T21:28:11.091 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.089+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.091 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.089+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1155383729}) 2024-09-15T21:28:11.091 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.089+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=576386244}) 2024-09-15T21:28:11.091 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.089+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.091 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.089+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1155383729}) 2024-09-15T21:28:11.091 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.089+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1526710981}) 2024-09-15T21:28:11.094 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.094 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2200805478}) 2024-09-15T21:28:11.094 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1537618855}) 2024-09-15T21:28:11.094 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.094 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2200805478}) 2024-09-15T21:28:11.095 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3866361381}) 2024-09-15T21:28:11.097 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.097 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=339259202}) 2024-09-15T21:28:11.097 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2881271414}) 2024-09-15T21:28:11.097 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.097 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=339259202}) 2024-09-15T21:28:11.098 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.093+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1406870878}) 2024-09-15T21:28:11.099 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.099 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568547554}) 2024-09-15T21:28:11.099 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1851382758}) 2024-09-15T21:28:11.099 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.099 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568547554}) 2024-09-15T21:28:11.099 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2494731719}) 2024-09-15T21:28:11.101 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.101 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3290052435}) 2024-09-15T21:28:11.101 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2639169986}) 2024-09-15T21:28:11.101 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.101 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3290052435}) 2024-09-15T21:28:11.101 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.097+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3502199962}) 2024-09-15T21:28:11.103 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.103 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1784403828}) 2024-09-15T21:28:11.103 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2027147494}) 2024-09-15T21:28:11.103 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.103 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1784403828}) 2024-09-15T21:28:11.103 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=875014515}) 2024-09-15T21:28:11.105 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.105 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4252289675}) 2024-09-15T21:28:11.105 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371730599}) 2024-09-15T21:28:11.105 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.105 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4252289675}) 2024-09-15T21:28:11.105 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.101+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=524256727}) 2024-09-15T21:28:11.106 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.106 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3851858436}) 2024-09-15T21:28:11.107 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580822158}) 2024-09-15T21:28:11.107 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.107 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3851858436}) 2024-09-15T21:28:11.107 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3875473180}) 2024-09-15T21:28:11.108 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.108 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2979488288}) 2024-09-15T21:28:11.108 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1635477587}) 2024-09-15T21:28:11.108 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.108 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2979488288}) 2024-09-15T21:28:11.108 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.105+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=123259143}) 2024-09-15T21:28:11.110 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.110 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615705884}) 2024-09-15T21:28:11.110 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1019031058}) 2024-09-15T21:28:11.110 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.110 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615705884}) 2024-09-15T21:28:11.110 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3980072279}) 2024-09-15T21:28:11.112 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.112 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4029481916}) 2024-09-15T21:28:11.112 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1076210858}) 2024-09-15T21:28:11.112 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.112 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4029481916}) 2024-09-15T21:28:11.112 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.109+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1624548003}) 2024-09-15T21:28:11.114 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.114 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158306364}) 2024-09-15T21:28:11.114 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=463204999}) 2024-09-15T21:28:11.114 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.114 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158306364}) 2024-09-15T21:28:11.114 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2692954745}) 2024-09-15T21:28:11.115 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.115 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944474000}) 2024-09-15T21:28:11.116 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=287167991}) 2024-09-15T21:28:11.116 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.116 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944474000}) 2024-09-15T21:28:11.116 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2362707269}) 2024-09-15T21:28:11.117 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.117 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545335334}) 2024-09-15T21:28:11.117 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273888580}) 2024-09-15T21:28:11.118 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.118 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545335334}) 2024-09-15T21:28:11.118 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.113+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3566459426}) 2024-09-15T21:28:11.119 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.120 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=128273225}) 2024-09-15T21:28:11.120 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757159863}) 2024-09-15T21:28:11.120 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.121 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=128273225}) 2024-09-15T21:28:11.121 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1625520361}) 2024-09-15T21:28:11.121 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2860635550}) 2024-09-15T21:28:11.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3027405530}) 2024-09-15T21:28:11.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2860635550}) 2024-09-15T21:28:11.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.117+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2458015943}) 2024-09-15T21:28:11.123 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.123 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2813751396}) 2024-09-15T21:28:11.123 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2258002432}) 2024-09-15T21:28:11.123 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.123 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2813751396}) 2024-09-15T21:28:11.123 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569930821}) 2024-09-15T21:28:11.124 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.125 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1661646009}) 2024-09-15T21:28:11.125 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2043668495}) 2024-09-15T21:28:11.125 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.125 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1661646009}) 2024-09-15T21:28:11.125 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.121+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1020068937}) 2024-09-15T21:28:11.126 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.126 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=521414408}) 2024-09-15T21:28:11.126 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986650609}) 2024-09-15T21:28:11.126 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.126 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=521414408}) 2024-09-15T21:28:11.126 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3188427692}) 2024-09-15T21:28:11.128 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.128 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2213575905}) 2024-09-15T21:28:11.128 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4023106744}) 2024-09-15T21:28:11.128 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.128 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2213575905}) 2024-09-15T21:28:11.128 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4172032483}) 2024-09-15T21:28:11.129 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.130 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1228938206}) 2024-09-15T21:28:11.130 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1291866551}) 2024-09-15T21:28:11.130 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.130 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1228938206}) 2024-09-15T21:28:11.130 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.125+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276535243}) 2024-09-15T21:28:11.131 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.131 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156192601}) 2024-09-15T21:28:11.131 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1359137341}) 2024-09-15T21:28:11.132 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.132 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156192601}) 2024-09-15T21:28:11.132 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691482433}) 2024-09-15T21:28:11.133 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.133 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3647982243}) 2024-09-15T21:28:11.133 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2823406635}) 2024-09-15T21:28:11.133 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.133 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3647982243}) 2024-09-15T21:28:11.134 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.129+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4164199252}) 2024-09-15T21:28:11.135 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.135 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2382022542}) 2024-09-15T21:28:11.135 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=794946342}) 2024-09-15T21:28:11.135 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.135 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2382022542}) 2024-09-15T21:28:11.135 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=722476257}) 2024-09-15T21:28:11.137 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.137 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=966666306}) 2024-09-15T21:28:11.137 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2355442283}) 2024-09-15T21:28:11.137 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.137 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=966666306}) 2024-09-15T21:28:11.137 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.133+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2764615406}) 2024-09-15T21:28:11.139 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.139 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=157190670}) 2024-09-15T21:28:11.139 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2399043003}) 2024-09-15T21:28:11.139 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.139 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=157190670}) 2024-09-15T21:28:11.139 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3479318528}) 2024-09-15T21:28:11.141 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.141 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=659860903}) 2024-09-15T21:28:11.141 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=265416409}) 2024-09-15T21:28:11.141 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.141 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=659860903}) 2024-09-15T21:28:11.141 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.137+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1802193274}) 2024-09-15T21:28:11.142 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.143 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2939765404}) 2024-09-15T21:28:11.143 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4124634762}) 2024-09-15T21:28:11.143 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.143 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2939765404}) 2024-09-15T21:28:11.143 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4149396053}) 2024-09-15T21:28:11.144 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.144 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1310376381}) 2024-09-15T21:28:11.144 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2349468983}) 2024-09-15T21:28:11.144 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.145 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1310376381}) 2024-09-15T21:28:11.145 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.141+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=493308936}) 2024-09-15T21:28:11.146 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.146 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1389204990}) 2024-09-15T21:28:11.146 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876695358}) 2024-09-15T21:28:11.146 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.146 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1389204990}) 2024-09-15T21:28:11.147 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=332639136}) 2024-09-15T21:28:11.148 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.148 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2945233511}) 2024-09-15T21:28:11.149 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282686896}) 2024-09-15T21:28:11.149 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.149 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2945233511}) 2024-09-15T21:28:11.149 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.145+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3411107221}) 2024-09-15T21:28:11.150 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.150 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702245526}) 2024-09-15T21:28:11.150 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3991889152}) 2024-09-15T21:28:11.150 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.150 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702245526}) 2024-09-15T21:28:11.151 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2905763568}) 2024-09-15T21:28:11.152 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.152 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1699044096}) 2024-09-15T21:28:11.152 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2069397257}) 2024-09-15T21:28:11.152 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.152 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1699044096}) 2024-09-15T21:28:11.152 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.149+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2510733394}) 2024-09-15T21:28:11.154 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.154 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3540604324}) 2024-09-15T21:28:11.154 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3337485898}) 2024-09-15T21:28:11.154 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.154 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3540604324}) 2024-09-15T21:28:11.154 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1066500688}) 2024-09-15T21:28:11.156 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.156 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235618558}) 2024-09-15T21:28:11.156 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3168336693}) 2024-09-15T21:28:11.156 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.156 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235618558}) 2024-09-15T21:28:11.156 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.153+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3546993493}) 2024-09-15T21:28:11.158 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.158 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3645180911}) 2024-09-15T21:28:11.158 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=681514982}) 2024-09-15T21:28:11.158 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.158 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3645180911}) 2024-09-15T21:28:11.158 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3131953529}) 2024-09-15T21:28:11.160 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.160 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=228807892}) 2024-09-15T21:28:11.160 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3146564183}) 2024-09-15T21:28:11.160 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.160 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=228807892}) 2024-09-15T21:28:11.160 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.157+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3146229119}) 2024-09-15T21:28:11.163 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.163 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407956665}) 2024-09-15T21:28:11.163 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=293488444}) 2024-09-15T21:28:11.163 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.163 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407956665}) 2024-09-15T21:28:11.163 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3478650648}) 2024-09-15T21:28:11.164 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.164 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2990829878}) 2024-09-15T21:28:11.164 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3559523272}) 2024-09-15T21:28:11.164 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.164 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2990829878}) 2024-09-15T21:28:11.164 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3008724833}) 2024-09-15T21:28:11.166 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.166 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836122485}) 2024-09-15T21:28:11.166 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=176345110}) 2024-09-15T21:28:11.166 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.166 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836122485}) 2024-09-15T21:28:11.167 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.161+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2442517744}) 2024-09-15T21:28:11.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715036931}) 2024-09-15T21:28:11.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246347543}) 2024-09-15T21:28:11.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715036931}) 2024-09-15T21:28:11.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1582121590}) 2024-09-15T21:28:11.169 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.170 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993709228}) 2024-09-15T21:28:11.170 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1710927464}) 2024-09-15T21:28:11.170 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.170 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993709228}) 2024-09-15T21:28:11.170 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.165+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1980143604}) 2024-09-15T21:28:11.172 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.172 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4027770678}) 2024-09-15T21:28:11.172 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1429100984}) 2024-09-15T21:28:11.172 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.172 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4027770678}) 2024-09-15T21:28:11.172 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1539372621}) 2024-09-15T21:28:11.173 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.173 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=664505680}) 2024-09-15T21:28:11.173 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3008156315}) 2024-09-15T21:28:11.173 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.173 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=664505680}) 2024-09-15T21:28:11.173 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.169+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3285875590}) 2024-09-15T21:28:11.174 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.175 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1139300340}) 2024-09-15T21:28:11.175 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=197245889}) 2024-09-15T21:28:11.175 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.175 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1139300340}) 2024-09-15T21:28:11.175 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279202738}) 2024-09-15T21:28:11.176 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.177 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=280583431}) 2024-09-15T21:28:11.177 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3927690277}) 2024-09-15T21:28:11.177 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.177 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=280583431}) 2024-09-15T21:28:11.177 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.173+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3865941961}) 2024-09-15T21:28:11.179 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.179 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=591516511}) 2024-09-15T21:28:11.179 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3422028416}) 2024-09-15T21:28:11.179 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.179 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=591516511}) 2024-09-15T21:28:11.179 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3075433974}) 2024-09-15T21:28:11.181 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.181 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2859416253}) 2024-09-15T21:28:11.181 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2530247761}) 2024-09-15T21:28:11.181 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.181 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2859416253}) 2024-09-15T21:28:11.181 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.177+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1481907098}) 2024-09-15T21:28:11.182 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.183 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197480858}) 2024-09-15T21:28:11.183 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2839800263}) 2024-09-15T21:28:11.183 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.183 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197480858}) 2024-09-15T21:28:11.183 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3707717272}) 2024-09-15T21:28:11.185 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.185 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1667530339}) 2024-09-15T21:28:11.185 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=200253119}) 2024-09-15T21:28:11.185 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.185 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1667530339}) 2024-09-15T21:28:11.185 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.181+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2256042585}) 2024-09-15T21:28:11.187 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.187 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=559779873}) 2024-09-15T21:28:11.187 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1135703533}) 2024-09-15T21:28:11.187 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.187 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=559779873}) 2024-09-15T21:28:11.187 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=138626372}) 2024-09-15T21:28:11.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1290361962}) 2024-09-15T21:28:11.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=116633255}) 2024-09-15T21:28:11.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1290361962}) 2024-09-15T21:28:11.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.185+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=487745067}) 2024-09-15T21:28:11.192 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.189+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.192 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.189+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=48,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=2} crc {logm=2385146569,mdsmap=173658045,mgr=3916736423,mgr_metadata=240635875,mgrstat=35510702,mon_config_key=3131655263,monmap=186996059,nvmeofgw=598337686,osd_metadata=1786881783,osd_pg_creating=543505836,osd_snap=3058936622,osdmap=809160886}) 2024-09-15T21:28:11.193 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.189+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=47,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=2} crc {logm=1894230738,mdsmap=173658045,mgr=4211821201,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3658046529,mon_config_key=3131655263,monmap=186996059,nvmeofgw=3517310312,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=809160886}) 2024-09-15T21:28:11.193 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.189+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.193 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.189+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=48,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=2} crc {logm=2385146569,mdsmap=173658045,mgr=3916736423,mgr_metadata=240635875,mgrstat=35510702,mon_config_key=3131655263,monmap=186996059,nvmeofgw=598337686,osd_metadata=1786881783,osd_pg_creating=543505836,osd_snap=3058936622,osdmap=809160886}) 2024-09-15T21:28:11.193 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.189+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=5} crc {logm=849210928,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=4137174909,mon_config_key=1677881253,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=726740008}) 2024-09-15T21:28:11.205 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.201+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.206 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.201+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=52} crc {osdmap=1336468258}) 2024-09-15T21:28:11.206 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.201+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=51} crc {osdmap=1787194630}) 2024-09-15T21:28:11.206 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.201+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:28:11.206 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.201+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=52} crc {osdmap=1336468258}) 2024-09-15T21:28:11.206 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:28:11.201+0000 7fef843be640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=49} crc {osdmap=2125956606}) 2024-09-15T21:28:12.087 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:28:12.087 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient: get_monmap_and_config 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient: monmap: 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:12.211668+0000 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:12.211668+0000 2024-09-15T21:28:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 1 Processor -- start 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 1 -- start start 2024-09-15T21:28:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient: init 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88cc073640 0x7f88cc073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient(hunting): picked mon.noname-a con 0x7f88cc073640 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f88cc073f60 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f88cc137da0 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88cc0a0890 con 0x7f88cc137da0 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88cc0746e0 con 0x7f88cc073f60 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f88cc0a1640 con 0x7f88cc073640 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88d1c3d640 10 monclient(hunting): authenticate will time out at 1615.607092s 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88cc073640 0x7f88cc073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88cc073640 0x7f88cc073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37100/0 (socket says 172.21.15.17:37100) 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 -- 172.21.15.17:0/1397519887 learned_addr learned my addr 172.21.15.17:0/1397519887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): get_auth_request con 0x7f88cc073640 auth_method 0 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_done global_id 124248 payload 274 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 -- 172.21.15.17:0/1397519887 >> v1:172.21.15.119:6789/0 conn(0x7f88cc073f60 legacy=0x7f88cc137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 -- 172.21.15.17:0/1397519887 >> v1:172.21.15.17:6789/0 conn(0x7f88cc137da0 legacy=0x7f88cc13fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 monclient: found mon.noname-a 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.209+0000 7f88cb7fe640 1 -- 172.21.15.17:0/1397519887 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc0a1920 con 0x7f88cc073640 2024-09-15T21:28:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88cb7fe640 1 --2- 172.21.15.17:0/1397519887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073640 0x7f88cc073a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009e50 tx=0x7f88bc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfa934aafd114c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 1 -- 172.21.15.17:0/1397519887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0a57a0 con 0x7f88cc073640 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: dump: 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 1 -- 172.21.15.17:0/1397519887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc0a5940 con 0x7f88cc073640 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 5 monclient: authenticate success, global_id 124248 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 1 -- 172.21.15.17:0/1397519887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0a5d10 con 0x7f88cc073640 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 4 set_mon_vals no callback set 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88ca7fc640 10 monclient: dump: 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 10 monclient: shutdown 2024-09-15T21:28:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 -- 172.21.15.17:0/1397519887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073640 msgr2=0x7f88cc073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/1397519887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073640 0x7f88cc073a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009e50 tx=0x7f88bc01c8c0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 -- 172.21.15.17:0/1397519887 shutdown_connections 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/1397519887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073640 0x7f88cc073a20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 -- 172.21.15.17:0/1397519887 >> 172.21.15.17:0/1397519887 conn(0x7f88cc0a66a0 msgr2=0x7f88cc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 -- 172.21.15.17:0/1397519887 shutdown_connections 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 -- 172.21.15.17:0/1397519887 wait complete. 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 10 monclient: monmap: 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 Processor -- start 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 1 -- start start 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.213+0000 7f88d1c3d640 10 monclient: init 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc073640 0x7f88cc13b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): picked mon.noname-c con 0x7f88cc073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 0x7f88cc13beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): picked mon.noname-a con 0x7f88cc073f60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc137da0 0x7f88cc13c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): picked mon.noname-b con 0x7f88cc137da0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc0a0e80 con 0x7f88cc073f60 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc0a0d00 con 0x7f88cc137da0 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc143b60 con 0x7f88cc073640 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient(hunting): authenticate will time out at 1615.612165s 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 0x7f88cc13beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc073640 0x7f88cc13b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 0x7f88cc13beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37108/0 (socket says 172.21.15.17:37108) 2024-09-15T21:28:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 -- 172.21.15.17:0/4154410643 learned_addr learned my addr 172.21.15.17:0/4154410643 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): get_auth_request con 0x7f88cc073f60 auth_method 0 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): get_auth_request con 0x7f88cc073640 auth_method 0 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cbfff640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc137da0 0x7f88cc13c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient(hunting): handle_auth_done global_id 124254 payload 824 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 -- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc073640 msgr2=0x7f88cc13b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc073640 0x7f88cc13b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 -- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc137da0 msgr2=0x7f88cc13c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc137da0 0x7f88cc13c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 monclient: found mon.noname-a 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 -- 172.21.15.17:0/4154410643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc13f3e0 con 0x7f88cc073f60 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cbfff640 10 monclient: get_auth_request con 0x7f88cc137da0 auth_method 0 2024-09-15T21:28:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cbfff640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc137da0 0x7f88cc13c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc073640 0x7f88cc13b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88caffd640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 0x7f88cc13beb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c480 tx=0x7f88c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0012830 con 0x7f88cc073f60 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: dump: 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88cc073f60 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 5 monclient: authenticate success, global_id 124254 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _renew_subs 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc13d5c0 con 0x7f88cc073f60 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0012c30 con 0x7f88cc073f60 2024-09-15T21:28:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 10 monclient: dump: 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _renew_subs 2024-09-15T21:28:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88c9ffb640 4 set_mon_vals no callback set 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc13dec0 con 0x7f88cc073f60 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc073a20 con 0x7f88cc073f60 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f88c001c460 con 0x7f88cc073f60 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b803b9c0 0x7f88b803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f88c0050fe0 con 0x7f88cc073f60 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b803b9c0 0x7f88b803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.217+0000 7f88cb7fe640 10 monclient: get_auth_request con 0x7f88b803b9c0 auth_method 0 2024-09-15T21:28:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.221+0000 7f88cb7fe640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b803b9c0 0x7f88b803de80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ae000 tx=0x7f88bc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.221+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f88c001ae30 con 0x7f88cc073f60 2024-09-15T21:28:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.221+0000 7f88b3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.221+0000 7f88b3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:12.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:12.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:12.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f88cc06d020 con 0x7f88cc073f60 2024-09-15T21:28:12.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88b3fff640 1 -- 172.21.15.17:0/4154410643 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f88c00249e0 con 0x7f88cc073f60 2024-09-15T21:28:12.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88b3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:12.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88b3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:12.464 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:12.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b803b9c0 msgr2=0x7f88b803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b803b9c0 0x7f88b803de80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ae000 tx=0x7f88bc0a7070 comp rx=0 tx=0).stop 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 10 monclient: shutdown 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 msgr2=0x7f88cc13beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 0x7f88cc13beb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c480 tx=0x7f88c000c940 comp rx=0 tx=0).stop 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 shutdown_connections 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b803b9c0 0x7f88b803de80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc137da0 0x7f88cc13c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc073f60 0x7f88cc13beb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 --2- 172.21.15.17:0/4154410643 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc073640 0x7f88cc13b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 >> 172.21.15.17:0/4154410643 conn(0x7f88cc0a66a0 msgr2=0x7f88cc0a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.461+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 shutdown_connections 2024-09-15T21:28:12.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.465+0000 7f88d1c3d640 1 -- 172.21.15.17:0/4154410643 wait complete. 2024-09-15T21:28:12.483 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:28:12.483 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:28:12.483 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:28:12.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient: get_monmap_and_config 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient: monmap: 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:12.606932+0000 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:12.606932+0000 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:12.608 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:12.608 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:12.608 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:12.608 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:12.608 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 Processor -- start 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 -- start start 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient: init 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3abc1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 0x7f3abc13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): picked mon.noname-b con 0x7f3abc1383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3abc13d2c0 0x7f3abc13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): picked mon.noname-c con 0x7f3abc13d2c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3abc0a1a20 con 0x7f3abc1378c0 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3abc0a1d00 con 0x7f3abc13d2c0 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3abc0a1fe0 con 0x7f3abc1383b0 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac3342640 10 monclient(hunting): authenticate will time out at 1616.003005s 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 0x7f3abc13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 0x7f3abc13cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44568/0 (socket says 172.21.15.17:44568) 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3945512879 learned_addr learned my addr 172.21.15.17:0/3945512879 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request con 0x7f3abc1383b0 auth_method 0 2024-09-15T21:28:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_done global_id 134129 payload 274 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac18df640 1 --2- 172.21.15.17:0/3945512879 >> v2:172.21.15.119:3300/0 conn(0x7f3abc13d2c0 0x7f3abc13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3945512879 >> v2:172.21.15.119:3300/0 conn(0x7f3abc13d2c0 msgr2=0x7f3abc13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3945512879 >> v2:172.21.15.119:3300/0 conn(0x7f3abc13d2c0 0x7f3abc13f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ab3fff640 1 -- 172.21.15.17:0/3945512879 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021055948 0 0) 0x7f3ab8001120 con 0x7f3abc1378c0 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3945512879 >> v1:172.21.15.17:6789/0 conn(0x7f3abc1378c0 legacy=0x7f3abc137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 monclient: found mon.noname-b 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3945512879 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc0a22c0 con 0x7f3abc1383b0 2024-09-15T21:28:12.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.605+0000 7f3ab3fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:28:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3945512879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 0x7f3abc13cd80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009870 tx=0x7f3aac01c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f893529b3ddf4490 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 1 -- 172.21.15.17:0/3945512879 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0b1070 con 0x7f3abc1383b0 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: dump: 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.624 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 1 -- 172.21.15.17:0/3945512879 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aac0a48b0 con 0x7f3abc1383b0 2024-09-15T21:28:12.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 1 -- 172.21.15.17:0/3945512879 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0a4c80 con 0x7f3abc1383b0 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 5 monclient: authenticate success, global_id 134129 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 4 set_mon_vals no callback set 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ab3fff640 10 monclient: dump: 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: shutdown 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- 172.21.15.17:0/3945512879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 msgr2=0x7f3abc13cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3945512879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 0x7f3abc13cd80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009870 tx=0x7f3aac01c100 comp rx=0 tx=0).stop 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- 172.21.15.17:0/3945512879 shutdown_connections 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3945512879 >> v2:172.21.15.119:3300/0 conn(0x7f3abc13d2c0 0x7f3abc13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3945512879 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1383b0 0x7f3abc13cd80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- 172.21.15.17:0/3945512879 >> 172.21.15.17:0/3945512879 conn(0x7f3abc0aac40 msgr2=0x7f3abc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- 172.21.15.17:0/3945512879 shutdown_connections 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- 172.21.15.17:0/3945512879 wait complete. 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: monmap: 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 Processor -- start 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- start start 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: init 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:28:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1378c0 0x7f3abc1d0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): picked mon.noname-c con 0x7f3abc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3abc1383b0 0x7f3abc136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): picked mon.noname-b con 0x7f3abc1383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13d2c0 0x7f3abc09dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): picked mon.noname-a con 0x7f3abc13d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc143040 con 0x7f3abc13d2c0 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc142ec0 con 0x7f3abc1383b0 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3abc1431c0 con 0x7f3abc1378c0 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.609+0000 7f3ac3342640 10 monclient(hunting): authenticate will time out at 1616.007937s 2024-09-15T21:28:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13d2c0 0x7f3abc09dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13d2c0 0x7f3abc09dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37128/0 (socket says 172.21.15.17:37128) 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1378c0 0x7f3abc1d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 1 -- 172.21.15.17:0/3127161385 learned_addr learned my addr 172.21.15.17:0/3127161385 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): get_auth_request con 0x7f3abc13d2c0 auth_method 0 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): get_auth_request con 0x7f3abc1378c0 auth_method 0 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3abc1383b0 0x7f3abc136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request con 0x7f3abc1383b0 auth_method 0 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient(hunting): handle_auth_done global_id 134132 payload 824 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3abc1383b0 msgr2=0x7f3abc136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3abc1383b0 0x7f3abc136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13d2c0 msgr2=0x7f3abc09dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13d2c0 0x7f3abc09dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 monclient: found mon.noname-c 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc1d1690 con 0x7f3abc1378c0 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13d2c0 0x7f3abc09dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac10de640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1378c0 0x7f3abc1d0780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c400 tx=0x7f3ab800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8017070 con 0x7f3abc1378c0 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: dump: 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.634 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.634 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.634 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 5 monclient: authenticate success, global_id 134132 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab8003600 con 0x7f3abc1378c0 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 10 monclient: _renew_subs 2024-09-15T21:28:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc0a0200 con 0x7f3abc1378c0 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80039f0 con 0x7f3abc1378c0 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: dump: 2024-09-15T21:28:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab37fe640 4 set_mon_vals no callback set 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 10 monclient: _renew_subs 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc09f6a0 con 0x7f3abc1378c0 2024-09-15T21:28:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc09ea40 con 0x7f3abc09e120 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3ab801a070 con 0x7f3abc1378c0 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient: get_auth_request con 0x7f3abc09e120 auth_method 0 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_done global_id 124263 payload 824 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3abc137ca0 con 0x7f3abc09e120 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3aa003b9f0 0x7f3aa003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3ab804f9c0 con 0x7f3abc1378c0 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0ab150 tx=0x7f3aac01a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 msgr2=0x7f3abc09e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 msgr2=0x7f3abc09e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0ab150 tx=0x7f3aac01a940 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0ab150 tx=0x7f3aac01a940 comp rx=0 tx=0).stop 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 msgr2=0x7f3abc09e500 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 0x7f3aa00466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 msgr2=0x7f3abc09e500 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc09e120 0x7f3abc09e500 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0046ce0 con 0x7f3aa0044280 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3aa003b9f0 0x7f3aa003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 10 monclient: get_auth_request con 0x7f3aa003b9f0 auth_method 0 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 0x7f3aa00466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient: get_auth_request con 0x7f3aa0044280 auth_method 0 2024-09-15T21:28:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_done global_id 124266 payload 824 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3aa0046e90 con 0x7f3aa0044280 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 0x7f3aa00466a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3aac01c140 tx=0x7f3aac009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0d6070 con 0x7f3aa0044280 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ac18df640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3aa003b9f0 0x7f3aa003deb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400d7d0 tx=0x7f3ab4007750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: dump: 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3aac0ac810 con 0x7f3aa0044280 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 msgr2=0x7f3aa00466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 0x7f3aa00466a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3aac01c140 tx=0x7f3aac009240 comp rx=0 tx=0).stop 2024-09-15T21:28:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 msgr2=0x7f3aa00466a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.613+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3aa0044280 0x7f3aa00466a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 0x7f3abc13b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc0a4490 con 0x7f3abc13ad70 2024-09-15T21:28:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 0x7f3abc13b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient: get_auth_request con 0x7f3abc13ad70 auth_method 0 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 10 monclient(hunting): handle_auth_done global_id 124269 payload 824 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 1 -- 172.21.15.17:0/3127161385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3abc069270 con 0x7f3abc13ad70 2024-09-15T21:28:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac08dd640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 0x7f3abc13b150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3aac01a7b0 tx=0x7f3aac0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0df070 con 0x7f3abc13ad70 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 10 monclient: dump: 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4881 (secure 0 0 0) 0x7f3aac0b68c0 con 0x7f3abc13ad70 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 msgr2=0x7f3abc13b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 0x7f3abc13b150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3aac01a7b0 tx=0x7f3aac0a6000 comp rx=0 tx=0).stop 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 msgr2=0x7f3abc13b150 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ab1ffb640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3abc13ad70 0x7f3abc13b150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac18df640 1 -- 172.21.15.17:0/3127161385 reap_dead start 2024-09-15T21:28:12.650 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 55571, 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:12.651 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.652 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:12.654 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.655 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:12.658 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:12.658 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.658 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.658 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.662 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.663 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:12.664 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3aa003b9f0 msgr2=0x7f3aa003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3aa003b9f0 0x7f3aa003deb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400d7d0 tx=0x7f3ab4007750 comp rx=0 tx=0).stop 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 10 monclient: shutdown 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1378c0 msgr2=0x7f3abc1d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1378c0 0x7f3abc1d0780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c400 tx=0x7f3ab800c8c0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 shutdown_connections 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3aa003b9f0 0x7f3aa003deb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 --2- 172.21.15.17:0/3127161385 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3abc1378c0 0x7f3abc1d0780 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 >> 172.21.15.17:0/3127161385 conn(0x7f3abc0aac40 msgr2=0x7f3abc0a8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 shutdown_connections 2024-09-15T21:28:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.621+0000 7f3ac3342640 1 -- 172.21.15.17:0/3127161385 wait complete. 2024-09-15T21:28:12.667 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:28:12.667 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:28:12.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:28:12.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient: get_monmap_and_config 2024-09-15T21:28:12.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient: monmap: 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:12.783627+0000 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:12.783627+0000 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:12.784 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 Processor -- start 2024-09-15T21:28:12.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 -- start start 2024-09-15T21:28:12.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient: init 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd6580a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd65809ffc0 0x7fd65813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): picked mon.noname-c con 0x7fd65809ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd6580a1380 0x7fd65813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): picked mon.noname-a con 0x7fd6580a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6580a3130 con 0x7fd6580a0890 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd658073df0 con 0x7fd6580a1380 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd6580a1bf0 con 0x7fd65809ffc0 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65e4e8640 10 monclient(hunting): authenticate will time out at 1616.179557s 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd6580a1380 0x7fd65813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd6580a1380 0x7fd65813d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37160/0 (socket says 172.21.15.17:37160) 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 1 -- 172.21.15.17:0/3504543285 learned_addr learned my addr 172.21.15.17:0/3504543285 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): get_auth_request con 0x7fd6580a1380 auth_method 0 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd65ca85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.781+0000 7fd64f7fe640 1 --2- 172.21.15.17:0/3504543285 >> v2:172.21.15.119:3300/0 conn(0x7fd65809ffc0 0x7fd65813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd64f7fe640 10 monclient(hunting): get_auth_request con 0x7fd65809ffc0 auth_method 0 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd64f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd64f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd64f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 10 monclient(hunting): handle_auth_done global_id 124272 payload 274 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 1 -- 172.21.15.17:0/3504543285 >> v2:172.21.15.119:3300/0 conn(0x7fd65809ffc0 msgr2=0x7fd65813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 1 --2- 172.21.15.17:0/3504543285 >> v2:172.21.15.119:3300/0 conn(0x7fd65809ffc0 0x7fd65813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 1 -- 172.21.15.17:0/3504543285 >> v1:172.21.15.17:6789/0 conn(0x7fd6580a0890 legacy=0x7fd6580a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 1 monclient: found mon.noname-a 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 1 -- 172.21.15.17:0/3504543285 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65809ccd0 con 0x7fd6580a1380 2024-09-15T21:28:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65ca85640 1 --2- 172.21.15.17:0/3504543285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd65813d010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd650009e50 tx=0x7fd65001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81dbf92d1a4117b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 1 -- 172.21.15.17:0/3504543285 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500a5680 con 0x7fd6580a1380 2024-09-15T21:28:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: dump: 2024-09-15T21:28:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 1 -- 172.21.15.17:0/3504543285 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6500a5820 con 0x7fd6580a1380 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 1 -- 172.21.15.17:0/3504543285 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500a5bf0 con 0x7fd6580a1380 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 5 monclient: authenticate success, global_id 124272 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 4 set_mon_vals no callback set 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd6577fe640 10 monclient: dump: 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 10 monclient: shutdown 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3504543285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 msgr2=0x7fd65813d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3504543285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd65813d010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd650009e50 tx=0x7fd65001c8b0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3504543285 shutdown_connections 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3504543285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd65813d010 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3504543285 >> v2:172.21.15.119:3300/0 conn(0x7fd65809ffc0 0x7fd65813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3504543285 >> 172.21.15.17:0/3504543285 conn(0x7fd6580a66a0 msgr2=0x7fd6580a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3504543285 shutdown_connections 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3504543285 wait complete. 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.785+0000 7fd65e4e8640 10 monclient: monmap: 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 Processor -- start 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- start start 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: init 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65809ffc0 0x7fd65811ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): picked mon.noname-c con 0x7fd65809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6580a0890 0x7fd658135b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): picked mon.noname-b con 0x7fd6580a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd6581cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): picked mon.noname-a con 0x7fd6580a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd658140100 con 0x7fd6580a1380 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd65813ff80 con 0x7fd6580a0890 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd658140280 con 0x7fd65809ffc0 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): authenticate will time out at 1616.184581s 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd6581cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd6581cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37174/0 (socket says 172.21.15.17:37174) 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 -- 172.21.15.17:0/3692872800 learned_addr learned my addr 172.21.15.17:0/3692872800 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): get_auth_request con 0x7fd6580a1380 auth_method 0 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65809ffc0 0x7fd65811ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): get_auth_request con 0x7fd65809ffc0 auth_method 0 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6580a0890 0x7fd658135b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 10 monclient(hunting): get_auth_request con 0x7fd6580a0890 auth_method 0 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient(hunting): handle_auth_done global_id 124275 payload 824 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65809ffc0 msgr2=0x7fd65811ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65809ffc0 0x7fd65811ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6580a0890 msgr2=0x7fd658135b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6580a0890 0x7fd658135b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 monclient: found mon.noname-a 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581204e0 con 0x7fd6580a1380 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65809ffc0 0x7fd65811ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65ca85640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd6581cd480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6500194c0 tx=0x7fd650019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500bc020 con 0x7fd6580a1380 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: dump: 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd650007bb0 con 0x7fd6580a1380 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 5 monclient: authenticate success, global_id 124275 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: _renew_subs 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65806f690 con 0x7fd6580a1380 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500a5680 con 0x7fd6580a1380 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: dump: 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd656ffd640 4 set_mon_vals no callback set 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: _renew_subs 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd658120200 con 0x7fd6580a1380 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 0x7fd6581cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd65809d0f0 con 0x7fd658136090 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 0x7fd6581cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient: get_auth_request con 0x7fd658136090 auth_method 0 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd6500bf020 con 0x7fd6580a1380 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 10 monclient(hunting): handle_auth_done global_id 134138 payload 824 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd65811dca0 con 0x7fd658136090 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd63803b9c0 0x7fd63803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd6500eb980 con 0x7fd6580a1380 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd657fff640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 0x7fd6581cd9c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd64800fae0 tx=0x7fd648001d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6480052e0 con 0x7fd658136090 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:12.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd63803b9c0 0x7fd63803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: dump: 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 10 monclient: get_auth_request con 0x7fd63803b9c0 auth_method 0 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd648018070 con 0x7fd658136090 2024-09-15T21:28:12.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:12.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 msgr2=0x7fd6581cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 0x7fd6581cd9c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd64800fae0 tx=0x7fd648001d80 comp rx=0 tx=0).stop 2024-09-15T21:28:12.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 msgr2=0x7fd6581cd9c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd6557fa640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd658136090 0x7fd6581cd9c0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd64800fae0 tx=0x7fd648001d80 comp rx=0 tx=0).stop 2024-09-15T21:28:12.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.789+0000 7fd64f7fe640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd63803b9c0 0x7fd63803de80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd6581ce540 tx=0x7fd640008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:12.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 0x7fd6581cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd6580a0c70 con 0x7fd65811cc10 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 0x7fd6581cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient: get_auth_request con 0x7fd65811cc10 auth_method 0 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 10 monclient(hunting): handle_auth_done global_id 134141 payload 824 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 1 -- 172.21.15.17:0/3692872800 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd65806d020 con 0x7fd65811cc10 2024-09-15T21:28:12.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd657fff640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 0x7fd6581cdf00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd648017000 tx=0x7fd648000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64800dca0 con 0x7fd65811cc10 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 10 monclient: dump: 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.813 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7fd64800dca0 con 0x7fd65811cc10 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 msgr2=0x7fd6581cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 0x7fd6581cdf00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd648017000 tx=0x7fd648000ee0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 msgr2=0x7fd6581cdf00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd6557fa640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd65811cc10 0x7fd6581cdf00 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd648017000 tx=0x7fd648000ee0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:12.814 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9951, 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:12.815 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:12.816 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.817 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:12.818 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.820 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.821 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:12.823 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.825 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:12.826 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd63803b9c0 msgr2=0x7fd63803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd63803b9c0 0x7fd63803de80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd6581ce540 tx=0x7fd640008040 comp rx=0 tx=0).stop 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 10 monclient: shutdown 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 msgr2=0x7fd6581cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd6581cd480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6500194c0 tx=0x7fd650019650 comp rx=0 tx=0).stop 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65ca85640 1 -- 172.21.15.17:0/3692872800 reap_dead start 2024-09-15T21:28:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 shutdown_connections 2024-09-15T21:28:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 --2- 172.21.15.17:0/3692872800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580a1380 0x7fd6581cd480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 >> 172.21.15.17:0/3692872800 conn(0x7fd6580a66a0 msgr2=0x7fd6580a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 shutdown_connections 2024-09-15T21:28:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.797+0000 7fd65e4e8640 1 -- 172.21.15.17:0/3692872800 wait complete. 2024-09-15T21:28:12.829 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:28:12.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:28:12.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:28:12.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient: get_monmap_and_config 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient: monmap: 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:12.946894+0000 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:12.946894+0000 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.947 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.948 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:12.948 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:12.948 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:12.948 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:12.948 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:12.948 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 1 Processor -- start 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 1 -- start start 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient: init 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70b00a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f70b009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70b00a1380 0x7f70b013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient(hunting): picked mon.noname-c con 0x7f70b00a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b00a3130 con 0x7f70b00a0890 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b0073df0 con 0x7f70b009ffc0 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f70b00a1bf0 con 0x7f70b00a1380 2024-09-15T21:28:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 10 monclient(hunting): authenticate will time out at 1616.342290s 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f70b00a0890 0x7f70b00a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41438/0 (socket says 172.21.15.17:41438) 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 1 -- 172.21.15.17:0/433609848 learned_addr learned my addr 172.21.15.17:0/433609848 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 --2- 172.21.15.17:0/433609848 >> v2:172.21.15.119:3300/0 conn(0x7f70b00a1380 0x7f70b013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760476926 0 0) 0x7f70b00a3130 con 0x7f70b00a0890 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): my global_id is 124278 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f709c003530 con 0x7f70b00a0890 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): get_auth_request con 0x7f70b00a1380 auth_method 0 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1219839503 0 0) 0x7f709c003530 con 0x7f70b00a0890 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 >> v2:172.21.15.119:3300/0 conn(0x7f70b00a1380 msgr2=0x7f70b013e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 --2- 172.21.15.17:0/433609848 >> v2:172.21.15.119:3300/0 conn(0x7f70b00a1380 0x7f70b013e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 >> v1:172.21.15.119:6789/0 conn(0x7f70b009ffc0 legacy=0x7f70b013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b009ccd0 con 0x7f70b00a0890 2024-09-15T21:28:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f7098001700 con 0x7f70b00a0890 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: dump: 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 >> v1:172.21.15.17:6789/0 conn(0x7f70b00a0890 legacy=0x7f70b00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f709c003a60 0x7f709c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): picked mon.c con 0x7f709c003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c004810 0x7f709c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): picked mon.b con 0x7f709c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709c0053d0 0x7f709c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): picked mon.a con 0x7f709c0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00a1bf0 con 0x7f709c0053d0 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f709c00c240 con 0x7f709c004810 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f709c00c440 con 0x7f709c003a60 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70af7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709c0053d0 0x7f709c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): get_auth_request con 0x7f709c0053d0 auth_method 0 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f709c003a60 0x7f709c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): get_auth_request con 0x7f709c003a60 auth_method 0 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient(hunting): handle_auth_done global_id 124278 payload 293 2024-09-15T21:28:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 -- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f709c003a60 msgr2=0x7f709c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f709c003a60 0x7f709c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 -- 172.21.15.17:0/433609848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c004810 msgr2=0x7f709c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c004810 0x7f709c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 monclient: found mon.a 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 1 -- 172.21.15.17:0/433609848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c003530 con 0x7f709c0053d0 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b5230640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 5 monclient: authenticate success, global_id 124278 2024-09-15T21:28:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.945+0000 7f70b6c93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b5230640 1 -- 172.21.15.17:0/433609848 reap_dead start 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b5230640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709c0053d0 0x7f709c00b9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009fc0 tx=0x7f70a801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7076c7134a3641be server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8009c10 con 0x7f709c0053d0 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: dump: 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a8003ba0 con 0x7f709c0053d0 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 1 -- 172.21.15.17:0/433609848 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80a59a0 con 0x7f709c0053d0 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 4 set_mon_vals no callback set 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70af7fe640 10 monclient: dump: 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 10 monclient: shutdown 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 -- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709c0053d0 msgr2=0x7f709c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709c0053d0 0x7f709c00b9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009fc0 tx=0x7f70a801cca0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 -- 172.21.15.17:0/433609848 shutdown_connections 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 --2- 172.21.15.17:0/433609848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709c0053d0 0x7f709c00b9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 -- 172.21.15.17:0/433609848 >> 172.21.15.17:0/433609848 conn(0x7f70b00a66a0 msgr2=0x7f70b00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 -- 172.21.15.17:0/433609848 shutdown_connections 2024-09-15T21:28:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 -- 172.21.15.17:0/433609848 wait complete. 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 10 monclient: build_initial_monmap 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 10 monclient: monmap: 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 Processor -- start 2024-09-15T21:28:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 1 -- start start 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.949+0000 7f70b6c93640 10 monclient: init 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c0053d0 0x7f70b01ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): picked mon.noname-b con 0x7f709c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70b01cf710 0x7f70b009d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): picked mon.noname-c con 0x7f70b01cf710 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b01cfb70 0x7f70b009d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): picked mon.noname-a con 0x7f70b01cfb70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00a1000 con 0x7f70b01cfb70 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00a0e80 con 0x7f709c0053d0 2024-09-15T21:28:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f70b00a1180 con 0x7f70b01cf710 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): authenticate will time out at 1616.348296s 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70b01cf710 0x7f70b009d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b01cfb70 0x7f70b009d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70b01cf710 0x7f70b009d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44630/0 (socket says 172.21.15.17:44630) 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 1 -- 172.21.15.17:0/797462588 learned_addr learned my addr 172.21.15.17:0/797462588 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): get_auth_request con 0x7f70b01cf710 auth_method 0 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): get_auth_request con 0x7f70b01cfb70 auth_method 0 2024-09-15T21:28:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c0053d0 0x7f70b01ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): get_auth_request con 0x7f709c0053d0 auth_method 0 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient(hunting): handle_auth_done global_id 124281 payload 824 2024-09-15T21:28:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70b01cf710 msgr2=0x7f70b009d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f70b01cf710 0x7f70b009d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c0053d0 msgr2=0x7f70b01ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f709c0053d0 0x7f70b01ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient: _finish_hunting 0 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 monclient: found mon.noname-a 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b013d0d0 con 0x7f70b01cfb70 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b5230640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b01cfb70 0x7f70b009d970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70b0074910 tx=0x7f70a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8009c10 con 0x7f70b01cfb70 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: dump: 2024-09-15T21:28:12.975 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: _finish_auth 0 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a80039a0 con 0x7f70b01cfb70 2024-09-15T21:28:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80adaa0 con 0x7f70b01cfb70 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: dump: 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.977 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 5 monclient: authenticate success, global_id 124281 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient: _renew_subs 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b013d3b0 con 0x7f70b01cfb70 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ae7fc640 4 set_mon_vals no callback set 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient: _renew_subs 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b009d0f0 con 0x7f70b01cfb70 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 0x7f70b009deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70b002e9b0 con 0x7f70b002e5d0 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f70a80c0020 con 0x7f70b01cfb70 2024-09-15T21:28:12.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 0x7f70b009deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient: get_auth_request con 0x7f70b002e5d0 auth_method 0 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f709003b9f0 0x7f709003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f70a80eb220 con 0x7f70b01cfb70 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f709003b9f0 0x7f709003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_done global_id 114316 payload 824 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70b013c050 con 0x7f70b002e5d0 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 10 monclient: get_auth_request con 0x7f709003b9f0 auth_method 0 2024-09-15T21:28:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70b4a2f640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 0x7f70b009deb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7098004ed0 tx=0x7f7098007430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098007740 con 0x7f70b002e5d0 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70ad7fa640 10 monclient: dump: 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.980 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.953+0000 7f70affff640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f709003b9f0 0x7f709003deb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f70b013da40 tx=0x7f70a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f709801a070 con 0x7f70b002e5d0 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 msgr2=0x7f70b009deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 0x7f70b009deb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7098004ed0 tx=0x7f7098007430 comp rx=0 tx=0).stop 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 msgr2=0x7f70b009deb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.957+0000 7f70ad7fa640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b002e5d0 0x7f70b009deb0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7098004ed0 tx=0x7f7098007430 comp rx=0 tx=0).stop 2024-09-15T21:28:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b6c93640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b6c93640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 0x7f70b00641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b6c93640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70b009eef0 con 0x7f70b0063e00 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 0x7f70b00641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient: get_auth_request con 0x7f70b0063e00 auth_method 0 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 10 monclient(hunting): handle_auth_done global_id 114319 payload 824 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.961+0000 7f70b4a2f640 1 -- 172.21.15.17:0/797462588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f70b01381b0 con 0x7f70b0063e00 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b4a2f640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 0x7f70b00641e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7098001170 tx=0x7f7098007900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098015880 con 0x7f70b0063e00 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 10 monclient: dump: 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7f7098015880 con 0x7f70b0063e00 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 msgr2=0x7f70b00641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 0x7f70b00641e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7098001170 tx=0x7f7098007900 comp rx=0 tx=0).stop 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 msgr2=0x7f70b00641e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70ad7fa640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0063e00 0x7f70b00641e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.985 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 101531, 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:12.986 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.987 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.988 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:12.989 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:12.990 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:12.992 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.994 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.995 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.996 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:12.997 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f709003b9f0 msgr2=0x7f709003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f709003b9f0 0x7f709003deb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f70b013da40 tx=0x7f70a0009040 comp rx=0 tx=0).stop 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 10 monclient: shutdown 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b01cfb70 msgr2=0x7f70b009d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b5230640 1 -- 172.21.15.17:0/797462588 reap_dead start 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b01cfb70 0x7f70b009d970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70b0074910 tx=0x7f70a8009750 comp rx=0 tx=0).stop 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 shutdown_connections 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 --2- 172.21.15.17:0/797462588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b01cfb70 0x7f70b009d970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 >> 172.21.15.17:0/797462588 conn(0x7f70b00a66a0 msgr2=0x7f70b00a4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 shutdown_connections 2024-09-15T21:28:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:12.965+0000 7f70b6c93640 1 -- 172.21.15.17:0/797462588 wait complete. 2024-09-15T21:28:13.000 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:28:13.000 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:28:13.000 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:28:13.000 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:28:13.000 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:28:13.000 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:28:13.001 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:28:13.015 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:28:13.015 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:28:13.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient: get_monmap_and_config 2024-09-15T21:28:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient: build_initial_monmap 2024-09-15T21:28:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient: monmap: 2024-09-15T21:28:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:13.172350+0000 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:13.172350+0000 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:13.173 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 Processor -- start 2024-09-15T21:28:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 -- start start 2024-09-15T21:28:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient: init 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61e01378c0 0x7f61e0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): picked mon.noname-c con 0x7f61e01378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f61e01381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61e00a5210 0x7f61e00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): picked mon.noname-a con 0x7f61e00a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e00a1d00 con 0x7f61e01381e0 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f61e00a1fe0 con 0x7f61e00a5210 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f61e00a22c0 con 0x7f61e01378c0 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e6854640 10 monclient(hunting): authenticate will time out at 1616.567728s 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e4df1640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f61e00a5210 msgr2=0x7f61e00a56a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.169+0000 7f61e4df1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61e00a5210 0x7f61e00a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61e01378c0 0x7f61e0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61e01378c0 0x7f61e0137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:36710/0 (socket says 172.21.15.17:36710) 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 -- 172.21.15.17:0/3975073089 learned_addr learned my addr 172.21.15.17:0/3975073089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): get_auth_request con 0x7f61e01378c0 auth_method 0 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient(hunting): handle_auth_done global_id 114322 payload 274 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 -- 172.21.15.17:0/3975073089 >> v2:172.21.15.17:3300/0 conn(0x7f61e00a5210 msgr2=0x7f61e00a56a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 --2- 172.21.15.17:0/3975073089 >> v2:172.21.15.17:3300/0 conn(0x7f61e00a5210 0x7f61e00a56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 -- 172.21.15.17:0/3975073089 >> v1:172.21.15.119:6789/0 conn(0x7f61e01381e0 legacy=0x7f61e00a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient: _finish_hunting 0 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 monclient: found mon.noname-c 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:28:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 -- 172.21.15.17:0/3975073089 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e00a4490 con 0x7f61e01378c0 2024-09-15T21:28:13.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61dffff640 1 --2- 172.21.15.17:0/3975073089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01378c0 0x7f61e0137ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009870 tx=0x7f61d001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f604841d6cf0aea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:13.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 1 -- 172.21.15.17:0/3975073089 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00ab620 con 0x7f61e01378c0 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: dump: 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: _finish_auth 0 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 1 -- 172.21.15.17:0/3975073089 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d00b0070 con 0x7f61e01378c0 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 1 -- 172.21.15.17:0/3975073089 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00ab9e0 con 0x7f61e01378c0 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:13.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61deffd640 10 monclient: dump: 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61e6854640 5 monclient: authenticate success, global_id 114322 2024-09-15T21:28:13.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61e6854640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.173+0000 7f61e6854640 4 set_mon_vals no callback set 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: shutdown 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/3975073089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01378c0 msgr2=0x7f61e0137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 --2- 172.21.15.17:0/3975073089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01378c0 0x7f61e0137ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009870 tx=0x7f61d001c400 comp rx=0 tx=0).stop 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/3975073089 shutdown_connections 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 --2- 172.21.15.17:0/3975073089 >> v2:172.21.15.17:3300/0 conn(0x7f61e00a5210 0x7f61e00a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 --2- 172.21.15.17:0/3975073089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01378c0 0x7f61e0137ca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/3975073089 >> 172.21.15.17:0/3975073089 conn(0x7f61e00aac40 msgr2=0x7f61e00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/3975073089 shutdown_connections 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/3975073089 wait complete. 2024-09-15T21:28:13.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: build_initial_monmap 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: monmap: 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 Processor -- start 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- start start 2024-09-15T21:28:13.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: init 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61e00a5210 0x7f61e01cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): picked mon.noname-a con 0x7f61e00a5210 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 0x7f61e01d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): picked mon.noname-c con 0x7f61e01378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01381e0 0x7f61e013cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): picked mon.noname-b con 0x7f61e01381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006f340 con 0x7f61e00a5210 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006f1c0 con 0x7f61e01381e0 2024-09-15T21:28:13.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f61e01425c0 con 0x7f61e01378c0 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient(hunting): authenticate will time out at 1616.573806s 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 0x7f61e01d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 0x7f61e01d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44662/0 (socket says 172.21.15.17:44662) 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 -- 172.21.15.17:0/1827492378 learned_addr learned my addr 172.21.15.17:0/1827492378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): get_auth_request con 0x7f61e01378c0 auth_method 0 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:13.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e4df1640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01381e0 0x7f61e013cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e4df1640 10 monclient(hunting): get_auth_request con 0x7f61e01381e0 auth_method 0 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e4df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e4df1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e4df1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient(hunting): handle_auth_done global_id 134147 payload 824 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 -- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01381e0 msgr2=0x7f61e013cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01381e0 0x7f61e013cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 -- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61e00a5210 msgr2=0x7f61e01cfaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61e00a5210 0x7f61e01cfaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 monclient: found mon.noname-c 2024-09-15T21:28:13.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 -- 172.21.15.17:0/1827492378 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e013e930 con 0x7f61e01378c0 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e4df1640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01381e0 0x7f61e013cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61df7fe640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 0x7f61e01d01d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c500 tx=0x7f61d400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017070 con 0x7f61e01378c0 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: dump: 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.193 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: _finish_auth 0 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d40052d0 con 0x7f61e01378c0 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 5 monclient: authenticate success, global_id 134147 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _renew_subs 2024-09-15T21:28:13.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e0120dd0 con 0x7f61e01378c0 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4012870 con 0x7f61e01378c0 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 10 monclient: dump: 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _renew_subs 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61de7fc640 4 set_mon_vals no callback set 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e013f490 con 0x7f61e01378c0 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e00a56a0 con 0x7f61e01378c0 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.177+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f61d401a070 con 0x7f61e01378c0 2024-09-15T21:28:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.181+0000 7f61dcff9640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61cc03b9c0 0x7f61cc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.181+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f61d404f880 con 0x7f61e01378c0 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.181+0000 7f61dffff640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61cc03b9c0 0x7f61cc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.181+0000 7f61dffff640 10 monclient: get_auth_request con 0x7f61cc03b9c0 auth_method 0 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.181+0000 7f61dffff640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61cc03b9c0 0x7f61cc03de80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f61d00af000 tx=0x7f61d00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.185+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f61d4005480 con 0x7f61e01378c0 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.185+0000 7f61dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.185+0000 7f61dcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:13.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61e6854640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:13.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61e6854640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:13.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61e6854640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:13.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f61e006cba0 con 0x7f61e01378c0 2024-09-15T21:28:13.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61dcff9640 1 -- 172.21.15.17:0/1827492378 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f61d4003d60 con 0x7f61e01378c0 2024-09-15T21:28:13.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:13.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.425+0000 7f61dcff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61cc03b9c0 msgr2=0x7f61cc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61cc03b9c0 0x7f61cc03de80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f61d00af000 tx=0x7f61d00a7000 comp rx=0 tx=0).stop 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 10 monclient: shutdown 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 msgr2=0x7f61e01d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 0x7f61e01d01d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c500 tx=0x7f61d400c9c0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 shutdown_connections 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61cc03b9c0 0x7f61cc03de80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61e01381e0 0x7f61e013cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61e01378c0 0x7f61e01d01d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 --2- 172.21.15.17:0/1827492378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61e00a5210 0x7f61e01cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 >> 172.21.15.17:0/1827492378 conn(0x7f61e00aac40 msgr2=0x7f61e006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 shutdown_connections 2024-09-15T21:28:13.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:13.429+0000 7f61e6854640 1 -- 172.21.15.17:0/1827492378 wait complete. 2024-09-15T21:28:16.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.577+0000 7fc021e70640 10 monclient: get_monmap_and_config 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.577+0000 7fc021e70640 10 monclient: build_initial_monmap 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.577+0000 7fc021e70640 10 monclient: monmap: 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:16.580127+0000 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:16.580127+0000 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:16.580 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:16.581 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:16.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.577+0000 7fc021e70640 1 Processor -- start 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.577+0000 7fc021e70640 1 -- start start 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.577+0000 7fc021e70640 10 monclient: init 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc01c1378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 0x7fc01c13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): picked mon.noname-b con 0x7fc01c1383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc01c13d2c0 0x7fc01c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): picked mon.noname-c con 0x7fc01c13d2c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0a1a20 con 0x7fc01c1378c0 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c0a1d00 con 0x7fc01c13d2c0 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0a1fe0 con 0x7fc01c1383b0 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient(hunting): authenticate will time out at 1619.976276s 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 0x7fc01c13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 0x7fc01c13cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44678/0 (socket says 172.21.15.17:44678) 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 -- 172.21.15.17:0/3155604839 learned_addr learned my addr 172.21.15.17:0/3155604839 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): get_auth_request con 0x7fc01c1383b0 auth_method 0 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01bfff640 1 --2- 172.21.15.17:0/3155604839 >> v2:172.21.15.119:3300/0 conn(0x7fc01c13d2c0 0x7fc01c13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01bfff640 10 monclient(hunting): get_auth_request con 0x7fc01c13d2c0 auth_method 0 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient(hunting): handle_auth_done global_id 134150 payload 274 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 -- 172.21.15.17:0/3155604839 >> v2:172.21.15.119:3300/0 conn(0x7fc01c13d2c0 msgr2=0x7fc01c13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 --2- 172.21.15.17:0/3155604839 >> v2:172.21.15.119:3300/0 conn(0x7fc01c13d2c0 0x7fc01c13f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 -- 172.21.15.17:0/3155604839 >> v1:172.21.15.119:6789/0 conn(0x7fc01c1378c0 legacy=0x7fc01c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:16.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:16.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 monclient: found mon.noname-b 2024-09-15T21:28:16.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:16.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:16.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 -- 172.21.15.17:0/3155604839 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0a22c0 con 0x7fc01c1383b0 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc012ffd640 1 --2- 172.21.15.17:0/3155604839 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 0x7fc01c13cd80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc004009870 tx=0x7fc00401c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36319b2b0cbe9651 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 1 -- 172.21.15.17:0/3155604839 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040b1070 con 0x7fc01c1383b0 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: dump: 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: _finish_auth 0 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 1 -- 172.21.15.17:0/3155604839 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0040a48b0 con 0x7fc01c1383b0 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 5 monclient: authenticate success, global_id 134150 2024-09-15T21:28:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 1 -- 172.21.15.17:0/3155604839 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040a4c80 con 0x7fc01c1383b0 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc021e70640 4 set_mon_vals no callback set 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.581+0000 7fc01affd640 10 monclient: dump: 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: shutdown 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/3155604839 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 msgr2=0x7fc01c13cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 --2- 172.21.15.17:0/3155604839 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 0x7fc01c13cd80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc004009870 tx=0x7fc00401c100 comp rx=0 tx=0).stop 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/3155604839 shutdown_connections 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 --2- 172.21.15.17:0/3155604839 >> v2:172.21.15.119:3300/0 conn(0x7fc01c13d2c0 0x7fc01c13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 --2- 172.21.15.17:0/3155604839 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1383b0 0x7fc01c13cd80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/3155604839 >> 172.21.15.17:0/3155604839 conn(0x7fc01c0aac40 msgr2=0x7fc01c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/3155604839 shutdown_connections 2024-09-15T21:28:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/3155604839 wait complete. 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: build_initial_monmap 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: monmap: 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 Processor -- start 2024-09-15T21:28:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- start start 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: init 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 0x7fc01c1d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): picked mon.noname-c con 0x7fc01c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc01c1383b0 0x7fc01c136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): picked mon.noname-a con 0x7fc01c1383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc01c13d2c0 0x7fc01c09ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): picked mon.noname-b con 0x7fc01c13d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c143040 con 0x7fc01c1383b0 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c142ec0 con 0x7fc01c13d2c0 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c1431c0 con 0x7fc01c1378c0 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient(hunting): authenticate will time out at 1619.982222s 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc012ffd640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc01c1383b0 msgr2=0x7fc01c136d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc01c1383b0 0x7fc01c136d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 0x7fc01c1d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 0x7fc01c1d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:44688/0 (socket says 172.21.15.17:44688) 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 -- 172.21.15.17:0/1773789267 learned_addr learned my addr 172.21.15.17:0/1773789267 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): get_auth_request con 0x7fc01c1378c0 auth_method 0 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01bfff640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc01c13d2c0 0x7fc01c09ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01bfff640 10 monclient(hunting): get_auth_request con 0x7fc01c13d2c0 auth_method 0 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient(hunting): handle_auth_done global_id 134153 payload 824 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 -- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc01c13d2c0 msgr2=0x7fc01c09ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc01c13d2c0 0x7fc01c09ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 -- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc01c1383b0 msgr2=0x7fc01c136d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc01c1383b0 0x7fc01c136d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 -- 172.21.15.17:0/1773789267 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c1216e0 con 0x7fc01c1378c0 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01b7fe640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 0x7fc01c1d07f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c4b0 tx=0x7fc00c00c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0052d0 con 0x7fc01c1378c0 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: dump: 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: _finish_auth 0 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c005470 con 0x7fc01c1378c0 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c012850 con 0x7fc01c1378c0 2024-09-15T21:28:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 5 monclient: authenticate success, global_id 134153 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: _renew_subs 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c120e40 con 0x7fc01c1378c0 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc01a7fc640 4 set_mon_vals no callback set 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc018ff9640 10 monclient: dump: 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: _renew_subs 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.585+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1d1670 con 0x7fc01c1378c0 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc021e70640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc021e70640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc021e70640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c137ca0 con 0x7fc01c1378c0 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc00c0129f0 con 0x7fc01c1378c0 2024-09-15T21:28:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc018ff9640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbffc03b9a0 0x7fbffc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc00c050220 con 0x7fc01c1378c0 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc012ffd640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbffc03b9a0 0x7fbffc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc012ffd640 10 monclient: get_auth_request con 0x7fbffc03b9a0 auth_method 0 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.589+0000 7fc012ffd640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbffc03b9a0 0x7fbffc03de60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc0040ae000 tx=0x7fc004008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.593+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc01c0a1a20 con 0x7fc01c1378c0 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.593+0000 7fc018ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.593+0000 7fc018ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc021e70640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc021e70640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc021e70640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc01c069270 con 0x7fc01c1378c0 2024-09-15T21:28:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc018ff9640 1 -- 172.21.15.17:0/1773789267 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fc00c019d10 con 0x7fc01c1378c0 2024-09-15T21:28:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc018ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.825+0000 7fc018ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:16.829 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbffc03b9a0 msgr2=0x7fbffc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbffc03b9a0 0x7fbffc03de60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc0040ae000 tx=0x7fc004008040 comp rx=0 tx=0).stop 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 10 monclient: shutdown 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 msgr2=0x7fc01c1d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 0x7fc01c1d07f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c4b0 tx=0x7fc00c00c970 comp rx=0 tx=0).stop 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 shutdown_connections 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbffc03b9a0 0x7fbffc03de60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc01c13d2c0 0x7fc01c09ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc01c1383b0 0x7fc01c136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 --2- 172.21.15.17:0/1773789267 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc01c1378c0 0x7fc01c1d07f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:16.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 >> 172.21.15.17:0/1773789267 conn(0x7fc01c0aac40 msgr2=0x7fc01c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:16.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 shutdown_connections 2024-09-15T21:28:16.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:16.829+0000 7fc021e70640 1 -- 172.21.15.17:0/1773789267 wait complete. 2024-09-15T21:28:19.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.973+0000 7f468812f640 10 monclient: get_monmap_and_config 2024-09-15T21:28:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.973+0000 7f468812f640 10 monclient: build_initial_monmap 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.973+0000 7f468812f640 10 monclient: monmap: 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:19.976785+0000 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:19.976785+0000 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:19.977 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.973+0000 7f468812f640 1 Processor -- start 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.973+0000 7f468812f640 1 -- start start 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.973+0000 7f468812f640 10 monclient: init 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f46801378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 0x7f468013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): picked mon.noname-b con 0x7f46801383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f468013d2c0 0x7f468013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): picked mon.noname-a con 0x7f468013d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46800a1a20 con 0x7f46801378c0 2024-09-15T21:28:19.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f46800a1d00 con 0x7f468013d2c0 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f46800a1fe0 con 0x7f46801383b0 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient(hunting): authenticate will time out at 1623.372255s 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46866cc640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f468013d2c0 msgr2=0x7f468013f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46866cc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f468013d2c0 0x7f468013f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 0x7f468013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 0x7f468013cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40642/0 (socket says 172.21.15.17:40642) 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 -- 172.21.15.17:0/1992958365 learned_addr learned my addr 172.21.15.17:0/1992958365 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): get_auth_request con 0x7f46801383b0 auth_method 0 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient(hunting): handle_auth_done global_id 134156 payload 274 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 -- 172.21.15.17:0/1992958365 >> v2:172.21.15.17:3300/0 conn(0x7f468013d2c0 msgr2=0x7f468013f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 --2- 172.21.15.17:0/1992958365 >> v2:172.21.15.17:3300/0 conn(0x7f468013d2c0 0x7f468013f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 -- 172.21.15.17:0/1992958365 >> v1:172.21.15.119:6789/0 conn(0x7f46801378c0 legacy=0x7f4680137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient: _finish_hunting 0 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 monclient: found mon.noname-b 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 -- 172.21.15.17:0/1992958365 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800a22c0 con 0x7f46801383b0 2024-09-15T21:28:19.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f46856ca640 1 --2- 172.21.15.17:0/1992958365 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 0x7f468013cd80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f467c009870 tx=0x7f467c01c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f18b2b160953fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:19.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 1 -- 172.21.15.17:0/1992958365 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c0b1070 con 0x7f46801383b0 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: dump: 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:19.988 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: _finish_auth 0 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: _check_auth_tickets 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 1 -- 172.21.15.17:0/1992958365 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c0a48d0 con 0x7f46801383b0 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 1 -- 172.21.15.17:0/1992958365 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a4ca0 con 0x7f46801383b0 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 5 monclient: authenticate success, global_id 134156 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 4 set_mon_vals no callback set 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f4684ec9640 10 monclient: dump: 2024-09-15T21:28:19.989 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 10 monclient: shutdown 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- 172.21.15.17:0/1992958365 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 msgr2=0x7f468013cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:19.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 --2- 172.21.15.17:0/1992958365 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 0x7f468013cd80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f467c009870 tx=0x7f467c01c120 comp rx=0 tx=0).stop 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- 172.21.15.17:0/1992958365 shutdown_connections 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 --2- 172.21.15.17:0/1992958365 >> v2:172.21.15.17:3300/0 conn(0x7f468013d2c0 0x7f468013f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 --2- 172.21.15.17:0/1992958365 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f46801383b0 0x7f468013cd80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- 172.21.15.17:0/1992958365 >> 172.21.15.17:0/1992958365 conn(0x7f46800aac40 msgr2=0x7f46800a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- 172.21.15.17:0/1992958365 shutdown_connections 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.977+0000 7f468812f640 1 -- 172.21.15.17:0/1992958365 wait complete. 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: build_initial_monmap 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: monmap: 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:19.991 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 Processor -- start 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- start start 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: init 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46801378c0 0x7f46800a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): picked mon.noname-a con 0x7f46801378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:19.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46801383b0 0x7f468009e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): picked mon.noname-b con 0x7f46801383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 0x7f4680136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): picked mon.noname-c con 0x7f468013d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4680143070 con 0x7f46801378c0 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4680142ef0 con 0x7f46801383b0 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f46801431f0 con 0x7f468013d2c0 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient(hunting): authenticate will time out at 1623.377276s 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f4685ecb640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46801378c0 msgr2=0x7f46800a0340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f4685ecb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46801378c0 0x7f46800a0340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 0x7f4680136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 0x7f4680136dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40644/0 (socket says 172.21.15.17:40644) 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 -- 172.21.15.17:0/4238015392 learned_addr learned my addr 172.21.15.17:0/4238015392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): get_auth_request con 0x7f468013d2c0 auth_method 0 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46856ca640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46801383b0 0x7f468009e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46856ca640 10 monclient(hunting): get_auth_request con 0x7f46801383b0 auth_method 0 2024-09-15T21:28:19.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46856ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46856ca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46856ca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient(hunting): handle_auth_done global_id 134159 payload 824 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 -- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46801383b0 msgr2=0x7f468009e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46801383b0 0x7f468009e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 -- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46801378c0 msgr2=0x7f46800a0340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46801378c0 0x7f46800a0340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient: _finish_hunting 0 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 monclient: found mon.noname-c 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 -- 172.21.15.17:0/4238015392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468009f7e0 con 0x7f468013d2c0 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46866cc640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 0x7f4680136dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f467800bdf0 tx=0x7f4678009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:19.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780064e0 con 0x7f468013d2c0 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: dump: 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:19.996 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: _finish_auth 0 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: _check_auth_tickets 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46780066a0 con 0x7f468013d2c0 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678006ad0 con 0x7f468013d2c0 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 10 monclient: dump: 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 5 monclient: authenticate success, global_id 134159 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f4677fff640 4 set_mon_vals no callback set 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _renew_subs 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4680120f10 con 0x7f468013d2c0 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _renew_subs 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f468009f0b0 con 0x7f468013d2c0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680137ca0 con 0x7f468013d2c0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4678015440 con 0x7f468013d2c0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f466403b970 0x7f466403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.981+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f46780517a0 con 0x7f468013d2c0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.985+0000 7f4685ecb640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f466403b970 0x7f466403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.985+0000 7f4685ecb640 10 monclient: get_auth_request con 0x7f466403b970 auth_method 0 2024-09-15T21:28:19.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.985+0000 7f4685ecb640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f466403b970 0x7f466403de30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4670000ee0 tx=0x7f46700079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:20.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.985+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4678024080 con 0x7f468013d2c0 2024-09-15T21:28:20.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.985+0000 7f46767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:20.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:19.985+0000 7f46767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:20.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.217+0000 7f468812f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:20.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.217+0000 7f468812f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:20.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.217+0000 7f468812f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:20.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.217+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f46801211f0 con 0x7f468013d2c0 2024-09-15T21:28:20.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f46767fc640 1 -- 172.21.15.17:0/4238015392 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f4680120f10 con 0x7f468013d2c0 2024-09-15T21:28:20.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f46767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:20.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f46767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:20.222 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:20.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f466403b970 msgr2=0x7f466403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:20.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f466403b970 0x7f466403de30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4670000ee0 tx=0x7f46700079e0 comp rx=0 tx=0).stop 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 10 monclient: shutdown 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 msgr2=0x7f4680136dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 0x7f4680136dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f467800bdf0 tx=0x7f4678009ee0 comp rx=0 tx=0).stop 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 shutdown_connections 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f466403b970 0x7f466403de30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f468013d2c0 0x7f4680136dc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46801383b0 0x7f468009e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 --2- 172.21.15.17:0/4238015392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46801378c0 0x7f46800a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 >> 172.21.15.17:0/4238015392 conn(0x7f46800aac40 msgr2=0x7f46800a32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 shutdown_connections 2024-09-15T21:28:20.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:20.221+0000 7f468812f640 1 -- 172.21.15.17:0/4238015392 wait complete. 2024-09-15T21:28:23.240 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient: get_monmap_and_config 2024-09-15T21:28:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient: build_initial_monmap 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient: monmap: 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:23.375634+0000 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:23.375634+0000 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:23.376 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 Processor -- start 2024-09-15T21:28:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 -- start start 2024-09-15T21:28:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient: init 2024-09-15T21:28:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 0x7fad040a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): picked mon.noname-a con 0x7fad040a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fad0409ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): picked mon.noname-c con 0x7fad040a11b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040a3130 con 0x7fad0409ffc0 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fad04073df0 con 0x7fad040a0890 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fad040a1bf0 con 0x7fad040a11b0 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad09688640 10 monclient(hunting): authenticate will time out at 1626.771246s 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad027fc640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fad0409ffc0 legacy=0x7fad04135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:28:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad02ffd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 msgr2=0x7fad040a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad02ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 0x7fad040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:57572/0 (socket says 172.21.15.17:57572) 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 learned_addr learned my addr 172.21.15.17:0/3115482996 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 10 monclient(hunting): get_auth_request con 0x7fad040a11b0 auth_method 0 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.373+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 10 monclient(hunting): get_auth_request con 0x7fad040a11b0 auth_method 0 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad02ffd640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 msgr2=0x7fad040a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> v1:172.21.15.17:6789/0 conn(0x7fad0409ffc0 legacy=0x7fad04135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 0x7fad040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:28:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.577+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:28:23.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> v1:172.21.15.17:6789/0 conn(0x7fad0409ffc0 legacy=0x7fad04135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:28:23.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad02ffd640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 msgr2=0x7fad040a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:23.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 0x7fad040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:28:23.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:23.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 10 monclient(hunting): get_auth_request con 0x7fad040a11b0 auth_method 0 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:23.981+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:28:24.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad02ffd640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 msgr2=0x7fad040a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:24.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> v1:172.21.15.17:6789/0 conn(0x7fad0409ffc0 legacy=0x7fad04135670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:28:24.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 0x7fad040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 10 monclient(hunting): get_auth_request con 0x7fad040a11b0 auth_method 0 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:24.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:24.785+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:28:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): tick 2024-09-15T21:28:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:28:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 monclient(hunting): continuing hunt 2024-09-15T21:28:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:28:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 msgr2=0x7fad04135d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.119:3300/0 conn(0x7fad040a11b0 0x7fad04135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 -- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 msgr2=0x7fad040a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 --2- 172.21.15.17:0/3115482996 >> v2:172.21.15.17:3300/0 conn(0x7fad040a0890 0x7fad040a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 -- 172.21.15.17:0/3115482996 >> v1:172.21.15.17:6789/0 conn(0x7fad0409ffc0 legacy=0x7fad04135670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient: _add_conns ranks=[1,2] 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7facf00036b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): picked mon.noname-b con 0x7facf00073a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 -- 172.21.15.17:0/3115482996 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040a15e0 con 0x7facf00036b0 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 1 -- 172.21.15.17:0/3115482996 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad04073df0 con 0x7facf00073a0 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad00ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.373+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 10 monclient(hunting): get_auth_request con 0x7facf00073a0 auth_method 0 2024-09-15T21:28:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:28:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.377+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 10 monclient(hunting): get_auth_request con 0x7facf00073a0 auth_method 0 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.581+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:28:26.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 10 monclient(hunting): get_auth_request con 0x7facf00073a0 auth_method 0 2024-09-15T21:28:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:26.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:28:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:26.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:26.985+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:28:27.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.537+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847541717 0 0) 0x7fad040a15e0 con 0x7facf00036b0 2024-09-15T21:28:27.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.537+0000 7fad01ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:28:27.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.537+0000 7fad01ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.537+0000 7fad01ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:27.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.537+0000 7fad01ffb640 10 monclient(hunting): my global_id is 114337 2024-09-15T21:28:27.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.537+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facec003530 con 0x7facf00036b0 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 123546661 0 0) 0x7facec003530 con 0x7facf00036b0 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 msgr2=0x7facf0009860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facf00073a0 0x7facf0009860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040a3130 con 0x7facf00036b0 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: _finish_auth 0 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:28:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7facfc0045c0 con 0x7facf00036b0 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: dump: 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.553 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 >> v1:172.21.15.119:6789/0 conn(0x7facf00036b0 legacy=0x7facf0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad037fe640 1 -- 172.21.15.17:0/3115482996 reap_dead start 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7facec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 10 monclient(hunting): picked mon.c con 0x7facec0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.545+0000 7fad01ffb640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004810 0x7facec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): picked mon.a con 0x7facec004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec0052f0 0x7facec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): picked mon.b con 0x7facec0052f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad04073df0 con 0x7facec004810 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7facec00c1a0 con 0x7facec0052f0 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7facec00c3a0 con 0x7facec0039c0 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad01ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad027fc640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004810 msgr2=0x7facec004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad027fc640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004810 0x7facec004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7facec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): get_auth_request con 0x7facec0039c0 auth_method 0 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec0052f0 0x7facec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): get_auth_request con 0x7facec0052f0 auth_method 0 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad037fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient(hunting): handle_auth_done global_id 114337 payload 293 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec0052f0 msgr2=0x7facec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:27.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec0052f0 0x7facec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004810 msgr2=0x7facec004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004810 0x7facec004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 monclient: found mon.c 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 1 -- 172.21.15.17:0/3115482996 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec003530 con 0x7facec0039c0 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient: _finish_auth 0 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad09688640 5 monclient: authenticate success, global_id 114337 2024-09-15T21:28:27.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.549+0000 7fad09688640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad02ffd640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7facec003dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7facfc003d10 tx=0x7facfc01cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d13d5b56e2486d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00c1e0 con 0x7facec0039c0 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: dump: 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.559 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc0a9030 con 0x7facec0039c0 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 1 -- 172.21.15.17:0/3115482996 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0060d0 con 0x7facec0039c0 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad09688640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad09688640 4 set_mon_vals no callback set 2024-09-15T21:28:27.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.553+0000 7fad01ffb640 10 monclient: dump: 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient: shutdown 2024-09-15T21:28:27.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 msgr2=0x7facec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7facec003dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7facfc003d10 tx=0x7facfc01cee0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- 172.21.15.17:0/3115482996 shutdown_connections 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec0052f0 0x7facec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004810 0x7facec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- 172.21.15.17:0/3115482996 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7facec003dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- 172.21.15.17:0/3115482996 >> 172.21.15.17:0/3115482996 conn(0x7fad040a66a0 msgr2=0x7fad040a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- 172.21.15.17:0/3115482996 shutdown_connections 2024-09-15T21:28:27.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- 172.21.15.17:0/3115482996 wait complete. 2024-09-15T21:28:27.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient: build_initial_monmap 2024-09-15T21:28:27.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient: monmap: 2024-09-15T21:28:27.566 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:27.566 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 Processor -- start 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- start start 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient: init 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:27.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7fad041cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): picked mon.noname-c con 0x7facec0039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec004810 0x7fad04134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): picked mon.noname-b con 0x7facec004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0052f0 0x7fad0409d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): picked mon.noname-a con 0x7facec0052f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad04136210 con 0x7facec0052f0 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad04136090 con 0x7facec004810 2024-09-15T21:28:27.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad04136390 con 0x7facec0039c0 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad09688640 10 monclient(hunting): authenticate will time out at 1630.955854s 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad037fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0052f0 msgr2=0x7fad0409d430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.557+0000 7fad037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0052f0 0x7fad0409d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7fad041cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7fad041cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40688/0 (socket says 172.21.15.17:40688) 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 -- 172.21.15.17:0/1495701243 learned_addr learned my addr 172.21.15.17:0/1495701243 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec004810 0x7fad04134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): get_auth_request con 0x7facec0039c0 auth_method 0 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:27.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): get_auth_request con 0x7facec004810 auth_method 0 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient(hunting): handle_auth_done global_id 134162 payload 824 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 -- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec004810 msgr2=0x7fad04134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec004810 0x7fad04134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 -- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0052f0 msgr2=0x7fad0409d430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0052f0 0x7fad0409d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 monclient: found mon.noname-c 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 -- 172.21.15.17:0/1495701243 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad04124080 con 0x7facec0039c0 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad02ffd640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7fad041cf260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7facfc004000 tx=0x7facec005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc003cf0 con 0x7facec0039c0 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: dump: 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.571 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc0060d0 con 0x7facec0039c0 2024-09-15T21:28:27.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 5 monclient: authenticate success, global_id 134162 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 monclient: _renew_subs 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad040a5a00 con 0x7facec0039c0 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0064a0 con 0x7facec0039c0 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 10 monclient: dump: 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.573 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad017fa640 4 set_mon_vals no callback set 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 monclient: _renew_subs 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad04123da0 con 0x7facec0039c0 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7facfc006860 con 0x7facec0039c0 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7facf003c170 0x7facf003e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:27.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad04138ab0 con 0x7facec0039c0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7facfc0ea770 con 0x7facec0039c0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7facf003c170 0x7facf003e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.561+0000 7fad027fc640 10 monclient: get_auth_request con 0x7facf003c170 auth_method 0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.565+0000 7fad027fc640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7facf003c170 0x7facf003e630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fad040732c0 tx=0x7facf8003ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.569+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7facfc016c90 con 0x7facec0039c0 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.569+0000 7faceb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:27.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.569+0000 7faceb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:27.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:27.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:27.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fad04135360 con 0x7facec0039c0 2024-09-15T21:28:27.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7faceb7fe640 1 -- 172.21.15.17:0/1495701243 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7facfc005b90 con 0x7facec0039c0 2024-09-15T21:28:27.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7faceb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:27.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7faceb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:27.827 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":104,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:27.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7facf003c170 msgr2=0x7facf003e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7facf003c170 0x7facf003e630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fad040732c0 tx=0x7facf8003ce0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 10 monclient: shutdown 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 msgr2=0x7fad041cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7fad041cf260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7facfc004000 tx=0x7facec005840 comp rx=0 tx=0).stop 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 shutdown_connections 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7facf003c170 0x7facf003e630 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0052f0 0x7fad0409d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facec004810 0x7fad04134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 --2- 172.21.15.17:0/1495701243 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7facec0039c0 0x7fad041cf260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 >> 172.21.15.17:0/1495701243 conn(0x7fad040a66a0 msgr2=0x7fad04070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 shutdown_connections 2024-09-15T21:28:27.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.825+0000 7fad09688640 1 -- 172.21.15.17:0/1495701243 wait complete. 2024-09-15T21:28:27.848 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:28:27.848 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:28:27.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient: get_monmap_and_config 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient: build_initial_monmap 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient: monmap: 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:27.975305+0000 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:27.975305+0000 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:27.975 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.976 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:27.976 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:27.976 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:27.976 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:27.976 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:27.976 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 1 Processor -- start 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 1 -- start start 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient: init 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb754073640 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb754074130 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 0x7fb754140710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient(hunting): picked mon.noname-b con 0x7fb75413e320 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7540a0890 con 0x7fb754073640 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7540746e0 con 0x7fb754074130 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb7540a1640 con 0x7fb75413e320 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb75938b640 10 monclient(hunting): authenticate will time out at 1631.370764s 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb752ffd640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fb754073640 legacy=0x7fb754073a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 0x7fb754140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 0x7fb754140710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57554/0 (socket says 172.21.15.17:57554) 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 -- 172.21.15.17:0/2416354337 learned_addr learned my addr 172.21.15.17:0/2416354337 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): get_auth_request con 0x7fb75413e320 auth_method 0 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient(hunting): handle_auth_done global_id 134165 payload 274 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 -- 172.21.15.17:0/2416354337 >> v1:172.21.15.119:6789/0 conn(0x7fb754074130 legacy=0x7fb75413cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 -- 172.21.15.17:0/2416354337 >> v1:172.21.15.17:6789/0 conn(0x7fb754073640 legacy=0x7fb754073a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 monclient: found mon.noname-b 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.973+0000 7fb7537fe640 1 -- 172.21.15.17:0/2416354337 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7540a1920 con 0x7fb75413e320 2024-09-15T21:28:27.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb7537fe640 1 --2- 172.21.15.17:0/2416354337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 0x7fb754140710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb744009e80 tx=0x7fb74401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14992690533a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:27.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 1 -- 172.21.15.17:0/2416354337 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7440a56e0 con 0x7fb75413e320 2024-09-15T21:28:27.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: dump: 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: _finish_auth 0 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:28:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 1 -- 172.21.15.17:0/2416354337 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7440a5880 con 0x7fb75413e320 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 5 monclient: authenticate success, global_id 134165 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 1 -- 172.21.15.17:0/2416354337 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7440a5c50 con 0x7fb75413e320 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 4 set_mon_vals no callback set 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb751ffb640 10 monclient: dump: 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 10 monclient: shutdown 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 1 -- 172.21.15.17:0/2416354337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 msgr2=0x7fb754140710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 1 --2- 172.21.15.17:0/2416354337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 0x7fb754140710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb744009e80 tx=0x7fb74401c8e0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 1 -- 172.21.15.17:0/2416354337 shutdown_connections 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 1 --2- 172.21.15.17:0/2416354337 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413e320 0x7fb754140710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.977+0000 7fb75938b640 1 -- 172.21.15.17:0/2416354337 >> 172.21.15.17:0/2416354337 conn(0x7fb7540a66a0 msgr2=0x7fb7540a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- 172.21.15.17:0/2416354337 shutdown_connections 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- 172.21.15.17:0/2416354337 wait complete. 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: build_initial_monmap 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: monmap: 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 Processor -- start 2024-09-15T21:28:27.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- start start 2024-09-15T21:28:27.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: init 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754073640 0x7fb75413cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): picked mon.noname-c con 0x7fb754073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb754074130 0x7fb754133570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): picked mon.noname-a con 0x7fb754074130 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75413e320 0x7fb75413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): picked mon.noname-b con 0x7fb75413e320 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541435f0 con 0x7fb754074130 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb754143470 con 0x7fb75413e320 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb754143770 con 0x7fb754073640 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): authenticate will time out at 1631.377584s 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb754074130 msgr2=0x7fb754133570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb754074130 0x7fb754133570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754073640 0x7fb75413cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754073640 0x7fb75413cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57562/0 (socket says 172.21.15.17:57562) 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 -- 172.21.15.17:0/3892569602 learned_addr learned my addr 172.21.15.17:0/3892569602 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): get_auth_request con 0x7fb754073640 auth_method 0 2024-09-15T21:28:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7537fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75413e320 0x7fb75413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient(hunting): handle_auth_done global_id 134168 payload 824 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75413e320 msgr2=0x7fb75413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75413e320 0x7fb75413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb754074130 msgr2=0x7fb754133570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb754074130 0x7fb754133570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 monclient: found mon.noname-c 2024-09-15T21:28:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7541d0710 con 0x7fb754073640 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb752ffd640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754073640 0x7fb75413cec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00b580 tx=0x7fb74c00ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c016070 con 0x7fb754073640 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: dump: 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c003df0 con 0x7fb754073640 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c005310 con 0x7fb754073640 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 10 monclient: dump: 2024-09-15T21:28:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 5 monclient: authenticate success, global_id 134168 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: _renew_subs 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7517fa640 4 set_mon_vals no callback set 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75411cef0 con 0x7fb754073640 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: _renew_subs 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75413d850 con 0x7fb754073640 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 0x7fb75413b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb75413b700 con 0x7fb75413ade0 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 0x7fb75413b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient: get_auth_request con 0x7fb75413ade0 auth_method 0 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb74c0054b0 con 0x7fb754073640 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 10 monclient(hunting): handle_auth_done global_id 134171 payload 824 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb754073a20 con 0x7fb75413ade0 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb73403b970 0x7fb73403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb74c04f8d0 con 0x7fb754073640 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.981+0000 7fb7527fc640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 0x7fb75413b1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb73c007fd0 tx=0x7fb73c00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c017070 con 0x7fb75413ade0 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 10 monclient: dump: 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb7537fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb73403b970 0x7fb73403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb7537fe640 10 monclient: get_auth_request con 0x7fb73403b970 auth_method 0 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb73c0074e0 con 0x7fb75413ade0 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 msgr2=0x7fb75413b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 0x7fb75413b1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb73c007fd0 tx=0x7fb73c00c810 comp rx=0 tx=0).stop 2024-09-15T21:28:28.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 msgr2=0x7fb75413b1c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:28.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb74b7fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb75413ade0 0x7fb75413b1c0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb73c007fd0 tx=0x7fb73c00c810 comp rx=0 tx=0).stop 2024-09-15T21:28:28.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.985+0000 7fb7537fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb73403b970 0x7fb73403de30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ae000 tx=0x7fb7440a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 0x7fb754136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb75411c710 con 0x7fb754136880 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 0x7fb754136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient: get_auth_request con 0x7fb754136880 auth_method 0 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 10 monclient(hunting): handle_auth_done global_id 134174 payload 824 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 1 -- 172.21.15.17:0/3892569602 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb75406d020 con 0x7fb754136880 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7527fc640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 0x7fb754136c90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb73c004600 tx=0x7fb73c0037f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c017030 con 0x7fb754136880 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 10 monclient: dump: 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7fb73c0192d0 con 0x7fb754136880 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 msgr2=0x7fb754136c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 0x7fb754136c90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb73c004600 tx=0x7fb73c0037f0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 msgr2=0x7fb754136c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb74b7fe640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754136880 0x7fb754136c90 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb73c004600 tx=0x7fb73c0037f0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 104, 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25143, 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:28.005 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.006 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:28.008 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:28.009 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:28.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:28.011 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:28.014 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:28.015 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.016 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:28.017 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb73403b970 msgr2=0x7fb73403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb73403b970 0x7fb73403de30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ae000 tx=0x7fb7440a7070 comp rx=0 tx=0).stop 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 10 monclient: shutdown 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754073640 msgr2=0x7fb75413cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 --2- 172.21.15.17:0/3892569602 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb754073640 0x7fb75413cec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00b580 tx=0x7fb74c00ba40 comp rx=0 tx=0).stop 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7537fe640 1 -- 172.21.15.17:0/3892569602 reap_dead start 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb7537fe640 1 -- 172.21.15.17:0/3892569602 reap_dead start 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 shutdown_connections 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 >> 172.21.15.17:0/3892569602 conn(0x7fb7540a66a0 msgr2=0x7fb7540a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 shutdown_connections 2024-09-15T21:28:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:27.989+0000 7fb75938b640 1 -- 172.21.15.17:0/3892569602 wait complete. 2024-09-15T21:28:28.020 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:28:28.020 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:28:28.020 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:28:28.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient: get_monmap_and_config 2024-09-15T21:28:28.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient: build_initial_monmap 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient: monmap: 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:28.138614+0000 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:28.138614+0000 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:28.139 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 Processor -- start 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 -- start start 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient: init 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4814073640 0x7f4814073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): picked mon.noname-a con 0x7f4814073640 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 0x7f4814137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): picked mon.noname-b con 0x7f4814073f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4814137bd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48140a0890 con 0x7f4814137bd0 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f48140746e0 con 0x7f4814073640 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f48140a1640 con 0x7f4814073f60 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4818b25640 10 monclient(hunting): authenticate will time out at 1631.534158s 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f481259c640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f4814073640 msgr2=0x7f4814073a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f481259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4814073640 0x7f4814073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 0x7f4814137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 0x7f4814137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57596/0 (socket says 172.21.15.17:57596) 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 -- 172.21.15.17:0/3773133303 learned_addr learned my addr 172.21.15.17:0/3773133303 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): get_auth_request con 0x7f4814073f60 auth_method 0 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient(hunting): handle_auth_done global_id 134177 payload 274 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 -- 172.21.15.17:0/3773133303 >> v2:172.21.15.17:3300/0 conn(0x7f4814073640 msgr2=0x7f4814073a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3773133303 >> v2:172.21.15.17:3300/0 conn(0x7f4814073640 0x7f4814073a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 -- 172.21.15.17:0/3773133303 >> v1:172.21.15.119:6789/0 conn(0x7f4814137bd0 legacy=0x7f481413ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 monclient: found mon.noname-b 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:28.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.137+0000 7f4811d9b640 1 -- 172.21.15.17:0/3773133303 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48140a1920 con 0x7f4814073f60 2024-09-15T21:28:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3773133303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 0x7f4814137690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4804009870 tx=0x7f480401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47aadc3f2655391 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 1 -- 172.21.15.17:0/3773133303 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040b0070 con 0x7f4814073f60 2024-09-15T21:28:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: dump: 2024-09-15T21:28:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: _finish_auth 0 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: _check_auth_tickets 2024-09-15T21:28:28.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 1 -- 172.21.15.17:0/3773133303 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48040ab600 con 0x7f4814073f60 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 1 -- 172.21.15.17:0/3773133303 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040ab9d0 con 0x7f4814073f60 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 5 monclient: authenticate success, global_id 134177 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 4 set_mon_vals no callback set 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481159a640 10 monclient: dump: 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.153 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: shutdown 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- 172.21.15.17:0/3773133303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 msgr2=0x7f4814137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 --2- 172.21.15.17:0/3773133303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 0x7f4814137690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4804009870 tx=0x7f480401c4d0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- 172.21.15.17:0/3773133303 shutdown_connections 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 --2- 172.21.15.17:0/3773133303 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073f60 0x7f4814137690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 --2- 172.21.15.17:0/3773133303 >> v2:172.21.15.17:3300/0 conn(0x7f4814073640 0x7f4814073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- 172.21.15.17:0/3773133303 >> 172.21.15.17:0/3773133303 conn(0x7f48140a66a0 msgr2=0x7f48140a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:28.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- 172.21.15.17:0/3773133303 shutdown_connections 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- 172.21.15.17:0/3773133303 wait complete. 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: build_initial_monmap 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: monmap: 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 Processor -- start 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- start start 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: init 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073640 0x7f48141cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): picked mon.noname-c con 0x7f4814073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814073f60 0x7f48141328b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): picked mon.noname-b con 0x7f4814073f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4814137bd0 0x7f481413cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): picked mon.noname-a con 0x7f4814137bd0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f481409ce50 con 0x7f4814137bd0 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f481409ccd0 con 0x7f4814073f60 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f48140a0d00 con 0x7f4814073640 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4818b25640 10 monclient(hunting): authenticate will time out at 1631.539730s 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4812d9d640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4814137bd0 msgr2=0x7f481413cc20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f4812d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4814137bd0 0x7f481413cc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073640 0x7f48141cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073640 0x7f48141cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57612/0 (socket says 172.21.15.17:57612) 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 1 -- 172.21.15.17:0/3836407363 learned_addr learned my addr 172.21.15.17:0/3836407363 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 10 monclient(hunting): get_auth_request con 0x7f4814073640 auth_method 0 2024-09-15T21:28:28.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.141+0000 7f481259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814073f60 0x7f48141328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): get_auth_request con 0x7f4814073f60 auth_method 0 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 10 monclient(hunting): handle_auth_done global_id 134180 payload 824 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814073f60 msgr2=0x7f48141328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814073f60 0x7f48141328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4814137bd0 msgr2=0x7f481413cc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4814137bd0 0x7f481413cc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 10 monclient: _finish_hunting 0 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 monclient: found mon.noname-c 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481411cf60 con 0x7f4814073640 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f481259c640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073640 0x7f48141cfb70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00b550 tx=0x7f47fc00ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc003dd0 con 0x7f4814073640 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: dump: 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: _finish_auth 0 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47fc015070 con 0x7f4814073640 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:28.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc005330 con 0x7f4814073640 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 5 monclient: authenticate success, global_id 134180 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 10 monclient: _renew_subs 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481413f180 con 0x7f4814073640 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480bfff640 4 set_mon_vals no callback set 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: dump: 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 10 monclient: _renew_subs 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f481413e620 con 0x7f4814073640 2024-09-15T21:28:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 0x7f481413d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4814073a20 con 0x7f4814132df0 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f47fc0054d0 con 0x7f4814073640 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 0x7f481413d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47f403b9a0 0x7f47f403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient: get_auth_request con 0x7f4814132df0 auth_method 0 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f47fc050440 con 0x7f4814073640 2024-09-15T21:28:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4812d9d640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47f403b9a0 0x7f47f403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4812d9d640 10 monclient: get_auth_request con 0x7f47f403b9a0 auth_method 0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 10 monclient(hunting): handle_auth_done global_id 114346 payload 824 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48141d09f0 con 0x7f4814132df0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 0x7f481413d160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48140a3d20 tx=0x7f4804019270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f4812d9d640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47f403b9a0 0x7f47f403de60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f480c003b00 tx=0x7f480c007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040bb070 con 0x7f4814132df0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.145+0000 7f480affd640 10 monclient: dump: 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4804009490 con 0x7f4814132df0 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 msgr2=0x7f481413d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 0x7f481413d160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48140a3d20 tx=0x7f4804019270 comp rx=0 tx=0).stop 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 msgr2=0x7f481413d160 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.149+0000 7f480affd640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4814132df0 0x7f481413d160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4818b25640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4818b25640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 0x7f481413d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4818b25640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f481406d020 con 0x7f481411c9a0 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 0x7f481413d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient: get_auth_request con 0x7f481411c9a0 auth_method 0 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 10 monclient(hunting): handle_auth_done global_id 114349 payload 824 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 1 -- 172.21.15.17:0/3836407363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f48141daee0 con 0x7f481411c9a0 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f4811d9b640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 0x7f481413d6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f480401cd70 tx=0x7f4804009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040b0040 con 0x7f481411c9a0 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f480affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.153+0000 7f480affd640 10 monclient: dump: 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4871 (secure 0 0 0) 0x7f4804003e50 con 0x7f481411c9a0 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 msgr2=0x7f481413d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 0x7f481413d6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f480401cd70 tx=0x7f4804009240 comp rx=0 tx=0).stop 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 msgr2=0x7f481413d6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:28:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f480affd640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f481411c9a0 0x7f481413d6a0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f480401cd70 tx=0x7f4804009240 comp rx=0 tx=0).stop 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 104, 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 116722, 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:28.171 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:28.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:28.173 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:28:28.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:28:28.175 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:28:28.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:28:28.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:28:28.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:28:28.180 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:28:28.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:28:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:28:28.183 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:28:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47f403b9a0 msgr2=0x7f47f403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f47f403b9a0 0x7f47f403de60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f480c003b00 tx=0x7f480c007d20 comp rx=0 tx=0).stop 2024-09-15T21:28:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 10 monclient: shutdown 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073640 msgr2=0x7f48141cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 --2- 172.21.15.17:0/3836407363 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4814073640 0x7f48141cfb70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00b550 tx=0x7f47fc00ba10 comp rx=0 tx=0).stop 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4812d9d640 1 -- 172.21.15.17:0/3836407363 reap_dead start 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4812d9d640 1 -- 172.21.15.17:0/3836407363 reap_dead start 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 shutdown_connections 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 >> 172.21.15.17:0/3836407363 conn(0x7f48140a66a0 msgr2=0x7f48140a54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 shutdown_connections 2024-09-15T21:28:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:28.157+0000 7f4818b25640 1 -- 172.21.15.17:0/3836407363 wait complete. 2024-09-15T21:28:28.185 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:28:28.186 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:28:48.187 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:28:48.187 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:28:48.188 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:28:48.191 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:28:48.191 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:28:48.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:48.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient: get_monmap_and_config 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient: build_initial_monmap 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient: monmap: 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:48.319032+0000 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:48.319032+0000 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:48.319 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:48.320 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:48.320 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:48.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:48.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 Processor -- start 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 -- start start 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient: init 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 0x7f3d5009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): picked mon.noname-a con 0x7f3d5009d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d5009dd20 0x7f3d50134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): picked mon.noname-c con 0x7f3d5009dd20 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3d50134ba0 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d500a3130 con 0x7f3d50134ba0 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3d500a1bc0 con 0x7f3d5009d400 2024-09-15T21:28:48.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3d500a3490 con 0x7f3d5009dd20 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d56510640 10 monclient(hunting): authenticate will time out at 1651.714254s 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 0x7f3d5009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d54aad640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3d50134ba0 0x7f3d5013bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48846/0 (socket says 172.21.15.17:48846) 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d54aad640 1 -- 172.21.15.17:0/114135460 learned_addr learned my addr 172.21.15.17:0/114135460 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 10 monclient(hunting): get_auth_request con 0x7f3d5009d400 auth_method 0 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 1 -- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 msgr2=0x7f3d5009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 1 -- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 msgr2=0x7f3d5009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:48.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 1 --2- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 0x7f3d5009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 1 --2- 172.21.15.17:0/114135460 >> v2:172.21.15.119:3300/0 conn(0x7f3d5009dd20 0x7f3d50134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4ffff640 1 --2- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 0x7f3d5009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): get_auth_request con 0x7f3d5009dd20 auth_method 0 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_done global_id 114352 payload 274 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 msgr2=0x7f3d5009d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 1 --2- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 0x7f3d5009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/114135460 >> v1:172.21.15.17:6789/0 conn(0x7f3d50134ba0 legacy=0x7f3d5013bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:48.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:48.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:28:48.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:48.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:28:48.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.317+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/114135460 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5009e7a0 con 0x7f3d5009dd20 2024-09-15T21:28:48.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4f7fe640 1 --2- 172.21.15.17:0/114135460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009dd20 0x7f3d50134660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d50074430 tx=0x7f3d4401c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c927f349970f139 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:48.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 1 -- 172.21.15.17:0/114135460 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440b0070 con 0x7f3d5009dd20 2024-09-15T21:28:48.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: dump: 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:48.332 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: _finish_auth 0 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 1 -- 172.21.15.17:0/114135460 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d440ab600 con 0x7f3d5009dd20 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 1 -- 172.21.15.17:0/114135460 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440ab9d0 con 0x7f3d5009dd20 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d4effd640 10 monclient: dump: 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:48.333 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 5 monclient: authenticate success, global_id 114352 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 4 set_mon_vals no callback set 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 10 monclient: shutdown 2024-09-15T21:28:48.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 -- 172.21.15.17:0/114135460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009dd20 msgr2=0x7f3d50134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 --2- 172.21.15.17:0/114135460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009dd20 0x7f3d50134660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d50074430 tx=0x7f3d4401c340 comp rx=0 tx=0).stop 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 -- 172.21.15.17:0/114135460 shutdown_connections 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 --2- 172.21.15.17:0/114135460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009dd20 0x7f3d50134660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 --2- 172.21.15.17:0/114135460 >> v2:172.21.15.17:3300/0 conn(0x7f3d5009d400 0x7f3d5009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 -- 172.21.15.17:0/114135460 >> 172.21.15.17:0/114135460 conn(0x7f3d500a66a0 msgr2=0x7f3d500a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 -- 172.21.15.17:0/114135460 shutdown_connections 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 -- 172.21.15.17:0/114135460 wait complete. 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 10 monclient: build_initial_monmap 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 10 monclient: monmap: 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:48.335 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 Processor -- start 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 1 -- start start 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.321+0000 7f3d56510640 10 monclient: init 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:28:48.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009d400 0x7f3d50132810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): picked mon.noname-b con 0x7f3d5009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 0x7f3d50132d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): picked mon.noname-c con 0x7f3d5009dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): picked mon.noname-a con 0x7f3d50134ba0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5009ce50 con 0x7f3d50134ba0 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5009ccd0 con 0x7f3d5009d400 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5009fa00 con 0x7f3d5009dd20 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:48.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient(hunting): authenticate will time out at 1651.720303s 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 0x7f3d50132d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 0x7f3d50132d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35888/0 (socket says 172.21.15.17:35888) 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/3207366460 learned_addr learned my addr 172.21.15.17:0/3207366460 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43900/0 (socket says 172.21.15.17:43900) 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): get_auth_request con 0x7f3d5009dd20 auth_method 0 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 10 monclient(hunting): get_auth_request con 0x7f3d50134ba0 auth_method 0 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:48.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 1 -- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 msgr2=0x7f3d5011c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 1 -- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 msgr2=0x7f3d5011c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d54aad640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009d400 0x7f3d50132810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 10 monclient(hunting): get_auth_request con 0x7f3d5009d400 auth_method 0 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient(hunting): handle_auth_done global_id 134183 payload 824 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009d400 msgr2=0x7f3d50132810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009d400 0x7f3d50132810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 msgr2=0x7f3d5011c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:28:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 -- 172.21.15.17:0/3207366460 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5011de70 con 0x7f3d5009dd20 2024-09-15T21:28:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009d400 0x7f3d50132810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:28:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4f7fe640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 0x7f3d50132d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d4401c820 tx=0x7f3d440190e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440b8020 con 0x7f3d5009dd20 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: dump: 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:48.344 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: _finish_auth 0 2024-09-15T21:28:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: _check_auth_tickets 2024-09-15T21:28:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d440b4600 con 0x7f3d5009dd20 2024-09-15T21:28:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 5 monclient: authenticate success, global_id 134183 2024-09-15T21:28:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _renew_subs 2024-09-15T21:28:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5009d0f0 con 0x7f3d5009dd20 2024-09-15T21:28:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440b0040 con 0x7f3d5009dd20 2024-09-15T21:28:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 10 monclient: dump: 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:48.347 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _renew_subs 2024-09-15T21:28:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4dffb640 4 set_mon_vals no callback set 2024-09-15T21:28:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5011f930 con 0x7f3d5009dd20 2024-09-15T21:28:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5009d7e0 con 0x7f3d5009dd20 2024-09-15T21:28:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3d440bb020 con 0x7f3d5009dd20 2024-09-15T21:28:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d3c03b9c0 0x7f3d3c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3d440ea640 con 0x7f3d5009dd20 2024-09-15T21:28:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d3c03b9c0 0x7f3d3c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 10 monclient: get_auth_request con 0x7f3d3c03b9c0 auth_method 0 2024-09-15T21:28:48.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.325+0000 7f3d4ffff640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d3c03b9c0 0x7f3d3c03de80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3d40006aa0 tx=0x7f3d40006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.329+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3d44016050 con 0x7f3d5009dd20 2024-09-15T21:28:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.329+0000 7f3d4cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:48.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.329+0000 7f3d4cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d56510640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d56510640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d56510640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3d5006d020 con 0x7f3d5009dd20 2024-09-15T21:28:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d4cff9640 1 -- 172.21.15.17:0/3207366460 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f3d440b49b0 con 0x7f3d5009dd20 2024-09-15T21:28:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d4cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.565+0000 7f3d4cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:48.569 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":104,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:48.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d3c03b9c0 msgr2=0x7f3d3c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d3c03b9c0 0x7f3d3c03de80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3d40006aa0 tx=0x7f3d40006a30 comp rx=0 tx=0).stop 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 10 monclient: shutdown 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 msgr2=0x7f3d50132d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 0x7f3d50132d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d4401c820 tx=0x7f3d440190e0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 shutdown_connections 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3d3c03b9c0 0x7f3d3c03de80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d50134ba0 0x7f3d5011c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3d5009dd20 0x7f3d50132d50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 --2- 172.21.15.17:0/3207366460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d5009d400 0x7f3d50132810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 >> 172.21.15.17:0/3207366460 conn(0x7f3d500a66a0 msgr2=0x7f3d50133800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 shutdown_connections 2024-09-15T21:28:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:48.569+0000 7f3d56510640 1 -- 172.21.15.17:0/3207366460 wait complete. 2024-09-15T21:28:51.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:51.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient: get_monmap_and_config 2024-09-15T21:28:51.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient: build_initial_monmap 2024-09-15T21:28:51.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient: monmap: 2024-09-15T21:28:51.723 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:51.723 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:51.723670+0000 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:51.723670+0000 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:51.724 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:51.725 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:51.725 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:51.725 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 Processor -- start 2024-09-15T21:28:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 -- start start 2024-09-15T21:28:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient: init 2024-09-15T21:28:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fad6c09d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 0x7fad6c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): picked mon.noname-b con 0x7fad6c09def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad6c139ff0 0x7fad6c13c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): picked mon.noname-c con 0x7fad6c139ff0 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c0a3130 con 0x7fad6c09d400 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c0a1bc0 con 0x7fad6c139ff0 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0a3490 con 0x7fad6c09def0 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad70ca7640 10 monclient(hunting): authenticate will time out at 1655.119114s 2024-09-15T21:28:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 0x7fad6c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 0x7fad6c139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35932/0 (socket says 172.21.15.17:35932) 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 -- 172.21.15.17:0/1745186868 learned_addr learned my addr 172.21.15.17:0/1745186868 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): get_auth_request con 0x7fad6c09def0 auth_method 0 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 1 --2- 172.21.15.17:0/1745186868 >> v2:172.21.15.119:3300/0 conn(0x7fad6c139ff0 0x7fad6c13c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): get_auth_request con 0x7fad6c139ff0 auth_method 0 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient(hunting): handle_auth_done global_id 134186 payload 274 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 -- 172.21.15.17:0/1745186868 >> v2:172.21.15.119:3300/0 conn(0x7fad6c139ff0 msgr2=0x7fad6c13c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 --2- 172.21.15.17:0/1745186868 >> v2:172.21.15.119:3300/0 conn(0x7fad6c139ff0 0x7fad6c13c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 -- 172.21.15.17:0/1745186868 >> v1:172.21.15.119:6789/0 conn(0x7fad6c09d400 legacy=0x7fad6c09d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:28:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 monclient: found mon.noname-b 2024-09-15T21:28:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.721+0000 7fad69d9b640 1 -- 172.21.15.17:0/1745186868 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c09e7a0 con 0x7fad6c09def0 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad69d9b640 1 --2- 172.21.15.17:0/1745186868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 0x7fad6c139ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00bb10 tx=0x7fad5c01eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa63e79780806cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 1 -- 172.21.15.17:0/1745186868 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0b4070 con 0x7fad6c09def0 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: dump: 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: _finish_auth 0 2024-09-15T21:28:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: _check_auth_tickets 2024-09-15T21:28:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 1 -- 172.21.15.17:0/1745186868 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5c0a78c0 con 0x7fad6c09def0 2024-09-15T21:28:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 5 monclient: authenticate success, global_id 134186 2024-09-15T21:28:51.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:51.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:51.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:51.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 1 -- 172.21.15.17:0/1745186868 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0a7c90 con 0x7fad6c09def0 2024-09-15T21:28:51.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 4 set_mon_vals no callback set 2024-09-15T21:28:51.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:51.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad6959a640 10 monclient: dump: 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:51.743 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 10 monclient: shutdown 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 -- 172.21.15.17:0/1745186868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 msgr2=0x7fad6c139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 --2- 172.21.15.17:0/1745186868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 0x7fad6c139ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00bb10 tx=0x7fad5c01eaf0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 -- 172.21.15.17:0/1745186868 shutdown_connections 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 --2- 172.21.15.17:0/1745186868 >> v2:172.21.15.119:3300/0 conn(0x7fad6c139ff0 0x7fad6c13c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 --2- 172.21.15.17:0/1745186868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c09def0 0x7fad6c139ab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 -- 172.21.15.17:0/1745186868 >> 172.21.15.17:0/1745186868 conn(0x7fad6c0a66a0 msgr2=0x7fad6c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 -- 172.21.15.17:0/1745186868 shutdown_connections 2024-09-15T21:28:51.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.725+0000 7fad70ca7640 1 -- 172.21.15.17:0/1745186868 wait complete. 2024-09-15T21:28:51.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: build_initial_monmap 2024-09-15T21:28:51.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: monmap: 2024-09-15T21:28:51.745 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:51.745 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:51.745 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:51.745 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:51.746 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:51.746 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:51.746 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:51.746 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:51.746 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 Processor -- start 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- start start 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: init 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): picked mon.noname-a con 0x7fad6c09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad6c09def0 0x7fad6c11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): picked mon.noname-b con 0x7fad6c09def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 0x7fad6c11cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): picked mon.noname-c con 0x7fad6c139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:51.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0a1220 con 0x7fad6c09d400 2024-09-15T21:28:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0a10a0 con 0x7fad6c09def0 2024-09-15T21:28:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0a13a0 con 0x7fad6c139ff0 2024-09-15T21:28:51.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient(hunting): authenticate will time out at 1655.125289s 2024-09-15T21:28:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 0x7fad6c11cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 0x7fad6c11cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35946/0 (socket says 172.21.15.17:35946) 2024-09-15T21:28:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 -- 172.21.15.17:0/230224022 learned_addr learned my addr 172.21.15.17:0/230224022 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): get_auth_request con 0x7fad6c139ff0 auth_method 0 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43916/0 (socket says 172.21.15.17:43916) 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 10 monclient(hunting): get_auth_request con 0x7fad6c09d400 auth_method 0 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad6c09def0 0x7fad6c11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): get_auth_request con 0x7fad6c09def0 auth_method 0 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 -- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 msgr2=0x7fad6c11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:28:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 -- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 msgr2=0x7fad6c11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient(hunting): handle_auth_done global_id 134189 payload 824 2024-09-15T21:28:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 -- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad6c09def0 msgr2=0x7fad6c11ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad6c09def0 0x7fad6c11ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 -- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 msgr2=0x7fad6c11c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 monclient: found mon.noname-c 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 -- 172.21.15.17:0/230224022 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c11e850 con 0x7fad6c139ff0 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6ad9d640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 0x7fad6c11cf60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fad6400efc0 tx=0x7fad6400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64018020 con 0x7fad6c139ff0 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: dump: 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: _finish_auth 0 2024-09-15T21:28:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad64014480 con 0x7fad6c139ff0 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64014870 con 0x7fad6c139ff0 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 5 monclient: authenticate success, global_id 134189 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _renew_subs 2024-09-15T21:28:51.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c073e30 con 0x7fad6c139ff0 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 10 monclient: dump: 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:51.756 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:51.758 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad68d99640 4 set_mon_vals no callback set 2024-09-15T21:28:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _renew_subs 2024-09-15T21:28:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c0a5640 con 0x7fad6c139ff0 2024-09-15T21:28:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:51.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fad6401b020 con 0x7fad6c139ff0 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c09d7e0 con 0x7fad6c139ff0 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad4c03b9c0 0x7fad4c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fad640514d0 con 0x7fad6c139ff0 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad4c03b9c0 0x7fad4c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.729+0000 7fad6a59c640 10 monclient: get_auth_request con 0x7fad4c03b9c0 auth_method 0 2024-09-15T21:28:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.733+0000 7fad6a59c640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad4c03b9c0 0x7fad4c03de80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fad60007c60 tx=0x7fad60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:51.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.737+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fad64022030 con 0x7fad6c139ff0 2024-09-15T21:28:51.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.737+0000 7fad5affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:51.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.737+0000 7fad5affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fad6c06d020 con 0x7fad6c139ff0 2024-09-15T21:28:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad5affd640 1 -- 172.21.15.17:0/230224022 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fad64010040 con 0x7fad6c139ff0 2024-09-15T21:28:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad5affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad5affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:51.979 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":104,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad4c03b9c0 msgr2=0x7fad4c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad4c03b9c0 0x7fad4c03de80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fad60007c60 tx=0x7fad60009040 comp rx=0 tx=0).stop 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 10 monclient: shutdown 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 msgr2=0x7fad6c11cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 0x7fad6c11cf60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fad6400efc0 tx=0x7fad6400c490 comp rx=0 tx=0).stop 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 shutdown_connections 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fad4c03b9c0 0x7fad4c03de80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fad6c139ff0 0x7fad6c11cf60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad6c09def0 0x7fad6c11ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 --2- 172.21.15.17:0/230224022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad6c09d400 0x7fad6c11c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 >> 172.21.15.17:0/230224022 conn(0x7fad6c0a66a0 msgr2=0x7fad6c13a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 shutdown_connections 2024-09-15T21:28:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:51.977+0000 7fad70ca7640 1 -- 172.21.15.17:0/230224022 wait complete. 2024-09-15T21:28:54.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient: get_monmap_and_config 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient: build_initial_monmap 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient: monmap: 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:55.132488+0000 2024-09-15T21:28:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:55.132488+0000 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:55.133 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:55.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 1 Processor -- start 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 1 -- start start 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient: init 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f94e00a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 0x7f94e013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient(hunting): picked mon.noname-b con 0x7f94e009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 10 monclient(hunting): picked mon.noname-a con 0x7f94e00a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.129+0000 7f94e597e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94e00a3130 con 0x7f94e00a0890 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:55.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f94e0073df0 con 0x7f94e00a1380 2024-09-15T21:28:55.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:55.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f94e00a1bf0 con 0x7f94e009ffc0 2024-09-15T21:28:55.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 10 monclient(hunting): authenticate will time out at 1658.528034s 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 0x7f94e013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94deffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f94e00a0890 0x7f94e00a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48854/0 (socket says 172.21.15.17:48854) 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 0x7f94e013aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35960/0 (socket says 172.21.15.17:35960) 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94deffd640 1 -- 172.21.15.17:0/4243391676 learned_addr learned my addr 172.21.15.17:0/4243391676 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43926/0 (socket says 172.21.15.17:43926) 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): get_auth_request con 0x7f94e009ffc0 auth_method 0 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 10 monclient(hunting): get_auth_request con 0x7f94e00a1380 auth_method 0 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 1 -- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 msgr2=0x7f94e013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 1 -- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 msgr2=0x7f94e013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 1 --2- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94df7fe640 1 --2- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient(hunting): handle_auth_done global_id 134192 payload 274 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 -- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 msgr2=0x7f94e013d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 --2- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 -- 172.21.15.17:0/4243391676 >> v1:172.21.15.17:6789/0 conn(0x7f94e00a0890 legacy=0x7f94e00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 monclient: found mon.noname-b 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 -- 172.21.15.17:0/4243391676 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e009ccd0 con 0x7f94e009ffc0 2024-09-15T21:28:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94de7fc640 1 --2- 172.21.15.17:0/4243391676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 0x7f94e013aac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d401c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6636a033ffc69c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 1 -- 172.21.15.17:0/4243391676 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40a6700 con 0x7f94e009ffc0 2024-09-15T21:28:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: dump: 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: _finish_auth 0 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: _check_auth_tickets 2024-09-15T21:28:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 1 -- 172.21.15.17:0/4243391676 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d40a68a0 con 0x7f94e009ffc0 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 1 -- 172.21.15.17:0/4243391676 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40a6c70 con 0x7f94e009ffc0 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94ddffb640 10 monclient: dump: 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 5 monclient: authenticate success, global_id 134192 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 4 set_mon_vals no callback set 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 10 monclient: shutdown 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- 172.21.15.17:0/4243391676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 msgr2=0x7f94e013aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 --2- 172.21.15.17:0/4243391676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 0x7f94e013aac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d401c830 comp rx=0 tx=0).stop 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- 172.21.15.17:0/4243391676 shutdown_connections 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 --2- 172.21.15.17:0/4243391676 >> v2:172.21.15.17:3300/0 conn(0x7f94e00a1380 0x7f94e013d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 --2- 172.21.15.17:0/4243391676 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e009ffc0 0x7f94e013aac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- 172.21.15.17:0/4243391676 >> 172.21.15.17:0/4243391676 conn(0x7f94e00a66a0 msgr2=0x7f94e00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- 172.21.15.17:0/4243391676 shutdown_connections 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.133+0000 7f94e597e640 1 -- 172.21.15.17:0/4243391676 wait complete. 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: build_initial_monmap 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: monmap: 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 Processor -- start 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- start start 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: init 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 0x7f94e011eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): picked mon.noname-a con 0x7f94e009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94e00a0890 0x7f94e011cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): picked mon.noname-b con 0x7f94e00a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 0x7f94e01359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): picked mon.noname-c con 0x7f94e00a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0140100 con 0x7f94e009ffc0 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94e013ff80 con 0x7f94e00a0890 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f94e0140280 con 0x7f94e00a1380 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient(hunting): authenticate will time out at 1658.533144s 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 0x7f94e01359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 0x7f94e011eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 0x7f94e01359e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35970/0 (socket says 172.21.15.17:35970) 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 -- 172.21.15.17:0/4041592025 learned_addr learned my addr 172.21.15.17:0/4041592025 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): get_auth_request con 0x7f94e00a1380 auth_method 0 2024-09-15T21:28:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 10 monclient(hunting): get_auth_request con 0x7f94e009ffc0 auth_method 0 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 1 -- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 msgr2=0x7f94e011eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 1 -- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 msgr2=0x7f94e011eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 0x7f94e011eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:55.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 0x7f94e011eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient(hunting): handle_auth_done global_id 134195 payload 824 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94de7fc640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94e00a0890 0x7f94e011cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 -- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94e00a0890 msgr2=0x7f94e011cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94e00a0890 0x7f94e011cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 -- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 msgr2=0x7f94e011eb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 0x7f94e011eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 monclient: found mon.noname-c 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 -- 172.21.15.17:0/4041592025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e013d0d0 con 0x7f94e00a1380 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94df7fe640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 0x7f94e01359e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006e90 tx=0x7f94d800ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:55.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d800f040 con 0x7f94e00a1380 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: dump: 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:55.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: _finish_auth 0 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d8018070 con 0x7f94e00a1380 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8013870 con 0x7f94e00a1380 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 10 monclient: dump: 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 5 monclient: authenticate success, global_id 134195 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _renew_subs 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94dd7fa640 4 set_mon_vals no callback set 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e009d0f0 con 0x7f94e00a1380 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _renew_subs 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e0120370 con 0x7f94e00a1380 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f94d8013a10 con 0x7f94e00a1380 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e00a0c70 con 0x7f94e00a1380 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f94cc03b9f0 0x7f94cc03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f94d8050300 con 0x7f94e00a1380 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f94cc03b9f0 0x7f94cc03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.137+0000 7f94deffd640 10 monclient: get_auth_request con 0x7f94cc03b9f0 auth_method 0 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.141+0000 7f94deffd640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f94cc03b9f0 0x7f94cc03deb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f94d0000f70 tx=0x7f94d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.141+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f94d801dc10 con 0x7f94e00a1380 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.141+0000 7f94c7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.141+0000 7f94c7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:55.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94e597e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:55.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94e597e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:55.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94e597e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:55.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f94e006d020 con 0x7f94e00a1380 2024-09-15T21:28:55.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94c7fff640 1 -- 172.21.15.17:0/4041592025 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f94d800c390 con 0x7f94e00a1380 2024-09-15T21:28:55.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94c7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:55.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.381+0000 7f94c7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:55.385 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":104,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:55.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f94cc03b9f0 msgr2=0x7f94cc03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:55.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f94cc03b9f0 0x7f94cc03deb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f94d0000f70 tx=0x7f94d0009040 comp rx=0 tx=0).stop 2024-09-15T21:28:55.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 10 monclient: shutdown 2024-09-15T21:28:55.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:55.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 msgr2=0x7f94e01359e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:55.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 0x7f94e01359e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006e90 tx=0x7f94d800ea60 comp rx=0 tx=0).stop 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 shutdown_connections 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f94cc03b9f0 0x7f94cc03deb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f94e00a1380 0x7f94e01359e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94e00a0890 0x7f94e011cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 --2- 172.21.15.17:0/4041592025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e009ffc0 0x7f94e011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 >> 172.21.15.17:0/4041592025 conn(0x7f94e00a66a0 msgr2=0x7f94e00a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 shutdown_connections 2024-09-15T21:28:55.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:55.385+0000 7f94e597e640 1 -- 172.21.15.17:0/4041592025 wait complete. 2024-09-15T21:28:58.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:28:58.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.537+0000 7fb0dca29640 10 monclient: get_monmap_and_config 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.537+0000 7fb0dca29640 10 monclient: build_initial_monmap 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.537+0000 7fb0dca29640 10 monclient: monmap: 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:28:58.540045+0000 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:28:58.540045+0000 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:28:58.540 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:28:58.541 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:28:58.541 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.537+0000 7fb0dca29640 1 Processor -- start 2024-09-15T21:28:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.537+0000 7fb0dca29640 1 -- start start 2024-09-15T21:28:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.537+0000 7fb0dca29640 10 monclient: init 2024-09-15T21:28:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb0d809d400 0x7fb0d809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): picked mon.noname-c con 0x7fb0d809d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 0x7fb0d8134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): picked mon.noname-b con 0x7fb0d809dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb0d8134ba0 addr v1:172.21.15.17:6789/0 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d80a3490 con 0x7fb0d8134ba0 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d80a1bc0 con 0x7fb0d809d400 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d80a0320 con 0x7fb0d809dd20 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient(hunting): authenticate will time out at 1661.936237s 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 0x7fb0d8134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb0d8134ba0 0x7fb0d813b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39328/0 (socket says 172.21.15.17:39328) 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 0x7fb0d8134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53982/0 (socket says 172.21.15.17:53982) 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d6d9d640 1 -- 172.21.15.17:0/2221399036 learned_addr learned my addr 172.21.15.17:0/2221399036 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): get_auth_request con 0x7fb0d809dd20 auth_method 0 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d659c640 1 --2- 172.21.15.17:0/2221399036 >> v2:172.21.15.119:3300/0 conn(0x7fb0d809d400 0x7fb0d809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d659c640 10 monclient(hunting): get_auth_request con 0x7fb0d809d400 auth_method 0 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient(hunting): handle_auth_done global_id 134198 payload 274 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 -- 172.21.15.17:0/2221399036 >> v2:172.21.15.119:3300/0 conn(0x7fb0d809d400 msgr2=0x7fb0d809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 --2- 172.21.15.17:0/2221399036 >> v2:172.21.15.119:3300/0 conn(0x7fb0d809d400 0x7fb0d809d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 -- 172.21.15.17:0/2221399036 >> v1:172.21.15.17:6789/0 conn(0x7fb0d8134ba0 legacy=0x7fb0d813b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient: _finish_hunting 0 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 monclient: found mon.noname-b 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:28:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 -- 172.21.15.17:0/2221399036 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d809e7a0 con 0x7fb0d809dd20 2024-09-15T21:28:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0cffff640 1 --2- 172.21.15.17:0/2221399036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 0x7fb0d8134660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097c0 tx=0x7fb0c001cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56f53c94a10c9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 1 -- 172.21.15.17:0/2221399036 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00b0070 con 0x7fb0d809dd20 2024-09-15T21:28:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:28:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: dump: 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: _finish_auth 0 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 1 -- 172.21.15.17:0/2221399036 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c0004230 con 0x7fb0d809dd20 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 5 monclient: authenticate success, global_id 134198 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient: get_monmap_and_config success 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 4 set_mon_vals no callback set 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 1 -- 172.21.15.17:0/2221399036 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003620 con 0x7fb0d809dd20 2024-09-15T21:28:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0d5d9b640 10 monclient: dump: 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 10 monclient: shutdown 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- 172.21.15.17:0/2221399036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 msgr2=0x7fb0d8134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 --2- 172.21.15.17:0/2221399036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 0x7fb0d8134660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097c0 tx=0x7fb0c001cd90 comp rx=0 tx=0).stop 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- 172.21.15.17:0/2221399036 shutdown_connections 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 --2- 172.21.15.17:0/2221399036 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d809dd20 0x7fb0d8134660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 --2- 172.21.15.17:0/2221399036 >> v2:172.21.15.119:3300/0 conn(0x7fb0d809d400 0x7fb0d809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- 172.21.15.17:0/2221399036 >> 172.21.15.17:0/2221399036 conn(0x7fb0d80a66a0 msgr2=0x7fb0d80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- 172.21.15.17:0/2221399036 shutdown_connections 2024-09-15T21:28:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.541+0000 7fb0dca29640 1 -- 172.21.15.17:0/2221399036 wait complete. 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: build_initial_monmap 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: monmap: 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:28:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 Processor -- start 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- start start 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: init 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _reopen_session rank -1 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 0x7fb0d81cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): picked mon.noname-a con 0x7fb0d809d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d809dd20 0x7fb0d8131910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): picked mon.noname-b con 0x7fb0d809dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:28:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 0x7fb0d81398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): picked mon.noname-c con 0x7fb0d8134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d80a3130 con 0x7fb0d809d400 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d809e130 con 0x7fb0d809dd20 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): start opening mon connection 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d80a32b0 con 0x7fb0d8134ba0 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): _renew_subs 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient(hunting): authenticate will time out at 1661.941388s 2024-09-15T21:28:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 0x7fb0d81398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 0x7fb0d81398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53984/0 (socket says 172.21.15.17:53984) 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 -- 172.21.15.17:0/302869322 learned_addr learned my addr 172.21.15.17:0/302869322 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): get_auth_request con 0x7fb0d8134ba0 auth_method 0 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 0x7fb0d81cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:28:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 10 monclient(hunting): get_auth_request con 0x7fb0d809d400 auth_method 0 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 1 -- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 msgr2=0x7fb0d81cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 1 -- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 msgr2=0x7fb0d81cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0cffff640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d809dd20 0x7fb0d8131910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 0x7fb0d81cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0cffff640 10 monclient(hunting): get_auth_request con 0x7fb0d809dd20 auth_method 0 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:28:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d659c640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 0x7fb0d81cd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient(hunting): handle_auth_done global_id 134201 payload 824 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 -- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d809dd20 msgr2=0x7fb0d8131910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d809dd20 0x7fb0d8131910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 -- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 msgr2=0x7fb0d81cd770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 0x7fb0d81cd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 monclient: found mon.noname-c 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:28:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 -- 172.21.15.17:0/302869322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d813b1f0 con 0x7fb0d8134ba0 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d6d9d640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 0x7fb0d81398f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000cb50 tx=0x7fb0d000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0017070 con 0x7fb0d8134ba0 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: dump: 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: _finish_auth 0 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: _check_auth_tickets 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d000ad40 con 0x7fb0d8134ba0 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 5 monclient: authenticate success, global_id 134201 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _renew_subs 2024-09-15T21:28:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d813cd60 con 0x7fb0d8134ba0 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0012600 con 0x7fb0d8134ba0 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 10 monclient: dump: 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:28:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0d559a640 4 set_mon_vals no callback set 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _renew_subs 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d811ca10 con 0x7fb0d8134ba0 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:58.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d809d7e0 con 0x7fb0d8134ba0 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb0d001a070 con 0x7fb0d8134ba0 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0b803b9c0 0x7fb0b803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.545+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb0d0050330 con 0x7fb0d8134ba0 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.549+0000 7fb0d659c640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0b803b9c0 0x7fb0b803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.549+0000 7fb0d659c640 10 monclient: get_auth_request con 0x7fb0b803b9c0 auth_method 0 2024-09-15T21:28:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.549+0000 7fb0d659c640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0b803b9c0 0x7fb0b803de80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b990 tx=0x7fb0c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:28:58.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.553+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb0d0007070 con 0x7fb0d8134ba0 2024-09-15T21:28:58.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.553+0000 7fb0ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:28:58.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.553+0000 7fb0ceffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:28:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0dca29640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:28:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0dca29640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0dca29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:28:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb0d813bd50 con 0x7fb0d8134ba0 2024-09-15T21:28:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0ceffd640 1 -- 172.21.15.17:0/302869322 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fb0d0011cc0 con 0x7fb0d8134ba0 2024-09-15T21:28:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:28:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.785+0000 7fb0ceffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:28:58.788 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":104,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":31,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:28:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0b803b9c0 msgr2=0x7fb0b803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0b803b9c0 0x7fb0b803de80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b990 tx=0x7fb0c8008040 comp rx=0 tx=0).stop 2024-09-15T21:28:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 10 monclient: shutdown 2024-09-15T21:28:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:28:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 msgr2=0x7fb0d81398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 0x7fb0d81398f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000cb50 tx=0x7fb0d000a430 comp rx=0 tx=0).stop 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 shutdown_connections 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb0b803b9c0 0x7fb0b803de80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb0d8134ba0 0x7fb0d81398f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d809dd20 0x7fb0d8131910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 --2- 172.21.15.17:0/302869322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0d809d400 0x7fb0d81cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 >> 172.21.15.17:0/302869322 conn(0x7fb0d80a66a0 msgr2=0x7fb0d80a5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 shutdown_connections 2024-09-15T21:28:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:28:58.789+0000 7fb0dca29640 1 -- 172.21.15.17:0/302869322 wait complete. 2024-09-15T21:29:01.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:01.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient: get_monmap_and_config 2024-09-15T21:29:01.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient: build_initial_monmap 2024-09-15T21:29:01.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient: monmap: 2024-09-15T21:29:01.939 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:01.939 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:01.939464+0000 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:01.939464+0000 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:01.940 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 Processor -- start 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 -- start start 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient: init 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): picked mon.noname-b con 0x7f5aac0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5aac09ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): picked mon.noname-c con 0x7f5aac0a11b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aac0a3130 con 0x7f5aac09ffc0 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5aac073df0 con 0x7f5aac0a11b0 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac0a1bf0 con 0x7f5aac0a0890 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab3291640 10 monclient(hunting): authenticate will time out at 1665.334859s 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:54008/0 (socket says 172.21.15.17:54008) 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 learned_addr learned my addr 172.21.15.17:0/1952564329 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5aac0a0890 auth_method 0 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 10 monclient(hunting): get_auth_request con 0x7f5aac0a11b0 auth_method 0 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:01.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:01.937+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5aac0a0890 auth_method 0 2024-09-15T21:29:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 10 monclient(hunting): get_auth_request con 0x7f5aac0a11b0 auth_method 0 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:02.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.141+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5aac0a0890 auth_method 0 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 10 monclient(hunting): get_auth_request con 0x7f5aac0a11b0 auth_method 0 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:02.545+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5aac0a0890 auth_method 0 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 10 monclient(hunting): get_auth_request con 0x7f5aac0a11b0 auth_method 0 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:03.349+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): tick 2024-09-15T21:29:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 monclient(hunting): continuing hunt 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac0a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5aac0a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 msgr2=0x7f5aac135d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.119:3300/0 conn(0x7f5aac0a11b0 0x7f5aac135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 -- 172.21.15.17:0/1952564329 >> v1:172.21.15.119:6789/0 conn(0x7f5aac09ffc0 legacy=0x7f5aac135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient: _add_conns ranks=[3,0] 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): picked mon.noname-a con 0x7f5a980036f0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5a98006220 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 -- 172.21.15.17:0/1952564329 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aac0a15e0 con 0x7f5a98006220 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 1 -- 172.21.15.17:0/1952564329 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5aac073df0 con 0x7f5a980036f0 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5aaaffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5a980036f0 auth_method 0 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:04.937+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5a980036f0 auth_method 0 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:05.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.141+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:05.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:05.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5a980036f0 auth_method 0 2024-09-15T21:29:05.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:05.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:05.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:05.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:05.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:05.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:05.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.545+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861582545 0 0) 0x7f5aac0a15e0 con 0x7f5a98006220 2024-09-15T21:29:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): _negotiate 2024-09-15T21:29:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): my global_id is 144099 2024-09-15T21:29:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a94003530 con 0x7f5a98006220 2024-09-15T21:29:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3964789485 0 0) 0x7f5a94003530 con 0x7f5a98006220 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 msgr2=0x7f5a98005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 --2- 172.21.15.17:0/1952564329 >> v2:172.21.15.17:3300/0 conn(0x7f5a980036f0 0x7f5a98005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: _finish_hunting 0 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9800af60 con 0x7f5a98006220 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: _finish_auth 0 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: _check_auth_tickets 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5a9c001b40 con 0x7f5a98006220 2024-09-15T21:29:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: dump: 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:05.984 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 >> v1:172.21.15.17:6789/0 conn(0x7f5a98006220 legacy=0x7f5a980086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 1 -- 172.21.15.17:0/1952564329 reap_dead start 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5a94003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): picked mon.c con 0x7f5aac0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a940040c0 0x7f5a94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): picked mon.b con 0x7f5a940040c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5a9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): picked mon.a con 0x7f5a94004e70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac073df0 con 0x7f5a94004e70 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94005690 con 0x7f5a940040c0 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9400bf60 con 0x7f5aac0a0890 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5aabfff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5a94003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5aac0a0890 auth_method 0 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5a9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a940040c0 0x7f5a94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): get_auth_request con 0x7f5a94004e70 auth_method 0 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): get_auth_request con 0x7f5a940040c0 auth_method 0 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab182e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab102d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.977+0000 7f5ab082c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 10 monclient(hunting): handle_auth_done global_id 144099 payload 293 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a940040c0 msgr2=0x7f5a94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a940040c0 0x7f5a94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 msgr2=0x7f5a9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5a9400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 10 monclient: _finish_hunting 0 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 1 monclient: found mon.c 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 1 -- 172.21.15.17:0/1952564329 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a94003530 con 0x7f5aac0a0890 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 10 monclient: _finish_auth 0 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 10 monclient: _check_auth_tickets 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab102d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab3291640 5 monclient: authenticate success, global_id 144099 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab3291640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.981+0000 7f5ab182e640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5a9400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:05.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab102d640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5a94003a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0018b20 tx=0x7f5aa001dd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206086e640ef54fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00ac070 con 0x7f5aac0a0890 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: dump: 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa00a6b00 con 0x7f5aac0a0890 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 4 set_mon_vals no callback set 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 1 -- 172.21.15.17:0/1952564329 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000f040 con 0x7f5aac0a0890 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5aabfff640 10 monclient: dump: 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 10 monclient: shutdown 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 1 -- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 msgr2=0x7f5a94003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.985+0000 7f5ab3291640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5a94003a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0018b20 tx=0x7f5aa001dd00 comp rx=0 tx=0).stop 2024-09-15T21:29:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 -- 172.21.15.17:0/1952564329 shutdown_connections 2024-09-15T21:29:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5a9400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a940040c0 0x7f5a94004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 --2- 172.21.15.17:0/1952564329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5aac0a0890 0x7f5a94003a60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 -- 172.21.15.17:0/1952564329 >> 172.21.15.17:0/1952564329 conn(0x7f5aac0a66a0 msgr2=0x7f5aac0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 -- 172.21.15.17:0/1952564329 shutdown_connections 2024-09-15T21:29:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 -- 172.21.15.17:0/1952564329 wait complete. 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 10 monclient: build_initial_monmap 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 10 monclient: monmap: 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:06.000 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 Processor -- start 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 1 -- start start 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.989+0000 7f5ab3291640 10 monclient: init 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 0x7f5aac133b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): picked mon.noname-c con 0x7f5a940040c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5aac11d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): picked mon.noname-a con 0x7f5a94004e70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aac0a0890 0x7f5aac11da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): picked mon.noname-b con 0x7f5aac0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:06.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0a0df0 con 0x7f5a94004e70 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0a0c70 con 0x7f5aac0a0890 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac0a0f70 con 0x7f5a940040c0 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient(hunting): authenticate will time out at 1669.388452s 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5aac11d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5aac11d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40354/0 (socket says 172.21.15.17:40354) 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 0x7f5aac133b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 -- 172.21.15.17:0/2882136421 learned_addr learned my addr 172.21.15.17:0/2882136421 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:06.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): get_auth_request con 0x7f5a94004e70 auth_method 0 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 10 monclient(hunting): get_auth_request con 0x7f5a940040c0 auth_method 0 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aac0a0890 0x7f5aac11da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:06.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 1 -- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 msgr2=0x7f5aac133b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 1 -- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 msgr2=0x7f5aac133b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 0x7f5aac133b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): get_auth_request con 0x7f5aac0a0890 auth_method 0 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab102d640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 0x7f5aac133b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:06.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab182e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient(hunting): handle_auth_done global_id 144102 payload 824 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 -- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 msgr2=0x7f5aac133b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 0x7f5aac133b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 -- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aac0a0890 msgr2=0x7f5aac11da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aac0a0890 0x7f5aac11da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient: _finish_hunting 0 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 monclient: found mon.noname-a 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 -- 172.21.15.17:0/2882136421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac09d0f0 con 0x7f5a94004e70 2024-09-15T21:29:06.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab082c640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5aac11d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c011670 tx=0x7f5a9c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c019020 con 0x7f5a94004e70 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: dump: 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:06.006 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c003980 con 0x7f5a94004e70 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c003d50 con 0x7f5a94004e70 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 5 monclient: authenticate success, global_id 144102 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _renew_subs 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac11ef00 con 0x7f5a94004e70 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:06.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 10 monclient: dump: 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _renew_subs 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:06.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac134040 con 0x7f5a94004e70 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aaaffd640 4 set_mon_vals no callback set 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac135670 con 0x7f5a94004e70 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5a9c01c020 con 0x7f5a94004e70 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.993+0000 7f5aa9ffb640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a9803c1e0 0x7f5a9803e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.997+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5a9c0534b0 con 0x7f5a94004e70 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.997+0000 7f5ab102d640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a9803c1e0 0x7f5a9803e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.997+0000 7f5ab102d640 10 monclient: get_auth_request con 0x7f5a9803c1e0 auth_method 0 2024-09-15T21:29:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:05.997+0000 7f5ab102d640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a9803c1e0 0x7f5a9803e6a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00b1000 tx=0x7f5aa00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.001+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5a9c023030 con 0x7f5a94004e70 2024-09-15T21:29:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.001+0000 7f5aa9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.001+0000 7f5aa9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5ab3291640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5ab3291640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5ab3291640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5aac121080 con 0x7f5a94004e70 2024-09-15T21:29:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5aa9ffb640 1 -- 172.21.15.17:0/2882136421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f5aac121080 con 0x7f5a94004e70 2024-09-15T21:29:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5aa9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:06.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.237+0000 7f5aa9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:06.240 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":108,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a9803c1e0 msgr2=0x7f5a9803e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a9803c1e0 0x7f5a9803e6a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00b1000 tx=0x7f5aa00046d0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 10 monclient: shutdown 2024-09-15T21:29:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 msgr2=0x7f5aac11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5aac11d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c011670 tx=0x7f5a9c011b30 comp rx=0 tx=0).stop 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 shutdown_connections 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5a9803c1e0 0x7f5a9803e6a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aac0a0890 0x7f5aac11da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a94004e70 0x7f5aac11d530 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 --2- 172.21.15.17:0/2882136421 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5a940040c0 0x7f5aac133b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 >> 172.21.15.17:0/2882136421 conn(0x7f5aac0a66a0 msgr2=0x7f5aac0a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 shutdown_connections 2024-09-15T21:29:06.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:06.241+0000 7f5ab3291640 1 -- 172.21.15.17:0/2882136421 wait complete. 2024-09-15T21:29:09.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient: get_monmap_and_config 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient: build_initial_monmap 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient: monmap: 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:09.395368+0000 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:09.395368+0000 2024-09-15T21:29:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:09.396 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:09.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 Processor -- start 2024-09-15T21:29:09.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 -- start start 2024-09-15T21:29:09.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient: init 2024-09-15T21:29:09.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): picked mon.noname-b con 0x7fa3d01378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa3d01381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): picked mon.noname-a con 0x7fa3d00a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d00a1d00 con 0x7fa3d01381e0 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d00a1fe0 con 0x7fa3d00a5210 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d00a22c0 con 0x7fa3d01378c0 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:09.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3d4a21640 10 monclient(hunting): authenticate will time out at 1672.790888s 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50052/0 (socket says 172.21.15.17:50052) 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 learned_addr learned my addr 172.21.15.17:0/1720439757 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request con 0x7fa3d00a5210 auth_method 0 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request con 0x7fa3d01378c0 auth_method 0 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:09.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:09.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.393+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:09.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:09.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:09.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request con 0x7fa3d01378c0 auth_method 0 2024-09-15T21:29:09.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request con 0x7fa3d00a5210 auth_method 0 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:09.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:09.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:09.597+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request con 0x7fa3d01378c0 auth_method 0 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request con 0x7fa3d00a5210 auth_method 0 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:10.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.001+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request con 0x7fa3d01378c0 auth_method 0 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:10.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request con 0x7fa3d00a5210 auth_method 0 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:10.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:10.805+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:11.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.985+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194126228 0 0) 0x7fa3d00a1d00 con 0x7fa3d01381e0 2024-09-15T21:29:11.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.985+0000 7fa3cd59a640 10 monclient(hunting): _negotiate 2024-09-15T21:29:11.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.985+0000 7fa3cd59a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:11.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.985+0000 7fa3cd59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:11.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.985+0000 7fa3cd59a640 10 monclient(hunting): my global_id is 114373 2024-09-15T21:29:11.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.985+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3bc003530 con 0x7fa3d01381e0 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3481608857 0 0) 0x7fa3bc003530 con 0x7fa3d01381e0 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3d0137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3d0137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 msgr2=0x7fa3d00a56c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 --2- 172.21.15.17:0/1720439757 >> v2:172.21.15.17:3300/0 conn(0x7fa3d00a5210 0x7fa3d00a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: _finish_hunting 0 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:29:11.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00a5c00 con 0x7fa3d01381e0 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: _finish_auth 0 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: _check_auth_tickets 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa3b8001700 con 0x7fa3d01381e0 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: dump: 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 >> v1:172.21.15.119:6789/0 conn(0x7fa3d01381e0 legacy=0x7fa3d00a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc003a10 0x7fa3bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): picked mon.b con 0x7fa3bc003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): picked mon.c con 0x7fa3d01378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3bc004d50 0x7fa3bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): picked mon.a con 0x7fa3bc004d50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00a1fe0 con 0x7fa3bc004d50 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00a22c0 con 0x7fa3bc003a10 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc00bc10 con 0x7fa3d01378c0 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cd59a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3bc004d50 0x7fa3bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:11.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request con 0x7fa3bc004d50 auth_method 0 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc003a10 0x7fa3bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): get_auth_request con 0x7fa3d01378c0 auth_method 0 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request con 0x7fa3bc003a10 auth_method 0 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3cdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient(hunting): handle_auth_done global_id 114373 payload 293 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 msgr2=0x7fa3bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01378c0 0x7fa3bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc003a10 msgr2=0x7fa3bc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc003a10 0x7fa3bc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient: _finish_hunting 0 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 monclient: found mon.a 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00a5c00 con 0x7fa3bc004d50 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient: _finish_auth 0 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 10 monclient: _check_auth_tickets 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3ced9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3d4a21640 5 monclient: authenticate success, global_id 114373 2024-09-15T21:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.989+0000 7fa3d4a21640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:11.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.993+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3bc004d50 0x7fa3bc0051d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80096d0 tx=0x7fa3c801d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e73dbbc9718210 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:11.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.993+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80b3070 con 0x7fa3bc004d50 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: dump: 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c80ae410 con 0x7fa3bc004d50 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 4 set_mon_vals no callback set 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 1 -- 172.21.15.17:0/1720439757 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80097e0 con 0x7fa3bc004d50 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3cd59a640 10 monclient: dump: 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 10 monclient: shutdown 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 -- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3bc004d50 msgr2=0x7fa3bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/1720439757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3bc004d50 0x7fa3bc0051d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80096d0 tx=0x7fa3c801d330 comp rx=0 tx=0).stop 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3ced9d640 1 -- 172.21.15.17:0/1720439757 reap_dead start 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 -- 172.21.15.17:0/1720439757 shutdown_connections 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 -- 172.21.15.17:0/1720439757 >> 172.21.15.17:0/1720439757 conn(0x7fa3d00aac40 msgr2=0x7fa3d00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:12.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 -- 172.21.15.17:0/1720439757 shutdown_connections 2024-09-15T21:29:12.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 -- 172.21.15.17:0/1720439757 wait complete. 2024-09-15T21:29:12.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 10 monclient: build_initial_monmap 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 10 monclient: monmap: 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 Processor -- start 2024-09-15T21:29:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 1 -- start start 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:11.997+0000 7fa3d4a21640 10 monclient: init 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 0x7fa3d009f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): picked mon.noname-a con 0x7fa3d0121340 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d009f800 0x7fa3d009fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): picked mon.noname-b con 0x7fa3d009f800 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 0x7fa3d01d6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): picked mon.noname-c con 0x7fa3d01d4630 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d006f340 con 0x7fa3d0121340 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d006f1c0 con 0x7fa3d009f800 2024-09-15T21:29:12.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d01405b0 con 0x7fa3d01d4630 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient(hunting): authenticate will time out at 1675.396755s 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 0x7fa3d009f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 0x7fa3d009f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50110/0 (socket says 172.21.15.17:50110) 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 0x7fa3d01d6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 -- 172.21.15.17:0/797785321 learned_addr learned my addr 172.21.15.17:0/797785321 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request con 0x7fa3d0121340 auth_method 0 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request con 0x7fa3d01d4630 auth_method 0 2024-09-15T21:29:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d009f800 0x7fa3d009fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): get_auth_request con 0x7fa3d009f800 auth_method 0 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 1 -- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 msgr2=0x7fa3d01d6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 1 -- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 msgr2=0x7fa3d01d6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 0x7fa3d01d6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:12.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ced9d640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 0x7fa3d01d6a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3cdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient(hunting): handle_auth_done global_id 144105 payload 824 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 -- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 msgr2=0x7fa3d01d6a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 0x7fa3d01d6a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 -- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d009f800 msgr2=0x7fa3d009fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d009f800 0x7fa3d009fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient: _finish_hunting 0 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 monclient: found mon.noname-a 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 -- 172.21.15.17:0/797785321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d0136560 con 0x7fa3d0121340 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ce59c640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 0x7fa3d009f2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0017bc0 tx=0x7fa3c0015730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000f040 con 0x7fa3d0121340 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: dump: 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c001c070 con 0x7fa3d0121340 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0004240 con 0x7fa3d0121340 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 5 monclient: authenticate success, global_id 144105 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _renew_subs 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d0122580 con 0x7fa3d0121340 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3ccd99640 4 set_mon_vals no callback set 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 10 monclient: dump: 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _renew_subs 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:12.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d0121cd0 con 0x7fa3d0121340 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa3c0003620 con 0x7fa3d0121340 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa3b003b9c0 0x7fa3b003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa3c00512c0 con 0x7fa3d0121340 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.001+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d00a56a0 con 0x7fa3d0121340 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.005+0000 7fa3cdd9b640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa3b003b9c0 0x7fa3b003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.005+0000 7fa3cdd9b640 10 monclient: get_auth_request con 0x7fa3b003b9c0 auth_method 0 2024-09-15T21:29:12.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.005+0000 7fa3cdd9b640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa3b003b9c0 0x7fa3b003de80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8000fc0 tx=0x7fa3b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:12.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.009+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa3c002e020 con 0x7fa3d0121340 2024-09-15T21:29:12.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.009+0000 7fa3c6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:12.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.009+0000 7fa3c6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3d4a21640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3d4a21640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3d4a21640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa3d01207e0 con 0x7fa3d0121340 2024-09-15T21:29:12.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3c6ffd640 1 -- 172.21.15.17:0/797785321 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fa3d0136560 con 0x7fa3d0121340 2024-09-15T21:29:12.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3c6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:12.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.253+0000 7fa3c6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:12.256 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":112,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:12.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa3b003b9c0 msgr2=0x7fa3b003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:12.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa3b003b9c0 0x7fa3b003de80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8000fc0 tx=0x7fa3b8009040 comp rx=0 tx=0).stop 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 10 monclient: shutdown 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 msgr2=0x7fa3d009f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 0x7fa3d009f2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0017bc0 tx=0x7fa3c0015730 comp rx=0 tx=0).stop 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 shutdown_connections 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa3b003b9c0 0x7fa3b003de80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa3d01d4630 0x7fa3d01d6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d009f800 0x7fa3d009fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 --2- 172.21.15.17:0/797785321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3d0121340 0x7fa3d009f2c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 >> 172.21.15.17:0/797785321 conn(0x7fa3d00aac40 msgr2=0x7fa3d0136ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 shutdown_connections 2024-09-15T21:29:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:12.257+0000 7fa3d4a21640 1 -- 172.21.15.17:0/797785321 wait complete. 2024-09-15T21:29:15.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient: get_monmap_and_config 2024-09-15T21:29:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient: build_initial_monmap 2024-09-15T21:29:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient: monmap: 2024-09-15T21:29:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:15.411794+0000 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:15.411794+0000 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:15.413 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 1 Processor -- start 2024-09-15T21:29:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 1 -- start start 2024-09-15T21:29:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient: init 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8a1c1340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8a1c09d6c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a1c13a570 0x7f8a1c13c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient(hunting): picked mon.noname-c con 0x7f8a1c13a570 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a1c09e1b0 con 0x7f8a1c09d6c0 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a1c09e490 con 0x7f8a1c1340c0 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a1c09d520 con 0x7f8a1c13a570 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 10 monclient(hunting): authenticate will time out at 1678.807057s 2024-09-15T21:29:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8a1c09d6c0 0x7f8a1c138e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55386/0 (socket says 172.21.15.17:55386) 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1bfff640 1 -- 172.21.15.17:0/1452689540 learned_addr learned my addr 172.21.15.17:0/1452689540 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069131746 0 0) 0x7f8a1c09e1b0 con 0x7f8a1c09d6c0 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient(hunting): my global_id is 144108 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 1 --2- 172.21.15.17:0/1452689540 >> v2:172.21.15.119:3300/0 conn(0x7f8a1c13a570 0x7f8a1c13c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89fc005740 con 0x7f8a1c09d6c0 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): get_auth_request con 0x7f8a1c13a570 auth_method 0 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1402883732 0 0) 0x7f89fc005740 con 0x7f8a1c09d6c0 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a212ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 >> v2:172.21.15.119:3300/0 conn(0x7f8a1c13a570 msgr2=0x7f8a1c13c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 --2- 172.21.15.17:0/1452689540 >> v2:172.21.15.119:3300/0 conn(0x7f8a1c13a570 0x7f8a1c13c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 >> v1:172.21.15.119:6789/0 conn(0x7f8a1c1340c0 legacy=0x7f8a1c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c09e770 con 0x7f8a1c09d6c0 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 5 monclient: authenticate success, global_id 144108 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a22d52640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f8a04001a80 con 0x7f8a1c09d6c0 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: dump: 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 >> v1:172.21.15.17:6789/0 conn(0x7f8a1c09d6c0 legacy=0x7f8a1c138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f89fc006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 10 monclient(hunting): picked mon.a con 0x7f89fc005c70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.409+0000 7f8a1b7fe640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89fc006a20 0x7f89fc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): picked mon.c con 0x7f89fc006a20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89fc0075e0 0x7f89fc00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): picked mon.b con 0x7f89fc0075e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c09d520 con 0x7f89fc005c70 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc00e450 con 0x7f89fc0075e0 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f89fc00e650 con 0x7f89fc006a20 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f89fc006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): get_auth_request con 0x7f89fc005c70 auth_method 0 2024-09-15T21:29:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89fc006a20 0x7f89fc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): get_auth_request con 0x7f89fc006a20 auth_method 0 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89fc0075e0 0x7f89fc00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 10 monclient(hunting): get_auth_request con 0x7f89fc0075e0 auth_method 0 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient(hunting): handle_auth_done global_id 144108 payload 293 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 -- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89fc006a20 msgr2=0x7f89fc006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89fc006a20 0x7f89fc006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 -- 172.21.15.17:0/1452689540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89fc0075e0 msgr2=0x7f89fc00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89fc0075e0 0x7f89fc00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 monclient: found mon.a 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 -- 172.21.15.17:0/1452689540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc005740 con 0x7f89fc005c70 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89fc0075e0 0x7f89fc00dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a212ef640 1 -- 172.21.15.17:0/1452689540 reap_dead start 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f89fc006090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c001790 tx=0x7f8a0c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6578922207d65c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0013e0 con 0x7f89fc005c70 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: dump: 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c0052d0 con 0x7f89fc005c70 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 1 -- 172.21.15.17:0/1452689540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c003840 con 0x7f89fc005c70 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a1b7fe640 10 monclient: dump: 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 4 set_mon_vals no callback set 2024-09-15T21:29:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 10 monclient: shutdown 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 -- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 msgr2=0x7f89fc006090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f89fc006090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c001790 tx=0x7f8a0c01c370 comp rx=0 tx=0).stop 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 -- 172.21.15.17:0/1452689540 shutdown_connections 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1452689540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f89fc006090 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 -- 172.21.15.17:0/1452689540 >> 172.21.15.17:0/1452689540 conn(0x7f8a1c0a66a0 msgr2=0x7f8a1c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 -- 172.21.15.17:0/1452689540 shutdown_connections 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.413+0000 7f8a22d52640 1 -- 172.21.15.17:0/1452689540 wait complete. 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: build_initial_monmap 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: monmap: 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 Processor -- start 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- start start 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: init 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f8a1c1221e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): picked mon.noname-a con 0x7f89fc005c70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a1c1208f0 0x7f8a1c120d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): picked mon.noname-c con 0x7f8a1c1208f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a1c121290 0x7f8a1c1d67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): picked mon.noname-b con 0x7f8a1c121290 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c0a0890 con 0x7f89fc005c70 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c0a2770 con 0x7f8a1c121290 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c0a0a10 con 0x7f8a1c1208f0 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient(hunting): authenticate will time out at 1678.812972s 2024-09-15T21:29:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f8a1c1221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f8a1c1221e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50152/0 (socket says 172.21.15.17:50152) 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 -- 172.21.15.17:0/1404966796 learned_addr learned my addr 172.21.15.17:0/1404966796 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): get_auth_request con 0x7f89fc005c70 auth_method 0 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a1c1208f0 0x7f8a1c120d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): get_auth_request con 0x7f8a1c1208f0 auth_method 0 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a212ef640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a1c121290 0x7f8a1c1d67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a212ef640 10 monclient(hunting): get_auth_request con 0x7f8a1c121290 auth_method 0 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a212ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a212ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a212ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient(hunting): handle_auth_done global_id 144111 payload 824 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 -- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a1c1208f0 msgr2=0x7f8a1c120d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a1c1208f0 0x7f8a1c120d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 -- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a1c121290 msgr2=0x7f8a1c1d67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a1c121290 0x7f8a1c1d67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 monclient: found mon.noname-a 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 -- 172.21.15.17:0/1404966796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c1335e0 con 0x7f89fc005c70 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a20aee640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f8a1c1221e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c01aa70 tx=0x7f89fc007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0013c0 con 0x7f89fc005c70 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: dump: 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c005480 con 0x7f89fc005c70 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0b4430 con 0x7f89fc005c70 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 10 monclient: dump: 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a1affd640 4 set_mon_vals no callback set 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 5 monclient: authenticate success, global_id 144111 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _renew_subs 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1c0a4dc0 con 0x7f89fc005c70 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _renew_subs 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1c123420 con 0x7f89fc005c70 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1c131320 con 0x7f89fc005c70 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8a0c0b45d0 con 0x7f89fc005c70 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a0003b9c0 0x7f8a0003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.417+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f8a0c0eab90 con 0x7f89fc005c70 2024-09-15T21:29:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.421+0000 7f8a1bfff640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a0003b9c0 0x7f8a0003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.421+0000 7f8a1bfff640 10 monclient: get_auth_request con 0x7f8a0003b9c0 auth_method 0 2024-09-15T21:29:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.421+0000 7f8a1bfff640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a0003b9c0 0x7f8a0003de80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c073dd0 tx=0x7f8a04006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.425+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8a1c0a4dc0 con 0x7f89fc005c70 2024-09-15T21:29:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.425+0000 7f8a197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.425+0000 7f8a197fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:15.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:15.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:15.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8a1c13a120 con 0x7f89fc005c70 2024-09-15T21:29:15.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a197fa640 1 -- 172.21.15.17:0/1404966796 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f8a0c0b8030 con 0x7f89fc005c70 2024-09-15T21:29:15.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:15.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a197fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:15.663 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:15.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a0003b9c0 msgr2=0x7f8a0003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a0003b9c0 0x7f8a0003de80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c073dd0 tx=0x7f8a04006040 comp rx=0 tx=0).stop 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 10 monclient: shutdown 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 msgr2=0x7f8a1c1221e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f8a1c1221e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c01aa70 tx=0x7f89fc007a50 comp rx=0 tx=0).stop 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 shutdown_connections 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8a0003b9c0 0x7f8a0003de80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a1c121290 0x7f8a1c1d67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8a1c1208f0 0x7f8a1c120d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 --2- 172.21.15.17:0/1404966796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89fc005c70 0x7f8a1c1221e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 >> 172.21.15.17:0/1404966796 conn(0x7f8a1c0a66a0 msgr2=0x7f8a1c132ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 shutdown_connections 2024-09-15T21:29:15.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.661+0000 7f8a22d52640 1 -- 172.21.15.17:0/1404966796 wait complete. 2024-09-15T21:29:15.681 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:29:15.681 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:29:15.682 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:29:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient: get_monmap_and_config 2024-09-15T21:29:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient: build_initial_monmap 2024-09-15T21:29:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient: monmap: 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:15.791040+0000 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:15.791040+0000 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:15.791 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 Processor -- start 2024-09-15T21:29:15.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 -- start start 2024-09-15T21:29:15.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient: init 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 0x7f064c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): picked mon.noname-b con 0x7f064c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f064c09ffc0 0x7f064c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): picked mon.noname-a con 0x7f064c09ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f064c135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c0a3130 con 0x7f064c135670 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f064c0a16f0 con 0x7f064c09ffc0 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f064c073df0 con 0x7f064c0a0890 2024-09-15T21:29:15.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f06530ee640 10 monclient(hunting): authenticate will time out at 1679.186402s 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 0x7f064c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f064c09ffc0 0x7f064c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f064c09ffc0 0x7f064c0a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50162/0 (socket says 172.21.15.17:50162) 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 -- 172.21.15.17:0/2493998462 learned_addr learned my addr 172.21.15.17:0/2493998462 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): get_auth_request con 0x7f064c09ffc0 auth_method 0 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 0x7f064c0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48002/0 (socket says 172.21.15.17:48002) 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): get_auth_request con 0x7f064c0a0890 auth_method 0 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient(hunting): handle_auth_done global_id 144114 payload 274 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 -- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 msgr2=0x7f064c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 --2- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 0x7f064c0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 -- 172.21.15.17:0/2493998462 >> v1:172.21.15.119:6789/0 conn(0x7f064c135670 legacy=0x7f064c135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 monclient: found mon.noname-a 2024-09-15T21:29:15.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f064bfff640 1 -- 172.21.15.17:0/2493998462 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c0a1bf0 con 0x7f064c09ffc0 2024-09-15T21:29:15.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.789+0000 7f0650e8a640 1 --2- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 0x7f064c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:15.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064bfff640 1 --2- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c0a11b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f064c0a1870 tx=0x7f063401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd1408ce685841b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 1 -- 172.21.15.17:0/2493998462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340b1070 con 0x7f064c09ffc0 2024-09-15T21:29:15.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: dump: 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 1 -- 172.21.15.17:0/2493998462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06340ac640 con 0x7f064c09ffc0 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 5 monclient: authenticate success, global_id 144114 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 4 set_mon_vals no callback set 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 1 -- 172.21.15.17:0/2493998462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340aca10 con 0x7f064c09ffc0 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064b7fe640 10 monclient: dump: 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.805 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: shutdown 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 msgr2=0x7f064c0a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 --2- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c0a11b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f064c0a1870 tx=0x7f063401c400 comp rx=0 tx=0).stop 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- 172.21.15.17:0/2493998462 shutdown_connections 2024-09-15T21:29:15.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 --2- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c0a11b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 --2- 172.21.15.17:0/2493998462 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c0a0890 0x7f064c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- 172.21.15.17:0/2493998462 >> 172.21.15.17:0/2493998462 conn(0x7f064c0a66a0 msgr2=0x7f064c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- 172.21.15.17:0/2493998462 shutdown_connections 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- 172.21.15.17:0/2493998462 wait complete. 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: build_initial_monmap 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: monmap: 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:15.807 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 Processor -- start 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- start start 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: init 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): picked mon.noname-a con 0x7f064c09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f064c0a0890 0x7f064c134630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): picked mon.noname-b con 0x7f064c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c135670 0x7f064c11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): picked mon.noname-c con 0x7f064c135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f064c140690 con 0x7f064c09ffc0 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f064c140510 con 0x7f064c0a0890 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f064c140810 con 0x7f064c135670 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f06530ee640 10 monclient(hunting): authenticate will time out at 1679.191378s 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c13c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50178/0 (socket says 172.21.15.17:50178) 2024-09-15T21:29:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 -- 172.21.15.17:0/2686926336 learned_addr learned my addr 172.21.15.17:0/2686926336 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): get_auth_request con 0x7f064c09ffc0 auth_method 0 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c135670 0x7f064c11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): get_auth_request con 0x7f064c135670 auth_method 0 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f065168b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient(hunting): handle_auth_done global_id 144117 payload 824 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064bfff640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f064c0a0890 0x7f064c134630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c135670 msgr2=0x7f064c11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f064c135670 0x7f064c11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f064c0a0890 msgr2=0x7f064c134630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f064c0a0890 0x7f064c134630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 monclient: found mon.noname-a 2024-09-15T21:29:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f0650e8a640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c09d0f0 con 0x7f064c09ffc0 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064bfff640 10 monclient: get_auth_request con 0x7f064c0a0890 auth_method 0 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.793+0000 7f064bfff640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f064c0a0890 0x7f064c134630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f0650e8a640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c13c7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0644009f60 tx=0x7f0644009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644018020 con 0x7f064c09ffc0 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: dump: 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0644014680 con 0x7f064c09ffc0 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 5 monclient: authenticate success, global_id 144117 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 10 monclient: _renew_subs 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c1ce5f0 con 0x7f064c09ffc0 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014a50 con 0x7f064c09ffc0 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: dump: 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 10 monclient: _renew_subs 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064affd640 4 set_mon_vals no callback set 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c0727d0 con 0x7f064c09ffc0 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 0x7f064c11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f064c11e980 con 0x7f064c134b70 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 0x7f064c11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient: get_auth_request con 0x7f064c134b70 auth_method 0 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 10 monclient(hunting): handle_auth_done global_id 144120 payload 824 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f064c0a5730 con 0x7f064c134b70 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f064401b020 con 0x7f064c09ffc0 2024-09-15T21:29:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f064bfff640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 0x7f064c11ca20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f064c06f1c0 tx=0x7f063401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f062c03b9f0 0x7f062c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0644051490 con 0x7f064c09ffc0 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340bb030 con 0x7f064c134b70 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: dump: 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f065168b640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f062c03b9f0 0x7f062c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0634009490 con 0x7f064c134b70 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 msgr2=0x7f064c11ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 0x7f064c11ca20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f064c06f1c0 tx=0x7f063401ac40 comp rx=0 tx=0).stop 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 msgr2=0x7f064c11ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f06497fa640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c134b70 0x7f064c11ca20 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f064c06f1c0 tx=0x7f063401ac40 comp rx=0 tx=0).stop 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f065168b640 10 monclient: get_auth_request con 0x7f062c03b9f0 auth_method 0 2024-09-15T21:29:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.797+0000 7f065168b640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f062c03b9f0 0x7f062c03deb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f063c007fd0 tx=0x7f063c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.801+0000 7f06530ee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.801+0000 7f06530ee640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 0x7f064c137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.801+0000 7f06530ee640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.801+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f064c11d340 con 0x7f064c11cf60 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 0x7f064c137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient: get_auth_request con 0x7f064c11cf60 auth_method 0 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 10 monclient(hunting): handle_auth_done global_id 144123 payload 824 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 1 -- 172.21.15.17:0/2686926336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f064c135aa0 con 0x7f064c11cf60 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f064bfff640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 0x7f064c137ab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f063401a030 tx=0x7f06340191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340c7070 con 0x7f064c11cf60 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 10 monclient: dump: 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f06340c1a10 con 0x7f064c11cf60 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 msgr2=0x7f064c137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 0x7f064c137ab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f063401a030 tx=0x7f06340191a0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 msgr2=0x7f064c137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06497fa640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c11cf60 0x7f064c137ab0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f063401a030 tx=0x7f06340191a0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 27531, 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.825 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:15.826 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:15.827 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:15.828 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:15.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:15.830 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:15.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:15.833 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.835 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:15.836 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:15.837 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f062c03b9f0 msgr2=0x7f062c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f062c03b9f0 0x7f062c03deb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f063c007fd0 tx=0x7f063c009040 comp rx=0 tx=0).stop 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 10 monclient: shutdown 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 msgr2=0x7f064c13c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 --2- 172.21.15.17:0/2686926336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f064c09ffc0 0x7f064c13c7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0644009f60 tx=0x7f0644009f90 comp rx=0 tx=0).stop 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f065168b640 1 -- 172.21.15.17:0/2686926336 reap_dead start 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f065168b640 1 -- 172.21.15.17:0/2686926336 reap_dead start 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 shutdown_connections 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 >> 172.21.15.17:0/2686926336 conn(0x7f064c0a66a0 msgr2=0x7f064c0a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 shutdown_connections 2024-09-15T21:29:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.805+0000 7f06530ee640 1 -- 172.21.15.17:0/2686926336 wait complete. 2024-09-15T21:29:15.840 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:29:15.840 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:29:15.840 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:29:15.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.953+0000 7f5665214640 10 monclient: get_monmap_and_config 2024-09-15T21:29:15.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.953+0000 7f5665214640 10 monclient: build_initial_monmap 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.953+0000 7f5665214640 10 monclient: monmap: 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:15.958061+0000 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:15.958061+0000 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:15.958 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 Processor -- start 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 -- start start 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient: init 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f56600a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566009ffc0 0x7f566013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): picked mon.noname-b con 0x7f566009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56600a1380 0x7f566013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): picked mon.noname-c con 0x7f56600a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56600a3130 con 0x7f56600a0890 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5660073df0 con 0x7f56600a1380 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f56600a1bf0 con 0x7f566009ffc0 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f5665214640 10 monclient(hunting): authenticate will time out at 1679.353414s 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565ed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f56600a0890 0x7f56600a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55388/0 (socket says 172.21.15.17:55388) 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565ed9d640 1 -- 172.21.15.17:0/406037569 learned_addr learned my addr 172.21.15.17:0/406037569 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 1 --2- 172.21.15.17:0/406037569 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566009ffc0 0x7f566013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): get_auth_request con 0x7f566009ffc0 auth_method 0 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565f59e640 1 --2- 172.21.15.17:0/406037569 >> v2:172.21.15.119:3300/0 conn(0x7f56600a1380 0x7f566013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565dd9b640 1 -- 172.21.15.17:0/406037569 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2660149232 0 0) 0x7f56600a3130 con 0x7f56600a0890 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565dd9b640 10 monclient(hunting): _negotiate 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565dd9b640 10 monclient(hunting): my global_id is 144126 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565dd9b640 1 -- 172.21.15.17:0/406037569 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f564c003530 con 0x7f56600a0890 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565f59e640 10 monclient(hunting): get_auth_request con 0x7f56600a1380 auth_method 0 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient(hunting): handle_auth_done global_id 134213 payload 274 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 1 -- 172.21.15.17:0/406037569 >> v2:172.21.15.119:3300/0 conn(0x7f56600a1380 msgr2=0x7f566013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 1 --2- 172.21.15.17:0/406037569 >> v2:172.21.15.119:3300/0 conn(0x7f56600a1380 0x7f566013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 1 -- 172.21.15.17:0/406037569 >> v1:172.21.15.17:6789/0 conn(0x7f56600a0890 legacy=0x7f56600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 1 monclient: found mon.noname-b 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:29:15.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.957+0000 7f565e59c640 1 -- 172.21.15.17:0/406037569 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566009ccd0 con 0x7f566009ffc0 2024-09-15T21:29:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 --2- 172.21.15.17:0/406037569 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566009ffc0 0x7f566013aac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d368cdd73da43023 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 1 -- 172.21.15.17:0/406037569 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b0070 con 0x7f566009ffc0 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: dump: 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 1 -- 172.21.15.17:0/406037569 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500ab600 con 0x7f566009ffc0 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 5 monclient: authenticate success, global_id 134213 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 1 -- 172.21.15.17:0/406037569 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500ab9d0 con 0x7f566009ffc0 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 4 set_mon_vals no callback set 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565dd9b640 10 monclient: dump: 2024-09-15T21:29:15.972 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: shutdown 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/406037569 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566009ffc0 msgr2=0x7f566013aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 --2- 172.21.15.17:0/406037569 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566009ffc0 0x7f566013aac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001c360 comp rx=0 tx=0).stop 2024-09-15T21:29:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/406037569 shutdown_connections 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 --2- 172.21.15.17:0/406037569 >> v2:172.21.15.119:3300/0 conn(0x7f56600a1380 0x7f566013d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 --2- 172.21.15.17:0/406037569 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566009ffc0 0x7f566013aac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/406037569 >> 172.21.15.17:0/406037569 conn(0x7f56600a66a0 msgr2=0x7f56600a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/406037569 shutdown_connections 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/406037569 wait complete. 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: build_initial_monmap 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: monmap: 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 Processor -- start 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- start start 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: init 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f566009ffc0 0x7f566011e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): picked mon.noname-a con 0x7f566009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56600a0890 0x7f56601cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): picked mon.noname-c con 0x7f56600a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:15.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600a1380 0x7f566011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): picked mon.noname-b con 0x7f56600a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5660140100 con 0x7f566009ffc0 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f566013ff80 con 0x7f56600a1380 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5660140280 con 0x7f56600a0890 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient(hunting): authenticate will time out at 1679.358571s 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56600a0890 0x7f56601cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f566009ffc0 0x7f566011e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56600a0890 0x7f56601cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48036/0 (socket says 172.21.15.17:48036) 2024-09-15T21:29:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f566009ffc0 0x7f566011e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50216/0 (socket says 172.21.15.17:50216) 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 -- 172.21.15.17:0/1746877453 learned_addr learned my addr 172.21.15.17:0/1746877453 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): get_auth_request con 0x7f56600a0890 auth_method 0 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): get_auth_request con 0x7f566009ffc0 auth_method 0 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient(hunting): handle_auth_done global_id 134216 payload 824 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565f59e640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600a1380 0x7f566011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600a1380 msgr2=0x7f566011c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600a1380 0x7f566011c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f566009ffc0 msgr2=0x7f566011e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f566009ffc0 0x7f566011e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 monclient: found mon.noname-c 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566011fcb0 con 0x7f56600a0890 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565ed9d640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f566009ffc0 0x7f566011e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565f59e640 10 monclient: get_auth_request con 0x7f56600a1380 auth_method 0 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565f59e640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600a1380 0x7f566011c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565e59c640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56600a0890 0x7f56601cd800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56500ae040 tx=0x7f565001aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500bc070 con 0x7f56600a0890 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: dump: 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.979 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500b1040 con 0x7f56600a0890 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b0050 con 0x7f56600a0890 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:15.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f56577fe640 10 monclient: dump: 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 5 monclient: authenticate success, global_id 134216 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: _renew_subs 2024-09-15T21:29:15.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566011ff90 con 0x7f56600a0890 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f565d59a640 4 set_mon_vals no callback set 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: _renew_subs 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.961+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f566011f120 con 0x7f56600a0890 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f56500ab4c0 con 0x7f56600a0890 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f5665214640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f5665214640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 0x7f566011ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f566006f5c0 con 0x7f56601cdd40 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f564003b9f0 0x7f564003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 0x7f566011ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f56500ea480 con 0x7f56600a0890 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient: get_auth_request con 0x7f56601cdd40 auth_method 0 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 10 monclient(hunting): handle_auth_done global_id 134219 payload 824 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56600a0c70 con 0x7f56601cdd40 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565f59e640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f564003b9f0 0x7f564003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565f59e640 10 monclient: get_auth_request con 0x7f564003b9f0 auth_method 0 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565ed9d640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 0x7f566011ca20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5648005890 tx=0x7f5648007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648005b90 con 0x7f56601cdd40 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 10 monclient: dump: 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5648018070 con 0x7f56601cdd40 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 msgr2=0x7f566011ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 0x7f566011ca20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5648005890 tx=0x7f5648007700 comp rx=0 tx=0).stop 2024-09-15T21:29:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 msgr2=0x7f566011ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f56577fe640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56601cdd40 0x7f566011ca20 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5648005890 tx=0x7f5648007700 comp rx=0 tx=0).stop 2024-09-15T21:29:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.965+0000 7f565f59e640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f564003b9f0 0x7f564003deb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f566011ded0 tx=0x7f565800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.969+0000 7f5665214640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.969+0000 7f5665214640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 0x7f5660137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f566011d340 con 0x7f566011cf60 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 0x7f5660137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient: get_auth_request con 0x7f566011cf60 auth_method 0 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 10 monclient(hunting): handle_auth_done global_id 134222 payload 824 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 1 -- 172.21.15.17:0/1746877453 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f566006d020 con 0x7f566011cf60 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565ed9d640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 0x7f5660137ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5648005890 tx=0x7f5648005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56480063f0 con 0x7f566011cf60 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 10 monclient: dump: 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f56480063f0 con 0x7f566011cf60 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 msgr2=0x7f5660137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 0x7f5660137ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5648005890 tx=0x7f5648005800 comp rx=0 tx=0).stop 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 msgr2=0x7f5660137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f56577fe640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f566011cf60 0x7f5660137ab0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5648005890 tx=0x7f5648005800 comp rx=0 tx=0).stop 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 73127, 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:29:15.991 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:15.992 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:15.993 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:15.994 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:15.995 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:15.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:15.999 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:16.000 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.001 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.002 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.003 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:16.004 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f564003b9f0 msgr2=0x7f564003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f564003b9f0 0x7f564003deb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f566011ded0 tx=0x7f565800b040 comp rx=0 tx=0).stop 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 10 monclient: shutdown 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56600a0890 msgr2=0x7f56601cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 --2- 172.21.15.17:0/1746877453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f56600a0890 0x7f56601cd800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56500ae040 tx=0x7f565001aba0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565f59e640 1 -- 172.21.15.17:0/1746877453 reap_dead start 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f565f59e640 1 -- 172.21.15.17:0/1746877453 reap_dead start 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 shutdown_connections 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 >> 172.21.15.17:0/1746877453 conn(0x7f56600a66a0 msgr2=0x7f566006fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 shutdown_connections 2024-09-15T21:29:16.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:15.973+0000 7f5665214640 1 -- 172.21.15.17:0/1746877453 wait complete. 2024-09-15T21:29:16.007 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:29:16.007 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:29:16.007 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:29:16.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient: get_monmap_and_config 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient: build_initial_monmap 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient: monmap: 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:16.119186+0000 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:16.119186+0000 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:16.119 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:16.120 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:16.120 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:16.120 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 Processor -- start 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 -- start start 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient: init 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faffc1378c0 0x7faffc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): picked mon.noname-a con 0x7faffc1378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1381e0 0x7faffc0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): picked mon.noname-b con 0x7faffc1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faffc0a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faffc0a1d00 con 0x7faffc0a5040 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faffc0a1fe0 con 0x7faffc1378c0 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faffc0a22c0 con 0x7faffc1381e0 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb0035f5640 10 monclient(hunting): authenticate will time out at 1679.514616s 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faffc1378c0 0x7faffc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faffc1378c0 0x7faffc137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50228/0 (socket says 172.21.15.17:50228) 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 -- 172.21.15.17:0/1649448329 learned_addr learned my addr 172.21.15.17:0/1649448329 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): get_auth_request con 0x7faffc1378c0 auth_method 0 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 1 --2- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1381e0 0x7faffc0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): get_auth_request con 0x7faffc1381e0 auth_method 0 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb000b90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient(hunting): handle_auth_done global_id 144132 payload 274 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 -- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1381e0 msgr2=0x7faffc0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 --2- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1381e0 0x7faffc0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 -- 172.21.15.17:0/1649448329 >> v1:172.21.15.119:6789/0 conn(0x7faffc0a5040 legacy=0x7faffc0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient: _finish_hunting 0 2024-09-15T21:29:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 monclient: found mon.noname-a 2024-09-15T21:29:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.117+0000 7fb001391640 1 -- 172.21.15.17:0/1649448329 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc0a5c00 con 0x7faffc1378c0 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001391640 1 --2- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1378c0 0x7faffc137ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b381e9955e17cb09 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 1 -- 172.21.15.17:0/1649448329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80b0070 con 0x7faffc1378c0 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: dump: 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: _finish_auth 0 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 1 -- 172.21.15.17:0/1649448329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff80ab600 con 0x7faffc1378c0 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 5 monclient: authenticate success, global_id 144132 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:16.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 1 -- 172.21.15.17:0/1649448329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80ab9d0 con 0x7faffc1378c0 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 4 set_mon_vals no callback set 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7faff3fff640 10 monclient: dump: 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.133 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: shutdown 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1378c0 msgr2=0x7faffc137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1378c0 0x7faffc137ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff801c420 comp rx=0 tx=0).stop 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- 172.21.15.17:0/1649448329 shutdown_connections 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1381e0 0x7faffc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1649448329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1378c0 0x7faffc137ca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- 172.21.15.17:0/1649448329 >> 172.21.15.17:0/1649448329 conn(0x7faffc0aac40 msgr2=0x7faffc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- 172.21.15.17:0/1649448329 shutdown_connections 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- 172.21.15.17:0/1649448329 wait complete. 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: build_initial_monmap 2024-09-15T21:29:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: monmap: 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 Processor -- start 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- start start 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: init 2024-09-15T21:29:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc0a5040 0x7faffc1cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): picked mon.noname-b con 0x7faffc0a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 0x7faffc136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): picked mon.noname-c con 0x7faffc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1381e0 0x7faffc13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): picked mon.noname-a con 0x7faffc1381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1425c0 con 0x7faffc1381e0 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faffc138700 con 0x7faffc0a5040 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faffc142740 con 0x7faffc1378c0 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb0035f5640 10 monclient(hunting): authenticate will time out at 1679.519524s 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 0x7faffc136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 0x7faffc136a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48068/0 (socket says 172.21.15.17:48068) 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1381e0 0x7faffc13cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 -- 172.21.15.17:0/1072946522 learned_addr learned my addr 172.21.15.17:0/1072946522 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): get_auth_request con 0x7faffc1378c0 auth_method 0 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): get_auth_request con 0x7faffc1381e0 auth_method 0 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001391640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc0a5040 0x7faffc1cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient(hunting): handle_auth_done global_id 134228 payload 824 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc0a5040 msgr2=0x7faffc1cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc0a5040 0x7faffc1cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1381e0 msgr2=0x7faffc13cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1381e0 0x7faffc13cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient: _finish_hunting 0 2024-09-15T21:29:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 monclient: found mon.noname-c 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb000b90640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc1d0890 con 0x7faffc1378c0 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001391640 10 monclient: get_auth_request con 0x7faffc0a5040 auth_method 0 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001b92640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc1381e0 0x7faffc13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.121+0000 7fb001391640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc0a5040 0x7faffc1cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb000b90640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 0x7faffc136a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9b0 tx=0x7fafec00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec017070 con 0x7faffc1378c0 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: dump: 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec00c9a0 con 0x7faffc1378c0 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec00cd70 con 0x7faffc1378c0 2024-09-15T21:29:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: dump: 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff37fe640 4 set_mon_vals no callback set 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 5 monclient: authenticate success, global_id 134228 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 10 monclient: _renew_subs 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc0a8a00 con 0x7faffc1378c0 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 10 monclient: _renew_subs 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc09ff80 con 0x7faffc1378c0 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 0x7faffc13d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1376a0 con 0x7faffc121010 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fafec01a070 con 0x7faffc1378c0 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 0x7faffc13d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafe003b9c0 0x7fafe003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient: get_auth_request con 0x7faffc121010 auth_method 0 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fafec0506a0 con 0x7faffc1378c0 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001b92640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafe003b9c0 0x7fafe003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001b92640 10 monclient: get_auth_request con 0x7fafe003b9c0 auth_method 0 2024-09-15T21:29:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 10 monclient(hunting): handle_auth_done global_id 114391 payload 824 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faffc137ca0 con 0x7faffc121010 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001391640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 0x7faffc13d140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faffc0a6150 tx=0x7faff801ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7fb001b92640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafe003b9c0 0x7fafe003de80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faff400f220 tx=0x7faff4003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80ba030 con 0x7faffc121010 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: dump: 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.144 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faff8009490 con 0x7faffc121010 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 msgr2=0x7faffc13d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 0x7faffc13d140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faffc0a6150 tx=0x7faff801ac60 comp rx=0 tx=0).stop 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 msgr2=0x7faffc13d140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.125+0000 7faff1ffb640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc121010 0x7faffc13d140 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faffc0a6150 tx=0x7faff801ac60 comp rx=0 tx=0).stop 2024-09-15T21:29:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb0035f5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 0x7faffc13d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb0035f5640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faffc069270 con 0x7faffc136f40 2024-09-15T21:29:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 0x7faffc13d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient: get_auth_request con 0x7faffc136f40 auth_method 0 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 10 monclient(hunting): handle_auth_done global_id 114394 payload 824 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 1 -- 172.21.15.17:0/1072946522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faffc120af0 con 0x7faffc136f40 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7fb001391640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 0x7faffc13d680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faff801c250 tx=0x7faff80191c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80ba050 con 0x7faffc136f40 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 10 monclient: dump: 2024-09-15T21:29:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7faff80044e0 con 0x7faffc136f40 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 msgr2=0x7faffc13d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 0x7faffc13d680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faff801c250 tx=0x7faff80191c0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 msgr2=0x7faffc13d680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.133+0000 7faff1ffb640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc136f40 0x7faffc13d680 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faff801c250 tx=0x7faff80191c0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 164702, 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:16.150 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:16.151 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:16.152 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:16.153 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:16.154 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.155 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:16.157 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.158 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:16.159 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:16.160 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:16.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:29:16.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:16.163 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafe003b9c0 msgr2=0x7fafe003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafe003b9c0 0x7fafe003de80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faff400f220 tx=0x7faff4003d50 comp rx=0 tx=0).stop 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 10 monclient: shutdown 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 msgr2=0x7faffc136a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 0x7faffc136a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9b0 tx=0x7fafec00ee70 comp rx=0 tx=0).stop 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb001b92640 1 -- 172.21.15.17:0/1072946522 reap_dead start 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 shutdown_connections 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 --2- 172.21.15.17:0/1072946522 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faffc1378c0 0x7faffc136a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 >> 172.21.15.17:0/1072946522 conn(0x7faffc0aac40 msgr2=0x7faffc0a04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 shutdown_connections 2024-09-15T21:29:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.137+0000 7fb0035f5640 1 -- 172.21.15.17:0/1072946522 wait complete. 2024-09-15T21:29:16.166 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:29:16.166 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:29:16.166 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.269+0000 7f0e8809e640 10 monclient: get_monmap_and_config 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.269+0000 7f0e8809e640 10 monclient: build_initial_monmap 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.269+0000 7f0e8809e640 10 monclient: monmap: 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:16.273391+0000 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:16.273391+0000 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 Processor -- start 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 -- start start 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient: init 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0e801378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0e801383b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e8013e4a0 0x7f0e80140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient(hunting): picked mon.noname-a con 0x7f0e8013e4a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e800a1a20 con 0x7f0e801383b0 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e800a1d00 con 0x7f0e801378c0 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0e800a1fe0 con 0x7f0e8013e4a0 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient(hunting): authenticate will time out at 1679.668827s 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e85639640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0e801383b0 0x7f0e8013cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55392/0 (socket says 172.21.15.17:55392) 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e85639640 1 -- 172.21.15.17:0/3992705736 learned_addr learned my addr 172.21.15.17:0/3992705736 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:16.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 1 --2- 172.21.15.17:0/3992705736 >> v2:172.21.15.17:3300/0 conn(0x7f0e8013e4a0 0x7f0e80140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): get_auth_request con 0x7f0e8013e4a0 auth_method 0 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient(hunting): handle_auth_done global_id 144138 payload 274 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 1 -- 172.21.15.17:0/3992705736 >> v1:172.21.15.119:6789/0 conn(0x7f0e801378c0 legacy=0x7f0e80137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 1 -- 172.21.15.17:0/3992705736 >> v1:172.21.15.17:6789/0 conn(0x7f0e801383b0 legacy=0x7f0e8013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient: _finish_hunting 0 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 1 monclient: found mon.noname-a 2024-09-15T21:29:16.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:16.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 1 -- 172.21.15.17:0/3992705736 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800a22c0 con 0x7f0e8013e4a0 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8663b640 1 --2- 172.21.15.17:0/3992705736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e8013e4a0 0x7f0e80140890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009dc0 tx=0x7f0e7801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e1214b85210356 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 1 -- 172.21.15.17:0/3992705736 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b1070 con 0x7f0e8013e4a0 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: dump: 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: _finish_auth 0 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: _check_auth_tickets 2024-09-15T21:29:16.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 5 monclient: authenticate success, global_id 144138 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 1 -- 172.21.15.17:0/3992705736 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780a4db0 con 0x7f0e8013e4a0 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 4 set_mon_vals no callback set 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 1 -- 172.21.15.17:0/3992705736 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780ac660 con 0x7f0e8013e4a0 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e84e38640 10 monclient: dump: 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 10 monclient: shutdown 2024-09-15T21:29:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 -- 172.21.15.17:0/3992705736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e8013e4a0 msgr2=0x7f0e80140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.273+0000 7f0e8809e640 1 --2- 172.21.15.17:0/3992705736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e8013e4a0 0x7f0e80140890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009dc0 tx=0x7f0e7801c4f0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/3992705736 shutdown_connections 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 --2- 172.21.15.17:0/3992705736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e8013e4a0 0x7f0e80140890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/3992705736 >> 172.21.15.17:0/3992705736 conn(0x7f0e800aac40 msgr2=0x7f0e800a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/3992705736 shutdown_connections 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/3992705736 wait complete. 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: build_initial_monmap 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: monmap: 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 Processor -- start 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- start start 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: init 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e801378c0 0x7f0e801367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): picked mon.noname-b con 0x7f0e801378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 0x7f0e80136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): picked mon.noname-a con 0x7f0e801383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e8013e4a0 0x7f0e8009cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): picked mon.noname-c con 0x7f0e8013e4a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80144280 con 0x7f0e801383b0 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80144100 con 0x7f0e801378c0 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0e80144400 con 0x7f0e8013e4a0 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient(hunting): authenticate will time out at 1679.673579s 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 0x7f0e80136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 0x7f0e80136d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50254/0 (socket says 172.21.15.17:50254) 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 -- 172.21.15.17:0/4274678133 learned_addr learned my addr 172.21.15.17:0/4274678133 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): get_auth_request con 0x7f0e801383b0 auth_method 0 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e8013e4a0 0x7f0e8009cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): get_auth_request con 0x7f0e8013e4a0 auth_method 0 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8663b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient(hunting): handle_auth_done global_id 144144 payload 824 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85e3a640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e801378c0 0x7f0e801367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 -- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e8013e4a0 msgr2=0x7f0e8009cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e8013e4a0 0x7f0e8009cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 -- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e801378c0 msgr2=0x7f0e801367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e801378c0 0x7f0e801367d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient: _finish_hunting 0 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 monclient: found mon.noname-a 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 -- 172.21.15.17:0/4274678133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800a02c0 con 0x7f0e801383b0 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85e3a640 10 monclient: get_auth_request con 0x7f0e801378c0 auth_method 0 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85e3a640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e801378c0 0x7f0e801367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85639640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 0x7f0e80136d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000c4d0 tx=0x7f0e7000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70017070 con 0x7f0e801383b0 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: dump: 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: _finish_auth 0 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: _check_auth_tickets 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e70005cf0 con 0x7f0e801383b0 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70007270 con 0x7f0e801383b0 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 5 monclient: authenticate success, global_id 144144 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _renew_subs 2024-09-15T21:29:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8009f080 con 0x7f0e801383b0 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e77fff640 4 set_mon_vals no callback set 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 10 monclient: dump: 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _renew_subs 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8013ddb0 con 0x7f0e801383b0 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e80137ca0 con 0x7f0e801383b0 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0e7001a070 con 0x7f0e801383b0 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0e6403b9f0 0x7f0e6403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:16.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0e7004f8d0 con 0x7f0e801383b0 2024-09-15T21:29:16.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85e3a640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0e6403b9f0 0x7f0e6403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:16.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.277+0000 7f0e85e3a640 10 monclient: get_auth_request con 0x7f0e6403b9f0 auth_method 0 2024-09-15T21:29:16.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.281+0000 7f0e85e3a640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0e6403b9f0 0x7f0e6403deb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c000f70 tx=0x7f0e7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:16.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.281+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0e800a1a20 con 0x7f0e801383b0 2024-09-15T21:29:16.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.281+0000 7f0e767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:16.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.281+0000 7f0e767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:16.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.509+0000 7f0e8809e640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:29:16.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.509+0000 7f0e8809e640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:29:16.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.509+0000 7f0e8809e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:16.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.509+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0e80069270 con 0x7f0e801383b0 2024-09-15T21:29:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e767fc640 1 -- 172.21.15.17:0/4274678133 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0e70012400 con 0x7f0e801383b0 2024-09-15T21:29:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:29:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:16.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.513+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.513+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=11,kv=12,logm=51} crc {auth=3899281877,config=3541143443,health=224038855,kv=3079713637,logm=78792102}) 2024-09-15T21:29:16.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.513+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=22,config=2,health=12,kv=11,logm=53} crc {auth=1916921785,config=3541143443,health=1555756363,kv=1618701482,logm=1588538528}) 2024-09-15T21:29:16.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.513+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.513+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=11,kv=12,logm=51} crc {auth=3899281877,config=3541143443,health=224038855,kv=3079713637,logm=78792102}) 2024-09-15T21:29:16.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.513+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=1,health=10,kv=12,logm=54} crc {auth=2846171063,config=1228700967,health=3785907680,kv=3079713637,logm=3258640186}) 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0e6403b9f0 msgr2=0x7f0e6403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0e6403b9f0 0x7f0e6403deb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c000f70 tx=0x7f0e7c009040 comp rx=0 tx=0).stop 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 10 monclient: shutdown 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 msgr2=0x7f0e80136d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 0x7f0e80136d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000c4d0 tx=0x7f0e7000c990 comp rx=0 tx=0).stop 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 shutdown_connections 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0e6403b9f0 0x7f0e6403deb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0e8013e4a0 0x7f0e8009cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e801383b0 0x7f0e80136d10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 --2- 172.21.15.17:0/4274678133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e801378c0 0x7f0e801367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 >> 172.21.15.17:0/4274678133 conn(0x7f0e800aac40 msgr2=0x7f0e8006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:16.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 shutdown_connections 2024-09-15T21:29:16.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:16.513+0000 7f0e8809e640 1 -- 172.21.15.17:0/4274678133 wait complete. 2024-09-15T21:29:16.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.517+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.517+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680053531}) 2024-09-15T21:29:16.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.517+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1346669720}) 2024-09-15T21:29:16.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.517+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.517+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680053531}) 2024-09-15T21:29:16.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.517+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2913767473}) 2024-09-15T21:29:16.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.525+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.525+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1150690097}) 2024-09-15T21:29:16.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.525+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1405638415}) 2024-09-15T21:29:16.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.525+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.525+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1150690097}) 2024-09-15T21:29:16.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.525+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1118707598}) 2024-09-15T21:29:16.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3119582172}) 2024-09-15T21:29:16.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=278466919}) 2024-09-15T21:29:16.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3119582172}) 2024-09-15T21:29:16.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284604695}) 2024-09-15T21:29:16.533 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:29:16.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1296625667}) 2024-09-15T21:29:16.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3796874734}) 2024-09-15T21:29:16.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1296625667}) 2024-09-15T21:29:16.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.529+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573012624}) 2024-09-15T21:29:16.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.533+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.533+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1714047786}) 2024-09-15T21:29:16.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.533+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4110249140}) 2024-09-15T21:29:16.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.533+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.533+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1714047786}) 2024-09-15T21:29:16.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.533+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3022145404}) 2024-09-15T21:29:16.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=8981896}) 2024-09-15T21:29:16.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1059764074}) 2024-09-15T21:29:16.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=8981896}) 2024-09-15T21:29:16.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1652598489}) 2024-09-15T21:29:16.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=382107117}) 2024-09-15T21:29:16.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174709671}) 2024-09-15T21:29:16.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=382107117}) 2024-09-15T21:29:16.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.537+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3995427808}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1043171002}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3720658220}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1043171002}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422348112}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1955159691}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=801543905}) 2024-09-15T21:29:16.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1955159691}) 2024-09-15T21:29:16.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=666641449}) 2024-09-15T21:29:16.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.541+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2445069977}) 2024-09-15T21:29:16.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3166611135}) 2024-09-15T21:29:16.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2445069977}) 2024-09-15T21:29:16.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2788207833}) 2024-09-15T21:29:16.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3945229932}) 2024-09-15T21:29:16.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1167729342}) 2024-09-15T21:29:16.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3945229932}) 2024-09-15T21:29:16.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2821327514}) 2024-09-15T21:29:16.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2242149305}) 2024-09-15T21:29:16.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1632978139}) 2024-09-15T21:29:16.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2242149305}) 2024-09-15T21:29:16.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.545+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3156057368}) 2024-09-15T21:29:16.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178248071}) 2024-09-15T21:29:16.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4147688193}) 2024-09-15T21:29:16.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178248071}) 2024-09-15T21:29:16.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4139228146}) 2024-09-15T21:29:16.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2726432672}) 2024-09-15T21:29:16.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432274707}) 2024-09-15T21:29:16.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2726432672}) 2024-09-15T21:29:16.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.549+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3429059182}) 2024-09-15T21:29:16.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4051593342}) 2024-09-15T21:29:16.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3910028945}) 2024-09-15T21:29:16.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4051593342}) 2024-09-15T21:29:16.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1625721278}) 2024-09-15T21:29:16.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=774069630}) 2024-09-15T21:29:16.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1173425005}) 2024-09-15T21:29:16.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=774069630}) 2024-09-15T21:29:16.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.553+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=661176765}) 2024-09-15T21:29:16.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1645506789}) 2024-09-15T21:29:16.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2774692106}) 2024-09-15T21:29:16.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1645506789}) 2024-09-15T21:29:16.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2873621705}) 2024-09-15T21:29:16.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=589437458}) 2024-09-15T21:29:16.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=439103012}) 2024-09-15T21:29:16.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=589437458}) 2024-09-15T21:29:16.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.557+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3039454170}) 2024-09-15T21:29:16.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2868979291}) 2024-09-15T21:29:16.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=36990169}) 2024-09-15T21:29:16.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2868979291}) 2024-09-15T21:29:16.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1339056613}) 2024-09-15T21:29:16.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725999545}) 2024-09-15T21:29:16.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2996577268}) 2024-09-15T21:29:16.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725999545}) 2024-09-15T21:29:16.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.561+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1130425992}) 2024-09-15T21:29:16.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2525281735}) 2024-09-15T21:29:16.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3308228157}) 2024-09-15T21:29:16.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2525281735}) 2024-09-15T21:29:16.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3133431169}) 2024-09-15T21:29:16.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557836205}) 2024-09-15T21:29:16.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2610295728}) 2024-09-15T21:29:16.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557836205}) 2024-09-15T21:29:16.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.565+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1204854990}) 2024-09-15T21:29:16.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2883750259}) 2024-09-15T21:29:16.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1462417629}) 2024-09-15T21:29:16.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2883750259}) 2024-09-15T21:29:16.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3914857870}) 2024-09-15T21:29:16.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767235359}) 2024-09-15T21:29:16.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2370845267}) 2024-09-15T21:29:16.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767235359}) 2024-09-15T21:29:16.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.569+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2235833416}) 2024-09-15T21:29:16.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=418084054}) 2024-09-15T21:29:16.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1285830329}) 2024-09-15T21:29:16.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=418084054}) 2024-09-15T21:29:16.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2954282405}) 2024-09-15T21:29:16.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=76272170}) 2024-09-15T21:29:16.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3085493851}) 2024-09-15T21:29:16.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=76272170}) 2024-09-15T21:29:16.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.573+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2825067204}) 2024-09-15T21:29:16.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1403160068}) 2024-09-15T21:29:16.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=176389979}) 2024-09-15T21:29:16.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1403160068}) 2024-09-15T21:29:16.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2440744603}) 2024-09-15T21:29:16.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207519542}) 2024-09-15T21:29:16.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2302218404}) 2024-09-15T21:29:16.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207519542}) 2024-09-15T21:29:16.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.577+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=408314466}) 2024-09-15T21:29:16.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013681956}) 2024-09-15T21:29:16.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2079630940}) 2024-09-15T21:29:16.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013681956}) 2024-09-15T21:29:16.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889625114}) 2024-09-15T21:29:16.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=226390174}) 2024-09-15T21:29:16.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540959021}) 2024-09-15T21:29:16.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=226390174}) 2024-09-15T21:29:16.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.581+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=523738349}) 2024-09-15T21:29:16.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=31985987}) 2024-09-15T21:29:16.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1528004958}) 2024-09-15T21:29:16.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=31985987}) 2024-09-15T21:29:16.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2414674772}) 2024-09-15T21:29:16.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3398421370}) 2024-09-15T21:29:16.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3633685540}) 2024-09-15T21:29:16.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3398421370}) 2024-09-15T21:29:16.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.585+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3160001671}) 2024-09-15T21:29:16.589 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.589 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250327392}) 2024-09-15T21:29:16.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2909092870}) 2024-09-15T21:29:16.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250327392}) 2024-09-15T21:29:16.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=392532364}) 2024-09-15T21:29:16.591 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.591 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2506221949}) 2024-09-15T21:29:16.591 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2784124248}) 2024-09-15T21:29:16.591 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2506221949}) 2024-09-15T21:29:16.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1649673216}) 2024-09-15T21:29:16.593 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.593 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3854554712}) 2024-09-15T21:29:16.593 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3760455494}) 2024-09-15T21:29:16.593 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.593 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3854554712}) 2024-09-15T21:29:16.593 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.589+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254353029}) 2024-09-15T21:29:16.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3345833883}) 2024-09-15T21:29:16.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3502154318}) 2024-09-15T21:29:16.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3345833883}) 2024-09-15T21:29:16.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=788360903}) 2024-09-15T21:29:16.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1045545512}) 2024-09-15T21:29:16.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=235445713}) 2024-09-15T21:29:16.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1045545512}) 2024-09-15T21:29:16.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.593+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016655986}) 2024-09-15T21:29:16.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1012447783}) 2024-09-15T21:29:16.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1662236969}) 2024-09-15T21:29:16.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1012447783}) 2024-09-15T21:29:16.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1346410387}) 2024-09-15T21:29:16.600 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706358749}) 2024-09-15T21:29:16.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798961112}) 2024-09-15T21:29:16.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706358749}) 2024-09-15T21:29:16.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.597+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2716159584}) 2024-09-15T21:29:16.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998187273}) 2024-09-15T21:29:16.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2008139838}) 2024-09-15T21:29:16.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998187273}) 2024-09-15T21:29:16.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1125137925}) 2024-09-15T21:29:16.604 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.604 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440701268}) 2024-09-15T21:29:16.604 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2980195760}) 2024-09-15T21:29:16.605 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.605 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440701268}) 2024-09-15T21:29:16.605 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.601+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1284429857}) 2024-09-15T21:29:16.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464144122}) 2024-09-15T21:29:16.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1403638590}) 2024-09-15T21:29:16.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464144122}) 2024-09-15T21:29:16.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2664485277}) 2024-09-15T21:29:16.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947636238}) 2024-09-15T21:29:16.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2210513901}) 2024-09-15T21:29:16.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.610 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947636238}) 2024-09-15T21:29:16.610 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.605+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3188912490}) 2024-09-15T21:29:16.611 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.611 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1744471650}) 2024-09-15T21:29:16.611 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3320546444}) 2024-09-15T21:29:16.611 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.611 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1744471650}) 2024-09-15T21:29:16.611 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=38754951}) 2024-09-15T21:29:16.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1505191314}) 2024-09-15T21:29:16.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=534935389}) 2024-09-15T21:29:16.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1505191314}) 2024-09-15T21:29:16.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.609+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1944712434}) 2024-09-15T21:29:16.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3296246598}) 2024-09-15T21:29:16.615 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2602981740}) 2024-09-15T21:29:16.615 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.615 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3296246598}) 2024-09-15T21:29:16.615 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3036660595}) 2024-09-15T21:29:16.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476247051}) 2024-09-15T21:29:16.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2536706291}) 2024-09-15T21:29:16.617 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.617 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476247051}) 2024-09-15T21:29:16.617 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.613+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=551664900}) 2024-09-15T21:29:16.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1073738904}) 2024-09-15T21:29:16.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1567656087}) 2024-09-15T21:29:16.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1073738904}) 2024-09-15T21:29:16.619 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=404021498}) 2024-09-15T21:29:16.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269579749}) 2024-09-15T21:29:16.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=719350034}) 2024-09-15T21:29:16.621 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.621 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269579749}) 2024-09-15T21:29:16.621 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.617+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=313775832}) 2024-09-15T21:29:16.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2333266781}) 2024-09-15T21:29:16.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1928827739}) 2024-09-15T21:29:16.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2333266781}) 2024-09-15T21:29:16.623 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4221230546}) 2024-09-15T21:29:16.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869726210}) 2024-09-15T21:29:16.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054125969}) 2024-09-15T21:29:16.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869726210}) 2024-09-15T21:29:16.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.621+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808819024}) 2024-09-15T21:29:16.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958787126}) 2024-09-15T21:29:16.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2181658434}) 2024-09-15T21:29:16.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958787126}) 2024-09-15T21:29:16.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=523172533}) 2024-09-15T21:29:16.627 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.627 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011783023}) 2024-09-15T21:29:16.627 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=276985467}) 2024-09-15T21:29:16.627 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011783023}) 2024-09-15T21:29:16.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2499142011}) 2024-09-15T21:29:16.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196465634}) 2024-09-15T21:29:16.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=361250490}) 2024-09-15T21:29:16.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196465634}) 2024-09-15T21:29:16.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.625+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410355076}) 2024-09-15T21:29:16.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4272201567}) 2024-09-15T21:29:16.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321269159}) 2024-09-15T21:29:16.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4272201567}) 2024-09-15T21:29:16.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=926086614}) 2024-09-15T21:29:16.632 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4274712899}) 2024-09-15T21:29:16.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=533244774}) 2024-09-15T21:29:16.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4274712899}) 2024-09-15T21:29:16.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.629+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436756531}) 2024-09-15T21:29:16.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=287102441}) 2024-09-15T21:29:16.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3500783441}) 2024-09-15T21:29:16.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=287102441}) 2024-09-15T21:29:16.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2326750916}) 2024-09-15T21:29:16.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131071443}) 2024-09-15T21:29:16.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1284303601}) 2024-09-15T21:29:16.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131071443}) 2024-09-15T21:29:16.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.633+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232153619}) 2024-09-15T21:29:16.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2778508774}) 2024-09-15T21:29:16.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2976407265}) 2024-09-15T21:29:16.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2778508774}) 2024-09-15T21:29:16.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4064005788}) 2024-09-15T21:29:16.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2411765812}) 2024-09-15T21:29:16.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3025188906}) 2024-09-15T21:29:16.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2411765812}) 2024-09-15T21:29:16.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685506154}) 2024-09-15T21:29:16.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3189231812}) 2024-09-15T21:29:16.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3464788236}) 2024-09-15T21:29:16.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.642 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3189231812}) 2024-09-15T21:29:16.642 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.637+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=874049654}) 2024-09-15T21:29:16.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3626948564}) 2024-09-15T21:29:16.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3303080009}) 2024-09-15T21:29:16.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3626948564}) 2024-09-15T21:29:16.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3101866130}) 2024-09-15T21:29:16.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719224067}) 2024-09-15T21:29:16.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2737747417}) 2024-09-15T21:29:16.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719224067}) 2024-09-15T21:29:16.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.641+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1076668623}) 2024-09-15T21:29:16.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100259549}) 2024-09-15T21:29:16.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445358450}) 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100259549}) 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1075313024}) 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2406589637}) 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1537601826}) 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2406589637}) 2024-09-15T21:29:16.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.645+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3569487593}) 2024-09-15T21:29:16.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.649+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.649+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7} crc {logm=71451421,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgrstat=577332420}) 2024-09-15T21:29:16.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.649+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=3487266739,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875}) 2024-09-15T21:29:16.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.649+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.649+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7} crc {logm=71451421,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgrstat=577332420}) 2024-09-15T21:29:16.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.649+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=85,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=2990110816,mdsmap=3508357332,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3728090950}) 2024-09-15T21:29:16.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.661+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.661+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=4,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=50} crc {mgrstat=1133892088,mon_config_key=2189608058,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3407157462}) 2024-09-15T21:29:16.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.661+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=5,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=51} crc {mgrstat=117461825,mon_config_key=1492161431,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3693463367}) 2024-09-15T21:29:16.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.661+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:29:16.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.661+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=4,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=50} crc {mgrstat=1133892088,mon_config_key=2189608058,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3407157462}) 2024-09-15T21:29:16.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:29:16.661+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=5,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=4,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {mgrstat=117461825,mon_config_key=2985153859,monmap=186996059,nvmeofgw=598337686,osd_metadata=3235920081,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3169519292}) 2024-09-15T21:29:17.534 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:29:17.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:17.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient: get_monmap_and_config 2024-09-15T21:29:17.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient: build_initial_monmap 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient: monmap: 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:17.664259+0000 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:17.664259+0000 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:17.666 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 Processor -- start 2024-09-15T21:29:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 -- start start 2024-09-15T21:29:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient: init 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 0x7f3fb009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): picked mon.noname-b con 0x7f3fb009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3fb009dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fb0134d70 0x7f3fb013cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): picked mon.noname-a con 0x7f3fb0134d70 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fb00a3130 con 0x7f3fb009dd20 2024-09-15T21:29:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb00a1bc0 con 0x7f3fb0134d70 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb00a3490 con 0x7f3fb009d400 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb6e60640 10 monclient(hunting): authenticate will time out at 1681.059866s 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.661+0000 7f3fb4bfc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 0x7f3fb009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fb0134d70 0x7f3fb013cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fb0134d70 0x7f3fb013cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39292/0 (socket says 172.21.15.17:39292) 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 0x7f3fb009d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53422/0 (socket says 172.21.15.17:53422) 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 -- 172.21.15.17:0/2466354231 learned_addr learned my addr 172.21.15.17:0/2466354231 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): get_auth_request con 0x7f3fb0134d70 auth_method 0 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): get_auth_request con 0x7f3fb009d400 auth_method 0 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_done global_id 144147 payload 274 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 -- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 msgr2=0x7f3fb009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 --2- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 0x7f3fb009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 -- 172.21.15.17:0/2466354231 >> v1:172.21.15.119:6789/0 conn(0x7f3fb009dd20 legacy=0x7f3fb0134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient: _finish_hunting 0 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 monclient: found mon.noname-a 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 -- 172.21.15.17:0/2466354231 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb009e7a0 con 0x7f3fb0134d70 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb4bfc640 1 --2- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 0x7f3fb009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb53fd640 1 --2- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013cad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009870 tx=0x7f3fa801e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107185e758567df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 1 -- 172.21.15.17:0/2466354231 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80b2070 con 0x7f3fb0134d70 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: dump: 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 1 -- 172.21.15.17:0/2466354231 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa80ad600 con 0x7f3fb0134d70 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 5 monclient: authenticate success, global_id 144147 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 1 -- 172.21.15.17:0/2466354231 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80ad9d0 con 0x7f3fb0134d70 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 4 set_mon_vals no callback set 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3faf7fe640 10 monclient: dump: 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:17.676 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:17.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 10 monclient: shutdown 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 -- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 msgr2=0x7f3fb013cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013cad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009870 tx=0x7f3fa801e630 comp rx=0 tx=0).stop 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 -- 172.21.15.17:0/2466354231 shutdown_connections 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013cad0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/2466354231 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009d400 0x7f3fb009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 -- 172.21.15.17:0/2466354231 >> 172.21.15.17:0/2466354231 conn(0x7f3fb00a66a0 msgr2=0x7f3fb00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 -- 172.21.15.17:0/2466354231 shutdown_connections 2024-09-15T21:29:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.665+0000 7f3fb6e60640 1 -- 172.21.15.17:0/2466354231 wait complete. 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: build_initial_monmap 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: monmap: 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 Processor -- start 2024-09-15T21:29:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- start start 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: init 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb009d400 0x7f3fb01d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): picked mon.noname-b con 0x7f3fb009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 0x7f3fb01d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): picked mon.noname-c con 0x7f3fb009dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): picked mon.noname-a con 0x7f3fb0134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb009fb80 con 0x7f3fb0134d70 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb009fa00 con 0x7f3fb009d400 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb00a10a0 con 0x7f3fb009dd20 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient(hunting): authenticate will time out at 1681.065194s 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 0x7f3fb01d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 0x7f3fb01d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53426/0 (socket says 172.21.15.17:53426) 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 -- 172.21.15.17:0/584444672 learned_addr learned my addr 172.21.15.17:0/584444672 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): get_auth_request con 0x7f3fb009dd20 auth_method 0 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:17.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): get_auth_request con 0x7f3fb0134d70 auth_method 0 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb4bfc640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb009d400 0x7f3fb01d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb4bfc640 10 monclient(hunting): get_auth_request con 0x7f3fb009d400 auth_method 0 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb4bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb4bfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:17.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb4bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient(hunting): handle_auth_done global_id 134237 payload 824 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 -- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb009d400 msgr2=0x7f3fb01d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb009d400 0x7f3fb01d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 -- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 msgr2=0x7f3fb013a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient: _finish_hunting 0 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 monclient: found mon.noname-c 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 -- 172.21.15.17:0/584444672 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb0121db0 con 0x7f3fb009dd20 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb53fd640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:17.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faffff640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 0x7f3fb01d0600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c870 tx=0x7f3fa000cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0003b80 con 0x7f3fb009dd20 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: dump: 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:17.687 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: _finish_auth 0 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa0003d20 con 0x7f3fb009dd20 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000aa90 con 0x7f3fb009dd20 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 10 monclient: dump: 2024-09-15T21:29:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3faeffd640 4 set_mon_vals no callback set 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 5 monclient: authenticate success, global_id 134237 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _renew_subs 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb0122090 con 0x7f3fb009dd20 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _renew_subs 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb006f330 con 0x7f3fb009dd20 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0073f10 con 0x7f3fb009dd20 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3fa000ac30 con 0x7f3fb009dd20 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f9003b9c0 0x7f3f9003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.669+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3fa0050510 con 0x7f3fb009dd20 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.673+0000 7f3fb4bfc640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f9003b9c0 0x7f3f9003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.673+0000 7f3fb4bfc640 10 monclient: get_auth_request con 0x7f3f9003b9c0 auth_method 0 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.673+0000 7f3fb4bfc640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f9003b9c0 0x7f3f9003de80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009580 tx=0x7f3f98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:17.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.677+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3fa0011c60 con 0x7f3fb009dd20 2024-09-15T21:29:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.677+0000 7f3fad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.677+0000 7f3fad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:17.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.901+0000 7f3fb6e60640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:17.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.901+0000 7f3fb6e60640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:17.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.901+0000 7f3fb6e60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:17.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.901+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3fb00a1500 con 0x7f3fb009dd20 2024-09-15T21:29:17.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.905+0000 7f3fad7fa640 1 -- 172.21.15.17:0/584444672 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f3fa0005080 con 0x7f3fb009dd20 2024-09-15T21:29:17.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.905+0000 7f3fad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:17.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.905+0000 7f3fad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:17.906 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f9003b9c0 msgr2=0x7f3f9003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f9003b9c0 0x7f3f9003de80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009580 tx=0x7f3f98008040 comp rx=0 tx=0).stop 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 10 monclient: shutdown 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 msgr2=0x7f3fb01d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 0x7f3fb01d0600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c870 tx=0x7f3fa000cd30 comp rx=0 tx=0).stop 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 shutdown_connections 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f9003b9c0 0x7f3f9003de80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb0134d70 0x7f3fb013a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3fb009dd20 0x7f3fb01d0600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 --2- 172.21.15.17:0/584444672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb009d400 0x7f3fb01d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:17.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 >> 172.21.15.17:0/584444672 conn(0x7f3fb00a66a0 msgr2=0x7f3fb006f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:17.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 shutdown_connections 2024-09-15T21:29:17.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:17.909+0000 7f3fb6e60640 1 -- 172.21.15.17:0/584444672 wait complete. 2024-09-15T21:29:17.925 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:29:17.925 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:29:17.925 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:29:17.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:29:18.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient: get_monmap_and_config 2024-09-15T21:29:18.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient: monmap: 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:18.046732+0000 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:18.046732+0000 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:18.047 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 1 Processor -- start 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 1 -- start start 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient: init 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4cf41340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4cf409d6c0 0x7f4cf4138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient(hunting): picked mon.noname-c con 0x7f4cf409d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:18.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4cf4139390 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf409e1b0 con 0x7f4cf4139390 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf409e490 con 0x7f4cf41340c0 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf409d520 con 0x7f4cf409d6c0 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 10 monclient(hunting): authenticate will time out at 1681.442227s 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4cf4139390 0x7f4cf413b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45522/0 (socket says 172.21.15.17:45522) 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 1 -- 172.21.15.17:0/3367450302 learned_addr learned my addr 172.21.15.17:0/3367450302 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 1 --2- 172.21.15.17:0/3367450302 >> v2:172.21.15.119:3300/0 conn(0x7f4cf409d6c0 0x7f4cf4138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): get_auth_request con 0x7f4cf409d6c0 auth_method 0 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367144714 0 0) 0x7f4cf409e1b0 con 0x7f4cf4139390 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): my global_id is 144153 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cdc003530 con 0x7f4cf4139390 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 258454704 0 0) 0x7f4cdc003530 con 0x7f4cf4139390 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 >> v2:172.21.15.119:3300/0 conn(0x7f4cf409d6c0 msgr2=0x7f4cf4138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 --2- 172.21.15.17:0/3367450302 >> v2:172.21.15.119:3300/0 conn(0x7f4cf409d6c0 0x7f4cf4138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 >> v1:172.21.15.119:6789/0 conn(0x7f4cf41340c0 legacy=0x7f4cf41344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf409e770 con 0x7f4cf4139390 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f4cec001a80 con 0x7f4cf4139390 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: dump: 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 >> v1:172.21.15.17:6789/0 conn(0x7f4cf4139390 legacy=0x7f4cf413b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cdc003a60 0x7f4cdc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): picked mon.c con 0x7f4cdc003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc004810 0x7f4cdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): picked mon.b con 0x7f4cdc004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cdc0053d0 0x7f4cdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): picked mon.a con 0x7f4cdc0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf409d520 con 0x7f4cdc0053d0 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc00c240 con 0x7f4cdc004810 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc00e650 con 0x7f4cdc003a60 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf1d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:29:18.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cdc0053d0 0x7f4cdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cdc003a60 0x7f4cdc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): get_auth_request con 0x7f4cdc003a60 auth_method 0 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): get_auth_request con 0x7f4cdc0053d0 auth_method 0 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc004810 0x7f4cdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): get_auth_request con 0x7f4cdc004810 auth_method 0 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_done global_id 144153 payload 293 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 -- 172.21.15.17:0/3367450302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc004810 msgr2=0x7f4cdc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc004810 0x7f4cdc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 -- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cdc0053d0 msgr2=0x7f4cdc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cdc0053d0 0x7f4cdc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 monclient: found mon.c 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 1 -- 172.21.15.17:0/3367450302 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc003530 con 0x7f4cdc003a60 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf2d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 5 monclient: authenticate success, global_id 144153 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf90a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cdc0053d0 0x7f4cdc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.045+0000 7f4cf359e640 1 -- 172.21.15.17:0/3367450302 reap_dead start 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cdc003a60 0x7f4cdc003e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4001790 tx=0x7f4ce401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1a2082b231e667 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40013e0 con 0x7f4cdc003a60 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: dump: 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:18.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce40052d0 con 0x7f4cdc003a60 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 4 set_mon_vals no callback set 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 1 -- 172.21.15.17:0/3367450302 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4003840 con 0x7f4cdc003a60 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf1d9b640 10 monclient: dump: 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 10 monclient: shutdown 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 -- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cdc003a60 msgr2=0x7f4cdc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cdc003a60 0x7f4cdc003e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4001790 tx=0x7f4ce401c370 comp rx=0 tx=0).stop 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 -- 172.21.15.17:0/3367450302 shutdown_connections 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 --2- 172.21.15.17:0/3367450302 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cdc003a60 0x7f4cdc003e80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 -- 172.21.15.17:0/3367450302 >> 172.21.15.17:0/3367450302 conn(0x7f4cf40a66a0 msgr2=0x7f4cf40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 -- 172.21.15.17:0/3367450302 shutdown_connections 2024-09-15T21:29:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 -- 172.21.15.17:0/3367450302 wait complete. 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 10 monclient: monmap: 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 Processor -- start 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 1 -- start start 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.049+0000 7f4cf90a1640 10 monclient: init 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc003a60 0x7f4cf41cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): picked mon.noname-b con 0x7f4cdc003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf412f7d0 0x7f4cf412fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): picked mon.noname-a con 0x7f4cf412f7d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cf40712a0 0x7f4cf4071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): picked mon.noname-c con 0x7f4cf40712a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40a0890 con 0x7f4cf412f7d0 2024-09-15T21:29:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40a27a0 con 0x7f4cdc003a60 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf40a0a10 con 0x7f4cf40712a0 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): authenticate will time out at 1681.448123s 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cf40712a0 0x7f4cf4071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf412f7d0 0x7f4cf412fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cf40712a0 0x7f4cf4071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53446/0 (socket says 172.21.15.17:53446) 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf412f7d0 0x7f4cf412fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39320/0 (socket says 172.21.15.17:39320) 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 -- 172.21.15.17:0/325678732 learned_addr learned my addr 172.21.15.17:0/325678732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): get_auth_request con 0x7f4cf40712a0 auth_method 0 2024-09-15T21:29:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): get_auth_request con 0x7f4cf412f7d0 auth_method 0 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc003a60 0x7f4cf41cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): get_auth_request con 0x7f4cdc003a60 auth_method 0 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient(hunting): handle_auth_done global_id 134240 payload 824 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc003a60 msgr2=0x7f4cf41cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cdc003a60 0x7f4cf41cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf412f7d0 msgr2=0x7f4cf412fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf412f7d0 0x7f4cf412fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 monclient: found mon.noname-c 2024-09-15T21:29:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf411c4e0 con 0x7f4cf40712a0 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf412f7d0 0x7f4cf412fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf359e640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cf40712a0 0x7f4cf4071730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40a2960 tx=0x7f4cec007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec018070 con 0x7f4cf40712a0 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: dump: 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.075 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec007a90 con 0x7f4cf40712a0 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00a4d0 con 0x7f4cf40712a0 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: dump: 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 5 monclient: authenticate success, global_id 134240 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient: _renew_subs 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf159a640 4 set_mon_vals no callback set 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf411d190 con 0x7f4cf40712a0 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient: _renew_subs 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf4130500 con 0x7f4cf40712a0 2024-09-15T21:29:18.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 0x7f4cf4071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40a0d40 con 0x7f4cf411cd60 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 0x7f4cf4071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient: get_auth_request con 0x7f4cf411cd60 auth_method 0 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4cec01b360 con 0x7f4cf40712a0 2024-09-15T21:29:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_done global_id 144159 payload 824 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cf411cb10 con 0x7f4cf411cd60 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ce003b9f0 0x7f4ce003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4cec04ff50 con 0x7f4cf40712a0 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 0x7f4cf4071c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4007fd0 tx=0x7f4ce4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40ba070 con 0x7f4cf411cd60 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: dump: 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ce003b9f0 0x7f4ce003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 10 monclient: get_auth_request con 0x7f4ce003b9f0 auth_method 0 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4ce4001480 con 0x7f4cf411cd60 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 msgr2=0x7f4cf4071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 0x7f4cf4071c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4007fd0 tx=0x7f4ce4019160 comp rx=0 tx=0).stop 2024-09-15T21:29:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 msgr2=0x7f4cf4071c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:18.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cdb7fe640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf411cd60 0x7f4cf4071c70 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4007fd0 tx=0x7f4ce4019160 comp rx=0 tx=0).stop 2024-09-15T21:29:18.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.053+0000 7f4cf259c640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ce003b9f0 0x7f4ce003deb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800af70 tx=0x7f4ce8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:18.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 0x7f4cf4137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40a4360 con 0x7f4cf4136e40 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 0x7f4cf4137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient: get_auth_request con 0x7f4cf4136e40 auth_method 0 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 10 monclient(hunting): handle_auth_done global_id 144162 payload 824 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 1 -- 172.21.15.17:0/325678732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4cf406d020 con 0x7f4cf4136e40 2024-09-15T21:29:18.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf2d9d640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 0x7f4cf4137220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005040 tx=0x7f4cdc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40ba030 con 0x7f4cf4136e40 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 10 monclient: dump: 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.087 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f4ce40069e0 con 0x7f4cf4136e40 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 msgr2=0x7f4cf4137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 0x7f4cf4137220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005040 tx=0x7f4cdc005840 comp rx=0 tx=0).stop 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 msgr2=0x7f4cf4137220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cdb7fe640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf4136e40 0x7f4cf4137220 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005040 tx=0x7f4cdc005840 comp rx=0 tx=0).stop 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:18.088 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 29787, 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:18.089 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:18.090 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:18.092 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.093 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:18.095 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:18.096 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.099 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:18.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:18.101 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ce003b9f0 msgr2=0x7f4ce003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ce003b9f0 0x7f4ce003deb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800af70 tx=0x7f4ce8008040 comp rx=0 tx=0).stop 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 10 monclient: shutdown 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cf40712a0 msgr2=0x7f4cf4071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 --2- 172.21.15.17:0/325678732 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4cf40712a0 0x7f4cf4071730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40a2960 tx=0x7f4cec007430 comp rx=0 tx=0).stop 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf359e640 1 -- 172.21.15.17:0/325678732 reap_dead start 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf359e640 1 -- 172.21.15.17:0/325678732 reap_dead start 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 shutdown_connections 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 >> 172.21.15.17:0/325678732 conn(0x7f4cf40a66a0 msgr2=0x7f4cf41325f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 shutdown_connections 2024-09-15T21:29:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.061+0000 7f4cf90a1640 1 -- 172.21.15.17:0/325678732 wait complete. 2024-09-15T21:29:18.104 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:29:18.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:29:18.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.217+0000 7f61c8d90640 10 monclient: get_monmap_and_config 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.217+0000 7f61c8d90640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.217+0000 7f61c8d90640 10 monclient: monmap: 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:18.221433+0000 2024-09-15T21:29:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:18.221433+0000 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:18.222 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 1 Processor -- start 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 1 -- start start 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient: init 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f61c40a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61c409ffc0 0x7f61c413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient(hunting): picked mon.noname-c con 0x7f61c409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f61c40a1380 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c40a3130 con 0x7f61c40a0890 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c4073df0 con 0x7f61c40a1380 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f61c40a1bf0 con 0x7f61c409ffc0 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 10 monclient(hunting): authenticate will time out at 1681.616930s 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f61c40a0890 0x7f61c40a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45530/0 (socket says 172.21.15.17:45530) 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 -- 172.21.15.17:0/414630277 learned_addr learned my addr 172.21.15.17:0/414630277 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 1 --2- 172.21.15.17:0/414630277 >> v2:172.21.15.119:3300/0 conn(0x7f61c409ffc0 0x7f61c413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request con 0x7f61c409ffc0 auth_method 0 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123284510 0 0) 0x7f61c40a3130 con 0x7f61c40a0890 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): _negotiate 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): my global_id is 144165 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61ac003530 con 0x7f61c40a0890 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4053003554 0 0) 0x7f61ac003530 con 0x7f61c40a0890 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 >> v2:172.21.15.119:3300/0 conn(0x7f61c409ffc0 msgr2=0x7f61c413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 --2- 172.21.15.17:0/414630277 >> v2:172.21.15.119:3300/0 conn(0x7f61c409ffc0 0x7f61c413aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 >> v1:172.21.15.119:6789/0 conn(0x7f61c40a1380 legacy=0x7f61c413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c409ccd0 con 0x7f61c40a0890 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f61b4001a80 con 0x7f61c40a0890 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: dump: 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 >> v1:172.21.15.17:6789/0 conn(0x7f61c40a0890 legacy=0x7f61c40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61ac003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): picked mon.a con 0x7f61ac003a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61ac004810 0x7f61ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): picked mon.c con 0x7f61ac004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61ac0053d0 0x7f61ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): picked mon.b con 0x7f61ac0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61c40a1bf0 con 0x7f61ac003a90 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac00c270 con 0x7f61ac0053d0 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f61ac00c470 con 0x7f61ac004810 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c159a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61ac003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): get_auth_request con 0x7f61ac003a90 auth_method 0 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61ac004810 0x7f61ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request con 0x7f61ac004810 auth_method 0 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c2d9d640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61ac0053d0 0x7f61ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient(hunting): handle_auth_done global_id 144165 payload 293 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 -- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61ac004810 msgr2=0x7f61ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61ac004810 0x7f61ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 -- 172.21.15.17:0/414630277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61ac0053d0 msgr2=0x7f61ac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61ac0053d0 0x7f61ac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 monclient: found mon.a 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 1 -- 172.21.15.17:0/414630277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac003530 con 0x7f61ac003a90 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 5 monclient: authenticate success, global_id 144165 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c8d90640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c2d9d640 10 monclient: get_auth_request con 0x7f61ac0053d0 auth_method 0 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c2d9d640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61ac0053d0 0x7f61ac00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.221+0000 7f61c2d9d640 1 -- 172.21.15.17:0/414630277 reap_dead start 2024-09-15T21:29:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61ac003eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61b400a8b0 tx=0x7f61b401d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2540f94304580d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b400a500 con 0x7f61ac003a90 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: dump: 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b40ad650 con 0x7f61ac003a90 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 1 -- 172.21.15.17:0/414630277 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40ada80 con 0x7f61ac003a90 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 4 set_mon_vals no callback set 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c159a640 10 monclient: dump: 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.237 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: shutdown 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 msgr2=0x7f61ac003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61ac003eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61b400a8b0 tx=0x7f61b401d2c0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- 172.21.15.17:0/414630277 shutdown_connections 2024-09-15T21:29:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 --2- 172.21.15.17:0/414630277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61ac003eb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- 172.21.15.17:0/414630277 >> 172.21.15.17:0/414630277 conn(0x7f61c40a66a0 msgr2=0x7f61c40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- 172.21.15.17:0/414630277 shutdown_connections 2024-09-15T21:29:18.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- 172.21.15.17:0/414630277 wait complete. 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: monmap: 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:18.246 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 Processor -- start 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- start start 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: init 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61c41cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): picked mon.noname-a con 0x7f61ac003a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c41ce270 0x7f61c4123140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): picked mon.noname-c con 0x7f61c41ce270 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c41ce6b0 0x7f61c4123680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): picked mon.noname-b con 0x7f61c41ce6b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4136210 con 0x7f61ac003a90 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4136090 con 0x7f61c41ce6b0 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f61c4136390 con 0x7f61c41ce270 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient(hunting): authenticate will time out at 1681.623000s 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61c41cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c41ce270 0x7f61c4123140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61c41cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39360/0 (socket says 172.21.15.17:39360) 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 1 -- 172.21.15.17:0/4073905577 learned_addr learned my addr 172.21.15.17:0/4073905577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:18.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): get_auth_request con 0x7f61ac003a90 auth_method 0 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request con 0x7f61c41ce270 auth_method 0 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_done global_id 134243 payload 824 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c2d9d640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c41ce6b0 0x7f61c4123680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c41ce6b0 msgr2=0x7f61c4123680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c41ce6b0 0x7f61c4123680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 msgr2=0x7f61c41cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61c41cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 monclient: found mon.noname-c 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c40725d0 con 0x7f61c41ce270 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c259c640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61ac003a90 0x7f61c41cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:18.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c1d9b640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c41ce270 0x7f61c4123140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61b8011960 tx=0x7f61b8011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8019070 con 0x7f61c41ce270 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: dump: 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.251 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8004240 con 0x7f61c41ce270 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 5 monclient: authenticate success, global_id 134243 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: _renew_subs 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c409d0f0 con 0x7f61c41ce270 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b800f880 con 0x7f61c41ce270 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b2ffd640 10 monclient: dump: 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61b3fff640 4 set_mon_vals no callback set 2024-09-15T21:29:18.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.225+0000 7f61c8d90640 10 monclient: _renew_subs 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c8d90640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c4127940 con 0x7f61c41ce270 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c8d90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c8d90640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 0x7f61c4123fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c8d90640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f61c4138ab0 con 0x7f61c4123bc0 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 0x7f61c4123fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient: get_auth_request con 0x7f61c4123bc0 auth_method 0 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f61b801c070 con 0x7f61c41ce270 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 10 monclient(hunting): handle_auth_done global_id 134246 payload 824 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61c41244e0 con 0x7f61c4123bc0 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61a403b9c0 0x7f61a403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f61b8050b70 con 0x7f61c41ce270 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c259c640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 0x7f61c4123fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005c30 tx=0x7f61b401a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40bd070 con 0x7f61c4123bc0 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 10 monclient: dump: 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c2d9d640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61a403b9c0 0x7f61a403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c2d9d640 10 monclient: get_auth_request con 0x7f61a403b9c0 auth_method 0 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f61b4001410 con 0x7f61c4123bc0 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 msgr2=0x7f61c4123fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 0x7f61c4123fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005c30 tx=0x7f61b401a100 comp rx=0 tx=0).stop 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 msgr2=0x7f61c4123fa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61b2ffd640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c4123bc0 0x7f61c4123fa0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005c30 tx=0x7f61b401a100 comp rx=0 tx=0).stop 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.229+0000 7f61c2d9d640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61a403b9c0 0x7f61a403de80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0039c0 tx=0x7f61bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 0x7f61c411e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f61c406d020 con 0x7f61c411e4f0 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 0x7f61c411e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient: get_auth_request con 0x7f61c411e4f0 auth_method 0 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 10 monclient(hunting): handle_auth_done global_id 134249 payload 824 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 1 -- 172.21.15.17:0/4073905577 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f61c4125760 con 0x7f61c411e4f0 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c259c640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 0x7f61c411e8d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000f90 tx=0x7f61ac005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40bd030 con 0x7f61c411e4f0 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 10 monclient: dump: 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f61b40c0b70 con 0x7f61c411e4f0 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 msgr2=0x7f61c411e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 0x7f61c411e8d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000f90 tx=0x7f61ac005840 comp rx=0 tx=0).stop 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 msgr2=0x7f61c411e8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61b2ffd640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c411e4f0 0x7f61c411e8d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 75391, 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:18.260 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.261 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.262 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:18.263 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:18.266 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:18.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.270 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:18.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.272 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:18.273 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61a403b9c0 msgr2=0x7f61a403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f61a403b9c0 0x7f61a403de80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0039c0 tx=0x7f61bc009040 comp rx=0 tx=0).stop 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 10 monclient: shutdown 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c41ce270 msgr2=0x7f61c4123140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c41ce270 0x7f61c4123140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61b8011960 tx=0x7f61b8011e20 comp rx=0 tx=0).stop 2024-09-15T21:29:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c2d9d640 1 -- 172.21.15.17:0/4073905577 reap_dead start 2024-09-15T21:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 shutdown_connections 2024-09-15T21:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 --2- 172.21.15.17:0/4073905577 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f61c41ce270 0x7f61c4123140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 >> 172.21.15.17:0/4073905577 conn(0x7f61c40a66a0 msgr2=0x7f61c4072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 shutdown_connections 2024-09-15T21:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.237+0000 7f61c8d90640 1 -- 172.21.15.17:0/4073905577 wait complete. 2024-09-15T21:29:18.275 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:29:18.275 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:29:18.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:29:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient: get_monmap_and_config 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient: monmap: 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:18.394324+0000 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:18.394324+0000 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 1 Processor -- start 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 1 -- start start 2024-09-15T21:29:18.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient: init 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81378c0 0x7f98a8137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient(hunting): picked mon.noname-b con 0x7f98a81378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f98a81381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f98a80a5210 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a80a1a20 con 0x7f98a81381e0 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a80a1d00 con 0x7f98a80a5210 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f98a80a1fe0 con 0x7f98a81378c0 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98ad713640 10 monclient(hunting): authenticate will time out at 1681.789722s 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f98a81381e0 0x7f98a80a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45544/0 (socket says 172.21.15.17:45544) 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a67fc640 1 -- 172.21.15.17:0/352070751 learned_addr learned my addr 172.21.15.17:0/352070751 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/352070751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81378c0 0x7f98a8137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): get_auth_request con 0x7f98a81378c0 auth_method 0 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_done global_id 134252 payload 274 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 1 -- 172.21.15.17:0/352070751 >> v1:172.21.15.119:6789/0 conn(0x7f98a80a5210 legacy=0x7f98a80a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a5ffb640 1 -- 172.21.15.17:0/352070751 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169875019 0 0) 0x7f9890001120 con 0x7f98a81381e0 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 1 -- 172.21.15.17:0/352070751 >> v1:172.21.15.17:6789/0 conn(0x7f98a81381e0 legacy=0x7f98a80a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 1 monclient: found mon.noname-b 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:29:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a6ffd640 1 -- 172.21.15.17:0/352070751 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a80a22c0 con 0x7f98a81378c0 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.393+0000 7f98a5ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/352070751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81378c0 0x7f98a8137ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009dc0 tx=0x7f98a001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa31ff4dc81b30ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 1 -- 172.21.15.17:0/352070751 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00b1070 con 0x7f98a81378c0 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: dump: 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 5 monclient: authenticate success, global_id 134252 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 1 -- 172.21.15.17:0/352070751 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a00a4cb0 con 0x7f98a81378c0 2024-09-15T21:29:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 4 set_mon_vals no callback set 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 1 -- 172.21.15.17:0/352070751 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00a4e60 con 0x7f98a81378c0 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a5ffb640 10 monclient: dump: 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: shutdown 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- 172.21.15.17:0/352070751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81378c0 msgr2=0x7f98a8137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 --2- 172.21.15.17:0/352070751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81378c0 0x7f98a8137ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009dc0 tx=0x7f98a001c530 comp rx=0 tx=0).stop 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- 172.21.15.17:0/352070751 shutdown_connections 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 --2- 172.21.15.17:0/352070751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81378c0 0x7f98a8137ca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- 172.21.15.17:0/352070751 >> 172.21.15.17:0/352070751 conn(0x7f98a80aac40 msgr2=0x7f98a80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- 172.21.15.17:0/352070751 shutdown_connections 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- 172.21.15.17:0/352070751 wait complete. 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: monmap: 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:18.408 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 Processor -- start 2024-09-15T21:29:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- start start 2024-09-15T21:29:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: init 2024-09-15T21:29:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:29:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a80a5210 0x7f98a8136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): picked mon.noname-b con 0x7f98a80a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 0x7f98a813ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): picked mon.noname-a con 0x7f98a81378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81381e0 0x7f98a813d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): picked mon.noname-c con 0x7f98a81381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8144270 con 0x7f98a81378c0 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81440f0 con 0x7f98a80a5210 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f98a81443f0 con 0x7f98a81381e0 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient(hunting): authenticate will time out at 1681.794989s 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 0x7f98a813ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81381e0 0x7f98a813d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 0x7f98a813ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39370/0 (socket says 172.21.15.17:39370) 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 -- 172.21.15.17:0/3412335586 learned_addr learned my addr 172.21.15.17:0/3412335586 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): get_auth_request con 0x7f98a81378c0 auth_method 0 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): get_auth_request con 0x7f98a81381e0 auth_method 0 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a80a5210 0x7f98a8136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a6ffd640 10 monclient(hunting): get_auth_request con 0x7f98a80a5210 auth_method 0 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient(hunting): handle_auth_done global_id 144174 payload 824 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81381e0 msgr2=0x7f98a813d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81381e0 0x7f98a813d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a80a5210 msgr2=0x7f98a8136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a80a5210 0x7f98a8136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 monclient: found mon.noname-a 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a81d1630 con 0x7f98a81378c0 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a77fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f98a81381e0 0x7f98a813d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98a67fc640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 0x7f98a813ce30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f989000c730 tx=0x7f989000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890017070 con 0x7f98a81378c0 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 10 monclient: dump: 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890007e50 con 0x7f98a81378c0 2024-09-15T21:29:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 5 monclient: authenticate success, global_id 144174 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: _renew_subs 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a813f5e0 con 0x7f98a81378c0 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.397+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890005d00 con 0x7f98a81378c0 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: dump: 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 10 monclient: _renew_subs 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a4ff9640 4 set_mon_vals no callback set 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a8140830 con 0x7f98a81378c0 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 0x7f98a813d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8137ca0 con 0x7f98a81370b0 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f989001a070 con 0x7f98a81378c0 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 0x7f98a813d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient: get_auth_request con 0x7f98a81370b0 auth_method 0 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f988c03ba20 0x7f988c03dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f989004f9f0 con 0x7f98a81378c0 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a77fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f988c03ba20 0x7f988c03dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a77fe640 10 monclient: get_auth_request con 0x7f988c03ba20 auth_method 0 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_done global_id 114418 payload 824 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98a80008e0 con 0x7f98a81370b0 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 0x7f98a813d8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98a80a7b50 tx=0x7f98a00195f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00bc070 con 0x7f98a81370b0 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: dump: 2024-09-15T21:29:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f98a77fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f988c03ba20 0x7f988c03dee0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9898007fd0 tx=0x7f9898007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f98a0003ba0 con 0x7f98a81370b0 2024-09-15T21:29:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 msgr2=0x7f98a813d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 0x7f98a813d8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98a80a7b50 tx=0x7f98a00195f0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 msgr2=0x7f98a813d8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.401+0000 7f989f7fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a81370b0 0x7f98a813d8b0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98a80a7b50 tx=0x7f98a00195f0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 0x7f98a8138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98a813e200 con 0x7f98a813ddf0 2024-09-15T21:29:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 0x7f98a8138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient: get_auth_request con 0x7f98a813ddf0 auth_method 0 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 10 monclient(hunting): handle_auth_done global_id 114421 payload 824 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 1 -- 172.21.15.17:0/3412335586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f98a806cba0 con 0x7f98a813ddf0 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 0x7f98a8138d60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98a001cdd0 tx=0x7f98a00a6000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00b1040 con 0x7f98a813ddf0 2024-09-15T21:29:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 10 monclient: dump: 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7f98a00b7440 con 0x7f98a813ddf0 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 msgr2=0x7f98a8138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 0x7f98a8138d60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98a001cdd0 tx=0x7f98a00a6000 comp rx=0 tx=0).stop 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 msgr2=0x7f98a8138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f989f7fe640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a813ddf0 0x7f98a8138d60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 166978, 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:18.438 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.439 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:18.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:18.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:29:18.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:18.444 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:18.445 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f988c03ba20 msgr2=0x7f988c03dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f988c03ba20 0x7f988c03dee0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9898007fd0 tx=0x7f9898007dc0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 10 monclient: shutdown 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 msgr2=0x7f98a813ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 0x7f98a813ce30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f989000c730 tx=0x7f989000cbf0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98a77fe640 1 -- 172.21.15.17:0/3412335586 reap_dead start 2024-09-15T21:29:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 shutdown_connections 2024-09-15T21:29:18.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 --2- 172.21.15.17:0/3412335586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a81378c0 0x7f98a813ce30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 >> 172.21.15.17:0/3412335586 conn(0x7f98a80aac40 msgr2=0x7f98a80a4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 shutdown_connections 2024-09-15T21:29:18.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.409+0000 7f98ad713640 1 -- 172.21.15.17:0/3412335586 wait complete. 2024-09-15T21:29:18.448 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:29:18.448 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:29:18.449 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:29:18.449 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:29:18.449 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:29:18.449 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:29:18.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:29:18.466 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:29:18.467 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:29:18.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.577+0000 7f3e62ea4640 10 monclient: get_monmap_and_config 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.577+0000 7f3e62ea4640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.577+0000 7f3e62ea4640 10 monclient: monmap: 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:18.580291+0000 2024-09-15T21:29:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:18.580291+0000 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:18.582 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.577+0000 7f3e62ea4640 1 Processor -- start 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.577+0000 7f3e62ea4640 1 -- start start 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.577+0000 7f3e62ea4640 10 monclient: init 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3e5c1378c0 0x7f3e5c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): picked mon.noname-a con 0x7f3e5c1378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e5c1381e0 0x7f3e5c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): picked mon.noname-c con 0x7f3e5c1381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3e5c0a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e5c0a1d00 con 0x7f3e5c0a5040 2024-09-15T21:29:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c0a1fe0 con 0x7f3e5c1378c0 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c0a22c0 con 0x7f3e5c1381e0 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e62ea4640 10 monclient(hunting): authenticate will time out at 1681.976383s 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e61441640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f3e5c0a5040 legacy=0x7f3e5c0a54d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e5bfff640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f3e5c1381e0 msgr2=0x7f3e5c0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e5bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e5c1381e0 0x7f3e5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3e5c1378c0 0x7f3e5c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3e5c1378c0 0x7f3e5c137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39392/0 (socket says 172.21.15.17:39392) 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 -- 172.21.15.17:0/1979996710 learned_addr learned my addr 172.21.15.17:0/1979996710 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): get_auth_request con 0x7f3e5c1378c0 auth_method 0 2024-09-15T21:29:18.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient(hunting): handle_auth_done global_id 144177 payload 274 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 -- 172.21.15.17:0/1979996710 >> v2:172.21.15.119:3300/0 conn(0x7f3e5c1381e0 msgr2=0x7f3e5c0a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:18.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 --2- 172.21.15.17:0/1979996710 >> v2:172.21.15.119:3300/0 conn(0x7f3e5c1381e0 0x7f3e5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 -- 172.21.15.17:0/1979996710 >> v1:172.21.15.119:6789/0 conn(0x7f3e5c0a5040 legacy=0x7f3e5c0a54d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 monclient: found mon.noname-a 2024-09-15T21:29:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 -- 172.21.15.17:0/1979996710 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0a4490 con 0x7f3e5c1378c0 2024-09-15T21:29:18.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.581+0000 7f3e60c40640 1 --2- 172.21.15.17:0/1979996710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1378c0 0x7f3e5c137ca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009870 tx=0x7f3e5001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ce56ea5821347c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 1 -- 172.21.15.17:0/1979996710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e500b1070 con 0x7f3e5c1378c0 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: dump: 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 1 -- 172.21.15.17:0/1979996710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e500a8050 con 0x7f3e5c1378c0 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 5 monclient: authenticate success, global_id 144177 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 1 -- 172.21.15.17:0/1979996710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e500ac860 con 0x7f3e5c1378c0 2024-09-15T21:29:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 4 set_mon_vals no callback set 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5b7fe640 10 monclient: dump: 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: shutdown 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1979996710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1378c0 msgr2=0x7f3e5c137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1979996710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1378c0 0x7f3e5c137ca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009870 tx=0x7f3e5001c400 comp rx=0 tx=0).stop 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1979996710 shutdown_connections 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1979996710 >> v2:172.21.15.119:3300/0 conn(0x7f3e5c1381e0 0x7f3e5c0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1979996710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1378c0 0x7f3e5c137ca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1979996710 >> 172.21.15.17:0/1979996710 conn(0x7f3e5c0aac40 msgr2=0x7f3e5c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1979996710 shutdown_connections 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1979996710 wait complete. 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: build_initial_monmap 2024-09-15T21:29:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: monmap: 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 Processor -- start 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- start start 2024-09-15T21:29:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: init 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e5c0a5040 0x7f3e5c1cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): picked mon.noname-b con 0x7f3e5c0a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e5c1378c0 0x7f3e5c136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): picked mon.noname-c con 0x7f3e5c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 0x7f3e5c13ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): picked mon.noname-a con 0x7f3e5c1381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c06f340 con 0x7f3e5c1381e0 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c06f1c0 con 0x7f3e5c0a5040 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c1425c0 con 0x7f3e5c1378c0 2024-09-15T21:29:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient(hunting): authenticate will time out at 1681.982343s 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 0x7f3e5c13ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e5c1378c0 0x7f3e5c136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e60c40640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e5c0a5040 msgr2=0x7f3e5c1cf8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 0x7f3e5c13ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39406/0 (socket says 172.21.15.17:39406) 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 -- 172.21.15.17:0/1841357767 learned_addr learned my addr 172.21.15.17:0/1841357767 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e5c1378c0 0x7f3e5c136b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53526/0 (socket says 172.21.15.17:53526) 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): get_auth_request con 0x7f3e5c1381e0 auth_method 0 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e60c40640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e5c0a5040 0x7f3e5c1cf8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): get_auth_request con 0x7f3e5c1378c0 auth_method 0 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient(hunting): handle_auth_done global_id 144180 payload 824 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 -- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e5c1378c0 msgr2=0x7f3e5c136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e5c1378c0 0x7f3e5c136b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 -- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e5c0a5040 msgr2=0x7f3e5c1cf8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e5c0a5040 0x7f3e5c1cf8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient: _finish_hunting 0 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 monclient: found mon.noname-a 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 -- 172.21.15.17:0/1841357767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c13f360 con 0x7f3e5c1381e0 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e61441640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 0x7f3e5c13ce30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b790 tx=0x7f3e5400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400c960 con 0x7f3e5c1381e0 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: dump: 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: _finish_auth 0 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: _check_auth_tickets 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e54010070 con 0x7f3e5c1381e0 2024-09-15T21:29:18.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 5 monclient: authenticate success, global_id 144180 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: _renew_subs 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c0a12f0 con 0x7f3e5c1381e0 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400cd40 con 0x7f3e5c1381e0 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e597fa640 10 monclient: dump: 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: _renew_subs 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e5affd640 4 set_mon_vals no callback set 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.585+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c121520 con 0x7f3e5c1381e0 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e62ea4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e62ea4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e62ea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c1210d0 con 0x7f3e5c1381e0 2024-09-15T21:29:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3e54015470 con 0x7f3e5c1381e0 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e597fa640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e4003b970 0x7f3e4003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3e54050460 con 0x7f3e5c1381e0 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e60c40640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e4003b970 0x7f3e4003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e60c40640 10 monclient: get_auth_request con 0x7f3e4003b970 auth_method 0 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.589+0000 7f3e60c40640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e4003b970 0x7f3e4003de30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3e500ae000 tx=0x7f3e50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.593+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3e5c1210d0 con 0x7f3e5c1381e0 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.593+0000 7f3e597fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.593+0000 7f3e597fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3e5c069270 con 0x7f3e5c1381e0 2024-09-15T21:29:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e597fa640 1 -- 172.21.15.17:0/1841357767 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f3e54007d30 con 0x7f3e5c1381e0 2024-09-15T21:29:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e597fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e597fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:18.835 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:18.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e4003b970 msgr2=0x7f3e4003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e4003b970 0x7f3e4003de30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3e500ae000 tx=0x7f3e50008040 comp rx=0 tx=0).stop 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 10 monclient: shutdown 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 msgr2=0x7f3e5c13ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 0x7f3e5c13ce30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b790 tx=0x7f3e5400bc50 comp rx=0 tx=0).stop 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 shutdown_connections 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3e4003b970 0x7f3e4003de30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e5c1381e0 0x7f3e5c13ce30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3e5c1378c0 0x7f3e5c136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 --2- 172.21.15.17:0/1841357767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e5c0a5040 0x7f3e5c1cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 >> 172.21.15.17:0/1841357767 conn(0x7f3e5c0aac40 msgr2=0x7f3e5c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 shutdown_connections 2024-09-15T21:29:18.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:18.833+0000 7f3e62ea4640 1 -- 172.21.15.17:0/1841357767 wait complete. 2024-09-15T21:29:21.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:21.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient: get_monmap_and_config 2024-09-15T21:29:21.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient: build_initial_monmap 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient: monmap: 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:21.988047+0000 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:21.988047+0000 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:21.989 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:21.989 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:21.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 Processor -- start 2024-09-15T21:29:21.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 -- start start 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient: init 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 0x7fc8e80a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): picked mon.noname-b con 0x7fc8e80a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8e80a53b0 0x7fc8e80a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): picked mon.noname-c con 0x7fc8e80a53b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc8e8137850 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e80a2c30 con 0x7fc8e8137850 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e80a0f00 con 0x7fc8e80a53b0 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e80a19b0 con 0x7fc8e80a4a90 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8efd17640 10 monclient(hunting): authenticate will time out at 1685.383397s 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8ee2b4640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fc8e8137850 legacy=0x7fc8e8137ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 0x7fc8e80a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8ed2b2640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7fc8e80a53b0 msgr2=0x7fc8e80a57d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8ed2b2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8e80a53b0 0x7fc8e80a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 0x7fc8e80a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53542/0 (socket says 172.21.15.17:53542) 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 1 -- 172.21.15.17:0/2454267494 learned_addr learned my addr 172.21.15.17:0/2454267494 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): get_auth_request con 0x7fc8e80a4a90 auth_method 0 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.985+0000 7fc8edab3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 10 monclient(hunting): handle_auth_done global_id 134261 payload 274 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 1 -- 172.21.15.17:0/2454267494 >> v2:172.21.15.119:3300/0 conn(0x7fc8e80a53b0 msgr2=0x7fc8e80a57d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 1 --2- 172.21.15.17:0/2454267494 >> v2:172.21.15.119:3300/0 conn(0x7fc8e80a53b0 0x7fc8e80a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 1 -- 172.21.15.17:0/2454267494 >> v1:172.21.15.119:6789/0 conn(0x7fc8e8137850 legacy=0x7fc8e8137ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 10 monclient: _finish_hunting 0 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 1 monclient: found mon.noname-b 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:29:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 1 -- 172.21.15.17:0/2454267494 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e80a1c90 con 0x7fc8e80a4a90 2024-09-15T21:29:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8edab3640 1 --2- 172.21.15.17:0/2454267494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 0x7fc8e80a4e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009870 tx=0x7fc8d801c2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af2f287dc5779ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 1 -- 172.21.15.17:0/2454267494 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80b0070 con 0x7fc8e80a4a90 2024-09-15T21:29:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:29:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: dump: 2024-09-15T21:29:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: _finish_auth 0 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: _check_auth_tickets 2024-09-15T21:29:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 1 -- 172.21.15.17:0/2454267494 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d80ab600 con 0x7fc8e80a4a90 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 5 monclient: authenticate success, global_id 134261 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 1 -- 172.21.15.17:0/2454267494 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80ab9d0 con 0x7fc8e80a4a90 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 4 set_mon_vals no callback set 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8ecab1640 10 monclient: dump: 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 10 monclient: shutdown 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 1 -- 172.21.15.17:0/2454267494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 msgr2=0x7fc8e80a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.989+0000 7fc8efd17640 1 --2- 172.21.15.17:0/2454267494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 0x7fc8e80a4e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009870 tx=0x7fc8d801c2f0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/2454267494 shutdown_connections 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 --2- 172.21.15.17:0/2454267494 >> v2:172.21.15.119:3300/0 conn(0x7fc8e80a53b0 0x7fc8e80a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 --2- 172.21.15.17:0/2454267494 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a4a90 0x7fc8e80a4e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/2454267494 >> 172.21.15.17:0/2454267494 conn(0x7fc8e80aabb0 msgr2=0x7fc8e80a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/2454267494 shutdown_connections 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/2454267494 wait complete. 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: build_initial_monmap 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: monmap: 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:22.003 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 Processor -- start 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- start start 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: init 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:22.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8e80a4a90 0x7fc8e81cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): picked mon.noname-a con 0x7fc8e80a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 0x7fc8e8136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): picked mon.noname-c con 0x7fc8e80a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8137850 0x7fc8e813bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): picked mon.noname-b con 0x7fc8e8137850 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8142710 con 0x7fc8e80a4a90 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8142590 con 0x7fc8e8137850 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e8142890 con 0x7fc8e80a53b0 2024-09-15T21:29:22.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient(hunting): authenticate will time out at 1685.389811s 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 0x7fc8e8136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 0x7fc8e8136960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53548/0 (socket says 172.21.15.17:53548) 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 -- 172.21.15.17:0/49466953 learned_addr learned my addr 172.21.15.17:0/49466953 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): get_auth_request con 0x7fc8e80a53b0 auth_method 0 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:22.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ee2b4640 1 -- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8137850 msgr2=0x7fc8e813bb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8e80a4a90 0x7fc8e81cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ee2b4640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8137850 0x7fc8e813bb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): get_auth_request con 0x7fc8e80a4a90 auth_method 0 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:22.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8edab3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient(hunting): handle_auth_done global_id 134264 payload 824 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 -- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8137850 msgr2=0x7fc8e813bb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8137850 0x7fc8e813bb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 -- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8e80a4a90 msgr2=0x7fc8e81cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8e80a4a90 0x7fc8e81cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient: _finish_hunting 0 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 monclient: found mon.noname-c 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 -- 172.21.15.17:0/49466953 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e8120d30 con 0x7fc8e80a53b0 2024-09-15T21:29:22.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8ed2b2640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 0x7fc8e8136960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00b530 tx=0x7fc8dc00b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc013020 con 0x7fc8e80a53b0 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: dump: 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:22.009 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: _finish_auth 0 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc003df0 con 0x7fc8e80a53b0 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 5 monclient: authenticate success, global_id 134264 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _renew_subs 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:22.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e8121010 con 0x7fc8e80a53b0 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0052b0 con 0x7fc8e80a53b0 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 10 monclient: dump: 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:22.011 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _renew_subs 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e7fff640 4 set_mon_vals no callback set 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e8120a50 con 0x7fc8e80a53b0 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e80a4e70 con 0x7fc8e80a53b0 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc8dc016020 con 0x7fc8e80a53b0 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc8d403b970 0x7fc8d403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:22.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.993+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc8dc0501d0 con 0x7fc8e80a53b0 2024-09-15T21:29:22.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.997+0000 7fc8edab3640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc8d403b970 0x7fc8d403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:22.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.997+0000 7fc8edab3640 10 monclient: get_auth_request con 0x7fc8d403b970 auth_method 0 2024-09-15T21:29:22.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:21.997+0000 7fc8edab3640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc8d403b970 0x7fc8d403de30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80ad000 tx=0x7fc8d80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:22.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.001+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc8dc01d020 con 0x7fc8e80a53b0 2024-09-15T21:29:22.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.001+0000 7fc8e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:22.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.001+0000 7fc8e67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:22.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8efd17640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:22.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8efd17640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:22.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8efd17640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:22.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc8e8069200 con 0x7fc8e80a53b0 2024-09-15T21:29:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8e67fc640 1 -- 172.21.15.17:0/49466953 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fc8dc00d380 con 0x7fc8e80a53b0 2024-09-15T21:29:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.237+0000 7fc8e67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:22.240 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:22.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc8d403b970 msgr2=0x7fc8d403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:22.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc8d403b970 0x7fc8d403de30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80ad000 tx=0x7fc8d80a6000 comp rx=0 tx=0).stop 2024-09-15T21:29:22.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 10 monclient: shutdown 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 msgr2=0x7fc8e8136960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 0x7fc8e8136960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00b530 tx=0x7fc8dc00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 shutdown_connections 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc8d403b970 0x7fc8d403de30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8137850 0x7fc8e813bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc8e80a53b0 0x7fc8e8136960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 --2- 172.21.15.17:0/49466953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8e80a4a90 0x7fc8e81cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 >> 172.21.15.17:0/49466953 conn(0x7fc8e80aabb0 msgr2=0x7fc8e813ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 shutdown_connections 2024-09-15T21:29:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:22.241+0000 7fc8efd17640 1 -- 172.21.15.17:0/49466953 wait complete. 2024-09-15T21:29:25.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient: get_monmap_and_config 2024-09-15T21:29:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient: build_initial_monmap 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient: monmap: 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:25.391839+0000 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:25.391839+0000 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:25.393 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 Processor -- start 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 -- start start 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient: init 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8e009d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e01398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): picked mon.noname-b con 0x7fd8e009def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8e0139de0 0x7fd8e013c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): picked mon.noname-a con 0x7fd8e0139de0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e00a3490 con 0x7fd8e009d400 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e009f0b0 con 0x7fd8e0139de0 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e00a1bc0 con 0x7fd8e009def0 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e773f640 10 monclient(hunting): authenticate will time out at 1688.787271s 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8e0139de0 0x7fd8e013c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8e0139de0 0x7fd8e013c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39414/0 (socket says 172.21.15.17:39414) 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 -- 172.21.15.17:0/4085020099 learned_addr learned my addr 172.21.15.17:0/4085020099 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): get_auth_request con 0x7fd8e0139de0 auth_method 0 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 1 --2- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e01398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): get_auth_request con 0x7fd8e009def0 auth_method 0 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e4cda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient(hunting): handle_auth_done global_id 144186 payload 274 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 -- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 msgr2=0x7fd8e01398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 --2- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e01398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 -- 172.21.15.17:0/4085020099 >> v1:172.21.15.17:6789/0 conn(0x7fd8e009d400 legacy=0x7fd8e009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient: _finish_hunting 0 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 monclient: found mon.noname-a 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.389+0000 7fd8e5cdc640 1 -- 172.21.15.17:0/4085020099 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e00a0320 con 0x7fd8e0139de0 2024-09-15T21:29:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e5cdc640 1 --2- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e0139de0 0x7fd8e013c1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009e50 tx=0x7fd8dc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7344c7f084339b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 1 -- 172.21.15.17:0/4085020099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0a5780 con 0x7fd8e0139de0 2024-09-15T21:29:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: dump: 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: _finish_auth 0 2024-09-15T21:29:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 5 monclient: authenticate success, global_id 144186 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 1 -- 172.21.15.17:0/4085020099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc0a5920 con 0x7fd8e0139de0 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 4 set_mon_vals no callback set 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 1 -- 172.21.15.17:0/4085020099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0a5cf0 con 0x7fd8e0139de0 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8d7fff640 10 monclient: dump: 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 10 monclient: shutdown 2024-09-15T21:29:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 -- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e0139de0 msgr2=0x7fd8e013c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 --2- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e0139de0 0x7fd8e013c1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009e50 tx=0x7fd8dc01c8c0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 -- 172.21.15.17:0/4085020099 shutdown_connections 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 --2- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e0139de0 0x7fd8e013c1d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 --2- 172.21.15.17:0/4085020099 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e01398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 -- 172.21.15.17:0/4085020099 >> 172.21.15.17:0/4085020099 conn(0x7fd8e00a66a0 msgr2=0x7fd8e00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 -- 172.21.15.17:0/4085020099 shutdown_connections 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 -- 172.21.15.17:0/4085020099 wait complete. 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 10 monclient: build_initial_monmap 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 10 monclient: monmap: 2024-09-15T21:29:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.393+0000 7fd8e773f640 1 Processor -- start 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- start start 2024-09-15T21:29:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: init 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 0x7fd8e011f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): picked mon.noname-a con 0x7fd8e009d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e011f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): picked mon.noname-c con 0x7fd8e009def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e0139de0 0x7fd8e011fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): picked mon.noname-b con 0x7fd8e0139de0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e009ce50 con 0x7fd8e009d400 2024-09-15T21:29:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e009ccd0 con 0x7fd8e0139de0 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e00a10a0 con 0x7fd8e009def0 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient(hunting): authenticate will time out at 1688.792540s 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 0x7fd8e011f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e011f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e011f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53566/0 (socket says 172.21.15.17:53566) 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e5cdc640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e0139de0 msgr2=0x7fd8e011fd30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 0x7fd8e011f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39420/0 (socket says 172.21.15.17:39420) 2024-09-15T21:29:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 1 -- 172.21.15.17:0/2122565064 learned_addr learned my addr 172.21.15.17:0/2122565064 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): get_auth_request con 0x7fd8e009def0 auth_method 0 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e5cdc640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e0139de0 0x7fd8e011fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): get_auth_request con 0x7fd8e009d400 auth_method 0 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient(hunting): handle_auth_done global_id 144189 payload 824 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 -- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 msgr2=0x7fd8e011f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e011f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 -- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e0139de0 msgr2=0x7fd8e011fd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e0139de0 0x7fd8e011fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient: _finish_hunting 0 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 monclient: found mon.noname-a 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 -- 172.21.15.17:0/2122565064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0123080 con 0x7fd8e009d400 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e011f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e54db640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 0x7fd8e011f2b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00a3660 tx=0x7fd8d800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80052d0 con 0x7fd8e009d400 2024-09-15T21:29:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: dump: 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d8005490 con 0x7fd8e009d400 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 5 monclient: authenticate success, global_id 144189 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _renew_subs 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e00a5330 con 0x7fd8e009d400 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8007240 con 0x7fd8e009d400 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 10 monclient: dump: 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _renew_subs 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d6ffd640 4 set_mon_vals no callback set 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e0120d20 con 0x7fd8e009d400 2024-09-15T21:29:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e00a14b0 con 0x7fd8e009d400 2024-09-15T21:29:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd8d80073e0 con 0x7fd8e009d400 2024-09-15T21:29:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd8c403b970 0x7fd8c403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd8d804fac0 con 0x7fd8e009d400 2024-09-15T21:29:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd8c403b970 0x7fd8c403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.397+0000 7fd8e4cda640 10 monclient: get_auth_request con 0x7fd8c403b970 auth_method 0 2024-09-15T21:29:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.401+0000 7fd8e4cda640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd8c403b970 0x7fd8c403de30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0011310 tx=0x7fd8d0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.401+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd8d801a360 con 0x7fd8e009d400 2024-09-15T21:29:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.401+0000 7fd8d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.401+0000 7fd8d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:25.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8e773f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:25.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8e773f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:25.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8e773f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:25.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd8e006d020 con 0x7fd8e009d400 2024-09-15T21:29:25.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8d5ffb640 1 -- 172.21.15.17:0/2122565064 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fd8d8011e10 con 0x7fd8e009d400 2024-09-15T21:29:25.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8d5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:25.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.633+0000 7fd8d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:25.638 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd8c403b970 msgr2=0x7fd8c403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd8c403b970 0x7fd8c403de30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0011310 tx=0x7fd8d0009190 comp rx=0 tx=0).stop 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 10 monclient: shutdown 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 msgr2=0x7fd8e011f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 0x7fd8e011f2b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00a3660 tx=0x7fd8d800c820 comp rx=0 tx=0).stop 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 shutdown_connections 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd8c403b970 0x7fd8c403de30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e0139de0 0x7fd8e011fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd8e009def0 0x7fd8e011f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 --2- 172.21.15.17:0/2122565064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e009d400 0x7fd8e011f2b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 >> 172.21.15.17:0/2122565064 conn(0x7fd8e00a66a0 msgr2=0x7fd8e00a3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 shutdown_connections 2024-09-15T21:29:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:25.637+0000 7fd8e773f640 1 -- 172.21.15.17:0/2122565064 wait complete. 2024-09-15T21:29:28.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:28.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient: get_monmap_and_config 2024-09-15T21:29:28.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient: build_initial_monmap 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient: monmap: 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:28.788032+0000 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:28.788032+0000 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:28.788 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 1 Processor -- start 2024-09-15T21:29:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 1 -- start start 2024-09-15T21:29:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient: init 2024-09-15T21:29:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2024-09-15T21:29:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient(hunting): picked mon.noname-b con 0x7f5e380a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e380a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e380a57d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e380a2c30 con 0x7f5e380a57d0 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e38138670 con 0x7f5e380a53b0 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e380a0f00 con 0x7f5e380a4a90 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.785+0000 7f5e3fdc1640 10 monclient(hunting): authenticate will time out at 1692.183914s 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55356/0 (socket says 172.21.15.17:55356) 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 learned_addr learned my addr 172.21.15.17:0/3370621918 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:28.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request con 0x7f5e380a4a90 auth_method 0 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e38137850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:28.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.789+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:28.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e38137850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:28.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:28.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request con 0x7f5e380a4a90 auth_method 0 2024-09-15T21:29:28.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:28.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:28.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:28.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:28.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:28.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:28.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:28.993+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:29.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e38137850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:29.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request con 0x7f5e380a4a90 auth_method 0 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:29.397+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:30.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e38137850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:30.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:30.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request con 0x7f5e380a4a90 auth_method 0 2024-09-15T21:29:30.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:30.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:30.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:30.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:30.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:30.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:30.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:30.201+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.785+0000 7f5e377fe640 10 monclient(hunting): tick 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e380a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e380a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e38137850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.17:6789/0 conn(0x7f5e380a57d0 legacy=0x7f5e38137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient: _add_conns ranks=[4,3] 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 0x7f5e1c005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): picked mon.noname-c con 0x7f5e1c0036e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): picked mon.noname-a con 0x7f5e1c0061b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5e380a0f00 con 0x7f5e1c0061b0 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c008d30 con 0x7f5e1c0036e0 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e377fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 msgr2=0x7f5e1c005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 0x7f5e1c005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request con 0x7f5e1c0061b0 auth_method 0 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.789+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:31.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 msgr2=0x7f5e1c005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:31.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 0x7f5e1c005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request con 0x7f5e1c0061b0 auth_method 0 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:31.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:31.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:31.993+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:29:32.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 msgr2=0x7f5e1c005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 0x7f5e1c005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request con 0x7f5e1c0061b0 auth_method 0 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:32.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:32.397+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:29:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request con 0x7f5e1c0061b0 auth_method 0 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 msgr2=0x7f5e1c005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:33.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:33.201+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 0x7f5e1c005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:29:34.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): tick 2024-09-15T21:29:34.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:29:34.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 monclient(hunting): continuing hunt 2024-09-15T21:29:34.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:29:34.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 msgr2=0x7f5e1c005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:34.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.119:3300/0 conn(0x7f5e1c0036e0 0x7f5e1c005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 msgr2=0x7f5e1c008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c0061b0 0x7f5e1c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 -- 172.21.15.17:0/3370621918 reap_dead start 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e380a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e1c00e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): picked mon.noname-b con 0x7f5e380a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c003b50 0x7f5e1c006660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): picked mon.noname-a con 0x7f5e1c003b50 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e380a19b0 con 0x7f5e380a53b0 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5e380a0f00 con 0x7f5e1c003b50 2024-09-15T21:29:34.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 1 -- 172.21.15.17:0/3370621918 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c008d30 con 0x7f5e380a4a90 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e377fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3db5d640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e1c00b150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 --2- 172.21.15.17:0/3370621918 >> v2:172.21.15.17:3300/0 conn(0x7f5e1c003b50 0x7f5e1c006660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e1c00e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): get_auth_request con 0x7f5e1c003b50 auth_method 0 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request con 0x7f5e380a4a90 auth_method 0 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:34.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3d35c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_done global_id 144192 payload 274 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 msgr2=0x7f5e1c00e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e1c00e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 -- 172.21.15.17:0/3370621918 >> v1:172.21.15.119:6789/0 conn(0x7f5e380a53b0 legacy=0x7f5e1c00b150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:34.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient: _finish_hunting 0 2024-09-15T21:29:34.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 monclient: found mon.noname-a 2024-09-15T21:29:34.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:34.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:34.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.789+0000 7f5e3e35e640 1 -- 172.21.15.17:0/3370621918 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e38138670 con 0x7f5e1c003b50 2024-09-15T21:29:34.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.793+0000 7f5e3e35e640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e1c003b50 0x7f5e1c006660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e380a15f0 tx=0x7f5e3001bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e88c17a8ec975e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.793+0000 7f5e3cb5b640 1 -- 172.21.15.17:0/3370621918 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30004020 con 0x7f5e1c003b50 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.793+0000 7f5e3cb5b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: dump: 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: _finish_auth 0 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: _check_auth_tickets 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 5 monclient: authenticate success, global_id 144192 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 1 -- 172.21.15.17:0/3370621918 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e300a8070 con 0x7f5e1c003b50 2024-09-15T21:29:34.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 4 set_mon_vals no callback set 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 1 -- 172.21.15.17:0/3370621918 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30004420 con 0x7f5e1c003b50 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3cb5b640 10 monclient: dump: 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.800 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:34.801 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:34.801 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:34.801 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:34.801 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:34.801 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: shutdown 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e1c003b50 msgr2=0x7f5e1c006660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e1c003b50 0x7f5e1c006660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e380a15f0 tx=0x7f5e3001bf70 comp rx=0 tx=0).stop 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/3370621918 shutdown_connections 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e1c003b50 0x7f5e1c006660 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/3370621918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a4a90 0x7f5e1c00e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/3370621918 >> 172.21.15.17:0/3370621918 conn(0x7f5e380aabb0 msgr2=0x7f5e380a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:34.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/3370621918 shutdown_connections 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/3370621918 wait complete. 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: build_initial_monmap 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: monmap: 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:34.806 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 Processor -- start 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- start start 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: init 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c003b50 0x7f5e381cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): picked mon.noname-b con 0x7f5e1c003b50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380a4a90 0x7f5e381d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): picked mon.noname-a con 0x7f5e380a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 0x7f5e381253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): picked mon.noname-c con 0x7f5e380a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:34.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3806f2d0 con 0x7f5e380a4a90 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3806f150 con 0x7f5e1c003b50 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e38138060 con 0x7f5e380a53b0 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.797+0000 7f5e3fdc1640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient(hunting): authenticate will time out at 1698.196016s 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3db5d640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c003b50 msgr2=0x7f5e381cfd70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3db5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c003b50 0x7f5e381cfd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 0x7f5e381253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380a4a90 0x7f5e381d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380a4a90 0x7f5e381d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52732/0 (socket says 172.21.15.17:52732) 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 0x7f5e381253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55410/0 (socket says 172.21.15.17:55410) 2024-09-15T21:29:34.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 1 -- 172.21.15.17:0/126258328 learned_addr learned my addr 172.21.15.17:0/126258328 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request con 0x7f5e380a4a90 auth_method 0 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): get_auth_request con 0x7f5e380a53b0 auth_method 0 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:34.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3d35c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient(hunting): handle_auth_done global_id 134276 payload 824 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 -- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c003b50 msgr2=0x7f5e381cfd70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c003b50 0x7f5e381cfd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 -- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380a4a90 msgr2=0x7f5e381d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380a4a90 0x7f5e381d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient: _finish_hunting 0 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 monclient: found mon.noname-c 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 -- 172.21.15.17:0/126258328 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e380a0080 con 0x7f5e380a53b0 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3e35e640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 0x7f5e381253d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c006180 tx=0x7f5e30018d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:34.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300072f0 con 0x7f5e380a53b0 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: dump: 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:34.811 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: _finish_auth 0 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: _check_auth_tickets 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e30007490 con 0x7f5e380a53b0 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300059f0 con 0x7f5e380a53b0 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 10 monclient: dump: 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 5 monclient: authenticate success, global_id 134276 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient: _renew_subs 2024-09-15T21:29:34.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e37fff640 4 set_mon_vals no callback set 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e380a1280 con 0x7f5e380a53b0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient: _renew_subs 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e38129b50 con 0x7f5e380a53b0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e30005b90 con 0x7f5e380a53b0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e381383f0 con 0x7f5e380a53b0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e1c03cae0 0x7f5e1c03efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5e300e8f20 con 0x7f5e380a53b0 2024-09-15T21:29:34.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e1c03cae0 0x7f5e1c03efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:34.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.801+0000 7f5e3db5d640 10 monclient: get_auth_request con 0x7f5e1c03cae0 auth_method 0 2024-09-15T21:29:34.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.805+0000 7f5e3db5d640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e1c03cae0 0x7f5e1c03efa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009440 tx=0x7f5e2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:34.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.809+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5e300be020 con 0x7f5e380a53b0 2024-09-15T21:29:34.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.809+0000 7f5e367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:34.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:34.809+0000 7f5e367fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:35.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.041+0000 7f5e3fdc1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:35.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.041+0000 7f5e3fdc1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:35.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.041+0000 7f5e3fdc1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.041+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5e38126df0 con 0x7f5e380a53b0 2024-09-15T21:29:35.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e367fc640 1 -- 172.21.15.17:0/126258328 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f5e380a0080 con 0x7f5e380a53b0 2024-09-15T21:29:35.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:35.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e367fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:35.046 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":120,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:35.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e1c03cae0 msgr2=0x7f5e1c03efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e1c03cae0 0x7f5e1c03efa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009440 tx=0x7f5e2c009040 comp rx=0 tx=0).stop 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 10 monclient: shutdown 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 msgr2=0x7f5e381253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 0x7f5e381253d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c006180 tx=0x7f5e30018d70 comp rx=0 tx=0).stop 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 shutdown_connections 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e1c03cae0 0x7f5e1c03efa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e380a53b0 0x7f5e381253d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380a4a90 0x7f5e381d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 --2- 172.21.15.17:0/126258328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c003b50 0x7f5e381cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 >> 172.21.15.17:0/126258328 conn(0x7f5e380aabb0 msgr2=0x7f5e3806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 shutdown_connections 2024-09-15T21:29:35.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.045+0000 7f5e3fdc1640 1 -- 172.21.15.17:0/126258328 wait complete. 2024-09-15T21:29:35.064 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:29:35.065 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:29:35.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:29:35.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.173+0000 7f81dd47a640 10 monclient: get_monmap_and_config 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.173+0000 7f81dd47a640 10 monclient: build_initial_monmap 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.173+0000 7f81dd47a640 10 monclient: monmap: 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:35.178089+0000 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:35.178089+0000 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:35.178 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:35.179 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 Processor -- start 2024-09-15T21:29:35.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 -- start start 2024-09-15T21:29:35.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient: init 2024-09-15T21:29:35.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81d809d400 0x7f81d809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): picked mon.noname-c con 0x7f81d809d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d8134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): picked mon.noname-b con 0x7f81d809dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f81d8134ba0 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d80a3130 con 0x7f81d8134ba0 2024-09-15T21:29:35.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f81d80a1bc0 con 0x7f81d809d400 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f81d80a3490 con 0x7f81d809dd20 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81dd47a640 10 monclient(hunting): authenticate will time out at 1698.573713s 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d8134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d77fe640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f81d8134ba0 legacy=0x7f81d813bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d6ffd640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f81d809d400 msgr2=0x7f81d809d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81d809d400 0x7f81d809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d8134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55424/0 (socket says 172.21.15.17:55424) 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 -- 172.21.15.17:0/1573433247 learned_addr learned my addr 172.21.15.17:0/1573433247 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): get_auth_request con 0x7f81d809dd20 auth_method 0 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient(hunting): handle_auth_done global_id 134279 payload 274 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 -- 172.21.15.17:0/1573433247 >> v2:172.21.15.119:3300/0 conn(0x7f81d809d400 msgr2=0x7f81d809d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 --2- 172.21.15.17:0/1573433247 >> v2:172.21.15.119:3300/0 conn(0x7f81d809d400 0x7f81d809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 -- 172.21.15.17:0/1573433247 >> v1:172.21.15.119:6789/0 conn(0x7f81d8134ba0 legacy=0x7f81d813bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 monclient: found mon.noname-b 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:35.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.177+0000 7f81d67fc640 1 -- 172.21.15.17:0/1573433247 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d809e7a0 con 0x7f81d809dd20 2024-09-15T21:29:35.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 --2- 172.21.15.17:0/1573433247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d8134660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81c000d150 tx=0x7f81c001eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b1e1e76804d3849 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 1 -- 172.21.15.17:0/1573433247 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c000b2d0 con 0x7f81d809dd20 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: dump: 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.191 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 1 -- 172.21.15.17:0/1573433247 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c000b470 con 0x7f81d809dd20 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 5 monclient: authenticate success, global_id 134279 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 1 -- 172.21.15.17:0/1573433247 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c00a7880 con 0x7f81d809dd20 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 4 set_mon_vals no callback set 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d5ffb640 10 monclient: dump: 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: shutdown 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/1573433247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 msgr2=0x7f81d8134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- 172.21.15.17:0/1573433247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d8134660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81c000d150 tx=0x7f81c001eb10 comp rx=0 tx=0).stop 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/1573433247 shutdown_connections 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- 172.21.15.17:0/1573433247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d8134660 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- 172.21.15.17:0/1573433247 >> v2:172.21.15.119:3300/0 conn(0x7f81d809d400 0x7f81d809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/1573433247 >> 172.21.15.17:0/1573433247 conn(0x7f81d80a66a0 msgr2=0x7f81d80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/1573433247 shutdown_connections 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/1573433247 wait complete. 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: build_initial_monmap 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: monmap: 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:35.194 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 Processor -- start 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- start start 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: init 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:35.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d809d400 0x7f81d81d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): picked mon.noname-a con 0x7f81d809d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d81d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): picked mon.noname-c con 0x7f81d809dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81d8134ba0 0x7f81d813ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): picked mon.noname-b con 0x7f81d8134ba0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81d809ce50 con 0x7f81d809d400 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f81d809ccd0 con 0x7f81d8134ba0 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f81d809fa20 con 0x7f81d809dd20 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): authenticate will time out at 1698.578283s 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d809d400 0x7f81d81d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d81d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d809d400 0x7f81d81d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52738/0 (socket says 172.21.15.17:52738) 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 1 -- 172.21.15.17:0/2615060565 learned_addr learned my addr 172.21.15.17:0/2615060565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d81d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55436/0 (socket says 172.21.15.17:55436) 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): get_auth_request con 0x7f81d809d400 auth_method 0 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): get_auth_request con 0x7f81d809dd20 auth_method 0 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d77fe640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81d8134ba0 msgr2=0x7f81d813ad10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d77fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81d8134ba0 0x7f81d813ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient(hunting): handle_auth_done global_id 134282 payload 824 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81d8134ba0 msgr2=0x7f81d813ad10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81d8134ba0 0x7f81d813ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d809d400 msgr2=0x7f81d81d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d809d400 0x7f81d81d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient: _finish_hunting 0 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 monclient: found mon.noname-c 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d8121e80 con 0x7f81d809dd20 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d809d400 0x7f81d81d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d67fc640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d81d0760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81c00ad000 tx=0x7f81c001b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0009600 con 0x7f81d809dd20 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: dump: 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:35.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 5 monclient: authenticate success, global_id 134282 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c00097e0 con 0x7f81d809dd20 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: _renew_subs 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d8122160 con 0x7f81d809dd20 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0009bd0 con 0x7f81d809dd20 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81cf7fe640 10 monclient: dump: 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d57fa640 4 set_mon_vals no callback set 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: _renew_subs 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d8121ba0 con 0x7f81d809dd20 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:35.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 0x7f81d802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81208f0 con 0x7f81d813ba10 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 0x7f81d802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient: get_auth_request con 0x7f81d813ba10 auth_method 0 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.181+0000 7f81d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f81c00c0020 con 0x7f81d809dd20 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_done global_id 144201 payload 824 2024-09-15T21:29:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d6ffd640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81d8073d60 con 0x7f81d813ba10 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81b803b970 0x7f81b803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f81c00eb2a0 con 0x7f81d809dd20 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d6ffd640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 0x7f81d802d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81d0005320 tx=0x7f81d00057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0016070 con 0x7f81d813ba10 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 10 monclient: dump: 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.204 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d77fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81b803b970 0x7f81b803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d77fe640 10 monclient: get_auth_request con 0x7f81b803b970 auth_method 0 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f81d000eaa0 con 0x7f81d813ba10 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 msgr2=0x7f81d802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 0x7f81d802d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81d0005320 tx=0x7f81d00057e0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 msgr2=0x7f81d802d450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81cf7fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813ba10 0x7f81d802d450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.185+0000 7f81d77fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81b803b970 0x7f81b803de30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f81c8003940 tx=0x7f81c8006c70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81dd47a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81dd47a640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 0x7f81d802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81dd47a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81371a0 con 0x7f81d813bdf0 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 0x7f81d802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient: get_auth_request con 0x7f81d813bdf0 auth_method 0 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 10 monclient(hunting): handle_auth_done global_id 144204 payload 824 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 1 -- 172.21.15.17:0/2615060565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f81d806d020 con 0x7f81d813bdf0 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81d6ffd640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 0x7f81d802d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f81d0005320 tx=0x7f81d0005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0016040 con 0x7f81d813bdf0 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 10 monclient: dump: 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4870 (secure 0 0 0) 0x7f81d00124d0 con 0x7f81d813bdf0 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 msgr2=0x7f81d802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 0x7f81d802d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f81d0005320 tx=0x7f81d0005980 comp rx=0 tx=0).stop 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 msgr2=0x7f81d802d990 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:35.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.189+0000 7f81cf7fe640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d813bdf0 0x7f81d802d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 120, 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 46915, 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:29:35.211 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:35.212 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:35.213 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:35.214 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:35.224 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81b803b970 msgr2=0x7f81b803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f81b803b970 0x7f81b803de30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f81c8003940 tx=0x7f81c8006c70 comp rx=0 tx=0).stop 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 10 monclient: shutdown 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 msgr2=0x7f81d81d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 --2- 172.21.15.17:0/2615060565 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f81d809dd20 0x7f81d81d0760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81c00ad000 tx=0x7f81c001b8b0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81d77fe640 1 -- 172.21.15.17:0/2615060565 reap_dead start 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81d77fe640 1 -- 172.21.15.17:0/2615060565 reap_dead start 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 shutdown_connections 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 >> 172.21.15.17:0/2615060565 conn(0x7f81d80a66a0 msgr2=0x7f81d80a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 shutdown_connections 2024-09-15T21:29:35.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.193+0000 7f81dd47a640 1 -- 172.21.15.17:0/2615060565 wait complete. 2024-09-15T21:29:35.227 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:29:35.227 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:29:35.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient: get_monmap_and_config 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient: build_initial_monmap 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient: monmap: 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:35.335350+0000 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:35.335350+0000 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.335 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:35.336 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:35.336 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:35.336 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:35.336 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:35.336 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 Processor -- start 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 -- start start 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient: init 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f140c1378c0 0x7f140c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): picked mon.noname-c con 0x7f140c1378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): picked mon.noname-b con 0x7f140c1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f140c0a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f140c0a1d00 con 0x7f140c0a5040 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f140c0a1fe0 con 0x7f140c1378c0 2024-09-15T21:29:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f140c0a22c0 con 0x7f140c1381e0 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141356f640 10 monclient(hunting): authenticate will time out at 1698.731106s 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1411b0c640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f140c0a5040 legacy=0x7f140c0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141130b640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f140c1378c0 msgr2=0x7f140c137ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f141130b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f140c1378c0 0x7f140c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c0a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55442/0 (socket says 172.21.15.17:55442) 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 -- 172.21.15.17:0/2039661501 learned_addr learned my addr 172.21.15.17:0/2039661501 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): get_auth_request con 0x7f140c1381e0 auth_method 0 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient(hunting): handle_auth_done global_id 134285 payload 274 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 -- 172.21.15.17:0/2039661501 >> v2:172.21.15.119:3300/0 conn(0x7f140c1378c0 msgr2=0x7f140c137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 --2- 172.21.15.17:0/2039661501 >> v2:172.21.15.119:3300/0 conn(0x7f140c1378c0 0x7f140c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 -- 172.21.15.17:0/2039661501 >> v1:172.21.15.119:6789/0 conn(0x7f140c0a5040 legacy=0x7f140c0a54f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient: _finish_hunting 0 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 monclient: found mon.noname-b 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.333+0000 7f1410b0a640 1 -- 172.21.15.17:0/2039661501 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c0a5c00 con 0x7f140c1381e0 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1410b0a640 1 --2- 172.21.15.17:0/2039661501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c0a4b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009870 tx=0x7f13fc01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5515198668094 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 1 -- 172.21.15.17:0/2039661501 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0b0070 con 0x7f140c1381e0 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: dump: 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.347 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: _finish_auth 0 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: _check_auth_tickets 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 1 -- 172.21.15.17:0/2039661501 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc004230 con 0x7f140c1381e0 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 1 -- 172.21.15.17:0/2039661501 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc003620 con 0x7f140c1381e0 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f1403fff640 10 monclient: dump: 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 5 monclient: authenticate success, global_id 134285 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 4 set_mon_vals no callback set 2024-09-15T21:29:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 10 monclient: shutdown 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 -- 172.21.15.17:0/2039661501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 msgr2=0x7f140c0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 --2- 172.21.15.17:0/2039661501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c0a4b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009870 tx=0x7f13fc01c440 comp rx=0 tx=0).stop 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 -- 172.21.15.17:0/2039661501 shutdown_connections 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 --2- 172.21.15.17:0/2039661501 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c0a4b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 --2- 172.21.15.17:0/2039661501 >> v2:172.21.15.119:3300/0 conn(0x7f140c1378c0 0x7f140c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 -- 172.21.15.17:0/2039661501 >> 172.21.15.17:0/2039661501 conn(0x7f140c0aac40 msgr2=0x7f140c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 -- 172.21.15.17:0/2039661501 shutdown_connections 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.337+0000 7f141356f640 1 -- 172.21.15.17:0/2039661501 wait complete. 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: build_initial_monmap 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: monmap: 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 Processor -- start 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- start start 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: init 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f140c0a5040 0x7f140c1cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): picked mon.noname-b con 0x7f140c0a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f140c1378c0 0x7f140c136be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): picked mon.noname-a con 0x7f140c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c13ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): picked mon.noname-c con 0x7f140c1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1425c0 con 0x7f140c1378c0 2024-09-15T21:29:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f140c138700 con 0x7f140c0a5040 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f140c142740 con 0x7f140c1381e0 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): authenticate will time out at 1698.737254s 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f140c1378c0 0x7f140c136be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f140c0a5040 msgr2=0x7f140c1cfce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c13ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f140c0a5040 0x7f140c1cfce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f140c1378c0 0x7f140c136be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52768/0 (socket says 172.21.15.17:52768) 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): get_auth_request con 0x7f140c1381e0 auth_method 0 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 1 -- 172.21.15.17:0/1833700804 learned_addr learned my addr 172.21.15.17:0/1833700804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): get_auth_request con 0x7f140c1378c0 auth_method 0 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient(hunting): handle_auth_done global_id 134288 payload 824 2024-09-15T21:29:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f140c0a5040 msgr2=0x7f140c1cfce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f140c0a5040 0x7f140c1cfce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f140c1378c0 msgr2=0x7f140c136be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f140c1378c0 0x7f140c136be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient: _finish_hunting 0 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 monclient: found mon.noname-c 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c121670 con 0x7f140c1381e0 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f140c1378c0 0x7f140c136be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1411b0c640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c13ce90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f140400ba60 tx=0x7f140400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140400cbe0 con 0x7f140c1381e0 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: dump: 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1404010070 con 0x7f140c1381e0 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404015470 con 0x7f140c1381e0 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 5 monclient: authenticate success, global_id 134288 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: _renew_subs 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c0a12f0 con 0x7f140c1381e0 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f14037fe640 4 set_mon_vals no callback set 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: dump: 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: _renew_subs 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c120dd0 con 0x7f140c1381e0 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 0x7f140c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f140c137ca0 con 0x7f140c137120 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 0x7f140c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient: get_auth_request con 0x7f140c137120 auth_method 0 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1404015610 con 0x7f140c1381e0 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 10 monclient(hunting): handle_auth_done global_id 134291 payload 824 2024-09-15T21:29:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f140c13f510 con 0x7f140c137120 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13f003b970 0x7f13f003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f14040506e0 con 0x7f140c1381e0 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f141130b640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 0x7f140c13d3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f140800b450 tx=0x7f140800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408003e20 con 0x7f140c137120 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: dump: 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13f003b970 0x7f13f003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1410b0a640 10 monclient: get_auth_request con 0x7f13f003b970 auth_method 0 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1408015070 con 0x7f140c137120 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 msgr2=0x7f140c13d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 0x7f140c13d3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f140800b450 tx=0x7f140800b910 comp rx=0 tx=0).stop 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 msgr2=0x7f140c13d3d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.341+0000 7f1401ffb640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c137120 0x7f140c13d3d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.345+0000 7f1410b0a640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13f003b970 0x7f13f003de30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000c10 tx=0x7f13fc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:29:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 0x7f140c138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f140c13dd20 con 0x7f140c13d910 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 0x7f140c138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient: get_auth_request con 0x7f140c13d910 auth_method 0 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 10 monclient(hunting): handle_auth_done global_id 134294 payload 824 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 1 -- 172.21.15.17:0/1833700804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f140c069270 con 0x7f140c13d910 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141130b640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 0x7f140c138d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f140800b450 tx=0x7f1408005a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408010480 con 0x7f140c13d910 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 10 monclient: dump: 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7f1408010480 con 0x7f140c13d910 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 msgr2=0x7f140c138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 0x7f140c138d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f140800b450 tx=0x7f1408005a30 comp rx=0 tx=0).stop 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 msgr2=0x7f140c138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1401ffb640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c13d910 0x7f140c138d60 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f140800b450 tx=0x7f1408005a30 comp rx=0 tx=0).stop 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:35.366 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 120, 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 92503, 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:29:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:29:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:29:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:29:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:29:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:29:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:29:35.380 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:29:35.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13f003b970 msgr2=0x7f13f003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f13f003b970 0x7f13f003de30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000c10 tx=0x7f13fc0a6000 comp rx=0 tx=0).stop 2024-09-15T21:29:35.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 10 monclient: shutdown 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 msgr2=0x7f140c13ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f1411b0c640 1 -- 172.21.15.17:0/1833700804 reap_dead start 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c13ce90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f140400ba60 tx=0x7f140400bf20 comp rx=0 tx=0).stop 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 shutdown_connections 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 --2- 172.21.15.17:0/1833700804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f140c1381e0 0x7f140c13ce90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 >> 172.21.15.17:0/1833700804 conn(0x7f140c0aac40 msgr2=0x7f140c0a8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 shutdown_connections 2024-09-15T21:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:35.349+0000 7f141356f640 1 -- 172.21.15.17:0/1833700804 wait complete. 2024-09-15T21:29:35.382 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:29:35.383 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:29:55.384 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:29:55.384 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:29:55.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:29:55.387 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:29:55.387 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:29:55.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient: get_monmap_and_config 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient: build_initial_monmap 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient: monmap: 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:55.520331+0000 2024-09-15T21:29:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:55.520331+0000 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:55.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 Processor -- start 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 -- start start 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient: init 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbfd00a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 0x7fbfd013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): picked mon.noname-c con 0x7fbfd009ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 0x7fbfd013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): picked mon.noname-b con 0x7fbfd00a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd00a3130 con 0x7fbfd00a0890 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd0073df0 con 0x7fbfd009ffc0 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd00a1bf0 con 0x7fbfd00a1380 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfd56ae640 10 monclient(hunting): authenticate will time out at 1718.915794s 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.516+0000 7fbfcf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 0x7fbfd013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 0x7fbfd013d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45572/0 (socket says 172.21.15.17:45572) 2024-09-15T21:29:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 -- 172.21.15.17:0/1198896338 learned_addr learned my addr 172.21.15.17:0/1198896338 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): get_auth_request con 0x7fbfd00a1380 auth_method 0 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 1 --2- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 0x7fbfd013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 10 monclient(hunting): get_auth_request con 0x7fbfd009ffc0 auth_method 0 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 1 -- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 msgr2=0x7fbfd013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 1 -- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 msgr2=0x7fbfd013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 1 --2- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 0x7fbfd013aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient(hunting): handle_auth_done global_id 134297 payload 274 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfce7fc640 1 --2- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 0x7fbfd013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 -- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 msgr2=0x7fbfd013aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 --2- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 0x7fbfd013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 -- 172.21.15.17:0/1198896338 >> v1:172.21.15.119:6789/0 conn(0x7fbfd00a0890 legacy=0x7fbfd00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 monclient: found mon.noname-b 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 -- 172.21.15.17:0/1198896338 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd009ccd0 con 0x7fbfd00a1380 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcf7fe640 1 --2- 172.21.15.17:0/1198896338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 0x7fbfd013d010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80098a0 tx=0x7fbfc801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a35f3669f2606bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 1 -- 172.21.15.17:0/1198896338 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80b1070 con 0x7fbfd00a1380 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: dump: 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 1 -- 172.21.15.17:0/1198896338 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc80ac640 con 0x7fbfd00a1380 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 1 -- 172.21.15.17:0/1198896338 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80aca10 con 0x7fbfd00a1380 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 5 monclient: authenticate success, global_id 134297 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 4 set_mon_vals no callback set 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfcdffb640 10 monclient: dump: 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:55.531 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 10 monclient: shutdown 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 -- 172.21.15.17:0/1198896338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 msgr2=0x7fbfd013d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/1198896338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 0x7fbfd013d010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80098a0 tx=0x7fbfc801c400 comp rx=0 tx=0).stop 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 -- 172.21.15.17:0/1198896338 shutdown_connections 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/1198896338 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd00a1380 0x7fbfd013d010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/1198896338 >> v2:172.21.15.119:3300/0 conn(0x7fbfd009ffc0 0x7fbfd013aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 -- 172.21.15.17:0/1198896338 >> 172.21.15.17:0/1198896338 conn(0x7fbfd00a66a0 msgr2=0x7fbfd00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 -- 172.21.15.17:0/1198896338 shutdown_connections 2024-09-15T21:29:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 1 -- 172.21.15.17:0/1198896338 wait complete. 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.520+0000 7fbfd56ae640 10 monclient: build_initial_monmap 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: monmap: 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 Processor -- start 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- start start 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: init 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 0x7fbfd01cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): picked mon.noname-c con 0x7fbfd009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00a0890 0x7fbfd0135b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): picked mon.noname-a con 0x7fbfd00a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 0x7fbfd01360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): picked mon.noname-b con 0x7fbfd00a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0140100 con 0x7fbfd00a0890 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd013ff80 con 0x7fbfd00a1380 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd0140280 con 0x7fbfd009ffc0 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient(hunting): authenticate will time out at 1718.920879s 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 0x7fbfd01cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00a0890 0x7fbfd0135b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 0x7fbfd01cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45584/0 (socket says 172.21.15.17:45584) 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 -- 172.21.15.17:0/470950739 learned_addr learned my addr 172.21.15.17:0/470950739 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): get_auth_request con 0x7fbfd009ffc0 auth_method 0 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): get_auth_request con 0x7fbfd00a0890 auth_method 0 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 0x7fbfd01360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 10 monclient(hunting): get_auth_request con 0x7fbfd00a1380 auth_method 0 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 1 -- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 msgr2=0x7fbfd01360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 1 -- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 msgr2=0x7fbfd01360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 0x7fbfd01360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient(hunting): handle_auth_done global_id 134300 payload 824 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcf7fe640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 0x7fbfd01360c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 -- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 msgr2=0x7fbfd01360c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 0x7fbfd01360c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 -- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00a0890 msgr2=0x7fbfd0135b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00a0890 0x7fbfd0135b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient: _finish_hunting 0 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 monclient: found mon.noname-c 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 -- 172.21.15.17:0/470950739 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd0122530 con 0x7fbfd009ffc0 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00a0890 0x7fbfd0135b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfceffd640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 0x7fbfd01cf4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c500 tx=0x7fbfc000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0017070 con 0x7fbfd009ffc0 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: dump: 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: _finish_auth 0 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc00052d0 con 0x7fbfd009ffc0 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0012870 con 0x7fbfd009ffc0 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 10 monclient: dump: 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 5 monclient: authenticate success, global_id 134300 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _renew_subs 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd006f3e0 con 0x7fbfd009ffc0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfcd7fa640 4 set_mon_vals no callback set 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _renew_subs 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd0122250 con 0x7fbfd009ffc0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd00a0c70 con 0x7fbfd009ffc0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbfc001a070 con 0x7fbfd009ffc0 2024-09-15T21:29:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfb003b9c0 0x7fbfb003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fbfc004f910 con 0x7fbfd009ffc0 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfb003b9c0 0x7fbfb003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.524+0000 7fbfce7fc640 10 monclient: get_auth_request con 0x7fbfb003b9c0 auth_method 0 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.528+0000 7fbfce7fc640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfb003b9c0 0x7fbfb003de80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0120a30 tx=0x7fbfb8009900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.528+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbfc0005480 con 0x7fbfd009ffc0 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.528+0000 7fbfc77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.528+0000 7fbfc77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:55.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfd56ae640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfd56ae640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfd56ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:29:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbfd0121e00 con 0x7fbfd009ffc0 2024-09-15T21:29:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfc77fe640 1 -- 172.21.15.17:0/470950739 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fbfc004f6e0 con 0x7fbfd009ffc0 2024-09-15T21:29:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfc77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.760+0000 7fbfc77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:55.766 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":120,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfb003b9c0 msgr2=0x7fbfb003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfb003b9c0 0x7fbfb003de80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0120a30 tx=0x7fbfb8009900 comp rx=0 tx=0).stop 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 10 monclient: shutdown 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 msgr2=0x7fbfd01cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 0x7fbfd01cf4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c500 tx=0x7fbfc000c9c0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 shutdown_connections 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfb003b9c0 0x7fbfb003de80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfd00a1380 0x7fbfd01360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00a0890 0x7fbfd0135b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 --2- 172.21.15.17:0/470950739 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbfd009ffc0 0x7fbfd01cf4a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 >> 172.21.15.17:0/470950739 conn(0x7fbfd00a66a0 msgr2=0x7fbfd00a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 shutdown_connections 2024-09-15T21:29:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:55.764+0000 7fbfd56ae640 1 -- 172.21.15.17:0/470950739 wait complete. 2024-09-15T21:29:58.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:29:58.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient: get_monmap_and_config 2024-09-15T21:29:58.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient: build_initial_monmap 2024-09-15T21:29:58.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient: monmap: 2024-09-15T21:29:58.912 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:58.912 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:29:58.912387+0000 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:29:58.912387+0000 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:29:58.913 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 Processor -- start 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 -- start start 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient: init 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcfb40a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 0x7fcfb413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): picked mon.noname-c con 0x7fcfb409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcfb40a1380 0x7fcfb413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): picked mon.noname-a con 0x7fcfb40a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb40a3130 con 0x7fcfb40a0890 2024-09-15T21:29:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb4073df0 con 0x7fcfb40a1380 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb40a1bf0 con 0x7fcfb409ffc0 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.908+0000 7fcfb9922640 10 monclient(hunting): authenticate will time out at 1722.307988s 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcfb40a1380 0x7fcfb413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcfb40a0890 0x7fcfb40a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35724/0 (socket says 172.21.15.17:35724) 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb2ffd640 1 -- 172.21.15.17:0/1283267880 learned_addr learned my addr 172.21.15.17:0/1283267880 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request con 0x7fcfb40a1380 auth_method 0 2024-09-15T21:29:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 1 --2- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 0x7fcfb413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 10 monclient(hunting): get_auth_request con 0x7fcfb409ffc0 auth_method 0 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_done global_id 144213 payload 274 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 -- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 msgr2=0x7fcfb413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 1 -- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 msgr2=0x7fcfb413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 1 -- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 msgr2=0x7fcfb413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 1 --2- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 0x7fcfb413aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb27fc640 1 --2- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 0x7fcfb413aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 --2- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 0x7fcfb413aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 -- 172.21.15.17:0/1283267880 >> v1:172.21.15.17:6789/0 conn(0x7fcfb40a0890 legacy=0x7fcfb40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 1 -- 172.21.15.17:0/1283267880 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886030001 0 0) 0x7fcfb40a3130 con 0x7fcfb40a0890 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:29:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 monclient: found mon.noname-a 2024-09-15T21:29:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 -- 172.21.15.17:0/1283267880 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb409ccd0 con 0x7fcfb40a1380 2024-09-15T21:29:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:29:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb37fe640 1 --2- 172.21.15.17:0/1283267880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb413d010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40097c0 tx=0x7fcfa401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1567d594d24f53ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 1 -- 172.21.15.17:0/1283267880 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40b0070 con 0x7fcfb40a1380 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: dump: 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 1 -- 172.21.15.17:0/1283267880 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa40ab600 con 0x7fcfb40a1380 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 5 monclient: authenticate success, global_id 144213 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:58.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 10 monclient: get_monmap_and_config success 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 4 set_mon_vals no callback set 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 1 -- 172.21.15.17:0/1283267880 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40ab9d0 con 0x7fcfb40a1380 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb1ffb640 10 monclient: dump: 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 10 monclient: shutdown 2024-09-15T21:29:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 -- 172.21.15.17:0/1283267880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 msgr2=0x7fcfb413d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 --2- 172.21.15.17:0/1283267880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb413d010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40097c0 tx=0x7fcfa401c320 comp rx=0 tx=0).stop 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 -- 172.21.15.17:0/1283267880 shutdown_connections 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 --2- 172.21.15.17:0/1283267880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb413d010 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 --2- 172.21.15.17:0/1283267880 >> v2:172.21.15.119:3300/0 conn(0x7fcfb409ffc0 0x7fcfb413aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 -- 172.21.15.17:0/1283267880 >> 172.21.15.17:0/1283267880 conn(0x7fcfb40a66a0 msgr2=0x7fcfb40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 -- 172.21.15.17:0/1283267880 shutdown_connections 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.912+0000 7fcfb9922640 1 -- 172.21.15.17:0/1283267880 wait complete. 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: build_initial_monmap 2024-09-15T21:29:58.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: monmap: 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 Processor -- start 2024-09-15T21:29:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- start start 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: init 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _reopen_session rank -1 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcfb409ffc0 0x7fcfb41cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): picked mon.noname-c con 0x7fcfb409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb40a0890 0x7fcfb4135920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): picked mon.noname-b con 0x7fcfb40a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb4135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): picked mon.noname-a con 0x7fcfb40a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4140100 con 0x7fcfb40a1380 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb413ff80 con 0x7fcfb40a0890 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): start opening mon connection 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb4140280 con 0x7fcfb409ffc0 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): _renew_subs 2024-09-15T21:29:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient(hunting): authenticate will time out at 1722.313249s 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb4135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb4135e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53874/0 (socket says 172.21.15.17:53874) 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 -- 172.21.15.17:0/3627248857 learned_addr learned my addr 172.21.15.17:0/3627248857 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request con 0x7fcfb40a1380 auth_method 0 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcfb409ffc0 0x7fcfb41cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:58.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): get_auth_request con 0x7fcfb409ffc0 auth_method 0 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb27fc640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb40a0890 0x7fcfb4135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb27fc640 10 monclient(hunting): get_auth_request con 0x7fcfb40a0890 auth_method 0 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:29:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient(hunting): handle_auth_done global_id 144219 payload 824 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 -- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcfb409ffc0 msgr2=0x7fcfb41cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcfb409ffc0 0x7fcfb41cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 -- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb40a0890 msgr2=0x7fcfb4135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb40a0890 0x7fcfb4135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 monclient: found mon.noname-a 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:29:58.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 -- 172.21.15.17:0/3627248857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb4121d30 con 0x7fcfb40a1380 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb37fe640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb4135e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4018f30 tx=0x7fcfa40190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40b8020 con 0x7fcfb40a1380 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: dump: 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: _finish_auth 0 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: _check_auth_tickets 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa40aa6d0 con 0x7fcfb40a1380 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 5 monclient: authenticate success, global_id 144219 2024-09-15T21:29:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _renew_subs 2024-09-15T21:29:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb4122010 con 0x7fcfb40a1380 2024-09-15T21:29:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:29:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40b0040 con 0x7fcfb40a1380 2024-09-15T21:29:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:29:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 10 monclient: dump: 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb17fa640 4 set_mon_vals no callback set 2024-09-15T21:29:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _renew_subs 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41222f0 con 0x7fcfb40a1380 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcfa40bb020 con 0x7fcfb40a1380 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb40a0c70 con 0x7fcfb40a1380 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf9403b9c0 0x7fcf9403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fcfa40ea730 con 0x7fcfb40a1380 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf9403b9c0 0x7fcf9403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.916+0000 7fcfb2ffd640 10 monclient: get_auth_request con 0x7fcf9403b9c0 auth_method 0 2024-09-15T21:29:58.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.920+0000 7fcfb2ffd640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf9403b9c0 0x7fcf9403de80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0012f0 tx=0x7fcfac001700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:29:58.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.924+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fcfa40c2030 con 0x7fcfb40a1380 2024-09-15T21:29:58.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.924+0000 7fcfabfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:29:58.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:58.924+0000 7fcfabfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:29:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfb9922640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:29:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfb9922640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfb9922640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:29:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcfb406f650 con 0x7fcfb40a1380 2024-09-15T21:29:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfabfff640 1 -- 172.21.15.17:0/3627248857 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fcfa4016030 con 0x7fcfb40a1380 2024-09-15T21:29:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfabfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:29:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.152+0000 7fcfabfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:29:59.158 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":120,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:29:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf9403b9c0 msgr2=0x7fcf9403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf9403b9c0 0x7fcf9403de80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0012f0 tx=0x7fcfac001700 comp rx=0 tx=0).stop 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 10 monclient: shutdown 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 msgr2=0x7fcfb4135e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb4135e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4018f30 tx=0x7fcfa40190c0 comp rx=0 tx=0).stop 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 shutdown_connections 2024-09-15T21:29:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcf9403b9c0 0x7fcf9403de80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfb40a1380 0x7fcfb4135e60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb40a0890 0x7fcfb4135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 --2- 172.21.15.17:0/3627248857 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcfb409ffc0 0x7fcfb41cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:29:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 >> 172.21.15.17:0/3627248857 conn(0x7fcfb40a66a0 msgr2=0x7fcfb40a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:29:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 shutdown_connections 2024-09-15T21:29:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:29:59.156+0000 7fcfb9922640 1 -- 172.21.15.17:0/3627248857 wait complete. 2024-09-15T21:30:02.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient: get_monmap_and_config 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient: build_initial_monmap 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient: monmap: 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:02.308013+0000 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:02.308013+0000 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:02.309 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 1 Processor -- start 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 1 -- start start 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient: init 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3c48073640 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3c48074130 0x7f3c4813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.304+0000 7f3c4d619640 10 monclient(hunting): picked mon.noname-a con 0x7f3c48074130 addr v2:172.21.15.17:3300/0 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 0x7f3c4813f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient(hunting): picked mon.noname-b con 0x7f3c4813d140 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c480a0890 con 0x7f3c48073640 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3c480746e0 con 0x7f3c48074130 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3c480a1640 con 0x7f3c4813d140 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient(hunting): authenticate will time out at 1725.704068s 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 0x7f3c4813f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 0x7f3c4813f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39160/0 (socket says 172.21.15.17:39160) 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 -- 172.21.15.17:0/1856069263 learned_addr learned my addr 172.21.15.17:0/1856069263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 1 --2- 172.21.15.17:0/1856069263 >> v2:172.21.15.17:3300/0 conn(0x7f3c48074130 0x7f3c4813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): get_auth_request con 0x7f3c4813d140 auth_method 0 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): get_auth_request con 0x7f3c48074130 auth_method 0 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c3ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient(hunting): handle_auth_done global_id 134306 payload 274 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 -- 172.21.15.17:0/1856069263 >> v2:172.21.15.17:3300/0 conn(0x7f3c48074130 msgr2=0x7f3c4813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 --2- 172.21.15.17:0/1856069263 >> v2:172.21.15.17:3300/0 conn(0x7f3c48074130 0x7f3c4813cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 -- 172.21.15.17:0/1856069263 >> v1:172.21.15.119:6789/0 conn(0x7f3c48073640 legacy=0x7f3c48073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient: _finish_hunting 0 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 monclient: found mon.noname-b 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 -- 172.21.15.17:0/1856069263 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480a1920 con 0x7f3c4813d140 2024-09-15T21:30:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c477fe640 1 --2- 172.21.15.17:0/1856069263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 0x7f3c4813f530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009e50 tx=0x7f3c4001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e363a08fb60c8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 1 -- 172.21.15.17:0/1856069263 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400a57a0 con 0x7f3c4813d140 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: dump: 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: _finish_auth 0 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: _check_auth_tickets 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 1 -- 172.21.15.17:0/1856069263 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c400a5940 con 0x7f3c4813d140 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 1 -- 172.21.15.17:0/1856069263 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400a5d10 con 0x7f3c4813d140 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 5 monclient: authenticate success, global_id 134306 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 4 set_mon_vals no callback set 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c467fc640 10 monclient: dump: 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 10 monclient: shutdown 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- 172.21.15.17:0/1856069263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 msgr2=0x7f3c4813f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 --2- 172.21.15.17:0/1856069263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 0x7f3c4813f530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009e50 tx=0x7f3c4001c8c0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- 172.21.15.17:0/1856069263 shutdown_connections 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 --2- 172.21.15.17:0/1856069263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c4813d140 0x7f3c4813f530 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 --2- 172.21.15.17:0/1856069263 >> v2:172.21.15.17:3300/0 conn(0x7f3c48074130 0x7f3c4813cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- 172.21.15.17:0/1856069263 >> 172.21.15.17:0/1856069263 conn(0x7f3c480a66a0 msgr2=0x7f3c480a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- 172.21.15.17:0/1856069263 shutdown_connections 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.308+0000 7f3c4d619640 1 -- 172.21.15.17:0/1856069263 wait complete. 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: build_initial_monmap 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: monmap: 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 Processor -- start 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- start start 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: init 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 0x7f3c48120c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): picked mon.noname-c con 0x7f3c48073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c48074130 0x7f3c4811eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): picked mon.noname-b con 0x7f3c48074130 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c4813d140 0x7f3c48137ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): picked mon.noname-a con 0x7f3c4813d140 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48142410 con 0x7f3c4813d140 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48142290 con 0x7f3c48074130 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3c48142590 con 0x7f3c48073640 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient(hunting): authenticate will time out at 1725.709394s 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 0x7f3c48120c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c4813d140 0x7f3c48137ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 0x7f3c48120c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39174/0 (socket says 172.21.15.17:39174) 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 -- 172.21.15.17:0/2520618223 learned_addr learned my addr 172.21.15.17:0/2520618223 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): get_auth_request con 0x7f3c48073640 auth_method 0 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): get_auth_request con 0x7f3c4813d140 auth_method 0 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3ffff640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c48074130 0x7f3c4811eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3ffff640 10 monclient(hunting): get_auth_request con 0x7f3c48074130 auth_method 0 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient(hunting): handle_auth_done global_id 134309 payload 824 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 -- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c48074130 msgr2=0x7f3c4811eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c48074130 0x7f3c4811eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 -- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c4813d140 msgr2=0x7f3c48137ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c4813d140 0x7f3c48137ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 monclient: found mon.noname-c 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 -- 172.21.15.17:0/2520618223 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c48138770 con 0x7f3c48073640 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c46ffd640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 0x7f3c48120c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c530 tx=0x7f3c3800c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38017070 con 0x7f3c48073640 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: dump: 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c380052d0 con 0x7f3c48073640 2024-09-15T21:30:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38012870 con 0x7f3c48073640 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 10 monclient: dump: 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c45ffb640 4 set_mon_vals no callback set 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 5 monclient: authenticate success, global_id 134309 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _renew_subs 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4811fac0 con 0x7f3c48073640 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _renew_subs 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c48121bc0 con 0x7f3c48073640 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c48073a20 con 0x7f3c48073640 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.312+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3c3801a070 con 0x7f3c48073640 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.316+0000 7f3c3f7fe640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c2803b9c0 0x7f3c2803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.316+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3c3804f880 con 0x7f3c48073640 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.316+0000 7f3c3ffff640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c2803b9c0 0x7f3c2803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.316+0000 7f3c3ffff640 10 monclient: get_auth_request con 0x7f3c2803b9c0 auth_method 0 2024-09-15T21:30:02.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.316+0000 7f3c3ffff640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c2803b9c0 0x7f3c2803de80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3c48121710 tx=0x7f3c30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.320+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3c38003d60 con 0x7f3c48073640 2024-09-15T21:30:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.320+0000 7f3c3f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.320+0000 7f3c3f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:02.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c4d619640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:30:02.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c4d619640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:30:02.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c4d619640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3c48120100 con 0x7f3c48073640 2024-09-15T21:30:02.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c3f7fe640 1 -- 172.21.15.17:0/2520618223 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f3c38019cd0 con 0x7f3c48073640 2024-09-15T21:30:02.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c3f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:30:02.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.556+0000 7f3c3f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:02.561 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:30:02.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c2803b9c0 msgr2=0x7f3c2803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c2803b9c0 0x7f3c2803de80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3c48121710 tx=0x7f3c30008040 comp rx=0 tx=0).stop 2024-09-15T21:30:02.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 10 monclient: shutdown 2024-09-15T21:30:02.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 msgr2=0x7f3c48120c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 0x7f3c48120c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c530 tx=0x7f3c3800c9f0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 shutdown_connections 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c2803b9c0 0x7f3c2803de80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c4813d140 0x7f3c48137ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c48074130 0x7f3c4811eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 --2- 172.21.15.17:0/2520618223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c48073640 0x7f3c48120c60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 >> 172.21.15.17:0/2520618223 conn(0x7f3c480a66a0 msgr2=0x7f3c4809e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 shutdown_connections 2024-09-15T21:30:02.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.560+0000 7f3c4d619640 1 -- 172.21.15.17:0/2520618223 wait complete. 2024-09-15T21:30:02.581 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:30:02.581 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:30:02.581 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:30:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient: get_monmap_and_config 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient: build_initial_monmap 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient: monmap: 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:02.711009+0000 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:02.711009+0000 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:02.712 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:02.712 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:02.712 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 Processor -- start 2024-09-15T21:30:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 -- start start 2024-09-15T21:30:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient: init 2024-09-15T21:30:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 0x7ff5ec0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): picked mon.noname-b con 0x7ff5ec0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5ec09ffc0 0x7ff5ec0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): picked mon.noname-a con 0x7ff5ec09ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5ec135670 0x7ff5ec135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): picked mon.noname-c con 0x7ff5ec135670 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff5ec0a16f0 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff5ec0a3130 con 0x7ff5ec135670 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec073df0 con 0x7ff5ec0a0890 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5f1ad6640 10 monclient(hunting): authenticate will time out at 1726.106423s 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5ec09ffc0 0x7ff5ec0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 0x7ff5ec0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 0x7ff5ec0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39176/0 (socket says 172.21.15.17:39176) 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5ec09ffc0 0x7ff5ec0a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53908/0 (socket says 172.21.15.17:53908) 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 1 -- 172.21.15.17:0/3485280181 learned_addr learned my addr 172.21.15.17:0/3485280181 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): get_auth_request con 0x7ff5ec0a0890 auth_method 0 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request con 0x7ff5ec09ffc0 auth_method 0 2024-09-15T21:30:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5ebfff640 1 --2- 172.21.15.17:0/3485280181 >> v2:172.21.15.119:3300/0 conn(0x7ff5ec135670 0x7ff5ec135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5ebfff640 10 monclient(hunting): get_auth_request con 0x7ff5ec135670 auth_method 0 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_done global_id 144228 payload 274 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 -- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 msgr2=0x7ff5ec0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 0x7ff5ec0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 -- 172.21.15.17:0/3485280181 >> v2:172.21.15.119:3300/0 conn(0x7ff5ec135670 msgr2=0x7ff5ec135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/3485280181 >> v2:172.21.15.119:3300/0 conn(0x7ff5ec135670 0x7ff5ec135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient: _finish_hunting 0 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 monclient: found mon.noname-a 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eaffd640 1 -- 172.21.15.17:0/3485280181 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec0a1bf0 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5eb7fe640 1 --2- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 0x7ff5ec0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.708+0000 7ff5ebfff640 1 --2- 172.21.15.17:0/3485280181 >> v2:172.21.15.119:3300/0 conn(0x7ff5ec135670 0x7ff5ec135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:30:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec09ffc0 0x7ff5ec0a11b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06f1c0 tx=0x7ff5d401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8273501eba0344 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 1 -- 172.21.15.17:0/3485280181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40b0070 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: dump: 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: _finish_auth 0 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:30:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 1 -- 172.21.15.17:0/3485280181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d40ab600 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 5 monclient: authenticate success, global_id 144228 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 1 -- 172.21.15.17:0/3485280181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40ab9d0 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 4 set_mon_vals no callback set 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5ea7fc640 10 monclient: dump: 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.727 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 10 monclient: shutdown 2024-09-15T21:30:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec09ffc0 msgr2=0x7ff5ec0a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec09ffc0 0x7ff5ec0a11b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06f1c0 tx=0x7ff5d401c340 comp rx=0 tx=0).stop 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/3485280181 shutdown_connections 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/3485280181 >> v2:172.21.15.119:3300/0 conn(0x7ff5ec135670 0x7ff5ec135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec09ffc0 0x7ff5ec0a11b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/3485280181 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec0a0890 0x7ff5ec0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/3485280181 >> 172.21.15.17:0/3485280181 conn(0x7ff5ec0a66a0 msgr2=0x7ff5ec0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/3485280181 shutdown_connections 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/3485280181 wait complete. 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 10 monclient: build_initial_monmap 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 10 monclient: monmap: 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 Processor -- start 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 1 -- start start 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.712+0000 7ff5f1ad6640 10 monclient: init 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec09ffc0 0x7ff5ec1cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): picked mon.noname-c con 0x7ff5ec09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0a0890 0x7ff5ec134700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): picked mon.noname-b con 0x7ff5ec0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec135670 0x7ff5ec11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): picked mon.noname-a con 0x7ff5ec135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec13f680 con 0x7ff5ec135670 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec13f500 con 0x7ff5ec0a0890 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec13f800 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): authenticate will time out at 1726.112277s 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec09ffc0 0x7ff5ec1cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec09ffc0 0x7ff5ec1cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39178/0 (socket says 172.21.15.17:39178) 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 -- 172.21.15.17:0/4100371896 learned_addr learned my addr 172.21.15.17:0/4100371896 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): get_auth_request con 0x7ff5ec09ffc0 auth_method 0 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec135670 0x7ff5ec11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): get_auth_request con 0x7ff5ec135670 auth_method 0 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0a0890 0x7ff5ec134700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request con 0x7ff5ec0a0890 auth_method 0 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_done global_id 134315 payload 824 2024-09-15T21:30:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0a0890 msgr2=0x7ff5ec134700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0a0890 0x7ff5ec134700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec135670 msgr2=0x7ff5ec11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec135670 0x7ff5ec11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 monclient: found mon.noname-c 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec072f10 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eb7fe640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec09ffc0 0x7ff5ec1cd800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009f20 tx=0x7ff5e400c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4014480 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: dump: 2024-09-15T21:30:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: _finish_auth 0 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e4019070 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 5 monclient: authenticate success, global_id 134315 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient: _renew_subs 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec09d0f0 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4014820 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: dump: 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e9ffb640 4 set_mon_vals no callback set 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient: _renew_subs 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec11f800 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 0x7ff5ec11d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec11fc70 con 0x7ff5ec134c40 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 0x7ff5ec11d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient: get_auth_request con 0x7ff5ec134c40 auth_method 0 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff5e4012050 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_done global_id 144234 payload 824 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5ec135aa0 con 0x7ff5ec134c40 2024-09-15T21:30:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5cc03b9f0 0x7ff5cc03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff5e4050c80 con 0x7ff5ec09ffc0 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 0x7ff5ec11d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06e7b0 tx=0x7ff5d401ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40b0050 con 0x7ff5ec134c40 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: dump: 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.739 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5cc03b9f0 0x7ff5cc03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 10 monclient: get_auth_request con 0x7ff5cc03b9f0 auth_method 0 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff5d40b1040 con 0x7ff5ec134c40 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 msgr2=0x7ff5ec11d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 0x7ff5ec11d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06e7b0 tx=0x7ff5d401ab80 comp rx=0 tx=0).stop 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 msgr2=0x7ff5ec11d1e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5e3fff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec134c40 0x7ff5ec11d1e0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06e7b0 tx=0x7ff5d401ab80 comp rx=0 tx=0).stop 2024-09-15T21:30:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.716+0000 7ff5ebfff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5cc03b9f0 0x7ff5cc03deb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0098c0 tx=0x7ff5dc004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 0x7ff5ec02d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1391c0 con 0x7ff5ec02d450 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 0x7ff5ec02d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient: get_auth_request con 0x7ff5ec02d450 auth_method 0 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_done global_id 144237 payload 824 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 1 -- 172.21.15.17:0/4100371896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff5ec06d020 con 0x7ff5ec02d450 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5eaffd640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 0x7ff5ec02d830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5d401c380 tx=0x7ff5d40190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4003b50 con 0x7ff5ec02d450 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 10 monclient: dump: 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4881 (secure 0 0 0) 0x7ff5d40093e0 con 0x7ff5ec02d450 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 msgr2=0x7ff5ec02d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 0x7ff5ec02d830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5d401c380 tx=0x7ff5d40190e0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 msgr2=0x7ff5ec02d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5e3fff640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec02d450 0x7ff5ec02d830 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5d401c380 tx=0x7ff5d40190e0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 74451, 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:30:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:02.746 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:02.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:02.748 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:02.749 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:02.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:02.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:02.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:02.759 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5cc03b9f0 msgr2=0x7ff5cc03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff5cc03b9f0 0x7ff5cc03deb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0098c0 tx=0x7ff5dc004740 comp rx=0 tx=0).stop 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 10 monclient: shutdown 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec09ffc0 msgr2=0x7ff5ec1cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 --2- 172.21.15.17:0/4100371896 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff5ec09ffc0 0x7ff5ec1cd800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009f20 tx=0x7ff5e400c550 comp rx=0 tx=0).stop 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5ebfff640 1 -- 172.21.15.17:0/4100371896 reap_dead start 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5ebfff640 1 -- 172.21.15.17:0/4100371896 reap_dead start 2024-09-15T21:30:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 shutdown_connections 2024-09-15T21:30:02.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 >> 172.21.15.17:0/4100371896 conn(0x7ff5ec0a66a0 msgr2=0x7ff5ec0a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:02.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 shutdown_connections 2024-09-15T21:30:02.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.724+0000 7ff5f1ad6640 1 -- 172.21.15.17:0/4100371896 wait complete. 2024-09-15T21:30:02.762 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:30:02.762 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:30:02.762 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:30:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient: get_monmap_and_config 2024-09-15T21:30:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient: build_initial_monmap 2024-09-15T21:30:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient: monmap: 2024-09-15T21:30:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:02.879614+0000 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:02.879614+0000 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:02.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 1 Processor -- start 2024-09-15T21:30:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 1 -- start start 2024-09-15T21:30:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient: init 2024-09-15T21:30:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f171c0a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f171c0a5580 0x7f171c13cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient(hunting): picked mon.noname-c con 0x7f171c0a5580 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f171c0a59d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f171c0a2c30 con 0x7f171c0a4a90 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f171c0a19b0 con 0x7f171c0a59d0 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f171c0a0f40 con 0x7f171c0a5580 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f1721fff640 10 monclient(hunting): authenticate will time out at 1726.275005s 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f171c0a4a90 0x7f171c0a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35726/0 (socket says 172.21.15.17:35726) 2024-09-15T21:30:02.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171b7fe640 1 -- 172.21.15.17:0/1302441750 learned_addr learned my addr 172.21.15.17:0/1302441750 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 1 --2- 172.21.15.17:0/1302441750 >> v2:172.21.15.119:3300/0 conn(0x7f171c0a5580 0x7f171c13cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): get_auth_request con 0x7f171c0a5580 auth_method 0 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 1 -- 172.21.15.17:0/1302441750 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808778749 0 0) 0x7f1710001120 con 0x7f171c0a4a90 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 10 monclient(hunting): my global_id is 144240 2024-09-15T21:30:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 1 -- 172.21.15.17:0/1302441750 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1704003530 con 0x7f171c0a4a90 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 1 -- 172.21.15.17:0/1302441750 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3290505487 0 0) 0x7f1704003530 con 0x7f171c0a4a90 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient(hunting): handle_auth_done global_id 154108 payload 274 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 1 -- 172.21.15.17:0/1302441750 >> v1:172.21.15.119:6789/0 conn(0x7f171c0a59d0 legacy=0x7f171c13f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 1 -- 172.21.15.17:0/1302441750 >> v1:172.21.15.17:6789/0 conn(0x7f171c0a4a90 legacy=0x7f171c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient: _finish_hunting 0 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 1 monclient: found mon.noname-c 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171affd640 1 -- 172.21.15.17:0/1302441750 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c0a1ce0 con 0x7f171c0a5580 2024-09-15T21:30:02.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.876+0000 7f171a7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:30:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171affd640 1 --2- 172.21.15.17:0/1302441750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a5580 0x7f171c13cd10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f170c0097f0 tx=0x7f170c01c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb674dcea8ca70d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 1 -- 172.21.15.17:0/1302441750 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0b0070 con 0x7f171c0a5580 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: dump: 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.894 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 1 -- 172.21.15.17:0/1302441750 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c0a4e50 con 0x7f171c0a5580 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 5 monclient: authenticate success, global_id 154108 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 4 set_mon_vals no callback set 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 1 -- 172.21.15.17:0/1302441750 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17100018d0 con 0x7f171c0a5580 2024-09-15T21:30:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f171a7fc640 10 monclient: dump: 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.897 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.897 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 10 monclient: shutdown 2024-09-15T21:30:02.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:02.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 -- 172.21.15.17:0/1302441750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a5580 msgr2=0x7f171c13cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 --2- 172.21.15.17:0/1302441750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a5580 0x7f171c13cd10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f170c0097f0 tx=0x7f170c01c2e0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 -- 172.21.15.17:0/1302441750 shutdown_connections 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 --2- 172.21.15.17:0/1302441750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a5580 0x7f171c13cd10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 -- 172.21.15.17:0/1302441750 >> 172.21.15.17:0/1302441750 conn(0x7f171c0aabb0 msgr2=0x7f171c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 -- 172.21.15.17:0/1302441750 shutdown_connections 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.880+0000 7f1721fff640 1 -- 172.21.15.17:0/1302441750 wait complete. 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: build_initial_monmap 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: monmap: 2024-09-15T21:30:02.898 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 Processor -- start 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- start start 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: init 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:02.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c0a4a90 0x7f171c133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): picked mon.noname-c con 0x7f171c0a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f171c0a5580 0x7f171c133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): picked mon.noname-a con 0x7f171c0a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a59d0 0x7f171c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): picked mon.noname-b con 0x7f171c0a59d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1437c0 con 0x7f171c0a5580 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f171c06f290 con 0x7f171c0a59d0 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f171c143940 con 0x7f171c0a4a90 2024-09-15T21:30:02.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): authenticate will time out at 1726.281074s 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f171c0a5580 0x7f171c133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f171c0a5580 0x7f171c133fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53946/0 (socket says 172.21.15.17:53946) 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c0a4a90 0x7f171c133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 -- 172.21.15.17:0/1750055475 learned_addr learned my addr 172.21.15.17:0/1750055475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): get_auth_request con 0x7f171c0a5580 auth_method 0 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): get_auth_request con 0x7f171c0a4a90 auth_method 0 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient(hunting): handle_auth_done global_id 144243 payload 824 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171bfff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a59d0 0x7f171c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c0a4a90 msgr2=0x7f171c133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c0a4a90 0x7f171c133a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a59d0 msgr2=0x7f171c138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c0a59d0 0x7f171c138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient: _finish_hunting 0 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 monclient: found mon.noname-a 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:02.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c13d220 con 0x7f171c0a5580 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171affd640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f171c0a5580 0x7f171c133fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f170c0ae040 tx=0x7f170c01ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0b0050 con 0x7f171c0a5580 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: dump: 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.904 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: _finish_auth 0 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: _check_auth_tickets 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 5 monclient: authenticate success, global_id 144243 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c0b1040 con 0x7f171c0a5580 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: _renew_subs 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c13a490 con 0x7f171c0a5580 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:02.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c004020 con 0x7f171c0a5580 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: dump: 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1719ffb640 4 set_mon_vals no callback set 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: _renew_subs 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c13fcf0 con 0x7f171c0a5580 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 0x7f171c139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f171c0a4e70 con 0x7f171c139230 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f170c0be020 con 0x7f171c0a5580 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 0x7f171c139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient: get_auth_request con 0x7f171c139230 auth_method 0 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f170803b9c0 0x7f170803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f170c0ea530 con 0x7f171c0a5580 2024-09-15T21:30:02.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 10 monclient(hunting): handle_auth_done global_id 134321 payload 824 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f171c0008e0 con 0x7f171c139230 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171b7fe640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 0x7f171c139610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1710001a40 tx=0x7f171000ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171bfff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f170803b9c0 0x7f170803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f171bfff640 10 monclient: get_auth_request con 0x7f170803b9c0 auth_method 0 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710017020 con 0x7f171c139230 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.884+0000 7f1703fff640 10 monclient: dump: 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.909 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f171000d860 con 0x7f171c139230 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 msgr2=0x7f171c139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 0x7f171c139610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1710001a40 tx=0x7f171000ff70 comp rx=0 tx=0).stop 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 msgr2=0x7f171c139610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f1703fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139230 0x7f171c139610 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1710001a40 tx=0x7f171000ff70 comp rx=0 tx=0).stop 2024-09-15T21:30:02.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.888+0000 7f171bfff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f170803b9c0 0x7f170803de80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f17140012f0 tx=0x7f1714009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 0x7f171c09c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f171c134610 con 0x7f171c139b50 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 0x7f171c09c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient: get_auth_request con 0x7f171c139b50 auth_method 0 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:02.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 10 monclient(hunting): handle_auth_done global_id 134324 payload 824 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 1 -- 172.21.15.17:0/1750055475 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f171c139f60 con 0x7f171c139b50 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171b7fe640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 0x7f171c09c580 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1710018000 tx=0x7f1710001450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710017040 con 0x7f171c139b50 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 10 monclient: dump: 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:02.914 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f171001c840 con 0x7f171c139b50 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 msgr2=0x7f171c09c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 0x7f171c09c580 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1710018000 tx=0x7f1710001450 comp rx=0 tx=0).stop 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 msgr2=0x7f171c09c580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1703fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f171c139b50 0x7f171c09c580 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1710018000 tx=0x7f1710001450 comp rx=0 tx=0).stop 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 120046, 2024-09-15T21:30:02.916 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:02.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:02.918 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:02.919 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:02.920 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:02.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.924 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:02.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:02.926 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.927 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:30:02.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:02.929 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:02.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:02.929 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:02.929 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f170803b9c0 msgr2=0x7f170803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f170803b9c0 0x7f170803de80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f17140012f0 tx=0x7f1714009040 comp rx=0 tx=0).stop 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 10 monclient: shutdown 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f171c0a5580 msgr2=0x7f171c133fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 --2- 172.21.15.17:0/1750055475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f171c0a5580 0x7f171c133fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f170c0ae040 tx=0x7f170c01ab20 comp rx=0 tx=0).stop 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171bfff640 1 -- 172.21.15.17:0/1750055475 reap_dead start 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f171bfff640 1 -- 172.21.15.17:0/1750055475 reap_dead start 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 shutdown_connections 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 >> 172.21.15.17:0/1750055475 conn(0x7f171c0aabb0 msgr2=0x7f171c136a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 shutdown_connections 2024-09-15T21:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:02.892+0000 7f1721fff640 1 -- 172.21.15.17:0/1750055475 wait complete. 2024-09-15T21:30:02.932 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:30:02.932 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:30:02.932 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:30:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient: get_monmap_and_config 2024-09-15T21:30:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient: build_initial_monmap 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient: monmap: 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:03.047032+0000 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:03.047032+0000 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:03.047 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 1 Processor -- start 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 1 -- start start 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient: init 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb3e80a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb3e80a5580 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a59d0 0x7fb3e8140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient(hunting): picked mon.noname-b con 0x7fb3e80a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e80a2c30 con 0x7fb3e80a5580 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e80a19b0 con 0x7fb3e80a4a90 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e80a1c90 con 0x7fb3e80a59d0 2024-09-15T21:30:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3edf2b640 10 monclient(hunting): authenticate will time out at 1726.443144s 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb3e80a5580 0x7fb3e813cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35742/0 (socket says 172.21.15.17:35742) 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a59d0 0x7fb3e8140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e6ffd640 1 -- 172.21.15.17:0/207830895 learned_addr learned my addr 172.21.15.17:0/207830895 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): get_auth_request con 0x7fb3e80a59d0 auth_method 0 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_done global_id 134327 payload 274 2024-09-15T21:30:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 1 -- 172.21.15.17:0/207830895 >> v1:172.21.15.119:6789/0 conn(0x7fb3e80a4a90 legacy=0x7fb3e80a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 1 -- 172.21.15.17:0/207830895 >> v1:172.21.15.17:6789/0 conn(0x7fb3e80a5580 legacy=0x7fb3e813cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 1 monclient: found mon.noname-b 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e7fff640 1 -- 172.21.15.17:0/207830895 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e80a1f70 con 0x7fb3e80a59d0 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e67fc640 1 -- 172.21.15.17:0/207830895 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711797080 0 0) 0x7fb3d8001120 con 0x7fb3e80a5580 2024-09-15T21:30:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.044+0000 7fb3e67fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:30:03.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e7fff640 1 --2- 172.21.15.17:0/207830895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a59d0 0x7fb3e8140440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009df0 tx=0x7fb3e001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a7dea5cb78b68e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 1 -- 172.21.15.17:0/207830895 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00b2070 con 0x7fb3e80a59d0 2024-09-15T21:30:03.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: dump: 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: _finish_auth 0 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 1 -- 172.21.15.17:0/207830895 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e00a5820 con 0x7fb3e80a59d0 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 5 monclient: authenticate success, global_id 134327 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 4 set_mon_vals no callback set 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 1 -- 172.21.15.17:0/207830895 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00a5bf0 con 0x7fb3e80a59d0 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3e67fc640 10 monclient: dump: 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 10 monclient: shutdown 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 -- 172.21.15.17:0/207830895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a59d0 msgr2=0x7fb3e8140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 --2- 172.21.15.17:0/207830895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a59d0 0x7fb3e8140440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009df0 tx=0x7fb3e001c850 comp rx=0 tx=0).stop 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 -- 172.21.15.17:0/207830895 shutdown_connections 2024-09-15T21:30:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 --2- 172.21.15.17:0/207830895 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a59d0 0x7fb3e8140440 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 -- 172.21.15.17:0/207830895 >> 172.21.15.17:0/207830895 conn(0x7fb3e80aabb0 msgr2=0x7fb3e80a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 -- 172.21.15.17:0/207830895 shutdown_connections 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 -- 172.21.15.17:0/207830895 wait complete. 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 10 monclient: build_initial_monmap 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 10 monclient: monmap: 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:03.064 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 Processor -- start 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 1 -- start start 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.048+0000 7fb3edf2b640 10 monclient: init 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e80a4a90 0x7fb3e8137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): picked mon.noname-b con 0x7fb3e80a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a5580 0x7fb3e8137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): picked mon.noname-c con 0x7fb3e80a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e80a59d0 0x7fb3e813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): picked mon.noname-a con 0x7fb3e80a59d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e81439d0 con 0x7fb3e80a59d0 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8143850 con 0x7fb3e80a4a90 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e8143b50 con 0x7fb3e80a5580 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): authenticate will time out at 1726.448289s 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a5580 0x7fb3e8137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e80a59d0 0x7fb3e813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a5580 0x7fb3e8137860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39234/0 (socket says 172.21.15.17:39234) 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 -- 172.21.15.17:0/1337778805 learned_addr learned my addr 172.21.15.17:0/1337778805 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): get_auth_request con 0x7fb3e80a5580 auth_method 0 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): get_auth_request con 0x7fb3e80a59d0 auth_method 0 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e80a4a90 0x7fb3e8137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): get_auth_request con 0x7fb3e80a4a90 auth_method 0 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient(hunting): handle_auth_done global_id 134330 payload 824 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e80a4a90 msgr2=0x7fb3e8137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e80a4a90 0x7fb3e8137320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e80a59d0 msgr2=0x7fb3e813cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e80a59d0 0x7fb3e813cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 monclient: found mon.noname-c 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:03.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e813fe10 con 0x7fb3e80a5580 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e80a59d0 0x7fb3e813cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e6ffd640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a5580 0x7fb3e8137860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c760 tx=0x7fb3d800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8018070 con 0x7fb3e80a5580 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: dump: 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.069 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: _finish_auth 0 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: _check_auth_tickets 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d80070d0 con 0x7fb3e80a5580 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80074c0 con 0x7fb3e80a5580 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 5 monclient: authenticate success, global_id 134330 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient: _renew_subs 2024-09-15T21:30:03.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e8120c80 con 0x7fb3e80a5580 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e57fa640 4 set_mon_vals no callback set 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: dump: 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.071 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient: _renew_subs 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e80a1280 con 0x7fb3e80a5580 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 0x7fb3e813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e80a4e70 con 0x7fb3e813d080 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb3d801b070 con 0x7fb3e80a5580 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 0x7fb3e813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient: get_auth_request con 0x7fb3e813d080 auth_method 0 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3d403b9c0 0x7fb3d403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb3d804f960 con 0x7fb3e80a5580 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3d403b9c0 0x7fb3d403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 10 monclient: get_auth_request con 0x7fb3d403b9c0 auth_method 0 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_done global_id 154117 payload 824 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3e8120a60 con 0x7fb3e813d080 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 0x7fb3e813d460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e4b0 tx=0x7fb3dc00e970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0052e0 con 0x7fb3e813d080 2024-09-15T21:30:03.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3e7fff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3d403b9c0 0x7fb3d403de80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00af000 tx=0x7fb3e00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.052+0000 7fb3cffff640 10 monclient: dump: 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.074 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb3dc018070 con 0x7fb3e813d080 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 msgr2=0x7fb3e813d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 0x7fb3e813d460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e4b0 tx=0x7fb3dc00e970 comp rx=0 tx=0).stop 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 msgr2=0x7fb3e813d460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:03.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.056+0000 7fb3cffff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d080 0x7fb3e813d460 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e4b0 tx=0x7fb3dc00e970 comp rx=0 tx=0).stop 2024-09-15T21:30:03.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3edf2b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3edf2b640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3edf2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8069200 con 0x7fb3e813d9a0 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient: get_auth_request con 0x7fb3e813d9a0 auth_method 0 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 msgr2=0x7fb3e8138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:30:03.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 msgr2=0x7fb3e8138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:30:03.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:03.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.060+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:03.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient: get_auth_request con 0x7fb3e813d9a0 auth_method 0 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 10 monclient(hunting): handle_auth_done global_id 154123 payload 824 2024-09-15T21:30:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 1 -- 172.21.15.17:0/1337778805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb3e813edc0 con 0x7fb3e813d9a0 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3e77fe640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e9b0 tx=0x7fb3dc001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc02b060 con 0x7fb3e813d9a0 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 10 monclient: dump: 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7fb3dc003810 con 0x7fb3e813d9a0 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 msgr2=0x7fb3e8138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e9b0 tx=0x7fb3dc001130 comp rx=0 tx=0).stop 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 msgr2=0x7fb3e8138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.264+0000 7fb3cffff640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3e813d9a0 0x7fb3e8138cf0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e9b0 tx=0x7fb3dc001130 comp rx=0 tx=0).stop 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7799, 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:30:03.271 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:03.272 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:03.273 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:03.274 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.276 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:03.277 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:03.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:03.279 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:03.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:03.281 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:03.282 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:03.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:03.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:03.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:03.283 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:03.283 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3d403b9c0 msgr2=0x7fb3d403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb3d403b9c0 0x7fb3d403de80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00af000 tx=0x7fb3e00a7070 comp rx=0 tx=0).stop 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 10 monclient: shutdown 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a5580 msgr2=0x7fb3e8137860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 --2- 172.21.15.17:0/1337778805 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3e80a5580 0x7fb3e8137860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c760 tx=0x7fb3d800cc20 comp rx=0 tx=0).stop 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3e7fff640 1 -- 172.21.15.17:0/1337778805 reap_dead start 2024-09-15T21:30:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3e7fff640 1 -- 172.21.15.17:0/1337778805 reap_dead start 2024-09-15T21:30:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 shutdown_connections 2024-09-15T21:30:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 >> 172.21.15.17:0/1337778805 conn(0x7fb3e80aabb0 msgr2=0x7fb3e80a8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 shutdown_connections 2024-09-15T21:30:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.268+0000 7fb3edf2b640 1 -- 172.21.15.17:0/1337778805 wait complete. 2024-09-15T21:30:03.290 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:30:03.290 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:30:03.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.408+0000 7fb5d2eca640 10 monclient: get_monmap_and_config 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.408+0000 7fb5d2eca640 10 monclient: build_initial_monmap 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.408+0000 7fb5d2eca640 10 monclient: monmap: 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:03.413507+0000 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:03.413507+0000 2024-09-15T21:30:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:03.414 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 Processor -- start 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 -- start start 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient: init 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5cc1378c0 0x7fb5cc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): picked mon.noname-c con 0x7fb5cc1378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 0x7fb5cc0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): picked mon.noname-a con 0x7fb5cc1381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 0x7fb5cc0a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): picked mon.noname-b con 0x7fb5cc0a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb5cc0a1d00 con 0x7fb5cc1381e0 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb5cc0a1fe0 con 0x7fb5cc1378c0 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb5cc0a22c0 con 0x7fb5cc0a5040 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient(hunting): authenticate will time out at 1726.808993s 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 0x7fb5cc0a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 0x7fb5cc0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 0x7fb5cc0a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53960/0 (socket says 172.21.15.17:53960) 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 0x7fb5cc0a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39248/0 (socket says 172.21.15.17:39248) 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 1 -- 172.21.15.17:0/206986057 learned_addr learned my addr 172.21.15.17:0/206986057 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): get_auth_request con 0x7fb5cc1381e0 auth_method 0 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): get_auth_request con 0x7fb5cc0a5040 auth_method 0 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d0c66640 1 --2- 172.21.15.17:0/206986057 >> v2:172.21.15.119:3300/0 conn(0x7fb5cc1378c0 0x7fb5cc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d0c66640 10 monclient(hunting): get_auth_request con 0x7fb5cc1378c0 auth_method 0 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d0c66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d0c66640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d0c66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient(hunting): handle_auth_done global_id 134333 payload 274 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 -- 172.21.15.17:0/206986057 >> v2:172.21.15.119:3300/0 conn(0x7fb5cc1378c0 msgr2=0x7fb5cc137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 --2- 172.21.15.17:0/206986057 >> v2:172.21.15.119:3300/0 conn(0x7fb5cc1378c0 0x7fb5cc137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 -- 172.21.15.17:0/206986057 >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 msgr2=0x7fb5cc0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 --2- 172.21.15.17:0/206986057 >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 0x7fb5cc0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient: _finish_hunting 0 2024-09-15T21:30:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 monclient: found mon.noname-b 2024-09-15T21:30:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:30:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 -- 172.21.15.17:0/206986057 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc0a4490 con 0x7fb5cc0a5040 2024-09-15T21:30:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cbfff640 1 --2- 172.21.15.17:0/206986057 >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 0x7fb5cc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d1467640 1 --2- 172.21.15.17:0/206986057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 0x7fb5cc0a54d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009d90 tx=0x7fb5c401c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c67196e4fd10293d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 1 -- 172.21.15.17:0/206986057 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40b2070 con 0x7fb5cc0a5040 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: dump: 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 1 -- 172.21.15.17:0/206986057 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c40a5890 con 0x7fb5cc0a5040 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 1 -- 172.21.15.17:0/206986057 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40a5c60 con 0x7fb5cc0a5040 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5cb7fe640 10 monclient: dump: 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 5 monclient: authenticate success, global_id 134333 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.412+0000 7fb5d2eca640 4 set_mon_vals no callback set 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient: shutdown 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- 172.21.15.17:0/206986057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 msgr2=0x7fb5cc0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/206986057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 0x7fb5cc0a54d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009d90 tx=0x7fb5c401c7f0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- 172.21.15.17:0/206986057 shutdown_connections 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/206986057 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc0a5040 0x7fb5cc0a54d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/206986057 >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1381e0 0x7fb5cc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/206986057 >> v2:172.21.15.119:3300/0 conn(0x7fb5cc1378c0 0x7fb5cc137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- 172.21.15.17:0/206986057 >> 172.21.15.17:0/206986057 conn(0x7fb5cc0aac40 msgr2=0x7fb5cc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- 172.21.15.17:0/206986057 shutdown_connections 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- 172.21.15.17:0/206986057 wait complete. 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient: build_initial_monmap 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient: monmap: 2024-09-15T21:30:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 Processor -- start 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- start start 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient: init 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5cc0a5040 0x7fb5cc1cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): picked mon.noname-b con 0x7fb5cc0a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc1378c0 0x7fb5cc136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): picked mon.noname-c con 0x7fb5cc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 0x7fb5cc1370e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): picked mon.noname-a con 0x7fb5cc1381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc141730 con 0x7fb5cc1381e0 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1415b0 con 0x7fb5cc0a5040 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb5cc1418b0 con 0x7fb5cc1378c0 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d2eca640 10 monclient(hunting): authenticate will time out at 1726.814892s 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 0x7fb5cc1370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 0x7fb5cc1370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53968/0 (socket says 172.21.15.17:53968) 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 -- 172.21.15.17:0/1574774018 learned_addr learned my addr 172.21.15.17:0/1574774018 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): get_auth_request con 0x7fb5cc1381e0 auth_method 0 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5cc0a5040 0x7fb5cc1cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc1378c0 0x7fb5cc136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): get_auth_request con 0x7fb5cc0a5040 auth_method 0 2024-09-15T21:30:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): get_auth_request con 0x7fb5cc1378c0 auth_method 0 2024-09-15T21:30:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d0c66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient(hunting): handle_auth_done global_id 144255 payload 824 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 -- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc1378c0 msgr2=0x7fb5cc136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc1378c0 0x7fb5cc136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 -- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5cc0a5040 msgr2=0x7fb5cc1cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5cc0a5040 0x7fb5cc1cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient: _finish_hunting 0 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 monclient: found mon.noname-a 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 -- 172.21.15.17:0/1574774018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc122560 con 0x7fb5cc1381e0 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5cbfff640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc1378c0 0x7fb5cc136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5d1467640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 0x7fb5cc1370e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40b0040 tx=0x7fb5c4019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.416+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40b2050 con 0x7fb5cc1381e0 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: dump: 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: _finish_auth 0 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c40be070 con 0x7fb5cc1381e0 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40ada70 con 0x7fb5cc1381e0 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 5 monclient: authenticate success, global_id 144255 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 10 monclient: dump: 2024-09-15T21:30:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 monclient: _renew_subs 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5ca7fc640 4 set_mon_vals no callback set 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc1d0ba0 con 0x7fb5cc1381e0 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 monclient: _renew_subs 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc13e7f0 con 0x7fb5cc1381e0 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb5c40a5710 con 0x7fb5cc1381e0 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc137ca0 con 0x7fb5cc1381e0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5b003b9f0 0x7fb5b003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb5c40eb530 con 0x7fb5cc1381e0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d0c66640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5b003b9f0 0x7fb5b003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d0c66640 10 monclient: get_auth_request con 0x7fb5b003b9f0 auth_method 0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.420+0000 7fb5d0c66640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5b003b9f0 0x7fb5b003deb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0006aa0 tx=0x7fb5c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.424+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb5c4016500 con 0x7fb5cc1381e0 2024-09-15T21:30:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.424+0000 7fb5c97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:03.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.424+0000 7fb5c97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.660+0000 7fb5d2eca640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:30:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.660+0000 7fb5d2eca640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:30:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.660+0000 7fb5d2eca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.660+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fb5cc069270 con 0x7fb5cc1381e0 2024-09-15T21:30:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fb5c97fa640 1 -- 172.21.15.17:0/1574774018 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fb5c40c39a0 con 0x7fb5cc1381e0 2024-09-15T21:30:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fb5c97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:30:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fb5c97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:03.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=1,health=11,kv=12,logm=49} crc {auth=4291237048,config=1228700967,health=4076111,kv=3079713637,logm=1887319242}) 2024-09-15T21:30:03.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=12,kv=12,logm=47} crc {auth=4258671806,config=3541143443,health=1301148488,kv=3079713637,logm=4046704952}) 2024-09-15T21:30:03.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=1,health=11,kv=12,logm=49} crc {auth=4291237048,config=1228700967,health=4076111,kv=3079713637,logm=1887319242}) 2024-09-15T21:30:03.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.664+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=2886029444,config=3541143443,health=1301148488,kv=3079713637,logm=819087547}) 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5b003b9f0 msgr2=0x7fb5b003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5b003b9f0 0x7fb5b003deb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0006aa0 tx=0x7fb5c0008040 comp rx=0 tx=0).stop 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 10 monclient: shutdown 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 msgr2=0x7fb5cc1370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 0x7fb5cc1370e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40b0040 tx=0x7fb5c4019590 comp rx=0 tx=0).stop 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 shutdown_connections 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5b003b9f0 0x7fb5b003deb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1381e0 0x7fb5cc1370e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5cc1378c0 0x7fb5cc136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 --2- 172.21.15.17:0/1574774018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5cc0a5040 0x7fb5cc1cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 >> 172.21.15.17:0/1574774018 conn(0x7fb5cc0aac40 msgr2=0x7fb5cc09da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 shutdown_connections 2024-09-15T21:30:03.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940744339}) 2024-09-15T21:30:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fb5d2eca640 1 -- 172.21.15.17:0/1574774018 wait complete. 2024-09-15T21:30:03.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3727991137}) 2024-09-15T21:30:03.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940744339}) 2024-09-15T21:30:03.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1205430497}) 2024-09-15T21:30:03.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=385738549}) 2024-09-15T21:30:03.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=697622978}) 2024-09-15T21:30:03.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=385738549}) 2024-09-15T21:30:03.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.668+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3706249935}) 2024-09-15T21:30:03.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3878025510}) 2024-09-15T21:30:03.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=950548178}) 2024-09-15T21:30:03.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3878025510}) 2024-09-15T21:30:03.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2645489388}) 2024-09-15T21:30:03.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578024688}) 2024-09-15T21:30:03.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=36332989}) 2024-09-15T21:30:03.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578024688}) 2024-09-15T21:30:03.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.672+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3426438115}) 2024-09-15T21:30:03.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463822090}) 2024-09-15T21:30:03.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3689172977}) 2024-09-15T21:30:03.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463822090}) 2024-09-15T21:30:03.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=339885351}) 2024-09-15T21:30:03.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2813662136}) 2024-09-15T21:30:03.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2889490742}) 2024-09-15T21:30:03.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2813662136}) 2024-09-15T21:30:03.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2495137666}) 2024-09-15T21:30:03.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2157962404}) 2024-09-15T21:30:03.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3022419830}) 2024-09-15T21:30:03.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2157962404}) 2024-09-15T21:30:03.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.676+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1943632050}) 2024-09-15T21:30:03.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1699084106}) 2024-09-15T21:30:03.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1180753817}) 2024-09-15T21:30:03.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1699084106}) 2024-09-15T21:30:03.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=435666039}) 2024-09-15T21:30:03.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146152503}) 2024-09-15T21:30:03.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2832389402}) 2024-09-15T21:30:03.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146152503}) 2024-09-15T21:30:03.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.680+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4005546734}) 2024-09-15T21:30:03.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014677771}) 2024-09-15T21:30:03.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2719392569}) 2024-09-15T21:30:03.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014677771}) 2024-09-15T21:30:03.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=424834806}) 2024-09-15T21:30:03.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.688 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=569584026}) 2024-09-15T21:30:03.688 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1830624868}) 2024-09-15T21:30:03.688 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.688 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=569584026}) 2024-09-15T21:30:03.688 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3328840387}) 2024-09-15T21:30:03.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245178861}) 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1704532846}) 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245178861}) 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.684+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=529355252}) 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3616087680}) 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=809784866}) 2024-09-15T21:30:03.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3616087680}) 2024-09-15T21:30:03.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3162476194}) 2024-09-15T21:30:03.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=753931630}) 2024-09-15T21:30:03.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1855932418}) 2024-09-15T21:30:03.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=753931630}) 2024-09-15T21:30:03.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.688+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2912619234}) 2024-09-15T21:30:03.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1858385984}) 2024-09-15T21:30:03.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=560667984}) 2024-09-15T21:30:03.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1858385984}) 2024-09-15T21:30:03.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1795494441}) 2024-09-15T21:30:03.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969019786}) 2024-09-15T21:30:03.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4251636658}) 2024-09-15T21:30:03.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969019786}) 2024-09-15T21:30:03.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.692+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1224842210}) 2024-09-15T21:30:03.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=562006816}) 2024-09-15T21:30:03.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3686414025}) 2024-09-15T21:30:03.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=562006816}) 2024-09-15T21:30:03.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2931933378}) 2024-09-15T21:30:03.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2605375038}) 2024-09-15T21:30:03.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441138339}) 2024-09-15T21:30:03.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2605375038}) 2024-09-15T21:30:03.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1502751846}) 2024-09-15T21:30:03.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908850068}) 2024-09-15T21:30:03.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2713585440}) 2024-09-15T21:30:03.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908850068}) 2024-09-15T21:30:03.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.696+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3169054085}) 2024-09-15T21:30:03.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1694660469}) 2024-09-15T21:30:03.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3142507017}) 2024-09-15T21:30:03.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1694660469}) 2024-09-15T21:30:03.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4137412649}) 2024-09-15T21:30:03.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3694534882}) 2024-09-15T21:30:03.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3039920594}) 2024-09-15T21:30:03.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3694534882}) 2024-09-15T21:30:03.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.700+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3901417526}) 2024-09-15T21:30:03.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1661329680}) 2024-09-15T21:30:03.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1662638962}) 2024-09-15T21:30:03.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1661329680}) 2024-09-15T21:30:03.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=604101747}) 2024-09-15T21:30:03.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1742047207}) 2024-09-15T21:30:03.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1215246131}) 2024-09-15T21:30:03.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1742047207}) 2024-09-15T21:30:03.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3748213574}) 2024-09-15T21:30:03.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=70896972}) 2024-09-15T21:30:03.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1225068569}) 2024-09-15T21:30:03.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=70896972}) 2024-09-15T21:30:03.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.704+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=364944433}) 2024-09-15T21:30:03.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=725683668}) 2024-09-15T21:30:03.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=376932159}) 2024-09-15T21:30:03.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=725683668}) 2024-09-15T21:30:03.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1501755705}) 2024-09-15T21:30:03.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3841073821}) 2024-09-15T21:30:03.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3494127753}) 2024-09-15T21:30:03.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3841073821}) 2024-09-15T21:30:03.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.708+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3214160008}) 2024-09-15T21:30:03.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1562636099}) 2024-09-15T21:30:03.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4073818708}) 2024-09-15T21:30:03.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1562636099}) 2024-09-15T21:30:03.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2222243434}) 2024-09-15T21:30:03.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3682108254}) 2024-09-15T21:30:03.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1241790756}) 2024-09-15T21:30:03.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3682108254}) 2024-09-15T21:30:03.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.712+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=785817554}) 2024-09-15T21:30:03.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=681326938}) 2024-09-15T21:30:03.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790719355}) 2024-09-15T21:30:03.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=681326938}) 2024-09-15T21:30:03.719 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1607113524}) 2024-09-15T21:30:03.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1946005004}) 2024-09-15T21:30:03.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2894446199}) 2024-09-15T21:30:03.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1946005004}) 2024-09-15T21:30:03.721 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2564604217}) 2024-09-15T21:30:03.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2372902665}) 2024-09-15T21:30:03.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1369911539}) 2024-09-15T21:30:03.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2372902665}) 2024-09-15T21:30:03.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.716+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888140196}) 2024-09-15T21:30:03.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2283635484}) 2024-09-15T21:30:03.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3440423550}) 2024-09-15T21:30:03.723 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2283635484}) 2024-09-15T21:30:03.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=822556650}) 2024-09-15T21:30:03.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291761746}) 2024-09-15T21:30:03.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1860367256}) 2024-09-15T21:30:03.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291761746}) 2024-09-15T21:30:03.725 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.720+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1513102390}) 2024-09-15T21:30:03.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=989920955}) 2024-09-15T21:30:03.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2504851894}) 2024-09-15T21:30:03.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=989920955}) 2024-09-15T21:30:03.727 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4049512330}) 2024-09-15T21:30:03.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421829777}) 2024-09-15T21:30:03.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=929322987}) 2024-09-15T21:30:03.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421829777}) 2024-09-15T21:30:03.729 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.724+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2986045908}) 2024-09-15T21:30:03.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=736014605}) 2024-09-15T21:30:03.731 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=363601481}) 2024-09-15T21:30:03.731 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.731 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=736014605}) 2024-09-15T21:30:03.731 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1026239747}) 2024-09-15T21:30:03.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=777661139}) 2024-09-15T21:30:03.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3485383960}) 2024-09-15T21:30:03.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=777661139}) 2024-09-15T21:30:03.733 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.728+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2702170598}) 2024-09-15T21:30:03.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247459116}) 2024-09-15T21:30:03.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=699550443}) 2024-09-15T21:30:03.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247459116}) 2024-09-15T21:30:03.735 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3156358522}) 2024-09-15T21:30:03.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4204504839}) 2024-09-15T21:30:03.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3906005765}) 2024-09-15T21:30:03.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4204504839}) 2024-09-15T21:30:03.737 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.732+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=722402074}) 2024-09-15T21:30:03.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3875481052}) 2024-09-15T21:30:03.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=642007893}) 2024-09-15T21:30:03.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3875481052}) 2024-09-15T21:30:03.739 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1630912613}) 2024-09-15T21:30:03.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3181550208}) 2024-09-15T21:30:03.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=147660763}) 2024-09-15T21:30:03.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3181550208}) 2024-09-15T21:30:03.741 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.736+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=585342925}) 2024-09-15T21:30:03.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635552223}) 2024-09-15T21:30:03.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=308109883}) 2024-09-15T21:30:03.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635552223}) 2024-09-15T21:30:03.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3063158546}) 2024-09-15T21:30:03.744 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3542122306}) 2024-09-15T21:30:03.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734317626}) 2024-09-15T21:30:03.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3542122306}) 2024-09-15T21:30:03.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.740+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=235514784}) 2024-09-15T21:30:03.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=888961737}) 2024-09-15T21:30:03.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3432559623}) 2024-09-15T21:30:03.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=888961737}) 2024-09-15T21:30:03.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2831438979}) 2024-09-15T21:30:03.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2499212770}) 2024-09-15T21:30:03.749 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3781386007}) 2024-09-15T21:30:03.749 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.749 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2499212770}) 2024-09-15T21:30:03.749 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.744+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1374915842}) 2024-09-15T21:30:03.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549366661}) 2024-09-15T21:30:03.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4059965549}) 2024-09-15T21:30:03.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549366661}) 2024-09-15T21:30:03.751 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1053960364}) 2024-09-15T21:30:03.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1585000171}) 2024-09-15T21:30:03.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4181385685}) 2024-09-15T21:30:03.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1585000171}) 2024-09-15T21:30:03.753 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.748+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127457102}) 2024-09-15T21:30:03.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582778510}) 2024-09-15T21:30:03.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=577172499}) 2024-09-15T21:30:03.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.755 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582778510}) 2024-09-15T21:30:03.755 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3971986485}) 2024-09-15T21:30:03.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3015544516}) 2024-09-15T21:30:03.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3814177582}) 2024-09-15T21:30:03.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3015544516}) 2024-09-15T21:30:03.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.752+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3618429658}) 2024-09-15T21:30:03.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187842050}) 2024-09-15T21:30:03.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2382249558}) 2024-09-15T21:30:03.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187842050}) 2024-09-15T21:30:03.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2418664037}) 2024-09-15T21:30:03.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=27534494}) 2024-09-15T21:30:03.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2802650945}) 2024-09-15T21:30:03.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=27534494}) 2024-09-15T21:30:03.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.756+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3103986969}) 2024-09-15T21:30:03.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4061618138}) 2024-09-15T21:30:03.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1426450049}) 2024-09-15T21:30:03.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4061618138}) 2024-09-15T21:30:03.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1869809543}) 2024-09-15T21:30:03.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1938959479}) 2024-09-15T21:30:03.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=757036289}) 2024-09-15T21:30:03.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1938959479}) 2024-09-15T21:30:03.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.760+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2274827215}) 2024-09-15T21:30:03.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833613426}) 2024-09-15T21:30:03.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1524797669}) 2024-09-15T21:30:03.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833613426}) 2024-09-15T21:30:03.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2904605073}) 2024-09-15T21:30:03.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2097645279}) 2024-09-15T21:30:03.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3456548440}) 2024-09-15T21:30:03.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2097645279}) 2024-09-15T21:30:03.769 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.764+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3753373010}) 2024-09-15T21:30:03.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702207420}) 2024-09-15T21:30:03.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4168743073}) 2024-09-15T21:30:03.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702207420}) 2024-09-15T21:30:03.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3382087675}) 2024-09-15T21:30:03.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1685239896}) 2024-09-15T21:30:03.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2280553481}) 2024-09-15T21:30:03.772 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1685239896}) 2024-09-15T21:30:03.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.768+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448505765}) 2024-09-15T21:30:03.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2558686773}) 2024-09-15T21:30:03.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=912227620}) 2024-09-15T21:30:03.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2558686773}) 2024-09-15T21:30:03.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=666367916}) 2024-09-15T21:30:03.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488482558}) 2024-09-15T21:30:03.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1322780278}) 2024-09-15T21:30:03.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488482558}) 2024-09-15T21:30:03.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.772+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696117404}) 2024-09-15T21:30:03.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4287126246}) 2024-09-15T21:30:03.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2674957561}) 2024-09-15T21:30:03.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.779 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4287126246}) 2024-09-15T21:30:03.779 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2741183373}) 2024-09-15T21:30:03.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1097222038}) 2024-09-15T21:30:03.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=112327689}) 2024-09-15T21:30:03.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.781 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1097222038}) 2024-09-15T21:30:03.781 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.776+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1886175449}) 2024-09-15T21:30:03.782 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.782 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=593825111}) 2024-09-15T21:30:03.784 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4135100694}) 2024-09-15T21:30:03.784 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.784 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=593825111}) 2024-09-15T21:30:03.784 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2787324548}) 2024-09-15T21:30:03.784 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.784 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1710651813}) 2024-09-15T21:30:03.785 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3790614179}) 2024-09-15T21:30:03.785 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.785 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1710651813}) 2024-09-15T21:30:03.785 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.780+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2420132786}) 2024-09-15T21:30:03.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=658576817}) 2024-09-15T21:30:03.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3083016176}) 2024-09-15T21:30:03.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=658576817}) 2024-09-15T21:30:03.786 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3910889396}) 2024-09-15T21:30:03.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112506502}) 2024-09-15T21:30:03.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1824024817}) 2024-09-15T21:30:03.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112506502}) 2024-09-15T21:30:03.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.784+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2212368441}) 2024-09-15T21:30:03.790 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.790 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949163036}) 2024-09-15T21:30:03.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4073355726}) 2024-09-15T21:30:03.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949163036}) 2024-09-15T21:30:03.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4037831304}) 2024-09-15T21:30:03.792 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.792 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4245877020}) 2024-09-15T21:30:03.792 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=255541106}) 2024-09-15T21:30:03.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4245877020}) 2024-09-15T21:30:03.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.788+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1282253849}) 2024-09-15T21:30:03.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745235234}) 2024-09-15T21:30:03.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3288798737}) 2024-09-15T21:30:03.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745235234}) 2024-09-15T21:30:03.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1693629615}) 2024-09-15T21:30:03.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1567221756}) 2024-09-15T21:30:03.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3605260847}) 2024-09-15T21:30:03.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1567221756}) 2024-09-15T21:30:03.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.792+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1907025068}) 2024-09-15T21:30:03.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3654731224}) 2024-09-15T21:30:03.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1682802605}) 2024-09-15T21:30:03.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3654731224}) 2024-09-15T21:30:03.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2851129266}) 2024-09-15T21:30:03.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=79,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=2480725711,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2862456999,mon_config_key=2661559954}) 2024-09-15T21:30:03.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=79,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=2558013360,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2862456999,mon_config_key=2661559954}) 2024-09-15T21:30:03.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=79,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=2480725711,mdsmap=173658045,mgr=1715286761,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2862456999,mon_config_key=2661559954}) 2024-09-15T21:30:03.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.796+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=85,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=1082381462,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1694918080}) 2024-09-15T21:30:03.817 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.812+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.817 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.812+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {mon_config_key=1373851027,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1236221888}) 2024-09-15T21:30:03.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.812+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {mon_config_key=1966449915,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=390463912}) 2024-09-15T21:30:03.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.812+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:30:03.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.812+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {mon_config_key=1373851027,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1236221888}) 2024-09-15T21:30:03.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:30:03.812+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {mon_config_key=1723083536,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=406562647}) 2024-09-15T21:30:04.690 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:30:04.691 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:30:04.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient: get_monmap_and_config 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient: build_initial_monmap 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient: monmap: 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:04.823956+0000 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:04.823956+0000 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:04.825 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 1 Processor -- start 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 1 -- start start 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient: init 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f499409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.820+0000 7f499c753640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f499409def0 0x7f49941398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 10 monclient(hunting): picked mon.noname-c con 0x7f499409def0 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:04.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4994139de0 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:04.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49940a3490 con 0x7f4994139de0 2024-09-15T21:30:04.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49940a0320 con 0x7f499409d400 2024-09-15T21:30:04.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f49940a1bc0 con 0x7f499409def0 2024-09-15T21:30:04.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:04.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 10 monclient(hunting): authenticate will time out at 1728.220115s 2024-09-15T21:30:04.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4994139de0 0x7f499413c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35756/0 (socket says 172.21.15.17:35756) 2024-09-15T21:30:04.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 1 -- 172.21.15.17:0/1572780359 learned_addr learned my addr 172.21.15.17:0/1572780359 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:04.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 1 --2- 172.21.15.17:0/1572780359 >> v2:172.21.15.119:3300/0 conn(0x7f499409def0 0x7f49941398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): get_auth_request con 0x7f499409def0 auth_method 0 2024-09-15T21:30:04.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:04.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641344240 0 0) 0x7f49940a3490 con 0x7f4994139de0 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): _negotiate 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): my global_id is 144258 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4980003530 con 0x7f4994139de0 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 343225070 0 0) 0x7f4980003530 con 0x7f4994139de0 2024-09-15T21:30:04.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 >> v2:172.21.15.119:3300/0 conn(0x7f499409def0 msgr2=0x7f49941398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 --2- 172.21.15.17:0/1572780359 >> v2:172.21.15.119:3300/0 conn(0x7f499409def0 0x7f49941398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 >> v1:172.21.15.119:6789/0 conn(0x7f499409d400 legacy=0x7f499409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: _finish_hunting 0 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499409e7a0 con 0x7f4994139de0 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: _finish_auth 0 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: _check_auth_tickets 2024-09-15T21:30:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f498c001700 con 0x7f4994139de0 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 5 monclient: authenticate success, global_id 144258 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: dump: 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 >> v1:172.21.15.17:6789/0 conn(0x7f4994139de0 legacy=0x7f499413c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4980003a60 0x7f4980003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): picked mon.a con 0x7f4980003a60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): picked mon.c con 0x7f4980004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49800053d0 0x7f498000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): picked mon.b con 0x7f49800053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f49940a1bc0 con 0x7f4980003a60 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f498000c240 con 0x7f49800053d0 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f498000c440 con 0x7f4980004810 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499c753640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4980003a60 0x7f4980003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): get_auth_request con 0x7f4980003a60 auth_method 0 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): get_auth_request con 0x7f4980004810 auth_method 0 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49800053d0 0x7f498000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 10 monclient(hunting): get_auth_request con 0x7f49800053d0 auth_method 0 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient(hunting): handle_auth_done global_id 144258 payload 293 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 -- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980004810 msgr2=0x7f4980004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 -- 172.21.15.17:0/1572780359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49800053d0 msgr2=0x7f498000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49800053d0 0x7f498000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient: _finish_hunting 0 2024-09-15T21:30:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 monclient: found mon.a 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499acf0640 1 -- 172.21.15.17:0/1572780359 reap_dead start 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 -- 172.21.15.17:0/1572780359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4980003530 con 0x7f4980003a60 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient: _finish_auth 0 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 10 monclient: _check_auth_tickets 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f4999cee640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4980003a60 0x7f4980003e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4990001e90 tx=0x7f499001d160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b38aca5dedccf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990001ae0 con 0x7f4980003a60 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.824+0000 7f49994ed640 10 monclient: dump: 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4990003b80 con 0x7f4980003a60 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f49994ed640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 4 set_mon_vals no callback set 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f49994ed640 1 -- 172.21.15.17:0/1572780359 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990005080 con 0x7f4980003a60 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f49994ed640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f49994ed640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f49994ed640 10 monclient: dump: 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: shutdown 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4980003a60 msgr2=0x7f4980003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4980003a60 0x7f4980003e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4990001e90 tx=0x7f499001d160 comp rx=0 tx=0).stop 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1572780359 shutdown_connections 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 --2- 172.21.15.17:0/1572780359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4980003a60 0x7f4980003e80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1572780359 >> 172.21.15.17:0/1572780359 conn(0x7f49940a66a0 msgr2=0x7f49940a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1572780359 shutdown_connections 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1572780359 wait complete. 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: build_initial_monmap 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: monmap: 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 Processor -- start 2024-09-15T21:30:04.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- start start 2024-09-15T21:30:04.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: init 2024-09-15T21:30:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 0x7f4994125120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): picked mon.noname-c con 0x7f4980003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4994122c00 0x7f4994123060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): picked mon.noname-b con 0x7f4994122c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49941235a0 0x7f49941d6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): picked mon.noname-a con 0x7f49941235a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f49940a32b0 con 0x7f49941235a0 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f49940a3130 con 0x7f4994122c00 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f499409ccd0 con 0x7f4980003a60 2024-09-15T21:30:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient(hunting): authenticate will time out at 1728.226287s 2024-09-15T21:30:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 0x7f4994125120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49941235a0 0x7f49941d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 0x7f4994125120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39268/0 (socket says 172.21.15.17:39268) 2024-09-15T21:30:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 1 -- 172.21.15.17:0/1368804073 learned_addr learned my addr 172.21.15.17:0/1368804073 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:04.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): get_auth_request con 0x7f4980003a60 auth_method 0 2024-09-15T21:30:04.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): get_auth_request con 0x7f49941235a0 auth_method 0 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f4999cee640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4994122c00 0x7f4994123060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f4999cee640 10 monclient(hunting): get_auth_request con 0x7f4994122c00 auth_method 0 2024-09-15T21:30:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f4999cee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f4999cee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f4999cee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient(hunting): handle_auth_done global_id 144261 payload 824 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 -- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 msgr2=0x7f4994125120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 0x7f4994125120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 -- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4994122c00 msgr2=0x7f4994123060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4994122c00 0x7f4994123060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient: _finish_hunting 0 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 monclient: found mon.noname-a 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 -- 172.21.15.17:0/1368804073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941342e0 con 0x7f49941235a0 2024-09-15T21:30:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 0x7f4994125120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499acf0640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49941235a0 0x7f49941d6080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f499409e300 tx=0x7f498c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c019070 con 0x7f49941235a0 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: dump: 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: _finish_auth 0 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: _check_auth_tickets 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c010510 con 0x7f49941235a0 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 5 monclient: authenticate success, global_id 144261 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: _renew_subs 2024-09-15T21:30:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4994126360 con 0x7f49941235a0 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c010900 con 0x7f49941235a0 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f498affd640 10 monclient: dump: 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f4998cec640 4 set_mon_vals no callback set 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: _renew_subs 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.828+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4994125ab0 con 0x7f49941235a0 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f498c01c070 con 0x7f49941235a0 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499c753640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499c753640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f498affd640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f497803b9c0 0x7f497803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4994134e70 con 0x7f49941235a0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f498c04faf0 con 0x7f49941235a0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f497803b9c0 0x7f497803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499a4ef640 10 monclient: get_auth_request con 0x7f497803b9c0 auth_method 0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.832+0000 7f499a4ef640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f497803b9c0 0x7f497803de80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4990007f70 tx=0x7f49900014b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.836+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f49941342e0 con 0x7f49941235a0 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.836+0000 7f498affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:04.836+0000 7f498affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:05.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f499c753640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:30:05.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f499c753640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:30:05.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f499c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:05.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f49941245c0 con 0x7f49941235a0 2024-09-15T21:30:05.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f498affd640 1 -- 172.21.15.17:0/1368804073 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f498c0133e0 con 0x7f49941235a0 2024-09-15T21:30:05.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f498affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:30:05.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.068+0000 7f498affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:05.073 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:30:05.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f497803b9c0 msgr2=0x7f497803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f497803b9c0 0x7f497803de80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4990007f70 tx=0x7f49900014b0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 10 monclient: shutdown 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49941235a0 msgr2=0x7f49941d6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49941235a0 0x7f49941d6080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f499409e300 tx=0x7f498c00e8a0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 shutdown_connections 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f497803b9c0 0x7f497803de80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49941235a0 0x7f49941d6080 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4994122c00 0x7f4994123060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 --2- 172.21.15.17:0/1368804073 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4980003a60 0x7f4994125120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 >> 172.21.15.17:0/1368804073 conn(0x7f49940a66a0 msgr2=0x7f4994137ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 shutdown_connections 2024-09-15T21:30:05.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.072+0000 7f499c753640 1 -- 172.21.15.17:0/1368804073 wait complete. 2024-09-15T21:30:05.090 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:30:05.090 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:30:05.090 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:30:05.090 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:30:05.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.208+0000 7f114a98f640 10 monclient: get_monmap_and_config 2024-09-15T21:30:05.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.208+0000 7f114a98f640 10 monclient: build_initial_monmap 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.208+0000 7f114a98f640 10 monclient: monmap: 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:05.213898+0000 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:05.213898+0000 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:05.215 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:05.215 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 Processor -- start 2024-09-15T21:30:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 -- start start 2024-09-15T21:30:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient: init 2024-09-15T21:30:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f114409d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f114409def0 0x7f1144139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): picked mon.noname-c con 0x7f114409def0 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1144139ff0 0x7f114413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): picked mon.noname-b con 0x7f1144139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11440a3130 con 0x7f114409d400 2024-09-15T21:30:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f11440a1bc0 con 0x7f114409def0 2024-09-15T21:30:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11440a3490 con 0x7f1144139ff0 2024-09-15T21:30:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f114a98f640 10 monclient(hunting): authenticate will time out at 1728.609359s 2024-09-15T21:30:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1143fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f114409d400 0x7f114409d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35770/0 (socket says 172.21.15.17:35770) 2024-09-15T21:30:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1144139ff0 0x7f114413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1143fff640 1 -- 172.21.15.17:0/3760190604 learned_addr learned my addr 172.21.15.17:0/3760190604 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): get_auth_request con 0x7f1144139ff0 auth_method 0 2024-09-15T21:30:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f11437fe640 1 --2- 172.21.15.17:0/3760190604 >> v2:172.21.15.119:3300/0 conn(0x7f114409def0 0x7f1144139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f11437fe640 10 monclient(hunting): get_auth_request con 0x7f114409def0 auth_method 0 2024-09-15T21:30:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f11437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f11437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f11437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient(hunting): handle_auth_done global_id 134342 payload 274 2024-09-15T21:30:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 -- 172.21.15.17:0/3760190604 >> v2:172.21.15.119:3300/0 conn(0x7f114409def0 msgr2=0x7f1144139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 --2- 172.21.15.17:0/3760190604 >> v2:172.21.15.119:3300/0 conn(0x7f114409def0 0x7f1144139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 -- 172.21.15.17:0/3760190604 >> v1:172.21.15.17:6789/0 conn(0x7f114409d400 legacy=0x7f114409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient: _finish_hunting 0 2024-09-15T21:30:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 monclient: found mon.noname-b 2024-09-15T21:30:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 -- 172.21.15.17:0/3760190604 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114409e7a0 con 0x7f1144139ff0 2024-09-15T21:30:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.212+0000 7f1148f2c640 1 --2- 172.21.15.17:0/3760190604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1144139ff0 0x7f114413c3e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f113c0097c0 tx=0x7f113c01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9793e71d80074e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 1 -- 172.21.15.17:0/3760190604 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0b0070 con 0x7f1144139ff0 2024-09-15T21:30:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:30:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: dump: 2024-09-15T21:30:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.231 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.231 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: _finish_auth 0 2024-09-15T21:30:05.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:30:05.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:05.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 1 -- 172.21.15.17:0/3760190604 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c0ab600 con 0x7f1144139ff0 2024-09-15T21:30:05.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 5 monclient: authenticate success, global_id 134342 2024-09-15T21:30:05.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:05.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:05.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 1 -- 172.21.15.17:0/3760190604 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0ab9d0 con 0x7f1144139ff0 2024-09-15T21:30:05.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:05.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1142ffd640 10 monclient: dump: 2024-09-15T21:30:05.233 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.233 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.233 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.234 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.234 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.234 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.234 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.234 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.234 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.235 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 4 set_mon_vals no callback set 2024-09-15T21:30:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: shutdown 2024-09-15T21:30:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/3760190604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1144139ff0 msgr2=0x7f114413c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 --2- 172.21.15.17:0/3760190604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1144139ff0 0x7f114413c3e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f113c0097c0 tx=0x7f113c01c360 comp rx=0 tx=0).stop 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/3760190604 shutdown_connections 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 --2- 172.21.15.17:0/3760190604 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1144139ff0 0x7f114413c3e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 --2- 172.21.15.17:0/3760190604 >> v2:172.21.15.119:3300/0 conn(0x7f114409def0 0x7f1144139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/3760190604 >> 172.21.15.17:0/3760190604 conn(0x7f11440a66a0 msgr2=0x7f11440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/3760190604 shutdown_connections 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/3760190604 wait complete. 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: build_initial_monmap 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: monmap: 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 Processor -- start 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- start start 2024-09-15T21:30:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: init 2024-09-15T21:30:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114409d400 0x7f11441c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): picked mon.noname-b con 0x7f114409d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f114409def0 0x7f11440738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): picked mon.noname-c con 0x7f114409def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144139ff0 0x7f11441cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): picked mon.noname-a con 0x7f1144139ff0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11440a0890 con 0x7f1144139ff0 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11440a0a10 con 0x7f114409d400 2024-09-15T21:30:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11440a0b90 con 0x7f114409def0 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient(hunting): authenticate will time out at 1728.614374s 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f114409def0 0x7f11440738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f114409def0 0x7f11440738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39274/0 (socket says 172.21.15.17:39274) 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 -- 172.21.15.17:0/1681249342 learned_addr learned my addr 172.21.15.17:0/1681249342 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): get_auth_request con 0x7f114409def0 auth_method 0 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144139ff0 0x7f11441cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): get_auth_request con 0x7f1144139ff0 auth_method 0 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1143fff640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114409d400 0x7f11441c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1143fff640 10 monclient(hunting): get_auth_request con 0x7f114409d400 auth_method 0 2024-09-15T21:30:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1143fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1143fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1143fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1148f2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient(hunting): handle_auth_done global_id 134345 payload 824 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114409d400 msgr2=0x7f11441c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114409d400 0x7f11441c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144139ff0 msgr2=0x7f11441cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144139ff0 0x7f11441cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient: _finish_hunting 0 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 monclient: found mon.noname-c 2024-09-15T21:30:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1144139710 con 0x7f114409def0 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11437fe640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f114409def0 0x7f11440738c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f112c00d830 tx=0x7f112c00dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c00b6c0 con 0x7f114409def0 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: dump: 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: _finish_auth 0 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 5 monclient: authenticate success, global_id 134345 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c019070 con 0x7f114409def0 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: _renew_subs 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1144137e10 con 0x7f114409def0 2024-09-15T21:30:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c00ba60 con 0x7f114409def0 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f1140ff9640 10 monclient: dump: 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: _renew_subs 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f11427fc640 4 set_mon_vals no callback set 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.216+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1144138250 con 0x7f114409def0 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f114a98f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f114a98f640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 0x7f11441cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f114409fc10 con 0x7f1144073e00 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 0x7f11441cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient: get_auth_request con 0x7f1144073e00 auth_method 0 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f112c00bc00 con 0x7f114409def0 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 10 monclient(hunting): handle_auth_done global_id 144270 payload 824 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f112403b9c0 0x7f112403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f114409d7e0 con 0x7f1144073e00 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f112c050a90 con 0x7f114409def0 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1143fff640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 0x7f11441cb960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f113400ec20 tx=0x7f113400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134005df0 con 0x7f1144073e00 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 10 monclient: dump: 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.255 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1148f2c640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f112403b9c0 0x7f112403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1148f2c640 10 monclient: get_auth_request con 0x7f112403b9c0 auth_method 0 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1134018070 con 0x7f1144073e00 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 msgr2=0x7f11441cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 0x7f11441cb960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f113400ec20 tx=0x7f113400c4b0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 msgr2=0x7f11441cb960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1140ff9640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1144073e00 0x7f11441cb960 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f113400ec20 tx=0x7f113400c4b0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.220+0000 7f1148f2c640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f112403b9c0 0x7f112403de80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f113c0ad000 tx=0x7f113c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f114a98f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f114a98f640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 0x7f1144131ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f114a98f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11440a5450 con 0x7f11441cbea0 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 0x7f1144131ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient: get_auth_request con 0x7f11441cbea0 auth_method 0 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 10 monclient(hunting): handle_auth_done global_id 144273 payload 824 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 1 -- 172.21.15.17:0/1681249342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f11441cc280 con 0x7f11441cbea0 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1143fff640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 0x7f1144131ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f113400ec20 tx=0x7f1134001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11340072d0 con 0x7f11441cbea0 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1140ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1140ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.224+0000 7f1140ff9640 10 monclient: dump: 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.258 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4881 (secure 0 0 0) 0x7f11340072d0 con 0x7f11441cbea0 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 msgr2=0x7f1144131ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 0x7f1144131ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f113400ec20 tx=0x7f1134001260 comp rx=0 tx=0).stop 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 msgr2=0x7f1144131ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:05.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1140ff9640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11441cbea0 0x7f1144131ad0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f113400ec20 tx=0x7f1134001260 comp rx=0 tx=0).stop 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 76951, 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:30:05.260 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.261 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.262 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:05.263 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:05.264 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.265 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.266 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.266 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:05.269 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.274 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.275 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:05.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.276 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.276 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:05.276 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f112403b9c0 msgr2=0x7f112403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f112403b9c0 0x7f112403de80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f113c0ad000 tx=0x7f113c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:30:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 10 monclient: shutdown 2024-09-15T21:30:05.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f114409def0 msgr2=0x7f11440738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 --2- 172.21.15.17:0/1681249342 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f114409def0 0x7f11440738c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f112c00d830 tx=0x7f112c00dcf0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1148f2c640 1 -- 172.21.15.17:0/1681249342 reap_dead start 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f1148f2c640 1 -- 172.21.15.17:0/1681249342 reap_dead start 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 shutdown_connections 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 >> 172.21.15.17:0/1681249342 conn(0x7f11440a66a0 msgr2=0x7f114413bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 shutdown_connections 2024-09-15T21:30:05.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.228+0000 7f114a98f640 1 -- 172.21.15.17:0/1681249342 wait complete. 2024-09-15T21:30:05.278 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:30:05.278 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:30:05.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient: get_monmap_and_config 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient: build_initial_monmap 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient: monmap: 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:05.394305+0000 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:05.394305+0000 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:05.395 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 Processor -- start 2024-09-15T21:30:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 -- start start 2024-09-15T21:30:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient: init 2024-09-15T21:30:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe3980a4c00 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe39809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): picked mon.noname-b con 0x7fe3980a56f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe39813ce10 0x7fe39813f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): picked mon.noname-a con 0x7fe39813ce10 addr v2:172.21.15.17:3300/0 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39809c700 con 0x7fe3980a4c00 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe39806f820 con 0x7fe39813ce10 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe3980a5fc0 con 0x7fe3980a56f0 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39c8a1640 10 monclient(hunting): authenticate will time out at 1728.789657s 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe3980a4c00 0x7fe3980a4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35786/0 (socket says 172.21.15.17:35786) 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39659c640 1 -- 172.21.15.17:0/2667600806 learned_addr learned my addr 172.21.15.17:0/2667600806 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 1 --2- 172.21.15.17:0/2667600806 >> v2:172.21.15.17:3300/0 conn(0x7fe39813ce10 0x7fe39813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): get_auth_request con 0x7fe39813ce10 auth_method 0 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 1 --2- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe39809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): get_auth_request con 0x7fe3980a56f0 auth_method 0 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe395d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient(hunting): handle_auth_done global_id 144276 payload 274 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 1 -- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 msgr2=0x7fe39809cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 1 --2- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe39809cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 1 -- 172.21.15.17:0/2667600806 >> v1:172.21.15.17:6789/0 conn(0x7fe3980a4c00 legacy=0x7fe3980a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 1 monclient: found mon.noname-a 2024-09-15T21:30:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39559a640 1 -- 172.21.15.17:0/2667600806 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155904536 0 0) 0x7fe39809c700 con 0x7fe3980a4c00 2024-09-15T21:30:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe396d9d640 1 -- 172.21.15.17:0/2667600806 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3980a5d10 con 0x7fe39813ce10 2024-09-15T21:30:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.392+0000 7fe39559a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 --2- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe39813f200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1866a98b966fbf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 1 -- 172.21.15.17:0/2667600806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900b2070 con 0x7fe39813ce10 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: dump: 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.407 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: _finish_auth 0 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: _check_auth_tickets 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 5 monclient: authenticate success, global_id 144276 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 1 -- 172.21.15.17:0/2667600806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3900a5780 con 0x7fe39813ce10 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 1 -- 172.21.15.17:0/2667600806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900a5b50 con 0x7fe39813ce10 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:05.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39559a640 10 monclient: dump: 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 4 set_mon_vals no callback set 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: shutdown 2024-09-15T21:30:05.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 msgr2=0x7fe39813f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe39813f200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c8c0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- 172.21.15.17:0/2667600806 shutdown_connections 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe39813f200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/2667600806 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe39809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- 172.21.15.17:0/2667600806 >> 172.21.15.17:0/2667600806 conn(0x7fe3980aaeb0 msgr2=0x7fe3980a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- 172.21.15.17:0/2667600806 shutdown_connections 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- 172.21.15.17:0/2667600806 wait complete. 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: build_initial_monmap 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: monmap: 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:05.410 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 Processor -- start 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- start start 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: init 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980a4c00 0x7fe398120d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): picked mon.noname-b con 0x7fe3980a4c00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:05.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe398137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): picked mon.noname-c con 0x7fe3980a56f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe3981cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): picked mon.noname-a con 0x7fe39813ce10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981422f0 con 0x7fe39813ce10 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe398142170 con 0x7fe3980a4c00 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe398142470 con 0x7fe3980a56f0 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 10 monclient(hunting): authenticate will time out at 1728.794847s 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe398137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe3981cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe398137bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39284/0 (socket says 172.21.15.17:39284) 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 1 -- 172.21.15.17:0/433201796 learned_addr learned my addr 172.21.15.17:0/433201796 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): get_auth_request con 0x7fe3980a56f0 auth_method 0 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): get_auth_request con 0x7fe39813ce10 auth_method 0 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39659c640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980a4c00 0x7fe398120d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39659c640 10 monclient(hunting): get_auth_request con 0x7fe3980a4c00 auth_method 0 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient(hunting): handle_auth_done global_id 144282 payload 824 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 msgr2=0x7fe398137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe398137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980a4c00 msgr2=0x7fe398120d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980a4c00 0x7fe398120d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 monclient: found mon.noname-a 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe398122560 con 0x7fe39813ce10 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe395d9b640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe3980a56f0 0x7fe398137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:30:05.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe396d9d640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe3981cf480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ac000 tx=0x7fe390019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900b2050 con 0x7fe39813ce10 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: dump: 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.418 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: _finish_auth 0 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3900b3040 con 0x7fe39813ce10 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe39c8a1640 5 monclient: authenticate success, global_id 144282 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390009450 con 0x7fe39813ce10 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe394d99640 4 set_mon_vals no callback set 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.396+0000 7fe386ffd640 10 monclient: dump: 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 10 monclient: _renew_subs 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe398138870 con 0x7fe39813ce10 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 10 monclient: _renew_subs 2024-09-15T21:30:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe398121fa0 con 0x7fe39813ce10 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 0x7fe39802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe39806a080 con 0x7fe398138110 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 0x7fe39802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient: get_auth_request con 0x7fe398138110 auth_method 0 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe3900adca0 con 0x7fe39813ce10 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe37803b9c0 0x7fe37803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 10 monclient(hunting): handle_auth_done global_id 134354 payload 824 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe39809cb20 con 0x7fe398138110 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe3900eaff0 con 0x7fe39813ce10 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe39659c640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 0x7fe39802d450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe38c000f70 tx=0x7fe38c0019c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c012590 con 0x7fe398138110 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 10 monclient: dump: 2024-09-15T21:30:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe395d9b640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe37803b9c0 0x7fe37803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe395d9b640 10 monclient: get_auth_request con 0x7fe37803b9c0 auth_method 0 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe38c018070 con 0x7fe398138110 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 msgr2=0x7fe39802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 0x7fe39802d450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe38c000f70 tx=0x7fe38c0019c0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 msgr2=0x7fe39802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe386ffd640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe398138110 0x7fe39802d450 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe38c000f70 tx=0x7fe38c0019c0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.400+0000 7fe395d9b640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe37803b9c0 0x7fe37803de80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe398120cf0 tx=0x7fe388009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.404+0000 7fe39c8a1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.404+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 0x7fe39802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.404+0000 7fe39c8a1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.404+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe39809f810 con 0x7fe39813b1e0 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 0x7fe39802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient: get_auth_request con 0x7fe39813b1e0 auth_method 0 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 10 monclient(hunting): handle_auth_done global_id 134357 payload 824 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 1 -- 172.21.15.17:0/433201796 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe398122170 con 0x7fe39813b1e0 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39659c640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 0x7fe39802d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe38c017000 tx=0x7fe38c001090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c012bb0 con 0x7fe39813b1e0 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 10 monclient: dump: 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7fe38c012bb0 con 0x7fe39813b1e0 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 msgr2=0x7fe39802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 0x7fe39802d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe38c017000 tx=0x7fe38c001090 comp rx=0 tx=0).stop 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 msgr2=0x7fe39802d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe386ffd640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe39813b1e0 0x7fe39802d990 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe38c017000 tx=0x7fe38c001090 comp rx=0 tx=0).stop 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-15T21:30:05.429 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 122562, 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 6, 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:05.442 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe37803b9c0 msgr2=0x7fe37803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe37803b9c0 0x7fe37803de80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe398120cf0 tx=0x7fe388009190 comp rx=0 tx=0).stop 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 10 monclient: shutdown 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 msgr2=0x7fe3981cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 --2- 172.21.15.17:0/433201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe39813ce10 0x7fe3981cf480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ac000 tx=0x7fe390019660 comp rx=0 tx=0).stop 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe396d9d640 1 -- 172.21.15.17:0/433201796 reap_dead start 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe396d9d640 1 -- 172.21.15.17:0/433201796 reap_dead start 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 shutdown_connections 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 >> 172.21.15.17:0/433201796 conn(0x7fe3980aaeb0 msgr2=0x7fe3980a37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 shutdown_connections 2024-09-15T21:30:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.408+0000 7fe39c8a1640 1 -- 172.21.15.17:0/433201796 wait complete. 2024-09-15T21:30:05.445 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:30:05.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:30:05.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.564+0000 7f7667d60640 10 monclient: get_monmap_and_config 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.564+0000 7f7667d60640 10 monclient: build_initial_monmap 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.564+0000 7f7667d60640 10 monclient: monmap: 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:05.569364+0000 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:05.569364+0000 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:05.570 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.564+0000 7f7667d60640 1 Processor -- start 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 1 -- start start 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient: init 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f766009c6a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7660137c60 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:05.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 0x7f76601401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient(hunting): picked mon.noname-b con 0x7f7660138040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7660074840 con 0x7f766009c6a0 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76600742d0 con 0x7f7660137c60 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f766009a860 con 0x7f7660138040 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient(hunting): authenticate will time out at 1728.964803s 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 0x7f76601401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 0x7f76601401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39302/0 (socket says 172.21.15.17:39302) 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 -- 172.21.15.17:0/466564423 learned_addr learned my addr 172.21.15.17:0/466564423 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): get_auth_request con 0x7f7660138040 auth_method 0 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient(hunting): handle_auth_done global_id 134360 payload 274 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 -- 172.21.15.17:0/466564423 >> v1:172.21.15.119:6789/0 conn(0x7f7660137c60 legacy=0x7f766013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 -- 172.21.15.17:0/466564423 >> v1:172.21.15.17:6789/0 conn(0x7f766009c6a0 legacy=0x7f766009ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient: _finish_hunting 0 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 1 -- 172.21.15.17:0/466564423 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205207139 0 0) 0x7f7660074840 con 0x7f766009c6a0 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 monclient: found mon.noname-b 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 -- 172.21.15.17:0/466564423 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766009a580 con 0x7f7660138040 2024-09-15T21:30:05.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:30:05.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f76662fd640 1 --2- 172.21.15.17:0/466564423 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 0x7f76601401f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f766009a9e0 tx=0x7f765001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71f9c405c1b994c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 1 -- 172.21.15.17:0/466564423 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76500b0070 con 0x7f7660138040 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: dump: 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.582 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: _finish_auth 0 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: _check_auth_tickets 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 5 monclient: authenticate success, global_id 134360 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 1 -- 172.21.15.17:0/466564423 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76500ab600 con 0x7f7660138040 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 1 -- 172.21.15.17:0/466564423 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76500ab9d0 con 0x7f7660138040 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:05.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7664afa640 10 monclient: dump: 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.568+0000 7f7667d60640 4 set_mon_vals no callback set 2024-09-15T21:30:05.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: shutdown 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/466564423 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 msgr2=0x7f76601401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 --2- 172.21.15.17:0/466564423 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 0x7f76601401f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f766009a9e0 tx=0x7f765001c440 comp rx=0 tx=0).stop 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/466564423 shutdown_connections 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 --2- 172.21.15.17:0/466564423 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7660138040 0x7f76601401f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/466564423 >> 172.21.15.17:0/466564423 conn(0x7f76600a2c50 msgr2=0x7f76600a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/466564423 shutdown_connections 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/466564423 wait complete. 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: build_initial_monmap 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: monmap: 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:05.585 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 Processor -- start 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- start start 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: init 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f766009c6a0 0x7f7660134590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): picked mon.noname-c con 0x7f766009c6a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660137c60 0x7f766013c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): picked mon.noname-b con 0x7f7660137c60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7660138040 0x7f766013cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): picked mon.noname-a con 0x7f7660138040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f76601432e0 con 0x7f7660138040 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7660143160 con 0x7f7660137c60 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7660143460 con 0x7f766009c6a0 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient(hunting): authenticate will time out at 1728.970512s 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f766009c6a0 0x7f7660134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7660138040 0x7f766013cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f766009c6a0 0x7f7660134590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39312/0 (socket says 172.21.15.17:39312) 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 -- 172.21.15.17:0/1260960014 learned_addr learned my addr 172.21.15.17:0/1260960014 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): get_auth_request con 0x7f766009c6a0 auth_method 0 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): get_auth_request con 0x7f7660138040 auth_method 0 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76662fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76652fb640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660137c60 0x7f766013c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76652fb640 10 monclient(hunting): get_auth_request con 0x7f7660137c60 auth_method 0 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76652fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76652fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76652fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient(hunting): handle_auth_done global_id 134363 payload 824 2024-09-15T21:30:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660137c60 msgr2=0x7f766013c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660137c60 0x7f766013c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7660138040 msgr2=0x7f766013cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7660138040 0x7f766013cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient: _finish_hunting 0 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 monclient: found mon.noname-c 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7660136b80 con 0x7f766009c6a0 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7665afc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f766009c6a0 0x7f7660134590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f765800c450 tx=0x7f765800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658005480 con 0x7f766009c6a0 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:30:05.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: dump: 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: _finish_auth 0 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: _check_auth_tickets 2024-09-15T21:30:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 5 monclient: authenticate success, global_id 134363 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: _renew_subs 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658003600 con 0x7f766009c6a0 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766009d340 con 0x7f766009c6a0 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76580039d0 con 0x7f766009c6a0 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f76567fc640 10 monclient: dump: 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: _renew_subs 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7657fff640 4 set_mon_vals no callback set 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.572+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f766013ffc0 con 0x7f766009c6a0 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f7667d60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f7667d60640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 0x7f766013d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f7667d60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f766013eb60 con 0x7f76601c88f0 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7658003b70 con 0x7f766009c6a0 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764c03b9f0 0x7f764c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 0x7f766013d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient: get_auth_request con 0x7f76601c88f0 auth_method 0 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7658050130 con 0x7f766009c6a0 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76662fd640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764c03b9f0 0x7f764c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76662fd640 10 monclient: get_auth_request con 0x7f764c03b9f0 auth_method 0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 10 monclient(hunting): handle_auth_done global_id 154153 payload 824 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f766009ca80 con 0x7f76601c88f0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76652fb640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 0x7f766013d060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f766013ea30 tx=0x7f765c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c003b90 con 0x7f76601c88f0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76662fd640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764c03b9f0 0x7f764c03deb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76500ad000 tx=0x7f76500a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 10 monclient: dump: 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.595 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f765c018070 con 0x7f76601c88f0 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 msgr2=0x7f766013d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 0x7f766013d060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f766013ea30 tx=0x7f765c00ecd0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 msgr2=0x7f766013d060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:05.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.576+0000 7f76567fc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601c88f0 0x7f766013d060 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:05.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 0x7f766013d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f766006d420 con 0x7f7660134ad0 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 0x7f766013d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient: get_auth_request con 0x7f7660134ad0 auth_method 0 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 10 monclient(hunting): handle_auth_done global_id 154156 payload 824 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 1 -- 172.21.15.17:0/1260960014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f76601da7f0 con 0x7f7660134ad0 2024-09-15T21:30:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76652fb640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 0x7f766013d5a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f765c017000 tx=0x7f765c0014b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c006b60 con 0x7f7660134ad0 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 10 monclient: dump: 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f765c006b60 con 0x7f7660134ad0 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 msgr2=0x7f766013d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 0x7f766013d5a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f765c017000 tx=0x7f765c0014b0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 msgr2=0x7f766013d5a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76567fc640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660134ad0 0x7f766013d5a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:05.601 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10119, 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 6, 2024-09-15T21:30:05.602 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.603 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:05.614 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764c03b9f0 msgr2=0x7f764c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f764c03b9f0 0x7f764c03deb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76500ad000 tx=0x7f76500a6000 comp rx=0 tx=0).stop 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 10 monclient: shutdown 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f766009c6a0 msgr2=0x7f7660134590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 --2- 172.21.15.17:0/1260960014 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f766009c6a0 0x7f7660134590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f765800c450 tx=0x7f765800c910 comp rx=0 tx=0).stop 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76662fd640 1 -- 172.21.15.17:0/1260960014 reap_dead start 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f76662fd640 1 -- 172.21.15.17:0/1260960014 reap_dead start 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 shutdown_connections 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 >> 172.21.15.17:0/1260960014 conn(0x7f76600a2c50 msgr2=0x7f7660135950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 shutdown_connections 2024-09-15T21:30:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:05.584+0000 7f7667d60640 1 -- 172.21.15.17:0/1260960014 wait complete. 2024-09-15T21:30:05.617 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:30:05.620 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T21:30:05.620 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-15T21:30:05.620 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T21:30:05.620 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T21:30:05.620 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:30:05.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:30:05.634 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:30:05.634 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T21:30:05.634 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T21:30:05.634 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:30:25.635 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T21:30:25.636 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T21:30:25.636 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:30:25.637 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:30:25.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient: get_monmap_and_config 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient: build_initial_monmap 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient: monmap: 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:25.768044+0000 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:25.768044+0000 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:25.769 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:25.769 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:25.769 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 Processor -- start 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 -- start start 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient: init 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): picked mon.noname-c con 0x7fb3681340c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): picked mon.noname-a con 0x7fb36809d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb3680a0dd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3681349e0 con 0x7fb3680a0dd0 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb36809d520 con 0x7fb36809d6c0 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb36809e1b0 con 0x7fb3681340c0 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb3706e9640 10 monclient(hunting): authenticate will time out at 1749.163399s 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36ec86640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb3680a0dd0 0x7fb36813ae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38728/0 (socket says 172.21.15.17:38728) 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44256/0 (socket says 172.21.15.17:44256) 2024-09-15T21:30:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36ec86640 1 -- 172.21.15.17:0/3215986772 learned_addr learned my addr 172.21.15.17:0/3215986772 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 10 monclient(hunting): get_auth_request con 0x7fb36809d6c0 auth_method 0 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36e485640 10 monclient(hunting): get_auth_request con 0x7fb3681340c0 auth_method 0 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36e485640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36e485640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:25.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.764+0000 7fb36e485640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.768+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.768+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.768+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:25.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.768+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 10 monclient(hunting): get_auth_request con 0x7fb36809d6c0 auth_method 0 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.968+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:30:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 10 monclient(hunting): get_auth_request con 0x7fb3681340c0 auth_method 0 2024-09-15T21:30:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:25.972+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 10 monclient(hunting): get_auth_request con 0x7fb36809d6c0 auth_method 0 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.372+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 10 monclient(hunting): get_auth_request con 0x7fb3681340c0 auth_method 0 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:26.376+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:30:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 10 monclient(hunting): get_auth_request con 0x7fb36809d6c0 auth_method 0 2024-09-15T21:30:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:27.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:27.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:27.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:27.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.176+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:30:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 10 monclient(hunting): get_auth_request con 0x7fb3681340c0 auth_method 0 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:30:27.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:27.180+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:30:28.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.288+0000 7fb36ec86640 1 -- 172.21.15.17:0/3215986772 >> v1:172.21.15.17:6789/0 conn(0x7fb3680a0dd0 legacy=0x7fb36813ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T21:30:28.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.288+0000 7fb36ec86640 1 -- 172.21.15.17:0/3215986772 >> v1:172.21.15.17:6789/0 conn(0x7fb3680a0dd0 legacy=0x7fb36813ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:30:28.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.288+0000 7fb36ec86640 1 --1- 172.21.15.17:0/3215986772 >> v1:172.21.15.17:6789/0 conn(0x7fb3680a0dd0 0x7fb36813ae60 :-1 s=OPENED pgs=120 cs=1 l=1).handle_message read tag failed 2024-09-15T21:30:28.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.288+0000 7fb36ec86640 1 --1- 172.21.15.17:0/3215986772 >> v1:172.21.15.17:6789/0 conn(0x7fb3680a0dd0 0x7fb36813ae60 :-1 s=OPENED pgs=120 cs=1 l=1).fault on lossy channel, failing 2024-09-15T21:30:28.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.288+0000 7fb36d483640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.17:6789/0 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): tick 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 monclient(hunting): continuing hunt 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 msgr2=0x7fb3681344a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.119:3300/0 conn(0x7fb3681340c0 0x7fb3681344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 -- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 msgr2=0x7fb3680a0890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 --2- 172.21.15.17:0/3215986772 >> v2:172.21.15.17:3300/0 conn(0x7fb36809d6c0 0x7fb3680a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 -- 172.21.15.17:0/3215986772 >> v1:172.21.15.17:6789/0 conn(0x7fb3680a0dd0 legacy=0x7fb36813ae60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient: _add_conns ranks=[1,2] 2024-09-15T21:30:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb34c005830 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 0x7fb34c00b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): picked mon.noname-b con 0x7fb34c0094d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 -- 172.21.15.17:0/3215986772 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb36809e490 con 0x7fb34c005830 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 1 -- 172.21.15.17:0/3215986772 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb36809d520 con 0x7fb34c0094d0 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb35bfff640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 msgr2=0x7fb34c00b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.768+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 0x7fb34c00b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:28.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.972+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 msgr2=0x7fb34c00b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:28.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:28.972+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 0x7fb34c00b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:30:29.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:29.376+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 msgr2=0x7fb34c00b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:29.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:29.376+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 0x7fb34c00b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:30:30.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.180+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 msgr2=0x7fb34c00b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:30.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.180+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 0x7fb34c00b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:30:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.688+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218883392 0 0) 0x7fb36809e490 con 0x7fb34c005830 2024-09-15T21:30:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.688+0000 7fb36d483640 10 monclient(hunting): _negotiate 2024-09-15T21:30:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.688+0000 7fb36d483640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.688+0000 7fb36d483640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.688+0000 7fb36d483640 10 monclient(hunting): my global_id is 154159 2024-09-15T21:30:30.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.688+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb354003530 con 0x7fb34c005830 2024-09-15T21:30:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3284038311 0 0) 0x7fb354003530 con 0x7fb34c005830 2024-09-15T21:30:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 msgr2=0x7fb34c00b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:30:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb34c0094d0 0x7fb34c00b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: _finish_hunting 0 2024-09-15T21:30:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:30:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:30:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c00d1b0 con 0x7fb34c005830 2024-09-15T21:30:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: _finish_auth 0 2024-09-15T21:30:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: _check_auth_tickets 2024-09-15T21:30:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:30.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb36400f040 con 0x7fb34c005830 2024-09-15T21:30:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb3706e9640 5 monclient: authenticate success, global_id 154159 2024-09-15T21:30:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb3706e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:30:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: dump: 2024-09-15T21:30:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:30.720 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:30:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 >> v1:172.21.15.119:6789/0 conn(0x7fb34c005830 legacy=0x7fb34c007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 1 -- 172.21.15.17:0/3215986772 reap_dead start 2024-09-15T21:30:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb354003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): picked mon.c con 0x7fb3540039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb354004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): picked mon.b con 0x7fb354004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb35400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): picked mon.a con 0x7fb3540053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb36809d520 con 0x7fb3540053d0 2024-09-15T21:30:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb35400c270 con 0x7fb354004810 2024-09-15T21:30:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb35400c470 con 0x7fb3540039c0 2024-09-15T21:30:30.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:30.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36d483640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:30:30.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb3706e9640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:30.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36e485640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 msgr2=0x7fb354003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:30.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36e485640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb354003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:30.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb354004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:30.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb35400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:30.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): get_auth_request con 0x7fb354004810 auth_method 0 2024-09-15T21:30:30.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:30.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:30.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): get_auth_request con 0x7fb3540053d0 auth_method 0 2024-09-15T21:30:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:30.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36dc84640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:30:30.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:30.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:30.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.696+0000 7fb36ec86640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:30:30.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 10 monclient(hunting): handle_auth_done global_id 154159 payload 293 2024-09-15T21:30:30.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 msgr2=0x7fb354003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:30:30.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb354003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 msgr2=0x7fb35400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:30.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb35400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 10 monclient: _finish_hunting 0 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 monclient: found mon.b 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 -- 172.21.15.17:0/3215986772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb354003530 con 0x7fb354004810 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 10 monclient: _finish_auth 0 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 10 monclient: _check_auth_tickets 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb354004c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb35c000ee0 tx=0x7fb35c01d4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e04ced8db3cfcb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0b3070 con 0x7fb354004810 2024-09-15T21:30:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: dump: 2024-09-15T21:30:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:30.732 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:30.732 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:30.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb35c004030 con 0x7fb354004810 2024-09-15T21:30:30.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb3706e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:30.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:30.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb3706e9640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:30.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb3706e9640 4 set_mon_vals no callback set 2024-09-15T21:30:30.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 1 -- 172.21.15.17:0/3215986772 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00f040 con 0x7fb354004810 2024-09-15T21:30:30.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:30.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:30.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.700+0000 7fb36d483640 10 monclient: dump: 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient: shutdown 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- 172.21.15.17:0/3215986772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 msgr2=0x7fb354004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb354004c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb35c000ee0 tx=0x7fb35c01d4d0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- 172.21.15.17:0/3215986772 shutdown_connections 2024-09-15T21:30:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb35400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb354004c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3215986772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb354003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- 172.21.15.17:0/3215986772 >> 172.21.15.17:0/3215986772 conn(0x7fb3680a66a0 msgr2=0x7fb3680a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- 172.21.15.17:0/3215986772 shutdown_connections 2024-09-15T21:30:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- 172.21.15.17:0/3215986772 wait complete. 2024-09-15T21:30:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient: build_initial_monmap 2024-09-15T21:30:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient: monmap: 2024-09-15T21:30:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:30.739 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:30.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 Processor -- start 2024-09-15T21:30:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- start start 2024-09-15T21:30:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient: init 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb368132ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): picked mon.noname-c con 0x7fb3540039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb36811c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): picked mon.noname-b con 0x7fb354004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb36811ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): picked mon.noname-a con 0x7fb3540053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb36809ce50 con 0x7fb3540053d0 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb36809ccd0 con 0x7fb354004810 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb3680a2590 con 0x7fb3540039c0 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb3706e9640 10 monclient(hunting): authenticate will time out at 1754.103655s 2024-09-15T21:30:30.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36e485640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 msgr2=0x7fb368132ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36e485640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb368132ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb36811ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb36811ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40154/0 (socket says 172.21.15.17:40154) 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 1 -- 172.21.15.17:0/3169624965 learned_addr learned my addr 172.21.15.17:0/3169624965 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 10 monclient(hunting): get_auth_request con 0x7fb3540053d0 auth_method 0 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.704+0000 7fb36ec86640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb36811c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): get_auth_request con 0x7fb354004810 auth_method 0 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36dc84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 10 monclient(hunting): handle_auth_done global_id 144291 payload 824 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 -- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 msgr2=0x7fb368132ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb368132ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 -- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 msgr2=0x7fb36811c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb36811c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 10 monclient: _finish_hunting 0 2024-09-15T21:30:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 monclient: found mon.noname-a 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 -- 172.21.15.17:0/3169624965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36811f430 con 0x7fb3540053d0 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36ec86640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb36811ced0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb36809eb90 tx=0x7fb360007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36000fab0 con 0x7fb3540053d0 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: dump: 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: _finish_auth 0 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: _check_auth_tickets 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb360018070 con 0x7fb3540053d0 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36000fe90 con 0x7fb3540053d0 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 5 monclient: authenticate success, global_id 144291 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 monclient: _renew_subs 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3681ca7e0 con 0x7fb3540053d0 2024-09-15T21:30:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35bfff640 4 set_mon_vals no callback set 2024-09-15T21:30:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 10 monclient: dump: 2024-09-15T21:30:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 monclient: _renew_subs 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb368120670 con 0x7fb3540053d0 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb36001d460 con 0x7fb3540053d0 2024-09-15T21:30:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:30.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:30.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:30.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3680a0890 con 0x7fb3540053d0 2024-09-15T21:30:30.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb34c03c480 0x7fb34c03e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:30.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb3600506f0 con 0x7fb3540053d0 2024-09-15T21:30:30.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36e485640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb34c03c480 0x7fb34c03e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:30.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36e485640 10 monclient: get_auth_request con 0x7fb34c03c480 auth_method 0 2024-09-15T21:30:30.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.708+0000 7fb36e485640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb34c03c480 0x7fb34c03e940 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb364000ee0 tx=0x7fb364010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:30.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.712+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb3600133f0 con 0x7fb3540053d0 2024-09-15T21:30:30.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.712+0000 7fb35affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:30.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.712+0000 7fb35affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:30.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.952+0000 7fb3706e9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:30:30.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.952+0000 7fb3706e9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:30:30.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.952+0000 7fb3706e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:30.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.952+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb36806d020 con 0x7fb3540053d0 2024-09-15T21:30:30.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb35affd640 1 -- 172.21.15.17:0/3169624965 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7fb36001bc80 con 0x7fb3540053d0 2024-09-15T21:30:30.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb35affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:30:30.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb35affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:30.958 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":124,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:30:30.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb34c03c480 msgr2=0x7fb34c03e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:30.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb34c03c480 0x7fb34c03e940 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb364000ee0 tx=0x7fb364010040 comp rx=0 tx=0).stop 2024-09-15T21:30:30.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 10 monclient: shutdown 2024-09-15T21:30:30.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:30.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 msgr2=0x7fb36811ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:30.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb36811ced0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb36809eb90 tx=0x7fb360007b00 comp rx=0 tx=0).stop 2024-09-15T21:30:30.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 shutdown_connections 2024-09-15T21:30:30.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb34c03c480 0x7fb34c03e940 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3540053d0 0x7fb36811ced0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354004810 0x7fb36811c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 --2- 172.21.15.17:0/3169624965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb3540039c0 0x7fb368132ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:30.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 >> 172.21.15.17:0/3169624965 conn(0x7fb3680a66a0 msgr2=0x7fb36809ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:30.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 shutdown_connections 2024-09-15T21:30:30.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:30.956+0000 7fb3706e9640 1 -- 172.21.15.17:0/3169624965 wait complete. 2024-09-15T21:30:30.979 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:30:30.979 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:30:30.979 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient: get_monmap_and_config 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient: build_initial_monmap 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient: monmap: 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:31.103351+0000 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:31.103351+0000 2024-09-15T21:30:31.103 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:31.104 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 1 Processor -- start 2024-09-15T21:30:31.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 1 -- start start 2024-09-15T21:30:31.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient: init 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5380a0890 0x7fb5380a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient(hunting): picked mon.noname-c con 0x7fb5380a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb53809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb5380a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb538136490 con 0x7fb5380a11b0 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5380a3130 con 0x7fb53809ffc0 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb538073df0 con 0x7fb5380a0890 2024-09-15T21:30:31.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53cd33640 10 monclient(hunting): authenticate will time out at 1754.498701s 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb536d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb5380a11b0 0x7fb538135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51062/0 (socket says 172.21.15.17:51062) 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb536d9d640 1 -- 172.21.15.17:0/943327107 learned_addr learned my addr 172.21.15.17:0/943327107 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 1 --2- 172.21.15.17:0/943327107 >> v2:172.21.15.119:3300/0 conn(0x7fb5380a0890 0x7fb5380a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): get_auth_request con 0x7fb5380a0890 auth_method 0 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790154416 0 0) 0x7fb538136490 con 0x7fb5380a11b0 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): _negotiate 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): my global_id is 144294 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb524003530 con 0x7fb5380a11b0 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3446676645 0 0) 0x7fb524003530 con 0x7fb5380a11b0 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 >> v2:172.21.15.119:3300/0 conn(0x7fb5380a0890 msgr2=0x7fb5380a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 --2- 172.21.15.17:0/943327107 >> v2:172.21.15.119:3300/0 conn(0x7fb5380a0890 0x7fb5380a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 >> v1:172.21.15.119:6789/0 conn(0x7fb53809ffc0 legacy=0x7fb538135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: _finish_hunting 0 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:30:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380a1bf0 con 0x7fb5380a11b0 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: _finish_auth 0 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: _check_auth_tickets 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fb530001700 con 0x7fb5380a11b0 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: dump: 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 >> v1:172.21.15.17:6789/0 conn(0x7fb5380a11b0 legacy=0x7fb538135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524003a60 0x7fb524003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): picked mon.b con 0x7fb524003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb524004810 0x7fb524004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): picked mon.a con 0x7fb524004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5240053d0 0x7fb52400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): picked mon.c con 0x7fb5240053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb538073df0 con 0x7fb524004810 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb52400c240 con 0x7fb524003a60 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb52400c440 con 0x7fb5240053d0 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb53559a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb535d9b640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb524004810 0x7fb524004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb536d9d640 1 -- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5240053d0 msgr2=0x7fb52400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.100+0000 7fb535d9b640 10 monclient(hunting): get_auth_request con 0x7fb524004810 auth_method 0 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb536d9d640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5240053d0 0x7fb52400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53659c640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524003a60 0x7fb524003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53659c640 10 monclient(hunting): get_auth_request con 0x7fb524003a60 auth_method 0 2024-09-15T21:30:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient(hunting): handle_auth_done global_id 144294 payload 293 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 -- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5240053d0 msgr2=0x7fb52400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5240053d0 0x7fb52400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 -- 172.21.15.17:0/943327107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524003a60 msgr2=0x7fb524003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524003a60 0x7fb524003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 monclient: found mon.a 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb536d9d640 1 -- 172.21.15.17:0/943327107 reap_dead start 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 -- 172.21.15.17:0/943327107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb524003530 con 0x7fb524004810 2024-09-15T21:30:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient: _finish_auth 0 2024-09-15T21:30:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:30:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 5 monclient: authenticate success, global_id 144294 2024-09-15T21:30:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb535d9b640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb524004810 0x7fb524004c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb520001790 tx=0x7fb52001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fdbddc618327eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200013e0 con 0x7fb524004810 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: dump: 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5200052d0 con 0x7fb524004810 2024-09-15T21:30:31.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 1 -- 172.21.15.17:0/943327107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520003840 con 0x7fb524004810 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 4 set_mon_vals no callback set 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53559a640 10 monclient: dump: 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 10 monclient: shutdown 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 -- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb524004810 msgr2=0x7fb524004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb524004810 0x7fb524004c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb520001790 tx=0x7fb52001c370 comp rx=0 tx=0).stop 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 -- 172.21.15.17:0/943327107 shutdown_connections 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 --2- 172.21.15.17:0/943327107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb524004810 0x7fb524004c90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 -- 172.21.15.17:0/943327107 >> 172.21.15.17:0/943327107 conn(0x7fb5380a66a0 msgr2=0x7fb5380a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 -- 172.21.15.17:0/943327107 shutdown_connections 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 1 -- 172.21.15.17:0/943327107 wait complete. 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 10 monclient: build_initial_monmap 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.104+0000 7fb53cd33640 10 monclient: monmap: 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 Processor -- start 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- start start 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: init 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524004810 0x7fb5381cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): picked mon.noname-b con 0x7fb524004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5381d0200 0x7fb538124640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): picked mon.noname-c con 0x7fb5381d0200 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381d0680 0x7fb538124b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): picked mon.noname-a con 0x7fb5381d0680 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb538136210 con 0x7fb5381d0680 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb538136090 con 0x7fb524004810 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb538138ab0 con 0x7fb5381d0200 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): authenticate will time out at 1754.504859s 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb535d9b640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5381d0200 msgr2=0x7fb538124640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb535d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5381d0200 0x7fb538124640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381d0680 0x7fb538124b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381d0680 0x7fb538124b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40172/0 (socket says 172.21.15.17:40172) 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 -- 172.21.15.17:0/185567621 learned_addr learned my addr 172.21.15.17:0/185567621 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): get_auth_request con 0x7fb5381d0680 auth_method 0 2024-09-15T21:30:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524004810 0x7fb5381cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): get_auth_request con 0x7fb524004810 auth_method 0 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient(hunting): handle_auth_done global_id 144297 payload 824 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5381d0200 msgr2=0x7fb538124640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:30:31.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb5381d0200 0x7fb538124640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524004810 msgr2=0x7fb5381cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524004810 0x7fb5381cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 monclient: found mon.noname-a 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5381260b0 con 0x7fb5381d0680 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb524004810 0x7fb5381cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb536d9d640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381d0680 0x7fb538124b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb538074910 tx=0x7fb53000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530007440 con 0x7fb5381d0680 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: dump: 2024-09-15T21:30:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: _finish_auth 0 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: _check_auth_tickets 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5300075e0 con 0x7fb5381d0680 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530004a10 con 0x7fb5381d0680 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: dump: 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.133 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 5 monclient: authenticate success, global_id 144297 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: _renew_subs 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb534d99640 4 set_mon_vals no callback set 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb538127b70 con 0x7fb5381d0680 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: _renew_subs 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381349a0 con 0x7fb5381d0680 2024-09-15T21:30:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 0x7fb5381254a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb538136420 con 0x7fb5381250c0 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 0x7fb5381254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient: get_auth_request con 0x7fb5381250c0 auth_method 0 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb530004bb0 con 0x7fb5381d0680 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 10 monclient(hunting): handle_auth_done global_id 144300 payload 824 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5381259e0 con 0x7fb5381250c0 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb51803b9a0 0x7fb51803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb53004f9f0 con 0x7fb5381d0680 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb53659c640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 0x7fb5381254a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb528007f50 tx=0x7fb5280126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52801c020 con 0x7fb5381250c0 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: dump: 2024-09-15T21:30:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb535d9b640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb51803b9a0 0x7fb51803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb535d9b640 10 monclient: get_auth_request con 0x7fb51803b9a0 auth_method 0 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb528018880 con 0x7fb5381250c0 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 msgr2=0x7fb5381254a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 0x7fb5381254a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb528007f50 tx=0x7fb5280126d0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 msgr2=0x7fb5381254a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.108+0000 7fb52effd640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381250c0 0x7fb5381254a0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb528007f50 tx=0x7fb5280126d0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.112+0000 7fb535d9b640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb51803b9a0 0x7fb51803de60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb5200ac000 tx=0x7fb520009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 0x7fb538120d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb538127ee0 con 0x7fb5381208f0 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 0x7fb538120d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient: get_auth_request con 0x7fb5381208f0 auth_method 0 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 10 monclient(hunting): handle_auth_done global_id 144303 payload 824 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 1 -- 172.21.15.17:0/185567621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb538126c10 con 0x7fb5381208f0 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53659c640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 0x7fb538120d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb528007f50 tx=0x7fb528004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52801c030 con 0x7fb5381208f0 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 10 monclient: dump: 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4871 (secure 0 0 0) 0x7fb528021420 con 0x7fb5381208f0 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 msgr2=0x7fb538120d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 0x7fb538120d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb528007f50 tx=0x7fb528004ac0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 msgr2=0x7fb538120d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb52effd640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381208f0 0x7fb538120d00 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb528007f50 tx=0x7fb528004ac0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:31.141 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 124, 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 102842, 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:31.144 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:31.145 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:31.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:31.147 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.148 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:31.150 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.153 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.154 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:31.155 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb51803b9a0 msgr2=0x7fb51803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb51803b9a0 0x7fb51803de60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb5200ac000 tx=0x7fb520009040 comp rx=0 tx=0).stop 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 10 monclient: shutdown 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381d0680 msgr2=0x7fb538124b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 --2- 172.21.15.17:0/185567621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5381d0680 0x7fb538124b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb538074910 tx=0x7fb53000c890 comp rx=0 tx=0).stop 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb536d9d640 1 -- 172.21.15.17:0/185567621 reap_dead start 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb536d9d640 1 -- 172.21.15.17:0/185567621 reap_dead start 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 shutdown_connections 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 >> 172.21.15.17:0/185567621 conn(0x7fb5380a66a0 msgr2=0x7fb538071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 shutdown_connections 2024-09-15T21:30:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.116+0000 7fb53cd33640 1 -- 172.21.15.17:0/185567621 wait complete. 2024-09-15T21:30:31.158 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:30:31.158 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:30:31.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:30:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient: get_monmap_and_config 2024-09-15T21:30:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient: build_initial_monmap 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient: monmap: 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:30:31.274839+0000 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:30:31.274839+0000 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:30:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:30:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:30:31.276 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 Processor -- start 2024-09-15T21:30:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 -- start start 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient: init 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b040a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b0409ffc0 0x7f5b0413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): picked mon.noname-a con 0x7f5b0409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5b040a1380 0x7f5b0413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): picked mon.noname-c con 0x7f5b040a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b040a3130 con 0x7f5b040a0890 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5b04073df0 con 0x7f5b0409ffc0 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5b040a1bf0 con 0x7f5b040a1380 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0bceb640 10 monclient(hunting): authenticate will time out at 1754.670203s 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b0409ffc0 0x7f5b0413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b0409ffc0 0x7f5b0413aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40208/0 (socket says 172.21.15.17:40208) 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 -- 172.21.15.17:0/3045671197 learned_addr learned my addr 172.21.15.17:0/3045671197 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): get_auth_request con 0x7f5b0409ffc0 auth_method 0 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0a288640 1 --2- 172.21.15.17:0/3045671197 >> v2:172.21.15.119:3300/0 conn(0x7f5b040a1380 0x7f5b0413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0a288640 10 monclient(hunting): get_auth_request con 0x7f5b040a1380 auth_method 0 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0a288640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0a288640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b0a288640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient(hunting): handle_auth_done global_id 144306 payload 274 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 -- 172.21.15.17:0/3045671197 >> v2:172.21.15.119:3300/0 conn(0x7f5b040a1380 msgr2=0x7f5b0413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:30:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 --2- 172.21.15.17:0/3045671197 >> v2:172.21.15.119:3300/0 conn(0x7f5b040a1380 0x7f5b0413d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 -- 172.21.15.17:0/3045671197 >> v1:172.21.15.119:6789/0 conn(0x7f5b040a0890 legacy=0x7f5b040a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient: _finish_hunting 0 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 monclient: found mon.noname-a 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.272+0000 7f5b09286640 1 -- 172.21.15.17:0/3045671197 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0409ccd0 con 0x7f5b0409ffc0 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09286640 1 --2- 172.21.15.17:0/3045671197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b0409ffc0 0x7f5b0413aac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009e50 tx=0x7f5b0001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef518cefe8bcc4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 1 -- 172.21.15.17:0/3045671197 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b000a5570 con 0x7f5b0409ffc0 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: dump: 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: _finish_auth 0 2024-09-15T21:30:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: _check_auth_tickets 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 1 -- 172.21.15.17:0/3045671197 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b000a5710 con 0x7f5b0409ffc0 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 5 monclient: authenticate success, global_id 144306 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: get_monmap_and_config success 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 1 -- 172.21.15.17:0/3045671197 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b000a5ae0 con 0x7f5b0409ffc0 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 4 set_mon_vals no callback set 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b08a85640 10 monclient: dump: 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: shutdown 2024-09-15T21:30:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- 172.21.15.17:0/3045671197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b0409ffc0 msgr2=0x7f5b0413aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/3045671197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b0409ffc0 0x7f5b0413aac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009e50 tx=0x7f5b0001c8b0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- 172.21.15.17:0/3045671197 shutdown_connections 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/3045671197 >> v2:172.21.15.119:3300/0 conn(0x7f5b040a1380 0x7f5b0413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/3045671197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b0409ffc0 0x7f5b0413aac0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- 172.21.15.17:0/3045671197 >> 172.21.15.17:0/3045671197 conn(0x7f5b040a66a0 msgr2=0x7f5b040a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- 172.21.15.17:0/3045671197 shutdown_connections 2024-09-15T21:30:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- 172.21.15.17:0/3045671197 wait complete. 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: build_initial_monmap 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: monmap: 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:30:31.294 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 Processor -- start 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- start start 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: init 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: _reopen_session rank -1 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b0409ffc0 0x7f5b041cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): picked mon.noname-c con 0x7f5b0409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:30:31.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b040a0890 0x7f5b04134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): picked mon.noname-b con 0x7f5b040a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b040a1380 0x7f5b04134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): picked mon.noname-a con 0x7f5b040a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04140100 con 0x7f5b040a1380 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0413ff80 con 0x7f5b040a0890 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5b04140280 con 0x7f5b0409ffc0 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): _renew_subs 2024-09-15T21:30:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0bceb640 10 monclient(hunting): authenticate will time out at 1754.675350s 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09a87640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b0409ffc0 msgr2=0x7f5b041cf4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09a87640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b0409ffc0 0x7f5b041cf4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b040a1380 0x7f5b04134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b040a1380 0x7f5b04134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40218/0 (socket says 172.21.15.17:40218) 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 -- 172.21.15.17:0/1664810263 learned_addr learned my addr 172.21.15.17:0/1664810263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): get_auth_request con 0x7f5b040a1380 auth_method 0 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09286640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b040a0890 0x7f5b04134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09286640 10 monclient(hunting): get_auth_request con 0x7f5b040a0890 auth_method 0 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09286640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09286640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b09286640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient(hunting): handle_auth_done global_id 144309 payload 824 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b0409ffc0 msgr2=0x7f5b041cf4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5b0409ffc0 0x7f5b041cf4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b040a0890 msgr2=0x7f5b04134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b040a0890 0x7f5b04134a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient: _finish_hunting 0 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 monclient: found mon.noname-a 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0409d0f0 con 0x7f5b040a1380 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.276+0000 7f5b0a288640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b040a1380 0x7f5b04134fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00d800 tx=0x7f5afc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00b6c0 con 0x7f5b040a1380 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: dump: 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.299 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: _finish_auth 0 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:30:31.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc019070 con 0x7f5b040a1380 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 5 monclient: authenticate success, global_id 144309 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 10 monclient: _renew_subs 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b04121150 con 0x7f5b040a1380 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00baa0 con 0x7f5b040a1380 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: dump: 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.301 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 10 monclient: _renew_subs 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af77fe640 4 set_mon_vals no callback set 2024-09-15T21:30:31.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0411f0d0 con 0x7f5b040a1380 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 0x7f5b0411d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040a0c70 con 0x7f5b0411fc30 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5afc00bc40 con 0x7f5b040a1380 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 0x7f5b0411d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient: get_auth_request con 0x7f5b0411fc30 auth_method 0 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae803b970 0x7f5ae803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5afc050ab0 con 0x7f5b040a1380 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09286640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae803b970 0x7f5ae803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09286640 10 monclient: get_auth_request con 0x7f5ae803b970 auth_method 0 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 10 monclient(hunting): handle_auth_done global_id 154183 payload 824 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b0413d330 con 0x7f5b0411fc30 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09a87640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 0x7f5b0411d610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b041208b0 tx=0x7f5af800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8017070 con 0x7f5b0411fc30 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: dump: 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5b09286640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae803b970 0x7f5ae803de30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5b000ae000 tx=0x7f5b000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5af80074e0 con 0x7f5b0411fc30 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 msgr2=0x7f5b0411d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 0x7f5b0411d610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b041208b0 tx=0x7f5af800c810 comp rx=0 tx=0).stop 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 msgr2=0x7f5b0411d610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.280+0000 7f5af67fc640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411fc30 0x7f5b0411d610 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b041208b0 tx=0x7f5af800c810 comp rx=0 tx=0).stop 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b0bceb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 0x7f5b0411df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b0bceb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:30:31.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04074580 con 0x7f5b0411db50 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 0x7f5b0411df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient: get_auth_request con 0x7f5b0411db50 auth_method 0 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 10 monclient(hunting): handle_auth_done global_id 154186 payload 824 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 1 -- 172.21.15.17:0/1664810263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5b0411e470 con 0x7f5b0411db50 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5b09a87640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 0x7f5b0411df30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5af8000c10 tx=0x7f5af8000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:30:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8017040 con 0x7f5b0411db50 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 10 monclient: dump: 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:30:31.308 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7f5af8022410 con 0x7f5b0411db50 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 msgr2=0x7f5b0411df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 0x7f5b0411df30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5af8000c10 tx=0x7f5af8000ee0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.309 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 124, 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 35823, 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:31.310 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:31.311 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:31.312 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:30:31.313 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:30:31.314 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.315 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:30:31.316 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:30:31.317 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:30:31.318 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:31.319 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:30:31.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:30:31.321 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:30:31.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:30:31.323 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:30:31.324 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 msgr2=0x7f5b0411df30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.288+0000 7f5af67fc640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0411db50 0x7f5b0411df30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae803b970 msgr2=0x7f5ae803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ae803b970 0x7f5ae803de30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5b000ae000 tx=0x7f5b000a7070 comp rx=0 tx=0).stop 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 10 monclient: shutdown 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b040a1380 msgr2=0x7f5b04134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:30:31.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 --2- 172.21.15.17:0/1664810263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b040a1380 0x7f5b04134fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00d800 tx=0x7f5afc00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:30:31.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0a288640 1 -- 172.21.15.17:0/1664810263 reap_dead start 2024-09-15T21:30:31.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0a288640 1 -- 172.21.15.17:0/1664810263 reap_dead start 2024-09-15T21:30:31.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 shutdown_connections 2024-09-15T21:30:31.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 >> 172.21.15.17:0/1664810263 conn(0x7f5b040a66a0 msgr2=0x7f5b0409dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:30:31.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 shutdown_connections 2024-09-15T21:30:31.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:30:31.292+0000 7f5b0bceb640 1 -- 172.21.15.17:0/1664810263 wait complete. 2024-09-15T21:30:31.327 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:30:31.327 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:30:51.329 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T21:30:51.329 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:30:51.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:30:51.333 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:30:51.333 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T21:30:51.333 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T21:30:51.334 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:31:11.334 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T21:31:11.335 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T21:31:11.335 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:31:11.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:11.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.467+0000 7fab9f004640 10 monclient: get_monmap_and_config 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.467+0000 7fab9f004640 10 monclient: build_initial_monmap 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.467+0000 7fab9f004640 10 monclient: monmap: 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:11.472791+0000 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:11.472791+0000 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:11.473 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:11.474 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:11.474 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:11.474 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:11.474 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.467+0000 7fab9f004640 1 Processor -- start 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 1 -- start start 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient: init 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 0x7fab9809cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): picked mon.noname-b con 0x7fab9809c7f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab98137b80 0x7fab980a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): picked mon.noname-c con 0x7fab98137b80 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fab98137f80 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab9809d110 con 0x7fab98137f80 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fab9809ad20 con 0x7fab98137b80 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fab98068f30 con 0x7fab9809c7f0 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient(hunting): authenticate will time out at 1794.868803s 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 0x7fab9809cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 0x7fab9809cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45744/0 (socket says 172.21.15.17:45744) 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 -- 172.21.15.17:0/3824897230 learned_addr learned my addr 172.21.15.17:0/3824897230 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): get_auth_request con 0x7fab9809c7f0 auth_method 0 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab97fff640 1 --2- 172.21.15.17:0/3824897230 >> v2:172.21.15.119:3300/0 conn(0x7fab98137b80 0x7fab980a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab97fff640 10 monclient(hunting): get_auth_request con 0x7fab98137b80 auth_method 0 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient(hunting): handle_auth_done global_id 164099 payload 274 2024-09-15T21:31:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 -- 172.21.15.17:0/3824897230 >> v2:172.21.15.119:3300/0 conn(0x7fab98137b80 msgr2=0x7fab980a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 --2- 172.21.15.17:0/3824897230 >> v2:172.21.15.119:3300/0 conn(0x7fab98137b80 0x7fab980a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 -- 172.21.15.17:0/3824897230 >> v1:172.21.15.119:6789/0 conn(0x7fab98137f80 legacy=0x7fab980a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient: _finish_hunting 0 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 monclient: found mon.noname-b 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:31:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 -- 172.21.15.17:0/3824897230 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9809a760 con 0x7fab9809c7f0 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9cda0640 1 --2- 172.21.15.17:0/3824897230 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 0x7fab9809cbd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009870 tx=0x7fab8c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd66089c35ea0654 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 1 -- 172.21.15.17:0/3824897230 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0b2070 con 0x7fab9809c7f0 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: dump: 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 1 -- 172.21.15.17:0/3824897230 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0a9070 con 0x7fab9809c7f0 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 5 monclient: authenticate success, global_id 164099 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 1 -- 172.21.15.17:0/3824897230 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0ad880 con 0x7fab9809c7f0 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab9f004640 4 set_mon_vals no callback set 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.471+0000 7fab977fe640 10 monclient: dump: 2024-09-15T21:31:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: shutdown 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3824897230 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 msgr2=0x7fab9809cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 --2- 172.21.15.17:0/3824897230 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 0x7fab9809cbd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009870 tx=0x7fab8c01c410 comp rx=0 tx=0).stop 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3824897230 shutdown_connections 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 --2- 172.21.15.17:0/3824897230 >> v2:172.21.15.119:3300/0 conn(0x7fab98137b80 0x7fab980a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 --2- 172.21.15.17:0/3824897230 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab9809c7f0 0x7fab9809cbd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3824897230 >> 172.21.15.17:0/3824897230 conn(0x7fab980a2ed0 msgr2=0x7fab980a32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3824897230 shutdown_connections 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3824897230 wait complete. 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: build_initial_monmap 2024-09-15T21:31:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: monmap: 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 Processor -- start 2024-09-15T21:31:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- start start 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: init 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 0x7fab981cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): picked mon.noname-a con 0x7fab9809c7f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98137b80 0x7fab980961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): picked mon.noname-b con 0x7fab98137b80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 0x7fab98096700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): picked mon.noname-c con 0x7fab98137f80 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fab98142220 con 0x7fab9809c7f0 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab980a2c40 con 0x7fab98137b80 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fab981423a0 con 0x7fab98137f80 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient(hunting): authenticate will time out at 1794.874087s 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 0x7fab981cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 0x7fab98096700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 0x7fab981cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41038/0 (socket says 172.21.15.17:41038) 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 10 monclient(hunting): get_auth_request con 0x7fab98137f80 auth_method 0 2024-09-15T21:31:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 -- 172.21.15.17:0/3372030682 learned_addr learned my addr 172.21.15.17:0/3372030682 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): get_auth_request con 0x7fab9809c7f0 auth_method 0 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 1 -- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 msgr2=0x7fab98096700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 1 -- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 msgr2=0x7fab98096700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 0x7fab98096700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9d5a1640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 0x7fab98096700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab97fff640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98137b80 0x7fab980961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab97fff640 10 monclient(hunting): get_auth_request con 0x7fab98137b80 auth_method 0 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient(hunting): handle_auth_done global_id 144312 payload 824 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 -- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 msgr2=0x7fab98096700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 0x7fab98096700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 -- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98137b80 msgr2=0x7fab980961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98137b80 0x7fab980961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient: _finish_hunting 0 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 monclient: found mon.noname-a 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 -- 172.21.15.17:0/3372030682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9813f3e0 con 0x7fab9809c7f0 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab97fff640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98137b80 0x7fab980961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9cda0640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 0x7fab981cf850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b0040 tx=0x7fab8c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0bb070 con 0x7fab9809c7f0 2024-09-15T21:31:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: dump: 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: _finish_auth 0 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: _check_auth_tickets 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 5 monclient: authenticate success, global_id 144312 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0addb0 con 0x7fab9809c7f0 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: _renew_subs 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0b2040 con 0x7fab9809c7f0 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9809d5c0 con 0x7fab9809c7f0 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 10 monclient: dump: 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab96ffd640 4 set_mon_vals no callback set 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: _renew_subs 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9813eb40 con 0x7fab9809c7f0 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.475+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fab8c0be070 con 0x7fab9809c7f0 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab9f004640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab9f004640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab9f004640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9809cbd0 con 0x7fab9809c7f0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab957fa640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab7c03b9c0 0x7fab7c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fab8c0e99f0 con 0x7fab9809c7f0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab97fff640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab7c03b9c0 0x7fab7c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab97fff640 10 monclient: get_auth_request con 0x7fab7c03b9c0 auth_method 0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.479+0000 7fab97fff640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab7c03b9c0 0x7fab7c03de80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fab980a28d0 tx=0x7fab84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.483+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fab8c0ebe20 con 0x7fab9809c7f0 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.483+0000 7fab957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.483+0000 7fab957fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:11.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab9f004640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:11.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab9f004640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:11.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab9f004640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:11.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fab9813dbe0 con 0x7fab9809c7f0 2024-09-15T21:31:11.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab957fa640 1 -- 172.21.15.17:0/3372030682 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fab8c0160f0 con 0x7fab9809c7f0 2024-09-15T21:31:11.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:11.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.719+0000 7fab957fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:11.726 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":128,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab7c03b9c0 msgr2=0x7fab7c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab7c03b9c0 0x7fab7c03de80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fab980a28d0 tx=0x7fab84008040 comp rx=0 tx=0).stop 2024-09-15T21:31:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 10 monclient: shutdown 2024-09-15T21:31:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 msgr2=0x7fab981cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 0x7fab981cf850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b0040 tx=0x7fab8c019210 comp rx=0 tx=0).stop 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 shutdown_connections 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fab7c03b9c0 0x7fab7c03de80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fab98137f80 0x7fab98096700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98137b80 0x7fab980961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 --2- 172.21.15.17:0/3372030682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab9809c7f0 0x7fab981cf850 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 >> 172.21.15.17:0/3372030682 conn(0x7fab980a2ed0 msgr2=0x7fab980972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 shutdown_connections 2024-09-15T21:31:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.723+0000 7fab9f004640 1 -- 172.21.15.17:0/3372030682 wait complete. 2024-09-15T21:31:11.744 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:31:11.744 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:31:11.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:31:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.859+0000 7f9aa42dd640 10 monclient: get_monmap_and_config 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.859+0000 7f9aa42dd640 10 monclient: build_initial_monmap 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.859+0000 7f9aa42dd640 10 monclient: monmap: 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:11.865917+0000 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:11.865917+0000 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:11.867 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 Processor -- start 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 -- start start 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient: init 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a9c0a0890 0x7f9a9c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): picked mon.noname-a con 0x7f9a9c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9a9c09ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): picked mon.noname-b con 0x7f9a9c0a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9c0a3130 con 0x7f9a9c09ffc0 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9a9c073df0 con 0x7f9a9c0a0890 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9c0a1bf0 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa42dd640 10 monclient(hunting): authenticate will time out at 1795.261277s 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a9c0a0890 0x7f9a9c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a9c0a0890 0x7f9a9c0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41040/0 (socket says 172.21.15.17:41040) 2024-09-15T21:31:11.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 -- 172.21.15.17:0/2977660162 learned_addr learned my addr 172.21.15.17:0/2977660162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): get_auth_request con 0x7f9a9c0a0890 auth_method 0 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 1 --2- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): get_auth_request con 0x7f9a9c0a11b0 auth_method 0 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa287a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient(hunting): handle_auth_done global_id 144315 payload 274 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 -- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 msgr2=0x7f9a9c135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 --2- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 -- 172.21.15.17:0/2977660162 >> v1:172.21.15.119:6789/0 conn(0x7f9a9c09ffc0 legacy=0x7f9a9c135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient: _finish_hunting 0 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 monclient: found mon.noname-a 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:11.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.863+0000 7f9aa2079640 1 -- 172.21.15.17:0/2977660162 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c0a15e0 con 0x7f9a9c0a0890 2024-09-15T21:31:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa2079640 1 --2- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 0x7f9a9c0a0c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0098a0 tx=0x7f9a8c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53cf0fbd2f20810 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 1 -- 172.21.15.17:0/2977660162 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0b0070 con 0x7f9a9c0a0890 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: dump: 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: _finish_auth 0 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: _check_auth_tickets 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 1 -- 172.21.15.17:0/2977660162 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c0ab600 con 0x7f9a9c0a0890 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 1 -- 172.21.15.17:0/2977660162 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0ab9d0 con 0x7f9a9c0a0890 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa1077640 10 monclient: dump: 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 5 monclient: authenticate success, global_id 144315 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 4 set_mon_vals no callback set 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: shutdown 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 msgr2=0x7f9a9c0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 0x7f9a9c0a0c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0098a0 tx=0x7f9a8c01c400 comp rx=0 tx=0).stop 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- 172.21.15.17:0/2977660162 shutdown_connections 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/2977660162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 0x7f9a9c0a0c70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- 172.21.15.17:0/2977660162 >> 172.21.15.17:0/2977660162 conn(0x7f9a9c0a66a0 msgr2=0x7f9a9c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- 172.21.15.17:0/2977660162 shutdown_connections 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- 172.21.15.17:0/2977660162 wait complete. 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: build_initial_monmap 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: monmap: 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 Processor -- start 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- start start 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: init 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c09ffc0 0x7f9a9c13d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): picked mon.noname-b con 0x7f9a9c09ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 0x7f9a9c135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): picked mon.noname-a con 0x7f9a9c0a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): picked mon.noname-c con 0x7f9a9c0a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c140510 con 0x7f9a9c0a0890 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1364d0 con 0x7f9a9c09ffc0 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9c140690 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.867+0000 7f9aa42dd640 10 monclient(hunting): authenticate will time out at 1795.267984s 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 0x7f9a9c135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45764/0 (socket says 172.21.15.17:45764) 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 -- 172.21.15.17:0/567813665 learned_addr learned my addr 172.21.15.17:0/567813665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): get_auth_request con 0x7f9a9c0a11b0 auth_method 0 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): get_auth_request con 0x7f9a9c0a0890 auth_method 0 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c09ffc0 0x7f9a9c13d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): get_auth_request con 0x7f9a9c09ffc0 auth_method 0 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient(hunting): handle_auth_done global_id 164105 payload 824 2024-09-15T21:31:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c09ffc0 msgr2=0x7f9a9c13d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c09ffc0 0x7f9a9c13d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 msgr2=0x7f9a9c135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0a0890 0x7f9a9c135400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient: _finish_hunting 0 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 monclient: found mon.noname-c 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c121af0 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa287a640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009dd0 tx=0x7f9a9400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94018020 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: dump: 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: _finish_auth 0 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94014480 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94014870 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: dump: 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 5 monclient: authenticate success, global_id 164105 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 10 monclient: _renew_subs 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c121dd0 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a93fff640 4 set_mon_vals no callback set 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 10 monclient: _renew_subs 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c121810 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 0x7f9a9c09d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c0a0c70 con 0x7f9a9c09d400 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 0x7f9a9c09d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient: get_auth_request con 0x7f9a9c09d400 auth_method 0 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9a9401b020 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 10 monclient(hunting): handle_auth_done global_id 144321 payload 824 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a9c0008e0 con 0x7f9a9c09d400 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a8003b9c0 0x7f9a8003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9a94051160 con 0x7f9a9c0a11b0 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa2079640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 0x7f9a9c09d7e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009870 tx=0x7f9a8c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0ba030 con 0x7f9a9c09d400 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: dump: 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a8003b9c0 0x7f9a8003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 10 monclient: get_auth_request con 0x7f9a8003b9c0 auth_method 0 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9a8c009490 con 0x7f9a9c09d400 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 msgr2=0x7f9a9c09d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 0x7f9a9c09d7e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009870 tx=0x7f9a8c01ac40 comp rx=0 tx=0).stop 2024-09-15T21:31:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 msgr2=0x7f9a9c09d7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9a92ffd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09d400 0x7f9a9c09d7e0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009870 tx=0x7f9a8c01ac40 comp rx=0 tx=0).stop 2024-09-15T21:31:11.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.871+0000 7f9aa1878640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a8003b9c0 0x7f9a8003de80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c09f5e0 tx=0x7f9a98007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 0x7f9a9c138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c09efa0 con 0x7f9a9c09dd20 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 0x7f9a9c138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient: get_auth_request con 0x7f9a9c09dd20 auth_method 0 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 10 monclient(hunting): handle_auth_done global_id 144324 payload 824 2024-09-15T21:31:11.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 1 -- 172.21.15.17:0/567813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9a9c09e130 con 0x7f9a9c09dd20 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa2079640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 0x7f9a9c138ab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c01c440 tx=0x7f9a8c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0ba040 con 0x7f9a9c09dd20 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 10 monclient: dump: 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:11.899 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4882 (secure 0 0 0) 0x7f9a8c0c0a10 con 0x7f9a9c09dd20 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 msgr2=0x7f9a9c138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 0x7f9a9c138ab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c01c440 tx=0x7f9a8c0191a0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 msgr2=0x7f9a9c138ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9a92ffd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c09dd20 0x7f9a9c138ab0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c01c440 tx=0x7f9a8c0191a0 comp rx=0 tx=0).stop 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:11.900 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 128, 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 143606, 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:11.913 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a8003b9c0 msgr2=0x7f9a8003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9a8003b9c0 0x7f9a8003de80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c09f5e0 tx=0x7f9a98007e40 comp rx=0 tx=0).stop 2024-09-15T21:31:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 10 monclient: shutdown 2024-09-15T21:31:11.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 msgr2=0x7f9a9c135940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 --2- 172.21.15.17:0/567813665 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9a9c0a11b0 0x7f9a9c135940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009dd0 tx=0x7f9a9400c490 comp rx=0 tx=0).stop 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa287a640 1 -- 172.21.15.17:0/567813665 reap_dead start 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa287a640 1 -- 172.21.15.17:0/567813665 reap_dead start 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 shutdown_connections 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 >> 172.21.15.17:0/567813665 conn(0x7f9a9c0a66a0 msgr2=0x7f9a9c072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 shutdown_connections 2024-09-15T21:31:11.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:11.879+0000 7f9aa42dd640 1 -- 172.21.15.17:0/567813665 wait complete. 2024-09-15T21:31:11.915 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:31:11.916 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:31:11.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient: get_monmap_and_config 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient: build_initial_monmap 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient: monmap: 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:12.038369+0000 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:12.038369+0000 2024-09-15T21:31:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:12.039 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 Processor -- start 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 -- start start 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient: init 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9240a0890 0x7fe9240a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): picked mon.noname-c con 0x7fe9240a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 0x7fe9240a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): picked mon.noname-b con 0x7fe92409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe924135670 0x7fe924135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): picked mon.noname-a con 0x7fe924135670 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe9240a16f0 con 0x7fe924135670 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe9240a3130 con 0x7fe9240a0890 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe924073df0 con 0x7fe92409ffc0 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe92a86a640 10 monclient(hunting): authenticate will time out at 1795.433545s 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 0x7fe9240a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 0x7fe9240a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45774/0 (socket says 172.21.15.17:45774) 2024-09-15T21:31:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 -- 172.21.15.17:0/1765281714 learned_addr learned my addr 172.21.15.17:0/1765281714 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): get_auth_request con 0x7fe92409ffc0 auth_method 0 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 1 --2- 172.21.15.17:0/1765281714 >> v2:172.21.15.17:3300/0 conn(0x7fe924135670 0x7fe924135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): get_auth_request con 0x7fe924135670 auth_method 0 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe923fff640 1 --2- 172.21.15.17:0/1765281714 >> v2:172.21.15.119:3300/0 conn(0x7fe9240a0890 0x7fe9240a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe923fff640 10 monclient(hunting): get_auth_request con 0x7fe9240a0890 auth_method 0 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe923fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe923fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe923fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe928e07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient(hunting): handle_auth_done global_id 164108 payload 274 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 -- 172.21.15.17:0/1765281714 >> v2:172.21.15.119:3300/0 conn(0x7fe9240a0890 msgr2=0x7fe9240a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 --2- 172.21.15.17:0/1765281714 >> v2:172.21.15.119:3300/0 conn(0x7fe9240a0890 0x7fe9240a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 -- 172.21.15.17:0/1765281714 >> v2:172.21.15.17:3300/0 conn(0x7fe924135670 msgr2=0x7fe924135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 --2- 172.21.15.17:0/1765281714 >> v2:172.21.15.17:3300/0 conn(0x7fe924135670 0x7fe924135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient: _finish_hunting 0 2024-09-15T21:31:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 monclient: found mon.noname-b 2024-09-15T21:31:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:31:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.035+0000 7fe9237fe640 1 -- 172.21.15.17:0/1765281714 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9240a1bf0 con 0x7fe92409ffc0 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 1 --2- 172.21.15.17:0/1765281714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 0x7fe9240a11b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe92409d0a0 tx=0x7fe90c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96940fe14b22282 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 1 -- 172.21.15.17:0/1765281714 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0b0070 con 0x7fe92409ffc0 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: dump: 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.052 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: _finish_auth 0 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 1 -- 172.21.15.17:0/1765281714 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c0ab600 con 0x7fe92409ffc0 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 5 monclient: authenticate success, global_id 164108 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 1 -- 172.21.15.17:0/1765281714 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0ab9d0 con 0x7fe92409ffc0 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe922ffd640 10 monclient: dump: 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 4 set_mon_vals no callback set 2024-09-15T21:31:12.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: shutdown 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/1765281714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 msgr2=0x7fe9240a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- 172.21.15.17:0/1765281714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 0x7fe9240a11b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe92409d0a0 tx=0x7fe90c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/1765281714 shutdown_connections 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- 172.21.15.17:0/1765281714 >> v2:172.21.15.17:3300/0 conn(0x7fe924135670 0x7fe924135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- 172.21.15.17:0/1765281714 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92409ffc0 0x7fe9240a11b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- 172.21.15.17:0/1765281714 >> v2:172.21.15.119:3300/0 conn(0x7fe9240a0890 0x7fe9240a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/1765281714 >> 172.21.15.17:0/1765281714 conn(0x7fe9240a66a0 msgr2=0x7fe9240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/1765281714 shutdown_connections 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/1765281714 wait complete. 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: build_initial_monmap 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: monmap: 2024-09-15T21:31:12.055 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 Processor -- start 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- start start 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: init 2024-09-15T21:31:12.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 0x7fe9241cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): picked mon.noname-a con 0x7fe92409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9240a0890 0x7fe924134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): picked mon.noname-c con 0x7fe9240a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe924135670 0x7fe92409d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): picked mon.noname-b con 0x7fe924135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe92413f680 con 0x7fe92409ffc0 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe92413f500 con 0x7fe924135670 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe92413f800 con 0x7fe9240a0890 2024-09-15T21:31:12.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient(hunting): authenticate will time out at 1795.438715s 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 0x7fe9241cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 0x7fe9241cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41084/0 (socket says 172.21.15.17:41084) 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 -- 172.21.15.17:0/3057366660 learned_addr learned my addr 172.21.15.17:0/3057366660 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): get_auth_request con 0x7fe92409ffc0 auth_method 0 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9240a0890 0x7fe924134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): get_auth_request con 0x7fe9240a0890 auth_method 0 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe928e07640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe924135670 0x7fe92409d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe928e07640 10 monclient(hunting): get_auth_request con 0x7fe924135670 auth_method 0 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe928e07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe928e07640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe928e07640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient(hunting): handle_auth_done global_id 144330 payload 824 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9240a0890 msgr2=0x7fe924134a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9240a0890 0x7fe924134a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe924135670 msgr2=0x7fe92409d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe924135670 0x7fe92409d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient: _finish_hunting 0 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 monclient: found mon.noname-a 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9240a53f0 con 0x7fe92409ffc0 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9237fe640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe9240a0890 0x7fe924134a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe923fff640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 0x7fe9241cfac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d800 tx=0x7fe91c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00b6c0 con 0x7fe92409ffc0 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:12.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: dump: 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: _finish_auth 0 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c019070 con 0x7fe92409ffc0 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00ba60 con 0x7fe92409ffc0 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 5 monclient: authenticate success, global_id 144330 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: _renew_subs 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe924122100 con 0x7fe92409ffc0 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe9227fc640 4 set_mon_vals no callback set 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe920ff9640 10 monclient: dump: 2024-09-15T21:31:12.062 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: _renew_subs 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.039+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe924072fb0 con 0x7fe92409ffc0 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe92a86a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe92a86a640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 0x7fe92402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe91c00bc00 con 0x7fe92409ffc0 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe92409d0f0 con 0x7fe924134f60 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 0x7fe92402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient: get_auth_request con 0x7fe924134f60 auth_method 0 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe90403b9f0 0x7fe90403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe91c050a90 con 0x7fe92409ffc0 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 10 monclient(hunting): handle_auth_done global_id 164114 payload 824 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe924135aa0 con 0x7fe924134f60 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe9237fe640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 0x7fe92402d450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92406f1c0 tx=0x7fe90c01ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe928e07640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe90403b9f0 0x7fe90403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0bb030 con 0x7fe924134f60 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 10 monclient: dump: 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.065 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe928e07640 10 monclient: get_auth_request con 0x7fe90403b9f0 auth_method 0 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe90c0abe50 con 0x7fe924134f60 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 msgr2=0x7fe92402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 0x7fe92402d450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92406f1c0 tx=0x7fe90c01ac20 comp rx=0 tx=0).stop 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 msgr2=0x7fe92402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe920ff9640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe924134f60 0x7fe92402d450 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92406f1c0 tx=0x7fe90c01ac20 comp rx=0 tx=0).stop 2024-09-15T21:31:12.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.043+0000 7fe928e07640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe90403b9f0 0x7fe90403deb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe9140074a0 tx=0x7fe91400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe92a86a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe92a86a640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 0x7fe92402dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe92a86a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe9241391c0 con 0x7fe92402d990 2024-09-15T21:31:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe9237fe640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 0x7fe92402dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe9237fe640 10 monclient: get_auth_request con 0x7fe92402d990 auth_method 0 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.047+0000 7fe9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe9237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe9237fe640 10 monclient(hunting): handle_auth_done global_id 164117 payload 824 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe9237fe640 1 -- 172.21.15.17:0/3057366660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe92406d020 con 0x7fe92402d990 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe9237fe640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 0x7fe92402dd70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe90c01aa90 tx=0x7fe90c019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0bb040 con 0x7fe92402d990 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 10 monclient: dump: 2024-09-15T21:31:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7fe90c0c1c20 con 0x7fe92402d990 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 msgr2=0x7fe92402dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 0x7fe92402dd70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe90c01aa90 tx=0x7fe90c019180 comp rx=0 tx=0).stop 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 msgr2=0x7fe92402dd70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe920ff9640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe92402d990 0x7fe92402dd70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 128, 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20635, 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:12.084 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe90403b9f0 msgr2=0x7fe90403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe90403b9f0 0x7fe90403deb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe9140074a0 tx=0x7fe91400b040 comp rx=0 tx=0).stop 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 10 monclient: shutdown 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 msgr2=0x7fe9241cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 0x7fe9241cfac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d800 tx=0x7fe91c00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe928e07640 1 -- 172.21.15.17:0/3057366660 reap_dead start 2024-09-15T21:31:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 shutdown_connections 2024-09-15T21:31:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 --2- 172.21.15.17:0/3057366660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92409ffc0 0x7fe9241cfac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 >> 172.21.15.17:0/3057366660 conn(0x7fe9240a66a0 msgr2=0x7fe9240a4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 shutdown_connections 2024-09-15T21:31:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.051+0000 7fe92a86a640 1 -- 172.21.15.17:0/3057366660 wait complete. 2024-09-15T21:31:12.086 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:31:12.086 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:31:12.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:31:12.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient: get_monmap_and_config 2024-09-15T21:31:12.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient: build_initial_monmap 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient: monmap: 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:12.202874+0000 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:12.202874+0000 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:12.203 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 Processor -- start 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 -- start start 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient: init 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0ae009c7c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0ae009cba0 0x7f0ae013ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): picked mon.noname-a con 0x7f0ae009cba0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ae013cf90 0x7f0ae013f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): picked mon.noname-c con 0x7f0ae013cf90 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae009a630 con 0x7f0ae009c7c0 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae0074810 con 0x7f0ae009cba0 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae009a8f0 con 0x7f0ae013cf90 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ae5b1e640 10 monclient(hunting): authenticate will time out at 1795.598989s 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0ae009cba0 0x7f0ae013ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adf7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0ae009c7c0 0x7f0ae0137bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40826/0 (socket says 172.21.15.17:40826) 2024-09-15T21:31:12.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adf7fe640 1 -- 172.21.15.17:0/1135598329 learned_addr learned my addr 172.21.15.17:0/1135598329 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): get_auth_request con 0x7f0ae009cba0 auth_method 0 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adffff640 1 --2- 172.21.15.17:0/1135598329 >> v2:172.21.15.119:3300/0 conn(0x7f0ae013cf90 0x7f0ae013f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adffff640 10 monclient(hunting): get_auth_request con 0x7f0ae013cf90 auth_method 0 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient(hunting): handle_auth_done global_id 144333 payload 274 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 1 -- 172.21.15.17:0/1135598329 >> v2:172.21.15.119:3300/0 conn(0x7f0ae013cf90 msgr2=0x7f0ae013f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 1 --2- 172.21.15.17:0/1135598329 >> v2:172.21.15.119:3300/0 conn(0x7f0ae013cf90 0x7f0ae013f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 1 -- 172.21.15.17:0/1135598329 >> v1:172.21.15.17:6789/0 conn(0x7f0ae009c7c0 legacy=0x7f0ae0137bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 1 monclient: found mon.noname-a 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ade7fc640 1 -- 172.21.15.17:0/1135598329 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080691285 0 0) 0x7f0ae009a630 con 0x7f0ae009c7c0 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adeffd640 1 -- 172.21.15.17:0/1135598329 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae0068f70 con 0x7f0ae009cba0 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0ade7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:31:12.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.199+0000 7f0adffff640 1 --2- 172.21.15.17:0/1135598329 >> v2:172.21.15.119:3300/0 conn(0x7f0ae013cf90 0x7f0ae013f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0adeffd640 1 --2- 172.21.15.17:0/1135598329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009cba0 0x7f0ae013ca50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00a3fd0 tx=0x7f0ad401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc6412982a86aec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 1 -- 172.21.15.17:0/1135598329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40b0070 con 0x7f0ae009cba0 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: dump: 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.216 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: _finish_auth 0 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 5 monclient: authenticate success, global_id 144333 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:12.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 1 -- 172.21.15.17:0/1135598329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad40ab600 con 0x7f0ae009cba0 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 1 -- 172.21.15.17:0/1135598329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40ab9d0 con 0x7f0ae009cba0 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 4 set_mon_vals no callback set 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ade7fc640 10 monclient: dump: 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.218 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: shutdown 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/1135598329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009cba0 msgr2=0x7f0ae013ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/1135598329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009cba0 0x7f0ae013ca50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00a3fd0 tx=0x7f0ad401c340 comp rx=0 tx=0).stop 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/1135598329 shutdown_connections 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/1135598329 >> v2:172.21.15.119:3300/0 conn(0x7f0ae013cf90 0x7f0ae013f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/1135598329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009cba0 0x7f0ae013ca50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/1135598329 >> 172.21.15.17:0/1135598329 conn(0x7f0ae00a2f40 msgr2=0x7f0ae00a3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/1135598329 shutdown_connections 2024-09-15T21:31:12.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/1135598329 wait complete. 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: build_initial_monmap 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: monmap: 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 Processor -- start 2024-09-15T21:31:12.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- start start 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: init 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009c7c0 0x7f0ae0136b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): picked mon.noname-a con 0x7f0ae009c7c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae009cba0 0x7f0ae0137070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): picked mon.noname-b con 0x7f0ae009cba0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ae013cf90 0x7f0ae01cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): picked mon.noname-c con 0x7f0ae013cf90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01422f0 con 0x7f0ae009c7c0 2024-09-15T21:31:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae009d200 con 0x7f0ae009cba0 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae0142470 con 0x7f0ae013cf90 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.203+0000 7f0ae5b1e640 10 monclient(hunting): authenticate will time out at 1795.603986s 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009c7c0 0x7f0ae0136b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ae013cf90 0x7f0ae01cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009c7c0 0x7f0ae0136b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41104/0 (socket says 172.21.15.17:41104) 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 -- 172.21.15.17:0/3190454571 learned_addr learned my addr 172.21.15.17:0/3190454571 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): get_auth_request con 0x7f0ae009c7c0 auth_method 0 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): get_auth_request con 0x7f0ae013cf90 auth_method 0 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae009cba0 0x7f0ae0137070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): get_auth_request con 0x7f0ae009cba0 auth_method 0 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient(hunting): handle_auth_done global_id 144339 payload 824 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ae013cf90 msgr2=0x7f0ae01cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0ae013cf90 0x7f0ae01cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae009cba0 msgr2=0x7f0ae0137070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae009cba0 0x7f0ae0137070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 monclient: found mon.noname-a 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae009d630 con 0x7f0ae009c7c0 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adf7fe640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009c7c0 0x7f0ae0136b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000c450 tx=0x7f0ad000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0018070 con 0x7f0ae009c7c0 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: dump: 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: _finish_auth 0 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 5 monclient: authenticate success, global_id 144339 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 10 monclient: _renew_subs 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad00074e0 con 0x7f0ae009c7c0 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae012dcf0 con 0x7f0ae009c7c0 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0004a30 con 0x7f0ae009c7c0 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: dump: 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0addffb640 4 set_mon_vals no callback set 2024-09-15T21:31:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 10 monclient: _renew_subs 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae013bf80 con 0x7f0ae009c7c0 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 0x7f0ae01cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae013c760 con 0x7f0ae012cd90 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0ad001b070 con 0x7f0ae009c7c0 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 0x7f0acc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 0x7f0ae01cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0ad004f880 con 0x7f0ae009c7c0 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient: get_auth_request con 0x7f0ae012cd90 auth_method 0 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 0x7f0acc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 10 monclient: get_auth_request con 0x7f0acc03b9c0 auth_method 0 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 10 monclient(hunting): handle_auth_done global_id 154213 payload 824 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ae0137bf0 con 0x7f0ae012cd90 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adeffd640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 0x7f0ae01cfed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0069c00 tx=0x7f0ad40190e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 msgr2=0x7f0acc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 msgr2=0x7f0acc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:31:12.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 0x7f0acc03de80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0074630 tx=0x7f0ad8009850 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0adffff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 0x7f0acc03de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40bb030 con 0x7f0ae012cd90 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.207+0000 7f0ac7fff640 10 monclient: dump: 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0ad40b1040 con 0x7f0ae012cd90 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 msgr2=0x7f0ae01cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 0x7f0ae01cfed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0069c00 tx=0x7f0ad40190e0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 msgr2=0x7f0ae01cfed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.211+0000 7f0ac7fff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae012cd90 0x7f0ae01cfed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ae5b1e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 0x7f0ae01387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ae5b1e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01d0820 con 0x7f0ae01d0410 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 0x7f0ae01387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient: get_auth_request con 0x7f0ae01d0410 auth_method 0 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 10 monclient(hunting): handle_auth_done global_id 154216 payload 824 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 1 -- 172.21.15.17:0/3190454571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0ae0069d90 con 0x7f0ae01d0410 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0adeffd640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 0x7f0ae01387d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4000c10 tx=0x7f0ad4009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40bb070 con 0x7f0ae01d0410 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ac7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ac7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.215+0000 7f0ac7fff640 10 monclient: dump: 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.234 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f0ad40b7800 con 0x7f0ae01d0410 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 msgr2=0x7f0ae01387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 0x7f0ae01387d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4000c10 tx=0x7f0ad4009190 comp rx=0 tx=0).stop 2024-09-15T21:31:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 msgr2=0x7f0ae01387d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ac7fff640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae01d0410 0x7f0ae01387d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 128, 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 76750, 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:12.236 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:12.237 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:12.238 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:12.239 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:12.240 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.241 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:12.242 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:12.243 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:12.246 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.247 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:12.248 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:12.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 msgr2=0x7f0acc03de80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:12.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0acc03b9c0 0x7f0acc03de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 10 monclient: shutdown 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009c7c0 msgr2=0x7f0ae0136b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 --2- 172.21.15.17:0/3190454571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ae009c7c0 0x7f0ae0136b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000c450 tx=0x7f0ad000c910 comp rx=0 tx=0).stop 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0adffff640 1 -- 172.21.15.17:0/3190454571 reap_dead start 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0adffff640 1 -- 172.21.15.17:0/3190454571 reap_dead start 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 shutdown_connections 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 >> 172.21.15.17:0/3190454571 conn(0x7f0ae00a2f40 msgr2=0x7f0ae00a1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 shutdown_connections 2024-09-15T21:31:12.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.219+0000 7f0ae5b1e640 1 -- 172.21.15.17:0/3190454571 wait complete. 2024-09-15T21:31:12.250 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:31:12.251 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:31:12.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient: get_monmap_and_config 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient: build_initial_monmap 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient: monmap: 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:12.367454+0000 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:12.367454+0000 2024-09-15T21:31:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:12.368 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 Processor -- start 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 -- start start 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient: init 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a00a0890 0x7f90a00a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): picked mon.noname-b con 0x7f90a00a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f90a009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:12.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90a00a11b0 0x7f90a0135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): picked mon.noname-a con 0x7f90a00a11b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a00a3130 con 0x7f90a009ffc0 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f90a0073df0 con 0x7f90a00a11b0 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90a00a1bf0 con 0x7f90a00a0890 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a83dd640 10 monclient(hunting): authenticate will time out at 1795.762809s 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90a00a11b0 0x7f90a0135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a00a0890 0x7f90a00a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90a00a11b0 0x7f90a0135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41116/0 (socket says 172.21.15.17:41116) 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 -- 172.21.15.17:0/2160254310 learned_addr learned my addr 172.21.15.17:0/2160254310 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): get_auth_request con 0x7f90a00a11b0 auth_method 0 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): get_auth_request con 0x7f90a00a0890 auth_method 0 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a6179640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient(hunting): handle_auth_done global_id 144342 payload 274 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 -- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a00a0890 msgr2=0x7f90a00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 --2- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a00a0890 0x7f90a00a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 -- 172.21.15.17:0/2160254310 >> v1:172.21.15.119:6789/0 conn(0x7f90a009ffc0 legacy=0x7f90a0135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient: _finish_hunting 0 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 monclient: found mon.noname-a 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.363+0000 7f90a697a640 1 -- 172.21.15.17:0/2160254310 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a00a15e0 con 0x7f90a00a11b0 2024-09-15T21:31:12.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a697a640 1 --2- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a0135d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90a00a1870 tx=0x7f909801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618ba8a129aacdd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 1 -- 172.21.15.17:0/2160254310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980a56a0 con 0x7f90a00a11b0 2024-09-15T21:31:12.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: dump: 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: _finish_auth 0 2024-09-15T21:31:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: _check_auth_tickets 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 1 -- 172.21.15.17:0/2160254310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90980a5840 con 0x7f90a00a11b0 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 5 monclient: authenticate success, global_id 144342 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 1 -- 172.21.15.17:0/2160254310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980a5c10 con 0x7f90a00a11b0 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 4 set_mon_vals no callback set 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a5177640 10 monclient: dump: 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: shutdown 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 msgr2=0x7f90a0135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 --2- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a0135d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90a00a1870 tx=0x7f909801c8d0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- 172.21.15.17:0/2160254310 shutdown_connections 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 --2- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a0135d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 --2- 172.21.15.17:0/2160254310 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a00a0890 0x7f90a00a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- 172.21.15.17:0/2160254310 >> 172.21.15.17:0/2160254310 conn(0x7f90a00a66a0 msgr2=0x7f90a00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- 172.21.15.17:0/2160254310 shutdown_connections 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- 172.21.15.17:0/2160254310 wait complete. 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: build_initial_monmap 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: monmap: 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 Processor -- start 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- start start 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: init 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 0x7f90a0135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): picked mon.noname-c con 0x7f90a009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a00a0890 0x7f90a01358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): picked mon.noname-b con 0x7f90a00a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a011def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): picked mon.noname-a con 0x7f90a00a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0140510 con 0x7f90a00a11b0 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01364d0 con 0x7f90a00a0890 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f90a0140690 con 0x7f90a009ffc0 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.367+0000 7f90a83dd640 10 monclient(hunting): authenticate will time out at 1795.767939s 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 0x7f90a0135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 0x7f90a0135360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45840/0 (socket says 172.21.15.17:45840) 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 -- 172.21.15.17:0/3428842812 learned_addr learned my addr 172.21.15.17:0/3428842812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a011def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): get_auth_request con 0x7f90a009ffc0 auth_method 0 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): get_auth_request con 0x7f90a00a11b0 auth_method 0 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a697a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a00a0890 0x7f90a01358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 10 monclient(hunting): get_auth_request con 0x7f90a00a0890 auth_method 0 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient(hunting): handle_auth_done global_id 164126 payload 824 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 -- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a00a0890 msgr2=0x7f90a01358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a00a0890 0x7f90a01358a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 -- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 msgr2=0x7f90a011def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a011def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient: _finish_hunting 0 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 monclient: found mon.noname-c 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 -- 172.21.15.17:0/3428842812 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a011f9b0 con 0x7f90a009ffc0 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a6179640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 0x7f90a0135360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f909000e930 tx=0x7f909000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090017070 con 0x7f90a009ffc0 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: dump: 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: _finish_auth 0 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909000ca00 con 0x7f90a009ffc0 2024-09-15T21:31:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 5 monclient: authenticate success, global_id 164126 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909000cdb0 con 0x7f90a009ffc0 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 10 monclient: dump: 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a4976640 4 set_mon_vals no callback set 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 monclient: _renew_subs 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a0121470 con 0x7f90a009ffc0 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 monclient: _renew_subs 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01cf900 con 0x7f90a009ffc0 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f909001a070 con 0x7f90a009ffc0 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a00a0c70 con 0x7f90a009ffc0 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f908403b9c0 0x7f908403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9090050750 con 0x7f90a009ffc0 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f908403b9c0 0x7f908403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 10 monclient: get_auth_request con 0x7f908403b9c0 auth_method 0 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.371+0000 7f90a5978640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f908403b9c0 0x7f908403de80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f90a0120a60 tx=0x7f909c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.375+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9090052cc0 con 0x7f90a009ffc0 2024-09-15T21:31:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.375+0000 7f9096ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.375+0000 7f9096ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.615+0000 7f90a83dd640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:31:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.615+0000 7f90a83dd640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:31:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.615+0000 7f90a83dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.615+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f90a006d020 con 0x7f90a009ffc0 2024-09-15T21:31:12.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f9096ffd640 1 -- 172.21.15.17:0/3428842812 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f9090013c70 con 0x7f90a009ffc0 2024-09-15T21:31:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f9096ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:31:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f9096ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:12.623 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=11,kv=12,logm=48} crc {auth=1940565336,config=3541143443,health=4195813507,kv=3079713637,logm=1613655125}) 2024-09-15T21:31:12.623 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=12,kv=12,logm=47} crc {auth=2347840487,config=3541143443,health=3798748726,kv=2840421642,logm=2502002888}) 2024-09-15T21:31:12.623 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.623 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=11,kv=12,logm=48} crc {auth=1940565336,config=3541143443,health=4195813507,kv=3079713637,logm=1613655125}) 2024-09-15T21:31:12.623 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=12,kv=12,logm=45} crc {auth=196913559,config=3541143443,health=4090972235,kv=3079713637,logm=3212288676}) 2024-09-15T21:31:12.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136150111}) 2024-09-15T21:31:12.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=855516122}) 2024-09-15T21:31:12.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3136150111}) 2024-09-15T21:31:12.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.619+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2504536420}) 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f908403b9c0 msgr2=0x7f908403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f908403b9c0 0x7f908403de80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f90a0120a60 tx=0x7f909c009040 comp rx=0 tx=0).stop 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 10 monclient: shutdown 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 msgr2=0x7f90a0135360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 0x7f90a0135360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f909000e930 tx=0x7f909000edf0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 shutdown_connections 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f908403b9c0 0x7f908403de80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a00a11b0 0x7f90a011def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a00a0890 0x7f90a01358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 --2- 172.21.15.17:0/3428842812 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f90a009ffc0 0x7f90a0135360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.619+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 >> 172.21.15.17:0/3428842812 conn(0x7f90a00a66a0 msgr2=0x7f90a013c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.623+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 shutdown_connections 2024-09-15T21:31:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:12.623+0000 7f90a83dd640 1 -- 172.21.15.17:0/3428842812 wait complete. 2024-09-15T21:31:12.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.623+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.623+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=723737717}) 2024-09-15T21:31:12.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.623+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1112896842}) 2024-09-15T21:31:12.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.623+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.623+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=723737717}) 2024-09-15T21:31:12.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.623+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2339658624}) 2024-09-15T21:31:12.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2796829169}) 2024-09-15T21:31:12.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1617329631}) 2024-09-15T21:31:12.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2796829169}) 2024-09-15T21:31:12.630 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667564278}) 2024-09-15T21:31:12.632 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.632 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2521670021}) 2024-09-15T21:31:12.632 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2429396696}) 2024-09-15T21:31:12.632 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2521670021}) 2024-09-15T21:31:12.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.627+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4028635327}) 2024-09-15T21:31:12.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695377630}) 2024-09-15T21:31:12.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1506767522}) 2024-09-15T21:31:12.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695377630}) 2024-09-15T21:31:12.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2293922154}) 2024-09-15T21:31:12.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2437032786}) 2024-09-15T21:31:12.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3147040389}) 2024-09-15T21:31:12.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2437032786}) 2024-09-15T21:31:12.636 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2003565978}) 2024-09-15T21:31:12.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1575526683}) 2024-09-15T21:31:12.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3055898262}) 2024-09-15T21:31:12.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1575526683}) 2024-09-15T21:31:12.638 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.631+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1516462496}) 2024-09-15T21:31:12.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=670054895}) 2024-09-15T21:31:12.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3932429066}) 2024-09-15T21:31:12.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=670054895}) 2024-09-15T21:31:12.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3054211520}) 2024-09-15T21:31:12.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448735979}) 2024-09-15T21:31:12.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2417301633}) 2024-09-15T21:31:12.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448735979}) 2024-09-15T21:31:12.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.635+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3971556387}) 2024-09-15T21:31:12.642 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.642 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624512812}) 2024-09-15T21:31:12.642 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1211083671}) 2024-09-15T21:31:12.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624512812}) 2024-09-15T21:31:12.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4110136232}) 2024-09-15T21:31:12.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480823512}) 2024-09-15T21:31:12.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2479168381}) 2024-09-15T21:31:12.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480823512}) 2024-09-15T21:31:12.644 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.639+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=567353534}) 2024-09-15T21:31:12.646 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.646 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2513512763}) 2024-09-15T21:31:12.646 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=238518988}) 2024-09-15T21:31:12.646 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.646 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2513512763}) 2024-09-15T21:31:12.646 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4133862664}) 2024-09-15T21:31:12.647 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=694979227}) 2024-09-15T21:31:12.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2006565486}) 2024-09-15T21:31:12.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=694979227}) 2024-09-15T21:31:12.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1986847731}) 2024-09-15T21:31:12.649 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:31:12.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=850471283}) 2024-09-15T21:31:12.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=314941735}) 2024-09-15T21:31:12.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.643+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=850471283}) 2024-09-15T21:31:12.650 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3910976304}) 2024-09-15T21:31:12.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2920268715}) 2024-09-15T21:31:12.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1578000232}) 2024-09-15T21:31:12.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2920268715}) 2024-09-15T21:31:12.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.647+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=305226262}) 2024-09-15T21:31:12.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=808849318}) 2024-09-15T21:31:12.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1938702242}) 2024-09-15T21:31:12.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=808849318}) 2024-09-15T21:31:12.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297513404}) 2024-09-15T21:31:12.656 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.656 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081723645}) 2024-09-15T21:31:12.656 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3157976121}) 2024-09-15T21:31:12.656 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081723645}) 2024-09-15T21:31:12.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.651+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2250898188}) 2024-09-15T21:31:12.658 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.658 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=182845648}) 2024-09-15T21:31:12.658 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1386634855}) 2024-09-15T21:31:12.658 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.658 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=182845648}) 2024-09-15T21:31:12.658 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3364821724}) 2024-09-15T21:31:12.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=108555954}) 2024-09-15T21:31:12.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4028753699}) 2024-09-15T21:31:12.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=108555954}) 2024-09-15T21:31:12.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198837822}) 2024-09-15T21:31:12.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=847429142}) 2024-09-15T21:31:12.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3976954033}) 2024-09-15T21:31:12.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=847429142}) 2024-09-15T21:31:12.662 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.655+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3848212216}) 2024-09-15T21:31:12.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407610790}) 2024-09-15T21:31:12.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=308786331}) 2024-09-15T21:31:12.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407610790}) 2024-09-15T21:31:12.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1651583180}) 2024-09-15T21:31:12.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3594331951}) 2024-09-15T21:31:12.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2871987595}) 2024-09-15T21:31:12.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3594331951}) 2024-09-15T21:31:12.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.659+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=760241840}) 2024-09-15T21:31:12.666 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.666 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=187767266}) 2024-09-15T21:31:12.666 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=396737537}) 2024-09-15T21:31:12.666 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.666 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=187767266}) 2024-09-15T21:31:12.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652578035}) 2024-09-15T21:31:12.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285125948}) 2024-09-15T21:31:12.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3008952367}) 2024-09-15T21:31:12.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285125948}) 2024-09-15T21:31:12.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3049533137}) 2024-09-15T21:31:12.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114621269}) 2024-09-15T21:31:12.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.663+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2763476328}) 2024-09-15T21:31:12.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114621269}) 2024-09-15T21:31:12.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3090170253}) 2024-09-15T21:31:12.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581812333}) 2024-09-15T21:31:12.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1739460830}) 2024-09-15T21:31:12.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581812333}) 2024-09-15T21:31:12.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3627233255}) 2024-09-15T21:31:12.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663672215}) 2024-09-15T21:31:12.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3214920293}) 2024-09-15T21:31:12.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663672215}) 2024-09-15T21:31:12.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.667+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4181589666}) 2024-09-15T21:31:12.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255704881}) 2024-09-15T21:31:12.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299148897}) 2024-09-15T21:31:12.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255704881}) 2024-09-15T21:31:12.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3205395203}) 2024-09-15T21:31:12.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2388481292}) 2024-09-15T21:31:12.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=993004644}) 2024-09-15T21:31:12.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2388481292}) 2024-09-15T21:31:12.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.671+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3918464947}) 2024-09-15T21:31:12.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=960764293}) 2024-09-15T21:31:12.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3084322809}) 2024-09-15T21:31:12.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=960764293}) 2024-09-15T21:31:12.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2517856673}) 2024-09-15T21:31:12.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3200958082}) 2024-09-15T21:31:12.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4204316633}) 2024-09-15T21:31:12.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3200958082}) 2024-09-15T21:31:12.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.675+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2106084428}) 2024-09-15T21:31:12.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3519676573}) 2024-09-15T21:31:12.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=303208718}) 2024-09-15T21:31:12.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3519676573}) 2024-09-15T21:31:12.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2573844335}) 2024-09-15T21:31:12.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3011758605}) 2024-09-15T21:31:12.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3912742635}) 2024-09-15T21:31:12.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3011758605}) 2024-09-15T21:31:12.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.679+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2524900030}) 2024-09-15T21:31:12.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=812864253}) 2024-09-15T21:31:12.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1820620869}) 2024-09-15T21:31:12.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=812864253}) 2024-09-15T21:31:12.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2862781803}) 2024-09-15T21:31:12.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631677393}) 2024-09-15T21:31:12.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1742488673}) 2024-09-15T21:31:12.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631677393}) 2024-09-15T21:31:12.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.683+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=391298753}) 2024-09-15T21:31:12.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=452133331}) 2024-09-15T21:31:12.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1620653719}) 2024-09-15T21:31:12.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=452133331}) 2024-09-15T21:31:12.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3206453712}) 2024-09-15T21:31:12.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3424427233}) 2024-09-15T21:31:12.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1488774128}) 2024-09-15T21:31:12.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3424427233}) 2024-09-15T21:31:12.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.687+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=279831995}) 2024-09-15T21:31:12.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2173767601}) 2024-09-15T21:31:12.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3385769869}) 2024-09-15T21:31:12.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2173767601}) 2024-09-15T21:31:12.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62330744}) 2024-09-15T21:31:12.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1819128312}) 2024-09-15T21:31:12.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=217091135}) 2024-09-15T21:31:12.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1819128312}) 2024-09-15T21:31:12.697 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.691+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3564605289}) 2024-09-15T21:31:12.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25327538}) 2024-09-15T21:31:12.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=851998026}) 2024-09-15T21:31:12.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25327538}) 2024-09-15T21:31:12.699 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3888100268}) 2024-09-15T21:31:12.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624901898}) 2024-09-15T21:31:12.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4164264934}) 2024-09-15T21:31:12.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624901898}) 2024-09-15T21:31:12.701 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.695+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=512686340}) 2024-09-15T21:31:12.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407189941}) 2024-09-15T21:31:12.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=26368212}) 2024-09-15T21:31:12.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407189941}) 2024-09-15T21:31:12.703 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2910416684}) 2024-09-15T21:31:12.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220310542}) 2024-09-15T21:31:12.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=267962600}) 2024-09-15T21:31:12.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220310542}) 2024-09-15T21:31:12.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.699+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3676989313}) 2024-09-15T21:31:12.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3779235006}) 2024-09-15T21:31:12.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1715385557}) 2024-09-15T21:31:12.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3779235006}) 2024-09-15T21:31:12.706 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=262791631}) 2024-09-15T21:31:12.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2773362760}) 2024-09-15T21:31:12.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4038124766}) 2024-09-15T21:31:12.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2773362760}) 2024-09-15T21:31:12.708 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.703+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756587963}) 2024-09-15T21:31:12.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=796570028}) 2024-09-15T21:31:12.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3458450568}) 2024-09-15T21:31:12.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=796570028}) 2024-09-15T21:31:12.710 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442355731}) 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3409586004}) 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1569958295}) 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3409586004}) 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.707+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3588768333}) 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2979232584}) 2024-09-15T21:31:12.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1003760362}) 2024-09-15T21:31:12.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2979232584}) 2024-09-15T21:31:12.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3661167183}) 2024-09-15T21:31:12.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1475479180}) 2024-09-15T21:31:12.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3979878817}) 2024-09-15T21:31:12.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1475479180}) 2024-09-15T21:31:12.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.711+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=529214894}) 2024-09-15T21:31:12.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=436782098}) 2024-09-15T21:31:12.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=856345448}) 2024-09-15T21:31:12.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=436782098}) 2024-09-15T21:31:12.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3850556555}) 2024-09-15T21:31:12.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3288019699}) 2024-09-15T21:31:12.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1418923010}) 2024-09-15T21:31:12.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3288019699}) 2024-09-15T21:31:12.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.715+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=216738420}) 2024-09-15T21:31:12.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3806976136}) 2024-09-15T21:31:12.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1901458342}) 2024-09-15T21:31:12.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3806976136}) 2024-09-15T21:31:12.722 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=373313738}) 2024-09-15T21:31:12.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1875983862}) 2024-09-15T21:31:12.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=63221166}) 2024-09-15T21:31:12.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1875983862}) 2024-09-15T21:31:12.724 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.719+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3483276693}) 2024-09-15T21:31:12.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087742019}) 2024-09-15T21:31:12.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1870200928}) 2024-09-15T21:31:12.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087742019}) 2024-09-15T21:31:12.726 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3861290387}) 2024-09-15T21:31:12.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1138256238}) 2024-09-15T21:31:12.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3135781140}) 2024-09-15T21:31:12.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1138256238}) 2024-09-15T21:31:12.728 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.723+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682422345}) 2024-09-15T21:31:12.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2562098611}) 2024-09-15T21:31:12.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3182671874}) 2024-09-15T21:31:12.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2562098611}) 2024-09-15T21:31:12.730 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2328303593}) 2024-09-15T21:31:12.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=791683677}) 2024-09-15T21:31:12.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1288865456}) 2024-09-15T21:31:12.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=791683677}) 2024-09-15T21:31:12.732 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.727+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4210278755}) 2024-09-15T21:31:12.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1100411160}) 2024-09-15T21:31:12.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1254935942}) 2024-09-15T21:31:12.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1100411160}) 2024-09-15T21:31:12.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2794465724}) 2024-09-15T21:31:12.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144332274}) 2024-09-15T21:31:12.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3221048534}) 2024-09-15T21:31:12.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144332274}) 2024-09-15T21:31:12.736 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.731+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3112993401}) 2024-09-15T21:31:12.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1576230928}) 2024-09-15T21:31:12.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3571146588}) 2024-09-15T21:31:12.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1576230928}) 2024-09-15T21:31:12.738 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122600917}) 2024-09-15T21:31:12.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3656984357}) 2024-09-15T21:31:12.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3999581984}) 2024-09-15T21:31:12.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3656984357}) 2024-09-15T21:31:12.740 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.735+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1006301395}) 2024-09-15T21:31:12.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327446967}) 2024-09-15T21:31:12.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2030405530}) 2024-09-15T21:31:12.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.742 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327446967}) 2024-09-15T21:31:12.743 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3490395856}) 2024-09-15T21:31:12.744 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4124474188}) 2024-09-15T21:31:12.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1147724845}) 2024-09-15T21:31:12.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4124474188}) 2024-09-15T21:31:12.745 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.739+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=70914794}) 2024-09-15T21:31:12.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=515893909}) 2024-09-15T21:31:12.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3321324665}) 2024-09-15T21:31:12.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=515893909}) 2024-09-15T21:31:12.746 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1726923726}) 2024-09-15T21:31:12.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3791260922}) 2024-09-15T21:31:12.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=374560010}) 2024-09-15T21:31:12.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3791260922}) 2024-09-15T21:31:12.748 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.743+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=668940320}) 2024-09-15T21:31:12.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=104994008}) 2024-09-15T21:31:12.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3700400947}) 2024-09-15T21:31:12.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=104994008}) 2024-09-15T21:31:12.750 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=869984280}) 2024-09-15T21:31:12.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4277005370}) 2024-09-15T21:31:12.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2057615824}) 2024-09-15T21:31:12.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4277005370}) 2024-09-15T21:31:12.752 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.747+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3535501793}) 2024-09-15T21:31:12.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1522077496}) 2024-09-15T21:31:12.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=871312944}) 2024-09-15T21:31:12.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1522077496}) 2024-09-15T21:31:12.754 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2332182210}) 2024-09-15T21:31:12.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1559656055}) 2024-09-15T21:31:12.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2992891385}) 2024-09-15T21:31:12.756 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1559656055}) 2024-09-15T21:31:12.757 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.751+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175604555}) 2024-09-15T21:31:12.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=390726912}) 2024-09-15T21:31:12.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3396965032}) 2024-09-15T21:31:12.758 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=390726912}) 2024-09-15T21:31:12.759 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=700471439}) 2024-09-15T21:31:12.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3433019129}) 2024-09-15T21:31:12.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1593975266}) 2024-09-15T21:31:12.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.760 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3433019129}) 2024-09-15T21:31:12.761 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.755+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2992731763}) 2024-09-15T21:31:12.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3008840132}) 2024-09-15T21:31:12.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2202720478}) 2024-09-15T21:31:12.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3008840132}) 2024-09-15T21:31:12.763 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3078091854}) 2024-09-15T21:31:12.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.764 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114488938}) 2024-09-15T21:31:12.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2562269653}) 2024-09-15T21:31:12.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114488938}) 2024-09-15T21:31:12.765 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.759+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3306644348}) 2024-09-15T21:31:12.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186452400}) 2024-09-15T21:31:12.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3391431990}) 2024-09-15T21:31:12.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186452400}) 2024-09-15T21:31:12.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4030903375}) 2024-09-15T21:31:12.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320946908}) 2024-09-15T21:31:12.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=911523788}) 2024-09-15T21:31:12.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320946908}) 2024-09-15T21:31:12.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.763+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=731777652}) 2024-09-15T21:31:12.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1867788666}) 2024-09-15T21:31:12.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1310850459}) 2024-09-15T21:31:12.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1867788666}) 2024-09-15T21:31:12.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3932326553}) 2024-09-15T21:31:12.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=10} crc {logm=3967742995,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2060384298,mon_config_key=1478676463,monmap=186996059,nvmeofgw=598337686,osd_metadata=1896677127,osd_pg_creating=543505836,osd_snap=841190635,osdmap=2029465972}) 2024-09-15T21:31:12.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=11} crc {logm=2231648212,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3654659336,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=3058936622,osdmap=3285260167}) 2024-09-15T21:31:12.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=10} crc {logm=3967742995,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2060384298,mon_config_key=1478676463,monmap=186996059,nvmeofgw=598337686,osd_metadata=1896677127,osd_pg_creating=543505836,osd_snap=841190635,osdmap=2029465972}) 2024-09-15T21:31:12.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.767+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=39,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_snap=2,osdmap=8} crc {logm=534493249,mdsmap=173658045,mgr=3869055957,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2060384298,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_snap=841190635,osdmap=2356159939}) 2024-09-15T21:31:12.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.783+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.783+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=43} crc {osdmap=2543434282}) 2024-09-15T21:31:12.790 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.783+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=43} crc {osdmap=2313035350}) 2024-09-15T21:31:12.790 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.783+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:31:12.790 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.783+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=43} crc {osdmap=2543434282}) 2024-09-15T21:31:12.790 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:31:12.783+0000 7fc13f4dc640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=43} crc {osdmap=1524960638}) 2024-09-15T21:31:13.650 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:31:13.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:13.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient: get_monmap_and_config 2024-09-15T21:31:13.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient: build_initial_monmap 2024-09-15T21:31:13.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient: monmap: 2024-09-15T21:31:13.782 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:13.782 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:13.782506+0000 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:13.782506+0000 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:13.783 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:13.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 Processor -- start 2024-09-15T21:31:13.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 -- start start 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient: init 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f69e80a0890 0x7f69e80a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): picked mon.noname-a con 0x7f69e80a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 0x7f69e80a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): picked mon.noname-b con 0x7f69e809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f69e8135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e80a3130 con 0x7f69e8135670 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f69e80a16f0 con 0x7f69e80a0890 2024-09-15T21:31:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f69e8073df0 con 0x7f69e809ffc0 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69ec9cf640 10 monclient(hunting): authenticate will time out at 1797.178081s 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 0x7f69e80a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f69e80a0890 0x7f69e80a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 0x7f69e80a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45846/0 (socket says 172.21.15.17:45846) 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f69e8135670 0x7f69e8135aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40828/0 (socket says 172.21.15.17:40828) 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 1 -- 172.21.15.17:0/465382953 learned_addr learned my addr 172.21.15.17:0/465382953 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): get_auth_request con 0x7f69e809ffc0 auth_method 0 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): get_auth_request con 0x7f69e80a0890 auth_method 0 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient(hunting): handle_auth_done global_id 144348 payload 274 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 1 -- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 msgr2=0x7f69e80a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 1 --2- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 0x7f69e80a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 1 -- 172.21.15.17:0/465382953 >> v1:172.21.15.17:6789/0 conn(0x7f69e8135670 legacy=0x7f69e8135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient: _finish_hunting 0 2024-09-15T21:31:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 1 monclient: found mon.noname-a 2024-09-15T21:31:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e659c640 1 -- 172.21.15.17:0/465382953 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e80a1bf0 con 0x7f69e80a0890 2024-09-15T21:31:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.779+0000 7f69e5d9b640 1 --2- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 0x7f69e80a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:31:13.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e659c640 1 --2- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e80a0c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f69e80a1870 tx=0x7f69d801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a96ca4d991e2a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 1 -- 172.21.15.17:0/465382953 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b0070 con 0x7f69e80a0890 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: dump: 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: _finish_auth 0 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: _check_auth_tickets 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 1 -- 172.21.15.17:0/465382953 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80ab600 con 0x7f69e80a0890 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 1 -- 172.21.15.17:0/465382953 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80ab9d0 con 0x7f69e80a0890 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 5 monclient: authenticate success, global_id 144348 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 4 set_mon_vals no callback set 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e559a640 10 monclient: dump: 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: shutdown 2024-09-15T21:31:13.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 msgr2=0x7f69e80a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e80a0c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f69e80a1870 tx=0x7f69d801c360 comp rx=0 tx=0).stop 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- 172.21.15.17:0/465382953 shutdown_connections 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e809ffc0 0x7f69e80a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/465382953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e80a0c70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- 172.21.15.17:0/465382953 >> 172.21.15.17:0/465382953 conn(0x7f69e80a66a0 msgr2=0x7f69e80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- 172.21.15.17:0/465382953 shutdown_connections 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- 172.21.15.17:0/465382953 wait complete. 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: build_initial_monmap 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: monmap: 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 Processor -- start 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- start start 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: init 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e809ffc0 0x7f69e81cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): picked mon.noname-b con 0x7f69e809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e811d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): picked mon.noname-a con 0x7f69e80a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e8135670 0x7f69e8134720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): picked mon.noname-c con 0x7f69e8135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8140690 con 0x7f69e80a0890 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8140510 con 0x7f69e809ffc0 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f69e8140810 con 0x7f69e8135670 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:13.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69ec9cf640 10 monclient(hunting): authenticate will time out at 1797.183203s 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e811d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e811d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41158/0 (socket says 172.21.15.17:41158) 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e8135670 0x7f69e8134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 -- 172.21.15.17:0/693147213 learned_addr learned my addr 172.21.15.17:0/693147213 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): get_auth_request con 0x7f69e80a0890 auth_method 0 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): get_auth_request con 0x7f69e8135670 auth_method 0 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:13.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e659c640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e809ffc0 0x7f69e81cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e659c640 10 monclient(hunting): get_auth_request con 0x7f69e809ffc0 auth_method 0 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient(hunting): handle_auth_done global_id 144354 payload 824 2024-09-15T21:31:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 -- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e8135670 msgr2=0x7f69e8134720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e8135670 0x7f69e8134720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 -- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e809ffc0 msgr2=0x7f69e81cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e809ffc0 0x7f69e81cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 monclient: found mon.noname-a 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 -- 172.21.15.17:0/693147213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e813d430 con 0x7f69e80a0890 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.783+0000 7f69e5d9b640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e811d9b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f69d000e9d0 tx=0x7f69d000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0017070 con 0x7f69e80a0890 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: dump: 2024-09-15T21:31:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: _finish_auth 0 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d000ca00 con 0x7f69e80a0890 2024-09-15T21:31:13.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d000cdb0 con 0x7f69e80a0890 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 10 monclient: dump: 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:13.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 5 monclient: authenticate success, global_id 144354 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69e4d99640 4 set_mon_vals no callback set 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 monclient: _renew_subs 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e811f1b0 con 0x7f69e80a0890 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 monclient: _renew_subs 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e8134e30 con 0x7f69e80a0890 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:13.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e8135aa0 con 0x7f69e80a0890 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f69d001a070 con 0x7f69e80a0890 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69c803b9c0 0x7f69c803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f69d0050540 con 0x7f69e80a0890 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69e659c640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69c803b9c0 0x7f69c803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69e659c640 10 monclient: get_auth_request con 0x7f69c803b9c0 auth_method 0 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.787+0000 7f69e659c640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69c803b9c0 0x7f69c803de80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f69d80ad000 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.791+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f69d0013cc0 con 0x7f69e80a0890 2024-09-15T21:31:13.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.791+0000 7f69deffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:13.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:13.791+0000 7f69deffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:31:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f69e806d020 con 0x7f69e80a0890 2024-09-15T21:31:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69deffd640 1 -- 172.21.15.17:0/693147213 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f69d001a360 con 0x7f69e80a0890 2024-09-15T21:31:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69deffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69deffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:14.031 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":128,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69c803b9c0 msgr2=0x7f69c803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69c803b9c0 0x7f69c803de80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f69d80ad000 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 10 monclient: shutdown 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 msgr2=0x7f69e811d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e811d9b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f69d000e9d0 tx=0x7f69d000ee90 comp rx=0 tx=0).stop 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 shutdown_connections 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f69c803b9c0 0x7f69c803de80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f69e8135670 0x7f69e8134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e80a0890 0x7f69e811d9b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 --2- 172.21.15.17:0/693147213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e809ffc0 0x7f69e81cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 >> 172.21.15.17:0/693147213 conn(0x7f69e80a66a0 msgr2=0x7f69e80a43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 shutdown_connections 2024-09-15T21:31:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.027+0000 7f69ec9cf640 1 -- 172.21.15.17:0/693147213 wait complete. 2024-09-15T21:31:14.050 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:31:14.050 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:31:14.050 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:31:14.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:31:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient: get_monmap_and_config 2024-09-15T21:31:14.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient: monmap: 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:14.174848+0000 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:14.174848+0000 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:14.175 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:14.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 Processor -- start 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 -- start start 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient: init 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe6180a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe61809ffc0 0x7fe61813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): picked mon.noname-a con 0x7fe61809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6180a1380 0x7fe61813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): picked mon.noname-c con 0x7fe6180a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6180a3130 con 0x7fe6180a0890 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe618073df0 con 0x7fe61809ffc0 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe6180a1bf0 con 0x7fe6180a1380 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61d209640 10 monclient(hunting): authenticate will time out at 1797.570278s 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe61809ffc0 0x7fe61813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe61809ffc0 0x7fe61813aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41160/0 (socket says 172.21.15.17:41160) 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 -- 172.21.15.17:0/1750973766 learned_addr learned my addr 172.21.15.17:0/1750973766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): get_auth_request con 0x7fe61809ffc0 auth_method 0 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61759e640 1 --2- 172.21.15.17:0/1750973766 >> v2:172.21.15.119:3300/0 conn(0x7fe6180a1380 0x7fe61813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61759e640 10 monclient(hunting): get_auth_request con 0x7fe6180a1380 auth_method 0 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient(hunting): handle_auth_done global_id 144357 payload 274 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 -- 172.21.15.17:0/1750973766 >> v2:172.21.15.119:3300/0 conn(0x7fe6180a1380 msgr2=0x7fe61813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 --2- 172.21.15.17:0/1750973766 >> v2:172.21.15.119:3300/0 conn(0x7fe6180a1380 0x7fe61813d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 -- 172.21.15.17:0/1750973766 >> v1:172.21.15.119:6789/0 conn(0x7fe6180a0890 legacy=0x7fe6180a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 monclient: found mon.noname-a 2024-09-15T21:31:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:14.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.171+0000 7fe61659c640 1 -- 172.21.15.17:0/1750973766 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61809ccd0 con 0x7fe61809ffc0 2024-09-15T21:31:14.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 1 --2- 172.21.15.17:0/1750973766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61809ffc0 0x7fe61813aac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009870 tx=0x7fe60c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4370c2b23263b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 1 -- 172.21.15.17:0/1750973766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0b0070 con 0x7fe61809ffc0 2024-09-15T21:31:14.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:14.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: dump: 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 1 -- 172.21.15.17:0/1750973766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c0ab600 con 0x7fe61809ffc0 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 5 monclient: authenticate success, global_id 144357 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 1 -- 172.21.15.17:0/1750973766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0ab9d0 con 0x7fe61809ffc0 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 4 set_mon_vals no callback set 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe615d9b640 10 monclient: dump: 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: shutdown 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- 172.21.15.17:0/1750973766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61809ffc0 msgr2=0x7fe61813aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 --2- 172.21.15.17:0/1750973766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61809ffc0 0x7fe61813aac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009870 tx=0x7fe60c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- 172.21.15.17:0/1750973766 shutdown_connections 2024-09-15T21:31:14.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 --2- 172.21.15.17:0/1750973766 >> v2:172.21.15.119:3300/0 conn(0x7fe6180a1380 0x7fe61813d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 --2- 172.21.15.17:0/1750973766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61809ffc0 0x7fe61813aac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- 172.21.15.17:0/1750973766 >> 172.21.15.17:0/1750973766 conn(0x7fe6180a66a0 msgr2=0x7fe6180a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- 172.21.15.17:0/1750973766 shutdown_connections 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- 172.21.15.17:0/1750973766 wait complete. 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: monmap: 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 Processor -- start 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- start start 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: init 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe61809ffc0 0x7fe6181cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): picked mon.noname-b con 0x7fe61809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6180a0890 0x7fe618132760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): picked mon.noname-c con 0x7fe6180a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6180a1380 0x7fe618139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): picked mon.noname-a con 0x7fe6180a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe618140100 con 0x7fe6180a1380 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe61813ff80 con 0x7fe61809ffc0 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe618140280 con 0x7fe6180a0890 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61d209640 10 monclient(hunting): authenticate will time out at 1797.575654s 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6180a0890 0x7fe618132760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6180a1380 0x7fe618139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6180a0890 0x7fe618132760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45864/0 (socket says 172.21.15.17:45864) 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 1 -- 172.21.15.17:0/696307532 learned_addr learned my addr 172.21.15.17:0/696307532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): get_auth_request con 0x7fe6180a0890 auth_method 0 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): get_auth_request con 0x7fe6180a1380 auth_method 0 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.175+0000 7fe61759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe61809ffc0 0x7fe6181cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): get_auth_request con 0x7fe61809ffc0 auth_method 0 2024-09-15T21:31:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 10 monclient(hunting): handle_auth_done global_id 164135 payload 824 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe61809ffc0 msgr2=0x7fe6181cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe61809ffc0 0x7fe6181cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6180a1380 msgr2=0x7fe618139930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6180a1380 0x7fe618139930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 monclient: found mon.noname-c 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61811d0f0 con 0x7fe6180a0890 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61759e640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6180a1380 0x7fe618139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:31:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61659c640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6180a0890 0x7fe618132760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe60c01a010 tx=0x7fe60c019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0b0040 con 0x7fe6180a0890 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: dump: 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c0b9070 con 0x7fe6180a0890 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 5 monclient: authenticate success, global_id 164135 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 10 monclient: _renew_subs 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61809d0f0 con 0x7fe6180a0890 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0aab10 con 0x7fe6180a0890 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: dump: 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 10 monclient: _renew_subs 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61811ce10 con 0x7fe6180a0890 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61559a640 4 set_mon_vals no callback set 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 0x7fe618139e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180a5c50 con 0x7fe618132ca0 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 0x7fe618139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient: get_auth_request con 0x7fe618132ca0 auth_method 0 2024-09-15T21:31:14.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 10 monclient(hunting): handle_auth_done global_id 144363 payload 824 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe60c0ab4c0 con 0x7fe6180a0890 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe61811c9c0 con 0x7fe618132ca0 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe616d9d640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 0x7fe618139e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e500 tx=0x7fe60800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe60403b9f0 0x7fe60403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe60c0ea7e0 con 0x7fe6180a0890 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080052e0 con 0x7fe618132ca0 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: dump: 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe608018070 con 0x7fe618132ca0 2024-09-15T21:31:14.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 msgr2=0x7fe618139e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 0x7fe618139e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e500 tx=0x7fe60800e9c0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 msgr2=0x7fe618139e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe5ff7fe640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe618132ca0 0x7fe618139e70 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e500 tx=0x7fe60800e9c0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61759e640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe60403b9f0 0x7fe60403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61759e640 10 monclient: get_auth_request con 0x7fe60403b9f0 auth_method 0 2024-09-15T21:31:14.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.179+0000 7fe61759e640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe60403b9f0 0x7fe60403deb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe61813b980 tx=0x7fe610009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 0x7fe618071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe61813a790 con 0x7fe61813a3b0 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 0x7fe618071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient: get_auth_request con 0x7fe61813a3b0 auth_method 0 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 10 monclient(hunting): handle_auth_done global_id 144366 payload 824 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 1 -- 172.21.15.17:0/696307532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe6180a0c70 con 0x7fe61813a3b0 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe616d9d640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 0x7fe618071360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e500 tx=0x7fe608000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608007460 con 0x7fe61813a3b0 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 10 monclient: dump: 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.204 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4882 (secure 0 0 0) 0x7fe6080054a0 con 0x7fe61813a3b0 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 msgr2=0x7fe618071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 0x7fe618071360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e500 tx=0x7fe608000ee0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 msgr2=0x7fe618071360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe5ff7fe640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe61813a3b0 0x7fe618071360 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e500 tx=0x7fe608000ee0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.205 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 128, 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 145914, 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:14.206 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:14.207 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:14.209 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.210 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:14.214 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.216 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.217 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:14.218 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe60403b9f0 msgr2=0x7fe60403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe60403b9f0 0x7fe60403deb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe61813b980 tx=0x7fe610009820 comp rx=0 tx=0).stop 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 10 monclient: shutdown 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6180a0890 msgr2=0x7fe618132760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 --2- 172.21.15.17:0/696307532 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe6180a0890 0x7fe618132760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe60c01a010 tx=0x7fe60c019180 comp rx=0 tx=0).stop 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61759e640 1 -- 172.21.15.17:0/696307532 reap_dead start 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61759e640 1 -- 172.21.15.17:0/696307532 reap_dead start 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 shutdown_connections 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 >> 172.21.15.17:0/696307532 conn(0x7fe6180a66a0 msgr2=0x7fe618135870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 shutdown_connections 2024-09-15T21:31:14.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.187+0000 7fe61d209640 1 -- 172.21.15.17:0/696307532 wait complete. 2024-09-15T21:31:14.221 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:31:14.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:31:14.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:31:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient: get_monmap_and_config 2024-09-15T21:31:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient: monmap: 2024-09-15T21:31:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:14.336456+0000 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:14.336456+0000 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:14.337 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 Processor -- start 2024-09-15T21:31:14.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 -- start start 2024-09-15T21:31:14.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient: init 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11cc09d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11cc09def0 0x7f11cc139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): picked mon.noname-c con 0x7f11cc09def0 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11cc139ff0 0x7f11cc13c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): picked mon.noname-a con 0x7f11cc139ff0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc0a3130 con 0x7f11cc09d400 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0a1bc0 con 0x7f11cc139ff0 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0a3490 con 0x7f11cc09def0 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.331+0000 7f11d1b24640 10 monclient(hunting): authenticate will time out at 1797.731917s 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11cc139ff0 0x7f11cc13c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11cc139ff0 0x7f11cc13c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41188/0 (socket says 172.21.15.17:41188) 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 -- 172.21.15.17:0/2440509086 learned_addr learned my addr 172.21.15.17:0/2440509086 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): get_auth_request con 0x7f11cc139ff0 auth_method 0 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11caffd640 1 --2- 172.21.15.17:0/2440509086 >> v2:172.21.15.119:3300/0 conn(0x7f11cc09def0 0x7f11cc139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11caffd640 10 monclient(hunting): get_auth_request con 0x7f11cc09def0 auth_method 0 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11caffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient(hunting): handle_auth_done global_id 144369 payload 274 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 -- 172.21.15.17:0/2440509086 >> v2:172.21.15.119:3300/0 conn(0x7f11cc09def0 msgr2=0x7f11cc139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 --2- 172.21.15.17:0/2440509086 >> v2:172.21.15.119:3300/0 conn(0x7f11cc09def0 0x7f11cc139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 -- 172.21.15.17:0/2440509086 >> v1:172.21.15.17:6789/0 conn(0x7f11cc09d400 legacy=0x7f11cc09d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 monclient: found mon.noname-a 2024-09-15T21:31:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:31:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 -- 172.21.15.17:0/2440509086 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc09e7a0 con 0x7f11cc139ff0 2024-09-15T21:31:14.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11cbfff640 1 --2- 172.21.15.17:0/2440509086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc139ff0 0x7f11cc13c3e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f11cc074430 tx=0x7f11c401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3b432f508483f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 1 -- 172.21.15.17:0/2440509086 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40a5570 con 0x7f11cc139ff0 2024-09-15T21:31:14.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: dump: 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 1 -- 172.21.15.17:0/2440509086 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c40a5710 con 0x7f11cc139ff0 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 1 -- 172.21.15.17:0/2440509086 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40a5ae0 con 0x7f11cc139ff0 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 5 monclient: authenticate success, global_id 144369 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 4 set_mon_vals no callback set 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11ca7fc640 10 monclient: dump: 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.355 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 10 monclient: shutdown 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 -- 172.21.15.17:0/2440509086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc139ff0 msgr2=0x7f11cc13c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 --2- 172.21.15.17:0/2440509086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc139ff0 0x7f11cc13c3e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f11cc074430 tx=0x7f11c401c8b0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 -- 172.21.15.17:0/2440509086 shutdown_connections 2024-09-15T21:31:14.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 --2- 172.21.15.17:0/2440509086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc139ff0 0x7f11cc13c3e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 --2- 172.21.15.17:0/2440509086 >> v2:172.21.15.119:3300/0 conn(0x7f11cc09def0 0x7f11cc139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 -- 172.21.15.17:0/2440509086 >> 172.21.15.17:0/2440509086 conn(0x7f11cc0a66a0 msgr2=0x7f11cc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 -- 172.21.15.17:0/2440509086 shutdown_connections 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 1 -- 172.21.15.17:0/2440509086 wait complete. 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.335+0000 7f11d1b24640 10 monclient: monmap: 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.357 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 Processor -- start 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- start start 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: init 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc09d400 0x7f11cc134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): picked mon.noname-a con 0x7f11cc09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc09def0 0x7f11cc13d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): picked mon.noname-c con 0x7f11cc09def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11cc139ff0 0x7f11cc13dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): picked mon.noname-b con 0x7f11cc139ff0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc13d130 con 0x7f11cc09d400 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc13d2b0 con 0x7f11cc139ff0 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc13d430 con 0x7f11cc09def0 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): authenticate will time out at 1797.736875s 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc09def0 0x7f11cc13d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc09d400 0x7f11cc134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc09def0 0x7f11cc13d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45874/0 (socket says 172.21.15.17:45874) 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 -- 172.21.15.17:0/1483659592 learned_addr learned my addr 172.21.15.17:0/1483659592 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): get_auth_request con 0x7f11cc09def0 auth_method 0 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): get_auth_request con 0x7f11cc09d400 auth_method 0 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11cc139ff0 0x7f11cc13dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 10 monclient(hunting): get_auth_request con 0x7f11cc139ff0 auth_method 0 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient(hunting): handle_auth_done global_id 164138 payload 824 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11cc139ff0 msgr2=0x7f11cc13dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11cc139ff0 0x7f11cc13dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc09d400 msgr2=0x7f11cc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc09d400 0x7f11cc134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 monclient: found mon.noname-c 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc073fe0 con 0x7f11cc09def0 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11cc09d400 0x7f11cc134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11caffd640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc09def0 0x7f11cc13d960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11b400d830 tx=0x7f11b400dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b400b6c0 con 0x7f11cc09def0 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: dump: 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.362 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b4019070 con 0x7f11cc09def0 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b400ba60 con 0x7f11cc09def0 2024-09-15T21:31:14.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 5 monclient: authenticate success, global_id 164138 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: _renew_subs 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc072040 con 0x7f11cc09def0 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c9ffb640 4 set_mon_vals no callback set 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: dump: 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.364 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: _renew_subs 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc09d0f0 con 0x7f11cc09def0 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 0x7f11cc02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc09fc10 con 0x7f11cc134bd0 2024-09-15T21:31:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f11b401e460 con 0x7f11cc09def0 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 0x7f11cc02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient: get_auth_request con 0x7f11cc134bd0 auth_method 0 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11ac03b9c0 0x7f11ac03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_done global_id 164141 payload 824 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f11b4050c70 con 0x7f11cc09def0 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11cc09d7e0 con 0x7f11cc134bd0 2024-09-15T21:31:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cb7fe640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 0x7f11cc02d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc00ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11ac03b9c0 0x7f11ac03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11cbfff640 10 monclient: get_auth_request con 0x7f11ac03b9c0 auth_method 0 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0052e0 con 0x7f11cc134bd0 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: dump: 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f11bc0052e0 con 0x7f11cc134bd0 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 msgr2=0x7f11cc02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 0x7f11cc02d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc00ff70 comp rx=0 tx=0).stop 2024-09-15T21:31:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 msgr2=0x7f11cc02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.339+0000 7f11c3fff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc134bd0 0x7f11cc02d450 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc00ff70 comp rx=0 tx=0).stop 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.343+0000 7f11cbfff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11ac03b9c0 0x7f11ac03de80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f11c40ae000 tx=0x7f11c40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 0x7f11cc02dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc13f290 con 0x7f11cc02d990 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 0x7f11cc02dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient: get_auth_request con 0x7f11cc02d990 auth_method 0 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 10 monclient(hunting): handle_auth_done global_id 164144 payload 824 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 1 -- 172.21.15.17:0/1483659592 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f11cc1391b0 con 0x7f11cc02d990 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cb7fe640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 0x7f11cc02dda0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc007540 con 0x7f11cc02d990 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 10 monclient: dump: 2024-09-15T21:31:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f11bc007540 con 0x7f11cc02d990 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 msgr2=0x7f11cc02dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 0x7f11cc02dda0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc000ee0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 msgr2=0x7f11cc02dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11c3fff640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc02d990 0x7f11cc02dda0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007fd0 tx=0x7f11bc000ee0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:31:14.372 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 128, 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22931, 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.373 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.374 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:14.375 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.377 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:14.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.383 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.384 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.385 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:31:14.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.385 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.385 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:14.385 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11ac03b9c0 msgr2=0x7f11ac03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f11ac03b9c0 0x7f11ac03de80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f11c40ae000 tx=0x7f11c40a7070 comp rx=0 tx=0).stop 2024-09-15T21:31:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 10 monclient: shutdown 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc09def0 msgr2=0x7f11cc13d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 --2- 172.21.15.17:0/1483659592 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11cc09def0 0x7f11cc13d960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11b400d830 tx=0x7f11b400dcf0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cbfff640 1 -- 172.21.15.17:0/1483659592 reap_dead start 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11cbfff640 1 -- 172.21.15.17:0/1483659592 reap_dead start 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 shutdown_connections 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 >> 172.21.15.17:0/1483659592 conn(0x7f11cc0a66a0 msgr2=0x7f11cc13b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 shutdown_connections 2024-09-15T21:31:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.347+0000 7f11d1b24640 1 -- 172.21.15.17:0/1483659592 wait complete. 2024-09-15T21:31:14.387 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:31:14.388 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:31:14.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:31:14.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient: get_monmap_and_config 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient: monmap: 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:14.503019+0000 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:14.503019+0000 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:14.504 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:14.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 1 Processor -- start 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 1 -- start start 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient: init 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc64409d400 0x7fc64409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient(hunting): picked mon.noname-c con 0x7fc64409d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc64409dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc644134d70 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6440a3490 con 0x7fc64409dd20 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6440a0320 con 0x7fc644134d70 2024-09-15T21:31:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc6440a1bc0 con 0x7fc64409d400 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 10 monclient(hunting): authenticate will time out at 1797.898438s 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc64409dd20 0x7fc644134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40852/0 (socket says 172.21.15.17:40852) 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 1 -- 172.21.15.17:0/3569372191 learned_addr learned my addr 172.21.15.17:0/3569372191 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 1 --2- 172.21.15.17:0/3569372191 >> v2:172.21.15.119:3300/0 conn(0x7fc64409d400 0x7fc64409d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): get_auth_request con 0x7fc64409d400 auth_method 0 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704033097 0 0) 0x7fc6440a3490 con 0x7fc64409dd20 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): _negotiate 2024-09-15T21:31:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): my global_id is 144375 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc62c005740 con 0x7fc64409dd20 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3274189117 0 0) 0x7fc62c005740 con 0x7fc64409dd20 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 >> v2:172.21.15.119:3300/0 conn(0x7fc64409d400 msgr2=0x7fc64409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 --2- 172.21.15.17:0/3569372191 >> v2:172.21.15.119:3300/0 conn(0x7fc64409d400 0x7fc64409d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 >> v1:172.21.15.119:6789/0 conn(0x7fc644134d70 legacy=0x7fc64413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64409e7a0 con 0x7fc64409dd20 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fc640001aa0 con 0x7fc64409dd20 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 5 monclient: authenticate success, global_id 144375 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 1 --2- 172.21.15.17:0/3569372191 >> v2:172.21.15.119:3300/0 conn(0x7fc64409d400 0x7fc64409d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: dump: 2024-09-15T21:31:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 >> v1:172.21.15.17:6789/0 conn(0x7fc64409dd20 legacy=0x7fc644134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:31:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc62c005cc0 0x7fc62c0060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): picked mon.b con 0x7fc62c005cc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc62c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): picked mon.c con 0x7fc62c006a20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc62c0075e0 0x7fc62c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): picked mon.a con 0x7fc62c0075e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440a1bc0 con 0x7fc62c0075e0 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c00e4a0 con 0x7fc62c005cc0 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc62c00e6a0 con 0x7fc62c006a20 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64941c640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64c682640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc62c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): get_auth_request con 0x7fc62c006a20 auth_method 0 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc62c0075e0 0x7fc62c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc62c005cc0 0x7fc62c0060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): get_auth_request con 0x7fc62c005cc0 auth_method 0 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): get_auth_request con 0x7fc62c0075e0 auth_method 0 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.499+0000 7fc64ac1f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 10 monclient(hunting): handle_auth_done global_id 144375 payload 293 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 -- 172.21.15.17:0/3569372191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc62c005cc0 msgr2=0x7fc62c0060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc62c005cc0 0x7fc62c0060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 -- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc62c0075e0 msgr2=0x7fc62c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc62c0075e0 0x7fc62c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 monclient: found mon.c 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 -- 172.21.15.17:0/3569372191 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c005740 con 0x7fc62c006a20 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64ac1f640 1 -- 172.21.15.17:0/3569372191 reap_dead start 2024-09-15T21:31:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc649c1d640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc62c006ea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc64000a8b0 tx=0x7fc64001dcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca80199e9626e385 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000a500 con 0x7fc62c006a20 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: dump: 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6400072d0 con 0x7fc62c006a20 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 4 set_mon_vals no callback set 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 1 -- 172.21.15.17:0/3569372191 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400076c0 con 0x7fc62c006a20 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64941c640 10 monclient: dump: 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 10 monclient: shutdown 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 -- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 msgr2=0x7fc62c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc62c006ea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc64000a8b0 tx=0x7fc64001dcb0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 -- 172.21.15.17:0/3569372191 shutdown_connections 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 --2- 172.21.15.17:0/3569372191 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc62c006ea0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 -- 172.21.15.17:0/3569372191 >> 172.21.15.17:0/3569372191 conn(0x7fc6440a66a0 msgr2=0x7fc6440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 -- 172.21.15.17:0/3569372191 shutdown_connections 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 -- 172.21.15.17:0/3569372191 wait complete. 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 10 monclient: monmap: 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 Processor -- start 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 1 -- start start 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.503+0000 7fc64c682640 10 monclient: init 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc6441cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): picked mon.noname-c con 0x7fc62c006a20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:14.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc644133a90 0x7fc644133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): picked mon.noname-a con 0x7fc644133a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644125750 0x7fc644125ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): picked mon.noname-b con 0x7fc644125750 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440a32b0 con 0x7fc644133a90 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440a3130 con 0x7fc644125750 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc64409ccd0 con 0x7fc62c006a20 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): authenticate will time out at 1797.904381s 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc6441cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc644133a90 0x7fc644133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc6441cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45910/0 (socket says 172.21.15.17:45910) 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 -- 172.21.15.17:0/4039233662 learned_addr learned my addr 172.21.15.17:0/4039233662 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): get_auth_request con 0x7fc62c006a20 auth_method 0 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): get_auth_request con 0x7fc644133a90 auth_method 0 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient(hunting): handle_auth_done global_id 164147 payload 824 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64ac1f640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644125750 0x7fc644125ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644125750 msgr2=0x7fc644125ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644125750 0x7fc644125ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc644133a90 msgr2=0x7fc644133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc644133a90 0x7fc644133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 monclient: found mon.noname-c 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64409d0f0 con 0x7fc62c006a20 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc644133a90 0x7fc644133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64a41e640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc6441cfb30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc63c011cd0 tx=0x7fc63c00f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c011e80 con 0x7fc62c006a20 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: dump: 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.532 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c004020 con 0x7fc62c006a20 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0043d0 con 0x7fc62c006a20 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 5 monclient: authenticate success, global_id 164147 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient: _renew_subs 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6441283d0 con 0x7fc62c006a20 2024-09-15T21:31:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc648c1b640 4 set_mon_vals no callback set 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: dump: 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient: _renew_subs 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6441d0070 con 0x7fc62c006a20 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 0x7fc6441260e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440008e0 con 0x7fc644134410 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc63c012040 con 0x7fc62c006a20 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc63003b9f0 0x7fc63003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc63c050d10 con 0x7fc62c006a20 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 0x7fc6441260e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient: get_auth_request con 0x7fc644134410 auth_method 0 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64ac1f640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc63003b9f0 0x7fc63003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64ac1f640 10 monclient: get_auth_request con 0x7fc63003b9f0 auth_method 0 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 10 monclient(hunting): handle_auth_done global_id 154249 payload 824 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc644127cd0 con 0x7fc644134410 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc64ac1f640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc63003b9f0 0x7fc63003deb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc64409e100 tx=0x7fc634009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc649c1d640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 0x7fc6441260e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a3660 tx=0x7fc64001a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400018a0 con 0x7fc644134410 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.507+0000 7fc63affd640 10 monclient: dump: 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc6400018a0 con 0x7fc644134410 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:14.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 msgr2=0x7fc6441260e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 0x7fc6441260e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a3660 tx=0x7fc64001a090 comp rx=0 tx=0).stop 2024-09-15T21:31:14.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 msgr2=0x7fc6441260e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:14.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.511+0000 7fc63affd640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc644134410 0x7fc6441260e0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a3660 tx=0x7fc64001a090 comp rx=0 tx=0).stop 2024-09-15T21:31:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc64c682640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc64c682640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 0x7fc64411d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc64c682640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc644127ee0 con 0x7fc64411d4f0 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 0x7fc64411d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient: get_auth_request con 0x7fc64411d4f0 auth_method 0 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 10 monclient(hunting): handle_auth_done global_id 154252 payload 824 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.515+0000 7fc649c1d640 1 -- 172.21.15.17:0/4039233662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc6441daea0 con 0x7fc64411d4f0 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc649c1d640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 0x7fc64411d900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc640000f90 tx=0x7fc62c007a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400be040 con 0x7fc64411d4f0 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 10 monclient: dump: 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.541 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7fc6400070d0 con 0x7fc64411d4f0 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 msgr2=0x7fc64411d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 0x7fc64411d900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc640000f90 tx=0x7fc62c007a50 comp rx=0 tx=0).stop 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 msgr2=0x7fc64411d900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc63affd640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64411d4f0 0x7fc64411d900 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc640000f90 tx=0x7fc62c007a50 comp rx=0 tx=0).stop 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:14.542 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 128, 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 79050, 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:14.543 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:14.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:14.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:14.547 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:14.548 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:14.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:14.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:14.555 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc63003b9f0 msgr2=0x7fc63003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc63003b9f0 0x7fc63003deb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc64409e100 tx=0x7fc634009040 comp rx=0 tx=0).stop 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 10 monclient: shutdown 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 msgr2=0x7fc6441cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 --2- 172.21.15.17:0/4039233662 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc62c006a20 0x7fc6441cfb30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc63c011cd0 tx=0x7fc63c00f610 comp rx=0 tx=0).stop 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64ac1f640 1 -- 172.21.15.17:0/4039233662 reap_dead start 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64ac1f640 1 -- 172.21.15.17:0/4039233662 reap_dead start 2024-09-15T21:31:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 shutdown_connections 2024-09-15T21:31:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 >> 172.21.15.17:0/4039233662 conn(0x7fc6440a66a0 msgr2=0x7fc644071f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 shutdown_connections 2024-09-15T21:31:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.519+0000 7fc64c682640 1 -- 172.21.15.17:0/4039233662 wait complete. 2024-09-15T21:31:14.557 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:31:14.557 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:31:14.557 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:31:14.557 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:31:14.557 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:31:14.558 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:31:14.558 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:14.574 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:31:14.574 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:31:14.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:14.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient: get_monmap_and_config 2024-09-15T21:31:14.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient: monmap: 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:14.739814+0000 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:14.739814+0000 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:14.740 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 Processor -- start 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 -- start start 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient: init 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faf4409f000 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf440a0fa0 0x7faf44139690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): picked mon.noname-c con 0x7faf440a0fa0 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf440a13b0 0x7faf4413bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): picked mon.noname-a con 0x7faf440a13b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:14.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4409f3e0 con 0x7faf4409f000 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faf440a1d60 con 0x7faf440a13b0 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faf440a23b0 con 0x7faf440a0fa0 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf4a12b640 10 monclient(hunting): authenticate will time out at 1798.135270s 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf43fff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7faf440a13b0 msgr2=0x7faf4413bbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf43fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf440a13b0 0x7faf4413bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf440a0fa0 0x7faf44139690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf440a0fa0 0x7faf44139690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:53410/0 (socket says 172.21.15.17:53410) 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 1 -- 172.21.15.17:0/2085808751 learned_addr learned my addr 172.21.15.17:0/2085808751 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): get_auth_request con 0x7faf440a0fa0 auth_method 0 2024-09-15T21:31:14.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.735+0000 7faf42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 10 monclient(hunting): handle_auth_done global_id 154255 payload 274 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 1 -- 172.21.15.17:0/2085808751 >> v2:172.21.15.17:3300/0 conn(0x7faf440a13b0 msgr2=0x7faf4413bbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 1 --2- 172.21.15.17:0/2085808751 >> v2:172.21.15.17:3300/0 conn(0x7faf440a13b0 0x7faf4413bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 1 -- 172.21.15.17:0/2085808751 >> v1:172.21.15.119:6789/0 conn(0x7faf4409f000 legacy=0x7faf440a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 1 monclient: found mon.noname-c 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:31:14.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 1 -- 172.21.15.17:0/2085808751 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4409e820 con 0x7faf440a0fa0 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf42ffd640 1 --2- 172.21.15.17:0/2085808751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf440a0fa0 0x7faf44139690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf44074aa0 tx=0x7faf3801eaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4acc93ec4ed6e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 1 -- 172.21.15.17:0/2085808751 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380b4070 con 0x7faf440a0fa0 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: dump: 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.751 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 1 -- 172.21.15.17:0/2085808751 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf380a7ad0 con 0x7faf440a0fa0 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 1 -- 172.21.15.17:0/2085808751 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380af440 con 0x7faf440a0fa0 2024-09-15T21:31:14.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf427fc640 10 monclient: dump: 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 5 monclient: authenticate success, global_id 154255 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 4 set_mon_vals no callback set 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 10 monclient: shutdown 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 -- 172.21.15.17:0/2085808751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf440a0fa0 msgr2=0x7faf44139690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 --2- 172.21.15.17:0/2085808751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf440a0fa0 0x7faf44139690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf44074aa0 tx=0x7faf3801eaa0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 -- 172.21.15.17:0/2085808751 shutdown_connections 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 --2- 172.21.15.17:0/2085808751 >> v2:172.21.15.17:3300/0 conn(0x7faf440a13b0 0x7faf4413bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 --2- 172.21.15.17:0/2085808751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf440a0fa0 0x7faf44139690 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 -- 172.21.15.17:0/2085808751 >> 172.21.15.17:0/2085808751 conn(0x7faf440a66a0 msgr2=0x7faf440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 -- 172.21.15.17:0/2085808751 shutdown_connections 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.739+0000 7faf4a12b640 1 -- 172.21.15.17:0/2085808751 wait complete. 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: build_initial_monmap 2024-09-15T21:31:14.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: monmap: 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 Processor -- start 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- start start 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: init 2024-09-15T21:31:14.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf4409f000 0x7faf441cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): picked mon.noname-b con 0x7faf4409f000 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 0x7faf4413b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): picked mon.noname-c con 0x7faf440a0fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf440a13b0 0x7faf4413bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): picked mon.noname-a con 0x7faf440a13b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faf4409ce50 con 0x7faf440a13b0 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faf4409ccd0 con 0x7faf4409f000 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faf440a25f0 con 0x7faf440a0fa0 2024-09-15T21:31:14.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient(hunting): authenticate will time out at 1798.141114s 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf43fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf440a13b0 msgr2=0x7faf4413bba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf440a13b0 0x7faf4413bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 0x7faf4413b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 0x7faf4413b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:45960/0 (socket says 172.21.15.17:45960) 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 -- 172.21.15.17:0/3823641061 learned_addr learned my addr 172.21.15.17:0/3823641061 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): get_auth_request con 0x7faf440a0fa0 auth_method 0 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf4409f000 0x7faf441cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 10 monclient(hunting): get_auth_request con 0x7faf4409f000 auth_method 0 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient(hunting): handle_auth_done global_id 164150 payload 824 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 -- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf4409f000 msgr2=0x7faf441cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf4409f000 0x7faf441cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 -- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf440a13b0 msgr2=0x7faf4413bba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf440a13b0 0x7faf4413bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 monclient: found mon.noname-c 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:14.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 -- 172.21.15.17:0/3823641061 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4406f7c0 con 0x7faf440a0fa0 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf4409f000 0x7faf441cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf42ffd640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 0x7faf4413b660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf3801b6b0 tx=0x7faf3801b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380b4030 con 0x7faf440a0fa0 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: dump: 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.759 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: _finish_auth 0 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 5 monclient: authenticate success, global_id 164150 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf380b6070 con 0x7faf440a0fa0 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380ad770 con 0x7faf440a0fa0 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _renew_subs 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf44127cd0 con 0x7faf440a0fa0 2024-09-15T21:31:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf41ffb640 4 set_mon_vals no callback set 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 10 monclient: dump: 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:14.761 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _renew_subs 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf44128830 con 0x7faf440a0fa0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf440a0890 con 0x7faf440a0fa0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7faf380c1060 con 0x7faf440a0fa0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf3003b9a0 0x7faf3003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7faf380eac70 con 0x7faf440a0fa0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf3003b9a0 0x7faf3003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.743+0000 7faf437fe640 10 monclient: get_auth_request con 0x7faf3003b9a0 auth_method 0 2024-09-15T21:31:14.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.747+0000 7faf437fe640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf3003b9a0 0x7faf3003de60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faf34001750 tx=0x7faf34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:14.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.747+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7faf380187e0 con 0x7faf440a0fa0 2024-09-15T21:31:14.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.747+0000 7faf2bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:14.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.747+0000 7faf2bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:14.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf4a12b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:14.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf4a12b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:14.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf4a12b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:14.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7faf4409d0f0 con 0x7faf440a0fa0 2024-09-15T21:31:14.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf2bfff640 1 -- 172.21.15.17:0/3823641061 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7faf380c2cd0 con 0x7faf440a0fa0 2024-09-15T21:31:14.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf2bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:14.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.987+0000 7faf2bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:14.992 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":128,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:14.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf3003b9a0 msgr2=0x7faf3003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf3003b9a0 0x7faf3003de60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faf34001750 tx=0x7faf34009040 comp rx=0 tx=0).stop 2024-09-15T21:31:14.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 10 monclient: shutdown 2024-09-15T21:31:14.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:14.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 msgr2=0x7faf4413b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:14.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 0x7faf4413b660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf3801b6b0 tx=0x7faf3801b840 comp rx=0 tx=0).stop 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 shutdown_connections 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf3003b9a0 0x7faf3003de60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf440a13b0 0x7faf4413bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf440a0fa0 0x7faf4413b660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 --2- 172.21.15.17:0/3823641061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf4409f000 0x7faf441cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 >> 172.21.15.17:0/3823641061 conn(0x7faf440a66a0 msgr2=0x7faf4406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 shutdown_connections 2024-09-15T21:31:14.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:14.991+0000 7faf4a12b640 1 -- 172.21.15.17:0/3823641061 wait complete. 2024-09-15T21:31:18.012 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient: get_monmap_and_config 2024-09-15T21:31:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient: build_initial_monmap 2024-09-15T21:31:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient: monmap: 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:18.147587+0000 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:18.147587+0000 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:18.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 Processor -- start 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 -- start start 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient: init 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4bc09d400 0x7ff4bc09d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): picked mon.noname-c con 0x7ff4bc09d400 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4bc09dd20 0x7ff4bc134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): picked mon.noname-a con 0x7ff4bc09dd20 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4bc134ba0 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc0a3130 con 0x7ff4bc134ba0 2024-09-15T21:31:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc0a1bc0 con 0x7ff4bc09dd20 2024-09-15T21:31:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc0a3490 con 0x7ff4bc09d400 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c307d640 10 monclient(hunting): authenticate will time out at 1801.543189s 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4bbfff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7ff4bc09dd20 msgr2=0x7ff4bc134660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4bbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4bc09dd20 0x7ff4bc134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4bc09d400 0x7ff4bc09d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4bc09d400 0x7ff4bc09d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:34372/0 (socket says 172.21.15.17:34372) 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 1 -- 172.21.15.17:0/2894866378 learned_addr learned my addr 172.21.15.17:0/2894866378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): get_auth_request con 0x7ff4bc09d400 auth_method 0 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.143+0000 7ff4c0e19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 10 monclient(hunting): handle_auth_done global_id 154261 payload 274 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 1 -- 172.21.15.17:0/2894866378 >> v2:172.21.15.17:3300/0 conn(0x7ff4bc09dd20 msgr2=0x7ff4bc134660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 1 --2- 172.21.15.17:0/2894866378 >> v2:172.21.15.17:3300/0 conn(0x7ff4bc09dd20 0x7ff4bc134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 1 -- 172.21.15.17:0/2894866378 >> v1:172.21.15.119:6789/0 conn(0x7ff4bc134ba0 legacy=0x7ff4bc13bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 10 monclient: _finish_hunting 0 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 1 monclient: found mon.noname-c 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:31:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 1 -- 172.21.15.17:0/2894866378 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc09e7a0 con 0x7ff4bc09d400 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c0e19640 1 --2- 172.21.15.17:0/2894866378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09d400 0x7ff4bc09d7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc09d0a0 tx=0x7ff4ac01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81292651e4831816 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 1 -- 172.21.15.17:0/2894866378 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0b0070 con 0x7ff4bc09d400 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: dump: 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 1 -- 172.21.15.17:0/2894866378 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ac0ab600 con 0x7ff4bc09d400 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 5 monclient: authenticate success, global_id 154261 2024-09-15T21:31:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 1 -- 172.21.15.17:0/2894866378 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0ab9d0 con 0x7ff4bc09d400 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 4 set_mon_vals no callback set 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4bb7fe640 10 monclient: dump: 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 10 monclient: shutdown 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 -- 172.21.15.17:0/2894866378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09d400 msgr2=0x7ff4bc09d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 --2- 172.21.15.17:0/2894866378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09d400 0x7ff4bc09d7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc09d0a0 tx=0x7ff4ac01c400 comp rx=0 tx=0).stop 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 -- 172.21.15.17:0/2894866378 shutdown_connections 2024-09-15T21:31:18.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 --2- 172.21.15.17:0/2894866378 >> v2:172.21.15.17:3300/0 conn(0x7ff4bc09dd20 0x7ff4bc134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 --2- 172.21.15.17:0/2894866378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09d400 0x7ff4bc09d7e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 -- 172.21.15.17:0/2894866378 >> 172.21.15.17:0/2894866378 conn(0x7ff4bc0a66a0 msgr2=0x7ff4bc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 -- 172.21.15.17:0/2894866378 shutdown_connections 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.147+0000 7ff4c307d640 1 -- 172.21.15.17:0/2894866378 wait complete. 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: build_initial_monmap 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: monmap: 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:18.164 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 Processor -- start 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- start start 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: init 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4bc09d400 0x7ff4bc133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): picked mon.noname-a con 0x7ff4bc09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09dd20 0x7ff4bc13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): picked mon.noname-b con 0x7ff4bc09dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 0x7ff4bc140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): picked mon.noname-c con 0x7ff4bc134ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:18.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc09ce50 con 0x7ff4bc09d400 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc09ccd0 con 0x7ff4bc09dd20 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc09fa20 con 0x7ff4bc134ba0 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient(hunting): authenticate will time out at 1801.549438s 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c0e19640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4bc09d400 msgr2=0x7ff4bc133e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c0e19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4bc09d400 0x7ff4bc133e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 0x7ff4bc140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 0x7ff4bc140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56140/0 (socket says 172.21.15.17:56140) 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 -- 172.21.15.17:0/1608568770 learned_addr learned my addr 172.21.15.17:0/1608568770 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:18.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): get_auth_request con 0x7ff4bc134ba0 auth_method 0 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4bbfff640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09dd20 0x7ff4bc13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4bbfff640 10 monclient(hunting): get_auth_request con 0x7ff4bc09dd20 auth_method 0 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient(hunting): handle_auth_done global_id 164153 payload 824 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 -- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09dd20 msgr2=0x7ff4bc13fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:18.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09dd20 0x7ff4bc13fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 -- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4bc09d400 msgr2=0x7ff4bc133e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4bc09d400 0x7ff4bc133e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient: _finish_hunting 0 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 monclient: found mon.noname-c 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 -- 172.21.15.17:0/1608568770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc13ae10 con 0x7ff4bc134ba0 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c161a640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 0x7ff4bc140000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400c500 tx=0x7ff4b400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40052d0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: dump: 2024-09-15T21:31:18.168 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: _finish_auth 0 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4005470 con 0x7ff4bc134ba0 2024-09-15T21:31:18.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 5 monclient: authenticate success, global_id 164153 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _renew_subs 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc13b0f0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4012850 con 0x7ff4bc134ba0 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 10 monclient: dump: 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:18.170 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _renew_subs 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4ba7fc640 4 set_mon_vals no callback set 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc13b3d0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc09d7e0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff4b40129f0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff49c03b970 0x7ff49c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:18.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.151+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff4b40501d0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.155+0000 7ff4c0e19640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff49c03b970 0x7ff49c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:18.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.155+0000 7ff4c0e19640 10 monclient: get_auth_request con 0x7ff49c03b970 auth_method 0 2024-09-15T21:31:18.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.155+0000 7ff4c0e19640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff49c03b970 0x7ff49c03de30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ad000 tx=0x7ff4ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:18.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.159+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff4b4003d60 con 0x7ff4bc134ba0 2024-09-15T21:31:18.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.159+0000 7ff4b97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:18.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.159+0000 7ff4b97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4c307d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4c307d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4c307d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff4bc073d00 con 0x7ff4bc134ba0 2024-09-15T21:31:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4b97fa640 1 -- 172.21.15.17:0/1608568770 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7ff4b4010cf0 con 0x7ff4bc134ba0 2024-09-15T21:31:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4b97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.395+0000 7ff4b97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:18.401 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":128,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff49c03b970 msgr2=0x7ff49c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff49c03b970 0x7ff49c03de30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ad000 tx=0x7ff4ac008040 comp rx=0 tx=0).stop 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 10 monclient: shutdown 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 msgr2=0x7ff4bc140000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 0x7ff4bc140000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400c500 tx=0x7ff4b400c9c0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 shutdown_connections 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff49c03b970 0x7ff49c03de30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff4bc134ba0 0x7ff4bc140000 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4bc09dd20 0x7ff4bc13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 --2- 172.21.15.17:0/1608568770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4bc09d400 0x7ff4bc133e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 >> 172.21.15.17:0/1608568770 conn(0x7ff4bc0a66a0 msgr2=0x7ff4bc0a5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 shutdown_connections 2024-09-15T21:31:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:18.399+0000 7ff4c307d640 1 -- 172.21.15.17:0/1608568770 wait complete. 2024-09-15T21:31:18.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=110 2024-09-15T21:31:18.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 110 -ne 110 ]] 2024-09-15T21:31:18.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:18.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ export CEPH_ARGS= 2024-09-15T21:31:18.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ CEPH_ARGS= 2024-09-15T21:31:18.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph auth del client.foo 2024-09-15T21:31:19.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient: get_monmap_and_config 2024-09-15T21:31:19.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient: build_initial_monmap 2024-09-15T21:31:19.100 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient: monmap: 2024-09-15T21:31:19.100 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:31:19.100 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:31:19.106413+0000 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:31:19.106413+0000 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:19.101 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 Processor -- start 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 -- start start 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient: init 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff824135c50 0x7ff824136030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.103 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): picked mon.noname-a con 0x7ff824135c50 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff82409fac0 0x7ff8240a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): picked mon.noname-c con 0x7ff82409fac0 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8240a1dd0 0x7ff8240a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): picked mon.noname-b con 0x7ff8240a1dd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff824074b00 con 0x7ff824135c50 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff82409d750 con 0x7ff82409fac0 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff82409ee20 con 0x7ff8240a1dd0 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82c3bf640 10 monclient(hunting): authenticate will time out at 1797.902925s 2024-09-15T21:31:19.104 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff82409fac0 0x7ff8240a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff82409fac0 0x7ff8240a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39022/0 (socket says 172.21.15.119:39022) 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 -- 172.21.15.119:0/4032408928 learned_addr learned my addr 172.21.15.119:0/4032408928 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82a15b640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7ff824135c50 msgr2=0x7ff824136030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): get_auth_request con 0x7ff82409fac0 auth_method 0 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82a15b640 1 --2- 172.21.15.119:0/4032408928 >> v2:172.21.15.17:3300/0 conn(0x7ff824135c50 0x7ff824136030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient(hunting): handle_auth_done global_id 154267 payload 274 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 -- 172.21.15.119:0/4032408928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8240a1dd0 msgr2=0x7ff8240a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:19.105 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 --2- 172.21.15.119:0/4032408928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8240a1dd0 0x7ff8240a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 -- 172.21.15.119:0/4032408928 >> v2:172.21.15.17:3300/0 conn(0x7ff824135c50 msgr2=0x7ff824136030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 --2- 172.21.15.119:0/4032408928 >> v2:172.21.15.17:3300/0 conn(0x7ff824135c50 0x7ff824136030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient: _finish_hunting 0 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 monclient: found mon.noname-c 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:31:19.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.101+0000 7ff82995a640 1 -- 172.21.15.119:0/4032408928 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82409d4d0 con 0x7ff82409fac0 2024-09-15T21:31:19.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82995a640 1 --2- 172.21.15.119:0/4032408928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82409fac0 0x7ff8240a1890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff82409f260 tx=0x7ff82001c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2695a7b7bc160c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:19.111 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 1 -- 172.21.15.119:0/4032408928 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a5660 con 0x7ff82409fac0 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: dump: 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:19.112 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: _finish_auth 0 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: _check_auth_tickets 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 1 -- 172.21.15.119:0/4032408928 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8200a5800 con 0x7ff82409fac0 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 5 monclient: authenticate success, global_id 154267 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 1 -- 172.21.15.119:0/4032408928 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a5bd0 con 0x7ff82409fac0 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 4 set_mon_vals no callback set 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff829159640 10 monclient: dump: 2024-09-15T21:31:19.113 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: shutdown 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- 172.21.15.119:0/4032408928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82409fac0 msgr2=0x7ff8240a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/4032408928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82409fac0 0x7ff8240a1890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff82409f260 tx=0x7ff82001c8d0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.114 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- 172.21.15.119:0/4032408928 shutdown_connections 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/4032408928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82409fac0 0x7ff8240a1890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/4032408928 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff8240a1dd0 0x7ff8240a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/4032408928 >> v2:172.21.15.17:3300/0 conn(0x7ff824135c50 0x7ff824136030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- 172.21.15.119:0/4032408928 >> 172.21.15.119:0/4032408928 conn(0x7ff8240a6480 msgr2=0x7ff8240a2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- 172.21.15.119:0/4032408928 shutdown_connections 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- 172.21.15.119:0/4032408928 wait complete. 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: build_initial_monmap 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: monmap: 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:19.115 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 Processor -- start 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- start start 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: init 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff82409fac0 0x7ff82413bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): picked mon.noname-c con 0x7ff82409fac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 0x7ff82406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): picked mon.noname-b con 0x7ff8240a1dd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:19.116 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff824135c50 0x7ff82406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): picked mon.noname-a con 0x7ff824135c50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff82413f4e0 con 0x7ff824135c50 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff82409ff80 con 0x7ff8240a1dd0 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff82413f660 con 0x7ff82409fac0 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82c3bf640 10 monclient(hunting): authenticate will time out at 1797.907770s 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82a95c640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff824135c50 msgr2=0x7ff82406e8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82a95c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff824135c50 0x7ff82406e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82995a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 0x7ff82406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:19.117 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.105+0000 7ff82995a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 0x7ff82406e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39036/0 (socket says 172.21.15.119:39036) 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 -- 172.21.15.119:0/469710439 learned_addr learned my addr 172.21.15.119:0/469710439 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): get_auth_request con 0x7ff8240a1dd0 auth_method 0 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff82409fac0 0x7ff82413bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): get_auth_request con 0x7ff82409fac0 auth_method 0 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:19.118 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient(hunting): handle_auth_done global_id 154270 payload 824 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 -- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff82409fac0 msgr2=0x7ff82413bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff82409fac0 0x7ff82413bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 -- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff824135c50 msgr2=0x7ff82406e8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff824135c50 0x7ff82406e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient: _finish_hunting 0 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 monclient: found mon.noname-b 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 -- 172.21.15.119:0/469710439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82413a040 con 0x7ff8240a1dd0 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82995a640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 0x7ff82406e380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff82001d170 tx=0x7ff820019670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:19.119 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200bc020 con 0x7ff8240a1dd0 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: dump: 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:19.120 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: _finish_auth 0 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8200b8670 con 0x7ff8240a1dd0 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200b8a40 con 0x7ff8240a1dd0 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 5 monclient: authenticate success, global_id 154270 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 monclient: _renew_subs 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82413a320 con 0x7ff8240a1dd0 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff817fff640 4 set_mon_vals no callback set 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:19.121 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 10 monclient: dump: 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 monclient: _renew_subs 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82409d070 con 0x7ff8240a1dd0 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:19.122 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff8200bf020 con 0x7ff8240a1dd0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8240a2250 con 0x7ff8240a1dd0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff80803b970 0x7ff80803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff8200eb4d0 con 0x7ff8240a1dd0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff80803b970 0x7ff80803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 10 monclient: get_auth_request con 0x7ff80803b970 auth_method 0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.109+0000 7ff82a15b640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff80803b970 0x7ff80803de30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0093d0 tx=0x7ff81c009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:19.123 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.113+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff8200c6070 con 0x7ff8240a1dd0 2024-09-15T21:31:19.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.113+0000 7ff816ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:19.124 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.113+0000 7ff816ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:19.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.337+0000 7ff82c3bf640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T21:31:19.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.337+0000 7ff82c3bf640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T21:31:19.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.337+0000 7ff82c3bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:19.337 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:19.337+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.foo"} v 0) -- 0x7ff8240a0350 con 0x7ff8240a1dd0 2024-09-15T21:31:21.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient: get_monmap_and_config 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient: build_initial_monmap 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient: monmap: 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:21.556055+0000 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:21.556055+0000 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:21.557 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:21.557 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:21.557 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 Processor -- start 2024-09-15T21:31:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 -- start start 2024-09-15T21:31:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient: init 2024-09-15T21:31:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2024-09-15T21:31:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54581379c0 0x7f545806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): picked mon.noname-a con 0x7f54581379c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f54580a63b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 0x7f545813ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): picked mon.noname-b con 0x7f54581384b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545809c2a0 con 0x7f54580a63b0 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f545809c840 con 0x7f54581379c0 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f545809c510 con 0x7f54581384b0 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f545d719640 10 monclient(hunting): authenticate will time out at 1804.951534s 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f5456ffd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f54581379c0 msgr2=0x7f545806f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f544ffff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f54580a63b0 legacy=0x7f5458137da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f5456ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54581379c0 0x7f545806f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 0x7f545813ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 0x7f545813ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56142/0 (socket says 172.21.15.17:56142) 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 1 -- 172.21.15.17:0/1027396798 learned_addr learned my addr 172.21.15.17:0/1027396798 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): get_auth_request con 0x7f54581384b0 auth_method 0 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.551+0000 7f54577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 10 monclient(hunting): handle_auth_done global_id 164159 payload 274 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 1 -- 172.21.15.17:0/1027396798 >> v2:172.21.15.17:3300/0 conn(0x7f54581379c0 msgr2=0x7f545806f5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 1 --2- 172.21.15.17:0/1027396798 >> v2:172.21.15.17:3300/0 conn(0x7f54581379c0 0x7f545806f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 1 -- 172.21.15.17:0/1027396798 >> v1:172.21.15.17:6789/0 conn(0x7f54580a63b0 legacy=0x7f5458137da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 10 monclient: _finish_hunting 0 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 1 monclient: found mon.noname-b 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:31:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 1 -- 172.21.15.17:0/1027396798 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545809d1d0 con 0x7f54581384b0 2024-09-15T21:31:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54577fe640 1 --2- 172.21.15.17:0/1027396798 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 0x7f545813ef00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5450009870 tx=0x7f545001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4691889adf2de9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 1 -- 172.21.15.17:0/1027396798 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500b0070 con 0x7f54581384b0 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: dump: 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: _finish_auth 0 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: _check_auth_tickets 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 5 monclient: authenticate success, global_id 164159 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 1 -- 172.21.15.17:0/1027396798 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54500ab600 con 0x7f54581384b0 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 4 set_mon_vals no callback set 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 1 -- 172.21.15.17:0/1027396798 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500ab9d0 con 0x7f54581384b0 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f54567fc640 10 monclient: dump: 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 10 monclient: shutdown 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 -- 172.21.15.17:0/1027396798 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 msgr2=0x7f545813ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 --2- 172.21.15.17:0/1027396798 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 0x7f545813ef00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5450009870 tx=0x7f545001c440 comp rx=0 tx=0).stop 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 -- 172.21.15.17:0/1027396798 shutdown_connections 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 --2- 172.21.15.17:0/1027396798 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54581384b0 0x7f545813ef00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 --2- 172.21.15.17:0/1027396798 >> v2:172.21.15.17:3300/0 conn(0x7f54581379c0 0x7f545806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 -- 172.21.15.17:0/1027396798 >> 172.21.15.17:0/1027396798 conn(0x7f54580a7100 msgr2=0x7f54580a7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 -- 172.21.15.17:0/1027396798 shutdown_connections 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 1 -- 172.21.15.17:0/1027396798 wait complete. 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 10 monclient: build_initial_monmap 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.555+0000 7f545d719640 10 monclient: monmap: 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:21.572 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 Processor -- start 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- start start 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: init 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:31:21.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 0x7f54581363a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): picked mon.noname-c con 0x7f54580a63b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54581379c0 0x7f54581368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): picked mon.noname-a con 0x7f54581379c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54581384b0 0x7f545813bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): picked mon.noname-b con 0x7f54581384b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5458142190 con 0x7f54581379c0 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5458142010 con 0x7f54581384b0 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:21.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5458142310 con 0x7f54580a63b0 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient(hunting): authenticate will time out at 1804.956721s 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544ffff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54581379c0 msgr2=0x7f54581368e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54581379c0 0x7f54581368e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 0x7f54581363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 0x7f54581363a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56156/0 (socket says 172.21.15.17:56156) 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 -- 172.21.15.17:0/1006946686 learned_addr learned my addr 172.21.15.17:0/1006946686 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): get_auth_request con 0x7f54580a63b0 auth_method 0 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:21.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient(hunting): handle_auth_done global_id 164162 payload 824 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f54577fe640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54581384b0 0x7f545813bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 -- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54581384b0 msgr2=0x7f545813bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54581384b0 0x7f545813bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 -- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54581379c0 msgr2=0x7f54581368e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54581379c0 0x7f54581368e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 monclient: found mon.noname-c 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 -- 172.21.15.17:0/1006946686 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5458121340 con 0x7f54580a63b0 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5456ffd640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 0x7f54581363a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f544800b9e0 tx=0x7f544800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800cba0 con 0x7f54580a63b0 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: dump: 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5448010070 con 0x7f54580a63b0 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448015470 con 0x7f54580a63b0 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 5 monclient: authenticate success, global_id 164162 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 10 monclient: dump: 2024-09-15T21:31:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _renew_subs 2024-09-15T21:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f5455ffb640 4 set_mon_vals no callback set 2024-09-15T21:31:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54580a13f0 con 0x7f54580a63b0 2024-09-15T21:31:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _renew_subs 2024-09-15T21:31:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f545813e410 con 0x7f54580a63b0 2024-09-15T21:31:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545806f5e0 con 0x7f54580a63b0 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f544801e3a0 con 0x7f54580a63b0 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f543803b970 0x7f543803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5448050900 con 0x7f54580a63b0 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544ffff640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f543803b970 0x7f543803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544ffff640 10 monclient: get_auth_request con 0x7f543803b970 auth_method 0 2024-09-15T21:31:21.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.559+0000 7f544ffff640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f543803b970 0x7f543803de30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:21.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.563+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5448005a80 con 0x7f54580a63b0 2024-09-15T21:31:21.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.563+0000 7f544f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:21.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.563+0000 7f544f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:21.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.799+0000 7f545d719640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:21.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.799+0000 7f545d719640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:21.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.799+0000 7f545d719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:21.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.799+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f545813d670 con 0x7f54580a63b0 2024-09-15T21:31:21.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f544f7fe640 1 -- 172.21.15.17:0/1006946686 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f5448007320 con 0x7f54580a63b0 2024-09-15T21:31:21.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f544f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:21.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f544f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:21.806 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":128,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f543803b970 msgr2=0x7f543803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f543803b970 0x7f543803de30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440006d40 comp rx=0 tx=0).stop 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 10 monclient: shutdown 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 msgr2=0x7f54581363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 0x7f54581363a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f544800b9e0 tx=0x7f544800bea0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 shutdown_connections 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f543803b970 0x7f543803de30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54581384b0 0x7f545813bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54581379c0 0x7f54581368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 --2- 172.21.15.17:0/1006946686 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f54580a63b0 0x7f54581363a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:21.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 >> 172.21.15.17:0/1006946686 conn(0x7f54580a7100 msgr2=0x7f54580a1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:21.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 shutdown_connections 2024-09-15T21:31:21.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:21.803+0000 7f545d719640 1 -- 172.21.15.17:0/1006946686 wait complete. 2024-09-15T21:31:22.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:22.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:22.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:22.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:22.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:22.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:22.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:22.109+0000 7ff815ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:23.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:23.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:23.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:23.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:23.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:23.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:24.106 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:24.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:24.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:24.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:24.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:24.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:24.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient: get_monmap_and_config 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient: build_initial_monmap 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient: monmap: 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:24.959339+0000 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:24.959339+0000 2024-09-15T21:31:24.959 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:24.960 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:24.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 Processor -- start 2024-09-15T21:31:24.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 -- start start 2024-09-15T21:31:24.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient: init 2024-09-15T21:31:24.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:24.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5d000a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): picked mon.noname-b con 0x7f5d000a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 0x7f5d0013f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): picked mon.noname-a con 0x7f5d000a59a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d000a2c30 con 0x7f5d000a4a90 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5d000a0f50 con 0x7f5d000a59a0 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5d000a1a00 con 0x7f5d000a5580 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5d0599c640 10 monclient(hunting): authenticate will time out at 1808.355502s 2024-09-15T21:31:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cfeffd640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f5d000a4a90 legacy=0x7f5d000a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cff7fe640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 msgr2=0x7f5d0013f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cff7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 0x7f5d0013f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:56162/0 (socket says 172.21.15.17:56162) 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 learned_addr learned my addr 172.21.15.17:0/2863684684 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request con 0x7f5d000a5580 auth_method 0 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:24.955+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:25.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:25.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:25.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:25.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:25.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:25.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:25.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 msgr2=0x7f5d0013f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:25.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v1:172.21.15.17:6789/0 conn(0x7f5d000a4a90 legacy=0x7f5d000a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:25.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 0x7f5d0013f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request con 0x7f5d000a5580 auth_method 0 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:25.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.159+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:31:25.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 msgr2=0x7f5d0013f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:25.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v1:172.21.15.17:6789/0 conn(0x7f5d000a4a90 legacy=0x7f5d000a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 0x7f5d0013f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request con 0x7f5d000a5580 auth_method 0 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:25.563+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:31:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:26.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:26.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:26.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:26.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 msgr2=0x7f5d0013f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:26.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v1:172.21.15.17:6789/0 conn(0x7f5d000a4a90 legacy=0x7f5d000a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:26.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 0x7f5d0013f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request con 0x7f5d000a5580 auth_method 0 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:26.367+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:31:27.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:27.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:27.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:27.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:27.108 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:27.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:27.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): tick 2024-09-15T21:31:27.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:31:27.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:31:27.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 msgr2=0x7f5d0013cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5d000a5580 0x7f5d0013cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 msgr2=0x7f5d0013f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.17:3300/0 conn(0x7f5d000a59a0 0x7f5d0013f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 -- 172.21.15.17:0/2863684684 >> v1:172.21.15.17:6789/0 conn(0x7f5d000a4a90 legacy=0x7f5d000a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient: _add_conns ranks=[4,1] 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): picked mon.noname-c con 0x7f5ce00036b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5ce0006220 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 -- 172.21.15.17:0/2863684684 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d000a1c90 con 0x7f5ce0006220 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 1 -- 172.21.15.17:0/2863684684 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5d000a0f50 con 0x7f5ce00036b0 2024-09-15T21:31:27.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfd7fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request con 0x7f5ce00036b0 auth_method 0 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:27.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:27.959+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:28.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:28.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:28.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:28.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:28.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:28.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request con 0x7f5ce00036b0 auth_method 0 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.163+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:31:28.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:28.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request con 0x7f5ce00036b0 auth_method 0 2024-09-15T21:31:28.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:28.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:28.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:28.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:28.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:28.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:28.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:28.567+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:31:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.109+0000 7ff815ffb640 10 monclient: tick 2024-09-15T21:31:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.109+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:29.107 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.109+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request con 0x7f5ce00036b0 auth_method 0 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.371+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:31:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.523+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203257830 0 0) 0x7f5d000a1c90 con 0x7f5ce0006220 2024-09-15T21:31:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.523+0000 7f5cfe7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:31:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.523+0000 7f5cfe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.523+0000 7f5cfe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.523+0000 7f5cfe7fc640 10 monclient(hunting): my global_id is 154273 2024-09-15T21:31:29.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.523+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cec003530 con 0x7f5ce0006220 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 752783220 0 0) 0x7f5cec003530 con 0x7f5ce0006220 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 msgr2=0x7f5ce0005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 --2- 172.21.15.17:0/2863684684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce00036b0 0x7f5ce0005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:31:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce000af90 con 0x7f5ce0006220 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: _finish_auth 0 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5ce800f040 con 0x7f5ce0006220 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: dump: 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 >> v1:172.21.15.119:6789/0 conn(0x7f5ce0006220 legacy=0x7f5ce00086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 reap_dead start 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5cec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): picked mon.b con 0x7f5cec0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5cec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): picked mon.a con 0x7f5cec004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5cec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): picked mon.c con 0x7f5cec0052a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5d000a0f50 con 0x7f5cec004810 2024-09-15T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec00c1f0 con 0x7f5cec0039c0 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5cec00c3f0 con 0x7f5cec0052a0 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfe7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cf67fc640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 msgr2=0x7f5cec004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5cec004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5cec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5cec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request con 0x7f5cec0039c0 auth_method 0 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): get_auth_request con 0x7f5cec0052a0 auth_method 0 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_done global_id 154273 payload 293 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 msgr2=0x7f5cec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5cec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 msgr2=0x7f5cec004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5cec004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 monclient: found mon.c 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 1 -- 172.21.15.17:0/2863684684 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec003530 con 0x7f5cec0052a0 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5cff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5d0599c640 5 monclient: authenticate success, global_id 154273 2024-09-15T21:31:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.535+0000 7f5d0599c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:29.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.541+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.foo"}]=0 v24) ==== 80+0+0 (secure 0 0 0) 0x7ff8200165e0 con 0x7ff8240a1dd0 2024-09-15T21:31:29.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.545+0000 7ff816ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T21:31:29.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.545+0000 7ff816ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cff7fe640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5cec00b9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d000a5d80 tx=0x7f5cf801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b711b767fbfc0509 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8007980 con 0x7f5cec0052a0 2024-09-15T21:31:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: dump: 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf80ac070 con 0x7f5cec0052a0 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 1 -- 172.21.15.17:0/2863684684 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80a68f0 con 0x7f5cec0052a0 2024-09-15T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 4 set_mon_vals no callback set 2024-09-15T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5cfe7fc640 10 monclient: dump: 2024-09-15T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 10 monclient: shutdown 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 -- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 msgr2=0x7f5cec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5cec00b9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d000a5d80 tx=0x7f5cf801c850 comp rx=0 tx=0).stop 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 -- 172.21.15.17:0/2863684684 shutdown_connections 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5cec00b9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5cec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 --2- 172.21.15.17:0/2863684684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5cec003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 -- 172.21.15.17:0/2863684684 >> 172.21.15.17:0/2863684684 conn(0x7f5d000aabb0 msgr2=0x7f5d000a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 -- 172.21.15.17:0/2863684684 shutdown_connections 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.543+0000 7f5d0599c640 1 -- 172.21.15.17:0/2863684684 wait complete. 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient: build_initial_monmap 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient: monmap: 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 Processor -- start 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 -- start start 2024-09-15T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient: init 2024-09-15T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5d00124120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): picked mon.noname-b con 0x7f5cec0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5d001cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): picked mon.noname-a con 0x7f5cec004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5d00136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): picked mon.noname-c con 0x7f5cec0052a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0006f2d0 con 0x7f5cec004810 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0006f150 con 0x7f5cec0039c0 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5d00138060 con 0x7f5cec0052a0 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5d0599c640 10 monclient(hunting): authenticate will time out at 1812.947330s 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cf67fc640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 msgr2=0x7f5d001cf890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cf67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5d001cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5d00136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5d00136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48530/0 (socket says 172.21.15.17:48530) 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 1 -- 172.21.15.17:0/1637990344 learned_addr learned my addr 172.21.15.17:0/1637990344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): get_auth_request con 0x7f5cec0052a0 auth_method 0 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5d00124120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request con 0x7f5cec0039c0 auth_method 0 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.547+0000 7f5cff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 10 monclient(hunting): handle_auth_done global_id 154276 payload 824 2024-09-15T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 -- 172.21.15.17:0/1637990344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 msgr2=0x7f5d00136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5d00136f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 -- 172.21.15.17:0/1637990344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 msgr2=0x7f5d001cf890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5d001cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 monclient: found mon.noname-b 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 -- 172.21.15.17:0/1637990344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d000a9e80 con 0x7f5cec0039c0 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfeffd640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5d00124120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0003fd0 tx=0x7f5cf0005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf001d020 con 0x7f5cec0039c0 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: dump: 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: _finish_auth 0 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf000cc20 con 0x7f5cec0039c0 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 5 monclient: authenticate success, global_id 154276 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 monclient: _renew_subs 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d00124a90 con 0x7f5cec0039c0 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0019670 con 0x7f5cec0039c0 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 10 monclient: dump: 2024-09-15T21:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfdffb640 4 set_mon_vals no callback set 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 monclient: _renew_subs 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d00125080 con 0x7f5cec0039c0 2024-09-15T21:31:29.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d001384f0 con 0x7f5cec0039c0 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5cf0020020 con 0x7f5cec0039c0 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ce003c330 0x7f5ce003e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5cf0052bf0 con 0x7f5cec0039c0 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ce003c330 0x7f5ce003e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.551+0000 7f5cf67fc640 10 monclient: get_auth_request con 0x7f5ce003c330 auth_method 0 2024-09-15T21:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.555+0000 7f5cf67fc640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ce003c330 0x7f5ce003e7f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007fd0 tx=0x7f5ce8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.563+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5cf0027030 con 0x7f5cec0039c0 2024-09-15T21:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.563+0000 7f5cfcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.563+0000 7f5cfcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:29.714 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T21:31:29.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T21:31:29.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:29.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.foo"} v 0) -- 0x7ff8240008e0 con 0x7ff8240a1dd0 2024-09-15T21:31:29.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff816ffd640 1 -- 172.21.15.119:0/469710439 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.foo"}]=0 v24) ==== 80+0+0 (secure 0 0 0) 0x7ff8200b73d0 con 0x7ff8240a1dd0 2024-09-15T21:31:29.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff816ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T21:31:29.715 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff816ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff80803b970 msgr2=0x7ff80803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff80803b970 0x7ff80803de30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0093d0 tx=0x7ff81c009360 comp rx=0 tx=0).stop 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 10 monclient: shutdown 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 msgr2=0x7ff82406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 0x7ff82406e380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff82001d170 tx=0x7ff820019670 comp rx=0 tx=0).stop 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 shutdown_connections 2024-09-15T21:31:29.718 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff80803b970 0x7ff80803de30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff824135c50 0x7ff82406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8240a1dd0 0x7ff82406e380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 --2- 172.21.15.119:0/469710439 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff82409fac0 0x7ff82413bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 >> 172.21.15.119:0/469710439 conn(0x7ff8240a6480 msgr2=0x7ff8241346e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 shutdown_connections 2024-09-15T21:31:29.719 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:31:29.717+0000 7ff82c3bf640 1 -- 172.21.15.119:0/469710439 wait complete. 2024-09-15T21:31:29.733 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 13 2024-09-15T21:31:29.734 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T21:31:29.734 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-09-15T21:31:29.734 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:31:29.734 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T21:31:29.735 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T21:31:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5d0599c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5d0599c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5d0599c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5d00069200 con 0x7f5cec0039c0 2024-09-15T21:31:29.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5cfcff9640 1 -- 172.21.15.17:0/1637990344 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f5cf0004000 con 0x7f5cec0039c0 2024-09-15T21:31:29.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5cfcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:29.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.815+0000 7f5cfcff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:29.821 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":132,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ce003c330 msgr2=0x7f5ce003e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ce003c330 0x7f5ce003e7f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007fd0 tx=0x7f5ce8008040 comp rx=0 tx=0).stop 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 10 monclient: shutdown 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 msgr2=0x7f5d00124120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5d00124120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0003fd0 tx=0x7f5cf0005900 comp rx=0 tx=0).stop 2024-09-15T21:31:29.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 shutdown_connections 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ce003c330 0x7f5ce003e7f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5cec0052a0 0x7f5d00136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cec004810 0x7f5d001cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 --2- 172.21.15.17:0/1637990344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cec0039c0 0x7f5d00124120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 >> 172.21.15.17:0/1637990344 conn(0x7f5d000aabb0 msgr2=0x7f5d001c1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 shutdown_connections 2024-09-15T21:31:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.819+0000 7f5d0599c640 1 -- 172.21.15.17:0/1637990344 wait complete. 2024-09-15T21:31:29.841 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:31:29.841 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:31:29.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:31:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:31:29.845 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=''\''allow command service with prefix=list, allow command quorum_status'\''' 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph auth get-or-create client.bar mon '\''allow command service with prefix=list, allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.bar.keyring' 0 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph auth get-or-create client.bar mon '\''allow command service with prefix=list, allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.bar.keyring' 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=0 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph auth get-or-create client.bar mon ''\''allow' command service with prefix=list, allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stdout:ceph auth get-or-create client.bar mon 'allow command service with prefix=list, allow command quorum_status' > /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T21:31:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph auth get-or-create client.bar mon ''\''allow' command service with prefix=list, allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient: get_monmap_and_config 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient: build_initial_monmap 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient: monmap: 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:29.967291+0000 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:29.967291+0000 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:29.967 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 1 Processor -- start 2024-09-15T21:31:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 1 -- start start 2024-09-15T21:31:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient: init 2024-09-15T21:31:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:29.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f99601378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99601383b0 0x7f996013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient(hunting): picked mon.noname-b con 0x7f99601383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f996013d2c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99600a1a20 con 0x7f996013d2c0 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99600a1d00 con 0x7f99601378c0 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f99600a1fe0 con 0x7f99601383b0 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f9966c4a640 10 monclient(hunting): authenticate will time out at 1813.363392s 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f99651e7640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f996013d2c0 legacy=0x7f996013f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f99649e6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f99601378c0 0x7f9960137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:44520/0 (socket says 172.21.15.17:44520) 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f99649e6640 1 -- 172.21.15.17:0/2727976285 learned_addr learned my addr 172.21.15.17:0/2727976285 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99601383b0 0x7f996013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): get_auth_request con 0x7f99601383b0 auth_method 0 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient(hunting): handle_auth_done global_id 164177 payload 274 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995f7fe640 1 -- 172.21.15.17:0/2727976285 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 860824410 0 0) 0x7f99600a1d00 con 0x7f99601378c0 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 1 -- 172.21.15.17:0/2727976285 >> v1:172.21.15.119:6789/0 conn(0x7f99601378c0 legacy=0x7f9960137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 1 -- 172.21.15.17:0/2727976285 >> v1:172.21.15.17:6789/0 conn(0x7f996013d2c0 legacy=0x7f996013f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 1 monclient: found mon.noname-b 2024-09-15T21:31:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.963+0000 7f995ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:31:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995ffff640 1 -- 172.21.15.17:0/2727976285 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99600a22c0 con 0x7f99601383b0 2024-09-15T21:31:29.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:31:29.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995ffff640 1 --2- 172.21.15.17:0/2727976285 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99601383b0 0x7f996013cd80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99540097f0 tx=0x7f995401bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6b2450288f7bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 1 -- 172.21.15.17:0/2727976285 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540ab070 con 0x7f99601383b0 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: dump: 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.979 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 1 -- 172.21.15.17:0/2727976285 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540a5620 con 0x7f99601383b0 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 1 -- 172.21.15.17:0/2727976285 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540a59f0 con 0x7f99601383b0 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 5 monclient: authenticate success, global_id 164177 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 4 set_mon_vals no callback set 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f995f7fe640 10 monclient: dump: 2024-09-15T21:31:29.980 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 10 monclient: shutdown 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 -- 172.21.15.17:0/2727976285 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99601383b0 msgr2=0x7f996013cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 --2- 172.21.15.17:0/2727976285 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99601383b0 0x7f996013cd80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99540097f0 tx=0x7f995401bea0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 -- 172.21.15.17:0/2727976285 shutdown_connections 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 --2- 172.21.15.17:0/2727976285 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f99601383b0 0x7f996013cd80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 -- 172.21.15.17:0/2727976285 >> 172.21.15.17:0/2727976285 conn(0x7f99600aac40 msgr2=0x7f99600a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 -- 172.21.15.17:0/2727976285 shutdown_connections 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 -- 172.21.15.17:0/2727976285 wait complete. 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 10 monclient: build_initial_monmap 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 10 monclient: monmap: 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.982 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.967+0000 7f9966c4a640 1 Processor -- start 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- start start 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: init 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99601378c0 0x7f99601d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): picked mon.noname-b con 0x7f99601378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99601383b0 0x7f9960136c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): picked mon.noname-a con 0x7f99601383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:29.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 0x7f9960137160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): picked mon.noname-c con 0x7f996013d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9960144340 con 0x7f99601383b0 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f99601441c0 con 0x7f99601378c0 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f99601444c0 con 0x7f996013d2c0 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): authenticate will time out at 1813.368605s 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995ffff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99601383b0 msgr2=0x7f9960136c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99601383b0 0x7f9960136c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 0x7f9960137160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 0x7f9960137160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48558/0 (socket says 172.21.15.17:48558) 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 -- 172.21.15.17:0/1045954884 learned_addr learned my addr 172.21.15.17:0/1045954884 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): get_auth_request con 0x7f996013d2c0 auth_method 0 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99601378c0 0x7f99601d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): get_auth_request con 0x7f99601378c0 auth_method 0 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient(hunting): handle_auth_done global_id 164180 payload 824 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99601378c0 msgr2=0x7f99601d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99601378c0 0x7f99601d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99601383b0 msgr2=0x7f9960136c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99601383b0 0x7f9960136c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient: _finish_hunting 0 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 monclient: found mon.noname-c 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99601d1910 con 0x7f996013d2c0 2024-09-15T21:31:29.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99651e7640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 0x7f9960137160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f995800b440 tx=0x7f995800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958017070 con 0x7f996013d2c0 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: dump: 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.987 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9958003e10 con 0x7f996013d2c0 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 5 monclient: authenticate success, global_id 164180 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: _renew_subs 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99600a0680 con 0x7f996013d2c0 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958006310 con 0x7f996013d2c0 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: dump: 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995e7fc640 4 set_mon_vals no callback set 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: _renew_subs 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f996009f5a0 con 0x7f996013d2c0 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 0x7f996009d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9960140760 con 0x7f996009cc30 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 0x7f996009d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient: get_auth_request con 0x7f996009cc30 auth_method 0 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f995801a070 con 0x7f996013d2c0 2024-09-15T21:31:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 10 monclient(hunting): handle_auth_done global_id 164183 payload 824 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9960137ca0 con 0x7f996009cc30 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f994c03b970 0x7f994c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f995804f930 con 0x7f996013d2c0 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f99649e6640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 0x7f996009d010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f90 tx=0x7f9950007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000f900 con 0x7f996009cc30 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995ffff640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f994c03b970 0x7f994c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: dump: 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995ffff640 10 monclient: get_auth_request con 0x7f994c03b970 auth_method 0 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9950018070 con 0x7f996009cc30 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 msgr2=0x7f996009d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 0x7f996009d010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f90 tx=0x7f9950007a90 comp rx=0 tx=0).stop 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 msgr2=0x7f996009d010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995d7fa640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009cc30 0x7f996009d010 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f90 tx=0x7f9950007a90 comp rx=0 tx=0).stop 2024-09-15T21:31:29.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.971+0000 7f995ffff640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f994c03b970 0x7f994c03de30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f99540ad000 tx=0x7f9954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 0x7f9960138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f996013cf50 con 0x7f996009d550 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 0x7f9960138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient: get_auth_request con 0x7f996009d550 auth_method 0 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 10 monclient(hunting): handle_auth_done global_id 164186 payload 824 2024-09-15T21:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 1 -- 172.21.15.17:0/1045954884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9960069270 con 0x7f996009d550 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99649e6640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 0x7f9960138d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f90 tx=0x7f9950001260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950013600 con 0x7f996009d550 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 10 monclient: dump: 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4868 (secure 0 0 0) 0x7f9950013600 con 0x7f996009d550 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 msgr2=0x7f9960138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 0x7f9960138d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f90 tx=0x7f9950001260 comp rx=0 tx=0).stop 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 msgr2=0x7f9960138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f995d7fa640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996009d550 0x7f9960138d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:29.997 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 132, 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 38563, 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:29.998 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:30.000 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:30.001 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:30.002 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:30.003 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:30.006 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:30.008 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.010 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:30.011 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f994c03b970 msgr2=0x7f994c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f994c03b970 0x7f994c03de30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f99540ad000 tx=0x7f9954008040 comp rx=0 tx=0).stop 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 10 monclient: shutdown 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 msgr2=0x7f9960137160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f99651e7640 1 -- 172.21.15.17:0/1045954884 reap_dead start 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 0x7f9960137160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f995800b440 tx=0x7f995800b900 comp rx=0 tx=0).stop 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 shutdown_connections 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 --2- 172.21.15.17:0/1045954884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f996013d2c0 0x7f9960137160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 >> 172.21.15.17:0/1045954884 conn(0x7f99600aac40 msgr2=0x7f996013f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 shutdown_connections 2024-09-15T21:31:30.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:29.979+0000 7f9966c4a640 1 -- 172.21.15.17:0/1045954884 wait complete. 2024-09-15T21:31:30.014 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:31:30.014 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:31:30.014 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.127+0000 7faa00bd6640 10 monclient: get_monmap_and_config 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.127+0000 7faa00bd6640 10 monclient: build_initial_monmap 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.127+0000 7faa00bd6640 10 monclient: monmap: 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:30.133523+0000 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:30.133523+0000 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 Processor -- start 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 -- start start 2024-09-15T21:31:30.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient: init 2024-09-15T21:31:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): picked mon.noname-b con 0x7fa9fc0a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9fc0a53b0 0x7fa9fc0a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): picked mon.noname-c con 0x7fa9fc0a53b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9fc137850 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9fc0a2c30 con 0x7fa9fc137850 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa9fc0a0f00 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc0a19b0 con 0x7fa9fc0a4a90 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7faa00bd6640 10 monclient(hunting): authenticate will time out at 1813.529063s 2024-09-15T21:31:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fad9d640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fa9fc137850 legacy=0x7fa9fc137ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48592/0 (socket says 172.21.15.17:48592) 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 -- 172.21.15.17:0/3589770644 learned_addr learned my addr 172.21.15.17:0/3589770644 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): get_auth_request con 0x7fa9fc0a4a90 auth_method 0 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9f9d9b640 1 --2- 172.21.15.17:0/3589770644 >> v2:172.21.15.119:3300/0 conn(0x7fa9fc0a53b0 0x7fa9fc0a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fa9fc0a53b0 auth_method 0 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_done global_id 164189 payload 274 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 -- 172.21.15.17:0/3589770644 >> v2:172.21.15.119:3300/0 conn(0x7fa9fc0a53b0 msgr2=0x7fa9fc0a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/3589770644 >> v2:172.21.15.119:3300/0 conn(0x7fa9fc0a53b0 0x7fa9fc0a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 -- 172.21.15.17:0/3589770644 >> v1:172.21.15.17:6789/0 conn(0x7fa9fc137850 legacy=0x7fa9fc137ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient: _finish_hunting 0 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 monclient: found mon.noname-b 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:31:30.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.131+0000 7fa9fa59c640 1 -- 172.21.15.17:0/3589770644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc0a1c90 con 0x7fa9fc0a4a90 2024-09-15T21:31:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/3589770644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc0a4e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009870 tx=0x7fa9ec01c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5466c546fce5ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 1 -- 172.21.15.17:0/3589770644 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0b0070 con 0x7fa9fc0a4a90 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: dump: 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: _finish_auth 0 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: _check_auth_tickets 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 1 -- 172.21.15.17:0/3589770644 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ec0ab600 con 0x7fa9fc0a4a90 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 5 monclient: authenticate success, global_id 164189 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 4 set_mon_vals no callback set 2024-09-15T21:31:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 1 -- 172.21.15.17:0/3589770644 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0ab9d0 con 0x7fa9fc0a4a90 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f959a640 10 monclient: dump: 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:30.149 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: shutdown 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- 172.21.15.17:0/3589770644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 msgr2=0x7fa9fc0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 --2- 172.21.15.17:0/3589770644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc0a4e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009870 tx=0x7fa9ec01c2d0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- 172.21.15.17:0/3589770644 shutdown_connections 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 --2- 172.21.15.17:0/3589770644 >> v2:172.21.15.119:3300/0 conn(0x7fa9fc0a53b0 0x7fa9fc0a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 --2- 172.21.15.17:0/3589770644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc0a4e70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- 172.21.15.17:0/3589770644 >> 172.21.15.17:0/3589770644 conn(0x7fa9fc0aabb0 msgr2=0x7fa9fc0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- 172.21.15.17:0/3589770644 shutdown_connections 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- 172.21.15.17:0/3589770644 wait complete. 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: build_initial_monmap 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: monmap: 2024-09-15T21:31:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 Processor -- start 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- start start 2024-09-15T21:31:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: init 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc1cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): picked mon.noname-c con 0x7fa9fc0a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc0a53b0 0x7fa9fc1369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): picked mon.noname-b con 0x7fa9fc0a53b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9fc137850 0x7fa9fc13bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): picked mon.noname-a con 0x7fa9fc137850 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc1426d0 con 0x7fa9fc137850 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc142550 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc142850 con 0x7fa9fc0a4a90 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7faa00bd6640 10 monclient(hunting): authenticate will time out at 1813.534977s 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc1cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fad9d640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9fc137850 msgr2=0x7fa9fc13bcf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc1cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48594/0 (socket says 172.21.15.17:48594) 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 -- 172.21.15.17:0/2192686775 learned_addr learned my addr 172.21.15.17:0/2192686775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 10 monclient(hunting): get_auth_request con 0x7fa9fc0a4a90 auth_method 0 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 msgr2=0x7fa9fc1cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send send error: (32) Broken pipe 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc1cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).write auth request write failed r=-32 ((32) Broken pipe) 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fad9d640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9fc137850 0x7fa9fc13bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc1cfc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc0a53b0 0x7fa9fc1369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fa9fc0a53b0 auth_method 0 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.135+0000 7fa9f9d9b640 10 monclient(hunting): handle_auth_done global_id 154300 payload 824 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 msgr2=0x7fa9fc1cfc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa9fc0a4a90 0x7fa9fc1cfc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9fc137850 msgr2=0x7fa9fc13bcf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9fc137850 0x7fa9fc13bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 monclient: found mon.noname-b 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc121360 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f9d9b640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc0a53b0 0x7fa9fc1369d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0007e60 tx=0x7fa9f000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0014620 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: dump: 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:30.156 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: _finish_auth 0 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 5 monclient: authenticate success, global_id 154300 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 10 monclient: _renew_subs 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc0a1280 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 10 monclient: _renew_subs 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc120ac0 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f0019070 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00149c0 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: dump: 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9f8d99640 4 set_mon_vals no callback set 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 0x7fa9e0004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0007cc0 con 0x7fa9e0002890 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 0x7fa9e0004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient: get_auth_request con 0x7fa9e0002890 auth_method 0 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa9f0014b60 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:30.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9dc03b970 0x7fa9dc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa9f0050e00 con 0x7fa9fc0a53b0 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_done global_id 154303 payload 824 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9e0007ec0 con 0x7fa9e0002890 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fad9d640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9dc03b970 0x7fa9dc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fad9d640 10 monclient: get_auth_request con 0x7fa9dc03b970 auth_method 0 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 0x7fa9e0004d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec01c100 tx=0x7fa9ec019070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0bb070 con 0x7fa9e0002890 2024-09-15T21:31:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: dump: 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:30.161 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9fad9d640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9dc03b970 0x7fa9dc03de30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4003940 tx=0x7fa9f4006c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa9ec0abcf0 con 0x7fa9e0002890 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 msgr2=0x7fa9e0004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 0x7fa9e0004d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec01c100 tx=0x7fa9ec019070 comp rx=0 tx=0).stop 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 msgr2=0x7fa9e0004d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:30.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.139+0000 7fa9eaffd640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0002890 0x7fa9e0004d30 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec01c100 tx=0x7fa9ec019070 comp rx=0 tx=0).stop 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7faa00bd6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7faa00bd6640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 0x7fa9e0007170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7faa00bd6640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e000a200 con 0x7fa9e0006ce0 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 0x7fa9e0007170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient: get_auth_request con 0x7fa9e0006ce0 auth_method 0 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:30.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 10 monclient(hunting): handle_auth_done global_id 154306 payload 824 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 1 -- 172.21.15.17:0/2192686775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa9e00076b0 con 0x7fa9e0006ce0 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9fa59c640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 0x7fa9e0007170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007e90 tx=0x7fa9ec008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0b0040 con 0x7fa9e0006ce0 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 10 monclient: dump: 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:30.165 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4536 (secure 0 0 0) 0x7fa9ec0b7460 con 0x7fa9e0006ce0 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 msgr2=0x7fa9e0007170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 0x7fa9e0007170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007e90 tx=0x7fa9ec008040 comp rx=0 tx=0).stop 2024-09-15T21:31:30.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 msgr2=0x7fa9e0007170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.147+0000 7fa9eaffd640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0006ce0 0x7fa9e0007170 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007e90 tx=0x7fa9ec008040 comp rx=0 tx=0).stop 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 132, 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 94682, 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:31:30.167 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:30.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:31:30.170 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:31:30.171 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:31:30.172 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:30.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:30.174 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:31:30.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:30.176 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:31:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:31:30.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:31:30.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:31:30.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:31:30.180 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:31:30.180 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:31:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9dc03b970 msgr2=0x7fa9dc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa9dc03b970 0x7fa9dc03de30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4003940 tx=0x7fa9f4006c50 comp rx=0 tx=0).stop 2024-09-15T21:31:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 10 monclient: shutdown 2024-09-15T21:31:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc0a53b0 msgr2=0x7fa9fc1369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 --2- 172.21.15.17:0/2192686775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc0a53b0 0x7fa9fc1369d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0007e60 tx=0x7fa9f000e4b0 comp rx=0 tx=0).stop 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7fa9fad9d640 1 -- 172.21.15.17:0/2192686775 reap_dead start 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7fa9fad9d640 1 -- 172.21.15.17:0/2192686775 reap_dead start 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 shutdown_connections 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 >> 172.21.15.17:0/2192686775 conn(0x7fa9fc0aabb0 msgr2=0x7fa9fc13ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 shutdown_connections 2024-09-15T21:31:30.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:30.151+0000 7faa00bd6640 1 -- 172.21.15.17:0/2192686775 wait complete. 2024-09-15T21:31:30.182 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:31:30.182 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:31:30.366 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=0 2024-09-15T21:31:30.367 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T21:31:30.367 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:30.367 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 0 2024-09-15T21:31:30.367 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 2024-09-15T21:31:30.367 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=0 2024-09-15T21:31:30.367 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T21:31:30.368 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T21:31:30.368 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T21:31:30.893 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=0 2024-09-15T21:31:30.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T21:31:30.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:30.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls' 13 2024-09-15T21:31:30.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls' 2024-09-15T21:31:30.894 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:31:30.895 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-09-15T21:31:30.895 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-09-15T21:31:30.895 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-09-15T21:31:31.226 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export' 13 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export' 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-09-15T21:31:31.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-09-15T21:31:31.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:31:31.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:31:31.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:31.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo' 13 2024-09-15T21:31:31.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo' 2024-09-15T21:31:31.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:31:31.573 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-09-15T21:31:31.573 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-09-15T21:31:31.573 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-09-15T21:31:31.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:31:31.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:31:31.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:31.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump' 13 2024-09-15T21:31:31.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump' 2024-09-15T21:31:31.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:31:31.934 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-09-15T21:31:31.934 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-09-15T21:31:31.934 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-09-15T21:31:32.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:31:32.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:31:32.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:31:32.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T21:31:32.217 INFO:tasks.workunit.client.0.smithi119.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T21:31:32.217 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-09-15T21:31:32.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump' 110 2024-09-15T21:31:32.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump' 2024-09-15T21:31:32.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=110 2024-09-15T21:31:32.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-09-15T21:31:32.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-09-15T21:31:50.183 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:31:50.183 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:31:50.184 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:31:50.187 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:31:50.187 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:31:50.187 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:50.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient: get_monmap_and_config 2024-09-15T21:31:50.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient: build_initial_monmap 2024-09-15T21:31:50.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient: monmap: 2024-09-15T21:31:50.307 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:50.307 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:50.307362+0000 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:50.307362+0000 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:50.308 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 Processor -- start 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 -- start start 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient: init 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f66a41378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a41383b0 0x7f66a413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): picked mon.noname-b con 0x7f66a41383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 0x7f66a413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): picked mon.noname-a con 0x7f66a413d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:50.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a40a1a20 con 0x7f66a41378c0 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f66a40a1d00 con 0x7f66a413d2c0 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f66a40a1fe0 con 0x7f66a41383b0 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66ab342640 10 monclient(hunting): authenticate will time out at 1833.702394s 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a41383b0 0x7f66a413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a90de640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f66a41378c0 0x7f66a4137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43154/0 (socket says 172.21.15.17:43154) 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 0x7f66a413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a90de640 1 -- 172.21.15.17:0/3980446938 learned_addr learned my addr 172.21.15.17:0/3980446938 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:31:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): get_auth_request con 0x7f66a41383b0 auth_method 0 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 10 monclient(hunting): get_auth_request con 0x7f66a413d2c0 auth_method 0 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 1 -- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 msgr2=0x7f66a413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:31:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 1 -- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 msgr2=0x7f66a413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 1 --2- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 0x7f66a413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a98df640 1 --2- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 0x7f66a413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient(hunting): handle_auth_done global_id 164213 payload 274 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 1 -- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 msgr2=0x7f66a413f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 1 --2- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 0x7f66a413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 1 -- 172.21.15.17:0/3980446938 >> v1:172.21.15.17:6789/0 conn(0x7f66a41378c0 legacy=0x7f66a4137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 1 monclient: found mon.noname-b 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.303+0000 7f66a88dd640 1 -- 172.21.15.17:0/3980446938 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a40a22c0 con 0x7f66a41383b0 2024-09-15T21:31:50.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a88dd640 1 --2- 172.21.15.17:0/3980446938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a41383b0 0x7f66a413cd80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66940097c0 tx=0x7f669401c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a04c20c94ae73d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 1 -- 172.21.15.17:0/3980446938 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940b1070 con 0x7f66a41383b0 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: dump: 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:50.316 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: _finish_auth 0 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 1 -- 172.21.15.17:0/3980446938 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66940a47f0 con 0x7f66a41383b0 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 5 monclient: authenticate success, global_id 164213 2024-09-15T21:31:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 4 set_mon_vals no callback set 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 1 -- 172.21.15.17:0/3980446938 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940a4bc0 con 0x7f66a41383b0 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f669bfff640 10 monclient: dump: 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: shutdown 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- 172.21.15.17:0/3980446938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a41383b0 msgr2=0x7f66a413cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 --2- 172.21.15.17:0/3980446938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a41383b0 0x7f66a413cd80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66940097c0 tx=0x7f669401c040 comp rx=0 tx=0).stop 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- 172.21.15.17:0/3980446938 shutdown_connections 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 --2- 172.21.15.17:0/3980446938 >> v2:172.21.15.17:3300/0 conn(0x7f66a413d2c0 0x7f66a413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 --2- 172.21.15.17:0/3980446938 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a41383b0 0x7f66a413cd80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- 172.21.15.17:0/3980446938 >> 172.21.15.17:0/3980446938 conn(0x7f66a40aac40 msgr2=0x7f66a40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- 172.21.15.17:0/3980446938 shutdown_connections 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- 172.21.15.17:0/3980446938 wait complete. 2024-09-15T21:31:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: build_initial_monmap 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: monmap: 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:50.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 Processor -- start 2024-09-15T21:31:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- start start 2024-09-15T21:31:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: init 2024-09-15T21:31:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:31:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): picked mon.noname-a con 0x7f66a41378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a41383b0 0x7f66a40a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): picked mon.noname-b con 0x7f66a41383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a413d2c0 0x7f66a40a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): picked mon.noname-c con 0x7f66a413d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4143040 con 0x7f66a41378c0 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4142ec0 con 0x7f66a41383b0 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f66a41431c0 con 0x7f66a413d2c0 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66ab342640 10 monclient(hunting): authenticate will time out at 1833.707332s 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a413d2c0 0x7f66a40a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50754/0 (socket says 172.21.15.17:50754) 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 -- 172.21.15.17:0/2148470605 learned_addr learned my addr 172.21.15.17:0/2148470605 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 10 monclient(hunting): get_auth_request con 0x7f66a41378c0 auth_method 0 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): get_auth_request con 0x7f66a413d2c0 auth_method 0 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 -- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 msgr2=0x7f66a413c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 -- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 msgr2=0x7f66a413c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a90de640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a88dd640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a41383b0 0x7f66a40a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:50.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient(hunting): handle_auth_done global_id 164216 payload 824 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 -- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a41383b0 msgr2=0x7f66a40a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a41383b0 0x7f66a40a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 -- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 msgr2=0x7f66a413c040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient: _finish_hunting 0 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 monclient: found mon.noname-c 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a98df640 1 -- 172.21.15.17:0/2148470605 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a409e870 con 0x7f66a413d2c0 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a88dd640 10 monclient: get_auth_request con 0x7f66a41383b0 auth_method 0 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.307+0000 7f66a88dd640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a41383b0 0x7f66a40a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:31:50.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66a98df640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a413d2c0 0x7f66a40a05c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66a40a1760 tx=0x7f669c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c00eda0 con 0x7f66a413d2c0 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: dump: 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:50.329 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: _finish_auth 0 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f669c0039a0 con 0x7f66a413d2c0 2024-09-15T21:31:50.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 5 monclient: authenticate success, global_id 164216 2024-09-15T21:31:50.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 monclient: _renew_subs 2024-09-15T21:31:50.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:50.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a413d010 con 0x7f66a413d2c0 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c003dd0 con 0x7f66a413d2c0 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 10 monclient: dump: 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:50.333 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 monclient: _renew_subs 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a409cc70 con 0x7f66a413d2c0 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f669affd640 4 set_mon_vals no callback set 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a4137ca0 con 0x7f66a413d2c0 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f669c01b020 con 0x7f66a413d2c0 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f668803b9c0 0x7f668803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:50.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f669c051650 con 0x7f66a413d2c0 2024-09-15T21:31:50.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66a90de640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f668803b9c0 0x7f668803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:50.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66a90de640 10 monclient: get_auth_request con 0x7f668803b9c0 auth_method 0 2024-09-15T21:31:50.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.311+0000 7f66a90de640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f668803b9c0 0x7f668803de80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f66a0000f70 tx=0x7f66a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:50.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.315+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f669c04e620 con 0x7f66a413d2c0 2024-09-15T21:31:50.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.315+0000 7f6699ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:50.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.315+0000 7f6699ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f66ab342640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f66ab342640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f66ab342640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f66a4069270 con 0x7f66a413d2c0 2024-09-15T21:31:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f6699ffb640 1 -- 172.21.15.17:0/2148470605 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f669c014420 con 0x7f66a413d2c0 2024-09-15T21:31:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f6699ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:50.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.547+0000 7f6699ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:50.553 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":132,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f668803b9c0 msgr2=0x7f668803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f668803b9c0 0x7f668803de80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f66a0000f70 tx=0x7f66a0009040 comp rx=0 tx=0).stop 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 10 monclient: shutdown 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a413d2c0 msgr2=0x7f66a40a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a413d2c0 0x7f66a40a05c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66a40a1760 tx=0x7f669c00e4b0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 shutdown_connections 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f668803b9c0 0x7f668803de80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f66a413d2c0 0x7f66a40a05c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a41383b0 0x7f66a40a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 --2- 172.21.15.17:0/2148470605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66a41378c0 0x7f66a413c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:50.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 >> 172.21.15.17:0/2148470605 conn(0x7f66a40aac40 msgr2=0x7f66a4136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:50.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 shutdown_connections 2024-09-15T21:31:50.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:50.551+0000 7f66ab342640 1 -- 172.21.15.17:0/2148470605 wait complete. 2024-09-15T21:31:53.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient: get_monmap_and_config 2024-09-15T21:31:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient: build_initial_monmap 2024-09-15T21:31:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient: monmap: 2024-09-15T21:31:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:53.707441+0000 2024-09-15T21:31:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:53.707441+0000 2024-09-15T21:31:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:53.709 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 1 Processor -- start 2024-09-15T21:31:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 1 -- start start 2024-09-15T21:31:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient: init 2024-09-15T21:31:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2024-09-15T21:31:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6080a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb60809ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb6080a1380 0x7fb60813e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient(hunting): picked mon.noname-c con 0x7fb6080a1380 addr v2:172.21.15.119:3300/0 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6080a3130 con 0x7fb60809ffc0 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb608073df0 con 0x7fb6080a0890 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb6080a1bf0 con 0x7fb6080a1380 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb60d7eb640 10 monclient(hunting): authenticate will time out at 1837.102828s 2024-09-15T21:31:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6067fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb60809ffc0 0x7fb60813aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43164/0 (socket says 172.21.15.17:43164) 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6067fc640 1 -- 172.21.15.17:0/1517493144 learned_addr learned my addr 172.21.15.17:0/1517493144 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 1 --2- 172.21.15.17:0/1517493144 >> v2:172.21.15.119:3300/0 conn(0x7fb6080a1380 0x7fb60813e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): get_auth_request con 0x7fb6080a1380 auth_method 0 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient(hunting): handle_auth_done global_id 154348 payload 274 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 1 -- 172.21.15.17:0/1517493144 >> v1:172.21.15.119:6789/0 conn(0x7fb6080a0890 legacy=0x7fb6080a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 1 -- 172.21.15.17:0/1517493144 >> v1:172.21.15.17:6789/0 conn(0x7fb60809ffc0 legacy=0x7fb60813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient: _finish_hunting 0 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 1 monclient: found mon.noname-c 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:31:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.703+0000 7fb6077fe640 1 -- 172.21.15.17:0/1517493144 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60809ccd0 con 0x7fb6080a1380 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb6077fe640 1 --2- 172.21.15.17:0/1517493144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb60813e1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb600009dc0 tx=0x7fb60001c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483cd79683e9a5b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 1 -- 172.21.15.17:0/1517493144 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6000b2070 con 0x7fb6080a1380 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: dump: 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: _finish_auth 0 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 5 monclient: authenticate success, global_id 154348 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 1 -- 172.21.15.17:0/1517493144 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6000a57e0 con 0x7fb6080a1380 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 1 -- 172.21.15.17:0/1517493144 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6000a5bb0 con 0x7fb6080a1380 2024-09-15T21:31:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb605ffb640 10 monclient: dump: 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 10 monclient: get_monmap_and_config success 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 4 set_mon_vals no callback set 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 10 monclient: shutdown 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 -- 172.21.15.17:0/1517493144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 msgr2=0x7fb60813e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/1517493144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb60813e1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb600009dc0 tx=0x7fb60001c810 comp rx=0 tx=0).stop 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 -- 172.21.15.17:0/1517493144 shutdown_connections 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/1517493144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb60813e1f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 -- 172.21.15.17:0/1517493144 >> 172.21.15.17:0/1517493144 conn(0x7fb6080a66a0 msgr2=0x7fb6080a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 -- 172.21.15.17:0/1517493144 shutdown_connections 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 1 -- 172.21.15.17:0/1517493144 wait complete. 2024-09-15T21:31:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 10 monclient: build_initial_monmap 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.707+0000 7fb60d7eb640 10 monclient: monmap: 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 Processor -- start 2024-09-15T21:31:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- start start 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: init 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 0x7fb60811d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): picked mon.noname-c con 0x7fb60809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 0x7fb6081cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): picked mon.noname-a con 0x7fb6080a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb6081cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): picked mon.noname-b con 0x7fb6080a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081412e0 con 0x7fb6080a0890 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb608141160 con 0x7fb6080a1380 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb608141460 con 0x7fb60809ffc0 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient(hunting): authenticate will time out at 1837.108763s 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 0x7fb60811d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 0x7fb60811d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57106/0 (socket says 172.21.15.17:57106) 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 -- 172.21.15.17:0/955245251 learned_addr learned my addr 172.21.15.17:0/955245251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): get_auth_request con 0x7fb60809ffc0 auth_method 0 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 0x7fb6081cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:53.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 10 monclient(hunting): get_auth_request con 0x7fb6080a0890 auth_method 0 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 1 -- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 msgr2=0x7fb6081cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 1 -- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 msgr2=0x7fb6081cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 0x7fb6081cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6077fe640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb6081cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 0x7fb6081cd8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6077fe640 10 monclient(hunting): get_auth_request con 0x7fb6080a1380 auth_method 0 2024-09-15T21:31:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient(hunting): handle_auth_done global_id 164219 payload 824 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 -- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 msgr2=0x7fb6081cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb6081cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 -- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 msgr2=0x7fb6081cd8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 0x7fb6081cd8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 monclient: found mon.noname-c 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 -- 172.21.15.17:0/955245251 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60811efe0 con 0x7fb60809ffc0 2024-09-15T21:31:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6077fe640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb6081cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb606ffd640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 0x7fb60811d7e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c500 tx=0x7fb5f800c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8017070 con 0x7fb60809ffc0 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: dump: 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: _finish_auth 0 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80052d0 con 0x7fb60809ffc0 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8012870 con 0x7fb60809ffc0 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 10 monclient: dump: 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 5 monclient: authenticate success, global_id 164219 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _renew_subs 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60811e170 con 0x7fb60809ffc0 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6057fa640 4 set_mon_vals no callback set 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _renew_subs 2024-09-15T21:31:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60811e740 con 0x7fb60809ffc0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb5f801a070 con 0x7fb60809ffc0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6080a0c70 con 0x7fb60809ffc0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b9c0 0x7fb5f403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb5f804f8a0 con 0x7fb60809ffc0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b9c0 0x7fb5f403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.711+0000 7fb6067fc640 10 monclient: get_auth_request con 0x7fb5f403b9c0 auth_method 0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.715+0000 7fb6067fc640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b9c0 0x7fb5f403de80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb608136860 tx=0x7fb5fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.715+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb5f8003d10 con 0x7fb60809ffc0 2024-09-15T21:31:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.715+0000 7fb5ef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:31:53.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.715+0000 7fb5ef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:31:53.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb60d7eb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:31:53.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb60d7eb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:53.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb60d7eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:31:53.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb608135c80 con 0x7fb60809ffc0 2024-09-15T21:31:53.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb5ef7fe640 1 -- 172.21.15.17:0/955245251 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fb5f8019cd0 con 0x7fb60809ffc0 2024-09-15T21:31:53.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb5ef7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:31:53.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.943+0000 7fb5ef7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:31:53.949 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":132,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b9c0 msgr2=0x7fb5f403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b9c0 0x7fb5f403de80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb608136860 tx=0x7fb5fc009040 comp rx=0 tx=0).stop 2024-09-15T21:31:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 10 monclient: shutdown 2024-09-15T21:31:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:31:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 msgr2=0x7fb60811d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 0x7fb60811d7e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c500 tx=0x7fb5f800c9c0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 shutdown_connections 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb5f403b9c0 0x7fb5f403de80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6080a1380 0x7fb6081cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6080a0890 0x7fb6081cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 --2- 172.21.15.17:0/955245251 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb60809ffc0 0x7fb60811d7e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 >> 172.21.15.17:0/955245251 conn(0x7fb6080a66a0 msgr2=0x7fb6080a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 shutdown_connections 2024-09-15T21:31:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:53.947+0000 7fb60d7eb640 1 -- 172.21.15.17:0/955245251 wait complete. 2024-09-15T21:31:56.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient: get_monmap_and_config 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient: build_initial_monmap 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient: monmap: 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:31:57.111348+0000 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:31:57.111348+0000 2024-09-15T21:31:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:31:57.112 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:31:57.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 Processor -- start 2024-09-15T21:31:57.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 -- start start 2024-09-15T21:31:57.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient: init 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient: _reopen_session rank -1 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): picked mon.noname-a con 0x7f02600a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f026009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): picked mon.noname-b con 0x7f02600a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02600a3130 con 0x7f026009ffc0 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0260073df0 con 0x7f02600a0890 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f02600a1bf0 con 0x7f02600a11b0 2024-09-15T21:31:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): _renew_subs 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f0265f4b640 10 monclient(hunting): authenticate will time out at 1840.506777s 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50774/0 (socket says 172.21.15.17:50774) 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 learned_addr learned my addr 172.21.15.17:0/2281043965 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 10 monclient(hunting): get_auth_request con 0x7f02600a0890 auth_method 0 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 10 monclient(hunting): get_auth_request con 0x7f02600a11b0 auth_method 0 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:57.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:57.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.107+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:31:57.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:57.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:57.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 10 monclient(hunting): get_auth_request con 0x7f02600a0890 auth_method 0 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 10 monclient(hunting): get_auth_request con 0x7f02600a11b0 auth_method 0 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:57.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:57.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:31:57.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.311+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:31:57.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 10 monclient(hunting): get_auth_request con 0x7f02600a0890 auth_method 0 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 10 monclient(hunting): get_auth_request con 0x7f02600a11b0 auth_method 0 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:57.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:57.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:31:57.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:57.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:57.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:31:57.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:57.715+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 10 monclient(hunting): get_auth_request con 0x7f02600a11b0 auth_method 0 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 10 monclient(hunting): get_auth_request con 0x7f02600a0890 auth_method 0 2024-09-15T21:31:58.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025f7fe640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:31:58.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:31:58.519+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:32:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): tick 2024-09-15T21:32:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f0260135d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f0260135d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 msgr2=0x7f02600a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f02600a0890 0x7f02600a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 >> v1:172.21.15.119:6789/0 conn(0x7f026009ffc0 legacy=0x7f0260135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient: _add_conns ranks=[0,4] 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02400036e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): picked mon.noname-c con 0x7f0240007430 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02600a15e0 con 0x7f02400036e0 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0260073df0 con 0x7f0240007430 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025d7fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 10 monclient(hunting): get_auth_request con 0x7f0240007430 auth_method 0 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:32:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.107+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:00.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.271+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v1:172.21.15.17:6789/0 conn(0x7f02400036e0 legacy=0x7f0240005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T21:32:00.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.271+0000 7f025f7fe640 1 -- 172.21.15.17:0/2281043965 >> v1:172.21.15.17:6789/0 conn(0x7f02400036e0 legacy=0x7f0240005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:32:00.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.271+0000 7f025f7fe640 1 --1- 172.21.15.17:0/2281043965 >> v1:172.21.15.17:6789/0 conn(0x7f02400036e0 0x7f0240005b60 :-1 s=OPENED pgs=26 cs=1 l=1).handle_message read tag failed 2024-09-15T21:32:00.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.271+0000 7f025f7fe640 1 --1- 172.21.15.17:0/2281043965 >> v1:172.21.15.17:6789/0 conn(0x7f02400036e0 0x7f0240005b60 :-1 s=OPENED pgs=26 cs=1 l=1).fault on lossy channel, failing 2024-09-15T21:32:00.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.271+0000 7f025e7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.17:6789/0 2024-09-15T21:32:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 10 monclient(hunting): get_auth_request con 0x7f0240007430 auth_method 0 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:00.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.311+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:32:00.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:00.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 10 monclient(hunting): get_auth_request con 0x7f0240007430 auth_method 0 2024-09-15T21:32:00.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:00.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:00.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:00.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:00.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:00.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:00.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:00.715+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:32:01.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:01.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 10 monclient(hunting): get_auth_request con 0x7f0240007430 auth_method 0 2024-09-15T21:32:01.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:01.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:01.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:01.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:01.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:01.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:01.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:01.519+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:32:03.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): tick 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.119:3300/0 conn(0x7f0240007430 0x7f02400098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 >> v1:172.21.15.17:6789/0 conn(0x7f02400036e0 legacy=0x7f0240005b60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02400036e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025ffff640 1 -- 172.21.15.17:0/2281043965 reap_dead start 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f024000a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): picked mon.noname-b con 0x7f02600a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f024000a590 0x7f02400098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): picked mon.noname-a con 0x7f024000a590 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f024000afc0 con 0x7f02400036e0 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0260073df0 con 0x7f024000a590 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 1 -- 172.21.15.17:0/2281043965 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f02600a1bf0 con 0x7f02600a11b0 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025d7fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f024000a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025effd640 10 monclient(hunting): get_auth_request con 0x7f02600a11b0 auth_method 0 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.107+0000 7f025ffff640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f024000a590 0x7f02400098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): get_auth_request con 0x7f024000a590 auth_method 0 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 10 monclient(hunting): handle_auth_done global_id 164222 payload 274 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f024000a590 msgr2=0x7f02400098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f024000a590 0x7f02400098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 >> v1:172.21.15.17:6789/0 conn(0x7f02400036e0 legacy=0x7f0240005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 1 monclient: found mon.noname-b 2024-09-15T21:32:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.111+0000 7f025effd640 1 -- 172.21.15.17:0/2281043965 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024000abf0 con 0x7f02600a11b0 2024-09-15T21:32:03.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025effd640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f024000a050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0248007c80 tx=0x7f024801ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f8cb7cb60e3cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 1 -- 172.21.15.17:0/2281043965 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480a5a00 con 0x7f02600a11b0 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: dump: 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.121 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 1 -- 172.21.15.17:0/2281043965 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480a9070 con 0x7f02600a11b0 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 5 monclient: authenticate success, global_id 164222 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 4 set_mon_vals no callback set 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 1 -- 172.21.15.17:0/2281043965 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480a5e20 con 0x7f02600a11b0 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f025e7fc640 10 monclient: dump: 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.123 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 10 monclient: shutdown 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 1 -- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 msgr2=0x7f024000a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.115+0000 7f0265f4b640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f024000a050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0248007c80 tx=0x7f024801ccb0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- 172.21.15.17:0/2281043965 shutdown_connections 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 --2- 172.21.15.17:0/2281043965 >> v2:172.21.15.17:3300/0 conn(0x7f024000a590 0x7f02400098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 --2- 172.21.15.17:0/2281043965 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02600a11b0 0x7f024000a050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- 172.21.15.17:0/2281043965 >> 172.21.15.17:0/2281043965 conn(0x7f02600a66a0 msgr2=0x7f02600a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- 172.21.15.17:0/2281043965 shutdown_connections 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- 172.21.15.17:0/2281043965 wait complete. 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient: monmap: 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.128 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 Processor -- start 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- start start 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient: init 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02400036e0 0x7f0260133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): picked mon.noname-c con 0x7f02400036e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024000a590 0x7f0260134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): picked mon.noname-b con 0x7f024000a590 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:03.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600a11b0 0x7f0260124840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): picked mon.noname-a con 0x7f02600a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0260136210 con 0x7f02600a11b0 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0260136090 con 0x7f024000a590 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0260136390 con 0x7f02400036e0 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0265f4b640 10 monclient(hunting): authenticate will time out at 1846.518722s 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02400036e0 0x7f0260133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600a11b0 0x7f0260124840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02400036e0 0x7f0260133c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47964/0 (socket says 172.21.15.17:47964) 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 1 -- 172.21.15.17:0/3299365074 learned_addr learned my addr 172.21.15.17:0/3299365074 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:03.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): get_auth_request con 0x7f02400036e0 auth_method 0 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): get_auth_request con 0x7f02600a11b0 auth_method 0 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025effd640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024000a590 0x7f0260134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025effd640 10 monclient(hunting): get_auth_request con 0x7f024000a590 auth_method 0 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient(hunting): handle_auth_done global_id 174105 payload 824 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 -- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02400036e0 msgr2=0x7f0260133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02400036e0 0x7f0260133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 -- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024000a590 msgr2=0x7f0260134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024000a590 0x7f0260134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 monclient: found mon.noname-a 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 -- 172.21.15.17:0/3299365074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0260128660 con 0x7f02600a11b0 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025effd640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024000a590 0x7f0260134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f025ffff640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600a11b0 0x7f0260124840 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f026009d070 tx=0x7f0258017e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.119+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025800f040 con 0x7f02600a11b0 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: dump: 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025801c070 con 0x7f02600a11b0 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 5 monclient: authenticate success, global_id 174105 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 monclient: _renew_subs 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0260126300 con 0x7f02600a11b0 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258015d10 con 0x7f02600a11b0 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 10 monclient: dump: 2024-09-15T21:32:03.135 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 monclient: _renew_subs 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f025dffb640 4 set_mon_vals no callback set 2024-09-15T21:32:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02601280a0 con 0x7f02600a11b0 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0258018450 con 0x7f02600a11b0 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0260138ab0 con 0x7f02600a11b0 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024003ce60 0x7f024003f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f02580511f0 con 0x7f02600a11b0 2024-09-15T21:32:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f025f7fe640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024003ce60 0x7f024003f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f025f7fe640 10 monclient: get_auth_request con 0x7f024003ce60 auth_method 0 2024-09-15T21:32:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.123+0000 7f025f7fe640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024003ce60 0x7f024003f320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0250009440 tx=0x7f0250009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.131+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0258026060 con 0x7f02600a11b0 2024-09-15T21:32:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.131+0000 7f0257fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.131+0000 7f0257fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:03.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:32:03.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:32:03.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f02601346b0 con 0x7f02600a11b0 2024-09-15T21:32:03.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0257fff640 1 -- 172.21.15.17:0/3299365074 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f025801ed10 con 0x7f02600a11b0 2024-09-15T21:32:03.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0257fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:32:03.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0257fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:03.375 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024003ce60 msgr2=0x7f024003f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024003ce60 0x7f024003f320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0250009440 tx=0x7f0250009040 comp rx=0 tx=0).stop 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 10 monclient: shutdown 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600a11b0 msgr2=0x7f0260124840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600a11b0 0x7f0260124840 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f026009d070 tx=0x7f0258017e60 comp rx=0 tx=0).stop 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 shutdown_connections 2024-09-15T21:32:03.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f024003ce60 0x7f024003f320 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.371+0000 7f0265f4b640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600a11b0 0x7f0260124840 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.375+0000 7f0265f4b640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024000a590 0x7f0260134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.375+0000 7f0265f4b640 1 --2- 172.21.15.17:0/3299365074 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f02400036e0 0x7f0260133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.375+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 >> 172.21.15.17:0/3299365074 conn(0x7f02600a66a0 msgr2=0x7f0260134ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.375+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 shutdown_connections 2024-09-15T21:32:03.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.375+0000 7f0265f4b640 1 -- 172.21.15.17:0/3299365074 wait complete. 2024-09-15T21:32:03.394 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:32:03.395 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:32:03.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient: get_monmap_and_config 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient: monmap: 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:03.519235+0000 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:03.519235+0000 2024-09-15T21:32:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:03.520 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 Processor -- start 2024-09-15T21:32:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 -- start start 2024-09-15T21:32:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient: init 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f080a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f080a5580 0x7f2f0813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): picked mon.noname-c con 0x7f2f080a5580 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a59d0 0x7f2f0813f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): picked mon.noname-b con 0x7f2f080a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f080a2c30 con 0x7f2f080a4a90 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2f080a0f50 con 0x7f2f080a5580 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2f080a1a00 con 0x7f2f080a59d0 2024-09-15T21:32:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f0e1e7640 10 monclient(hunting): authenticate will time out at 1846.914636s 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f077fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2f080a4a90 0x7f2f080a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42370/0 (socket says 172.21.15.17:42370) 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f077fe640 1 -- 172.21.15.17:0/1054188346 learned_addr learned my addr 172.21.15.17:0/1054188346 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 1 --2- 172.21.15.17:0/1054188346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a59d0 0x7f2f0813f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): get_auth_request con 0x7f2f080a59d0 auth_method 0 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/1054188346 >> v2:172.21.15.119:3300/0 conn(0x7f2f080a5580 0x7f2f0813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f067fc640 1 -- 172.21.15.17:0/1054188346 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546203960 0 0) 0x7f2f000014a0 con 0x7f2f080a4a90 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f067fc640 10 monclient(hunting): _negotiate 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f067fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f067fc640 10 monclient(hunting): my global_id is 174108 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f067fc640 1 -- 172.21.15.17:0/1054188346 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ef0003530 con 0x7f2f080a4a90 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient(hunting): handle_auth_done global_id 164228 payload 274 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 1 -- 172.21.15.17:0/1054188346 >> v2:172.21.15.119:3300/0 conn(0x7f2f080a5580 msgr2=0x7f2f0813cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 1 --2- 172.21.15.17:0/1054188346 >> v2:172.21.15.119:3300/0 conn(0x7f2f080a5580 0x7f2f0813cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 1 -- 172.21.15.17:0/1054188346 >> v1:172.21.15.17:6789/0 conn(0x7f2f080a4a90 legacy=0x7f2f080a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 1 monclient: found mon.noname-b 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f07fff640 1 -- 172.21.15.17:0/1054188346 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f080a1c90 con 0x7f2f080a59d0 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f06ffd640 10 monclient: get_auth_request con 0x7f2f080a5580 auth_method 0 2024-09-15T21:32:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.515+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/1054188346 >> v2:172.21.15.119:3300/0 conn(0x7f2f080a5580 0x7f2f0813cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f07fff640 1 --2- 172.21.15.17:0/1054188346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a59d0 0x7f2f0813f260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80097f0 tx=0x7f2ef801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0deb12ab50b4c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 1 -- 172.21.15.17:0/1054188346 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b0070 con 0x7f2f080a59d0 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: dump: 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 5 monclient: authenticate success, global_id 164228 2024-09-15T21:32:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 1 -- 172.21.15.17:0/1054188346 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef80ab600 con 0x7f2f080a59d0 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 4 set_mon_vals no callback set 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 1 -- 172.21.15.17:0/1054188346 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80ab9b0 con 0x7f2f080a59d0 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f067fc640 10 monclient: dump: 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: shutdown 2024-09-15T21:32:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/1054188346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a59d0 msgr2=0x7f2f0813f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/1054188346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a59d0 0x7f2f0813f260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80097f0 tx=0x7f2ef801c330 comp rx=0 tx=0).stop 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/1054188346 shutdown_connections 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/1054188346 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a59d0 0x7f2f0813f260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/1054188346 >> v2:172.21.15.119:3300/0 conn(0x7f2f080a5580 0x7f2f0813cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/1054188346 >> 172.21.15.17:0/1054188346 conn(0x7f2f080aabb0 msgr2=0x7f2f080a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/1054188346 shutdown_connections 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/1054188346 wait complete. 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: monmap: 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 Processor -- start 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- start start 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: init 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a4a90 0x7f2f08138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): picked mon.noname-c con 0x7f2f080a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f080a5580 0x7f2f08139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): picked mon.noname-b con 0x7f2f080a5580 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f080a59d0 0x7f2f081cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): picked mon.noname-a con 0x7f2f080a59d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f081421d0 con 0x7f2f080a59d0 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0806f2a0 con 0x7f2f080a5580 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08142350 con 0x7f2f080a4a90 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f0e1e7640 10 monclient(hunting): authenticate will time out at 1846.919638s 2024-09-15T21:32:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a4a90 0x7f2f08138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a4a90 0x7f2f08138f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47982/0 (socket says 172.21.15.17:47982) 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 1 -- 172.21.15.17:0/2218632512 learned_addr learned my addr 172.21.15.17:0/2218632512 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): get_auth_request con 0x7f2f080a4a90 auth_method 0 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f07fff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f080a59d0 0x7f2f081cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f07fff640 10 monclient(hunting): get_auth_request con 0x7f2f080a59d0 auth_method 0 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.519+0000 7f2f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f080a5580 0x7f2f08139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 10 monclient(hunting): handle_auth_done global_id 164231 payload 824 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f080a5580 msgr2=0x7f2f08139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f080a5580 0x7f2f08139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f080a59d0 msgr2=0x7f2f081cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f080a59d0 0x7f2f081cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 monclient: found mon.noname-c 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f080a1280 con 0x7f2f080a4a90 2024-09-15T21:32:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient: get_auth_request con 0x7f2f080a5580 auth_method 0 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f07fff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f080a59d0 0x7f2f081cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f080a5580 0x7f2f08139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f077fe640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a4a90 0x7f2f08138f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cbc0 tx=0x7f2f0000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000ad40 con 0x7f2f080a4a90 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: dump: 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 5 monclient: authenticate success, global_id 164231 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 10 monclient: _renew_subs 2024-09-15T21:32:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f00012400 con 0x7f2f080a4a90 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0813f480 con 0x7f2f080a4a90 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f000127f0 con 0x7f2f080a4a90 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: dump: 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 10 monclient: _renew_subs 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0813c070 con 0x7f2f080a4a90 2024-09-15T21:32:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f05ffb640 4 set_mon_vals no callback set 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 0x7f2f081cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08076dd0 con 0x7f2f081cfa70 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 0x7f2f081cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2f00012990 con 0x7f2f080a4a90 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient: get_auth_request con 0x7f2f081cfa70 auth_method 0 2024-09-15T21:32:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ef403b9c0 0x7f2ef403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_done global_id 174114 payload 824 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f080a4e70 con 0x7f2f081cfa70 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f2f00050d30 con 0x7f2f080a4a90 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 0x7f2f081cfe50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f0813eac0 tx=0x7f2efc00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc016070 con 0x7f2f081cfa70 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: dump: 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.548 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f07fff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ef403b9c0 0x7f2ef403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f07fff640 10 monclient: get_auth_request con 0x7f2ef403b9c0 auth_method 0 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2efc008900 con 0x7f2f081cfa70 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 msgr2=0x7f2f081cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 0x7f2f081cfe50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f0813eac0 tx=0x7f2efc00dbf0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 msgr2=0x7f2f081cfe50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2eeffff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f081cfa70 0x7f2f081cfe50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.523+0000 7f2f07fff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ef403b9c0 0x7f2ef403de80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ad000 tx=0x7f2ef80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 0x7f2f0809c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f081d0390 con 0x7f2f0809c480 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 0x7f2f0809c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient: get_auth_request con 0x7f2f0809c480 auth_method 0 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 10 monclient(hunting): handle_auth_done global_id 174117 payload 824 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 1 -- 172.21.15.17:0/2218632512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2f08069200 con 0x7f2f0809c480 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f06ffd640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 0x7f2f0809c8b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f08068d50 tx=0x7f2efc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc01bad0 con 0x7f2f0809c480 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 10 monclient: dump: 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4547 (secure 0 0 0) 0x7f2efc01bad0 con 0x7f2f0809c480 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 msgr2=0x7f2f0809c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 0x7f2f0809c8b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f08068d50 tx=0x7f2efc004740 comp rx=0 tx=0).stop 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 msgr2=0x7f2f0809c8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2eeffff640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f0809c480 0x7f2f0809c8b0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f08068d50 tx=0x7f2efc004740 comp rx=0 tx=0).stop 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 140, 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 13267, 2024-09-15T21:32:03.554 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.556 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.557 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:03.558 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.559 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:03.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.563 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.566 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:03.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.567 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.567 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:03.567 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ef403b9c0 msgr2=0x7f2ef403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2ef403b9c0 0x7f2ef403de80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ad000 tx=0x7f2ef80a6000 comp rx=0 tx=0).stop 2024-09-15T21:32:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 10 monclient: shutdown 2024-09-15T21:32:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a4a90 msgr2=0x7f2f08138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 --2- 172.21.15.17:0/2218632512 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2f080a4a90 0x7f2f08138f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cbc0 tx=0x7f2f0000a430 comp rx=0 tx=0).stop 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f07fff640 1 -- 172.21.15.17:0/2218632512 reap_dead start 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f07fff640 1 -- 172.21.15.17:0/2218632512 reap_dead start 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 shutdown_connections 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 >> 172.21.15.17:0/2218632512 conn(0x7f2f080aabb0 msgr2=0x7f2f08137ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 shutdown_connections 2024-09-15T21:32:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.531+0000 7f2f0e1e7640 1 -- 172.21.15.17:0/2218632512 wait complete. 2024-09-15T21:32:03.569 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:32:03.569 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:32:03.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient: get_monmap_and_config 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient: monmap: 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:03.687550+0000 2024-09-15T21:32:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:03.687550+0000 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:03.688 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 1 Processor -- start 2024-09-15T21:32:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 1 -- start start 2024-09-15T21:32:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient: init 2024-09-15T21:32:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc2781378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc2781383b0 0x7fc27813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient(hunting): picked mon.noname-a con 0x7fc2781383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc27813d2c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780a1a20 con 0x7fc2781378c0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780a1d00 con 0x7fc27813d2c0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc2780a1fe0 con 0x7fc2781383b0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc27e730640 10 monclient(hunting): authenticate will time out at 1847.082923s 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc2781383b0 0x7fc27813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc2781383b0 0x7fc27813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46480/0 (socket says 172.21.15.17:46480) 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 -- 172.21.15.17:0/325596033 learned_addr learned my addr 172.21.15.17:0/325596033 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): get_auth_request con 0x7fc2781383b0 auth_method 0 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc277fff640 1 --1- 172.21.15.17:0/325596033 >> v1:172.21.15.17:6789/0 conn(0x7fc2781378c0 0x7fc278137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42386/0 (socket says 172.21.15.17:42386) 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient(hunting): handle_auth_done global_id 174120 payload 274 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 -- 172.21.15.17:0/325596033 >> v1:172.21.15.119:6789/0 conn(0x7fc27813d2c0 legacy=0x7fc27813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 -- 172.21.15.17:0/325596033 >> v1:172.21.15.17:6789/0 conn(0x7fc2781378c0 legacy=0x7fc278137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 monclient: found mon.noname-a 2024-09-15T21:32:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.683+0000 7fc2777fe640 1 -- 172.21.15.17:0/325596033 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2780a22c0 con 0x7fc2781383b0 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc2777fe640 1 --2- 172.21.15.17:0/325596033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc27813cd80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098a0 tx=0x7fc26c01c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22cc46bc9767d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 1 -- 172.21.15.17:0/325596033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0b1070 con 0x7fc2781383b0 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: dump: 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 5 monclient: authenticate success, global_id 174120 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 1 -- 172.21.15.17:0/325596033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c0a4890 con 0x7fc2781383b0 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 4 set_mon_vals no callback set 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 1 -- 172.21.15.17:0/325596033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0a4c60 con 0x7fc2781383b0 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc276ffd640 10 monclient: dump: 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 10 monclient: shutdown 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 -- 172.21.15.17:0/325596033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 msgr2=0x7fc27813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 --2- 172.21.15.17:0/325596033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc27813cd80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098a0 tx=0x7fc26c01c0e0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 -- 172.21.15.17:0/325596033 shutdown_connections 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 --2- 172.21.15.17:0/325596033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc27813cd80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 -- 172.21.15.17:0/325596033 >> 172.21.15.17:0/325596033 conn(0x7fc2780aac40 msgr2=0x7fc2780a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 -- 172.21.15.17:0/325596033 shutdown_connections 2024-09-15T21:32:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.687+0000 7fc27e730640 1 -- 172.21.15.17:0/325596033 wait complete. 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: monmap: 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 Processor -- start 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- start start 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: init 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2781378c0 0x7fc2781d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): picked mon.noname-c con 0x7fc2781378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc278136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): picked mon.noname-a con 0x7fc2781383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc27813d2c0 0x7fc278137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): picked mon.noname-b con 0x7fc27813d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc278144230 con 0x7fc2781383b0 2024-09-15T21:32:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2781440b0 con 0x7fc27813d2c0 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc2781443b0 con 0x7fc2781378c0 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): authenticate will time out at 1847.089468s 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2781378c0 0x7fc2781d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc278136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2781378c0 0x7fc2781d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47996/0 (socket says 172.21.15.17:47996) 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 1 -- 172.21.15.17:0/1283165467 learned_addr learned my addr 172.21.15.17:0/1283165467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): get_auth_request con 0x7fc2781378c0 auth_method 0 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): get_auth_request con 0x7fc2781383b0 auth_method 0 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient(hunting): handle_auth_done global_id 174126 payload 824 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27cccd640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc27813d2c0 0x7fc278137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2781378c0 msgr2=0x7fc2781d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2781378c0 0x7fc2781d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc27813d2c0 msgr2=0x7fc278137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc27813d2c0 0x7fc278137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 monclient: found mon.noname-a 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27809f6c0 con 0x7fc2781383b0 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2777fe640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc278136d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0ab000 tx=0x7fc26c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0b9070 con 0x7fc2781383b0 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: dump: 2024-09-15T21:32:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.708 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c0ac870 con 0x7fc2781383b0 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0b1040 con 0x7fc2781383b0 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 5 monclient: authenticate success, global_id 174126 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 10 monclient: dump: 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: _renew_subs 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc2767fc640 4 set_mon_vals no callback set 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27809e720 con 0x7fc2781383b0 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: _renew_subs 2024-09-15T21:32:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27809f280 con 0x7fc2781383b0 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 0x7fc27809d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc2780a12f0 con 0x7fc27809cd00 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc26c0bc070 con 0x7fc2781383b0 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 0x7fc27809d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient: get_auth_request con 0x7fc27809cd00 auth_method 0 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc26403b9c0 0x7fc26403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 10 monclient(hunting): handle_auth_done global_id 164237 payload 824 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2781d0ea0 con 0x7fc27809cd00 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc26c0e9e80 con 0x7fc2781383b0 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc277fff640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 0x7fc27809d0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2680015a0 tx=0x7fc268001a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27cccd640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc26403b9c0 0x7fc26403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.691+0000 7fc27cccd640 10 monclient: get_auth_request con 0x7fc26403b9c0 auth_method 0 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26800f780 con 0x7fc27809cd00 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 10 monclient: dump: 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc268018070 con 0x7fc27809cd00 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:03.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 msgr2=0x7fc27809d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 0x7fc27809d0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2680015a0 tx=0x7fc268001a60 comp rx=0 tx=0).stop 2024-09-15T21:32:03.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 msgr2=0x7fc27809d0e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc274ff9640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809cd00 0x7fc27809d0e0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2680015a0 tx=0x7fc268001a60 comp rx=0 tx=0).stop 2024-09-15T21:32:03.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.695+0000 7fc27cccd640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc26403b9c0 0x7fc26403de80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc2780a6b30 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 0x7fc278138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc278137ca0 con 0x7fc27809d620 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 0x7fc278138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient: get_auth_request con 0x7fc27809d620 auth_method 0 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 10 monclient(hunting): handle_auth_done global_id 164240 payload 824 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 1 -- 172.21.15.17:0/1283165467 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc27813ce40 con 0x7fc27809d620 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc277fff640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 0x7fc278138d60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2680015a0 tx=0x7fc268000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26800fd20 con 0x7fc27809d620 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 10 monclient: dump: 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.717 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7fc26800fd20 con 0x7fc27809d620 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 msgr2=0x7fc278138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 0x7fc278138d60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2680015a0 tx=0x7fc268000ee0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 msgr2=0x7fc278138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc274ff9640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc27809d620 0x7fc278138d60 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2680015a0 tx=0x7fc268000ee0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 140, 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 72283, 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:32:03.719 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.720 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.721 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:03.722 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.723 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:03.726 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:03.729 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.730 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.731 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:03.732 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc26403b9c0 msgr2=0x7fc26403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc26403b9c0 0x7fc26403de80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc2780a6b30 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 10 monclient: shutdown 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 msgr2=0x7fc278136d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 --2- 172.21.15.17:0/1283165467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2781383b0 0x7fc278136d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0ab000 tx=0x7fc26c0191a0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27cccd640 1 -- 172.21.15.17:0/1283165467 reap_dead start 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27cccd640 1 -- 172.21.15.17:0/1283165467 reap_dead start 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 shutdown_connections 2024-09-15T21:32:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.699+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 >> 172.21.15.17:0/1283165467 conn(0x7fc2780aac40 msgr2=0x7fc2780a3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.703+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 shutdown_connections 2024-09-15T21:32:03.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.703+0000 7fc27e730640 1 -- 172.21.15.17:0/1283165467 wait complete. 2024-09-15T21:32:03.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:32:03.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:32:03.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:32:03.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.839+0000 7f8946252640 10 monclient: get_monmap_and_config 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.839+0000 7f8946252640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.839+0000 7f8946252640 10 monclient: monmap: 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:03.845927+0000 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:03.845927+0000 2024-09-15T21:32:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:03.847 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 Processor -- start 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 -- start start 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient: init 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f89400a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): picked mon.noname-a con 0x7f89400a5580 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): picked mon.noname-b con 0x7f89400a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89400a2c30 con 0x7f89400a4a90 2024-09-15T21:32:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f89400a0f50 con 0x7f89400a5580 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f89400a1a00 con 0x7f89400a59d0 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient(hunting): authenticate will time out at 1847.240947s 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48020/0 (socket says 172.21.15.17:48020) 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46496/0 (socket says 172.21.15.17:46496) 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 1 -- 172.21.15.17:0/1858389270 learned_addr learned my addr 172.21.15.17:0/1858389270 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): get_auth_request con 0x7f89400a59d0 auth_method 0 2024-09-15T21:32:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): get_auth_request con 0x7f89400a5580 auth_method 0 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient(hunting): handle_auth_done global_id 174129 payload 274 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 -- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 msgr2=0x7f894013f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 --2- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 -- 172.21.15.17:0/1858389270 >> v1:172.21.15.119:6789/0 conn(0x7f89400a4a90 legacy=0x7f89400a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 monclient: found mon.noname-a 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 -- 172.21.15.17:0/1858389270 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89400a1c90 con 0x7f89400a5580 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893ffff640 1 --2- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893effd640 1 --2- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013cd10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89300097f0 tx=0x7f893001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e8ecb6f09bec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 1 -- 172.21.15.17:0/1858389270 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89300b0070 con 0x7f89400a5580 2024-09-15T21:32:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: dump: 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.861 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 1 -- 172.21.15.17:0/1858389270 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89300ab600 con 0x7f89400a5580 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 1 -- 172.21.15.17:0/1858389270 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89300ab9d0 con 0x7f89400a5580 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 5 monclient: authenticate success, global_id 174129 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f8946252640 4 set_mon_vals no callback set 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.843+0000 7f893e7fc640 10 monclient: dump: 2024-09-15T21:32:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: shutdown 2024-09-15T21:32:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 msgr2=0x7f894013cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013cd10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89300097f0 tx=0x7f893001c330 comp rx=0 tx=0).stop 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/1858389270 shutdown_connections 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- 172.21.15.17:0/1858389270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013cd10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/1858389270 >> 172.21.15.17:0/1858389270 conn(0x7f89400aabb0 msgr2=0x7f89400a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/1858389270 shutdown_connections 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/1858389270 wait complete. 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: build_initial_monmap 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: monmap: 2024-09-15T21:32:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 Processor -- start 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- start start 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: init 2024-09-15T21:32:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a4a90 0x7f89401cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): picked mon.noname-b con 0x7f89400a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): picked mon.noname-a con 0x7f89400a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): picked mon.noname-c con 0x7f89400a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8940142140 con 0x7f89400a5580 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8940141fc0 con 0x7f89400a4a90 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f89401422c0 con 0x7f89400a59d0 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): authenticate will time out at 1847.245989s 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48028/0 (socket says 172.21.15.17:48028) 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 -- 172.21.15.17:0/3839835555 learned_addr learned my addr 172.21.15.17:0/3839835555 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46498/0 (socket says 172.21.15.17:46498) 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): get_auth_request con 0x7f89400a59d0 auth_method 0 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): get_auth_request con 0x7f89400a5580 auth_method 0 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a4a90 0x7f89401cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): get_auth_request con 0x7f89400a4a90 auth_method 0 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient(hunting): handle_auth_done global_id 164246 payload 824 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a4a90 msgr2=0x7f89401cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89400a4a90 0x7f89401cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 msgr2=0x7f894013dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 monclient: found mon.noname-c 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89400a1280 con 0x7f89400a59d0 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893effd640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400a5580 0x7f894013dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893ffff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013e520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f893800efc0 tx=0x7f893800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893800cbe0 con 0x7f89400a59d0 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.870 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: _finish_auth 0 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: _check_auth_tickets 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893800cd80 con 0x7f89400a59d0 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 5 monclient: authenticate success, global_id 164246 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: _renew_subs 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894013ea60 con 0x7f89400a59d0 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938003bc0 con 0x7f89400a59d0 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:32:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: _renew_subs 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f894013a1f0 con 0x7f89400a59d0 2024-09-15T21:32:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893dffb640 4 set_mon_vals no callback set 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 0x7f894009eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89400a4e70 con 0x7f894009e7c0 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 0x7f894009eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f893801b030 con 0x7f89400a59d0 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient: get_auth_request con 0x7f894009e7c0 auth_method 0 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.847+0000 7f893f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f8938050b10 con 0x7f89400a59d0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893effd640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893f7fe640 10 monclient(hunting): handle_auth_done global_id 154366 payload 824 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893f7fe640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89400008e0 con 0x7f894009e7c0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893effd640 10 monclient: get_auth_request con 0x7f892c03b9c0 auth_method 0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893f7fe640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 0x7f894009eba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f893400e500 tx=0x7f893400e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340052e0 con 0x7f894009e7c0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f893effd640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89300ad000 tx=0x7f8930008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8934018070 con 0x7f894009e7c0 2024-09-15T21:32:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 msgr2=0x7f894009eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 0x7f894009eba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f893400e500 tx=0x7f893400e9c0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 msgr2=0x7f894009eba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.851+0000 7f8927fff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009e7c0 0x7f894009eba0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f893400e500 tx=0x7f893400e9c0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.855+0000 7f8946252640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.855+0000 7f8946252640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 0x7f8940136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.855+0000 7f8946252640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.855+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f894009f4f0 con 0x7f894009f0e0 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 0x7f8940136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient: get_auth_request con 0x7f894009f0e0 auth_method 0 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 10 monclient(hunting): handle_auth_done global_id 154369 payload 824 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 1 -- 172.21.15.17:0/3839835555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f894013a9f0 con 0x7f894009f0e0 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893f7fe640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 0x7f8940136ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f893400e500 tx=0x7f8934000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934007480 con 0x7f894009f0e0 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 10 monclient: dump: 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7f8934005480 con 0x7f894009f0e0 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 msgr2=0x7f8940136ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 0x7f8940136ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f893400e500 tx=0x7f8934000ee0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 msgr2=0x7f8940136ce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8927fff640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f894009f0e0 0x7f8940136ce0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f893400e500 tx=0x7f8934000ee0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 140, 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 128397, 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:03.881 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.882 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.883 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:03.884 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:03.885 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:03.886 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.889 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:03.890 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:03.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:03.892 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:03.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:03.893 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:03.893 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 msgr2=0x7f892c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f892c03b9c0 0x7f892c03de80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89300ad000 tx=0x7f8930008040 comp rx=0 tx=0).stop 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 10 monclient: shutdown 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 msgr2=0x7f894013e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f893ffff640 1 -- 172.21.15.17:0/3839835555 reap_dead start 2024-09-15T21:32:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013e520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f893800efc0 tx=0x7f893800c490 comp rx=0 tx=0).stop 2024-09-15T21:32:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 shutdown_connections 2024-09-15T21:32:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 --2- 172.21.15.17:0/3839835555 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f89400a59d0 0x7f894013e520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 >> 172.21.15.17:0/3839835555 conn(0x7f89400aabb0 msgr2=0x7f8940134990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 shutdown_connections 2024-09-15T21:32:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.859+0000 7f8946252640 1 -- 172.21.15.17:0/3839835555 wait complete. 2024-09-15T21:32:03.895 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:32:03.895 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:32:03.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.999+0000 7fa43000a640 10 monclient: get_monmap_and_config 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.999+0000 7fa43000a640 10 monclient: build_initial_monmap 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.999+0000 7fa43000a640 10 monclient: monmap: 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:04.005385+0000 2024-09-15T21:32:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:04.005385+0000 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:04.006 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:04.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:03.999+0000 7fa43000a640 1 Processor -- start 2024-09-15T21:32:04.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 1 -- start start 2024-09-15T21:32:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient: init 2024-09-15T21:32:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:32:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa42809d400 0x7fa42809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): picked mon.noname-a con 0x7fa42809d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa42809dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa428134d70 0x7fa42813cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): picked mon.noname-c con 0x7fa428134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4280a3130 con 0x7fa42809dd20 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa4280a1bc0 con 0x7fa42809d400 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa4280a3490 con 0x7fa428134d70 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa43000a640 10 monclient(hunting): authenticate will time out at 1847.400829s 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa42809d400 0x7fa42809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa42809d400 0x7fa42809d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46514/0 (socket says 172.21.15.17:46514) 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 -- 172.21.15.17:0/3904322626 learned_addr learned my addr 172.21.15.17:0/3904322626 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): get_auth_request con 0x7fa42809d400 auth_method 0 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42e5a7640 1 --2- 172.21.15.17:0/3904322626 >> v2:172.21.15.119:3300/0 conn(0x7fa428134d70 0x7fa42813cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42e5a7640 10 monclient(hunting): get_auth_request con 0x7fa428134d70 auth_method 0 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42e5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42e5a7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42e5a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient(hunting): handle_auth_done global_id 174135 payload 274 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 -- 172.21.15.17:0/3904322626 >> v2:172.21.15.119:3300/0 conn(0x7fa428134d70 msgr2=0x7fa42813cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 --2- 172.21.15.17:0/3904322626 >> v2:172.21.15.119:3300/0 conn(0x7fa428134d70 0x7fa42813cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 -- 172.21.15.17:0/3904322626 >> v1:172.21.15.119:6789/0 conn(0x7fa42809dd20 legacy=0x7fa428134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient: _finish_hunting 0 2024-09-15T21:32:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 monclient: found mon.noname-a 2024-09-15T21:32:04.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:04.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:04.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.003+0000 7fa42dda6640 1 -- 172.21.15.17:0/3904322626 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42809e7a0 con 0x7fa42809d400 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42dda6640 1 --2- 172.21.15.17:0/3904322626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa42809d400 0x7fa42809d7e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa42809f4f0 tx=0x7fa42401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33a5d05ac7b791f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 1 -- 172.21.15.17:0/3904322626 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b0070 con 0x7fa42809d400 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: dump: 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.018 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: _finish_auth 0 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: _check_auth_tickets 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 5 monclient: authenticate success, global_id 174135 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 1 -- 172.21.15.17:0/3904322626 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4240ab600 con 0x7fa42809d400 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 1 -- 172.21.15.17:0/3904322626 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240ab9d0 con 0x7fa42809d400 2024-09-15T21:32:04.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa42cda4640 10 monclient: dump: 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:04.020 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 4 set_mon_vals no callback set 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 10 monclient: shutdown 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 -- 172.21.15.17:0/3904322626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa42809d400 msgr2=0x7fa42809d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 --2- 172.21.15.17:0/3904322626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa42809d400 0x7fa42809d7e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa42809f4f0 tx=0x7fa42401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 -- 172.21.15.17:0/3904322626 shutdown_connections 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 --2- 172.21.15.17:0/3904322626 >> v2:172.21.15.119:3300/0 conn(0x7fa428134d70 0x7fa42813cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 --2- 172.21.15.17:0/3904322626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa42809d400 0x7fa42809d7e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 -- 172.21.15.17:0/3904322626 >> 172.21.15.17:0/3904322626 conn(0x7fa4280a66a0 msgr2=0x7fa4280a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:04.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 -- 172.21.15.17:0/3904322626 shutdown_connections 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 1 -- 172.21.15.17:0/3904322626 wait complete. 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 10 monclient: build_initial_monmap 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.007+0000 7fa43000a640 10 monclient: monmap: 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:04.022 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 Processor -- start 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- start start 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: init 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa42809d400 0x7fa42812c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): picked mon.noname-c con 0x7fa42809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42809dd20 0x7fa42812cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): picked mon.noname-b con 0x7fa42809dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 0x7fa4281318e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): picked mon.noname-a con 0x7fa428134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa42809fba0 con 0x7fa428134d70 2024-09-15T21:32:04.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa42809fa20 con 0x7fa42809dd20 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa42813f830 con 0x7fa42809d400 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient(hunting): authenticate will time out at 1847.408625s 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 0x7fa4281318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa42809d400 0x7fa42812c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 0x7fa4281318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46528/0 (socket says 172.21.15.17:46528) 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa42809d400 0x7fa42812c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48030/0 (socket says 172.21.15.17:48030) 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 -- 172.21.15.17:0/3232547542 learned_addr learned my addr 172.21.15.17:0/3232547542 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): get_auth_request con 0x7fa428134d70 auth_method 0 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): get_auth_request con 0x7fa42809d400 auth_method 0 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient(hunting): handle_auth_done global_id 174138 payload 824 2024-09-15T21:32:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42d5a5640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42809dd20 0x7fa42812cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 -- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa42809d400 msgr2=0x7fa42812c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa42809d400 0x7fa42812c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 -- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42809dd20 msgr2=0x7fa42812cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42809dd20 0x7fa42812cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient: _finish_hunting 0 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 monclient: found mon.noname-a 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 -- 172.21.15.17:0/3232547542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa428136110 con 0x7fa428134d70 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42e5a7640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 0x7fa4281318e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa420007e40 tx=0x7fa42000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420014620 con 0x7fa428134d70 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: dump: 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:32:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa420019070 con 0x7fa428134d70 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420014a20 con 0x7fa428134d70 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 10 monclient: dump: 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41ffff640 4 set_mon_vals no callback set 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 5 monclient: authenticate success, global_id 174138 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _renew_subs 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281363f0 con 0x7fa428134d70 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _renew_subs 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa428133360 con 0x7fa428134d70 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa428133f80 con 0x7fa428134d70 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa420014bc0 con 0x7fa428134d70 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa40c03b9c0 0x7fa40c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa420050ed0 con 0x7fa428134d70 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa40c03b9c0 0x7fa40c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.011+0000 7fa42dda6640 10 monclient: get_auth_request con 0x7fa40c03b9c0 auth_method 0 2024-09-15T21:32:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.015+0000 7fa42dda6640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa40c03b9c0 0x7fa40c03de80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa4240ad000 tx=0x7fa424008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:04.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.015+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa42004edd0 con 0x7fa428134d70 2024-09-15T21:32:04.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.015+0000 7fa41e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:04.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.015+0000 7fa41e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.251+0000 7fa43000a640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:32:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.251+0000 7fa43000a640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:32:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.251+0000 7fa43000a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.251+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa42806d020 con 0x7fa428134d70 2024-09-15T21:32:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.255+0000 7fa41e7fc640 1 -- 172.21.15.17:0/3232547542 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa42001c360 con 0x7fa428134d70 2024-09-15T21:32:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.255+0000 7fa41e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:32:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.255+0000 7fa41e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:04.260 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.255+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.260 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.255+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=12,logm=45} crc {auth=2076488302,config=3541143443,health=2515098687,kv=3079713637,logm=454156840}) 2024-09-15T21:32:04.260 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.255+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=32,config=2,health=12,kv=11,logm=43} crc {auth=2475230223,config=3541143443,health=1353537582,kv=2290493906,logm=1053368012}) 2024-09-15T21:32:04.260 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.255+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.260 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.255+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=12,logm=45} crc {auth=2076488302,config=3541143443,health=2515098687,kv=3079713637,logm=454156840}) 2024-09-15T21:32:04.261 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.255+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=32,config=1,health=10,kv=12,logm=45} crc {auth=3414449613,config=555582061,health=1980244553,kv=3079713637,logm=979691889}) 2024-09-15T21:32:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa40c03b9c0 msgr2=0x7fa40c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa40c03b9c0 0x7fa40c03de80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa4240ad000 tx=0x7fa424008040 comp rx=0 tx=0).stop 2024-09-15T21:32:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 10 monclient: shutdown 2024-09-15T21:32:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 msgr2=0x7fa4281318e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 0x7fa4281318e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa420007e40 tx=0x7fa42000e4b0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 shutdown_connections 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa40c03b9c0 0x7fa40c03de80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa428134d70 0x7fa4281318e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42809dd20 0x7fa42812cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 --2- 172.21.15.17:0/3232547542 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa42809d400 0x7fa42812c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 >> 172.21.15.17:0/3232547542 conn(0x7fa4280a66a0 msgr2=0x7fa4280a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 shutdown_connections 2024-09-15T21:32:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:04.259+0000 7fa43000a640 1 -- 172.21.15.17:0/3232547542 wait complete. 2024-09-15T21:32:04.264 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.259+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.264 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.259+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2989973423}) 2024-09-15T21:32:04.264 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.259+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4210417554}) 2024-09-15T21:32:04.264 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.259+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.264 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.259+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2989973423}) 2024-09-15T21:32:04.264 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.259+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1404943889}) 2024-09-15T21:32:04.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917313567}) 2024-09-15T21:32:04.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1053712745}) 2024-09-15T21:32:04.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.266 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917313567}) 2024-09-15T21:32:04.267 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3543043801}) 2024-09-15T21:32:04.268 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197738644}) 2024-09-15T21:32:04.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=886087573}) 2024-09-15T21:32:04.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197738644}) 2024-09-15T21:32:04.269 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.263+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3532375996}) 2024-09-15T21:32:04.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=922067875}) 2024-09-15T21:32:04.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4249329335}) 2024-09-15T21:32:04.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=922067875}) 2024-09-15T21:32:04.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836130485}) 2024-09-15T21:32:04.273 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1873510806}) 2024-09-15T21:32:04.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1040055465}) 2024-09-15T21:32:04.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.267+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1873510806}) 2024-09-15T21:32:04.274 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1736721373}) 2024-09-15T21:32:04.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132595286}) 2024-09-15T21:32:04.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1892316878}) 2024-09-15T21:32:04.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132595286}) 2024-09-15T21:32:04.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.271+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540854501}) 2024-09-15T21:32:04.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1032226470}) 2024-09-15T21:32:04.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1374192745}) 2024-09-15T21:32:04.278 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.279 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1032226470}) 2024-09-15T21:32:04.279 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=733601634}) 2024-09-15T21:32:04.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=881109803}) 2024-09-15T21:32:04.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=650938593}) 2024-09-15T21:32:04.281 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.281 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=881109803}) 2024-09-15T21:32:04.281 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.275+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=144931848}) 2024-09-15T21:32:04.281 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:32:04.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1280191979}) 2024-09-15T21:32:04.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134476021}) 2024-09-15T21:32:04.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1280191979}) 2024-09-15T21:32:04.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2328660030}) 2024-09-15T21:32:04.284 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.284 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3289846237}) 2024-09-15T21:32:04.284 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2704427524}) 2024-09-15T21:32:04.284 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3289846237}) 2024-09-15T21:32:04.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.279+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322123145}) 2024-09-15T21:32:04.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=652504467}) 2024-09-15T21:32:04.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4141396437}) 2024-09-15T21:32:04.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.286 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=652504467}) 2024-09-15T21:32:04.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3560985064}) 2024-09-15T21:32:04.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1957045309}) 2024-09-15T21:32:04.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=832394813}) 2024-09-15T21:32:04.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1957045309}) 2024-09-15T21:32:04.288 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.283+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1909327395}) 2024-09-15T21:32:04.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011703971}) 2024-09-15T21:32:04.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3368128301}) 2024-09-15T21:32:04.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011703971}) 2024-09-15T21:32:04.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=202638868}) 2024-09-15T21:32:04.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=929939310}) 2024-09-15T21:32:04.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3216887370}) 2024-09-15T21:32:04.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=929939310}) 2024-09-15T21:32:04.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.287+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2863461829}) 2024-09-15T21:32:04.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695076910}) 2024-09-15T21:32:04.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3371121579}) 2024-09-15T21:32:04.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695076910}) 2024-09-15T21:32:04.294 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1397665434}) 2024-09-15T21:32:04.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.296 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3094317094}) 2024-09-15T21:32:04.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2892658960}) 2024-09-15T21:32:04.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3094317094}) 2024-09-15T21:32:04.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.291+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3471701359}) 2024-09-15T21:32:04.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3490668062}) 2024-09-15T21:32:04.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4275853601}) 2024-09-15T21:32:04.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3490668062}) 2024-09-15T21:32:04.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2986449542}) 2024-09-15T21:32:04.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2612742175}) 2024-09-15T21:32:04.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2385878906}) 2024-09-15T21:32:04.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2612742175}) 2024-09-15T21:32:04.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.295+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217465781}) 2024-09-15T21:32:04.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3937232059}) 2024-09-15T21:32:04.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398542288}) 2024-09-15T21:32:04.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3937232059}) 2024-09-15T21:32:04.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4272607876}) 2024-09-15T21:32:04.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986294324}) 2024-09-15T21:32:04.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3322985309}) 2024-09-15T21:32:04.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986294324}) 2024-09-15T21:32:04.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.299+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2144026979}) 2024-09-15T21:32:04.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=817717532}) 2024-09-15T21:32:04.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=325544429}) 2024-09-15T21:32:04.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=817717532}) 2024-09-15T21:32:04.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931960995}) 2024-09-15T21:32:04.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224622014}) 2024-09-15T21:32:04.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=893205563}) 2024-09-15T21:32:04.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224622014}) 2024-09-15T21:32:04.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.303+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4018190560}) 2024-09-15T21:32:04.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3876799408}) 2024-09-15T21:32:04.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2875625439}) 2024-09-15T21:32:04.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.312 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3876799408}) 2024-09-15T21:32:04.312 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454178322}) 2024-09-15T21:32:04.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=414895021}) 2024-09-15T21:32:04.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416299935}) 2024-09-15T21:32:04.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=414895021}) 2024-09-15T21:32:04.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.307+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=215709873}) 2024-09-15T21:32:04.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1574203881}) 2024-09-15T21:32:04.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=884663499}) 2024-09-15T21:32:04.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1574203881}) 2024-09-15T21:32:04.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2029894024}) 2024-09-15T21:32:04.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543911293}) 2024-09-15T21:32:04.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1251584372}) 2024-09-15T21:32:04.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.318 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543911293}) 2024-09-15T21:32:04.318 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.311+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1627879193}) 2024-09-15T21:32:04.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3318329723}) 2024-09-15T21:32:04.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1722023820}) 2024-09-15T21:32:04.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.319 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3318329723}) 2024-09-15T21:32:04.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1918059535}) 2024-09-15T21:32:04.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=208888320}) 2024-09-15T21:32:04.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=775036397}) 2024-09-15T21:32:04.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.321 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=208888320}) 2024-09-15T21:32:04.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.315+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3173037694}) 2024-09-15T21:32:04.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146503969}) 2024-09-15T21:32:04.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2571548549}) 2024-09-15T21:32:04.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146503969}) 2024-09-15T21:32:04.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573139239}) 2024-09-15T21:32:04.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=473754635}) 2024-09-15T21:32:04.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3956679971}) 2024-09-15T21:32:04.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=473754635}) 2024-09-15T21:32:04.326 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.319+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2032127802}) 2024-09-15T21:32:04.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838089887}) 2024-09-15T21:32:04.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1223603188}) 2024-09-15T21:32:04.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838089887}) 2024-09-15T21:32:04.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1007248429}) 2024-09-15T21:32:04.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958266389}) 2024-09-15T21:32:04.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1238085302}) 2024-09-15T21:32:04.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958266389}) 2024-09-15T21:32:04.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.323+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1829899919}) 2024-09-15T21:32:04.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969118194}) 2024-09-15T21:32:04.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3139149880}) 2024-09-15T21:32:04.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969118194}) 2024-09-15T21:32:04.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1911165583}) 2024-09-15T21:32:04.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875931469}) 2024-09-15T21:32:04.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3257680863}) 2024-09-15T21:32:04.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875931469}) 2024-09-15T21:32:04.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.327+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3214283018}) 2024-09-15T21:32:04.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2226307817}) 2024-09-15T21:32:04.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3167360518}) 2024-09-15T21:32:04.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2226307817}) 2024-09-15T21:32:04.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1283756668}) 2024-09-15T21:32:04.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=210540128}) 2024-09-15T21:32:04.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2532818242}) 2024-09-15T21:32:04.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=210540128}) 2024-09-15T21:32:04.337 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.331+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=387730136}) 2024-09-15T21:32:04.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=482333442}) 2024-09-15T21:32:04.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1613437661}) 2024-09-15T21:32:04.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=482333442}) 2024-09-15T21:32:04.339 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4154830736}) 2024-09-15T21:32:04.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2627850038}) 2024-09-15T21:32:04.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2744712951}) 2024-09-15T21:32:04.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2627850038}) 2024-09-15T21:32:04.341 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.335+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685606167}) 2024-09-15T21:32:04.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164360228}) 2024-09-15T21:32:04.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1880911529}) 2024-09-15T21:32:04.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164360228}) 2024-09-15T21:32:04.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4151570458}) 2024-09-15T21:32:04.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381204785}) 2024-09-15T21:32:04.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3522729213}) 2024-09-15T21:32:04.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381204785}) 2024-09-15T21:32:04.345 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.339+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2361752552}) 2024-09-15T21:32:04.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2222930157}) 2024-09-15T21:32:04.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3838666260}) 2024-09-15T21:32:04.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2222930157}) 2024-09-15T21:32:04.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2425496149}) 2024-09-15T21:32:04.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=765236789}) 2024-09-15T21:32:04.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2492291788}) 2024-09-15T21:32:04.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=765236789}) 2024-09-15T21:32:04.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.343+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3401208095}) 2024-09-15T21:32:04.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3543895206}) 2024-09-15T21:32:04.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3017614819}) 2024-09-15T21:32:04.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3543895206}) 2024-09-15T21:32:04.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3382366563}) 2024-09-15T21:32:04.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2637167003}) 2024-09-15T21:32:04.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3939340752}) 2024-09-15T21:32:04.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2637167003}) 2024-09-15T21:32:04.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.347+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2489326320}) 2024-09-15T21:32:04.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3586439004}) 2024-09-15T21:32:04.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3834224182}) 2024-09-15T21:32:04.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3586439004}) 2024-09-15T21:32:04.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1169184526}) 2024-09-15T21:32:04.355 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3134668209}) 2024-09-15T21:32:04.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1133765419}) 2024-09-15T21:32:04.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3134668209}) 2024-09-15T21:32:04.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2701195266}) 2024-09-15T21:32:04.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4160159969}) 2024-09-15T21:32:04.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2731945577}) 2024-09-15T21:32:04.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4160159969}) 2024-09-15T21:32:04.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.351+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2050974516}) 2024-09-15T21:32:04.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=540791912}) 2024-09-15T21:32:04.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3309554184}) 2024-09-15T21:32:04.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=540791912}) 2024-09-15T21:32:04.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3965317064}) 2024-09-15T21:32:04.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212792902}) 2024-09-15T21:32:04.362 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=772549279}) 2024-09-15T21:32:04.362 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.362 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212792902}) 2024-09-15T21:32:04.362 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.355+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=168638804}) 2024-09-15T21:32:04.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=563223552}) 2024-09-15T21:32:04.364 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3518988442}) 2024-09-15T21:32:04.364 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.364 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=563223552}) 2024-09-15T21:32:04.364 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2014187989}) 2024-09-15T21:32:04.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4293870379}) 2024-09-15T21:32:04.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2448800611}) 2024-09-15T21:32:04.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4293870379}) 2024-09-15T21:32:04.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.359+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1714258759}) 2024-09-15T21:32:04.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170586731}) 2024-09-15T21:32:04.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=384879240}) 2024-09-15T21:32:04.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170586731}) 2024-09-15T21:32:04.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3518596267}) 2024-09-15T21:32:04.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815588022}) 2024-09-15T21:32:04.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3285180064}) 2024-09-15T21:32:04.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815588022}) 2024-09-15T21:32:04.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.363+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2353330829}) 2024-09-15T21:32:04.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1837832640}) 2024-09-15T21:32:04.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1205237162}) 2024-09-15T21:32:04.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1837832640}) 2024-09-15T21:32:04.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2145405474}) 2024-09-15T21:32:04.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3535081802}) 2024-09-15T21:32:04.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3856615588}) 2024-09-15T21:32:04.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3535081802}) 2024-09-15T21:32:04.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.367+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=319284542}) 2024-09-15T21:32:04.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4162647644}) 2024-09-15T21:32:04.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4260219148}) 2024-09-15T21:32:04.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4162647644}) 2024-09-15T21:32:04.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2850908629}) 2024-09-15T21:32:04.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4768176}) 2024-09-15T21:32:04.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2055962112}) 2024-09-15T21:32:04.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4768176}) 2024-09-15T21:32:04.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.371+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=849821172}) 2024-09-15T21:32:04.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935844152}) 2024-09-15T21:32:04.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3384786761}) 2024-09-15T21:32:04.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935844152}) 2024-09-15T21:32:04.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=13846874}) 2024-09-15T21:32:04.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1700280156}) 2024-09-15T21:32:04.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3397447202}) 2024-09-15T21:32:04.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1700280156}) 2024-09-15T21:32:04.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.375+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4226536015}) 2024-09-15T21:32:04.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1600629294}) 2024-09-15T21:32:04.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094335503}) 2024-09-15T21:32:04.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1600629294}) 2024-09-15T21:32:04.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2424911775}) 2024-09-15T21:32:04.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2221935472}) 2024-09-15T21:32:04.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633002861}) 2024-09-15T21:32:04.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2221935472}) 2024-09-15T21:32:04.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.379+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3244927421}) 2024-09-15T21:32:04.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3451173910}) 2024-09-15T21:32:04.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1539192716}) 2024-09-15T21:32:04.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3451173910}) 2024-09-15T21:32:04.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1243444318}) 2024-09-15T21:32:04.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2444793808}) 2024-09-15T21:32:04.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=809952294}) 2024-09-15T21:32:04.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2444793808}) 2024-09-15T21:32:04.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=264980372}) 2024-09-15T21:32:04.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=114902336}) 2024-09-15T21:32:04.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4073264433}) 2024-09-15T21:32:04.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=114902336}) 2024-09-15T21:32:04.390 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.383+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2109863776}) 2024-09-15T21:32:04.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3140126965}) 2024-09-15T21:32:04.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=285377379}) 2024-09-15T21:32:04.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3140126965}) 2024-09-15T21:32:04.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2542559058}) 2024-09-15T21:32:04.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399723328}) 2024-09-15T21:32:04.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1340998196}) 2024-09-15T21:32:04.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399723328}) 2024-09-15T21:32:04.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.387+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=96120134}) 2024-09-15T21:32:04.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1158651563}) 2024-09-15T21:32:04.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=698759856}) 2024-09-15T21:32:04.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1158651563}) 2024-09-15T21:32:04.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3159078593}) 2024-09-15T21:32:04.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1568554338}) 2024-09-15T21:32:04.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3713077897}) 2024-09-15T21:32:04.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1568554338}) 2024-09-15T21:32:04.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.391+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=586384983}) 2024-09-15T21:32:04.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3772280346}) 2024-09-15T21:32:04.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3972027086}) 2024-09-15T21:32:04.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3772280346}) 2024-09-15T21:32:04.400 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1437012203}) 2024-09-15T21:32:04.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=589495125}) 2024-09-15T21:32:04.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2153194183}) 2024-09-15T21:32:04.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=589495125}) 2024-09-15T21:32:04.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.395+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2127693533}) 2024-09-15T21:32:04.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193994821}) 2024-09-15T21:32:04.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4017048060}) 2024-09-15T21:32:04.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193994821}) 2024-09-15T21:32:04.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2207302212}) 2024-09-15T21:32:04.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1904297893}) 2024-09-15T21:32:04.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3504406276}) 2024-09-15T21:32:04.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1904297893}) 2024-09-15T21:32:04.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.399+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450051284}) 2024-09-15T21:32:04.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780207027}) 2024-09-15T21:32:04.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=127877382}) 2024-09-15T21:32:04.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780207027}) 2024-09-15T21:32:04.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1305361422}) 2024-09-15T21:32:04.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2713686087}) 2024-09-15T21:32:04.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3581280478}) 2024-09-15T21:32:04.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2713686087}) 2024-09-15T21:32:04.409 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.403+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284368247}) 2024-09-15T21:32:04.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1544250309}) 2024-09-15T21:32:04.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3743257887}) 2024-09-15T21:32:04.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1544250309}) 2024-09-15T21:32:04.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936758803}) 2024-09-15T21:32:04.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491806533}) 2024-09-15T21:32:04.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=131156829}) 2024-09-15T21:32:04.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491806533}) 2024-09-15T21:32:04.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.407+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=899772526}) 2024-09-15T21:32:04.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766718827}) 2024-09-15T21:32:04.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=582750878}) 2024-09-15T21:32:04.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766718827}) 2024-09-15T21:32:04.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=343327033}) 2024-09-15T21:32:04.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2919128956}) 2024-09-15T21:32:04.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695176905}) 2024-09-15T21:32:04.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2919128956}) 2024-09-15T21:32:04.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.411+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1085244045}) 2024-09-15T21:32:04.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2309679813}) 2024-09-15T21:32:04.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3032447189}) 2024-09-15T21:32:04.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2309679813}) 2024-09-15T21:32:04.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=710098130}) 2024-09-15T21:32:04.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=380947909}) 2024-09-15T21:32:04.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1354612351}) 2024-09-15T21:32:04.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=380947909}) 2024-09-15T21:32:04.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.415+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3908848560}) 2024-09-15T21:32:04.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4177622467}) 2024-09-15T21:32:04.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1597809638}) 2024-09-15T21:32:04.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4177622467}) 2024-09-15T21:32:04.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4130470737}) 2024-09-15T21:32:04.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2470773591}) 2024-09-15T21:32:04.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3577437630}) 2024-09-15T21:32:04.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2470773591}) 2024-09-15T21:32:04.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.419+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2304128814}) 2024-09-15T21:32:04.427 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.423+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.423+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=22,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=27} crc {logm=2306000021,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2251144053,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1642619227}) 2024-09-15T21:32:04.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.423+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=25} crc {logm=3317209643,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3811985176,mon_config_key=521969047,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1775701575}) 2024-09-15T21:32:04.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.423+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.423+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=22,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=27} crc {logm=2306000021,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2251144053,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1642619227}) 2024-09-15T21:32:04.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.423+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=26} crc {logm=2814283451,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3811985176,mon_config_key=3562240218,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1360937367}) 2024-09-15T21:32:04.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.435+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.435+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=22} crc {osdmap=227051593}) 2024-09-15T21:32:04.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.435+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=22} crc {osdmap=712409319}) 2024-09-15T21:32:04.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.435+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:32:04.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.435+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=22} crc {osdmap=227051593}) 2024-09-15T21:32:04.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:32:04.435+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=22} crc {osdmap=464844186}) 2024-09-15T21:32:05.283 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:32:05.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:05.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient: get_monmap_and_config 2024-09-15T21:32:05.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient: build_initial_monmap 2024-09-15T21:32:05.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient: monmap: 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:05.415735+0000 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:05.415735+0000 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:05.416 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 1 Processor -- start 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 1 -- start start 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient: init 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f71ac0a3a80 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:05.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71ac0a4570 0x7f71ac13ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient(hunting): picked mon.noname-c con 0x7f71ac0a4570 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f71ac13d3a0 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71ac074910 con 0x7f71ac13d3a0 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71ac09ef70 con 0x7f71ac0a3a80 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f71ac09fa50 con 0x7f71ac0a4570 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b2d5e640 10 monclient(hunting): authenticate will time out at 1848.811112s 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b12fb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f71ac13d3a0 0x7f71ac13f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42400/0 (socket says 172.21.15.17:42400) 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71b12fb640 1 -- 172.21.15.17:0/2408534199 learned_addr learned my addr 172.21.15.17:0/2408534199 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 1 --2- 172.21.15.17:0/2408534199 >> v2:172.21.15.119:3300/0 conn(0x7f71ac0a4570 0x7f71ac13ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): get_auth_request con 0x7f71ac0a4570 auth_method 0 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209169478 0 0) 0x7f71a40014a0 con 0x7f71ac13d3a0 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 10 monclient(hunting): my global_id is 174141 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7194003530 con 0x7f71ac13d3a0 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1294945963 0 0) 0x7f7194003530 con 0x7f71ac13d3a0 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 >> v2:172.21.15.119:3300/0 conn(0x7f71ac0a4570 msgr2=0x7f71ac13ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 --2- 172.21.15.17:0/2408534199 >> v2:172.21.15.119:3300/0 conn(0x7f71ac0a4570 0x7f71ac13ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 >> v1:172.21.15.119:6789/0 conn(0x7f71ac0a3a80 legacy=0x7f71ac0a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.411+0000 7f71ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac06e790 con 0x7f71ac13d3a0 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b2d5e640 5 monclient: authenticate success, global_id 174141 2024-09-15T21:32:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f71a4001c50 con 0x7f71ac13d3a0 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: dump: 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 >> v1:172.21.15.17:6789/0 conn(0x7f71ac13d3a0 legacy=0x7f71ac13f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7194003a60 0x7f7194003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): picked mon.c con 0x7f7194003a60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7194004810 0x7f7194004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): picked mon.b con 0x7f7194004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71940053d0 0x7f719400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): picked mon.a con 0x7f71940053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac09fa50 con 0x7f71940053d0 2024-09-15T21:32:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f719400c240 con 0x7f7194004810 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f719400c440 con 0x7f7194003a60 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b2d5e640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7194003a60 0x7f7194003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71940053d0 0x7f719400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): get_auth_request con 0x7f71940053d0 auth_method 0 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): get_auth_request con 0x7f7194003a60 auth_method 0 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b0afa640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71abfff640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7194004810 0x7f7194004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient(hunting): handle_auth_done global_id 174141 payload 293 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 -- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7194003a60 msgr2=0x7f7194003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7194003a60 0x7f7194003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 -- 172.21.15.17:0/2408534199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7194004810 msgr2=0x7f7194004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7194004810 0x7f7194004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 monclient: found mon.a 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 -- 172.21.15.17:0/2408534199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194003530 con 0x7f71940053d0 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71abfff640 10 monclient: get_auth_request con 0x7f7194004810 auth_method 0 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71abfff640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7194004810 0x7f7194004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 -- 172.21.15.17:0/2408534199 reap_dead start 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b12fb640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71940053d0 0x7f719400b9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71a400aa00 tx=0x7f71a401d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ed4a0d437ca88c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400a650 con 0x7f71940053d0 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: dump: 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.430 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b2d5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a40ac660 con 0x7f71940053d0 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b2d5e640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71b2d5e640 4 set_mon_vals no callback set 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 1 -- 172.21.15.17:0/2408534199 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40aca50 con 0x7f71940053d0 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.415+0000 7f71ab7fe640 10 monclient: dump: 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.431 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.432 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.432 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.432 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.432 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.432 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: shutdown 2024-09-15T21:32:05.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:05.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71940053d0 msgr2=0x7f719400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71940053d0 0x7f719400b9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71a400aa00 tx=0x7f71a401d280 comp rx=0 tx=0).stop 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2408534199 shutdown_connections 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2408534199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71940053d0 0x7f719400b9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2408534199 >> 172.21.15.17:0/2408534199 conn(0x7f71ac0a8980 msgr2=0x7f71ac0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2408534199 shutdown_connections 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2408534199 wait complete. 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: build_initial_monmap 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: monmap: 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 Processor -- start 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- start start 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: init 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71940053d0 0x7f71ac09c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): picked mon.noname-b con 0x7f71940053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 0x7f71ac09d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): picked mon.noname-c con 0x7f71ac09cf10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71ac1cf5c0 0x7f71ac1cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): picked mon.noname-a con 0x7f71ac1cf5c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0a3e60 con 0x7f71ac1cf5c0 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0a5890 con 0x7f71940053d0 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f71ac0a3fe0 con 0x7f71ac09cf10 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient(hunting): authenticate will time out at 1848.817962s 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 0x7f71ac09d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71ac1cf5c0 0x7f71ac1cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 0x7f71ac09d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48044/0 (socket says 172.21.15.17:48044) 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 -- 172.21.15.17:0/2499486340 learned_addr learned my addr 172.21.15.17:0/2499486340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:05.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): get_auth_request con 0x7f71ac09cf10 auth_method 0 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): get_auth_request con 0x7f71ac1cf5c0 auth_method 0 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b0afa640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71940053d0 0x7f71ac09c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b0afa640 10 monclient(hunting): get_auth_request con 0x7f71940053d0 auth_method 0 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b0afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b0afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b0afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient(hunting): handle_auth_done global_id 164252 payload 824 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 -- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71940053d0 msgr2=0x7f71ac09c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71940053d0 0x7f71ac09c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 -- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71ac1cf5c0 msgr2=0x7f71ac1cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71ac1cf5c0 0x7f71ac1cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 monclient: found mon.noname-c 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 -- 172.21.15.17:0/2499486340 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac135120 con 0x7f71ac09cf10 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b12fb640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71ac1cf5c0 0x7f71ac1cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71abfff640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 0x7f71ac09d310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71a00108d0 tx=0x7f71a0010d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a000f040 con 0x7f71ac09cf10 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: dump: 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 5 monclient: authenticate success, global_id 164252 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a0004220 con 0x7f71ac09cf10 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a000e960 con 0x7f71ac09cf10 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 10 monclient: dump: 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _renew_subs 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71aaffd640 4 set_mon_vals no callback set 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac135400 con 0x7f71ac09cf10 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _renew_subs 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1409d0 con 0x7f71ac09cf10 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac13d010 con 0x7f71ac09cf10 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.419+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f71a000eb00 con 0x7f71ac09cf10 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.423+0000 7f71a97fa640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f719803b9f0 0x7f719803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.423+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f71a00506b0 con 0x7f71ac09cf10 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.423+0000 7f71b0afa640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f719803b9f0 0x7f719803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.423+0000 7f71b0afa640 10 monclient: get_auth_request con 0x7f719803b9f0 auth_method 0 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.423+0000 7f71b0afa640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f719803b9f0 0x7f719803deb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f719c007fd0 tx=0x7f719c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.427+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f71a0018030 con 0x7f71ac09cf10 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.427+0000 7f71a97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.427+0000 7f71a97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:05.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71b2d5e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:32:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71b2d5e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:32:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71b2d5e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f71ac133f10 con 0x7f71ac09cf10 2024-09-15T21:32:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71a97fa640 1 -- 172.21.15.17:0/2499486340 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f71a000edf0 con 0x7f71ac09cf10 2024-09-15T21:32:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71a97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:32:05.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.663+0000 7f71a97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:05.670 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f719803b9f0 msgr2=0x7f719803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f719803b9f0 0x7f719803deb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f719c007fd0 tx=0x7f719c009040 comp rx=0 tx=0).stop 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 10 monclient: shutdown 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 msgr2=0x7f71ac09d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 0x7f71ac09d310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71a00108d0 tx=0x7f71a0010d90 comp rx=0 tx=0).stop 2024-09-15T21:32:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 shutdown_connections 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f719803b9f0 0x7f719803deb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71ac1cf5c0 0x7f71ac1cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71ac09cf10 0x7f71ac09d310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 --2- 172.21.15.17:0/2499486340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71940053d0 0x7f71ac09c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 >> 172.21.15.17:0/2499486340 conn(0x7f71ac0a8980 msgr2=0x7f71ac0a7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 shutdown_connections 2024-09-15T21:32:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.667+0000 7f71b2d5e640 1 -- 172.21.15.17:0/2499486340 wait complete. 2024-09-15T21:32:05.688 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:32:05.688 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:32:05.688 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:32:05.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient: get_monmap_and_config 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient: build_initial_monmap 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient: monmap: 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:05.804200+0000 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:05.804200+0000 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:05.805 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:05.805 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:05.805 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:05.805 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:05.805 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 1 Processor -- start 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 1 -- start start 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient: init 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa6e809d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa6e809def0 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e813afc0 0x7fa6e813d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient(hunting): picked mon.noname-b con 0x7fa6e813afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e80a3490 con 0x7fa6e809def0 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e80a0320 con 0x7fa6e809d400 2024-09-15T21:32:05.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e80a1bc0 con 0x7fa6e813afc0 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6ed1d2640 10 monclient(hunting): authenticate will time out at 1849.199501s 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa6e809def0 0x7fa6e81398a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42402/0 (socket says 172.21.15.17:42402) 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e659c640 1 -- 172.21.15.17:0/4040695485 learned_addr learned my addr 172.21.15.17:0/4040695485 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 1 --2- 172.21.15.17:0/4040695485 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e813afc0 0x7fa6e813d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): get_auth_request con 0x7fa6e813afc0 auth_method 0 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e5d9b640 1 -- 172.21.15.17:0/4040695485 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470949553 0 0) 0x7fa6e80a3490 con 0x7fa6e809def0 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e5d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e5d9b640 10 monclient(hunting): my global_id is 174147 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e5d9b640 1 -- 172.21.15.17:0/4040695485 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d0003530 con 0x7fa6e809def0 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 10 monclient(hunting): handle_auth_done global_id 164255 payload 274 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 1 -- 172.21.15.17:0/4040695485 >> v1:172.21.15.119:6789/0 conn(0x7fa6e809d400 legacy=0x7fa6e809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.799+0000 7fa6e759e640 1 -- 172.21.15.17:0/4040695485 >> v1:172.21.15.17:6789/0 conn(0x7fa6e809def0 legacy=0x7fa6e81398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e759e640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e759e640 1 monclient: found mon.noname-b 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e759e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:32:05.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e759e640 1 -- 172.21.15.17:0/4040695485 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e809e7a0 con 0x7fa6e813afc0 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e759e640 1 --2- 172.21.15.17:0/4040695485 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e813afc0 0x7fa6e813d3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009dc0 tx=0x7fa6e001d220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23e1c23ce5d942 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 1 -- 172.21.15.17:0/4040695485 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b2070 con 0x7fa6e813afc0 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: dump: 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.816 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 1 -- 172.21.15.17:0/4040695485 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e00a58a0 con 0x7fa6e813afc0 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:05.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 1 -- 172.21.15.17:0/4040695485 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00a5c70 con 0x7fa6e813afc0 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 5 monclient: authenticate success, global_id 164255 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 4 set_mon_vals no callback set 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6e5d9b640 10 monclient: dump: 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 10 monclient: shutdown 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/4040695485 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e813afc0 msgr2=0x7fa6e813d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 --2- 172.21.15.17:0/4040695485 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e813afc0 0x7fa6e813d3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009dc0 tx=0x7fa6e001d220 comp rx=0 tx=0).stop 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/4040695485 shutdown_connections 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 --2- 172.21.15.17:0/4040695485 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e813afc0 0x7fa6e813d3b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/4040695485 >> 172.21.15.17:0/4040695485 conn(0x7fa6e80a66a0 msgr2=0x7fa6e80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/4040695485 shutdown_connections 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/4040695485 wait complete. 2024-09-15T21:32:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 10 monclient: build_initial_monmap 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 10 monclient: monmap: 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 Processor -- start 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 1 -- start start 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.803+0000 7fa6ed1d2640 10 monclient: init 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e809d400 0x7fa6e8139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): picked mon.noname-a con 0x7fa6e809d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e809def0 0x7fa6e8139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): picked mon.noname-c con 0x7fa6e809def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e813afc0 0x7fa6e811f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): picked mon.noname-b con 0x7fa6e813afc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e809ce50 con 0x7fa6e809d400 2024-09-15T21:32:05.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e809ccd0 con 0x7fa6e813afc0 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e8140520 con 0x7fa6e809def0 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): authenticate will time out at 1849.204150s 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e809d400 0x7fa6e8139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e809d400 0x7fa6e8139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46574/0 (socket says 172.21.15.17:46574) 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 -- 172.21.15.17:0/1124814368 learned_addr learned my addr 172.21.15.17:0/1124814368 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): get_auth_request con 0x7fa6e809d400 auth_method 0 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e809def0 0x7fa6e8139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): get_auth_request con 0x7fa6e809def0 auth_method 0 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e759e640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e813afc0 0x7fa6e811f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient(hunting): handle_auth_done global_id 174150 payload 824 2024-09-15T21:32:05.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e809def0 msgr2=0x7fa6e8139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa6e809def0 0x7fa6e8139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e813afc0 msgr2=0x7fa6e811f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e813afc0 0x7fa6e811f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 monclient: found mon.noname-a 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e8125f90 con 0x7fa6e809d400 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e759e640 10 monclient: get_auth_request con 0x7fa6e813afc0 auth_method 0 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e759e640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e813afc0 0x7fa6e811f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e6d9d640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e809d400 0x7fa6e8139610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c450 tx=0x7fa6d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011790 con 0x7fa6e809d400 2024-09-15T21:32:05.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: dump: 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 5 monclient: authenticate success, global_id 174150 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient: _renew_subs 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d8017070 con 0x7fa6e809d400 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e8126270 con 0x7fa6e809d400 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011b90 con 0x7fa6e809d400 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: dump: 2024-09-15T21:32:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient: _renew_subs 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e559a640 4 set_mon_vals no callback set 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e8125cb0 con 0x7fa6e809d400 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 0x7fa6e802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e8120440 con 0x7fa6e811fe60 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 0x7fa6e802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient: get_auth_request con 0x7fa6e811fe60 auth_method 0 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa6d8011d30 con 0x7fa6e809d400 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 10 monclient(hunting): handle_auth_done global_id 174153 payload 824 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6e809d7e0 con 0x7fa6e811fe60 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa6d403b9f0 0x7fa6d403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa6d8050430 con 0x7fa6e809d400 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e659c640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 0x7fa6e802d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8125900 tx=0x7fa6dc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc0058c0 con 0x7fa6e811fe60 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: dump: 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e759e640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa6d403b9f0 0x7fa6d403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e759e640 10 monclient: get_auth_request con 0x7fa6d403b9f0 auth_method 0 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa6dc018070 con 0x7fa6e811fe60 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:05.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 msgr2=0x7fa6e802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 0x7fa6e802d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8125900 tx=0x7fa6dc001a00 comp rx=0 tx=0).stop 2024-09-15T21:32:05.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 msgr2=0x7fa6e802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:05.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6cf7fe640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e811fe60 0x7fa6e802d450 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8125900 tx=0x7fa6dc001a00 comp rx=0 tx=0).stop 2024-09-15T21:32:05.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.807+0000 7fa6e759e640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa6d403b9f0 0x7fa6d403deb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00af000 tx=0x7fa6e0004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 0x7fa6e802dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e813b860 con 0x7fa6e802d990 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 0x7fa6e802dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient: get_auth_request con 0x7fa6e802d990 auth_method 0 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 10 monclient(hunting): handle_auth_done global_id 174156 payload 824 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 1 -- 172.21.15.17:0/1124814368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa6e80729e0 con 0x7fa6e802d990 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e659c640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 0x7fa6e802dda0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa6e81205c0 tx=0x7fa6dc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc006440 con 0x7fa6e802d990 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 10 monclient: dump: 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4547 (secure 0 0 0) 0x7fa6dc006440 con 0x7fa6e802d990 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 msgr2=0x7fa6e802dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 0x7fa6e802dda0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa6e81205c0 tx=0x7fa6dc005800 comp rx=0 tx=0).stop 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 msgr2=0x7fa6e802dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6cf7fe640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e802d990 0x7fa6e802dda0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa6e81205c0 tx=0x7fa6dc005800 comp rx=0 tx=0).stop 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 140, 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 15551, 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:32:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:05.848 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa6d403b9f0 msgr2=0x7fa6d403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa6d403b9f0 0x7fa6d403deb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00af000 tx=0x7fa6e0004ac0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 10 monclient: shutdown 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e809d400 msgr2=0x7fa6e8139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 --2- 172.21.15.17:0/1124814368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e809d400 0x7fa6e8139610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c450 tx=0x7fa6d800c910 comp rx=0 tx=0).stop 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e759e640 1 -- 172.21.15.17:0/1124814368 reap_dead start 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6e759e640 1 -- 172.21.15.17:0/1124814368 reap_dead start 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 shutdown_connections 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 >> 172.21.15.17:0/1124814368 conn(0x7fa6e80a66a0 msgr2=0x7fa6e813ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 shutdown_connections 2024-09-15T21:32:05.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.815+0000 7fa6ed1d2640 1 -- 172.21.15.17:0/1124814368 wait complete. 2024-09-15T21:32:05.850 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:32:05.851 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:32:05.851 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient: get_monmap_and_config 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient: build_initial_monmap 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient: monmap: 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:05.963258+0000 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:05.963258+0000 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:05.964 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 Processor -- start 2024-09-15T21:32:05.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 -- start start 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient: init 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff0340a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 0x7ff03413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): picked mon.noname-b con 0x7ff03409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff0340a1380 0x7ff03413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): picked mon.noname-a con 0x7ff0340a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0340a3130 con 0x7ff0340a0890 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff034073df0 con 0x7ff0340a1380 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff0340a1bf0 con 0x7ff03409ffc0 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff038964640 10 monclient(hunting): authenticate will time out at 1849.358608s 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 0x7ff03413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 0x7ff03413aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48068/0 (socket says 172.21.15.17:48068) 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 1 -- 172.21.15.17:0/3011591477 learned_addr learned my addr 172.21.15.17:0/3011591477 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): get_auth_request con 0x7ff03409ffc0 auth_method 0 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 1 --2- 172.21.15.17:0/3011591477 >> v2:172.21.15.17:3300/0 conn(0x7ff0340a1380 0x7ff03413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): get_auth_request con 0x7ff0340a1380 auth_method 0 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient(hunting): handle_auth_done global_id 174159 payload 274 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 1 -- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 msgr2=0x7ff03413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 1 --2- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 0x7ff03413aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 1 -- 172.21.15.17:0/3011591477 >> v1:172.21.15.119:6789/0 conn(0x7ff0340a0890 legacy=0x7ff0340a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 1 monclient: found mon.noname-a 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff032d9d640 1 -- 172.21.15.17:0/3011591477 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03409ccd0 con 0x7ff0340a1380 2024-09-15T21:32:05.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.959+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 0x7ff03413aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff032d9d640 1 --2- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0340a1380 0x7ff03413d010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009d90 tx=0x7ff02c01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3820fe6b1f9af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 1 -- 172.21.15.17:0/3011591477 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0b2070 con 0x7ff0340a1380 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: dump: 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.976 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 1 -- 172.21.15.17:0/3011591477 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c0a5800 con 0x7ff0340a1380 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 1 -- 172.21.15.17:0/3011591477 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0a5bd0 con 0x7ff0340a1380 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff031d9b640 10 monclient: dump: 2024-09-15T21:32:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff038964640 5 monclient: authenticate success, global_id 174159 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff038964640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.963+0000 7ff038964640 4 set_mon_vals no callback set 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: shutdown 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0340a1380 msgr2=0x7ff03413d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0340a1380 0x7ff03413d010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009d90 tx=0x7ff02c01c830 comp rx=0 tx=0).stop 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3011591477 shutdown_connections 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0340a1380 0x7ff03413d010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- 172.21.15.17:0/3011591477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03409ffc0 0x7ff03413aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3011591477 >> 172.21.15.17:0/3011591477 conn(0x7ff0340a66a0 msgr2=0x7ff0340a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3011591477 shutdown_connections 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3011591477 wait complete. 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: build_initial_monmap 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: monmap: 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 Processor -- start 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- start start 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: init 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 0x7ff03413bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): picked mon.noname-a con 0x7ff03409ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0340a0890 0x7ff0341cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): picked mon.noname-b con 0x7ff0340a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0340a1380 0x7ff034139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): picked mon.noname-c con 0x7ff0340a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff034140100 con 0x7ff03409ffc0 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff03413ff80 con 0x7ff0340a0890 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff034140280 con 0x7ff0340a1380 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): authenticate will time out at 1849.365803s 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 0x7ff03413bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0340a1380 0x7ff034139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 0x7ff03413bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46616/0 (socket says 172.21.15.17:46616) 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): get_auth_request con 0x7ff0340a1380 auth_method 0 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 -- 172.21.15.17:0/3741663361 learned_addr learned my addr 172.21.15.17:0/3741663361 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): get_auth_request con 0x7ff03409ffc0 auth_method 0 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0340a0890 0x7ff0341cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient(hunting): handle_auth_done global_id 174162 payload 824 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0340a1380 msgr2=0x7ff034139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0340a1380 0x7ff034139740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0340a0890 msgr2=0x7ff0341cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0340a0890 0x7ff0341cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient: _finish_hunting 0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 monclient: found mon.noname-a 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03411caa0 con 0x7ff03409ffc0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient: get_auth_request con 0x7ff0340a0890 auth_method 0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0340a0890 0x7ff0341cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff032d9d640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0340a1380 0x7ff034139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff03259c640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 0x7ff03413bd10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00c500 tx=0x7ff01c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c017070 con 0x7ff03409ffc0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: dump: 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: _finish_auth 0 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: _check_auth_tickets 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01c0052d0 con 0x7ff03409ffc0 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c012870 con 0x7ff03409ffc0 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 10 monclient: dump: 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 5 monclient: authenticate success, global_id 174162 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: _renew_subs 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff030d99640 4 set_mon_vals no callback set 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03411cd80 con 0x7ff03409ffc0 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: _renew_subs 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03411c7c0 con 0x7ff03409ffc0 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff01c01a070 con 0x7ff03409ffc0 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 0x7ff034139c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff0340a0c70 con 0x7ff0341cdd40 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 0x7ff034139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient: get_auth_request con 0x7ff0341cdd40 auth_method 0 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff01403b9c0 0x7ff01403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff01c04f860 con 0x7ff03409ffc0 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.967+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02bfff640 10 monclient(hunting): handle_auth_done global_id 164267 payload 824 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02bfff640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff03409d0f0 con 0x7ff0341cdd40 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff032d9d640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff01403b9c0 0x7ff01403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff032d9d640 10 monclient: get_auth_request con 0x7ff01403b9c0 auth_method 0 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 0x7ff034139c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff034132f80 tx=0x7ff024010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400f040 con 0x7ff0341cdd40 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 10 monclient: dump: 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff024010e70 con 0x7ff0341cdd40 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 msgr2=0x7ff034139c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 0x7ff034139c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff034132f80 tx=0x7ff024010c20 comp rx=0 tx=0).stop 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 msgr2=0x7ff034139c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff02affd640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff0341cdd40 0x7ff034139c80 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff034132f80 tx=0x7ff024010c20 comp rx=0 tx=0).stop 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.971+0000 7ff032d9d640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff01403b9c0 0x7ff01403de80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0af000 tx=0x7ff02c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff038964640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff038964640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 0x7ff034071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff038964640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff03413a5d0 con 0x7ff03413a1c0 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 0x7ff034071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient: get_auth_request con 0x7ff03413a1c0 auth_method 0 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 10 monclient(hunting): handle_auth_done global_id 164270 payload 824 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 1 -- 172.21.15.17:0/3741663361 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff03406d020 con 0x7ff03413a1c0 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02bfff640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 0x7ff034071550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff034132f80 tx=0x7ff0240046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024025070 con 0x7ff03413a1c0 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02affd640 10 monclient: dump: 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.975+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4879 (secure 0 0 0) 0x7ff024006540 con 0x7ff03413a1c0 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff02affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff02affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 msgr2=0x7ff034071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff02affd640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 0x7ff034071550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff034132f80 tx=0x7ff0240046d0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff02affd640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 msgr2=0x7ff034071550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff02affd640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff03413a1c0 0x7ff034071550 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff034132f80 tx=0x7ff0240046d0 comp rx=0 tx=0).stop 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 140, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 74559, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff01403b9c0 msgr2=0x7ff01403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff01403b9c0 0x7ff01403de80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0af000 tx=0x7ff02c0a7070 comp rx=0 tx=0).stop 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 10 monclient: shutdown 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 msgr2=0x7ff03413bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 0x7ff03413bd10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00c500 tx=0x7ff01c00c9c0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff032d9d640 1 -- 172.21.15.17:0/3741663361 reap_dead start 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 shutdown_connections 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 --2- 172.21.15.17:0/3741663361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03409ffc0 0x7ff03413bd10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 >> 172.21.15.17:0/3741663361 conn(0x7ff0340a66a0 msgr2=0x7ff034135870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 shutdown_connections 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:05.979+0000 7ff038964640 1 -- 172.21.15.17:0/3741663361 wait complete. 2024-09-15T21:32:06.010 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:32:06.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:32:06.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient: get_monmap_and_config 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient: build_initial_monmap 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient: monmap: 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:06.127240+0000 2024-09-15T21:32:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:06.127240+0000 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:06.128 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 Processor -- start 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 -- start start 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient: init 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f991c0a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f991c0a5580 0x7f991c13cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): picked mon.noname-c con 0x7f991c0a5580 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 0x7f991c13f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): picked mon.noname-b con 0x7f991c0a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c0a2c30 con 0x7f991c0a4a90 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f991c0a0f50 con 0x7f991c0a5580 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0a1a00 con 0x7f991c0a59d0 2024-09-15T21:32:06.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f99214cb640 10 monclient(hunting): authenticate will time out at 1849.522646s 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 0x7f991c13f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 0x7f991c13f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48084/0 (socket says 172.21.15.17:48084) 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 -- 172.21.15.17:0/3061075272 learned_addr learned my addr 172.21.15.17:0/3061075272 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): get_auth_request con 0x7f991c0a59d0 auth_method 0 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 1 --2- 172.21.15.17:0/3061075272 >> v2:172.21.15.119:3300/0 conn(0x7f991c0a5580 0x7f991c13cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): get_auth_request con 0x7f991c0a5580 auth_method 0 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient(hunting): handle_auth_done global_id 164273 payload 274 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 -- 172.21.15.17:0/3061075272 >> v2:172.21.15.119:3300/0 conn(0x7f991c0a5580 msgr2=0x7f991c13cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 --2- 172.21.15.17:0/3061075272 >> v2:172.21.15.119:3300/0 conn(0x7f991c0a5580 0x7f991c13cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 -- 172.21.15.17:0/3061075272 >> v1:172.21.15.119:6789/0 conn(0x7f991c0a4a90 legacy=0x7f991c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:32:06.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:32:06.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:06.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:32:06.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.123+0000 7f991b7fe640 1 -- 172.21.15.17:0/3061075272 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0a1c90 con 0x7f991c0a59d0 2024-09-15T21:32:06.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f991b7fe640 1 --2- 172.21.15.17:0/3061075272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 0x7f991c13f260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9914009e80 tx=0x7f991401c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a451909d3c9656e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 1 -- 172.21.15.17:0/3061075272 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140a5680 con 0x7f991c0a59d0 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: dump: 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:06.141 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:06.142 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:06.142 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: _finish_auth 0 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 1 -- 172.21.15.17:0/3061075272 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99140a5820 con 0x7f991c0a59d0 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 5 monclient: authenticate success, global_id 164273 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 1 -- 172.21.15.17:0/3061075272 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140a5bf0 con 0x7f991c0a59d0 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 4 set_mon_vals no callback set 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f9919ffb640 10 monclient: dump: 2024-09-15T21:32:06.143 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 10 monclient: shutdown 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 -- 172.21.15.17:0/3061075272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 msgr2=0x7f991c13f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 --2- 172.21.15.17:0/3061075272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 0x7f991c13f260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9914009e80 tx=0x7f991401c8a0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 -- 172.21.15.17:0/3061075272 shutdown_connections 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 --2- 172.21.15.17:0/3061075272 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a59d0 0x7f991c13f260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 --2- 172.21.15.17:0/3061075272 >> v2:172.21.15.119:3300/0 conn(0x7f991c0a5580 0x7f991c13cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 -- 172.21.15.17:0/3061075272 >> 172.21.15.17:0/3061075272 conn(0x7f991c0aabb0 msgr2=0x7f991c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 -- 172.21.15.17:0/3061075272 shutdown_connections 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 -- 172.21.15.17:0/3061075272 wait complete. 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 10 monclient: build_initial_monmap 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 10 monclient: monmap: 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:06.145 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 Processor -- start 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 1 -- start start 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.127+0000 7f99214cb640 10 monclient: init 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0a4a90 0x7f991c139b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): picked mon.noname-b con 0x7f991c0a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 0x7f991c13a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): picked mon.noname-c con 0x7f991c0a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0a59d0 0x7f991c13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): picked mon.noname-a con 0x7f991c0a59d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1421d0 con 0x7f991c0a59d0 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f991c06f290 con 0x7f991c0a4a90 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f991c142350 con 0x7f991c0a5580 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): authenticate will time out at 1849.528120s 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 0x7f991c13a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0a59d0 0x7f991c13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 0x7f991c13a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:48088/0 (socket says 172.21.15.17:48088) 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 -- 172.21.15.17:0/2239987632 learned_addr learned my addr 172.21.15.17:0/2239987632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): get_auth_request con 0x7f991c0a5580 auth_method 0 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): get_auth_request con 0x7f991c0a59d0 auth_method 0 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:06.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0a4a90 0x7f991c139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): get_auth_request con 0x7f991c0a4a90 auth_method 0 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient(hunting): handle_auth_done global_id 164276 payload 824 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0a4a90 msgr2=0x7f991c139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0a4a90 0x7f991c139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0a59d0 msgr2=0x7f991c13a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0a59d0 0x7f991c13a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:32:06.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 monclient: found mon.noname-c 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c13b550 con 0x7f991c0a5580 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991a7fc640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 0x7f991c13a050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f990c00e930 tx=0x7f990c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c017070 con 0x7f991c0a5580 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: dump: 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.150 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: _finish_auth 0 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: _check_auth_tickets 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c00ca00 con 0x7f991c0a5580 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 5 monclient: authenticate success, global_id 164276 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient: _renew_subs 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:06.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c13d8b0 con 0x7f991c0a5580 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c00cdb0 con 0x7f991c0a5580 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: dump: 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f9918ff9640 4 set_mon_vals no callback set 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient: _renew_subs 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c13c280 con 0x7f991c0a5580 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 0x7f991c134820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0a1280 con 0x7f991c13e030 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f990c01a070 con 0x7f991c0a5580 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 0x7f991c134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient: get_auth_request con 0x7f991c13e030 auth_method 0 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98fc03b9c0 0x7f98fc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f990c050730 con 0x7f991c0a5580 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98fc03b9c0 0x7f98fc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 10 monclient(hunting): handle_auth_done global_id 154396 payload 824 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f991c0a4e70 con 0x7f991c13e030 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 10 monclient: get_auth_request con 0x7f98fc03b9c0 auth_method 0 2024-09-15T21:32:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991affd640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 0x7f991c134820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f990400e4b0 tx=0x7f990400e970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040052e0 con 0x7f991c13e030 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f99137fe640 10 monclient: dump: 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:06.155 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.131+0000 7f991b7fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98fc03b9c0 0x7f98fc03de80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f99140ae000 tx=0x7f99140a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9904018070 con 0x7f991c13e030 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 msgr2=0x7f991c134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 0x7f991c134820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f990400e4b0 tx=0x7f990400e970 comp rx=0 tx=0).stop 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 msgr2=0x7f991c134820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.135+0000 7f99137fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c13e030 0x7f991c134820 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99214cb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99214cb640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 0x7f991c135170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99214cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f991c1356b0 con 0x7f991c134d60 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 0x7f991c135170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient: get_auth_request con 0x7f991c134d60 auth_method 0 2024-09-15T21:32:06.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 10 monclient(hunting): handle_auth_done global_id 154399 payload 824 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 1 -- 172.21.15.17:0/2239987632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f991c13d3b0 con 0x7f991c134d60 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f991affd640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 0x7f991c135170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9904017000 tx=0x7f9904001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904003a10 con 0x7f991c134d60 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.139+0000 7f99137fe640 10 monclient: dump: 2024-09-15T21:32:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7f9904003a10 con 0x7f991c134d60 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 msgr2=0x7f991c135170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 0x7f991c135170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9904017000 tx=0x7f9904001130 comp rx=0 tx=0).stop 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 msgr2=0x7f991c135170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99137fe640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c134d60 0x7f991c135170 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9904017000 tx=0x7f9904001130 comp rx=0 tx=0).stop 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 140, 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 130677, 2024-09-15T21:32:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98fc03b9c0 msgr2=0x7f98fc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f98fc03b9c0 0x7f98fc03de80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f99140ae000 tx=0x7f99140a7070 comp rx=0 tx=0).stop 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 10 monclient: shutdown 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 msgr2=0x7f991c13a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 0x7f991c13a050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f990c00e930 tx=0x7f990c00edf0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f991b7fe640 1 -- 172.21.15.17:0/2239987632 reap_dead start 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 shutdown_connections 2024-09-15T21:32:06.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 --2- 172.21.15.17:0/2239987632 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f991c0a5580 0x7f991c13a050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:06.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 >> 172.21.15.17:0/2239987632 conn(0x7f991c0aabb0 msgr2=0x7f991c137a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:06.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 shutdown_connections 2024-09-15T21:32:06.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:06.143+0000 7f99214cb640 1 -- 172.21.15.17:0/2239987632 wait complete. 2024-09-15T21:32:06.177 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:32:06.178 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:32:06.178 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-15T21:32:06.178 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:32:06.178 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:32:06.178 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:32:06.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:32:06.205 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:32:06.206 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-15T21:32:06.206 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-15T21:32:06.206 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:32:26.207 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-15T21:32:26.208 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-15T21:32:26.208 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:32:26.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:26.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient: get_monmap_and_config 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient: build_initial_monmap 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient: monmap: 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:26.343909+0000 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:26.343909+0000 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:26.345 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:26.345 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:26.345 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:26.345 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:26.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 Processor -- start 2024-09-15T21:32:26.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 -- start start 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient: init 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): picked mon.noname-b con 0x7f9d5c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 0x7f9d5c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): picked mon.noname-c con 0x7f9d5c1381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9d5c0a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0a1d00 con 0x7f9d5c0a5040 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c0a1fe0 con 0x7f9d5c1381e0 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5c0a22c0 con 0x7f9d5c1378c0 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.338+0000 7f9d60ba3640 10 monclient(hunting): authenticate will time out at 1869.739988s 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5ad9d640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f9d5c0a5040 legacy=0x7f9d5c0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d51d9b640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 msgr2=0x7f9d5c0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57266/0 (socket says 172.21.15.17:57266) 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 learned_addr learned my addr 172.21.15.17:0/4126696937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d5c1378c0 auth_method 0 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d51d9b640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 0x7f9d5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d51d9b640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 msgr2=0x7f9d5c0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.119:6789/0 conn(0x7f9d5c0a5040 legacy=0x7f9d5c0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d51d9b640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 0x7f9d5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d5c1378c0 auth_method 0 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.546+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:32:26.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d51d9b640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 msgr2=0x7f9d5c0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:26.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.119:6789/0 conn(0x7f9d5c0a5040 legacy=0x7f9d5c0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d51d9b640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 0x7f9d5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d5c1378c0 auth_method 0 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:26.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:26.950+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:32:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d5c1378c0 auth_method 0 2024-09-15T21:32:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.119:6789/0 conn(0x7f9d5c0a5040 legacy=0x7f9d5c0a54f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:32:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d51d9b640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 msgr2=0x7f9d5c0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d51d9b640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 0x7f9d5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:32:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:27.754+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.338+0000 7f9d58d99640 10 monclient(hunting): tick 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 monclient(hunting): continuing hunt 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 msgr2=0x7f9d5c137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d5c1378c0 0x7f9d5c137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 msgr2=0x7f9d5c0a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d5c1381e0 0x7f9d5c0a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.119:6789/0 conn(0x7f9d5c0a5040 legacy=0x7f9d5c0a54f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient: _add_conns ranks=[3,0] 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient(hunting): picked mon.noname-a con 0x7f9d3c0036b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d3c006220 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0a5c00 con 0x7f9d3c006220 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c0a1fe0 con 0x7f9d3c0036b0 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d58d99640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d3c0036b0 auth_method 0 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d3c0036b0 auth_method 0 2024-09-15T21:32:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.546+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:32:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d3c0036b0 auth_method 0 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:29.950+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.270+0000 7f9d51d9b640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.17:6789/0 conn(0x7f9d3c006220 legacy=0x7f9d3c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.270+0000 7f9d51d9b640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.17:6789/0 conn(0x7f9d3c006220 legacy=0x7f9d3c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.270+0000 7f9d51d9b640 1 --1- 172.21.15.17:0/4126696937 >> v1:172.21.15.17:6789/0 conn(0x7f9d3c006220 0x7f9d3c0086e0 :-1 s=OPENED pgs=63 cs=1 l=1).handle_message read tag failed 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.270+0000 7f9d51d9b640 1 --1- 172.21.15.17:0/4126696937 >> v1:172.21.15.17:6789/0 conn(0x7f9d3c006220 0x7f9d3c0086e0 :-1 s=OPENED pgs=63 cs=1 l=1).fault on lossy channel, failing 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.270+0000 7f9d59d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.17:6789/0 2024-09-15T21:32:30.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:30.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d3c0036b0 auth_method 0 2024-09-15T21:32:30.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:30.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:30.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:32:30.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:32:30.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:32:30.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:32:30.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:30.754+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): tick 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 monclient(hunting): continuing hunt 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 msgr2=0x7f9d3c005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c0036b0 0x7f9d3c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.17:6789/0 conn(0x7f9d3c006220 legacy=0x7f9d3c0086e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/4126696937 reap_dead start 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c009fd0 0x7f9d3c005aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): picked mon.noname-a con 0x7f9d3c009fd0 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d3c00a480 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d3c00a9d0 0x7f9d3c00d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): picked mon.noname-c con 0x7f9d3c00a9d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0a1d00 con 0x7f9d3c00a480 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c0a1fe0 con 0x7f9d3c009fd0 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 1 -- 172.21.15.17:0/4126696937 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c0a22c0 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d58d99640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d3c00a9d0 msgr2=0x7f9d3c00d100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5ad9d640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d3c00a9d0 0x7f9d3c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.17:3300/0 conn(0x7f9d3c009fd0 0x7f9d3c005aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request con 0x7f9d3c009fd0 auth_method 0 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient(hunting): handle_auth_done global_id 174168 payload 274 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d3c00a9d0 msgr2=0x7f9d3c00d100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d3c00a9d0 0x7f9d3c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 >> v1:172.21.15.17:6789/0 conn(0x7f9d3c00a480 legacy=0x7f9d3c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:32:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 1 monclient: found mon.noname-a 2024-09-15T21:32:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.342+0000 7f9d5a59c640 1 -- 172.21.15.17:0/4126696937 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c00e710 con 0x7f9d3c009fd0 2024-09-15T21:32:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c009fd0 0x7f9d3c005aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005fe0 tx=0x7f9d4c01de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f21a3ceb68daf04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 1 -- 172.21.15.17:0/4126696937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00f040 con 0x7f9d3c009fd0 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 10 monclient: dump: 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 10 monclient: _finish_auth 0 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 1 -- 172.21.15.17:0/4126696937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0a6c50 con 0x7f9d3c009fd0 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d60ba3640 5 monclient: authenticate success, global_id 174168 2024-09-15T21:32:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d60ba3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.346+0000 7f9d59d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 4 set_mon_vals no callback set 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d59d9b640 1 -- 172.21.15.17:0/4126696937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c004030 con 0x7f9d3c009fd0 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d59d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d59d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d59d9b640 10 monclient: dump: 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.357 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 10 monclient: shutdown 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 -- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c009fd0 msgr2=0x7f9d3c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c009fd0 0x7f9d3c005aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005fe0 tx=0x7f9d4c01de90 comp rx=0 tx=0).stop 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 -- 172.21.15.17:0/4126696937 shutdown_connections 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/4126696937 >> v2:172.21.15.119:3300/0 conn(0x7f9d3c00a9d0 0x7f9d3c00d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/4126696937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c009fd0 0x7f9d3c005aa0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 -- 172.21.15.17:0/4126696937 >> 172.21.15.17:0/4126696937 conn(0x7f9d5c0aac40 msgr2=0x7f9d5c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 -- 172.21.15.17:0/4126696937 shutdown_connections 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 -- 172.21.15.17:0/4126696937 wait complete. 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 10 monclient: build_initial_monmap 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 10 monclient: monmap: 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 Processor -- start 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 1 -- start start 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.350+0000 7f9d60ba3640 10 monclient: init 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d3c009fd0 0x7f9d5c124300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): picked mon.noname-b con 0x7f9d3c009fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 0x7f9d5c121d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): picked mon.noname-a con 0x7f9d3c00a480 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 0x7f9d5c122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): picked mon.noname-c con 0x7f9d3c00a9d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c06f340 con 0x7f9d3c00a480 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c06f1c0 con 0x7f9d3c009fd0 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5c0a54f0 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient(hunting): authenticate will time out at 1875.752643s 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 0x7f9d5c122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5a59c640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d3c009fd0 msgr2=0x7f9d5c124300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 0x7f9d5c121d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d3c009fd0 0x7f9d5c124300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 0x7f9d5c122260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53954/0 (socket says 172.21.15.17:53954) 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/3470071309 learned_addr learned my addr 172.21.15.17:0/3470071309 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d3c00a9d0 auth_method 0 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 0x7f9d5c121d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46204/0 (socket says 172.21.15.17:46204) 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): get_auth_request con 0x7f9d3c00a480 auth_method 0 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient(hunting): handle_auth_done global_id 164279 payload 824 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d3c009fd0 msgr2=0x7f9d5c124300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d3c009fd0 0x7f9d5c124300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 msgr2=0x7f9d5c121d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 0x7f9d5c121d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:32:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 monclient: found mon.noname-c 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 -- 172.21.15.17:0/3470071309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c125b00 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d51d9b640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 0x7f9d5c121d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5ad9d640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 0x7f9d5c122260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d54003cc0 tx=0x7f9d5400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400ce60 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: dump: 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: _finish_auth 0 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: _check_auth_tickets 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d54010030 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54016650 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 5 monclient: authenticate success, global_id 164279 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _renew_subs 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5c0a12f0 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d5959a640 4 set_mon_vals no callback set 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 10 monclient: dump: 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _renew_subs 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5c124c90 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c0a58d0 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9d540153e0 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d3c03d660 0x7f9d3c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.354+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9d540506e0 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.358+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d3c03d660 0x7f9d3c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.358+0000 7f9d5a59c640 10 monclient: get_auth_request con 0x7f9d3c03d660 auth_method 0 2024-09-15T21:32:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.358+0000 7f9d5a59c640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d3c03d660 0x7f9d3c03fb20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0af000 tx=0x7f9d4c0a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.362+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9d54008070 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.362+0000 7f9d537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.362+0000 7f9d537fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d60ba3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:32:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d60ba3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:32:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d60ba3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9d5c069270 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d537fe640 1 -- 172.21.15.17:0/3470071309 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f9d5401e350 con 0x7f9d3c00a9d0 2024-09-15T21:32:32.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d537fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:32:32.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.598+0000 7f9d537fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:32.604 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":142,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d3c03d660 msgr2=0x7f9d3c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d3c03d660 0x7f9d3c03fb20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0af000 tx=0x7f9d4c0a8070 comp rx=0 tx=0).stop 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 10 monclient: shutdown 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 msgr2=0x7f9d5c122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 0x7f9d5c122260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d54003cc0 tx=0x7f9d5400bea0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 shutdown_connections 2024-09-15T21:32:32.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d3c03d660 0x7f9d3c03fb20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9d3c00a9d0 0x7f9d5c122260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c00a480 0x7f9d5c121d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 --2- 172.21.15.17:0/3470071309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d3c009fd0 0x7f9d5c124300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 >> 172.21.15.17:0/3470071309 conn(0x7f9d5c0aac40 msgr2=0x7f9d5c0a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:32.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 shutdown_connections 2024-09-15T21:32:32.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.602+0000 7f9d60ba3640 1 -- 172.21.15.17:0/3470071309 wait complete. 2024-09-15T21:32:32.622 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:32:32.622 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:32:32.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient: get_monmap_and_config 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient: build_initial_monmap 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient: monmap: 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:32.735308+0000 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:32.735308+0000 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 Processor -- start 2024-09-15T21:32:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 -- start start 2024-09-15T21:32:32.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient: init 2024-09-15T21:32:32.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:32.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbcd40a2990 addr v1:172.21.15.119:6789/0 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a3480 0x7fbcd409cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): picked mon.noname-b con 0x7fbcd40a3480 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 0x7fbcd413f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): picked mon.noname-a con 0x7fbcd413cc50 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd409c740 con 0x7fbcd40a2990 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd4074870 con 0x7fbcd413cc50 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd406d040 con 0x7fbcd40a3480 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcdc3b8640 10 monclient(hunting): authenticate will time out at 1876.131510s 2024-09-15T21:32:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 0x7fbcd413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a3480 0x7fbcd409cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda154640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fbcd40a2990 legacy=0x7fbcd40a2d70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 0x7fbcd413f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46206/0 (socket says 172.21.15.17:46206) 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 1 -- 172.21.15.17:0/3958238248 learned_addr learned my addr 172.21.15.17:0/3958238248 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): get_auth_request con 0x7fbcd413cc50 auth_method 0 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): get_auth_request con 0x7fbcd40a3480 auth_method 0 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.730+0000 7fbcda955640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 10 monclient(hunting): handle_auth_done global_id 164282 payload 274 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 1 -- 172.21.15.17:0/3958238248 >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 msgr2=0x7fbcd413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3958238248 >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 0x7fbcd413f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 1 -- 172.21.15.17:0/3958238248 >> v1:172.21.15.119:6789/0 conn(0x7fbcd40a2990 legacy=0x7fbcd40a2d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 10 monclient: _finish_hunting 0 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 1 monclient: found mon.noname-b 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 1 -- 172.21.15.17:0/3958238248 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd4074a40 con 0x7fbcd40a3480 2024-09-15T21:32:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcda955640 1 --2- 172.21.15.17:0/3958238248 >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 0x7fbcd413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3958238248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a3480 0x7fbcd409cc40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40097c0 tx=0x7fbcc401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea8ccd4059bf18df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 1 -- 172.21.15.17:0/3958238248 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40b0070 con 0x7fbcd40a3480 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: dump: 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: _finish_auth 0 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: _check_auth_tickets 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 1 -- 172.21.15.17:0/3958238248 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc40ab600 con 0x7fbcd40a3480 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 5 monclient: authenticate success, global_id 164282 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 1 -- 172.21.15.17:0/3958238248 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40ab9d0 con 0x7fbcd40a3480 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 4 set_mon_vals no callback set 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcd9152640 10 monclient: dump: 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 10 monclient: shutdown 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3958238248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a3480 msgr2=0x7fbcd409cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3958238248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a3480 0x7fbcd409cc40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40097c0 tx=0x7fbcc401c360 comp rx=0 tx=0).stop 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3958238248 shutdown_connections 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3958238248 >> v2:172.21.15.17:3300/0 conn(0x7fbcd413cc50 0x7fbcd413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3958238248 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a3480 0x7fbcd409cc40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3958238248 >> 172.21.15.17:0/3958238248 conn(0x7fbcd40a7580 msgr2=0x7fbcd40a3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3958238248 shutdown_connections 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3958238248 wait complete. 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 10 monclient: build_initial_monmap 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.734+0000 7fbcdc3b8640 10 monclient: monmap: 2024-09-15T21:32:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 Processor -- start 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- start start 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: init 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a2990 0x7fbcd41338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): picked mon.noname-c con 0x7fbcd40a2990 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd40a3480 0x7fbcd4133de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): picked mon.noname-b con 0x7fbcd40a3480 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd413cc50 0x7fbcd409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): picked mon.noname-a con 0x7fbcd413cc50 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41421f0 con 0x7fbcd413cc50 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4142070 con 0x7fbcd40a3480 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd4142370 con 0x7fbcd40a2990 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): authenticate will time out at 1876.137206s 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a2990 0x7fbcd41338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd413cc50 0x7fbcd409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd40a3480 msgr2=0x7fbcd4133de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a2990 0x7fbcd41338a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53972/0 (socket says 172.21.15.17:53972) 2024-09-15T21:32:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd413cc50 0x7fbcd409e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46212/0 (socket says 172.21.15.17:46212) 2024-09-15T21:32:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 -- 172.21.15.17:0/3468312120 learned_addr learned my addr 172.21.15.17:0/3468312120 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): get_auth_request con 0x7fbcd40a2990 auth_method 0 2024-09-15T21:32:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd40a3480 0x7fbcd4133de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): get_auth_request con 0x7fbcd413cc50 auth_method 0 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient(hunting): handle_auth_done global_id 164285 payload 824 2024-09-15T21:32:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd40a3480 msgr2=0x7fbcd4133de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd40a3480 0x7fbcd4133de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd413cc50 msgr2=0x7fbcd409e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd413cc50 0x7fbcd409e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient: _finish_hunting 0 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 monclient: found mon.noname-c 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd413d030 con 0x7fbcd40a2990 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd413cc50 0x7fbcd409e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda154640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a2990 0x7fbcd41338a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000b4e0 tx=0x7fbcd000b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0003d80 con 0x7fbcd40a2990 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: dump: 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: _finish_auth 0 2024-09-15T21:32:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd0015070 con 0x7fbcd40a2990 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 5 monclient: authenticate success, global_id 164285 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: _renew_subs 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd411dde0 con 0x7fbcd40a2990 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00052b0 con 0x7fbcd40a2990 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: dump: 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd8951640 4 set_mon_vals no callback set 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: _renew_subs 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd413d580 con 0x7fbcd40a2990 2024-09-15T21:32:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 0x7fbcd409eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd409cc40 con 0x7fbcd409eb10 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbcd0005450 con 0x7fbcd40a2990 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 0x7fbcd409eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient: get_auth_request con 0x7fbcd409eb10 auth_method 0 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcb803b970 0x7fbcb803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 10 monclient(hunting): handle_auth_done global_id 174183 payload 824 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcd40a0a00 con 0x7fbcd409eb10 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fbcd00503c0 con 0x7fbcd40a2990 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 0x7fbcd409eef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40a7fd0 tx=0x7fbcc401aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40bb070 con 0x7fbcd409eb10 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbccaffd640 10 monclient: dump: 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcb803b970 0x7fbcb803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.738+0000 7fbcda955640 10 monclient: get_auth_request con 0x7fbcb803b970 auth_method 0 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbcc4004230 con 0x7fbcd409eb10 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 msgr2=0x7fbcd409eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 0x7fbcd409eef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40a7fd0 tx=0x7fbcc401aba0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 msgr2=0x7fbcd409eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbccaffd640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409eb10 0x7fbcd409eef0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40a7fd0 tx=0x7fbcc401aba0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.742+0000 7fbcda955640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcb803b970 0x7fbcb803de30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbcd411da30 tx=0x7fbccc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 0x7fbcd4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd411db00 con 0x7fbcd409f430 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 0x7fbcd4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient: get_auth_request con 0x7fbcd409f430 auth_method 0 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 10 monclient(hunting): handle_auth_done global_id 174186 payload 824 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 1 -- 172.21.15.17:0/3468312120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbcd409f840 con 0x7fbcd409f430 2024-09-15T21:32:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcd9953640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 0x7fbcd4138c30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbcc401c190 tx=0x7fbcc4019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40b0040 con 0x7fbcd409f430 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 10 monclient: dump: 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4703 (secure 0 0 0) 0x7fbcc40c0c80 con 0x7fbcd409f430 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 msgr2=0x7fbcd4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 0x7fbcd4138c30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbcc401c190 tx=0x7fbcc4019100 comp rx=0 tx=0).stop 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 msgr2=0x7fbcd4138c30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbccaffd640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcd409f430 0x7fbcd4138c30 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbcc401c190 tx=0x7fbcc4019100 comp rx=0 tx=0).stop 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:32.770 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 142, 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 42483, 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:32.771 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:32.772 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:32.773 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:32.775 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:32.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:32.777 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:32.778 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:32.779 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.781 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:32.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcb803b970 msgr2=0x7fbcb803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbcb803b970 0x7fbcb803de30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbcd411da30 tx=0x7fbccc00b040 comp rx=0 tx=0).stop 2024-09-15T21:32:32.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 10 monclient: shutdown 2024-09-15T21:32:32.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:32.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a2990 msgr2=0x7fbcd41338a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 --2- 172.21.15.17:0/3468312120 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbcd40a2990 0x7fbcd41338a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000b4e0 tx=0x7fbcd000b9a0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcda955640 1 -- 172.21.15.17:0/3468312120 reap_dead start 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcda955640 1 -- 172.21.15.17:0/3468312120 reap_dead start 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 shutdown_connections 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 >> 172.21.15.17:0/3468312120 conn(0x7fbcd40a7580 msgr2=0x7fbcd413c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 shutdown_connections 2024-09-15T21:32:32.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.746+0000 7fbcdc3b8640 1 -- 172.21.15.17:0/3468312120 wait complete. 2024-09-15T21:32:32.786 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:32:32.786 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:32:32.787 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.886+0000 7fb941c98640 10 monclient: get_monmap_and_config 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.886+0000 7fb941c98640 10 monclient: build_initial_monmap 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.886+0000 7fb941c98640 10 monclient: monmap: 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:32:32.894034+0000 2024-09-15T21:32:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:32:32.894034+0000 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 Processor -- start 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 -- start start 2024-09-15T21:32:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient: init 2024-09-15T21:32:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb93c1340c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb93c09d6c0 0x7fb93c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): picked mon.noname-a con 0x7fb93c09d6c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c13b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): picked mon.noname-b con 0x7fb93c139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c09e1b0 con 0x7fb93c1340c0 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb93c09d520 con 0x7fb93c09d6c0 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c09e490 con 0x7fb93c139390 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient(hunting): authenticate will time out at 1876.289063s 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb93c1340c0 0x7fb93c1344a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50938/0 (socket says 172.21.15.17:50938) 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93b7fe640 1 -- 172.21.15.17:0/3960463035 learned_addr learned my addr 172.21.15.17:0/3960463035 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c13b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 --2- 172.21.15.17:0/3960463035 >> v2:172.21.15.17:3300/0 conn(0x7fb93c09d6c0 0x7fb93c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): get_auth_request con 0x7fb93c09d6c0 auth_method 0 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): get_auth_request con 0x7fb93c139390 auth_method 0 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 1 -- 172.21.15.17:0/3960463035 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529670732 0 0) 0x7fb93c09e1b0 con 0x7fb93c1340c0 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient(hunting): my global_id is 174189 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 1 -- 172.21.15.17:0/3960463035 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb924005740 con 0x7fb93c1340c0 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient(hunting): handle_auth_done global_id 174192 payload 274 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 -- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 msgr2=0x7fb93c13b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c13b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 -- 172.21.15.17:0/3960463035 >> v1:172.21.15.17:6789/0 conn(0x7fb93c1340c0 legacy=0x7fb93c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient: _finish_hunting 0 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 monclient: found mon.noname-a 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 -- 172.21.15.17:0/3960463035 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c09e770 con 0x7fb93c09d6c0 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93bfff640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c13b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93affd640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c138e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9300097c0 tx=0x7fb93001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29655e78d6a5a617 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 1 -- 172.21.15.17:0/3960463035 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300b0070 con 0x7fb93c09d6c0 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: dump: 2024-09-15T21:32:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 1 -- 172.21.15.17:0/3960463035 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9300ab600 con 0x7fb93c09d6c0 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 5 monclient: authenticate success, global_id 174192 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 10 monclient: get_monmap_and_config success 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 1 -- 172.21.15.17:0/3960463035 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300ab9d0 con 0x7fb93c09d6c0 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb941c98640 4 set_mon_vals no callback set 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.890+0000 7fb93a7fc640 10 monclient: dump: 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: shutdown 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 msgr2=0x7fb93c138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c138e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9300097c0 tx=0x7fb93001c360 comp rx=0 tx=0).stop 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/3960463035 shutdown_connections 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c13b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- 172.21.15.17:0/3960463035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c138e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/3960463035 >> 172.21.15.17:0/3960463035 conn(0x7fb93c0a66a0 msgr2=0x7fb93c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/3960463035 shutdown_connections 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/3960463035 wait complete. 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: build_initial_monmap 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: monmap: 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:32:32.911 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 Processor -- start 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- start start 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: init 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: _reopen_session rank -1 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c1cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): picked mon.noname-a con 0x7fb93c09d6c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c1340c0 0x7fb93c130a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): picked mon.noname-b con 0x7fb93c1340c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): picked mon.noname-c con 0x7fb93c139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c13e060 con 0x7fb93c09d6c0 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c13dee0 con 0x7fb93c1340c0 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c13e1e0 con 0x7fb93c139390 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): _renew_subs 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): authenticate will time out at 1876.293868s 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c1cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c130fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c1340c0 msgr2=0x7fb93c130a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c1340c0 0x7fb93c130a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c1cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46262/0 (socket says 172.21.15.17:46262) 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c130fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:53990/0 (socket says 172.21.15.17:53990) 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 -- 172.21.15.17:0/2772477868 learned_addr learned my addr 172.21.15.17:0/2772477868 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:32:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): get_auth_request con 0x7fb93c09d6c0 auth_method 0 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): get_auth_request con 0x7fb93c139390 auth_method 0 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient(hunting): handle_auth_done global_id 174195 payload 824 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 msgr2=0x7fb93c130fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c139390 0x7fb93c130fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c1340c0 msgr2=0x7fb93c130a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c1340c0 0x7fb93c130a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 monclient: found mon.noname-a 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c138930 con 0x7fb93c09d6c0 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93b7fe640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c1cf9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00cbf0 tx=0x7fb92c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00aca0 con 0x7fb93c09d6c0 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: dump: 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.916 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: _finish_auth 0 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: _check_auth_tickets 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c00ae40 con 0x7fb93c09d6c0 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 5 monclient: authenticate success, global_id 174195 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: _renew_subs 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c11db90 con 0x7fb93c09d6c0 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c012600 con 0x7fb93c09d6c0 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 10 monclient: dump: 2024-09-15T21:32:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb9397fa640 4 set_mon_vals no callback set 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: _renew_subs 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c11cc30 con 0x7fb93c09d6c0 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 0x7fb93c137250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c1cff30 con 0x7fb93c136e70 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 0x7fb93c137250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient: get_auth_request con 0x7fb93c136e70 auth_method 0 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb92c0127a0 con 0x7fb93c09d6c0 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 10 monclient(hunting): handle_auth_done global_id 164294 payload 824 2024-09-15T21:32:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb93affd640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb93c1344a0 con 0x7fb93c136e70 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.894+0000 7fb923fff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb92803b970 0x7fb92803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb92c050c10 con 0x7fb93c09d6c0 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb93affd640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 0x7fb93c137250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb93c09d0a0 tx=0x7fb93001aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300b0050 con 0x7fb93c136e70 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 10 monclient: dump: 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.920 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb93bfff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb92803b970 0x7fb92803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb93bfff640 10 monclient: get_auth_request con 0x7fb92803b970 auth_method 0 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb9300b1040 con 0x7fb93c136e70 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 msgr2=0x7fb93c137250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 0x7fb93c137250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb93c09d0a0 tx=0x7fb93001aba0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 msgr2=0x7fb93c137250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb923fff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c136e70 0x7fb93c137250 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb93c09d0a0 tx=0x7fb93001aba0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.898+0000 7fb93bfff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb92803b970 0x7fb92803de30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb93c134bd0 tx=0x7fb934009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 0x7fb93c131960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 10 monclient(hunting): start opening mon connection 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0a59a0 con 0x7fb93c137790 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 0x7fb93c131960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient: get_auth_request con 0x7fb93c137790 auth_method 0 2024-09-15T21:32:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 10 monclient(hunting): handle_auth_done global_id 164297 payload 824 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 1 -- 172.21.15.17:0/2772477868 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb93c06d020 con 0x7fb93c137790 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93affd640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 0x7fb93c131960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb93001aa10 tx=0x7fb930019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300b5ac0 con 0x7fb93c137790 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 10 monclient: dump: 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:32:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7fb9300aba20 con 0x7fb93c137790 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 msgr2=0x7fb93c131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 0x7fb93c131960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb93001aa10 tx=0x7fb930019100 comp rx=0 tx=0).stop 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 msgr2=0x7fb93c131960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb923fff640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb93c137790 0x7fb93c131960 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb93001aa10 tx=0x7fb930019100 comp rx=0 tx=0).stop 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 142, 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 101486, 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:32:32.927 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:32.928 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:32:32.930 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:32:32.931 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.933 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:32:32.935 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:32:32.937 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:32:32.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:32:32.939 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:32:32.940 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:32:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb92803b970 msgr2=0x7fb92803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb92803b970 0x7fb92803de30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb93c134bd0 tx=0x7fb934009190 comp rx=0 tx=0).stop 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 10 monclient: shutdown 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 msgr2=0x7fb93c1cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c1cf9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00cbf0 tx=0x7fb92c00a060 comp rx=0 tx=0).stop 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb93bfff640 1 -- 172.21.15.17:0/2772477868 reap_dead start 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 shutdown_connections 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 --2- 172.21.15.17:0/2772477868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb93c09d6c0 0x7fb93c1cf9f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 >> 172.21.15.17:0/2772477868 conn(0x7fb93c0a66a0 msgr2=0x7fb93c073b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 shutdown_connections 2024-09-15T21:32:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:32:32.902+0000 7fb941c98640 1 -- 172.21.15.17:0/2772477868 wait complete. 2024-09-15T21:32:32.943 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:32:32.943 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:32:52.943 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:32:52.944 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:32:52.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:32:52.947 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:32:52.947 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-15T21:32:52.947 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-15T21:32:52.947 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:33:12.949 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-15T21:33:12.950 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-15T21:33:12.950 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:12.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:13.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient: get_monmap_and_config 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient: monmap: 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:13.099782+0000 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:13.099782+0000 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:13.101 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:13.101 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:13.101 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 1 Processor -- start 2024-09-15T21:33:13.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 1 -- start start 2024-09-15T21:33:13.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient: init 2024-09-15T21:33:13.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa2f40a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2f409ffc0 0x7fa2f413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient(hunting): picked mon.noname-c con 0x7fa2f409ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa2f40a1380 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f40a3130 con 0x7fa2f40a1380 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f4073df0 con 0x7fa2f40a0890 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f40a1bf0 con 0x7fa2f409ffc0 2024-09-15T21:33:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f9c07640 10 monclient(hunting): authenticate will time out at 1916.495624s 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa2f40a1380 0x7fa2f413d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58140/0 (socket says 172.21.15.17:58140) 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.094+0000 7fa2f3fff640 1 -- 172.21.15.17:0/2713191148 learned_addr learned my addr 172.21.15.17:0/2713191148 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 --2- 172.21.15.17:0/2713191148 >> v2:172.21.15.119:3300/0 conn(0x7fa2f409ffc0 0x7fa2f413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): get_auth_request con 0x7fa2f409ffc0 auth_method 0 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269670030 0 0) 0x7fa2f40a3130 con 0x7fa2f40a1380 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): my global_id is 174198 2024-09-15T21:33:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2e0003530 con 0x7fa2f40a1380 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1632156362 0 0) 0x7fa2e0003530 con 0x7fa2f40a1380 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 >> v2:172.21.15.119:3300/0 conn(0x7fa2f409ffc0 msgr2=0x7fa2f413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 --2- 172.21.15.17:0/2713191148 >> v2:172.21.15.119:3300/0 conn(0x7fa2f409ffc0 0x7fa2f413aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 >> v1:172.21.15.119:6789/0 conn(0x7fa2f40a0890 legacy=0x7fa2f40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f409ccd0 con 0x7fa2f40a1380 2024-09-15T21:33:13.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa2ec001a80 con 0x7fa2f40a1380 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: dump: 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 >> v1:172.21.15.17:6789/0 conn(0x7fa2f40a1380 legacy=0x7fa2f413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2e0003a90 0x7fa2e0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): picked mon.b con 0x7fa2e0003a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): picked mon.a con 0x7fa2e0004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2e00053d0 0x7fa2e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): picked mon.c con 0x7fa2e00053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f40a1bf0 con 0x7fa2e0004810 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000c270 con 0x7fa2e0003a90 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e000c470 con 0x7fa2e00053d0 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): get_auth_request con 0x7fa2e0004810 auth_method 0 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2e00053d0 0x7fa2e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2e0003a90 0x7fa2e0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): get_auth_request con 0x7fa2e0003a90 auth_method 0 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): get_auth_request con 0x7fa2e00053d0 auth_method 0 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient(hunting): handle_auth_done global_id 174198 payload 293 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 -- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2e00053d0 msgr2=0x7fa2e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2e00053d0 0x7fa2e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 -- 172.21.15.17:0/2713191148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2e0003a90 msgr2=0x7fa2e0003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2e0003a90 0x7fa2e0003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 monclient: found mon.a 2024-09-15T21:33:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 -- 172.21.15.17:0/2713191148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e0003530 con 0x7fa2e0004810 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f3fff640 1 -- 172.21.15.17:0/2713191148 reap_dead start 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f9c07640 5 monclient: authenticate success, global_id 174198 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f9c07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2ebfff640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2e0004c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009ae0 tx=0x7fa2dc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d304989e597a1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0096b0 con 0x7fa2e0004810 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: dump: 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2dc0a6430 con 0x7fa2e0004810 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f9c07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f9c07640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 1 -- 172.21.15.17:0/2713191148 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0a6820 con 0x7fa2e0004810 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f9c07640 4 set_mon_vals no callback set 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.098+0000 7fa2f2ffd640 10 monclient: dump: 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.122 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: shutdown 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 msgr2=0x7fa2e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2e0004c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009ae0 tx=0x7fa2dc01ca50 comp rx=0 tx=0).stop 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/2713191148 shutdown_connections 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/2713191148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2e0004c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/2713191148 >> 172.21.15.17:0/2713191148 conn(0x7fa2f40a66a0 msgr2=0x7fa2f40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/2713191148 shutdown_connections 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/2713191148 wait complete. 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: monmap: 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 Processor -- start 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- start start 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: init 2024-09-15T21:33:13.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2f4123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): picked mon.noname-a con 0x7fa2e0004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f4120d80 0x7fa2f41211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): picked mon.noname-b con 0x7fa2f4120d80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 0x7fa2f41cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): picked mon.noname-c con 0x7fa2f4121720 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4136210 con 0x7fa2e0004810 2024-09-15T21:33:13.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4136090 con 0x7fa2f4120d80 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f4136390 con 0x7fa2f4121720 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient(hunting): authenticate will time out at 1916.501699s 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 0x7fa2f41cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 0x7fa2f41cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37068/0 (socket says 172.21.15.17:37068) 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 -- 172.21.15.17:0/1072890123 learned_addr learned my addr 172.21.15.17:0/1072890123 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): get_auth_request con 0x7fa2f4121720 auth_method 0 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f37fe640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2f4123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f37fe640 10 monclient(hunting): get_auth_request con 0x7fa2e0004810 auth_method 0 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2ebfff640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f4120d80 0x7fa2f41211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient(hunting): handle_auth_done global_id 164300 payload 824 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 -- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f4120d80 msgr2=0x7fa2f41211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f4120d80 0x7fa2f41211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 -- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 msgr2=0x7fa2f4123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2f4123360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 monclient: found mon.noname-c 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 -- 172.21.15.17:0/1072890123 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f4124880 con 0x7fa2f4121720 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f3fff640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 0x7fa2f41cdad0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000f90 tx=0x7fa2ec007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec017020 con 0x7fa2f4121720 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:13.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: dump: 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 5 monclient: authenticate success, global_id 164300 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec00ab40 con 0x7fa2f4121720 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _renew_subs 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f40724d0 con 0x7fa2f4121720 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0133f0 con 0x7fa2f4121720 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:13.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 10 monclient: dump: 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f27fc640 4 set_mon_vals no callback set 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _renew_subs 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:13.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f4121b80 con 0x7fa2f4121720 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f4138ab0 con 0x7fa2f4121720 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa2ec01a020 con 0x7fa2f4121720 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.102+0000 7fa2f0ff9640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2d403b9f0 0x7fa2d403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.106+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa2ec050d70 con 0x7fa2f4121720 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.106+0000 7fa2f37fe640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2d403b9f0 0x7fa2d403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.106+0000 7fa2f37fe640 10 monclient: get_auth_request con 0x7fa2d403b9f0 auth_method 0 2024-09-15T21:33:13.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.106+0000 7fa2f37fe640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2d403b9f0 0x7fa2d403deb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c10 tx=0x7fa2e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.110+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa2ec021020 con 0x7fa2f4121720 2024-09-15T21:33:13.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.110+0000 7fa2f0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.110+0000 7fa2f0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f9c07640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f9c07640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f9c07640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa2f40008e0 con 0x7fa2f4121720 2024-09-15T21:33:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f0ff9640 1 -- 172.21.15.17:0/1072890123 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fa2ec01b3b0 con 0x7fa2f4121720 2024-09-15T21:33:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.342+0000 7fa2f0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2d403b9f0 msgr2=0x7fa2d403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2d403b9f0 0x7fa2d403deb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c10 tx=0x7fa2e4009040 comp rx=0 tx=0).stop 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 10 monclient: shutdown 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 msgr2=0x7fa2f41cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 0x7fa2f41cdad0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000f90 tx=0x7fa2ec007a90 comp rx=0 tx=0).stop 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 shutdown_connections 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa2d403b9f0 0x7fa2d403deb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa2f4121720 0x7fa2f41cdad0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f4120d80 0x7fa2f41211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 --2- 172.21.15.17:0/1072890123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e0004810 0x7fa2f4123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 >> 172.21.15.17:0/1072890123 conn(0x7fa2f40a66a0 msgr2=0x7fa2f406fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 shutdown_connections 2024-09-15T21:33:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.346+0000 7fa2f9c07640 1 -- 172.21.15.17:0/1072890123 wait complete. 2024-09-15T21:33:13.368 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:33:13.368 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:33:13.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.486+0000 7f403148a640 10 monclient: get_monmap_and_config 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.486+0000 7f403148a640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient: monmap: 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:13.494187+0000 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:13.494187+0000 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:13.495 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 Processor -- start 2024-09-15T21:33:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 -- start start 2024-09-15T21:33:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient: init 2024-09-15T21:33:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f402c0a0890 0x7f402c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): picked mon.noname-c con 0x7f402c0a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f402c09ffc0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f402c0a11b0 0x7f402c135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): picked mon.noname-a con 0x7f402c0a11b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c0a3130 con 0x7f402c09ffc0 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f402c073df0 con 0x7f402c0a11b0 2024-09-15T21:33:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0a1bf0 con 0x7f402c0a0890 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f403148a640 10 monclient(hunting): authenticate will time out at 1916.889623s 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f402c0a11b0 0x7f402c135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402a7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f402c09ffc0 0x7f402c135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58144/0 (socket says 172.21.15.17:58144) 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f402c0a11b0 0x7f402c135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55640/0 (socket says 172.21.15.17:55640) 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402a7fc640 1 -- 172.21.15.17:0/463128294 learned_addr learned my addr 172.21.15.17:0/463128294 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): get_auth_request con 0x7f402c0a11b0 auth_method 0 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402affd640 1 --2- 172.21.15.17:0/463128294 >> v2:172.21.15.119:3300/0 conn(0x7f402c0a0890 0x7f402c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402affd640 10 monclient(hunting): get_auth_request con 0x7f402c0a0890 auth_method 0 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient(hunting): handle_auth_done global_id 174204 payload 274 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 -- 172.21.15.17:0/463128294 >> v2:172.21.15.119:3300/0 conn(0x7f402c0a0890 msgr2=0x7f402c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 --2- 172.21.15.17:0/463128294 >> v2:172.21.15.119:3300/0 conn(0x7f402c0a0890 0x7f402c0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 -- 172.21.15.17:0/463128294 >> v1:172.21.15.17:6789/0 conn(0x7f402c09ffc0 legacy=0x7f402c135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 monclient: found mon.noname-a 2024-09-15T21:33:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f4029ffb640 1 -- 172.21.15.17:0/463128294 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 589336529 0 0) 0x7f402c0a3130 con 0x7f402c09ffc0 2024-09-15T21:33:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f402b7fe640 1 -- 172.21.15.17:0/463128294 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0a15e0 con 0x7f402c0a11b0 2024-09-15T21:33:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.490+0000 7f4029ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402b7fe640 1 --2- 172.21.15.17:0/463128294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0a11b0 0x7f402c135d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40240097c0 tx=0x7f402401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b5a3f41b68fe70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 1 -- 172.21.15.17:0/463128294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40240b0070 con 0x7f402c0a11b0 2024-09-15T21:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: dump: 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 1 -- 172.21.15.17:0/463128294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40240ab600 con 0x7f402c0a11b0 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 1 -- 172.21.15.17:0/463128294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40240ab9d0 con 0x7f402c0a11b0 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 5 monclient: authenticate success, global_id 174204 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 4 set_mon_vals no callback set 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f4029ffb640 10 monclient: dump: 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: shutdown 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/463128294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0a11b0 msgr2=0x7f402c135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 --2- 172.21.15.17:0/463128294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0a11b0 0x7f402c135d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40240097c0 tx=0x7f402401c320 comp rx=0 tx=0).stop 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/463128294 shutdown_connections 2024-09-15T21:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 --2- 172.21.15.17:0/463128294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0a11b0 0x7f402c135d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 --2- 172.21.15.17:0/463128294 >> v2:172.21.15.119:3300/0 conn(0x7f402c0a0890 0x7f402c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/463128294 >> 172.21.15.17:0/463128294 conn(0x7f402c0a66a0 msgr2=0x7f402c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/463128294 shutdown_connections 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/463128294 wait complete. 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: monmap: 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 Processor -- start 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- start start 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: init 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c09ffc0 0x7f402c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): picked mon.noname-a con 0x7f402c09ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f402c0a0890 0x7f402c136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): picked mon.noname-c con 0x7f402c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f402c0a11b0 0x7f402c137520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): picked mon.noname-b con 0x7f402c0a11b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f402c140510 con 0x7f402c09ffc0 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1364d0 con 0x7f402c0a11b0 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f402c140690 con 0x7f402c0a0890 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient(hunting): authenticate will time out at 1916.894765s 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c09ffc0 0x7f402c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c09ffc0 0x7f402c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55644/0 (socket says 172.21.15.17:55644) 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 -- 172.21.15.17:0/631994636 learned_addr learned my addr 172.21.15.17:0/631994636 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): get_auth_request con 0x7f402c09ffc0 auth_method 0 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402a7fc640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f402c0a0890 0x7f402c136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402a7fc640 10 monclient(hunting): get_auth_request con 0x7f402c0a0890 auth_method 0 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402b7fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f402c0a11b0 0x7f402c137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient(hunting): handle_auth_done global_id 174210 payload 824 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f402c0a0890 msgr2=0x7f402c136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f402c0a0890 0x7f402c136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f402c0a11b0 msgr2=0x7f402c137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f402c0a11b0 0x7f402c137520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 monclient: found mon.noname-a 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c139170 con 0x7f402c09ffc0 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402b7fe640 10 monclient: get_auth_request con 0x7f402c0a11b0 auth_method 0 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402b7fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f402c0a11b0 0x7f402c137520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f402affd640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c09ffc0 0x7f402c136aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d830 tx=0x7f401c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00b6c0 con 0x7f402c09ffc0 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: dump: 2024-09-15T21:33:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 5 monclient: authenticate success, global_id 174210 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401c019070 con 0x7f402c09ffc0 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: _renew_subs 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c06f650 con 0x7f402c09ffc0 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00ba60 con 0x7f402c09ffc0 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40237fe640 10 monclient: dump: 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f40297fa640 4 set_mon_vals no callback set 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: _renew_subs 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.494+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c138e90 con 0x7f402c09ffc0 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f403148a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f403148a640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 0x7f402c070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f402c138b10 con 0x7f402c070a90 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 0x7f402c070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient: get_auth_request con 0x7f402c070a90 auth_method 0 2024-09-15T21:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f401c01e460 con 0x7f402c09ffc0 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 10 monclient(hunting): handle_auth_done global_id 174213 payload 824 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f400c03b9c0 0x7f400c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f402c0a0c70 con 0x7f402c070a90 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f401c050d10 con 0x7f402c09ffc0 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402a7fc640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 0x7f402c070e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4014007fd0 tx=0x7f4014001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401400e940 con 0x7f402c070a90 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 10 monclient: dump: 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402b7fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f400c03b9c0 0x7f400c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402b7fe640 10 monclient: get_auth_request con 0x7f400c03b9c0 auth_method 0 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4014018070 con 0x7f402c070a90 2024-09-15T21:33:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 msgr2=0x7f402c070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 0x7f402c070e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4014007fd0 tx=0x7f4014001f10 comp rx=0 tx=0).stop 2024-09-15T21:33:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 msgr2=0x7f402c070e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f40237fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c070a90 0x7f402c070e70 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4014007fd0 tx=0x7f4014001f10 comp rx=0 tx=0).stop 2024-09-15T21:33:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.498+0000 7f402b7fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f400c03b9c0 0x7f400c03de80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f40240ad000 tx=0x7f4024008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f403148a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f403148a640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 0x7f402c1d75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f403148a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f402c13ca10 con 0x7f402c0713b0 2024-09-15T21:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 0x7f402c1d75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient: get_auth_request con 0x7f402c0713b0 auth_method 0 2024-09-15T21:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 10 monclient(hunting): handle_auth_done global_id 174216 payload 824 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.502+0000 7f402a7fc640 1 -- 172.21.15.17:0/631994636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f402c134fa0 con 0x7f402c0713b0 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f402a7fc640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 0x7f402c1d75f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4014005fd0 tx=0x7f4014001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401401dd90 con 0x7f402c0713b0 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 10 monclient: dump: 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.531 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f4014027430 con 0x7f402c0713b0 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 msgr2=0x7f402c1d75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 0x7f402c1d75f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4014005fd0 tx=0x7f4014001260 comp rx=0 tx=0).stop 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 msgr2=0x7f402c1d75f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f40237fe640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c0713b0 0x7f402c1d75f0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4014005fd0 tx=0x7f4014001260 comp rx=0 tx=0).stop 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:13.532 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 146, 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 83243, 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.533 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f400c03b9c0 msgr2=0x7f400c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f400c03b9c0 0x7f400c03de80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f40240ad000 tx=0x7f4024008040 comp rx=0 tx=0).stop 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 10 monclient: shutdown 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c09ffc0 msgr2=0x7f402c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 --2- 172.21.15.17:0/631994636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f402c09ffc0 0x7f402c136aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d830 tx=0x7f401c00dcf0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f402b7fe640 1 -- 172.21.15.17:0/631994636 reap_dead start 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f402b7fe640 1 -- 172.21.15.17:0/631994636 reap_dead start 2024-09-15T21:33:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 shutdown_connections 2024-09-15T21:33:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 >> 172.21.15.17:0/631994636 conn(0x7f402c0a66a0 msgr2=0x7f402c06fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 shutdown_connections 2024-09-15T21:33:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.506+0000 7f403148a640 1 -- 172.21.15.17:0/631994636 wait complete. 2024-09-15T21:33:13.549 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:13.549 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:33:13.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:33:13.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.662+0000 7f5ce5621640 10 monclient: get_monmap_and_config 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.662+0000 7f5ce5621640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.662+0000 7f5ce5621640 10 monclient: monmap: 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:13.670049+0000 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:13.670049+0000 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:13.671 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 Processor -- start 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 -- start start 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient: init 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ce01378c0 0x7f5ce0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): picked mon.noname-a con 0x7f5ce01378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5ce01381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce00a5210 0x7f5ce00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): picked mon.noname-b con 0x7f5ce00a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce00a1d00 con 0x7f5ce01381e0 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce00a1fe0 con 0x7f5ce01378c0 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce00a22c0 con 0x7f5ce00a5210 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5ce5621640 10 monclient(hunting): authenticate will time out at 1917.065586s 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ce01378c0 0x7f5ce0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ce01378c0 0x7f5ce0137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55664/0 (socket says 172.21.15.17:55664) 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 -- 172.21.15.17:0/3732604801 learned_addr learned my addr 172.21.15.17:0/3732604801 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): get_auth_request con 0x7f5ce01378c0 auth_method 0 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 1 --2- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce00a5210 0x7f5ce00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f5ce00a5210 auth_method 0 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_done global_id 174219 payload 274 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 -- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce00a5210 msgr2=0x7f5ce00a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 --2- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce00a5210 0x7f5ce00a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 -- 172.21.15.17:0/3732604801 >> v1:172.21.15.119:6789/0 conn(0x7f5ce01381e0 legacy=0x7f5ce00a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 monclient: found mon.noname-a 2024-09-15T21:33:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.666+0000 7f5cdeffd640 1 -- 172.21.15.17:0/3732604801 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce00a4490 con 0x7f5ce01378c0 2024-09-15T21:33:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 1 --2- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce01378c0 0x7f5ce0137ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00098a0 tx=0x7f5cd001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26ff2a7bd8b9552 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 1 -- 172.21.15.17:0/3732604801 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00b1070 con 0x7f5ce01378c0 2024-09-15T21:33:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: dump: 2024-09-15T21:33:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 1 -- 172.21.15.17:0/3732604801 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd00ac600 con 0x7f5ce01378c0 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 5 monclient: authenticate success, global_id 174219 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 1 -- 172.21.15.17:0/3732604801 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00ac9d0 con 0x7f5ce01378c0 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 4 set_mon_vals no callback set 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cddffb640 10 monclient: dump: 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: shutdown 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce01378c0 msgr2=0x7f5ce0137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 --2- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce01378c0 0x7f5ce0137ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00098a0 tx=0x7f5cd001c400 comp rx=0 tx=0).stop 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- 172.21.15.17:0/3732604801 shutdown_connections 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 --2- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce00a5210 0x7f5ce00a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 --2- 172.21.15.17:0/3732604801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce01378c0 0x7f5ce0137ca0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- 172.21.15.17:0/3732604801 >> 172.21.15.17:0/3732604801 conn(0x7f5ce00aac40 msgr2=0x7f5ce00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- 172.21.15.17:0/3732604801 shutdown_connections 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- 172.21.15.17:0/3732604801 wait complete. 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: monmap: 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 Processor -- start 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- start start 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: init 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce00a5210 0x7f5ce0136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): picked mon.noname-a con 0x7f5ce00a5210 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce01378c0 0x7f5ce0136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): picked mon.noname-b con 0x7f5ce01378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce01381e0 0x7f5ce013cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): picked mon.noname-c con 0x7f5ce01381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce006f340 con 0x7f5ce00a5210 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce006f1c0 con 0x7f5ce01378c0 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce01425c0 con 0x7f5ce01381e0 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5ce5621640 10 monclient(hunting): authenticate will time out at 1917.071763s 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce01381e0 0x7f5ce013cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce00a5210 0x7f5ce0136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce00a5210 0x7f5ce0136900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55668/0 (socket says 172.21.15.17:55668) 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce01381e0 0x7f5ce013cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37086/0 (socket says 172.21.15.17:37086) 2024-09-15T21:33:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 1 -- 172.21.15.17:0/41912258 learned_addr learned my addr 172.21.15.17:0/41912258 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 10 monclient(hunting): get_auth_request con 0x7f5ce00a5210 auth_method 0 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f5ce01381e0 auth_method 0 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.670+0000 7f5cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce01378c0 0x7f5ce0136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): get_auth_request con 0x7f5ce01378c0 auth_method 0 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 10 monclient(hunting): handle_auth_done global_id 174222 payload 824 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce01381e0 msgr2=0x7f5ce013cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce01381e0 0x7f5ce013cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce01378c0 msgr2=0x7f5ce0136e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce01378c0 0x7f5ce0136e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 monclient: found mon.noname-a 2024-09-15T21:33:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce0120750 con 0x7f5ce00a5210 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce01381e0 0x7f5ce013cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdeffd640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce00a5210 0x7f5ce0136900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cd001cca0 tx=0x7f5cd001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00b1040 con 0x7f5ce00a5210 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: dump: 2024-09-15T21:33:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.694 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd00bc070 con 0x7f5ce00a5210 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00b7690 con 0x7f5ce00a5210 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 5 monclient: authenticate success, global_id 174222 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 10 monclient: _renew_subs 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce0121400 con 0x7f5ce00a5210 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdd7fa640 4 set_mon_vals no callback set 2024-09-15T21:33:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: dump: 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 10 monclient: _renew_subs 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce013e6f0 con 0x7f5ce00a5210 2024-09-15T21:33:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 0x7f5ce013d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce00a56a0 con 0x7f5ce013d170 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 0x7f5ce013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient: get_auth_request con 0x7f5ce013d170 auth_method 0 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5cd00b7830 con 0x7f5ce00a5210 2024-09-15T21:33:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_done global_id 164312 payload 824 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ce00008e0 con 0x7f5ce013d170 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ccc03b9c0 0x7f5ccc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5cd00ea3f0 con 0x7f5ce00a5210 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cde7fc640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 0x7f5ce013d550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ce013f660 tx=0x7f5cd400e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4007240 con 0x7f5ce013d170 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: dump: 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ccc03b9c0 0x7f5ccc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 10 monclient: get_auth_request con 0x7f5ccc03b9c0 auth_method 0 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5cd4007240 con 0x7f5ce013d170 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 msgr2=0x7f5ce013d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 0x7f5ce013d550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ce013f660 tx=0x7f5cd400e960 comp rx=0 tx=0).stop 2024-09-15T21:33:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 msgr2=0x7f5ce013d550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cc77fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013d170 0x7f5ce013d550 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ce013f660 tx=0x7f5cd400e960 comp rx=0 tx=0).stop 2024-09-15T21:33:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.674+0000 7f5cdf7fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ccc03b9c0 0x7f5ccc03de80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5ce013e2a0 tx=0x7f5cd800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 0x7f5ce0138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce006ccc0 con 0x7f5ce013da90 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 0x7f5ce0138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient: get_auth_request con 0x7f5ce013da90 auth_method 0 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 10 monclient(hunting): handle_auth_done global_id 164315 payload 824 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 1 -- 172.21.15.17:0/41912258 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5ce013dea0 con 0x7f5ce013da90 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cde7fc640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 0x7f5ce0138d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400efd0 tx=0x7f5cd4003770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd401e760 con 0x7f5ce013da90 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 10 monclient: dump: 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.703 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f5cd4013420 con 0x7f5ce013da90 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 msgr2=0x7f5ce0138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 0x7f5ce0138d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400efd0 tx=0x7f5cd4003770 comp rx=0 tx=0).stop 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 msgr2=0x7f5ce0138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cc77fe640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ce013da90 0x7f5ce0138d60 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400efd0 tx=0x7f5cd4003770 comp rx=0 tx=0).stop 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:13.704 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 146, 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 142266, 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:13.719 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ccc03b9c0 msgr2=0x7f5ccc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5ccc03b9c0 0x7f5ccc03de80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5ce013e2a0 tx=0x7f5cd800b040 comp rx=0 tx=0).stop 2024-09-15T21:33:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 10 monclient: shutdown 2024-09-15T21:33:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce00a5210 msgr2=0x7f5ce0136900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 --2- 172.21.15.17:0/41912258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ce00a5210 0x7f5ce0136900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cd001cca0 tx=0x7f5cd001ac40 comp rx=0 tx=0).stop 2024-09-15T21:33:13.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cdf7fe640 1 -- 172.21.15.17:0/41912258 reap_dead start 2024-09-15T21:33:13.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5cdf7fe640 1 -- 172.21.15.17:0/41912258 reap_dead start 2024-09-15T21:33:13.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 shutdown_connections 2024-09-15T21:33:13.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 >> 172.21.15.17:0/41912258 conn(0x7f5ce00aac40 msgr2=0x7f5ce006e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 shutdown_connections 2024-09-15T21:33:13.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.682+0000 7f5ce5621640 1 -- 172.21.15.17:0/41912258 wait complete. 2024-09-15T21:33:13.722 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:13.722 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:33:13.723 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.838+0000 7fcff8153640 10 monclient: get_monmap_and_config 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.838+0000 7fcff8153640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.838+0000 7fcff8153640 10 monclient: monmap: 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:13.846157+0000 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:13.846157+0000 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.846 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.847 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:13.847 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:13.847 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:13.847 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:13.847 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:13.847 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 Processor -- start 2024-09-15T21:33:13.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 -- start start 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient: init 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcff00a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcff009ffc0 0x7fcff013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): picked mon.noname-a con 0x7fcff009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): picked mon.noname-b con 0x7fcff00a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff00a3130 con 0x7fcff00a0890 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcff0073df0 con 0x7fcff009ffc0 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcff00a1bf0 con 0x7fcff00a1380 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff8153640 10 monclient(hunting): authenticate will time out at 1917.241655s 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff5eef640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcff00a0890 0x7fcff00a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58158/0 (socket says 172.21.15.17:58158) 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff5eef640 1 -- 172.21.15.17:0/2513805861 learned_addr learned my addr 172.21.15.17:0/2513805861 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 1 --2- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2513805861 >> v2:172.21.15.17:3300/0 conn(0x7fcff009ffc0 0x7fcff013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): get_auth_request con 0x7fcff009ffc0 auth_method 0 2024-09-15T21:33:13.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): get_auth_request con 0x7fcff00a1380 auth_method 0 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff66f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient(hunting): handle_auth_done global_id 174225 payload 274 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 1 -- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 msgr2=0x7fcff013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 1 -- 172.21.15.17:0/2513805861 >> v1:172.21.15.17:6789/0 conn(0x7fcff00a0890 legacy=0x7fcff00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 1 monclient: found mon.noname-a 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff56ee640 1 -- 172.21.15.17:0/2513805861 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff009ccd0 con 0x7fcff009ffc0 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff4eed640 1 -- 172.21.15.17:0/2513805861 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335503162 0 0) 0x7fcff00a3130 con 0x7fcff00a0890 2024-09-15T21:33:13.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.842+0000 7fcff4eed640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:13.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff013aac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00097f0 tx=0x7fcfe001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819f6ea30db5b49f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 1 -- 172.21.15.17:0/2513805861 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00b0070 con 0x7fcff009ffc0 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: dump: 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 1 -- 172.21.15.17:0/2513805861 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe00ab600 con 0x7fcff009ffc0 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 1 -- 172.21.15.17:0/2513805861 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00ab9d0 con 0x7fcff009ffc0 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 5 monclient: authenticate success, global_id 174225 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 4 set_mon_vals no callback set 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff4eed640 10 monclient: dump: 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: shutdown 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 msgr2=0x7fcff013aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 --2- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff013aac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00097f0 tx=0x7fcfe001c360 comp rx=0 tx=0).stop 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2513805861 shutdown_connections 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 --2- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff013d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 --2- 172.21.15.17:0/2513805861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff013aac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2513805861 >> 172.21.15.17:0/2513805861 conn(0x7fcff00a66a0 msgr2=0x7fcff00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2513805861 shutdown_connections 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2513805861 wait complete. 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: build_initial_monmap 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: monmap: 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:13.864 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 Processor -- start 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- start start 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: init 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:13.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff0134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): picked mon.noname-a con 0x7fcff009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff00a0890 0x7fcff01354c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): picked mon.noname-b con 0x7fcff00a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff011db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): picked mon.noname-c con 0x7fcff00a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0140100 con 0x7fcff009ffc0 2024-09-15T21:33:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcff013ff80 con 0x7fcff00a0890 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fcff0140280 con 0x7fcff00a1380 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient(hunting): authenticate will time out at 1917.246776s 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff0134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff011db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff0134f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55686/0 (socket says 172.21.15.17:55686) 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 -- 172.21.15.17:0/2582538247 learned_addr learned my addr 172.21.15.17:0/2582538247 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): get_auth_request con 0x7fcff009ffc0 auth_method 0 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): get_auth_request con 0x7fcff00a1380 auth_method 0 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff00a0890 0x7fcff01354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff56ee640 10 monclient(hunting): get_auth_request con 0x7fcff00a0890 auth_method 0 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff56ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff56ee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff56ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient(hunting): handle_auth_done global_id 174231 payload 824 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 msgr2=0x7fcff011db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff011db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff00a0890 msgr2=0x7fcff01354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff00a0890 0x7fcff01354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient: _finish_hunting 0 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 monclient: found mon.noname-a 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff0121650 con 0x7fcff009ffc0 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff66f0640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fcff00a1380 0x7fcff011db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:13.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff5eef640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff0134f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c450 tx=0x7fcfec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec018070 con 0x7fcff009ffc0 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: dump: 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec0074e0 con 0x7fcff009ffc0 2024-09-15T21:33:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec004a30 con 0x7fcff009ffc0 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 5 monclient: authenticate success, global_id 174231 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe67fc640 10 monclient: dump: 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.873 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcfe7fff640 4 set_mon_vals no callback set 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: _renew_subs 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff0135d90 con 0x7fcff009ffc0 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: _renew_subs 2024-09-15T21:33:13.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.846+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff0121090 con 0x7fcff009ffc0 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff8153640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff8153640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 0x7fcff0063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01212d0 con 0x7fcff0120130 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fcfec004bd0 con 0x7fcff009ffc0 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 0x7fcff0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcfd403b9c0 0x7fcfd403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient: get_auth_request con 0x7fcff0120130 auth_method 0 2024-09-15T21:33:13.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fcfec04f910 con 0x7fcff009ffc0 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff66f0640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcfd403b9c0 0x7fcfd403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff66f0640 10 monclient: get_auth_request con 0x7fcfd403b9c0 auth_method 0 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 10 monclient(hunting): handle_auth_done global_id 184114 payload 824 2024-09-15T21:33:13.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcff00a0c70 con 0x7fcff0120130 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff66f0640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcfd403b9c0 0x7fcfd403de80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e740 tx=0x7fcfe800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 0x7fcff0063e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcff009d0a0 tx=0x7fcfe001aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00bb070 con 0x7fcff0120130 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 10 monclient: dump: 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.877 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcfe00b1040 con 0x7fcff0120130 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:13.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:13.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 msgr2=0x7fcff0063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 0x7fcff0063e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcff009d0a0 tx=0x7fcfe001aba0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 msgr2=0x7fcff0063e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:13.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.850+0000 7fcfe67fc640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0120130 0x7fcff0063e00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:13.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:13.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 0x7fcff002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:13.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:13.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0070a90 con 0x7fcff0064340 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 0x7fcff002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient: get_auth_request con 0x7fcff0064340 auth_method 0 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 10 monclient(hunting): handle_auth_done global_id 184117 payload 824 2024-09-15T21:33:13.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 1 -- 172.21.15.17:0/2582538247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcff01da4b0 con 0x7fcff0064340 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff56ee640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 0x7fcff002d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001c3a0 tx=0x7fcfe0019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00bb040 con 0x7fcff0064340 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 10 monclient: dump: 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7fcfe00b8e10 con 0x7fcff0064340 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 msgr2=0x7fcff002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 0x7fcff002d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001c3a0 tx=0x7fcfe0019100 comp rx=0 tx=0).stop 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 msgr2=0x7fcff002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcfe67fc640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0064340 0x7fcff002d450 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001c3a0 tx=0x7fcfe0019100 comp rx=0 tx=0).stop 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 146, 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20835, 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:13.898 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:13.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcfd403b9c0 msgr2=0x7fcfd403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcfd403b9c0 0x7fcfd403de80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e740 tx=0x7fcfe800b040 comp rx=0 tx=0).stop 2024-09-15T21:33:13.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 10 monclient: shutdown 2024-09-15T21:33:13.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:13.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 msgr2=0x7fcff0134f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 --2- 172.21.15.17:0/2582538247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcff009ffc0 0x7fcff0134f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c450 tx=0x7fcfec00c910 comp rx=0 tx=0).stop 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff66f0640 1 -- 172.21.15.17:0/2582538247 reap_dead start 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff66f0640 1 -- 172.21.15.17:0/2582538247 reap_dead start 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 shutdown_connections 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 >> 172.21.15.17:0/2582538247 conn(0x7fcff00a66a0 msgr2=0x7fcff00a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 shutdown_connections 2024-09-15T21:33:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:13.858+0000 7fcff8153640 1 -- 172.21.15.17:0/2582538247 wait complete. 2024-09-15T21:33:13.901 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:13.902 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:33:13.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:33:14.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient: get_monmap_and_config 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient: build_initial_monmap 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient: monmap: 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:14.018949+0000 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:14.018949+0000 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:14.020 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:14.020 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:14.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 Processor -- start 2024-09-15T21:33:14.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 -- start start 2024-09-15T21:33:14.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient: init 2024-09-15T21:33:14.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:14.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-09-15T21:33:14.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf700a0890 0x7faf700a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): picked mon.noname-a con 0x7faf700a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf7009e8c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 0x7faf70143490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): picked mon.noname-b con 0x7faf701410a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf700a31c0 con 0x7faf7009e8c0 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faf700a1af0 con 0x7faf700a0890 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faf7009d640 con 0x7faf701410a0 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf78247640 10 monclient(hunting): authenticate will time out at 1917.414314s 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 0x7faf70143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:14.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 0x7faf70143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37132/0 (socket says 172.21.15.17:37132) 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 -- 172.21.15.17:0/724762154 learned_addr learned my addr 172.21.15.17:0/724762154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): get_auth_request con 0x7faf701410a0 auth_method 0 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 1 --2- 172.21.15.17:0/724762154 >> v2:172.21.15.17:3300/0 conn(0x7faf700a0890 0x7faf700a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): get_auth_request con 0x7faf700a0890 auth_method 0 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient(hunting): handle_auth_done global_id 164324 payload 274 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 -- 172.21.15.17:0/724762154 >> v2:172.21.15.17:3300/0 conn(0x7faf700a0890 msgr2=0x7faf700a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf74fe1640 1 -- 172.21.15.17:0/724762154 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298524680 0 0) 0x7faf700a31c0 con 0x7faf7009e8c0 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 --2- 172.21.15.17:0/724762154 >> v2:172.21.15.17:3300/0 conn(0x7faf700a0890 0x7faf700a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 -- 172.21.15.17:0/724762154 >> v1:172.21.15.17:6789/0 conn(0x7faf7009e8c0 legacy=0x7faf700a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient: _finish_hunting 0 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 monclient: found mon.noname-b 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf767e4640 1 -- 172.21.15.17:0/724762154 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7009d920 con 0x7faf701410a0 2024-09-15T21:33:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf75fe3640 1 --2- 172.21.15.17:0/724762154 >> v2:172.21.15.17:3300/0 conn(0x7faf700a0890 0x7faf700a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.014+0000 7faf74fe1640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf767e4640 1 --2- 172.21.15.17:0/724762154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 0x7faf70143490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf68009e80 tx=0x7faf6801c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245e2e87b5097245 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 1 -- 172.21.15.17:0/724762154 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680a5590 con 0x7faf701410a0 2024-09-15T21:33:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: dump: 2024-09-15T21:33:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: _finish_auth 0 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: _check_auth_tickets 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 1 -- 172.21.15.17:0/724762154 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf680a5730 con 0x7faf701410a0 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 5 monclient: authenticate success, global_id 164324 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 4 set_mon_vals no callback set 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 1 -- 172.21.15.17:0/724762154 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680a5b00 con 0x7faf701410a0 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf74fe1640 10 monclient: dump: 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: shutdown 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- 172.21.15.17:0/724762154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 msgr2=0x7faf70143490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 --2- 172.21.15.17:0/724762154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 0x7faf70143490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf68009e80 tx=0x7faf6801c8d0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- 172.21.15.17:0/724762154 shutdown_connections 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 --2- 172.21.15.17:0/724762154 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf701410a0 0x7faf70143490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 --2- 172.21.15.17:0/724762154 >> v2:172.21.15.17:3300/0 conn(0x7faf700a0890 0x7faf700a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- 172.21.15.17:0/724762154 >> 172.21.15.17:0/724762154 conn(0x7faf700a66a0 msgr2=0x7faf700a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- 172.21.15.17:0/724762154 shutdown_connections 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- 172.21.15.17:0/724762154 wait complete. 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: build_initial_monmap 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: monmap: 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:14.034 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 Processor -- start 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- start start 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: init 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 0x7faf701d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): picked mon.noname-c con 0x7faf700a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf701410a0 0x7faf7013bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): picked mon.noname-a con 0x7faf701410a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf7013c400 0x7faf700720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): picked mon.noname-b con 0x7faf7013c400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faf70146580 con 0x7faf701410a0 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faf70146400 con 0x7faf7013c400 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faf70146700 con 0x7faf700a0890 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf78247640 10 monclient(hunting): authenticate will time out at 1917.419400s 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 0x7faf701d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 0x7faf701d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37142/0 (socket says 172.21.15.17:37142) 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 -- 172.21.15.17:0/514968256 learned_addr learned my addr 172.21.15.17:0/514968256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): get_auth_request con 0x7faf700a0890 auth_method 0 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf701410a0 0x7faf7013bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): get_auth_request con 0x7faf701410a0 auth_method 0 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf757e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf767e4640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf7013c400 0x7faf700720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient(hunting): handle_auth_done global_id 164327 payload 824 2024-09-15T21:33:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 -- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf7013c400 msgr2=0x7faf700720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf7013c400 0x7faf700720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 -- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf701410a0 msgr2=0x7faf7013bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf701410a0 0x7faf7013bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient: _finish_hunting 0 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 monclient: found mon.noname-c 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf75fe3640 1 -- 172.21.15.17:0/514968256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf701d47f0 con 0x7faf700a0890 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf767e4640 10 monclient: get_auth_request con 0x7faf7013c400 auth_method 0 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.018+0000 7faf767e4640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf7013c400 0x7faf700720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf75fe3640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 0x7faf701d38e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00ec40 tx=0x7faf6c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c017070 con 0x7faf700a0890 2024-09-15T21:33:14.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: dump: 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:14.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c00cbd0 con 0x7faf700a0890 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 5 monclient: authenticate success, global_id 164327 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 monclient: _renew_subs 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf70126570 con 0x7faf700a0890 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c013470 con 0x7faf700a0890 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 10 monclient: dump: 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf677fe640 4 set_mon_vals no callback set 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 monclient: _renew_subs 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf70142e70 con 0x7faf700a0890 2024-09-15T21:33:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf700a4c00 con 0x7faf700a0890 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7faf6c01a070 con 0x7faf700a0890 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf5403b9c0 0x7faf5403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7faf6c050570 con 0x7faf700a0890 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf757e2640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf5403b9c0 0x7faf5403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf757e2640 10 monclient: get_auth_request con 0x7faf5403b9c0 auth_method 0 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.022+0000 7faf757e2640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf5403b9c0 0x7faf5403de80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faf701253b0 tx=0x7faf60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.026+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7faf701d47f0 con 0x7faf700a0890 2024-09-15T21:33:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.026+0000 7faf667fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.026+0000 7faf667fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:14.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.262+0000 7faf78247640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:33:14.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.262+0000 7faf78247640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:33:14.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.262+0000 7faf78247640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:14.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.262+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7faf700a0c70 con 0x7faf700a0890 2024-09-15T21:33:14.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=26,config=2,health=10,kv=12,logm=50} crc {auth=333146793,config=3541143443,health=2026282893,kv=3079713637,logm=3885471056}) 2024-09-15T21:33:14.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=25,config=2,health=12,kv=11,logm=50} crc {auth=4069855157,config=3541143443,health=1675380437,kv=3059833255,logm=1433837657}) 2024-09-15T21:33:14.271 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf667fc640 1 -- 172.21.15.17:0/514968256 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7faf6c013ae0 con 0x7faf700a0890 2024-09-15T21:33:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf667fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:33:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf667fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:14.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=26,config=2,health=10,kv=12,logm=50} crc {auth=333146793,config=3541143443,health=2026282893,kv=3079713637,logm=3885471056}) 2024-09-15T21:33:14.272 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=26,config=2,health=12,kv=11,logm=49} crc {auth=3958456616,config=3541143443,health=1675380437,kv=2435832620,logm=1140124169}) 2024-09-15T21:33:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf5403b9c0 msgr2=0x7faf5403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf5403b9c0 0x7faf5403de80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faf701253b0 tx=0x7faf60009040 comp rx=0 tx=0).stop 2024-09-15T21:33:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 10 monclient: shutdown 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 msgr2=0x7faf701d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 0x7faf701d38e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00ec40 tx=0x7faf6c00c520 comp rx=0 tx=0).stop 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 shutdown_connections 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7faf5403b9c0 0x7faf5403de80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf7013c400 0x7faf700720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf701410a0 0x7faf7013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 --2- 172.21.15.17:0/514968256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faf700a0890 0x7faf701d38e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 >> 172.21.15.17:0/514968256 conn(0x7faf700a66a0 msgr2=0x7faf70141db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 shutdown_connections 2024-09-15T21:33:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:14.266+0000 7faf78247640 1 -- 172.21.15.17:0/514968256 wait complete. 2024-09-15T21:33:14.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.266+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=858610754}) 2024-09-15T21:33:14.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3987577177}) 2024-09-15T21:33:14.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.275 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=858610754}) 2024-09-15T21:33:14.276 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3029505928}) 2024-09-15T21:33:14.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=658863299}) 2024-09-15T21:33:14.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1715592985}) 2024-09-15T21:33:14.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=658863299}) 2024-09-15T21:33:14.277 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.270+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=304734096}) 2024-09-15T21:33:14.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.274+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.274+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1943545709}) 2024-09-15T21:33:14.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.274+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1247218354}) 2024-09-15T21:33:14.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.274+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.274+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1943545709}) 2024-09-15T21:33:14.280 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.274+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3841284771}) 2024-09-15T21:33:14.282 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3052106640}) 2024-09-15T21:33:14.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3952654081}) 2024-09-15T21:33:14.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3052106640}) 2024-09-15T21:33:14.283 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2449706635}) 2024-09-15T21:33:14.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2055479124}) 2024-09-15T21:33:14.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4224096290}) 2024-09-15T21:33:14.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2055479124}) 2024-09-15T21:33:14.285 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.278+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4156444204}) 2024-09-15T21:33:14.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2148148659}) 2024-09-15T21:33:14.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371731027}) 2024-09-15T21:33:14.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2148148659}) 2024-09-15T21:33:14.287 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3401061472}) 2024-09-15T21:33:14.289 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.289 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=259424886}) 2024-09-15T21:33:14.289 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1437814755}) 2024-09-15T21:33:14.289 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.290 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:33:14.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=259424886}) 2024-09-15T21:33:14.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.282+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1620125036}) 2024-09-15T21:33:14.291 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.291 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1248631929}) 2024-09-15T21:33:14.291 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3969244086}) 2024-09-15T21:33:14.291 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1248631929}) 2024-09-15T21:33:14.292 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=564090999}) 2024-09-15T21:33:14.293 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.293 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947967553}) 2024-09-15T21:33:14.293 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375893522}) 2024-09-15T21:33:14.293 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.293 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947967553}) 2024-09-15T21:33:14.293 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.286+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2519282449}) 2024-09-15T21:33:14.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2329687635}) 2024-09-15T21:33:14.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632275539}) 2024-09-15T21:33:14.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2329687635}) 2024-09-15T21:33:14.295 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1037768449}) 2024-09-15T21:33:14.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3436943903}) 2024-09-15T21:33:14.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3854367064}) 2024-09-15T21:33:14.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3436943903}) 2024-09-15T21:33:14.298 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.290+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2887490764}) 2024-09-15T21:33:14.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=726370396}) 2024-09-15T21:33:14.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3882292573}) 2024-09-15T21:33:14.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=726370396}) 2024-09-15T21:33:14.299 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1668108167}) 2024-09-15T21:33:14.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744380585}) 2024-09-15T21:33:14.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=796581724}) 2024-09-15T21:33:14.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.301 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744380585}) 2024-09-15T21:33:14.302 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.294+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3775203196}) 2024-09-15T21:33:14.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2316835038}) 2024-09-15T21:33:14.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1739851575}) 2024-09-15T21:33:14.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2316835038}) 2024-09-15T21:33:14.303 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2596522884}) 2024-09-15T21:33:14.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700190278}) 2024-09-15T21:33:14.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=670190696}) 2024-09-15T21:33:14.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700190278}) 2024-09-15T21:33:14.305 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.298+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=221628394}) 2024-09-15T21:33:14.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4126612420}) 2024-09-15T21:33:14.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3367500679}) 2024-09-15T21:33:14.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4126612420}) 2024-09-15T21:33:14.307 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1689505541}) 2024-09-15T21:33:14.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467447973}) 2024-09-15T21:33:14.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=954314763}) 2024-09-15T21:33:14.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467447973}) 2024-09-15T21:33:14.309 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.302+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2266570140}) 2024-09-15T21:33:14.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493601570}) 2024-09-15T21:33:14.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=744013985}) 2024-09-15T21:33:14.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493601570}) 2024-09-15T21:33:14.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2236327200}) 2024-09-15T21:33:14.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1857575076}) 2024-09-15T21:33:14.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837043019}) 2024-09-15T21:33:14.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.313 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1857575076}) 2024-09-15T21:33:14.314 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.306+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=989429157}) 2024-09-15T21:33:14.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.315 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266079335}) 2024-09-15T21:33:14.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=54254360}) 2024-09-15T21:33:14.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266079335}) 2024-09-15T21:33:14.316 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3599538382}) 2024-09-15T21:33:14.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1241095000}) 2024-09-15T21:33:14.317 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3797305080}) 2024-09-15T21:33:14.318 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.318 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1241095000}) 2024-09-15T21:33:14.318 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.310+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3978697901}) 2024-09-15T21:33:14.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.314+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.314+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2842031283}) 2024-09-15T21:33:14.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.314+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3304991300}) 2024-09-15T21:33:14.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.314+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.314+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2842031283}) 2024-09-15T21:33:14.320 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.314+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3775976127}) 2024-09-15T21:33:14.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=374423119}) 2024-09-15T21:33:14.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1318654621}) 2024-09-15T21:33:14.322 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=374423119}) 2024-09-15T21:33:14.323 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1939067474}) 2024-09-15T21:33:14.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3033073432}) 2024-09-15T21:33:14.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=841286982}) 2024-09-15T21:33:14.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3033073432}) 2024-09-15T21:33:14.325 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.318+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2415188203}) 2024-09-15T21:33:14.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767528181}) 2024-09-15T21:33:14.327 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=430437037}) 2024-09-15T21:33:14.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767528181}) 2024-09-15T21:33:14.328 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2605161046}) 2024-09-15T21:33:14.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356328286}) 2024-09-15T21:33:14.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1918073437}) 2024-09-15T21:33:14.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356328286}) 2024-09-15T21:33:14.329 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.322+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3588488273}) 2024-09-15T21:33:14.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294911308}) 2024-09-15T21:33:14.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=520851566}) 2024-09-15T21:33:14.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294911308}) 2024-09-15T21:33:14.331 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2177059341}) 2024-09-15T21:33:14.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=769190272}) 2024-09-15T21:33:14.333 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2182002700}) 2024-09-15T21:33:14.334 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.334 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=769190272}) 2024-09-15T21:33:14.334 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.326+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2455884688}) 2024-09-15T21:33:14.335 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2380304938}) 2024-09-15T21:33:14.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1597136626}) 2024-09-15T21:33:14.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2380304938}) 2024-09-15T21:33:14.336 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1163161349}) 2024-09-15T21:33:14.338 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.338 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.330+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306800070}) 2024-09-15T21:33:14.338 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1429469860}) 2024-09-15T21:33:14.338 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.338 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3306800070}) 2024-09-15T21:33:14.338 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1890857254}) 2024-09-15T21:33:14.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2534071002}) 2024-09-15T21:33:14.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2110562308}) 2024-09-15T21:33:14.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2534071002}) 2024-09-15T21:33:14.340 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.334+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2980126744}) 2024-09-15T21:33:14.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130848148}) 2024-09-15T21:33:14.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2853363301}) 2024-09-15T21:33:14.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130848148}) 2024-09-15T21:33:14.342 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329893467}) 2024-09-15T21:33:14.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651389700}) 2024-09-15T21:33:14.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=758514637}) 2024-09-15T21:33:14.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651389700}) 2024-09-15T21:33:14.344 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3340958668}) 2024-09-15T21:33:14.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3202317872}) 2024-09-15T21:33:14.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.338+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2204676380}) 2024-09-15T21:33:14.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3202317872}) 2024-09-15T21:33:14.346 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3446004387}) 2024-09-15T21:33:14.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2635584104}) 2024-09-15T21:33:14.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3024007732}) 2024-09-15T21:33:14.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2635584104}) 2024-09-15T21:33:14.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1227445695}) 2024-09-15T21:33:14.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.342+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2686528736}) 2024-09-15T21:33:14.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2098603031}) 2024-09-15T21:33:14.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2686528736}) 2024-09-15T21:33:14.350 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2365960430}) 2024-09-15T21:33:14.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1146106809}) 2024-09-15T21:33:14.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2418029217}) 2024-09-15T21:33:14.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1146106809}) 2024-09-15T21:33:14.352 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.346+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4003313021}) 2024-09-15T21:33:14.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117786857}) 2024-09-15T21:33:14.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=998441202}) 2024-09-15T21:33:14.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117786857}) 2024-09-15T21:33:14.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1733965206}) 2024-09-15T21:33:14.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3216805750}) 2024-09-15T21:33:14.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=270814024}) 2024-09-15T21:33:14.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3216805750}) 2024-09-15T21:33:14.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.350+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4011468431}) 2024-09-15T21:33:14.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874880631}) 2024-09-15T21:33:14.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3128361809}) 2024-09-15T21:33:14.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874880631}) 2024-09-15T21:33:14.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1252438742}) 2024-09-15T21:33:14.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2273938630}) 2024-09-15T21:33:14.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1586906051}) 2024-09-15T21:33:14.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2273938630}) 2024-09-15T21:33:14.360 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=242226778}) 2024-09-15T21:33:14.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1134345603}) 2024-09-15T21:33:14.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2685651725}) 2024-09-15T21:33:14.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.362 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1134345603}) 2024-09-15T21:33:14.362 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.354+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=811739518}) 2024-09-15T21:33:14.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2374290548}) 2024-09-15T21:33:14.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=661664498}) 2024-09-15T21:33:14.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2374290548}) 2024-09-15T21:33:14.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1960638608}) 2024-09-15T21:33:14.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1057216953}) 2024-09-15T21:33:14.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2339168159}) 2024-09-15T21:33:14.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1057216953}) 2024-09-15T21:33:14.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.358+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=494168851}) 2024-09-15T21:33:14.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=743808462}) 2024-09-15T21:33:14.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=821835610}) 2024-09-15T21:33:14.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=743808462}) 2024-09-15T21:33:14.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1585744720}) 2024-09-15T21:33:14.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=792536829}) 2024-09-15T21:33:14.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=286049616}) 2024-09-15T21:33:14.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=792536829}) 2024-09-15T21:33:14.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4190291869}) 2024-09-15T21:33:14.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2830181067}) 2024-09-15T21:33:14.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1753232725}) 2024-09-15T21:33:14.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2830181067}) 2024-09-15T21:33:14.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.362+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1661823381}) 2024-09-15T21:33:14.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693666677}) 2024-09-15T21:33:14.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=871211023}) 2024-09-15T21:33:14.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693666677}) 2024-09-15T21:33:14.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1412453699}) 2024-09-15T21:33:14.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85478407}) 2024-09-15T21:33:14.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=429970721}) 2024-09-15T21:33:14.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=85478407}) 2024-09-15T21:33:14.373 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.366+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=177615569}) 2024-09-15T21:33:14.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430897098}) 2024-09-15T21:33:14.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1143485533}) 2024-09-15T21:33:14.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430897098}) 2024-09-15T21:33:14.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3577503423}) 2024-09-15T21:33:14.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2558448495}) 2024-09-15T21:33:14.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=587981119}) 2024-09-15T21:33:14.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2558448495}) 2024-09-15T21:33:14.377 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.370+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3878355878}) 2024-09-15T21:33:14.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526711847}) 2024-09-15T21:33:14.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=648798063}) 2024-09-15T21:33:14.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526711847}) 2024-09-15T21:33:14.379 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2005253172}) 2024-09-15T21:33:14.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=383423721}) 2024-09-15T21:33:14.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=213473198}) 2024-09-15T21:33:14.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=383423721}) 2024-09-15T21:33:14.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3542752770}) 2024-09-15T21:33:14.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092735859}) 2024-09-15T21:33:14.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932939523}) 2024-09-15T21:33:14.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.374+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092735859}) 2024-09-15T21:33:14.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1681013245}) 2024-09-15T21:33:14.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379938935}) 2024-09-15T21:33:14.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3354160015}) 2024-09-15T21:33:14.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379938935}) 2024-09-15T21:33:14.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4073513932}) 2024-09-15T21:33:14.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2847896353}) 2024-09-15T21:33:14.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1745873135}) 2024-09-15T21:33:14.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2847896353}) 2024-09-15T21:33:14.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.378+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3449201810}) 2024-09-15T21:33:14.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3607597106}) 2024-09-15T21:33:14.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4140983426}) 2024-09-15T21:33:14.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3607597106}) 2024-09-15T21:33:14.388 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1013901562}) 2024-09-15T21:33:14.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266247152}) 2024-09-15T21:33:14.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169331764}) 2024-09-15T21:33:14.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266247152}) 2024-09-15T21:33:14.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.382+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2253746832}) 2024-09-15T21:33:14.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4278360322}) 2024-09-15T21:33:14.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3448053961}) 2024-09-15T21:33:14.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4278360322}) 2024-09-15T21:33:14.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=513067446}) 2024-09-15T21:33:14.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775293382}) 2024-09-15T21:33:14.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3778273508}) 2024-09-15T21:33:14.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775293382}) 2024-09-15T21:33:14.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.386+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766840563}) 2024-09-15T21:33:14.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139210220}) 2024-09-15T21:33:14.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2576921696}) 2024-09-15T21:33:14.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139210220}) 2024-09-15T21:33:14.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125378991}) 2024-09-15T21:33:14.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2207248340}) 2024-09-15T21:33:14.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1486052845}) 2024-09-15T21:33:14.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2207248340}) 2024-09-15T21:33:14.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3837685147}) 2024-09-15T21:33:14.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=837146117}) 2024-09-15T21:33:14.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585830951}) 2024-09-15T21:33:14.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=837146117}) 2024-09-15T21:33:14.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.390+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2810225273}) 2024-09-15T21:33:14.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2306672962}) 2024-09-15T21:33:14.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643170931}) 2024-09-15T21:33:14.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2306672962}) 2024-09-15T21:33:14.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1150031846}) 2024-09-15T21:33:14.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3557586245}) 2024-09-15T21:33:14.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2976076308}) 2024-09-15T21:33:14.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3557586245}) 2024-09-15T21:33:14.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.394+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4021527335}) 2024-09-15T21:33:14.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3002099988}) 2024-09-15T21:33:14.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=798691017}) 2024-09-15T21:33:14.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3002099988}) 2024-09-15T21:33:14.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2571910777}) 2024-09-15T21:33:14.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1227221355}) 2024-09-15T21:33:14.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2987004692}) 2024-09-15T21:33:14.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1227221355}) 2024-09-15T21:33:14.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.398+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1224111851}) 2024-09-15T21:33:14.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588540701}) 2024-09-15T21:33:14.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3928178331}) 2024-09-15T21:33:14.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588540701}) 2024-09-15T21:33:14.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3472707504}) 2024-09-15T21:33:14.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225884557}) 2024-09-15T21:33:14.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846522089}) 2024-09-15T21:33:14.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225884557}) 2024-09-15T21:33:14.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.402+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2801372898}) 2024-09-15T21:33:14.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1185719217}) 2024-09-15T21:33:14.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4250032114}) 2024-09-15T21:33:14.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1185719217}) 2024-09-15T21:33:14.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1234232577}) 2024-09-15T21:33:14.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2288909019}) 2024-09-15T21:33:14.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3024768082}) 2024-09-15T21:33:14.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2288909019}) 2024-09-15T21:33:14.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2050255697}) 2024-09-15T21:33:14.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102597216}) 2024-09-15T21:33:14.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.406+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=35750644}) 2024-09-15T21:33:14.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102597216}) 2024-09-15T21:33:14.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3834108128}) 2024-09-15T21:33:14.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2790823960}) 2024-09-15T21:33:14.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3632473878}) 2024-09-15T21:33:14.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2790823960}) 2024-09-15T21:33:14.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1477148309}) 2024-09-15T21:33:14.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445034835}) 2024-09-15T21:33:14.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1216439883}) 2024-09-15T21:33:14.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445034835}) 2024-09-15T21:33:14.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.410+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2000362334}) 2024-09-15T21:33:14.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1386432019}) 2024-09-15T21:33:14.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925938539}) 2024-09-15T21:33:14.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1386432019}) 2024-09-15T21:33:14.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=705301509}) 2024-09-15T21:33:14.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3405354653}) 2024-09-15T21:33:14.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4082825323}) 2024-09-15T21:33:14.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.414+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3405354653}) 2024-09-15T21:33:14.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=291353952}) 2024-09-15T21:33:14.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2253175985}) 2024-09-15T21:33:14.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1356020780}) 2024-09-15T21:33:14.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2253175985}) 2024-09-15T21:33:14.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2071372167}) 2024-09-15T21:33:14.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1582988467}) 2024-09-15T21:33:14.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=978385691}) 2024-09-15T21:33:14.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1582988467}) 2024-09-15T21:33:14.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.418+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=828413941}) 2024-09-15T21:33:14.427 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1062794407}) 2024-09-15T21:33:14.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762286127}) 2024-09-15T21:33:14.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1062794407}) 2024-09-15T21:33:14.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2336948968}) 2024-09-15T21:33:14.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632799709}) 2024-09-15T21:33:14.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3473900174}) 2024-09-15T21:33:14.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.422+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632799709}) 2024-09-15T21:33:14.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4219157482}) 2024-09-15T21:33:14.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1250383793}) 2024-09-15T21:33:14.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=604432822}) 2024-09-15T21:33:14.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1250383793}) 2024-09-15T21:33:14.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.426+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949964092}) 2024-09-15T21:33:14.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432125117}) 2024-09-15T21:33:14.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2599726903}) 2024-09-15T21:33:14.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432125117}) 2024-09-15T21:33:14.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=599043973}) 2024-09-15T21:33:14.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3674830527}) 2024-09-15T21:33:14.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3307291904}) 2024-09-15T21:33:14.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3674830527}) 2024-09-15T21:33:14.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.430+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3200322644}) 2024-09-15T21:33:14.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=291736078}) 2024-09-15T21:33:14.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=870607223}) 2024-09-15T21:33:14.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=291736078}) 2024-09-15T21:33:14.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=154823529}) 2024-09-15T21:33:14.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1278325656}) 2024-09-15T21:33:14.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=664981906}) 2024-09-15T21:33:14.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1278325656}) 2024-09-15T21:33:14.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.434+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2802651710}) 2024-09-15T21:33:14.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=106200487}) 2024-09-15T21:33:14.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=591808875}) 2024-09-15T21:33:14.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=106200487}) 2024-09-15T21:33:14.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=120967592}) 2024-09-15T21:33:14.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1623627729}) 2024-09-15T21:33:14.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3816405961}) 2024-09-15T21:33:14.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1623627729}) 2024-09-15T21:33:14.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.438+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3909878072}) 2024-09-15T21:33:14.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.442+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.447 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.442+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=89,mdsmap=3,mgr=8} crc {logm=116010690,mdsmap=173658045,mgr=3916736423}) 2024-09-15T21:33:14.447 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.442+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=336386161,mdsmap=173658045,mgr=259257734}) 2024-09-15T21:33:14.447 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.442+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.447 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.442+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=89,mdsmap=3,mgr=8} crc {logm=116010690,mdsmap=173658045,mgr=3916736423}) 2024-09-15T21:33:14.447 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.442+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=95,mdsmap=3,mgr=2} crc {logm=3114070310,mdsmap=173658045,mgr=1430609620}) 2024-09-15T21:33:14.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.454+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.454+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=14,monmap=3,nvmeofgw=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=700485134,mon_config_key=521969046,monmap=186996059,nvmeofgw=1596038039,osd_metadata=1154780841,osd_pg_creating=543505836,osd_snap=841190635,osdmap=4054534628}) 2024-09-15T21:33:14.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.454+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=50} crc {mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3293125510,mon_config_key=3254240883,monmap=186996059,nvmeofgw=598337686,osd_metadata=166226148,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3014104468}) 2024-09-15T21:33:14.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.454+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:14.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.454+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=14,monmap=3,nvmeofgw=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=700485134,mon_config_key=521969046,monmap=186996059,nvmeofgw=1596038039,osd_metadata=1154780841,osd_pg_creating=543505836,osd_snap=841190635,osdmap=4054534628}) 2024-09-15T21:33:14.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:14.454+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=700485134,mon_config_key=171072070,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3176747619}) 2024-09-15T21:33:15.291 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:15.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient: get_monmap_and_config 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient: build_initial_monmap 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient: monmap: 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:15.423527+0000 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:15.423527+0000 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 Processor -- start 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 -- start start 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient: init 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048073640 0x7f8048073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): picked mon.noname-b con 0x7f8048073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8048073f60 0x7f8048137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): picked mon.noname-a con 0x7f8048073f60 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8048137bd0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80480a0890 con 0x7f8048137bd0 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f80480746e0 con 0x7f8048073f60 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f80480a1640 con 0x7f8048073640 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f804d3e3640 10 monclient(hunting): authenticate will time out at 1918.818928s 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048073640 0x7f8048073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8048073f60 0x7f8048137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80477fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8048137bd0 0x7f804813ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58182/0 (socket says 172.21.15.17:58182) 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80477fe640 1 -- 172.21.15.17:0/354330446 learned_addr learned my addr 172.21.15.17:0/354330446 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): get_auth_request con 0x7f8048073f60 auth_method 0 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): get_auth_request con 0x7f8048073640 auth_method 0 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient(hunting): handle_auth_done global_id 174243 payload 274 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 1 -- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048073640 msgr2=0x7f8048073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 1 --2- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048073640 0x7f8048073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 1 -- 172.21.15.17:0/354330446 >> v1:172.21.15.17:6789/0 conn(0x7f8048137bd0 legacy=0x7f804813ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient: _finish_hunting 0 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 1 monclient: found mon.noname-a 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f80467fc640 1 -- 172.21.15.17:0/354330446 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80480a1920 con 0x7f8048073f60 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.418+0000 7f8046ffd640 1 --2- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048073640 0x7f8048073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f80467fc640 1 --2- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048137690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80380097c0 tx=0x7f803801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549c2cb8e008f5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 1 -- 172.21.15.17:0/354330446 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380b0070 con 0x7f8048073f60 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: dump: 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: _finish_auth 0 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 1 -- 172.21.15.17:0/354330446 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80380ab600 con 0x7f8048073f60 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 5 monclient: authenticate success, global_id 174243 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 1 -- 172.21.15.17:0/354330446 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380ab9d0 con 0x7f8048073f60 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 4 set_mon_vals no callback set 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f8045ffb640 10 monclient: dump: 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: shutdown 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 msgr2=0x7f8048137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 --2- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048137690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80380097c0 tx=0x7f803801c3f0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- 172.21.15.17:0/354330446 shutdown_connections 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 --2- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048137690 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 --2- 172.21.15.17:0/354330446 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048073640 0x7f8048073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- 172.21.15.17:0/354330446 >> 172.21.15.17:0/354330446 conn(0x7f80480a66a0 msgr2=0x7f80480a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- 172.21.15.17:0/354330446 shutdown_connections 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- 172.21.15.17:0/354330446 wait complete. 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: build_initial_monmap 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: monmap: 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 Processor -- start 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- start start 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: init 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8048073640 0x7f80481cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): picked mon.noname-b con 0x7f8048073640 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048132ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): picked mon.noname-a con 0x7f8048073f60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048137bd0 0x7f804813cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): picked mon.noname-c con 0x7f8048137bd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f804809ce50 con 0x7f8048073f60 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f804809ccd0 con 0x7f8048073640 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f80480a0d00 con 0x7f8048137bd0 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.422+0000 7f804d3e3640 10 monclient(hunting): authenticate will time out at 1918.823994s 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048132ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048137bd0 0x7f804813cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048132ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55716/0 (socket says 172.21.15.17:55716) 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 1 -- 172.21.15.17:0/2271467557 learned_addr learned my addr 172.21.15.17:0/2271467557 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): get_auth_request con 0x7f8048073f60 auth_method 0 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): get_auth_request con 0x7f8048137bd0 auth_method 0 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8048073640 0x7f80481cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 10 monclient(hunting): get_auth_request con 0x7f8048073640 auth_method 0 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient(hunting): handle_auth_done global_id 164333 payload 824 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 -- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8048073640 msgr2=0x7f80481cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8048073640 0x7f80481cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 -- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 msgr2=0x7f8048132ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048132ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient: _finish_hunting 0 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 monclient: found mon.noname-c 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 -- 172.21.15.17:0/2271467557 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804813e820 con 0x7f8048137bd0 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80467fc640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80477fe640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048137bd0 0x7f804813cd60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f804000cad0 tx=0x7f804000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040017070 con 0x7f8048137bd0 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: dump: 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804000abf0 con 0x7f8048137bd0 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040012400 con 0x7f8048137bd0 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 10 monclient: dump: 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 5 monclient: authenticate success, global_id 164333 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 monclient: _renew_subs 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80481d0a70 con 0x7f8048137bd0 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f80457fa640 4 set_mon_vals no callback set 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 monclient: _renew_subs 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f804813f380 con 0x7f8048137bd0 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8048073a20 con 0x7f8048137bd0 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f804001a070 con 0x7f8048137bd0 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f803403b9f0 0x7f803403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f8040050200 con 0x7f8048137bd0 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f803403b9f0 0x7f803403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 10 monclient: get_auth_request con 0x7f803403b9f0 auth_method 0 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.426+0000 7f8046ffd640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f803403b9f0 0x7f803403deb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f803c009580 tx=0x7f803c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.430+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8040007a00 con 0x7f8048137bd0 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.430+0000 7f802f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.430+0000 7f802f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:15.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.666+0000 7f804d3e3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:15.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.666+0000 7f804d3e3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:15.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.666+0000 7f804d3e3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.666+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f804806d020 con 0x7f8048137bd0 2024-09-15T21:33:15.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f802f7fe640 1 -- 172.21.15.17:0/2271467557 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f8040019cd0 con 0x7f8048137bd0 2024-09-15T21:33:15.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f802f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:15.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f802f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:15.675 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f803403b9f0 msgr2=0x7f803403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f803403b9f0 0x7f803403deb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f803c009580 tx=0x7f803c008040 comp rx=0 tx=0).stop 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 10 monclient: shutdown 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048137bd0 msgr2=0x7f804813cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048137bd0 0x7f804813cd60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f804000cad0 tx=0x7f804000cf90 comp rx=0 tx=0).stop 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 shutdown_connections 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f803403b9f0 0x7f803403deb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8048137bd0 0x7f804813cd60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8048073f60 0x7f8048132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 --2- 172.21.15.17:0/2271467557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8048073640 0x7f80481cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 >> 172.21.15.17:0/2271467557 conn(0x7f80480a66a0 msgr2=0x7f80480a5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:15.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 shutdown_connections 2024-09-15T21:33:15.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.670+0000 7f804d3e3640 1 -- 172.21.15.17:0/2271467557 wait complete. 2024-09-15T21:33:15.693 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:33:15.693 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:33:15.693 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:33:15.694 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:33:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient: get_monmap_and_config 2024-09-15T21:33:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient: build_initial_monmap 2024-09-15T21:33:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient: monmap: 2024-09-15T21:33:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:15.811 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:15.811561+0000 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:15.811561+0000 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:15.812 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 Processor -- start 2024-09-15T21:33:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 -- start start 2024-09-15T21:33:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient: init 2024-09-15T21:33:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fce880a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce8809ffc0 0x7fce8813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): picked mon.noname-a con 0x7fce8809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a1380 0x7fce8813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): picked mon.noname-b con 0x7fce880a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce880a3130 con 0x7fce880a0890 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fce88073df0 con 0x7fce8809ffc0 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fce880a1bf0 con 0x7fce880a1380 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8ea89640 10 monclient(hunting): authenticate will time out at 1919.206931s 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce8809ffc0 0x7fce8813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a1380 0x7fce8813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8c825640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fce880a0890 0x7fce880a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58186/0 (socket says 172.21.15.17:58186) 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8c825640 1 -- 172.21.15.17:0/4294227456 learned_addr learned my addr 172.21.15.17:0/4294227456 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): get_auth_request con 0x7fce8809ffc0 auth_method 0 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): get_auth_request con 0x7fce880a1380 auth_method 0 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient(hunting): handle_auth_done global_id 164336 payload 274 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 1 -- 172.21.15.17:0/4294227456 >> v2:172.21.15.17:3300/0 conn(0x7fce8809ffc0 msgr2=0x7fce8813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 1 --2- 172.21.15.17:0/4294227456 >> v2:172.21.15.17:3300/0 conn(0x7fce8809ffc0 0x7fce8813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 1 -- 172.21.15.17:0/4294227456 >> v1:172.21.15.17:6789/0 conn(0x7fce880a0890 legacy=0x7fce880a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient: _finish_hunting 0 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 1 monclient: found mon.noname-b 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce877fe640 1 -- 172.21.15.17:0/4294227456 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1380040704 0 0) 0x7fce880a3130 con 0x7fce880a0890 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce8d026640 1 -- 172.21.15.17:0/4294227456 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8809ccd0 con 0x7fce880a1380 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce877fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.806+0000 7fce87fff640 1 --2- 172.21.15.17:0/4294227456 >> v2:172.21.15.17:3300/0 conn(0x7fce8809ffc0 0x7fce8813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8d026640 1 --2- 172.21.15.17:0/4294227456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a1380 0x7fce8813d010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce800097f0 tx=0x7fce8001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a2e90e205bd997 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 1 -- 172.21.15.17:0/4294227456 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800b0070 con 0x7fce880a1380 2024-09-15T21:33:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: dump: 2024-09-15T21:33:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 1 -- 172.21.15.17:0/4294227456 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce800ab600 con 0x7fce880a1380 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 1 -- 172.21.15.17:0/4294227456 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800ab9d0 con 0x7fce880a1380 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 5 monclient: authenticate success, global_id 164336 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 4 set_mon_vals no callback set 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce877fe640 10 monclient: dump: 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 10 monclient: shutdown 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 -- 172.21.15.17:0/4294227456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a1380 msgr2=0x7fce8813d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 --2- 172.21.15.17:0/4294227456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a1380 0x7fce8813d010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce800097f0 tx=0x7fce8001c360 comp rx=0 tx=0).stop 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 -- 172.21.15.17:0/4294227456 shutdown_connections 2024-09-15T21:33:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 --2- 172.21.15.17:0/4294227456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a1380 0x7fce8813d010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 --2- 172.21.15.17:0/4294227456 >> v2:172.21.15.17:3300/0 conn(0x7fce8809ffc0 0x7fce8813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 -- 172.21.15.17:0/4294227456 >> 172.21.15.17:0/4294227456 conn(0x7fce880a66a0 msgr2=0x7fce880a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 -- 172.21.15.17:0/4294227456 shutdown_connections 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 -- 172.21.15.17:0/4294227456 wait complete. 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 10 monclient: build_initial_monmap 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 10 monclient: monmap: 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 Processor -- start 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 1 -- start start 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.810+0000 7fce8ea89640 10 monclient: init 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809ffc0 0x7fce881353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): picked mon.noname-a con 0x7fce8809ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a0890 0x7fce88135930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): picked mon.noname-c con 0x7fce880a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce880a1380 0x7fce8809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): picked mon.noname-b con 0x7fce880a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fce88140100 con 0x7fce8809ffc0 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fce8813ff80 con 0x7fce880a1380 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fce88140280 con 0x7fce880a0890 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): authenticate will time out at 1919.212209s 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809ffc0 0x7fce881353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a0890 0x7fce88135930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a0890 0x7fce88135930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37174/0 (socket says 172.21.15.17:37174) 2024-09-15T21:33:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): get_auth_request con 0x7fce8809ffc0 auth_method 0 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 -- 172.21.15.17:0/2438592332 learned_addr learned my addr 172.21.15.17:0/2438592332 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): get_auth_request con 0x7fce880a0890 auth_method 0 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient(hunting): handle_auth_done global_id 174258 payload 824 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a0890 msgr2=0x7fce88135930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a0890 0x7fce88135930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce880a1380 msgr2=0x7fce8809d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce880a1380 0x7fce8809d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient: _finish_hunting 0 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 monclient: found mon.noname-a 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce88121af0 con 0x7fce8809ffc0 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce880a0890 0x7fce88135930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8c825640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809ffc0 0x7fce881353f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce7000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700074e0 con 0x7fce8809ffc0 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: dump: 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.833 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: _finish_auth 0 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: _check_auth_tickets 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce700047f0 con 0x7fce8809ffc0 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70007680 con 0x7fce8809ffc0 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: dump: 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce86ffd640 4 set_mon_vals no callback set 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 5 monclient: authenticate success, global_id 174258 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient: _renew_subs 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8809ea40 con 0x7fce8809ffc0 2024-09-15T21:33:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient: _renew_subs 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8809d0f0 con 0x7fce8809ffc0 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 0x7fce8809dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fce880a0c70 con 0x7fce8809d940 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 0x7fce8809dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient: get_auth_request con 0x7fce8809d940 auth_method 0 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fce70011070 con 0x7fce8809ffc0 2024-09-15T21:33:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 10 monclient(hunting): handle_auth_done global_id 174261 payload 824 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce6803b9c0 0x7fce6803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce881208f0 con 0x7fce8809d940 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fce70050170 con 0x7fce8809ffc0 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce87fff640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 0x7fce8809dd20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce78004950 tx=0x7fce7800e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78019070 con 0x7fce8809d940 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: dump: 2024-09-15T21:33:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8d026640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce6803b9c0 0x7fce6803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8d026640 10 monclient: get_auth_request con 0x7fce6803b9c0 auth_method 0 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fce78015920 con 0x7fce8809d940 2024-09-15T21:33:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 msgr2=0x7fce8809dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 0x7fce8809dd20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce78004950 tx=0x7fce7800e8c0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 msgr2=0x7fce8809dd20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce857fa640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809d940 0x7fce8809dd20 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce78004950 tx=0x7fce7800e8c0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.814+0000 7fce8d026640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce6803b9c0 0x7fce6803de80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fce800ad000 tx=0x7fce80004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 0x7fce88138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fce88122110 con 0x7fce8809e260 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 0x7fce88138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient: get_auth_request con 0x7fce8809e260 auth_method 0 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 10 monclient(hunting): handle_auth_done global_id 174264 payload 824 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 1 -- 172.21.15.17:0/2438592332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fce8806d020 con 0x7fce8809e260 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce87fff640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 0x7fce88138ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce780049e0 tx=0x7fce78004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780295f0 con 0x7fce8809e260 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 10 monclient: dump: 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7fce780143b0 con 0x7fce8809e260 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 msgr2=0x7fce88138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 0x7fce88138ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce780049e0 tx=0x7fce78004ac0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 msgr2=0x7fce88138ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce857fa640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809e260 0x7fce88138ab0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce780049e0 tx=0x7fce78004ac0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 146, 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 85558, 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:15.845 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:15.846 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:15.847 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:15.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:15.849 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:15.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:15.851 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:15.852 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:15.855 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:15.856 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce6803b9c0 msgr2=0x7fce6803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fce6803b9c0 0x7fce6803de80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fce800ad000 tx=0x7fce80004740 comp rx=0 tx=0).stop 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 10 monclient: shutdown 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809ffc0 msgr2=0x7fce881353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809ffc0 0x7fce881353f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce7000c890 comp rx=0 tx=0).stop 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8d026640 1 -- 172.21.15.17:0/2438592332 reap_dead start 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 shutdown_connections 2024-09-15T21:33:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 --2- 172.21.15.17:0/2438592332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce8809ffc0 0x7fce881353f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 >> 172.21.15.17:0/2438592332 conn(0x7fce880a66a0 msgr2=0x7fce880a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:15.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 shutdown_connections 2024-09-15T21:33:15.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.822+0000 7fce8ea89640 1 -- 172.21.15.17:0/2438592332 wait complete. 2024-09-15T21:33:15.859 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:15.859 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:33:15.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.966+0000 7f0176d3e640 10 monclient: get_monmap_and_config 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.966+0000 7f0176d3e640 10 monclient: build_initial_monmap 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.966+0000 7f0176d3e640 10 monclient: monmap: 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:15.973321+0000 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:15.973321+0000 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.966+0000 7f0176d3e640 1 Processor -- start 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 1 -- start start 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient: init 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01700a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f017009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01700a1380 0x7f017013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient(hunting): picked mon.noname-a con 0x7f01700a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01700a3130 con 0x7f01700a0890 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0170073df0 con 0x7f017009ffc0 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f01700a1bf0 con 0x7f01700a1380 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient(hunting): authenticate will time out at 1919.368801s 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0174ada640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01700a0890 0x7f01700a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58198/0 (socket says 172.21.15.17:58198) 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0174ada640 1 -- 172.21.15.17:0/3128434138 learned_addr learned my addr 172.21.15.17:0/3128434138 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 1 --2- 172.21.15.17:0/3128434138 >> v2:172.21.15.17:3300/0 conn(0x7f01700a1380 0x7f017013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): get_auth_request con 0x7f01700a1380 auth_method 0 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient(hunting): handle_auth_done global_id 174267 payload 274 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 1 -- 172.21.15.17:0/3128434138 >> v1:172.21.15.119:6789/0 conn(0x7f017009ffc0 legacy=0x7f017013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 1 -- 172.21.15.17:0/3128434138 >> v1:172.21.15.17:6789/0 conn(0x7f01700a0890 legacy=0x7f01700a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient: _finish_hunting 0 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 1 monclient: found mon.noname-a 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 1 -- 172.21.15.17:0/3128434138 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017009ccd0 con 0x7f01700a1380 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f01752db640 1 --2- 172.21.15.17:0/3128434138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01700a1380 0x7f017013e1f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01680097f0 tx=0x7f016801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea36565f81fe928 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 1 -- 172.21.15.17:0/3128434138 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680b0070 con 0x7f01700a1380 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: dump: 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 1 -- 172.21.15.17:0/3128434138 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01680ab600 con 0x7f01700a1380 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 5 monclient: authenticate success, global_id 174267 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f0176d3e640 4 set_mon_vals no callback set 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 1 -- 172.21.15.17:0/3128434138 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680ab9d0 con 0x7f01700a1380 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.970+0000 7f016f7fe640 10 monclient: dump: 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: shutdown 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3128434138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01700a1380 msgr2=0x7f017013e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 --2- 172.21.15.17:0/3128434138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01700a1380 0x7f017013e1f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01680097f0 tx=0x7f016801c360 comp rx=0 tx=0).stop 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3128434138 shutdown_connections 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 --2- 172.21.15.17:0/3128434138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01700a1380 0x7f017013e1f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3128434138 >> 172.21.15.17:0/3128434138 conn(0x7f01700a66a0 msgr2=0x7f01700a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3128434138 shutdown_connections 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3128434138 wait complete. 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: build_initial_monmap 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: monmap: 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 Processor -- start 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- start start 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: init 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f017009ffc0 0x7f017009d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): picked mon.noname-a con 0x7f017009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01700a0890 0x7f017009d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): picked mon.noname-c con 0x7f01700a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01700a1380 0x7f017009de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): picked mon.noname-b con 0x7f01700a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f01701412e0 con 0x7f017009ffc0 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0170141160 con 0x7f01700a1380 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0170141460 con 0x7f01700a0890 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): authenticate will time out at 1919.374008s 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01700a0890 0x7f017009d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01700a0890 0x7f017009d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37182/0 (socket says 172.21.15.17:37182) 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 -- 172.21.15.17:0/3168292610 learned_addr learned my addr 172.21.15.17:0/3168292610 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): get_auth_request con 0x7f01700a0890 auth_method 0 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f017009ffc0 0x7f017009d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): get_auth_request con 0x7f017009ffc0 auth_method 0 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f01752db640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01700a1380 0x7f017009de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f01752db640 10 monclient(hunting): get_auth_request con 0x7f01700a1380 auth_method 0 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f01752db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f01752db640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f01752db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient(hunting): handle_auth_done global_id 164342 payload 824 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01700a1380 msgr2=0x7f017009de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01700a1380 0x7f017009de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f017009ffc0 msgr2=0x7f017009d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f017009ffc0 0x7f017009d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 monclient: found mon.noname-c 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017011cf20 con 0x7f01700a0890 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016ffff640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01700a0890 0x7f017009d940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f016000c550 tx=0x7f016000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160016020 con 0x7f01700a0890 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: dump: 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0160012830 con 0x7f01700a0890 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 5 monclient: authenticate success, global_id 164342 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: _renew_subs 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017009d0f0 con 0x7f01700a0890 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160012c20 con 0x7f01700a0890 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016d7fa640 10 monclient: dump: 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: _renew_subs 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f017011cc40 con 0x7f01700a0890 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f016e7fc640 4 set_mon_vals no callback set 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 0x7f0170131260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01700a0c70 con 0x7f0170130e80 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 0x7f0170131260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient: get_auth_request con 0x7f0170130e80 auth_method 0 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.974+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0160019020 con 0x7f01700a0890 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f0174ada640 10 monclient(hunting): handle_auth_done global_id 164345 payload 824 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f0174ada640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01700008e0 con 0x7f0170130e80 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f015003b9c0 0x7f015003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f01600505b0 con 0x7f01700a0890 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f0174ada640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 0x7f0170131260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0158005c10 tx=0x7f015800d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f01752db640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f015003b9c0 0x7f015003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015800dde0 con 0x7f0170130e80 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 10 monclient: dump: 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f01752db640 10 monclient: get_auth_request con 0x7f015003b9c0 auth_method 0 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0158018070 con 0x7f0170130e80 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 msgr2=0x7f0170131260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 0x7f0170131260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0158005c10 tx=0x7f015800d520 comp rx=0 tx=0).stop 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 msgr2=0x7f0170131260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f016d7fa640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0170130e80 0x7f0170131260 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0158005c10 tx=0x7f015800d520 comp rx=0 tx=0).stop 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.978+0000 7f01752db640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f015003b9c0 0x7f015003de80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f01680ad000 tx=0x7f0168008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0176d3e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0176d3e640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 0x7f0170133860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0176d3e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f017009e550 con 0x7f01701317a0 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 0x7f0170133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient: get_auth_request con 0x7f01701317a0 auth_method 0 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 10 monclient(hunting): handle_auth_done global_id 164348 payload 824 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 1 -- 172.21.15.17:0/3168292610 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0170131bb0 con 0x7f01701317a0 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f0174ada640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 0x7f0170133860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0158017000 tx=0x7f0158005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158007460 con 0x7f01701317a0 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f016d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f016d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.982+0000 7f016d7fa640 10 monclient: dump: 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f0158007460 con 0x7f01701317a0 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 msgr2=0x7f0170133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 0x7f0170133860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0158017000 tx=0x7f0158005b80 comp rx=0 tx=0).stop 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 msgr2=0x7f0170133860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f016d7fa640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01701317a0 0x7f0170133860 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0158017000 tx=0x7f0158005b80 comp rx=0 tx=0).stop 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 146, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 144566, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f015003b9c0 msgr2=0x7f015003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f015003b9c0 0x7f015003de80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f01680ad000 tx=0x7f0168008040 comp rx=0 tx=0).stop 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 10 monclient: shutdown 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01700a0890 msgr2=0x7f017009d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 --2- 172.21.15.17:0/3168292610 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01700a0890 0x7f017009d940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f016000c550 tx=0x7f016000ca10 comp rx=0 tx=0).stop 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f01752db640 1 -- 172.21.15.17:0/3168292610 reap_dead start 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f01752db640 1 -- 172.21.15.17:0/3168292610 reap_dead start 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 shutdown_connections 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 >> 172.21.15.17:0/3168292610 conn(0x7f01700a66a0 msgr2=0x7f017006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 shutdown_connections 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:15.986+0000 7f0176d3e640 1 -- 172.21.15.17:0/3168292610 wait complete. 2024-09-15T21:33:16.020 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:16.020 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:33:16.020 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.130+0000 7f1543bcf640 10 monclient: get_monmap_and_config 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.130+0000 7f1543bcf640 10 monclient: build_initial_monmap 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.130+0000 7f1543bcf640 10 monclient: monmap: 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:16.137247+0000 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:16.137247+0000 2024-09-15T21:33:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:16.138 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.130+0000 7f1543bcf640 1 Processor -- start 2024-09-15T21:33:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.130+0000 7f1543bcf640 1 -- start start 2024-09-15T21:33:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.130+0000 7f1543bcf640 10 monclient: init 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f153c0a2920 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f153c0a3410 0x7f153c13ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): picked mon.noname-c con 0x7f153c0a3410 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f153c13b070 0x7f153c13d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): picked mon.noname-a con 0x7f153c13b070 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f153c09ecf0 con 0x7f153c0a2920 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f153c09ea10 con 0x7f153c13b070 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f153c0a1230 con 0x7f153c0a3410 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f1543bcf640 10 monclient(hunting): authenticate will time out at 1919.532701s 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f153c13b070 0x7f153c13d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f153c13b070 0x7f153c13d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55790/0 (socket says 172.21.15.17:55790) 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 -- 172.21.15.17:0/3763788635 learned_addr learned my addr 172.21.15.17:0/3763788635 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): get_auth_request con 0x7f153c13b070 auth_method 0 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154116a640 1 --2- 172.21.15.17:0/3763788635 >> v2:172.21.15.119:3300/0 conn(0x7f153c0a3410 0x7f153c13ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154116a640 10 monclient(hunting): get_auth_request con 0x7f153c0a3410 auth_method 0 2024-09-15T21:33:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154116a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154116a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154116a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient(hunting): handle_auth_done global_id 174276 payload 274 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 -- 172.21.15.17:0/3763788635 >> v2:172.21.15.119:3300/0 conn(0x7f153c0a3410 msgr2=0x7f153c13ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 --2- 172.21.15.17:0/3763788635 >> v2:172.21.15.119:3300/0 conn(0x7f153c0a3410 0x7f153c13ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 -- 172.21.15.17:0/3763788635 >> v1:172.21.15.119:6789/0 conn(0x7f153c0a2920 legacy=0x7f153c0a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient: _finish_hunting 0 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 monclient: found mon.noname-a 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154216c640 1 -- 172.21.15.17:0/3763788635 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c06d1b0 con 0x7f153c13b070 2024-09-15T21:33:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.134+0000 7f154116a640 1 --2- 172.21.15.17:0/3763788635 >> v2:172.21.15.119:3300/0 conn(0x7f153c0a3410 0x7f153c13ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:33:16.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f154216c640 1 --2- 172.21.15.17:0/3763788635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c13b070 0x7f153c13d460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1534009870 tx=0x7f153401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be883626c73bd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 1 -- 172.21.15.17:0/3763788635 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15340ab600 con 0x7f153c13b070 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: dump: 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.146 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: _finish_auth 0 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: _check_auth_tickets 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 5 monclient: authenticate success, global_id 174276 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 1 -- 172.21.15.17:0/3763788635 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15340b0070 con 0x7f153c13b070 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 1 -- 172.21.15.17:0/3763788635 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15340ab9c0 con 0x7f153c13b070 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 4 set_mon_vals no callback set 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:16.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1540969640 10 monclient: dump: 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 10 monclient: shutdown 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 -- 172.21.15.17:0/3763788635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c13b070 msgr2=0x7f153c13d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 --2- 172.21.15.17:0/3763788635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c13b070 0x7f153c13d460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1534009870 tx=0x7f153401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 -- 172.21.15.17:0/3763788635 shutdown_connections 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 --2- 172.21.15.17:0/3763788635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c13b070 0x7f153c13d460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 --2- 172.21.15.17:0/3763788635 >> v2:172.21.15.119:3300/0 conn(0x7f153c0a3410 0x7f153c13ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 -- 172.21.15.17:0/3763788635 >> 172.21.15.17:0/3763788635 conn(0x7f153c0a7510 msgr2=0x7f153c0a3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 -- 172.21.15.17:0/3763788635 shutdown_connections 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 -- 172.21.15.17:0/3763788635 wait complete. 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 10 monclient: build_initial_monmap 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 10 monclient: monmap: 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 Processor -- start 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 1 -- start start 2024-09-15T21:33:16.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.138+0000 7f1543bcf640 10 monclient: init 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c0a2920 0x7f153c1cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): picked mon.noname-a con 0x7f153c0a2920 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f153c0a3410 0x7f153c132b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): picked mon.noname-c con 0x7f153c0a3410 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c13b070 0x7f153c133090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): picked mon.noname-b con 0x7f153c13b070 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f153c1401c0 con 0x7f153c0a2920 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f153c140340 con 0x7f153c13b070 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f153c1404c0 con 0x7f153c0a3410 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): authenticate will time out at 1919.540500s 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c0a2920 0x7f153c1cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f153c0a3410 0x7f153c132b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f153c0a3410 0x7f153c132b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37202/0 (socket says 172.21.15.17:37202) 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c0a2920 0x7f153c1cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55794/0 (socket says 172.21.15.17:55794) 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 -- 172.21.15.17:0/4035255946 learned_addr learned my addr 172.21.15.17:0/4035255946 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): get_auth_request con 0x7f153c0a3410 auth_method 0 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): get_auth_request con 0x7f153c0a2920 auth_method 0 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c13b070 0x7f153c133090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 10 monclient(hunting): get_auth_request con 0x7f153c13b070 auth_method 0 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient(hunting): handle_auth_done global_id 164351 payload 824 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c13b070 msgr2=0x7f153c133090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c13b070 0x7f153c133090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c0a2920 msgr2=0x7f153c1cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c0a2920 0x7f153c1cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient: _finish_hunting 0 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 monclient: found mon.noname-c 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c11d960 con 0x7f153c0a3410 2024-09-15T21:33:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f153c0a2920 0x7f153c1cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154116a640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f153c0a3410 0x7f153c132b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f152c00e810 tx=0x7f152c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c017070 con 0x7f153c0a3410 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: dump: 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152c00c8d0 con 0x7f153c0a3410 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c00f040 con 0x7f153c0a3410 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: dump: 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 5 monclient: authenticate success, global_id 164351 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1533fff640 4 set_mon_vals no callback set 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient: _renew_subs 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c1cfed0 con 0x7f153c0a3410 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient: _renew_subs 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c1d08a0 con 0x7f153c0a3410 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 0x7f153c072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f153c11eea0 con 0x7f153c11dd00 2024-09-15T21:33:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f152c01a070 con 0x7f153c0a3410 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 0x7f153c072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f152003b9c0 0x7f152003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient: get_auth_request con 0x7f153c11dd00 auth_method 0 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f152c0504d0 con 0x7f153c0a3410 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f152003b9c0 0x7f152003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 10 monclient: get_auth_request con 0x7f152003b9c0 auth_method 0 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 10 monclient(hunting): handle_auth_done global_id 184135 payload 824 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f153c0a2d00 con 0x7f153c11dd00 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154196b640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 0x7f153c072b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1538001f60 tx=0x7f153800d520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153800de10 con 0x7f153c11dd00 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f15327fc640 10 monclient: dump: 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.142+0000 7f154216c640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f152003b9c0 0x7f152003de80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f15340af000 tx=0x7f15340a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1538018070 con 0x7f153c11dd00 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 msgr2=0x7f153c072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 0x7f153c072b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1538001f60 tx=0x7f153800d520 comp rx=0 tx=0).stop 2024-09-15T21:33:16.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 msgr2=0x7f153c072b90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.146+0000 7f15327fc640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c11dd00 0x7f153c072b90 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1538001f60 tx=0x7f153800d520 comp rx=0 tx=0).stop 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f1543bcf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f1543bcf640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 0x7f153c02d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f1543bcf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f153c06cfb0 con 0x7f153c02d450 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 0x7f153c02d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient: get_auth_request con 0x7f153c02d450 auth_method 0 2024-09-15T21:33:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 10 monclient(hunting): handle_auth_done global_id 184138 payload 824 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.150+0000 7f154196b640 1 -- 172.21.15.17:0/4035255946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f153c134fe0 con 0x7f153c02d450 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f154196b640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 0x7f153c02d860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1538001f60 tx=0x7f1538001260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153801ed70 con 0x7f153c02d450 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 10 monclient: dump: 2024-09-15T21:33:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f15380050d0 con 0x7f153c02d450 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 msgr2=0x7f153c02d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 0x7f153c02d860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1538001f60 tx=0x7f1538001260 comp rx=0 tx=0).stop 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 msgr2=0x7f153c02d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f15327fc640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f153c02d450 0x7f153c02d860 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 146, 2024-09-15T21:33:16.170 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 23127, 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:16.171 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:16.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:16.173 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:16.174 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:16.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.177 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:16.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:16.180 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:16.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:16.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:16.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:16.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:16.183 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:16.183 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:16.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f152003b9c0 msgr2=0x7f152003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f152003b9c0 0x7f152003de80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f15340af000 tx=0x7f15340a7000 comp rx=0 tx=0).stop 2024-09-15T21:33:16.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 10 monclient: shutdown 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f153c0a3410 msgr2=0x7f153c132b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 --2- 172.21.15.17:0/4035255946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f153c0a3410 0x7f153c132b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f152c00e810 tx=0x7f152c00ecd0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f154216c640 1 -- 172.21.15.17:0/4035255946 reap_dead start 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f154216c640 1 -- 172.21.15.17:0/4035255946 reap_dead start 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 shutdown_connections 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 >> 172.21.15.17:0/4035255946 conn(0x7f153c0a7510 msgr2=0x7f153c0745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 shutdown_connections 2024-09-15T21:33:16.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.154+0000 7f1543bcf640 1 -- 172.21.15.17:0/4035255946 wait complete. 2024-09-15T21:33:16.185 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:16.186 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:33:16.186 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:33:16.186 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:33:16.186 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:33:16.186 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:33:16.186 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:33:16.203 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:33:16.203 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:33:16.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient: get_monmap_and_config 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient: build_initial_monmap 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient: monmap: 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:16.324032+0000 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:16.324032+0000 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:16.325 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 Processor -- start 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 -- start start 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient: init 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f138009cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): picked mon.noname-b con 0x7f13800a4b90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f13800a4f70 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f138013ce10 0x7f138013f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): picked mon.noname-c con 0x7f138013ce10 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13800a6000 con 0x7f13800a4f70 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f13800a5e30 con 0x7f138013ce10 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f138009ba60 con 0x7f13800a4b90 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f13855c8640 10 monclient(hunting): authenticate will time out at 1919.719290s 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137e7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f13800a4f70 0x7f13800a5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58202/0 (socket says 172.21.15.17:58202) 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f138009cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137e7fc640 1 -- 172.21.15.17:0/2055445579 learned_addr learned my addr 172.21.15.17:0/2055445579 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137f7fe640 1 -- 172.21.15.17:0/2055445579 >> v2:172.21.15.119:3300/0 conn(0x7f138013ce10 msgr2=0x7f138013f200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): get_auth_request con 0x7f13800a4b90 auth_method 0 2024-09-15T21:33:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137f7fe640 1 --2- 172.21.15.17:0/2055445579 >> v2:172.21.15.119:3300/0 conn(0x7f138013ce10 0x7f138013f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient(hunting): handle_auth_done global_id 164354 payload 274 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 1 -- 172.21.15.17:0/2055445579 >> v2:172.21.15.119:3300/0 conn(0x7f138013ce10 msgr2=0x7f138013f200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 1 --2- 172.21.15.17:0/2055445579 >> v2:172.21.15.119:3300/0 conn(0x7f138013ce10 0x7f138013f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137dffb640 1 -- 172.21.15.17:0/2055445579 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 337757158 0 0) 0x7f1374001120 con 0x7f13800a4f70 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 1 -- 172.21.15.17:0/2055445579 >> v1:172.21.15.17:6789/0 conn(0x7f13800a4f70 legacy=0x7f13800a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 1 monclient: found mon.noname-b 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137effd640 1 -- 172.21.15.17:0/2055445579 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138009c7b0 con 0x7f13800a4b90 2024-09-15T21:33:16.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.318+0000 7f137dffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137effd640 1 --2- 172.21.15.17:0/2055445579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f138009cc60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13700097c0 tx=0x7f137001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b3d3a5554f3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 1 -- 172.21.15.17:0/2055445579 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700b0070 con 0x7f13800a4b90 2024-09-15T21:33:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: dump: 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: _finish_auth 0 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:33:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 1 -- 172.21.15.17:0/2055445579 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13700ab600 con 0x7f13800a4b90 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 5 monclient: authenticate success, global_id 164354 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 1 -- 172.21.15.17:0/2055445579 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700ab9d0 con 0x7f13800a4b90 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 4 set_mon_vals no callback set 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f137dffb640 10 monclient: dump: 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 10 monclient: shutdown 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 -- 172.21.15.17:0/2055445579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 msgr2=0x7f138009cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 --2- 172.21.15.17:0/2055445579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f138009cc60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13700097c0 tx=0x7f137001c410 comp rx=0 tx=0).stop 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 -- 172.21.15.17:0/2055445579 shutdown_connections 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 --2- 172.21.15.17:0/2055445579 >> v2:172.21.15.119:3300/0 conn(0x7f138013ce10 0x7f138013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 --2- 172.21.15.17:0/2055445579 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f138009cc60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 -- 172.21.15.17:0/2055445579 >> 172.21.15.17:0/2055445579 conn(0x7f13800aae60 msgr2=0x7f13800a6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 -- 172.21.15.17:0/2055445579 shutdown_connections 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 1 -- 172.21.15.17:0/2055445579 wait complete. 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 10 monclient: build_initial_monmap 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.322+0000 7f13855c8640 10 monclient: monmap: 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:16.340 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 Processor -- start 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- start start 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: init 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f1380120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): picked mon.noname-c con 0x7f13800a4b90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13800a4f70 0x7f138009f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): picked mon.noname-b con 0x7f13800a4f70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f138013ce10 0x7f1380137c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): picked mon.noname-a con 0x7f138013ce10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13801422f0 con 0x7f138013ce10 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1380142170 con 0x7f13800a4f70 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1380142470 con 0x7f13800a4b90 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient(hunting): authenticate will time out at 1919.724725s 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f1380120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f138013ce10 0x7f1380137c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f1380120df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:37216/0 (socket says 172.21.15.17:37216) 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 -- 172.21.15.17:0/995231932 learned_addr learned my addr 172.21.15.17:0/995231932 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): get_auth_request con 0x7f13800a4b90 auth_method 0 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): get_auth_request con 0x7f138013ce10 auth_method 0 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137e7fc640 1 -- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13800a4f70 msgr2=0x7f138009f160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137e7fc640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13800a4f70 0x7f138009f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient(hunting): handle_auth_done global_id 164357 payload 824 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 -- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13800a4f70 msgr2=0x7f138009f160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13800a4f70 0x7f138009f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 -- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f138013ce10 msgr2=0x7f1380137c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f138013ce10 0x7f1380137c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 monclient: found mon.noname-c 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 -- 172.21.15.17:0/995231932 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13800a1380 con 0x7f13800a4b90 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137f7fe640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f138013ce10 0x7f1380137c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137effd640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f1380120df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f137001aac0 tx=0x7f137001ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700b0050 con 0x7f13800a4b90 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: dump: 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13700bc070 con 0x7f13800a4b90 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700aab30 con 0x7f13800a4b90 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 10 monclient: dump: 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137d7fa640 4 set_mon_vals no callback set 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 5 monclient: authenticate success, global_id 164357 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _renew_subs 2024-09-15T21:33:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13800a0760 con 0x7f13800a4b90 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _renew_subs 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13801381a0 con 0x7f13800a4b90 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13800a01b0 con 0x7f13800a4b90 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f13700aacd0 con 0x7f13800a4b90 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f136c03b9a0 0x7f136c03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f13700eb2e0 con 0x7f13800a4b90 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137e7fc640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f136c03b9a0 0x7f136c03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.326+0000 7f137e7fc640 10 monclient: get_auth_request con 0x7f136c03b9a0 auth_method 0 2024-09-15T21:33:16.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.330+0000 7f137e7fc640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f136c03b9a0 0x7f136c03de60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f13740012c0 tx=0x7f13740078e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:16.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.330+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1370016090 con 0x7f13800a4b90 2024-09-15T21:33:16.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.330+0000 7f13677fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:16.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.330+0000 7f13677fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:16.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13855c8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:16.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13855c8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:16.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13855c8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:16.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f138009cc60 con 0x7f13800a4b90 2024-09-15T21:33:16.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13677fe640 1 -- 172.21.15.17:0/995231932 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f13700ab5a0 con 0x7f13800a4b90 2024-09-15T21:33:16.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13677fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:16.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.562+0000 7f13677fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:16.570 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f136c03b9a0 msgr2=0x7f136c03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f136c03b9a0 0x7f136c03de60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f13740012c0 tx=0x7f13740078e0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 10 monclient: shutdown 2024-09-15T21:33:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 msgr2=0x7f1380120df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f1380120df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f137001aac0 tx=0x7f137001ac50 comp rx=0 tx=0).stop 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 shutdown_connections 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f136c03b9a0 0x7f136c03de60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f138013ce10 0x7f1380137c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13800a4f70 0x7f138009f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 --2- 172.21.15.17:0/995231932 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f13800a4b90 0x7f1380120df0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 >> 172.21.15.17:0/995231932 conn(0x7f13800aae60 msgr2=0x7f13800a37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 shutdown_connections 2024-09-15T21:33:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:16.566+0000 7f13855c8640 1 -- 172.21.15.17:0/995231932 wait complete. 2024-09-15T21:33:19.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.714+0000 7f7dbadf0640 10 monclient: get_monmap_and_config 2024-09-15T21:33:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.714+0000 7f7dbadf0640 10 monclient: build_initial_monmap 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.714+0000 7f7dbadf0640 10 monclient: monmap: 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:19.720657+0000 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:19.720657+0000 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:19.722 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.714+0000 7f7dbadf0640 1 Processor -- start 2024-09-15T21:33:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.714+0000 7f7dbadf0640 1 -- start start 2024-09-15T21:33:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.714+0000 7f7dbadf0640 10 monclient: init 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): picked mon.noname-b con 0x7f7db41378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7db41381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7db40a5210 0x7f7db40a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): picked mon.noname-a con 0x7f7db40a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db40a1d00 con 0x7f7db41381e0 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7db40a1fe0 con 0x7f7db40a5210 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7db40a22c0 con 0x7f7db41378c0 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient(hunting): authenticate will time out at 1923.116136s 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7db41381e0 0x7f7db40a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51616/0 (socket says 172.21.15.17:51616) 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db3fff640 1 -- 172.21.15.17:0/1549473510 learned_addr learned my addr 172.21.15.17:0/1549473510 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): get_auth_request con 0x7f7db41378c0 auth_method 0 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:19.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 --2- 172.21.15.17:0/1549473510 >> v2:172.21.15.17:3300/0 conn(0x7f7db40a5210 0x7f7db40a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): get_auth_request con 0x7f7db40a5210 auth_method 0 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 1 -- 172.21.15.17:0/1549473510 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851789837 0 0) 0x7f7db40a1d00 con 0x7f7db41381e0 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient(hunting): _negotiate 2024-09-15T21:33:19.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient(hunting): my global_id is 174291 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 1 -- 172.21.15.17:0/1549473510 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7da0003530 con 0x7f7db41381e0 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient(hunting): handle_auth_done global_id 174288 payload 274 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 -- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 msgr2=0x7f7db4137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 --2- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db4137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 -- 172.21.15.17:0/1549473510 >> v1:172.21.15.17:6789/0 conn(0x7f7db41381e0 legacy=0x7f7db40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient: _finish_hunting 0 2024-09-15T21:33:19.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 monclient: found mon.noname-a 2024-09-15T21:33:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 -- 172.21.15.17:0/1549473510 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db40a5c00 con 0x7f7db40a5210 2024-09-15T21:33:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db8b8c640 1 --2- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db4137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db938d640 1 --2- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db40a5210 0x7f7db40a56c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e2023cae7b4dca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 1 -- 172.21.15.17:0/1549473510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0b2070 con 0x7f7db40a5210 2024-09-15T21:33:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: dump: 2024-09-15T21:33:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 5 monclient: authenticate success, global_id 174288 2024-09-15T21:33:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 1 -- 172.21.15.17:0/1549473510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac0a5770 con 0x7f7db40a5210 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 1 -- 172.21.15.17:0/1549473510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0a5b20 con 0x7f7db40a5210 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7db37fe640 10 monclient: dump: 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:19.736 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 4 set_mon_vals no callback set 2024-09-15T21:33:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 10 monclient: shutdown 2024-09-15T21:33:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:19.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db40a5210 msgr2=0x7f7db40a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:19.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db40a5210 0x7f7db40a56c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac01c830 comp rx=0 tx=0).stop 2024-09-15T21:33:19.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- 172.21.15.17:0/1549473510 shutdown_connections 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db40a5210 0x7f7db40a56c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/1549473510 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db4137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- 172.21.15.17:0/1549473510 >> 172.21.15.17:0/1549473510 conn(0x7f7db40aac40 msgr2=0x7f7db40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.718+0000 7f7dbadf0640 1 -- 172.21.15.17:0/1549473510 shutdown_connections 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- 172.21.15.17:0/1549473510 wait complete. 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: build_initial_monmap 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: monmap: 2024-09-15T21:33:19.741 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:19.742 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:19.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 Processor -- start 2024-09-15T21:33:19.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- start start 2024-09-15T21:33:19.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: init 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40a5210 0x7f7db4073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): picked mon.noname-b con 0x7f7db40a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db413cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): picked mon.noname-c con 0x7f7db41378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 0x7f7db413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): picked mon.noname-a con 0x7f7db41381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7db406f340 con 0x7f7db41381e0 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:19.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7db406f1c0 con 0x7f7db40a5210 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7db41425c0 con 0x7f7db41378c0 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient(hunting): authenticate will time out at 1923.121576s 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db413cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 0x7f7db413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db8b8c640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40a5210 msgr2=0x7f7db4073b00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db8b8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40a5210 0x7f7db4073b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db413cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36464/0 (socket says 172.21.15.17:36464) 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 0x7f7db413d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47726/0 (socket says 172.21.15.17:47726) 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 -- 172.21.15.17:0/2577336480 learned_addr learned my addr 172.21.15.17:0/2577336480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:19.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): get_auth_request con 0x7f7db41378c0 auth_method 0 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): get_auth_request con 0x7f7db41381e0 auth_method 0 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:19.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient(hunting): handle_auth_done global_id 164363 payload 824 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 -- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40a5210 msgr2=0x7f7db4073b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40a5210 0x7f7db4073b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 -- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 msgr2=0x7f7db413d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 0x7f7db413d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 monclient: found mon.noname-c 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:19.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 -- 172.21.15.17:0/2577336480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db4070510 con 0x7f7db41378c0 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db938d640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 0x7f7db413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db3fff640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db413cb70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c500 tx=0x7f7d9c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c017070 con 0x7f7db41378c0 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: dump: 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:19.751 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.752 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.752 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:19.752 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:19.752 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:19.752 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: _finish_auth 0 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: _check_auth_tickets 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c005cf0 con 0x7f7db41378c0 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 5 monclient: authenticate success, global_id 164363 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _renew_subs 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db40707f0 con 0x7f7db41378c0 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c007270 con 0x7f7db41378c0 2024-09-15T21:33:19.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 10 monclient: dump: 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:19.754 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db2ffd640 4 set_mon_vals no callback set 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _renew_subs 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db40a8910 con 0x7f7db41378c0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db40a56c0 con 0x7f7db41378c0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7d9c01a070 con 0x7f7db41378c0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d9403b970 0x7f7d9403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.722+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7d9c04f920 con 0x7f7db41378c0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.726+0000 7f7db8b8c640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d9403b970 0x7f7d9403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.726+0000 7f7db8b8c640 10 monclient: get_auth_request con 0x7f7d9403b970 auth_method 0 2024-09-15T21:33:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.726+0000 7f7db8b8c640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d9403b970 0x7f7d9403de30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009650 tx=0x7f7da4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:19.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.730+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7d9c01b3b0 con 0x7f7db41378c0 2024-09-15T21:33:19.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.730+0000 7f7db17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:19.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.730+0000 7f7db17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.954+0000 7f7dbadf0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.954+0000 7f7dbadf0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.954+0000 7f7dbadf0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.954+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7db40a1270 con 0x7f7db41378c0 2024-09-15T21:33:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7db17fa640 1 -- 172.21.15.17:0/2577336480 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f7d9c01b730 con 0x7f7db41378c0 2024-09-15T21:33:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7db17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7db17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:19.963 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d9403b970 msgr2=0x7f7d9403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d9403b970 0x7f7d9403de30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009650 tx=0x7f7da4009190 comp rx=0 tx=0).stop 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 10 monclient: shutdown 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 msgr2=0x7f7db413cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db413cb70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c500 tx=0x7f7d9c00c9c0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 shutdown_connections 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7d9403b970 0x7f7d9403de30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7db41381e0 0x7f7db413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7db41378c0 0x7f7db413cb70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 --2- 172.21.15.17:0/2577336480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40a5210 0x7f7db4073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 >> 172.21.15.17:0/2577336480 conn(0x7f7db40aac40 msgr2=0x7f7db406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 shutdown_connections 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:19.958+0000 7f7dbadf0640 1 -- 172.21.15.17:0/2577336480 wait complete. 2024-09-15T21:33:22.980 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient: get_monmap_and_config 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient: build_initial_monmap 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient: monmap: 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:23.112109+0000 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:23.112109+0000 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 Processor -- start 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 -- start start 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient: init 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): picked mon.noname-b con 0x7fb174073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb174073f60 0x7fb174137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): picked mon.noname-c con 0x7fb174073f60 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb174137bd0 0x7fb174138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): picked mon.noname-a con 0x7fb174137bd0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb1740a3640 con 0x7fb174137bd0 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb1740a0890 con 0x7fb174073f60 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb1740746e0 con 0x7fb174073640 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb179472640 10 monclient(hunting): authenticate will time out at 1926.507561s 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb174137bd0 0x7fb174138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb174137bd0 0x7fb174138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47728/0 (socket says 172.21.15.17:47728) 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 1 -- 172.21.15.17:0/1879909934 learned_addr learned my addr 172.21.15.17:0/1879909934 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): get_auth_request con 0x7fb174137bd0 auth_method 0 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1727fc640 1 -- 172.21.15.17:0/1879909934 >> v2:172.21.15.119:3300/0 conn(0x7fb174073f60 msgr2=0x7fb174137690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1727fc640 1 --2- 172.21.15.17:0/1879909934 >> v2:172.21.15.119:3300/0 conn(0x7fb174073f60 0x7fb174137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb172ffd640 1 --2- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb1737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb172ffd640 10 monclient(hunting): get_auth_request con 0x7fb174073640 auth_method 0 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb172ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb172ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.106+0000 7fb172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb172ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb172ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb172ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 10 monclient(hunting): handle_auth_done global_id 174297 payload 274 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 -- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 msgr2=0x7fb174073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 --2- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 -- 172.21.15.17:0/1879909934 >> v2:172.21.15.119:3300/0 conn(0x7fb174073f60 msgr2=0x7fb174137690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 --2- 172.21.15.17:0/1879909934 >> v2:172.21.15.119:3300/0 conn(0x7fb174073f60 0x7fb174137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 monclient: found mon.noname-a 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 -- 172.21.15.17:0/1879909934 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1740a1640 con 0x7fb174137bd0 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb1737fe640 1 --2- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174137bd0 0x7fb174138000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb17406e7b0 tx=0x7fb16401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21334cfad972c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 1 -- 172.21.15.17:0/1879909934 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640b0070 con 0x7fb174137bd0 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: dump: 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: _finish_auth 0 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 1 -- 172.21.15.17:0/1879909934 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1640ab600 con 0x7fb174137bd0 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 1 -- 172.21.15.17:0/1879909934 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640ab9d0 con 0x7fb174137bd0 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 5 monclient: authenticate success, global_id 174297 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 4 set_mon_vals no callback set 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb171ffb640 10 monclient: dump: 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 10 monclient: shutdown 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 1 -- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174137bd0 msgr2=0x7fb174138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.110+0000 7fb179472640 1 --2- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174137bd0 0x7fb174138000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb17406e7b0 tx=0x7fb16401c490 comp rx=0 tx=0).stop 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- 172.21.15.17:0/1879909934 shutdown_connections 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 --2- 172.21.15.17:0/1879909934 >> v2:172.21.15.119:3300/0 conn(0x7fb174073f60 0x7fb174137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 --2- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174137bd0 0x7fb174138000 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 --2- 172.21.15.17:0/1879909934 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- 172.21.15.17:0/1879909934 >> 172.21.15.17:0/1879909934 conn(0x7fb1740a66a0 msgr2=0x7fb1740a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- 172.21.15.17:0/1879909934 shutdown_connections 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- 172.21.15.17:0/1879909934 wait complete. 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: build_initial_monmap 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: monmap: 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 Processor -- start 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- start start 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: init 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174120930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): picked mon.noname-c con 0x7fb174073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 0x7fb17411e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): picked mon.noname-a con 0x7fb174073f60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb174137bd0 0x7fb174136790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): picked mon.noname-b con 0x7fb174137bd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb17409ce50 con 0x7fb174073f60 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb17409ccd0 con 0x7fb174137bd0 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb1740a0d00 con 0x7fb174073640 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient(hunting): authenticate will time out at 1926.514200s 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 0x7fb17411e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 0x7fb17411e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47734/0 (socket says 172.21.15.17:47734) 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 -- 172.21.15.17:0/1716352981 learned_addr learned my addr 172.21.15.17:0/1716352981 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): get_auth_request con 0x7fb174073f60 auth_method 0 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174120930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1737fe640 1 -- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb174137bd0 msgr2=0x7fb174136790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1737fe640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb174137bd0 0x7fb174136790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): get_auth_request con 0x7fb174073640 auth_method 0 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb172ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient(hunting): handle_auth_done global_id 174300 payload 824 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 -- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 msgr2=0x7fb174120930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174120930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 -- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb174137bd0 msgr2=0x7fb174136790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb174137bd0 0x7fb174136790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient: _finish_hunting 0 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 monclient: found mon.noname-a 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 -- 172.21.15.17:0/1716352981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17409c540 con 0x7fb174073f60 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1727fc640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 0x7fb17411e7d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b530 tx=0x7fb15c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c013020 con 0x7fb174073f60 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: dump: 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c003df0 con 0x7fb174073f60 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0052b0 con 0x7fb174073f60 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 5 monclient: authenticate success, global_id 174300 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: _renew_subs 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb1717fa640 4 set_mon_vals no callback set 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17409d0f0 con 0x7fb174073f60 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb16b7fe640 10 monclient: dump: 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: _renew_subs 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.114+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb174122130 con 0x7fb174073f60 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb179472640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb179472640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb179472640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb174137690 con 0x7fb174073f60 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb15c016020 con 0x7fb174073f60 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb16b7fe640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb15403b9a0 0x7fb15403de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb15c050240 con 0x7fb174073f60 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb172ffd640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb15403b9a0 0x7fb15403de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb172ffd640 10 monclient: get_auth_request con 0x7fb15403b9a0 auth_method 0 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.118+0000 7fb172ffd640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb15403b9a0 0x7fb15403de60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0113e0 tx=0x7fb16c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.122+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fb15c01d020 con 0x7fb174073f60 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.122+0000 7fb16b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.122+0000 7fb16b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb179472640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb179472640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb179472640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb174136cd0 con 0x7fb174073f60 2024-09-15T21:33:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb16b7fe640 1 -- 172.21.15.17:0/1716352981 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fb17409c540 con 0x7fb174073f60 2024-09-15T21:33:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb16b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.358+0000 7fb16b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:23.366 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb15403b9a0 msgr2=0x7fb15403de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb15403b9a0 0x7fb15403de60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0113e0 tx=0x7fb16c00b040 comp rx=0 tx=0).stop 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 10 monclient: shutdown 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 msgr2=0x7fb17411e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 0x7fb17411e7d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b530 tx=0x7fb15c00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 shutdown_connections 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb15403b9a0 0x7fb15403de60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb174137bd0 0x7fb174136790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb174073f60 0x7fb17411e7d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 --2- 172.21.15.17:0/1716352981 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb174073640 0x7fb174120930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 >> 172.21.15.17:0/1716352981 conn(0x7fb1740a66a0 msgr2=0x7fb1740a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 shutdown_connections 2024-09-15T21:33:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:23.362+0000 7fb179472640 1 -- 172.21.15.17:0/1716352981 wait complete. 2024-09-15T21:33:26.385 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:26.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient: get_monmap_and_config 2024-09-15T21:33:26.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient: build_initial_monmap 2024-09-15T21:33:26.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient: monmap: 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:26.519693+0000 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:26.519693+0000 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:26.520 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 Processor -- start 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 -- start start 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient: init 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): picked mon.noname-b con 0x7f1ac009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1ac009dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:26.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1ac013cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): picked mon.noname-c con 0x7f1ac0134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac00a3130 con 0x7f1ac009dd20 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac00a1bc0 con 0x7f1ac0134d70 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac00a3490 con 0x7f1ac009d400 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac6899640 10 monclient(hunting): authenticate will time out at 1929.915112s 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abf7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1ac009dd20 0x7f1ac0134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51626/0 (socket says 172.21.15.17:51626) 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 learned_addr learned my addr 172.21.15.17:0/1455207906 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac4e36640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1ac013cad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:26.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1ac4e36640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1ac013cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1ac009d400 auth_method 0 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:26.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.514+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1ac4e36640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1ac013cad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1ac4e36640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1ac013cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1ac009d400 auth_method 0 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:26.718+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:33:27.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1ac4e36640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1ac013cad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:27.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1ac4e36640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1ac013cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:33:27.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:27.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1ac009d400 auth_method 0 2024-09-15T21:33:27.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:27.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:27.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:27.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:27.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:27.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:27.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.122+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:33:27.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1ac4e36640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1ac013cad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1ac009d400 auth_method 0 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1ac4e36640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1ac013cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:27.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:27.926+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): tick 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 monclient(hunting): continuing hunt 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac009d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac009d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1ac013cad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1ac013cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.17:6789/0 conn(0x7f1ac009dd20 legacy=0x7f1ac0134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient: _add_conns ranks=[3,1] 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): picked mon.noname-a con 0x7f1aa00036d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1aa0006240 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac009e7a0 con 0x7f1aa0006240 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac00a3490 con 0x7f1aa00036d0 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abdffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1aa00036d0 auth_method 0 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.518+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:29.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:29.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:29.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1aa00036d0 auth_method 0 2024-09-15T21:33:29.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:29.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:29.722+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1aa00036d0 auth_method 0 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.126+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:33:30.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:30.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:30.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1aa00036d0 auth_method 0 2024-09-15T21:33:30.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:30.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:30.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:30.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:33:30.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:30.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:30.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:30.934+0000 7f1abffff640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): tick 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 monclient(hunting): continuing hunt 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 msgr2=0x7f1aa0005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.17:3300/0 conn(0x7f1aa00036d0 0x7f1aa0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1aa0006240 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1ac4e36640 1 -- 172.21.15.17:0/1455207906 reap_dead start 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1aa0009f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): picked mon.noname-b con 0x7f1ac009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1aa000a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): picked mon.noname-c con 0x7f1ac0134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa000b070 con 0x7f1aa0006240 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac00a3490 con 0x7f1ac0134d70 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 1 -- 172.21.15.17:0/1455207906 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa000aad0 con 0x7f1ac009d400 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abdffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abffff640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1ac4e36640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1aa000a500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1ac4e36640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1aa000a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1aa0009f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): get_auth_request con 0x7f1ac009d400 auth_method 0 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_done global_id 164372 payload 274 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 msgr2=0x7f1aa000a500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1aa000a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 >> v1:172.21.15.119:6789/0 conn(0x7f1aa0006240 legacy=0x7f1aa0008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:33:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 1 monclient: found mon.noname-b 2024-09-15T21:33:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.518+0000 7f1abf7fe640 1 -- 172.21.15.17:0/1455207906 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac009e7a0 con 0x7f1ac009d400 2024-09-15T21:33:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abf7fe640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1aa0009f40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8001170 tx=0x7f1aa801c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dac43bf1a78af13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 1 -- 172.21.15.17:0/1455207906 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8004ab0 con 0x7f1ac009d400 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: dump: 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 1 -- 172.21.15.17:0/1455207906 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa8004c50 con 0x7f1ac009d400 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 1 -- 172.21.15.17:0/1455207906 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8007070 con 0x7f1ac009d400 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1abeffd640 10 monclient: dump: 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1ac6899640 5 monclient: authenticate success, global_id 164372 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1ac6899640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.522+0000 7f1ac6899640 4 set_mon_vals no callback set 2024-09-15T21:33:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient: shutdown 2024-09-15T21:33:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1aa0009f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1aa0009f40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8001170 tx=0x7f1aa801c6a0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- 172.21.15.17:0/1455207906 shutdown_connections 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1455207906 >> v2:172.21.15.119:3300/0 conn(0x7f1ac0134d70 0x7f1aa000a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1455207906 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1aa0009f40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- 172.21.15.17:0/1455207906 >> 172.21.15.17:0/1455207906 conn(0x7f1ac00a66a0 msgr2=0x7f1ac00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- 172.21.15.17:0/1455207906 shutdown_connections 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- 172.21.15.17:0/1455207906 wait complete. 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient: build_initial_monmap 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient: monmap: 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 Processor -- start 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- start start 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient: init 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 0x7f1ac01cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): picked mon.noname-a con 0x7f1aa0006240 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac0133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): picked mon.noname-c con 0x7f1ac009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ac0134d70 0x7f1ac01cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): picked mon.noname-b con 0x7f1ac0134d70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac009fb80 con 0x7f1aa0006240 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac009fa00 con 0x7f1ac0134d70 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac0134660 con 0x7f1ac009d400 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac6899640 10 monclient(hunting): authenticate will time out at 1935.927171s 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac4e36640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ac0134d70 msgr2=0x7f1ac01cca20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1ac4e36640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ac0134d70 0x7f1ac01cca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 0x7f1ac01cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac0133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 0x7f1ac01cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45592/0 (socket says 172.21.15.17:45592) 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 1 -- 172.21.15.17:0/1875701831 learned_addr learned my addr 172.21.15.17:0/1875701831 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): get_auth_request con 0x7f1aa0006240 auth_method 0 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): get_auth_request con 0x7f1ac009d400 auth_method 0 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 10 monclient(hunting): handle_auth_done global_id 174303 payload 824 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 1 -- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 msgr2=0x7f1ac0133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.526+0000 7f1abffff640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac0133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 1 -- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ac0134d70 msgr2=0x7f1ac01cca20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ac0134d70 0x7f1ac01cca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 1 monclient: found mon.noname-a 2024-09-15T21:33:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 1 -- 172.21.15.17:0/1875701831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac0127420 con 0x7f1aa0006240 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abffff640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 0x7f1ac01cffa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000acc0 tx=0x7f1ab0017780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0017e90 con 0x7f1aa0006240 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: dump: 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: _finish_auth 0 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 5 monclient: authenticate success, global_id 174303 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 monclient: _renew_subs 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab000adb0 con 0x7f1aa0006240 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac0129780 con 0x7f1aa0006240 2024-09-15T21:33:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0021650 con 0x7f1aa0006240 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 10 monclient: dump: 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 monclient: _renew_subs 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abe7fc640 4 set_mon_vals no callback set 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac0073ed0 con 0x7f1aa0006240 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1ab0015040 con 0x7f1aa0006240 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac0134a70 con 0x7f1aa0006240 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1aa003cab0 0x7f1aa003ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f1ab0051c50 con 0x7f1aa0006240 2024-09-15T21:33:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abf7fe640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1aa003cab0 0x7f1aa003ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abf7fe640 10 monclient: get_auth_request con 0x7f1aa003cab0 auth_method 0 2024-09-15T21:33:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.530+0000 7f1abf7fe640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1aa003cab0 0x7f1aa003ef70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00061e0 tx=0x7f1aa80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.534+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1ab0029db0 con 0x7f1aa0006240 2024-09-15T21:33:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.534+0000 7f1abcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.534+0000 7f1abcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:32.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:32.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:32.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:32.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1ac0128330 con 0x7f1aa0006240 2024-09-15T21:33:32.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1abcff9640 1 -- 172.21.15.17:0/1875701831 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f1ab001f380 con 0x7f1aa0006240 2024-09-15T21:33:32.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1abcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:32.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1abcff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:32.780 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1aa003cab0 msgr2=0x7f1aa003ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1aa003cab0 0x7f1aa003ef70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00061e0 tx=0x7f1aa80a6000 comp rx=0 tx=0).stop 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 10 monclient: shutdown 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 msgr2=0x7f1ac01cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 0x7f1ac01cffa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000acc0 tx=0x7f1ab0017780 comp rx=0 tx=0).stop 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 shutdown_connections 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1aa003cab0 0x7f1aa003ef70 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ac0134d70 0x7f1ac01cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1ac009d400 0x7f1ac0133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 --2- 172.21.15.17:0/1875701831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0006240 0x7f1ac01cffa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.774+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 >> 172.21.15.17:0/1875701831 conn(0x7f1ac00a66a0 msgr2=0x7f1ac00729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:32.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.778+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 shutdown_connections 2024-09-15T21:33:32.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.778+0000 7f1ac6899640 1 -- 172.21.15.17:0/1875701831 wait complete. 2024-09-15T21:33:32.799 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:33:32.799 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:33:32.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:33:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: get_monmap_and_config 2024-09-15T21:33:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: build_initial_monmap 2024-09-15T21:33:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: monmap: 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:32.918518+0000 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:32.918518+0000 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:32.920 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 Processor -- start 2024-09-15T21:33:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- start start 2024-09-15T21:33:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: init 2024-09-15T21:33:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6e80a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e809ffc0 0x7fb6e813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): picked mon.noname-b con 0x7fb6e809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6e80a1380 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e80a3130 con 0x7fb6e80a1380 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e8073df0 con 0x7fb6e80a0890 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e80a1bf0 con 0x7fb6e809ffc0 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): authenticate will time out at 1936.313867s 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e809ffc0 0x7fb6e813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fb6e80a0890 legacy=0x7fb6e80a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb6e80a1380 0x7fb6e813d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38982/0 (socket says 172.21.15.17:38982) 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 -- 172.21.15.17:0/4118104866 learned_addr learned my addr 172.21.15.17:0/4118104866 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): get_auth_request con 0x7fb6e809ffc0 auth_method 0 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient(hunting): handle_auth_done global_id 164378 payload 274 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 -- 172.21.15.17:0/4118104866 >> v1:172.21.15.119:6789/0 conn(0x7fb6e80a0890 legacy=0x7fb6e80a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 -- 172.21.15.17:0/4118104866 >> v1:172.21.15.17:6789/0 conn(0x7fb6e80a1380 legacy=0x7fb6e813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient: _finish_hunting 0 2024-09-15T21:33:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 monclient: found mon.noname-b 2024-09-15T21:33:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 -- 172.21.15.17:0/4118104866 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e809ccd0 con 0x7fb6e809ffc0 2024-09-15T21:33:32.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 --2- 172.21.15.17:0/4118104866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e809ffc0 0x7fb6e813aac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80097f0 tx=0x7fb6d801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=971299eab0262570 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 1 -- 172.21.15.17:0/4118104866 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80b0070 con 0x7fb6e809ffc0 2024-09-15T21:33:32.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:32.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: dump: 2024-09-15T21:33:32.930 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: _finish_auth 0 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: _check_auth_tickets 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 1 -- 172.21.15.17:0/4118104866 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d80ab600 con 0x7fb6e809ffc0 2024-09-15T21:33:32.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 5 monclient: authenticate success, global_id 164378 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 1 -- 172.21.15.17:0/4118104866 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80ab9d0 con 0x7fb6e809ffc0 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 4 set_mon_vals no callback set 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ed244640 10 monclient: dump: 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.932 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: shutdown 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/4118104866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e809ffc0 msgr2=0x7fb6e813aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 --2- 172.21.15.17:0/4118104866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e809ffc0 0x7fb6e813aac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80097f0 tx=0x7fb6d801c3a0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/4118104866 shutdown_connections 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 --2- 172.21.15.17:0/4118104866 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e809ffc0 0x7fb6e813aac0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/4118104866 >> 172.21.15.17:0/4118104866 conn(0x7fb6e80a66a0 msgr2=0x7fb6e80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:32.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/4118104866 shutdown_connections 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/4118104866 wait complete. 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: build_initial_monmap 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: monmap: 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:32.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 Processor -- start 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- start start 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: init 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e809ffc0 0x7fb6e809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): picked mon.noname-a con 0x7fb6e809ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6e80a0890 0x7fb6e809d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): picked mon.noname-b con 0x7fb6e80a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e80a1380 0x7fb6e809de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): picked mon.noname-c con 0x7fb6e80a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81412e0 con 0x7fb6e809ffc0 2024-09-15T21:33:32.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8141160 con 0x7fb6e80a0890 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e8141460 con 0x7fb6e80a1380 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient(hunting): authenticate will time out at 1936.318774s 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e809ffc0 0x7fb6e809d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e80a1380 0x7fb6e809de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e809ffc0 0x7fb6e809d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45608/0 (socket says 172.21.15.17:45608) 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e80a1380 0x7fb6e809de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47506/0 (socket says 172.21.15.17:47506) 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 1 -- 172.21.15.17:0/181339634 learned_addr learned my addr 172.21.15.17:0/181339634 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): get_auth_request con 0x7fb6e809ffc0 auth_method 0 2024-09-15T21:33:32.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6e80a0890 msgr2=0x7fb6e809d940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): get_auth_request con 0x7fb6e80a1380 auth_method 0 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eda45640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6e80a0890 0x7fb6e809d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient(hunting): handle_auth_done global_id 164381 payload 824 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6e80a0890 msgr2=0x7fb6e809d940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6e80a0890 0x7fb6e809d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e809ffc0 msgr2=0x7fb6e809d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e809ffc0 0x7fb6e809d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient: _finish_hunting 0 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 monclient: found mon.noname-c 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e8120f50 con 0x7fb6e80a1380 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6eea47640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e80a1380 0x7fb6e809de80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000cae0 tx=0x7fb6e000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0017070 con 0x7fb6e80a1380 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: dump: 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:32.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e000abe0 con 0x7fb6e80a1380 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 5 monclient: authenticate success, global_id 164381 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _renew_subs 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e8121230 con 0x7fb6e80a1380 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0012600 con 0x7fb6e80a1380 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6deffd640 10 monclient: dump: 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.940 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _renew_subs 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6dffff640 4 set_mon_vals no callback set 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.918+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e809e480 con 0x7fb6e80a1380 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6f04aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6f04aa640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 0x7fb6e8135930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6f04aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80a0c70 con 0x7fb6e813ca00 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 0x7fb6e8135930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient: get_auth_request con 0x7fb6e813ca00 auth_method 0 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fb6e001a070 con 0x7fb6e80a1380 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 10 monclient(hunting): handle_auth_done global_id 174312 payload 824 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6e813da80 con 0x7fb6e813ca00 2024-09-15T21:33:32.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb6cc03b9a0 0x7fb6cc03de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fb6e00501d0 con 0x7fb6e80a1380 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6ee246640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 0x7fb6e8135930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009710 tx=0x7fb6e4009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4016070 con 0x7fb6e813ca00 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 10 monclient: dump: 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.943 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6eda45640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb6cc03b9a0 0x7fb6cc03de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6eda45640 10 monclient: get_auth_request con 0x7fb6cc03b9a0 auth_method 0 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb6e400e7a0 con 0x7fb6e813ca00 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 msgr2=0x7fb6e8135930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 0x7fb6e8135930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009710 tx=0x7fb6e4009b10 comp rx=0 tx=0).stop 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 msgr2=0x7fb6e8135930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6deffd640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813ca00 0x7fb6e8135930 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009710 tx=0x7fb6e4009b10 comp rx=0 tx=0).stop 2024-09-15T21:33:32.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.922+0000 7fb6eda45640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb6cc03b9a0 0x7fb6cc03de60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80ad000 tx=0x7fb6d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6f04aa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6f04aa640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 0x7fb6e802d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6f04aa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80a5480 con 0x7fb6e802d450 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 0x7fb6e802d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient: get_auth_request con 0x7fb6e802d450 auth_method 0 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:32.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.926+0000 7fb6ee246640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6ee246640 10 monclient(hunting): handle_auth_done global_id 174315 payload 824 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6ee246640 1 -- 172.21.15.17:0/181339634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6e806d020 con 0x7fb6e802d450 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6ee246640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 0x7fb6e802d860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009710 tx=0x7fb6e40059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4016040 con 0x7fb6e802d450 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 10 monclient: dump: 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:32.947 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4704 (secure 0 0 0) 0x7fb6e401b450 con 0x7fb6e802d450 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 msgr2=0x7fb6e802d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 0x7fb6e802d860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009710 tx=0x7fb6e40059e0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 msgr2=0x7fb6e802d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6deffd640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e802d450 0x7fb6e802d860 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009710 tx=0x7fb6e40059e0 comp rx=0 tx=0).stop 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 102666, 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:32.949 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:32.950 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:32.951 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:32.952 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:32.953 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:32.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:32.956 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:32.957 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:32.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.959 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:32.960 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:32.961 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb6cc03b9a0 msgr2=0x7fb6cc03de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb6cc03b9a0 0x7fb6cc03de60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80ad000 tx=0x7fb6d8008040 comp rx=0 tx=0).stop 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 10 monclient: shutdown 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e80a1380 msgr2=0x7fb6e809de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 --2- 172.21.15.17:0/181339634 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fb6e80a1380 0x7fb6e809de80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000cae0 tx=0x7fb6e000a060 comp rx=0 tx=0).stop 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6eea47640 1 -- 172.21.15.17:0/181339634 reap_dead start 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6eea47640 1 -- 172.21.15.17:0/181339634 reap_dead start 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 shutdown_connections 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 >> 172.21.15.17:0/181339634 conn(0x7fb6e80a66a0 msgr2=0x7fb6e80710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:32.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 shutdown_connections 2024-09-15T21:33:32.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:32.930+0000 7fb6f04aa640 1 -- 172.21.15.17:0/181339634 wait complete. 2024-09-15T21:33:32.964 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:33:32.964 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:33:32.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:33:33.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.074+0000 7f88d087c640 10 monclient: get_monmap_and_config 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.074+0000 7f88d087c640 10 monclient: build_initial_monmap 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.074+0000 7f88d087c640 10 monclient: monmap: 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:33.077955+0000 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:33.077955+0000 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:33.079 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 1 Processor -- start 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 1 -- start start 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient: init 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f88cc09d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f88cc09def0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc13d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient(hunting): picked mon.noname-b con 0x7f88cc13afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88cc0a3490 con 0x7f88cc09def0 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88cc0a0320 con 0x7f88cc09d400 2024-09-15T21:33:33.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc0a1bc0 con 0x7f88cc13afc0 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88d087c640 10 monclient(hunting): authenticate will time out at 1936.473166s 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88ca59c640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7f88cc09d400 legacy=0x7f88cc09d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c9d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f88cc09def0 0x7f88cc1398a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38984/0 (socket says 172.21.15.17:38984) 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc13d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c9d9b640 1 -- 172.21.15.17:0/2840102408 learned_addr learned my addr 172.21.15.17:0/2840102408 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): get_auth_request con 0x7f88cc13afc0 auth_method 0 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c959a640 1 -- 172.21.15.17:0/2840102408 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090400443 0 0) 0x7f88cc0a3490 con 0x7f88cc09def0 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c959a640 10 monclient(hunting): _negotiate 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c959a640 10 monclient(hunting): my global_id is 174318 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88c959a640 1 -- 172.21.15.17:0/2840102408 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88b8003530 con 0x7f88cc09def0 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient(hunting): handle_auth_done global_id 164384 payload 274 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 1 -- 172.21.15.17:0/2840102408 >> v1:172.21.15.119:6789/0 conn(0x7f88cc09d400 legacy=0x7f88cc09d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 1 -- 172.21.15.17:0/2840102408 >> v1:172.21.15.17:6789/0 conn(0x7f88cc09def0 legacy=0x7f88cc1398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 1 monclient: found mon.noname-b 2024-09-15T21:33:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:33.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:33.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.078+0000 7f88cad9d640 1 -- 172.21.15.17:0/2840102408 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc09e7a0 con 0x7f88cc13afc0 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 --2- 172.21.15.17:0/2840102408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc13d3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88c40097f0 tx=0x7f88c401cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0af11a568a630e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 1 -- 172.21.15.17:0/2840102408 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40b0070 con 0x7f88cc13afc0 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: dump: 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.091 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: _finish_auth 0 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: _check_auth_tickets 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 1 -- 172.21.15.17:0/2840102408 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c40ab600 con 0x7f88cc13afc0 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 1 -- 172.21.15.17:0/2840102408 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40ab9d0 con 0x7f88cc13afc0 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 5 monclient: authenticate success, global_id 164384 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 4 set_mon_vals no callback set 2024-09-15T21:33:33.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c959a640 10 monclient: dump: 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: shutdown 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2840102408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 msgr2=0x7f88cc13d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 --2- 172.21.15.17:0/2840102408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc13d3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88c40097f0 tx=0x7f88c401cd70 comp rx=0 tx=0).stop 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2840102408 shutdown_connections 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 --2- 172.21.15.17:0/2840102408 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc13d3b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2840102408 >> 172.21.15.17:0/2840102408 conn(0x7f88cc0a66a0 msgr2=0x7f88cc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2840102408 shutdown_connections 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2840102408 wait complete. 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: build_initial_monmap 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: monmap: 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:33.094 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 Processor -- start 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- start start 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: init 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc09d400 0x7f88cc13a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): picked mon.noname-a con 0x7f88cc09d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc09def0 0x7f88cc13a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): picked mon.noname-b con 0x7f88cc09def0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc122f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): picked mon.noname-c con 0x7f88cc13afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc09ce50 con 0x7f88cc09d400 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc09ccd0 con 0x7f88cc09def0 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc0a0890 con 0x7f88cc13afc0 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): authenticate will time out at 1936.478947s 2024-09-15T21:33:33.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc09d400 0x7f88cc13a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc122f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc09d400 0x7f88cc13a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45644/0 (socket says 172.21.15.17:45644) 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 -- 172.21.15.17:0/2184785188 learned_addr learned my addr 172.21.15.17:0/2184785188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): get_auth_request con 0x7f88cc09d400 auth_method 0 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): get_auth_request con 0x7f88cc13afc0 auth_method 0 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c9d9b640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc09def0 msgr2=0x7f88cc13a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:33:33.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c9d9b640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc09def0 0x7f88cc13a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient(hunting): handle_auth_done global_id 164387 payload 824 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc09def0 msgr2=0x7f88cc13a840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88cc09def0 0x7f88cc13a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc09d400 msgr2=0x7f88cc13a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc09d400 0x7f88cc13a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:33:33.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 monclient: found mon.noname-c 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc124950 con 0x7f88cc13afc0 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88cc09d400 0x7f88cc13a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88cad9d640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc122f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f88c40ab000 tx=0x7f88c4019110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40bb070 con 0x7f88cc13afc0 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: dump: 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.099 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c40b7460 con 0x7f88cc13afc0 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40b0040 con 0x7f88cc13afc0 2024-09-15T21:33:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 5 monclient: authenticate success, global_id 164387 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: _renew_subs 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc0a1640 con 0x7f88cc13afc0 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c8d99640 4 set_mon_vals no callback set 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: dump: 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.101 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: _renew_subs 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc126cb0 con 0x7f88cc13afc0 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 0x7f88cc02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc09d7e0 con 0x7f88cc1254b0 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 0x7f88cc02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f88c40be070 con 0x7f88cc13afc0 2024-09-15T21:33:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient: get_auth_request con 0x7f88cc1254b0 auth_method 0 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 10 monclient(hunting): handle_auth_done global_id 164390 payload 824 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88ac03b970 0x7f88ac03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88cc11fc00 con 0x7f88cc1254b0 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f88c40eaad0 con 0x7f88cc13afc0 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88ca59c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 0x7f88cc02d450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0052f0 tx=0x7f88bc0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c9d9b640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88ac03b970 0x7f88ac03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c9d9b640 10 monclient: get_auth_request con 0x7f88ac03b970 auth_method 0 2024-09-15T21:33:33.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc016070 con 0x7f88cc1254b0 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: dump: 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:33.104 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f88bc00eaf0 con 0x7f88cc1254b0 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 msgr2=0x7f88cc02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 0x7f88cc02d450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0052f0 tx=0x7f88bc0057b0 comp rx=0 tx=0).stop 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 msgr2=0x7f88cc02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c2ffd640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1254b0 0x7f88cc02d450 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0052f0 tx=0x7f88bc0057b0 comp rx=0 tx=0).stop 2024-09-15T21:33:33.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.082+0000 7f88c9d9b640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88ac03b970 0x7f88ac03de30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f88cc126340 tx=0x7f88b4001780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:33.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:33.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 0x7f88cc02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc06d020 con 0x7f88cc1cf8b0 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 0x7f88cc02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient: get_auth_request con 0x7f88cc1cf8b0 auth_method 0 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 10 monclient(hunting): handle_auth_done global_id 164393 payload 824 2024-09-15T21:33:33.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 1 -- 172.21.15.17:0/2184785188 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f88cc126410 con 0x7f88cc1cf8b0 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88ca59c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 0x7f88cc02d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0052f0 tx=0x7f88bc005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc016040 con 0x7f88cc1cf8b0 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 10 monclient: dump: 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:33.108 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7f88bc005a00 con 0x7f88cc1cf8b0 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 msgr2=0x7f88cc02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 0x7f88cc02d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0052f0 tx=0x7f88bc005990 comp rx=0 tx=0).stop 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 msgr2=0x7f88cc02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88c2ffd640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc1cf8b0 0x7f88cc02d990 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0052f0 tx=0x7f88bc005990 comp rx=0 tx=0).stop 2024-09-15T21:33:33.109 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 161674, 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:33.110 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:33.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:33.113 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:33.114 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:33.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:33.116 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:33.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:33.118 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:33.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:33.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.121 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:33.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:33.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:33.123 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:33.123 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88ac03b970 msgr2=0x7f88ac03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88ac03b970 0x7f88ac03de30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f88cc126340 tx=0x7f88b4001780 comp rx=0 tx=0).stop 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 10 monclient: shutdown 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 msgr2=0x7f88cc122f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 --2- 172.21.15.17:0/2184785188 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88cc13afc0 0x7f88cc122f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f88c40ab000 tx=0x7f88c4019110 comp rx=0 tx=0).stop 2024-09-15T21:33:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88cad9d640 1 -- 172.21.15.17:0/2184785188 reap_dead start 2024-09-15T21:33:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88cad9d640 1 -- 172.21.15.17:0/2184785188 reap_dead start 2024-09-15T21:33:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 shutdown_connections 2024-09-15T21:33:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 >> 172.21.15.17:0/2184785188 conn(0x7f88cc0a66a0 msgr2=0x7f88cc13b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 shutdown_connections 2024-09-15T21:33:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:33.090+0000 7f88d087c640 1 -- 172.21.15.17:0/2184785188 wait complete. 2024-09-15T21:33:33.125 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:33:33.125 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:33:53.126 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:33:53.126 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:33:53.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:33:53.129 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:33:53.129 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:53.130 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient: get_monmap_and_config 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient: build_initial_monmap 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient: monmap: 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:53.260312+0000 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:53.260312+0000 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 1 Processor -- start 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 1 -- start start 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient: init 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff1e81378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 0x7ff1e813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient(hunting): picked mon.noname-c con 0x7ff1e81383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff1e813d2c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e80a1a20 con 0x7ff1e81378c0 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e80a1d00 con 0x7ff1e813d2c0 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff1e80a1fe0 con 0x7ff1e81383b0 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1ec8af640 10 monclient(hunting): authenticate will time out at 1956.655792s 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1e659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff1e81378c0 0x7ff1e8137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43684/0 (socket says 172.21.15.17:43684) 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.257+0000 7ff1e659c640 1 -- 172.21.15.17:0/2767188409 learned_addr learned my addr 172.21.15.17:0/2767188409 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 0x7ff1e813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44114984 0 0) 0x7ff1d8001120 con 0x7ff1e81378c0 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): get_auth_request con 0x7ff1e81383b0 auth_method 0 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): _negotiate 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): my global_id is 174324 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1d0003530 con 0x7ff1e81378c0 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 -- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 msgr2=0x7ff1e813cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 -- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 msgr2=0x7ff1e813cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 0x7ff1e813cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 0x7ff1e813cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1758613708 0 0) 0x7ff1d0003530 con 0x7ff1e81378c0 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 msgr2=0x7ff1e813cd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 --2- 172.21.15.17:0/2767188409 >> v2:172.21.15.119:3300/0 conn(0x7ff1e81383b0 0x7ff1e813cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 >> v1:172.21.15.119:6789/0 conn(0x7ff1e813d2c0 legacy=0x7ff1e813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: _finish_hunting 0 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e80a22c0 con 0x7ff1e81378c0 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: _finish_auth 0 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: _check_auth_tickets 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7ff1d80018d0 con 0x7ff1e81378c0 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: dump: 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 >> v1:172.21.15.17:6789/0 conn(0x7ff1e81378c0 legacy=0x7ff1e8137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0003a90 0x7ff1d0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): picked mon.b con 0x7ff1d0003a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1d0004810 0x7ff1d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): picked mon.c con 0x7ff1d0004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d0005420 0x7ff1d000ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): picked mon.a con 0x7ff1d0005420 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e80a1fe0 con 0x7ff1d0005420 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d000c330 con 0x7ff1d0003a90 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d000c530 con 0x7ff1d0004810 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1d0004810 0x7ff1d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d0005420 0x7ff1d000ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): get_auth_request con 0x7ff1d0004810 auth_method 0 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): get_auth_request con 0x7ff1d0005420 auth_method 0 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e659c640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0003a90 0x7ff1d0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e659c640 10 monclient(hunting): get_auth_request con 0x7ff1d0003a90 auth_method 0 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_done global_id 174324 payload 293 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 -- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1d0004810 msgr2=0x7ff1d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1d0004810 0x7ff1d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 -- 172.21.15.17:0/2767188409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0003a90 msgr2=0x7ff1d0003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0003a90 0x7ff1d0003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 monclient: found mon.a 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 -- 172.21.15.17:0/2767188409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d0003530 con 0x7ff1d0005420 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient: _finish_auth 0 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 10 monclient: _check_auth_tickets 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1ec8af640 5 monclient: authenticate success, global_id 174324 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1ec8af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e659c640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0003a90 0x7ff1d0003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1d0004810 0x7ff1d0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 -- 172.21.15.17:0/2767188409 reap_dead start 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e6d9d640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d0005420 0x7ff1d000ba50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80a6b30 tx=0x7ff1e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768835b776de2f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00f040 con 0x7ff1d0005420 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: dump: 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0003b80 con 0x7ff1d0005420 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1ec8af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1ec8af640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1ec8af640 4 set_mon_vals no callback set 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 1 -- 172.21.15.17:0/2767188409 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0001ae0 con 0x7ff1d0005420 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.261+0000 7ff1e559a640 10 monclient: dump: 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: shutdown 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d0005420 msgr2=0x7ff1d000ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d0005420 0x7ff1d000ba50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80a6b30 tx=0x7ff1e001c410 comp rx=0 tx=0).stop 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/2767188409 shutdown_connections 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/2767188409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d0005420 0x7ff1d000ba50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/2767188409 >> 172.21.15.17:0/2767188409 conn(0x7ff1e80aac40 msgr2=0x7ff1e80a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/2767188409 shutdown_connections 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/2767188409 wait complete. 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: build_initial_monmap 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: monmap: 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 Processor -- start 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- start start 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: init 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 0x7ff1e8136590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): picked mon.noname-b con 0x7ff1d0005420 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 0x7ff1e809d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): picked mon.noname-c con 0x7ff1e8136ad0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e809db70 0x7ff1e809dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): picked mon.noname-a con 0x7ff1e809db70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8138030 con 0x7ff1e809db70 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8137eb0 con 0x7ff1d0005420 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e81381b0 con 0x7ff1e8136ad0 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient(hunting): authenticate will time out at 1956.661956s 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 0x7ff1e809d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 0x7ff1e8136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 0x7ff1e809d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47038/0 (socket says 172.21.15.17:47038) 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 -- 172.21.15.17:0/1767753685 learned_addr learned my addr 172.21.15.17:0/1767753685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): get_auth_request con 0x7ff1e8136ad0 auth_method 0 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 10 monclient(hunting): get_auth_request con 0x7ff1d0005420 auth_method 0 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e809db70 0x7ff1e809dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 1 -- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 msgr2=0x7ff1e8136590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 1 -- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 msgr2=0x7ff1e8136590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 0x7ff1e8136590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): get_auth_request con 0x7ff1e809db70 auth_method 0 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e659c640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 0x7ff1e8136590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient(hunting): handle_auth_done global_id 164396 payload 824 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 -- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 msgr2=0x7ff1e8136590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 0x7ff1e8136590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 -- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e809db70 msgr2=0x7ff1e809dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e809db70 0x7ff1e809dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 monclient: found mon.noname-c 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 -- 172.21.15.17:0/1767753685 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e8120db0 con 0x7ff1e8136ad0 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e5d9b640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 0x7ff1e809d630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e5c0 tx=0x7ff1dc00ea80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc017810 con 0x7ff1e8136ad0 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: dump: 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0179b0 con 0x7ff1e8136ad0 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc017da0 con 0x7ff1e8136ad0 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1d6ffd640 10 monclient: dump: 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1e4d99640 4 set_mon_vals no callback set 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 5 monclient: authenticate success, global_id 164396 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: _renew_subs 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e80a8a00 con 0x7ff1e8136ad0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: _renew_subs 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.265+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e809fc50 con 0x7ff1e8136ad0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1ec8af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1ec8af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1ec8af640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e80a4d00 con 0x7ff1e8136ad0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff1dc016070 con 0x7ff1e8136ad0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1d6ffd640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1c803ba20 0x7ff1c803dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff1dc0506b0 con 0x7ff1e8136ad0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1e659c640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1c803ba20 0x7ff1c803dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1e659c640 10 monclient: get_auth_request con 0x7ff1c803ba20 auth_method 0 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.269+0000 7ff1e659c640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1c803ba20 0x7ff1c803dee0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8000fc0 tx=0x7ff1d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.273+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff1dc020da0 con 0x7ff1e8136ad0 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.273+0000 7ff1d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.273+0000 7ff1d6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:53.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.501+0000 7ff1ec8af640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:53.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.501+0000 7ff1ec8af640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:53.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.501+0000 7ff1ec8af640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:53.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.501+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff1e80a0090 con 0x7ff1e8136ad0 2024-09-15T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1767753685 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7ff1dc027d00 con 0x7ff1e8136ad0 2024-09-15T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1d6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1d6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1c803ba20 msgr2=0x7ff1c803dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1c803ba20 0x7ff1c803dee0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8000fc0 tx=0x7ff1d8009040 comp rx=0 tx=0).stop 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 10 monclient: shutdown 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 msgr2=0x7ff1e809d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 0x7ff1e809d630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e5c0 tx=0x7ff1dc00ea80 comp rx=0 tx=0).stop 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 shutdown_connections 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff1c803ba20 0x7ff1c803dee0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e809db70 0x7ff1e809dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1e8136ad0 0x7ff1e809d630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 --2- 172.21.15.17:0/1767753685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d0005420 0x7ff1e8136590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 >> 172.21.15.17:0/1767753685 conn(0x7ff1e80aac40 msgr2=0x7ff1e806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 shutdown_connections 2024-09-15T21:33:53.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:53.505+0000 7ff1ec8af640 1 -- 172.21.15.17:0/1767753685 wait complete. 2024-09-15T21:33:56.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient: get_monmap_and_config 2024-09-15T21:33:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient: build_initial_monmap 2024-09-15T21:33:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient: monmap: 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:56.659746+0000 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:56.659746+0000 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:56.660 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:56.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 1 Processor -- start 2024-09-15T21:33:56.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 1 -- start start 2024-09-15T21:33:56.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient: init 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa65c1340c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c09d6c0 0x7fa65c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient(hunting): picked mon.noname-b con 0x7fa65c09d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa65c139390 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c09e1b0 con 0x7fa65c1340c0 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c09e490 con 0x7fa65c139390 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c09d520 con 0x7fa65c09d6c0 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa662255640 10 monclient(hunting): authenticate will time out at 1960.055321s 2024-09-15T21:33:56.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa65c1340c0 0x7fa65c1344a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43692/0 (socket says 172.21.15.17:43692) 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65b7fe640 1 -- 172.21.15.17:0/2823287710 learned_addr learned my addr 172.21.15.17:0/2823287710 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 1 --2- 172.21.15.17:0/2823287710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c09d6c0 0x7fa65c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): get_auth_request con 0x7fa65c09d6c0 auth_method 0 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 10 monclient(hunting): handle_auth_done global_id 164399 payload 274 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65a7fc640 1 -- 172.21.15.17:0/2823287710 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671708034 0 0) 0x7fa65c09e1b0 con 0x7fa65c1340c0 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.657+0000 7fa65affd640 1 -- 172.21.15.17:0/2823287710 >> v1:172.21.15.119:6789/0 conn(0x7fa65c139390 legacy=0x7fa65c13b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 1 -- 172.21.15.17:0/2823287710 >> v1:172.21.15.17:6789/0 conn(0x7fa65c1340c0 legacy=0x7fa65c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:56.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 1 monclient: found mon.noname-b 2024-09-15T21:33:56.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:56.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:56.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 1 -- 172.21.15.17:0/2823287710 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c09e770 con 0x7fa65c09d6c0 2024-09-15T21:33:56.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65affd640 1 --2- 172.21.15.17:0/2823287710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c09d6c0 0x7fa65c138e50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6440097f0 tx=0x7fa64401d4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee5c4eb42303264 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 1 -- 172.21.15.17:0/2823287710 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b1070 con 0x7fa65c09d6c0 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: dump: 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:56.671 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 1 -- 172.21.15.17:0/2823287710 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6440ac600 con 0x7fa65c09d6c0 2024-09-15T21:33:56.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 1 -- 172.21.15.17:0/2823287710 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440ac9d0 con 0x7fa65c09d6c0 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 5 monclient: authenticate success, global_id 164399 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 4 set_mon_vals no callback set 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa65a7fc640 10 monclient: dump: 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:56.673 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 10 monclient: shutdown 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 -- 172.21.15.17:0/2823287710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c09d6c0 msgr2=0x7fa65c138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 --2- 172.21.15.17:0/2823287710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c09d6c0 0x7fa65c138e50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6440097f0 tx=0x7fa64401d4b0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 -- 172.21.15.17:0/2823287710 shutdown_connections 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 --2- 172.21.15.17:0/2823287710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c09d6c0 0x7fa65c138e50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 -- 172.21.15.17:0/2823287710 >> 172.21.15.17:0/2823287710 conn(0x7fa65c0a66a0 msgr2=0x7fa65c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 -- 172.21.15.17:0/2823287710 shutdown_connections 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.661+0000 7fa662255640 1 -- 172.21.15.17:0/2823287710 wait complete. 2024-09-15T21:33:56.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: build_initial_monmap 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: monmap: 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 Processor -- start 2024-09-15T21:33:56.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- start start 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: init 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa65c09d6c0 0x7fa65c1cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): picked mon.noname-b con 0x7fa65c09d6c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 0x7fa65c1cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): picked mon.noname-a con 0x7fa65c1340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 0x7fa65c1d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): picked mon.noname-c con 0x7fa65c139390 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c13f240 con 0x7fa65c1340c0 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:56.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c13f0c0 con 0x7fa65c09d6c0 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c13f3c0 con 0x7fa65c139390 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient(hunting): authenticate will time out at 1960.061472s 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 0x7fa65c1cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 0x7fa65c1d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 0x7fa65c1d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47050/0 (socket says 172.21.15.17:47050) 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 1 -- 172.21.15.17:0/494341300 learned_addr learned my addr 172.21.15.17:0/494341300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): get_auth_request con 0x7fa65c139390 auth_method 0 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 0x7fa65c1cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33494/0 (socket says 172.21.15.17:33494) 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:56.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): get_auth_request con 0x7fa65c1340c0 auth_method 0 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65b7fe640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa65c09d6c0 0x7fa65c1cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:56.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65b7fe640 10 monclient(hunting): get_auth_request con 0x7fa65c09d6c0 auth_method 0 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient(hunting): handle_auth_done global_id 174333 payload 824 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 -- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 msgr2=0x7fa65c1d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 0x7fa65c1d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 -- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa65c09d6c0 msgr2=0x7fa65c1cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa65c09d6c0 0x7fa65c1cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 monclient: found mon.noname-a 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 -- 172.21.15.17:0/494341300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c09d0f0 con 0x7fa65c1340c0 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65bfff640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 0x7fa65c1d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:56.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa65affd640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 0x7fa65c1cfd10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6440ac000 tx=0x7fa64401a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b9020 con 0x7fa65c1340c0 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: dump: 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:56.680 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: _finish_auth 0 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: _check_auth_tickets 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6440b5410 con 0x7fa65c1340c0 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b1040 con 0x7fa65c1340c0 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 10 monclient: dump: 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:56.681 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 5 monclient: authenticate success, global_id 174333 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _renew_subs 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c133660 con 0x7fa65c1340c0 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa659ffb640 4 set_mon_vals no callback set 2024-09-15T21:33:56.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _renew_subs 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c0a41e0 con 0x7fa65c1340c0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa6440bc020 con 0x7fa65c1340c0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65c1344a0 con 0x7fa65c1340c0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9f0 0x7fa63c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.665+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa6440ea3f0 con 0x7fa65c1340c0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.669+0000 7fa65b7fe640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9f0 0x7fa63c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.669+0000 7fa65b7fe640 10 monclient: get_auth_request con 0x7fa63c03b9f0 auth_method 0 2024-09-15T21:33:56.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.669+0000 7fa65b7fe640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9f0 0x7fa63c03deb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000f70 tx=0x7fa64c007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:56.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.673+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa6440c3030 con 0x7fa65c1340c0 2024-09-15T21:33:56.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.673+0000 7fa653fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:56.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.673+0000 7fa653fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:56.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa662255640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:56.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa662255640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:56.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa662255640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:56.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa65c12f250 con 0x7fa65c1340c0 2024-09-15T21:33:56.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa653fff640 1 -- 172.21.15.17:0/494341300 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fa65c09d0f0 con 0x7fa65c1340c0 2024-09-15T21:33:56.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa653fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:56.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.909+0000 7fa653fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:56.912 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:56.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9f0 msgr2=0x7fa63c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9f0 0x7fa63c03deb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000f70 tx=0x7fa64c007b60 comp rx=0 tx=0).stop 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 10 monclient: shutdown 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 msgr2=0x7fa65c1cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 0x7fa65c1cfd10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6440ac000 tx=0x7fa64401a250 comp rx=0 tx=0).stop 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 shutdown_connections 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa63c03b9f0 0x7fa63c03deb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa65c139390 0x7fa65c1d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa65c1340c0 0x7fa65c1cfd10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 --2- 172.21.15.17:0/494341300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa65c09d6c0 0x7fa65c1cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 >> 172.21.15.17:0/494341300 conn(0x7fa65c0a66a0 msgr2=0x7fa65c13be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:56.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 shutdown_connections 2024-09-15T21:33:56.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:56.913+0000 7fa662255640 1 -- 172.21.15.17:0/494341300 wait complete. 2024-09-15T21:33:56.931 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:33:56.932 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:33:56.932 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:33:57.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.045+0000 7fca85d87640 10 monclient: get_monmap_and_config 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.045+0000 7fca85d87640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.045+0000 7fca85d87640 10 monclient: monmap: 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:57.049959+0000 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:57.049959+0000 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:57.050 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:57.051 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 1 Processor -- start 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 1 -- start start 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient: init 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fca80099600 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca800999e0 0x7fca8013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient(hunting): picked mon.noname-a con 0x7fca800999e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fca80099e10 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca80075200 con 0x7fca80099600 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca80074d80 con 0x7fca80099e10 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fca800748c0 con 0x7fca800999e0 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 10 monclient(hunting): authenticate will time out at 1960.445140s 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fca80099600 0x7fca8009ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43696/0 (socket says 172.21.15.17:43696) 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7f7fe640 1 -- 172.21.15.17:0/123482667 learned_addr learned my addr 172.21.15.17:0/123482667 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 1 --2- 172.21.15.17:0/123482667 >> v2:172.21.15.17:3300/0 conn(0x7fca800999e0 0x7fca8013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): get_auth_request con 0x7fca800999e0 auth_method 0 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient(hunting): handle_auth_done global_id 174336 payload 274 2024-09-15T21:33:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 1 -- 172.21.15.17:0/123482667 >> v1:172.21.15.119:6789/0 conn(0x7fca80099e10 legacy=0x7fca8013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 1 -- 172.21.15.17:0/123482667 >> v1:172.21.15.17:6789/0 conn(0x7fca80099600 legacy=0x7fca8009ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 1 monclient: found mon.noname-a 2024-09-15T21:33:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 1 -- 172.21.15.17:0/123482667 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8009dbc0 con 0x7fca800999e0 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7effd640 1 --2- 172.21.15.17:0/123482667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca8013cac0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca680097c0 tx=0x7fca6801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b6003b20bd160 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 1 -- 172.21.15.17:0/123482667 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680b0070 con 0x7fca800999e0 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 10 monclient: dump: 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 1 -- 172.21.15.17:0/123482667 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca68004230 con 0x7fca800999e0 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca7e7fc640 1 -- 172.21.15.17:0/123482667 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68003620 con 0x7fca800999e0 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.049+0000 7fca85d87640 5 monclient: authenticate success, global_id 174336 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 4 set_mon_vals no callback set 2024-09-15T21:33:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7e7fc640 10 monclient: dump: 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: shutdown 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/123482667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 msgr2=0x7fca8013cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 --2- 172.21.15.17:0/123482667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca8013cac0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca680097c0 tx=0x7fca6801c330 comp rx=0 tx=0).stop 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/123482667 shutdown_connections 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 --2- 172.21.15.17:0/123482667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca8013cac0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/123482667 >> 172.21.15.17:0/123482667 conn(0x7fca800a6820 msgr2=0x7fca800a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/123482667 shutdown_connections 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/123482667 wait complete. 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: monmap: 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:57.062 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 Processor -- start 2024-09-15T21:33:57.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- start start 2024-09-15T21:33:57.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: init 2024-09-15T21:33:57.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80099600 0x7fca800963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): picked mon.noname-b con 0x7fca80099600 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca80090d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): picked mon.noname-a con 0x7fca800999e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca80099e10 0x7fca800912c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): picked mon.noname-c con 0x7fca80099e10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca80143340 con 0x7fca800999e0 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca801431c0 con 0x7fca80099600 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fca801434c0 con 0x7fca80099e10 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): authenticate will time out at 1960.450274s 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca80090d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca80090d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33516/0 (socket says 172.21.15.17:33516) 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 -- 172.21.15.17:0/1729932281 learned_addr learned my addr 172.21.15.17:0/1729932281 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): get_auth_request con 0x7fca800999e0 auth_method 0 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca80099e10 0x7fca800912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): get_auth_request con 0x7fca80099e10 auth_method 0 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7f7fe640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80099600 0x7fca800963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7f7fe640 10 monclient(hunting): get_auth_request con 0x7fca80099600 auth_method 0 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient(hunting): handle_auth_done global_id 174342 payload 824 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca80099e10 msgr2=0x7fca800912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca80099e10 0x7fca800912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80099600 msgr2=0x7fca800963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80099600 0x7fca800963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 monclient: found mon.noname-a 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca800a5f10 con 0x7fca800999e0 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7effd640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca80090d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca68019f60 tx=0x7fca680190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680b9020 con 0x7fca800999e0 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: dump: 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca680043e0 con 0x7fca800999e0 2024-09-15T21:33:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 5 monclient: authenticate success, global_id 174342 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: _renew_subs 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca800927f0 con 0x7fca800999e0 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680b0040 con 0x7fca800999e0 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca77fff640 10 monclient: dump: 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: _renew_subs 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8013fdb0 con 0x7fca800999e0 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca7dffb640 4 set_mon_vals no callback set 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 0x7fca80091800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.053+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca8013ffe0 con 0x7fca80096910 2024-09-15T21:33:57.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 0x7fca80091800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient: get_auth_request con 0x7fca80096910 auth_method 0 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fca680bc020 con 0x7fca800999e0 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca6003b9f0 0x7fca6003deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fca680ea8a0 con 0x7fca800999e0 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7ffff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca6003b9f0 0x7fca6003deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7ffff640 10 monclient: get_auth_request con 0x7fca6003b9f0 auth_method 0 2024-09-15T21:33:57.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_done global_id 174345 payload 824 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca8009ee60 con 0x7fca80096910 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7f7fe640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 0x7fca80091800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f70 tx=0x7fca7000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70019070 con 0x7fca80096910 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 10 monclient: dump: 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.077 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fca70005a70 con 0x7fca80096910 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 msgr2=0x7fca80091800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 0x7fca80091800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f70 tx=0x7fca7000e940 comp rx=0 tx=0).stop 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 msgr2=0x7fca80091800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca77fff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80096910 0x7fca80091800 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f70 tx=0x7fca7000e940 comp rx=0 tx=0).stop 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.057+0000 7fca7ffff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca6003b9f0 0x7fca6003deb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fca78005600 tx=0x7fca78003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca85d87640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca85d87640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 0x7fca8008cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca85d87640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca80091d40 con 0x7fca80093350 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 0x7fca8008cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient: get_auth_request con 0x7fca80093350 auth_method 0 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.061+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca7f7fe640 10 monclient(hunting): handle_auth_done global_id 174348 payload 824 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca7f7fe640 1 -- 172.21.15.17:0/1729932281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fca80069db0 con 0x7fca80093350 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca7f7fe640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 0x7fca8008cff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f70 tx=0x7fca70000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70019040 con 0x7fca80093350 2024-09-15T21:33:57.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 10 monclient: dump: 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4715 (secure 0 0 0) 0x7fca700073d0 con 0x7fca80093350 2024-09-15T21:33:57.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 msgr2=0x7fca8008cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 0x7fca8008cff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f70 tx=0x7fca70000ee0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 msgr2=0x7fca8008cff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca77fff640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80093350 0x7fca8008cff0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca70000f70 tx=0x7fca70000ee0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 152, 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 126802, 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:57.085 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.086 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:57.090 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.092 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.093 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.094 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:33:57.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.094 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.094 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:57.094 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:57.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca6003b9f0 msgr2=0x7fca6003deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca6003b9f0 0x7fca6003deb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fca78005600 tx=0x7fca78003d50 comp rx=0 tx=0).stop 2024-09-15T21:33:57.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 10 monclient: shutdown 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 msgr2=0x7fca80090d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 --2- 172.21.15.17:0/1729932281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca800999e0 0x7fca80090d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca68019f60 tx=0x7fca680190d0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca7ffff640 1 -- 172.21.15.17:0/1729932281 reap_dead start 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca7ffff640 1 -- 172.21.15.17:0/1729932281 reap_dead start 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 shutdown_connections 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 >> 172.21.15.17:0/1729932281 conn(0x7fca800a6820 msgr2=0x7fca800a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 shutdown_connections 2024-09-15T21:33:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.065+0000 7fca85d87640 1 -- 172.21.15.17:0/1729932281 wait complete. 2024-09-15T21:33:57.096 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:57.097 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:33:57.097 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:33:57.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient: get_monmap_and_config 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient: monmap: 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:57.211922+0000 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:57.211922+0000 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:57.213 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:57.213 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 Processor -- start 2024-09-15T21:33:57.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 -- start start 2024-09-15T21:33:57.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient: init 2024-09-15T21:33:57.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2024-09-15T21:33:57.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 0x7f24a40a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): picked mon.noname-b con 0x7f24a40a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f24a409ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24a40a11b0 0x7f24a4135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): picked mon.noname-c con 0x7f24a40a11b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a40a3130 con 0x7f24a409ffc0 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f24a4073df0 con 0x7f24a40a11b0 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24a40a1bf0 con 0x7f24a40a0890 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a8817640 10 monclient(hunting): authenticate will time out at 1960.607380s 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 0x7f24a40a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 0x7f24a40a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47068/0 (socket says 172.21.15.17:47068) 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 1 -- 172.21.15.17:0/1375272758 learned_addr learned my addr 172.21.15.17:0/1375272758 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): get_auth_request con 0x7f24a40a0890 auth_method 0 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a2d9d640 1 --2- 172.21.15.17:0/1375272758 >> v2:172.21.15.119:3300/0 conn(0x7f24a40a11b0 0x7f24a4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a2d9d640 10 monclient(hunting): get_auth_request con 0x7f24a40a11b0 auth_method 0 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 10 monclient(hunting): handle_auth_done global_id 164408 payload 274 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 1 -- 172.21.15.17:0/1375272758 >> v2:172.21.15.119:3300/0 conn(0x7f24a40a11b0 msgr2=0x7f24a4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 1 --2- 172.21.15.17:0/1375272758 >> v2:172.21.15.119:3300/0 conn(0x7f24a40a11b0 0x7f24a4135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.209+0000 7f24a259c640 1 -- 172.21.15.17:0/1375272758 >> v1:172.21.15.119:6789/0 conn(0x7f24a409ffc0 legacy=0x7f24a4135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a259c640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a259c640 1 monclient: found mon.noname-b 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a259c640 1 -- 172.21.15.17:0/1375272758 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a40a15e0 con 0x7f24a40a0890 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a259c640 1 --2- 172.21.15.17:0/1375272758 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 0x7f24a40a0c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24a40a1870 tx=0x7f249c01c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ffbc2aa7376d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 1 -- 172.21.15.17:0/1375272758 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a56a0 con 0x7f24a40a0890 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: dump: 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 1 -- 172.21.15.17:0/1375272758 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c0a5840 con 0x7f24a40a0890 2024-09-15T21:33:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 5 monclient: authenticate success, global_id 164408 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 1 -- 172.21.15.17:0/1375272758 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5c10 con 0x7f24a40a0890 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 4 set_mon_vals no callback set 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a159a640 10 monclient: dump: 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.226 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 10 monclient: shutdown 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 -- 172.21.15.17:0/1375272758 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 msgr2=0x7f24a40a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 --2- 172.21.15.17:0/1375272758 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 0x7f24a40a0c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24a40a1870 tx=0x7f249c01c8d0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 -- 172.21.15.17:0/1375272758 shutdown_connections 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 --2- 172.21.15.17:0/1375272758 >> v2:172.21.15.119:3300/0 conn(0x7f24a40a11b0 0x7f24a4135d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 --2- 172.21.15.17:0/1375272758 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a40a0890 0x7f24a40a0c70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 -- 172.21.15.17:0/1375272758 >> 172.21.15.17:0/1375272758 conn(0x7f24a40a66a0 msgr2=0x7f24a40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 -- 172.21.15.17:0/1375272758 shutdown_connections 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 1 -- 172.21.15.17:0/1375272758 wait complete. 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.213+0000 7f24a8817640 10 monclient: monmap: 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:57.228 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 Processor -- start 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- start start 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: init 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a409ffc0 0x7f24a41cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): picked mon.noname-c con 0x7f24a409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24a40a0890 0x7f24a41352c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): picked mon.noname-b con 0x7f24a40a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a40a11b0 0x7f24a4135800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): picked mon.noname-a con 0x7f24a40a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4140510 con 0x7f24a40a11b0 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41364d0 con 0x7f24a40a0890 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24a4140690 con 0x7f24a409ffc0 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): authenticate will time out at 1960.612818s 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a409ffc0 0x7f24a41cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a409ffc0 0x7f24a41cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47078/0 (socket says 172.21.15.17:47078) 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 -- 172.21.15.17:0/4039438281 learned_addr learned my addr 172.21.15.17:0/4039438281 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): get_auth_request con 0x7f24a409ffc0 auth_method 0 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a40a11b0 0x7f24a4135800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): get_auth_request con 0x7f24a40a11b0 auth_method 0 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24a40a0890 0x7f24a41352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): get_auth_request con 0x7f24a40a0890 auth_method 0 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient(hunting): handle_auth_done global_id 164411 payload 824 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24a40a0890 msgr2=0x7f24a41352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24a40a0890 0x7f24a41352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a40a11b0 msgr2=0x7f24a4135800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a40a11b0 0x7f24a4135800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 monclient: found mon.noname-c 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:57.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a4121990 con 0x7f24a409ffc0 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a259c640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a409ffc0 0x7f24a41cf630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f249c01a500 tx=0x7f249c019670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0bc020 con 0x7f24a409ffc0 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: dump: 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c0b86b0 con 0x7f24a409ffc0 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0b8aa0 con 0x7f24a409ffc0 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: dump: 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a0d99640 4 set_mon_vals no callback set 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 5 monclient: authenticate success, global_id 164411 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: _renew_subs 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a409d0f0 con 0x7f24a409ffc0 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: _renew_subs 2024-09-15T21:33:57.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41213d0 con 0x7f24a409ffc0 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 0x7f24a411dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24a40a0c70 con 0x7f24a4120470 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 0x7f24a411dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient: get_auth_request con 0x7f24a4120470 auth_method 0 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f249c0bf020 con 0x7f24a409ffc0 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f248403b9f0 0x7f248403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_done global_id 164414 payload 824 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24a411f910 con 0x7f24a4120470 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f249c0eb070 con 0x7f24a409ffc0 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a1d9b640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 0x7f24a411dfa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24a4120400 tx=0x7f249800e820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f248403b9f0 0x7f248403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f24a2d9d640 10 monclient: get_auth_request con 0x7f248403b9f0 auth_method 0 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498019070 con 0x7f24a4120470 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: dump: 2024-09-15T21:33:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f24980052e0 con 0x7f24a4120470 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 msgr2=0x7f24a411dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 0x7f24a411dfa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24a4120400 tx=0x7f249800e820 comp rx=0 tx=0).stop 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 msgr2=0x7f24a411dfa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.217+0000 7f2492ffd640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a4120470 0x7f24a411dfa0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24a4120400 tx=0x7f249800e820 comp rx=0 tx=0).stop 2024-09-15T21:33:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.221+0000 7f24a2d9d640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f248403b9f0 0x7f248403deb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2494006aa0 tx=0x7f2494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 0x7f24a411e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24a411ee30 con 0x7f24a411e4e0 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 0x7f24a411e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient: get_auth_request con 0x7f24a411e4e0 auth_method 0 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 10 monclient(hunting): handle_auth_done global_id 164417 payload 824 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 1 -- 172.21.15.17:0/4039438281 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f24a41215a0 con 0x7f24a411e4e0 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a1d9b640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 0x7f24a411e8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24a4120400 tx=0x7f2498000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498019040 con 0x7f24a411e4e0 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 10 monclient: dump: 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.242 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7f2498007050 con 0x7f24a411e4e0 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 msgr2=0x7f24a411e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 0x7f24a411e8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24a4120400 tx=0x7f2498000ee0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 msgr2=0x7f24a411e8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f2492ffd640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a411e4e0 0x7f24a411e8f0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24a4120400 tx=0x7f2498000ee0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 152, 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 185809, 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:57.257 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f248403b9f0 msgr2=0x7f248403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f248403b9f0 0x7f248403deb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2494006aa0 tx=0x7f2494008040 comp rx=0 tx=0).stop 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 10 monclient: shutdown 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a409ffc0 msgr2=0x7f24a41cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 --2- 172.21.15.17:0/4039438281 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24a409ffc0 0x7f24a41cf630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f249c01a500 tx=0x7f249c019670 comp rx=0 tx=0).stop 2024-09-15T21:33:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a2d9d640 1 -- 172.21.15.17:0/4039438281 reap_dead start 2024-09-15T21:33:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a2d9d640 1 -- 172.21.15.17:0/4039438281 reap_dead start 2024-09-15T21:33:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 shutdown_connections 2024-09-15T21:33:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 >> 172.21.15.17:0/4039438281 conn(0x7f24a40a66a0 msgr2=0x7f24a40a5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 shutdown_connections 2024-09-15T21:33:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.225+0000 7f24a8817640 1 -- 172.21.15.17:0/4039438281 wait complete. 2024-09-15T21:33:57.259 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:57.259 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:33:57.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:33:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient: get_monmap_and_config 2024-09-15T21:33:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient: monmap: 2024-09-15T21:33:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:57.386418+0000 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:57.386418+0000 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:57.387 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 Processor -- start 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 -- start start 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient: init 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 0x7f5e8c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): picked mon.noname-b con 0x7f5e8c1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e8c1381e0 0x7f5e8c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): picked mon.noname-c con 0x7f5e8c1381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e8c0a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8c0a1d00 con 0x7f5e8c0a5040 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e8c0a1fe0 con 0x7f5e8c1381e0 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c0a22c0 con 0x7f5e8c1378c0 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e9281e640 10 monclient(hunting): authenticate will time out at 1960.781774s 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 0x7f5e8c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 0x7f5e8c137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47112/0 (socket says 172.21.15.17:47112) 2024-09-15T21:33:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 -- 172.21.15.17:0/3793282477 learned_addr learned my addr 172.21.15.17:0/3793282477 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): get_auth_request con 0x7f5e8c1378c0 auth_method 0 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8b7fe640 1 --2- 172.21.15.17:0/3793282477 >> v2:172.21.15.119:3300/0 conn(0x7f5e8c1381e0 0x7f5e8c0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_done global_id 164420 payload 274 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 -- 172.21.15.17:0/3793282477 >> v2:172.21.15.119:3300/0 conn(0x7f5e8c1381e0 msgr2=0x7f5e8c0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/3793282477 >> v2:172.21.15.119:3300/0 conn(0x7f5e8c1381e0 0x7f5e8c0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 -- 172.21.15.17:0/3793282477 >> v1:172.21.15.119:6789/0 conn(0x7f5e8c0a5040 legacy=0x7f5e8c0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 monclient: found mon.noname-b 2024-09-15T21:33:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8bfff640 1 -- 172.21.15.17:0/3793282477 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c0a5c00 con 0x7f5e8c1378c0 2024-09-15T21:33:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8b7fe640 10 monclient: get_auth_request con 0x7f5e8c1381e0 auth_method 0 2024-09-15T21:33:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.385+0000 7f5e8b7fe640 1 --2- 172.21.15.17:0/3793282477 >> v2:172.21.15.119:3300/0 conn(0x7f5e8c1381e0 0x7f5e8c0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:33:57.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/3793282477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 0x7f5e8c137ca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009e50 tx=0x7f5e8401c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923cd19c64da0e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 1 -- 172.21.15.17:0/3793282477 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840a5640 con 0x7f5e8c1378c0 2024-09-15T21:33:57.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:57.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: dump: 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 1 -- 172.21.15.17:0/3793282477 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e840a57e0 con 0x7f5e8c1378c0 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 5 monclient: authenticate success, global_id 164420 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 1 -- 172.21.15.17:0/3793282477 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840a5bb0 con 0x7f5e8c1378c0 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 4 set_mon_vals no callback set 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8affd640 10 monclient: dump: 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.400 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: shutdown 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- 172.21.15.17:0/3793282477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 msgr2=0x7f5e8c137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 --2- 172.21.15.17:0/3793282477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 0x7f5e8c137ca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009e50 tx=0x7f5e8401c980 comp rx=0 tx=0).stop 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- 172.21.15.17:0/3793282477 shutdown_connections 2024-09-15T21:33:57.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 --2- 172.21.15.17:0/3793282477 >> v2:172.21.15.119:3300/0 conn(0x7f5e8c1381e0 0x7f5e8c0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 --2- 172.21.15.17:0/3793282477 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c1378c0 0x7f5e8c137ca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- 172.21.15.17:0/3793282477 >> 172.21.15.17:0/3793282477 conn(0x7f5e8c0aac40 msgr2=0x7f5e8c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- 172.21.15.17:0/3793282477 shutdown_connections 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- 172.21.15.17:0/3793282477 wait complete. 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: monmap: 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 Processor -- start 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- start start 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: init 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c0a5040 0x7f5e8c1cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): picked mon.noname-c con 0x7f5e8c0a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c1378c0 0x7f5e8c136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): picked mon.noname-b con 0x7f5e8c1378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e8c1381e0 0x7f5e8c13ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): picked mon.noname-a con 0x7f5e8c1381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c06f340 con 0x7f5e8c1381e0 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c06f1c0 con 0x7f5e8c1378c0 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c1425c0 con 0x7f5e8c0a5040 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e9281e640 10 monclient(hunting): authenticate will time out at 1960.787540s 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e8c1381e0 0x7f5e8c13ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c0a5040 0x7f5e8c1cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e8c1381e0 0x7f5e8c13ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33548/0 (socket says 172.21.15.17:33548) 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 1 -- 172.21.15.17:0/4003363460 learned_addr learned my addr 172.21.15.17:0/4003363460 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): get_auth_request con 0x7f5e8c1381e0 auth_method 0 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c0a5040 0x7f5e8c1cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47122/0 (socket says 172.21.15.17:47122) 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): get_auth_request con 0x7f5e8c0a5040 auth_method 0 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e90dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.389+0000 7f5e8b7fe640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c1378c0 0x7f5e8c136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 10 monclient(hunting): get_auth_request con 0x7f5e8c1378c0 auth_method 0 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 10 monclient(hunting): handle_auth_done global_id 174354 payload 824 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c0a5040 msgr2=0x7f5e8c1cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c0a5040 0x7f5e8c1cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c1378c0 msgr2=0x7f5e8c136b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c1378c0 0x7f5e8c136b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 monclient: found mon.noname-a 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c0a12f0 con 0x7f5e8c1381e0 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e8c0a5040 0x7f5e8c1cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c1378c0 0x7f5e8c136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e90dbb640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e8c1381e0 0x7f5e8c13ccc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c3c0 tx=0x7f5e7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c011790 con 0x7f5e8c1381e0 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: dump: 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 5 monclient: authenticate success, global_id 174354 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 10 monclient: _renew_subs 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c121490 con 0x7f5e8c1381e0 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c017070 con 0x7f5e8c1381e0 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c011b70 con 0x7f5e8c1381e0 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: dump: 2024-09-15T21:33:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8a7fc640 4 set_mon_vals no callback set 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 10 monclient: _renew_subs 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1d07d0 con 0x7f5e8c1381e0 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 0x7f5e8c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c121040 con 0x7f5e8c137040 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e7c011d10 con 0x7f5e8c1381e0 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6c03b9c0 0x7f5e6c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 0x7f5e8c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient: get_auth_request con 0x7f5e8c137040 auth_method 0 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5e7c050560 con 0x7f5e8c1381e0 2024-09-15T21:33:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6c03b9c0 0x7f5e6c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 10 monclient: get_auth_request con 0x7f5e6c03b9c0 auth_method 0 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_done global_id 194113 payload 824 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e8c137ca0 con 0x7f5e8c137040 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8b7fe640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6c03b9c0 0x7f5e6c03de80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5e74006fd0 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 0x7f5e8c02d450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0a6150 tx=0x7f5e84019720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84009510 con 0x7f5e8c137040 2024-09-15T21:33:57.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.393+0000 7f5e88ff9640 10 monclient: dump: 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5e840a5640 con 0x7f5e8c137040 2024-09-15T21:33:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 msgr2=0x7f5e8c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 0x7f5e8c02d450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0a6150 tx=0x7f5e84019720 comp rx=0 tx=0).stop 2024-09-15T21:33:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 msgr2=0x7f5e8c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.397+0000 7f5e88ff9640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c137040 0x7f5e8c02d450 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0a6150 tx=0x7f5e84019720 comp rx=0 tx=0).stop 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e9281e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e9281e640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 0x7f5e8c02dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e9281e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c09f3c0 con 0x7f5e8c02d990 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 0x7f5e8c02dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient: get_auth_request con 0x7f5e8c02d990 auth_method 0 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 10 monclient(hunting): handle_auth_done global_id 194116 payload 824 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.401+0000 7f5e8bfff640 1 -- 172.21.15.17:0/4003363460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5e8c069270 con 0x7f5e8c02d990 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e8bfff640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 0x7f5e8c02dda0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009e50 tx=0x7f5e840a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840c9410 con 0x7f5e8c02d990 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 10 monclient: dump: 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.417 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f5e840c9410 con 0x7f5e8c02d990 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:57.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 msgr2=0x7f5e8c02dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 0x7f5e8c02dda0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009e50 tx=0x7f5e840a7000 comp rx=0 tx=0).stop 2024-09-15T21:33:57.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 msgr2=0x7f5e8c02dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:57.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e88ff9640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c02d990 0x7f5e8c02dda0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009e50 tx=0x7f5e840a7000 comp rx=0 tx=0).stop 2024-09-15T21:33:57.420 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:57.420 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:57.420 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:57.420 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 152, 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 4195, 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:57.434 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:57.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6c03b9c0 msgr2=0x7f5e6c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e6c03b9c0 0x7f5e6c03de80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5e74006fd0 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 10 monclient: shutdown 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e8c1381e0 msgr2=0x7f5e8c13ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 --2- 172.21.15.17:0/4003363460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e8c1381e0 0x7f5e8c13ccc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c3c0 tx=0x7f5e7c00c880 comp rx=0 tx=0).stop 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e90dbb640 1 -- 172.21.15.17:0/4003363460 reap_dead start 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e90dbb640 1 -- 172.21.15.17:0/4003363460 reap_dead start 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 shutdown_connections 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 >> 172.21.15.17:0/4003363460 conn(0x7f5e8c0aac40 msgr2=0x7f5e8c0a0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 shutdown_connections 2024-09-15T21:33:57.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.405+0000 7f5e9281e640 1 -- 172.21.15.17:0/4003363460 wait complete. 2024-09-15T21:33:57.437 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:57.438 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:33:57.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:33:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.549+0000 7fd36a8eb640 10 monclient: get_monmap_and_config 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.549+0000 7fd36a8eb640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.549+0000 7fd36a8eb640 10 monclient: monmap: 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:57.553836+0000 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:57.553836+0000 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:57.555 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:57.555 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:57.555 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:57.555 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 Processor -- start 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 -- start start 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient: init 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd36409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd364139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): picked mon.noname-b con 0x7fd36409def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd364139ff0 0x7fd36413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): picked mon.noname-a con 0x7fd364139ff0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3640a3130 con 0x7fd36409d400 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd3640a1bc0 con 0x7fd364139ff0 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd3640a3490 con 0x7fd36409def0 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd36a8eb640 10 monclient(hunting): authenticate will time out at 1960.949343s 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd364139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd364139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:47126/0 (socket says 172.21.15.17:47126) 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 1 -- 172.21.15.17:0/1087508574 learned_addr learned my addr 172.21.15.17:0/1087508574 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): get_auth_request con 0x7fd36409def0 auth_method 0 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 1 --2- 172.21.15.17:0/1087508574 >> v2:172.21.15.17:3300/0 conn(0x7fd364139ff0 0x7fd36413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): get_auth_request con 0x7fd364139ff0 auth_method 0 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient(hunting): handle_auth_done global_id 174357 payload 274 2024-09-15T21:33:57.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 1 -- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 msgr2=0x7fd364139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 1 --2- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd364139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 1 -- 172.21.15.17:0/1087508574 >> v1:172.21.15.119:6789/0 conn(0x7fd36409d400 legacy=0x7fd36409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 1 monclient: found mon.noname-a 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd368e88640 1 -- 172.21.15.17:0/1087508574 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36409e7a0 con 0x7fd364139ff0 2024-09-15T21:33:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.553+0000 7fd3637fe640 1 --2- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd364139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 --2- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd36413c3e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097c0 tx=0x7fd35c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814f14efd894ebe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 1 -- 172.21.15.17:0/1087508574 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0b0070 con 0x7fd364139ff0 2024-09-15T21:33:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: dump: 2024-09-15T21:33:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 1 -- 172.21.15.17:0/1087508574 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c0ab600 con 0x7fd364139ff0 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 1 -- 172.21.15.17:0/1087508574 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0ab9d0 con 0x7fd364139ff0 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 5 monclient: authenticate success, global_id 174357 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 4 set_mon_vals no callback set 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd362ffd640 10 monclient: dump: 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: shutdown 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 msgr2=0x7fd36413c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd36413c3e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097c0 tx=0x7fd35c01c360 comp rx=0 tx=0).stop 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/1087508574 shutdown_connections 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd36413c3e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/1087508574 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd364139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/1087508574 >> 172.21.15.17:0/1087508574 conn(0x7fd3640a66a0 msgr2=0x7fd3640a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/1087508574 shutdown_connections 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/1087508574 wait complete. 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: build_initial_monmap 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: monmap: 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:57.572 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 Processor -- start 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- start start 2024-09-15T21:33:57.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: init 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd36409d400 0x7fd36412db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): picked mon.noname-b con 0x7fd36409d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd36412e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): picked mon.noname-c con 0x7fd36409def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd3641cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): picked mon.noname-a con 0x7fd364139ff0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:57.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd3640a0a10 con 0x7fd364139ff0 2024-09-15T21:33:57.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3640a0890 con 0x7fd36409d400 2024-09-15T21:33:57.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd3640a0b90 con 0x7fd36409def0 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient(hunting): authenticate will time out at 1960.954496s 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd3641cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd3641cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33566/0 (socket says 172.21.15.17:33566) 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 -- 172.21.15.17:0/4012909553 learned_addr learned my addr 172.21.15.17:0/4012909553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): get_auth_request con 0x7fd364139ff0 auth_method 0 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd36412e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): get_auth_request con 0x7fd36409def0 auth_method 0 2024-09-15T21:33:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd363fff640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd36409d400 0x7fd36412db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient(hunting): handle_auth_done global_id 174360 payload 824 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 -- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 msgr2=0x7fd36412e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd36412e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 -- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd36409d400 msgr2=0x7fd36412db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd36409d400 0x7fd36412db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient: _finish_hunting 0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 monclient: found mon.noname-a 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 -- 172.21.15.17:0/4012909553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd364139990 con 0x7fd364139ff0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd363fff640 10 monclient: get_auth_request con 0x7fd36409d400 auth_method 0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd363fff640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd36409d400 0x7fd36412db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3637fe640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd36412e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd368e88640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd3641cb1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd35c01c190 tx=0x7fd35c019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0b0040 con 0x7fd364139ff0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: dump: 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: _finish_auth 0 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:33:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c0b9070 con 0x7fd364139ff0 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0b4650 con 0x7fd364139ff0 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd360ff9640 10 monclient: dump: 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 5 monclient: authenticate success, global_id 174360 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: _renew_subs 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36406f230 con 0x7fd364139ff0 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd3627fc640 4 set_mon_vals no callback set 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: _renew_subs 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.557+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641396b0 con 0x7fd364139ff0 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd36a8eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd36a8eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd36a8eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd35c0b47f0 con 0x7fd364139ff0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36409d7e0 con 0x7fd364139ff0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd360ff9640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd35003b9c0 0x7fd35003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd35c0ea330 con 0x7fd364139ff0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd363fff640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd35003b9c0 0x7fd35003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd363fff640 10 monclient: get_auth_request con 0x7fd35003b9c0 auth_method 0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.561+0000 7fd363fff640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd35003b9c0 0x7fd35003de80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd354003e40 tx=0x7fd354009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.565+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd35c016070 con 0x7fd364139ff0 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.565+0000 7fd360ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.565+0000 7fd360ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd36a8eb640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:33:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd36a8eb640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:33:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd36a8eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd364074090 con 0x7fd364139ff0 2024-09-15T21:33:57.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd360ff9640 1 -- 172.21.15.17:0/4012909553 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd35c0b3390 con 0x7fd364139ff0 2024-09-15T21:33:57.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd360ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:33:57.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.805+0000 7fd360ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:57.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.805+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.805+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=11,kv=12,logm=48} crc {auth=1545915,config=3541143443,health=1933779806,kv=3079713637,logm=2713072998}) 2024-09-15T21:33:57.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.805+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=10,kv=11,logm=50} crc {auth=2293225617,config=3541143443,health=3692557161,kv=3059833255,logm=2954859313}) 2024-09-15T21:33:57.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.805+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.805+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=11,kv=12,logm=48} crc {auth=1545915,config=3541143443,health=1933779806,kv=3079713637,logm=2713072998}) 2024-09-15T21:33:57.810 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.805+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=1631040406,config=3541143443,health=1350913232,kv=3079713637,logm=4249352678}) 2024-09-15T21:33:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd35003b9c0 msgr2=0x7fd35003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd35003b9c0 0x7fd35003de80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd354003e40 tx=0x7fd354009040 comp rx=0 tx=0).stop 2024-09-15T21:33:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 10 monclient: shutdown 2024-09-15T21:33:57.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.809+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.809+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=618922835}) 2024-09-15T21:33:57.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.809+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=903771753}) 2024-09-15T21:33:57.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.809+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.809+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=618922835}) 2024-09-15T21:33:57.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.809+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4096566188}) 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 msgr2=0x7fd3641cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd3641cb1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd35c01c190 tx=0x7fd35c019100 comp rx=0 tx=0).stop 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 shutdown_connections 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd35003b9c0 0x7fd35003de80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd364139ff0 0x7fd3641cb1b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd36409def0 0x7fd36412e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 --2- 172.21.15.17:0/4012909553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd36409d400 0x7fd36412db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:57.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 >> 172.21.15.17:0/4012909553 conn(0x7fd3640a66a0 msgr2=0x7fd36413c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:57.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 shutdown_connections 2024-09-15T21:33:57.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:57.809+0000 7fd36a8eb640 1 -- 172.21.15.17:0/4012909553 wait complete. 2024-09-15T21:33:57.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.815 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1498896287}) 2024-09-15T21:33:57.815 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2850921569}) 2024-09-15T21:33:57.815 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.815 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1498896287}) 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3000770452}) 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646007190}) 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3949505722}) 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646007190}) 2024-09-15T21:33:57.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.813+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1053379891}) 2024-09-15T21:33:57.819 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.819 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454587832}) 2024-09-15T21:33:57.819 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3598812965}) 2024-09-15T21:33:57.819 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.819 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454587832}) 2024-09-15T21:33:57.819 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989588208}) 2024-09-15T21:33:57.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1136526920}) 2024-09-15T21:33:57.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=68043372}) 2024-09-15T21:33:57.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1136526920}) 2024-09-15T21:33:57.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.817+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=667977008}) 2024-09-15T21:33:57.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207226626}) 2024-09-15T21:33:57.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=389285454}) 2024-09-15T21:33:57.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207226626}) 2024-09-15T21:33:57.824 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25622684}) 2024-09-15T21:33:57.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2330582609}) 2024-09-15T21:33:57.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4020279142}) 2024-09-15T21:33:57.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2330582609}) 2024-09-15T21:33:57.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.821+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2716909961}) 2024-09-15T21:33:57.827 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1688485814}) 2024-09-15T21:33:57.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=618784925}) 2024-09-15T21:33:57.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1688485814}) 2024-09-15T21:33:57.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3412570408}) 2024-09-15T21:33:57.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:33:57.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014037872}) 2024-09-15T21:33:57.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3643797229}) 2024-09-15T21:33:57.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014037872}) 2024-09-15T21:33:57.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.825+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2009466586}) 2024-09-15T21:33:57.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2328695254}) 2024-09-15T21:33:57.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3566228269}) 2024-09-15T21:33:57.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2328695254}) 2024-09-15T21:33:57.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1281698053}) 2024-09-15T21:33:57.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624594658}) 2024-09-15T21:33:57.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=908770724}) 2024-09-15T21:33:57.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624594658}) 2024-09-15T21:33:57.833 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.829+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2909637558}) 2024-09-15T21:33:57.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2878462087}) 2024-09-15T21:33:57.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1628156016}) 2024-09-15T21:33:57.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2878462087}) 2024-09-15T21:33:57.835 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1092555828}) 2024-09-15T21:33:57.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2677970008}) 2024-09-15T21:33:57.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2526779424}) 2024-09-15T21:33:57.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2677970008}) 2024-09-15T21:33:57.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.833+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1267748089}) 2024-09-15T21:33:57.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=144300458}) 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3512326229}) 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=144300458}) 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3274285669}) 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524560340}) 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3703725942}) 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.841 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524560340}) 2024-09-15T21:33:57.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.837+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3234573335}) 2024-09-15T21:33:57.843 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.843 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051188138}) 2024-09-15T21:33:57.843 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=617538716}) 2024-09-15T21:33:57.843 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051188138}) 2024-09-15T21:33:57.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3185470724}) 2024-09-15T21:33:57.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1870665217}) 2024-09-15T21:33:57.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2935055068}) 2024-09-15T21:33:57.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1870665217}) 2024-09-15T21:33:57.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296741446}) 2024-09-15T21:33:57.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.841+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2416595867}) 2024-09-15T21:33:57.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2625337760}) 2024-09-15T21:33:57.846 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2416595867}) 2024-09-15T21:33:57.847 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1383473226}) 2024-09-15T21:33:57.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3580050462}) 2024-09-15T21:33:57.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=743378}) 2024-09-15T21:33:57.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3580050462}) 2024-09-15T21:33:57.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.845+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3571362617}) 2024-09-15T21:33:57.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1838416703}) 2024-09-15T21:33:57.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2290135044}) 2024-09-15T21:33:57.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1838416703}) 2024-09-15T21:33:57.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4116071380}) 2024-09-15T21:33:57.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1296660950}) 2024-09-15T21:33:57.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=812709711}) 2024-09-15T21:33:57.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1296660950}) 2024-09-15T21:33:57.852 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=911676103}) 2024-09-15T21:33:57.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=378610408}) 2024-09-15T21:33:57.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2130950780}) 2024-09-15T21:33:57.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.849+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=378610408}) 2024-09-15T21:33:57.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1751451911}) 2024-09-15T21:33:57.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=965541774}) 2024-09-15T21:33:57.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4159661912}) 2024-09-15T21:33:57.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=965541774}) 2024-09-15T21:33:57.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.853+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2688700811}) 2024-09-15T21:33:57.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407928291}) 2024-09-15T21:33:57.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2248888734}) 2024-09-15T21:33:57.859 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.859 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407928291}) 2024-09-15T21:33:57.859 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=823393197}) 2024-09-15T21:33:57.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520076930}) 2024-09-15T21:33:57.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2189040623}) 2024-09-15T21:33:57.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520076930}) 2024-09-15T21:33:57.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.857+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620775147}) 2024-09-15T21:33:57.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=870597655}) 2024-09-15T21:33:57.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3161613727}) 2024-09-15T21:33:57.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=870597655}) 2024-09-15T21:33:57.862 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035564250}) 2024-09-15T21:33:57.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3858345421}) 2024-09-15T21:33:57.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1916094410}) 2024-09-15T21:33:57.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3858345421}) 2024-09-15T21:33:57.864 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.861+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2084454183}) 2024-09-15T21:33:57.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2371122795}) 2024-09-15T21:33:57.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1110320649}) 2024-09-15T21:33:57.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.866 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2371122795}) 2024-09-15T21:33:57.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3331265601}) 2024-09-15T21:33:57.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1661412583}) 2024-09-15T21:33:57.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=861806130}) 2024-09-15T21:33:57.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1661412583}) 2024-09-15T21:33:57.868 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.865+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3042267846}) 2024-09-15T21:33:57.870 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2852863714}) 2024-09-15T21:33:57.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3227980498}) 2024-09-15T21:33:57.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2852863714}) 2024-09-15T21:33:57.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3670875253}) 2024-09-15T21:33:57.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3058701959}) 2024-09-15T21:33:57.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3745058773}) 2024-09-15T21:33:57.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3058701959}) 2024-09-15T21:33:57.873 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3279477148}) 2024-09-15T21:33:57.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2067807731}) 2024-09-15T21:33:57.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1163752356}) 2024-09-15T21:33:57.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.869+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2067807731}) 2024-09-15T21:33:57.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4201579313}) 2024-09-15T21:33:57.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=413346384}) 2024-09-15T21:33:57.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3351030659}) 2024-09-15T21:33:57.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=413346384}) 2024-09-15T21:33:57.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2023261234}) 2024-09-15T21:33:57.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2364226801}) 2024-09-15T21:33:57.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2349264731}) 2024-09-15T21:33:57.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2364226801}) 2024-09-15T21:33:57.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.873+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3414071527}) 2024-09-15T21:33:57.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196851528}) 2024-09-15T21:33:57.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2463648947}) 2024-09-15T21:33:57.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196851528}) 2024-09-15T21:33:57.881 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3649590721}) 2024-09-15T21:33:57.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2061327092}) 2024-09-15T21:33:57.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3816379910}) 2024-09-15T21:33:57.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2061327092}) 2024-09-15T21:33:57.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.877+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=700217711}) 2024-09-15T21:33:57.883 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014386735}) 2024-09-15T21:33:57.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2369905515}) 2024-09-15T21:33:57.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014386735}) 2024-09-15T21:33:57.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3492500469}) 2024-09-15T21:33:57.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4084066722}) 2024-09-15T21:33:57.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=344871272}) 2024-09-15T21:33:57.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4084066722}) 2024-09-15T21:33:57.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.881+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=690799582}) 2024-09-15T21:33:57.887 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813149641}) 2024-09-15T21:33:57.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4266471953}) 2024-09-15T21:33:57.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813149641}) 2024-09-15T21:33:57.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1964013679}) 2024-09-15T21:33:57.890 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.890 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1338028092}) 2024-09-15T21:33:57.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540206358}) 2024-09-15T21:33:57.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1338028092}) 2024-09-15T21:33:57.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.885+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2733770077}) 2024-09-15T21:33:57.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808875425}) 2024-09-15T21:33:57.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237955350}) 2024-09-15T21:33:57.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808875425}) 2024-09-15T21:33:57.892 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1478005439}) 2024-09-15T21:33:57.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1908752866}) 2024-09-15T21:33:57.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2276316535}) 2024-09-15T21:33:57.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1908752866}) 2024-09-15T21:33:57.894 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.889+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=367552684}) 2024-09-15T21:33:57.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1017405369}) 2024-09-15T21:33:57.896 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3567047680}) 2024-09-15T21:33:57.896 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.896 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1017405369}) 2024-09-15T21:33:57.896 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3483840024}) 2024-09-15T21:33:57.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1236136184}) 2024-09-15T21:33:57.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1913618807}) 2024-09-15T21:33:57.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1236136184}) 2024-09-15T21:33:57.898 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.893+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3916748910}) 2024-09-15T21:33:57.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2175980873}) 2024-09-15T21:33:57.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2617804298}) 2024-09-15T21:33:57.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2175980873}) 2024-09-15T21:33:57.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2804569749}) 2024-09-15T21:33:57.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2671820843}) 2024-09-15T21:33:57.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3525878118}) 2024-09-15T21:33:57.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2671820843}) 2024-09-15T21:33:57.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.897+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364762101}) 2024-09-15T21:33:57.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980860618}) 2024-09-15T21:33:57.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3938568733}) 2024-09-15T21:33:57.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980860618}) 2024-09-15T21:33:57.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1343653800}) 2024-09-15T21:33:57.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932564797}) 2024-09-15T21:33:57.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3783387977}) 2024-09-15T21:33:57.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932564797}) 2024-09-15T21:33:57.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.901+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=146956132}) 2024-09-15T21:33:57.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=352643602}) 2024-09-15T21:33:57.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2256283177}) 2024-09-15T21:33:57.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=352643602}) 2024-09-15T21:33:57.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=198981861}) 2024-09-15T21:33:57.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940962841}) 2024-09-15T21:33:57.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2438001335}) 2024-09-15T21:33:57.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3940962841}) 2024-09-15T21:33:57.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.905+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2320100856}) 2024-09-15T21:33:57.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1351045176}) 2024-09-15T21:33:57.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1093725196}) 2024-09-15T21:33:57.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1351045176}) 2024-09-15T21:33:57.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1117987754}) 2024-09-15T21:33:57.912 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=678817781}) 2024-09-15T21:33:57.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=386799710}) 2024-09-15T21:33:57.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=678817781}) 2024-09-15T21:33:57.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.909+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1159800062}) 2024-09-15T21:33:57.914 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.914 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=320531843}) 2024-09-15T21:33:57.914 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1290500514}) 2024-09-15T21:33:57.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=320531843}) 2024-09-15T21:33:57.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1816593432}) 2024-09-15T21:33:57.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=600777656}) 2024-09-15T21:33:57.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811537630}) 2024-09-15T21:33:57.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=600777656}) 2024-09-15T21:33:57.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.913+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2957586978}) 2024-09-15T21:33:57.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785649284}) 2024-09-15T21:33:57.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2856785977}) 2024-09-15T21:33:57.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785649284}) 2024-09-15T21:33:57.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3643011592}) 2024-09-15T21:33:57.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752553318}) 2024-09-15T21:33:57.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3696151278}) 2024-09-15T21:33:57.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2752553318}) 2024-09-15T21:33:57.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.917+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2850456738}) 2024-09-15T21:33:57.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2935252666}) 2024-09-15T21:33:57.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2204223052}) 2024-09-15T21:33:57.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2935252666}) 2024-09-15T21:33:57.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2372824062}) 2024-09-15T21:33:57.924 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.924 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1108357973}) 2024-09-15T21:33:57.924 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4101249152}) 2024-09-15T21:33:57.924 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.924 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1108357973}) 2024-09-15T21:33:57.925 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.921+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249369117}) 2024-09-15T21:33:57.926 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.926 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2425403808}) 2024-09-15T21:33:57.926 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3504025965}) 2024-09-15T21:33:57.926 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2425403808}) 2024-09-15T21:33:57.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=668496293}) 2024-09-15T21:33:57.928 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.928 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1123475673}) 2024-09-15T21:33:57.928 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029815971}) 2024-09-15T21:33:57.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1123475673}) 2024-09-15T21:33:57.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.925+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=882225507}) 2024-09-15T21:33:57.930 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.930 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1963553140}) 2024-09-15T21:33:57.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=726754910}) 2024-09-15T21:33:57.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1963553140}) 2024-09-15T21:33:57.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3337336334}) 2024-09-15T21:33:57.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3501984963}) 2024-09-15T21:33:57.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3823720699}) 2024-09-15T21:33:57.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3501984963}) 2024-09-15T21:33:57.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.929+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3011011462}) 2024-09-15T21:33:57.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1995653479}) 2024-09-15T21:33:57.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4116121337}) 2024-09-15T21:33:57.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1995653479}) 2024-09-15T21:33:57.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434264961}) 2024-09-15T21:33:57.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.938 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3770055248}) 2024-09-15T21:33:57.938 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=438097435}) 2024-09-15T21:33:57.938 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.938 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3770055248}) 2024-09-15T21:33:57.938 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.933+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2437902131}) 2024-09-15T21:33:57.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3432338504}) 2024-09-15T21:33:57.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1696828533}) 2024-09-15T21:33:57.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.940 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3432338504}) 2024-09-15T21:33:57.940 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3177037481}) 2024-09-15T21:33:57.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3637795349}) 2024-09-15T21:33:57.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2182428358}) 2024-09-15T21:33:57.942 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.942 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3637795349}) 2024-09-15T21:33:57.942 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.937+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3685783091}) 2024-09-15T21:33:57.944 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.941+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.944 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.941+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=443835294}) 2024-09-15T21:33:57.944 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.941+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=884760414}) 2024-09-15T21:33:57.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.941+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.941+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=443835294}) 2024-09-15T21:33:57.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.941+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1965241086}) 2024-09-15T21:33:57.946 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.946 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3442372659}) 2024-09-15T21:33:57.946 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3052537039}) 2024-09-15T21:33:57.946 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.946 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3442372659}) 2024-09-15T21:33:57.946 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=248746831}) 2024-09-15T21:33:57.948 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.948 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=789319180}) 2024-09-15T21:33:57.948 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=818847981}) 2024-09-15T21:33:57.948 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.948 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=789319180}) 2024-09-15T21:33:57.948 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.945+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=45709822}) 2024-09-15T21:33:57.950 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.950 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977086791}) 2024-09-15T21:33:57.950 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2846985081}) 2024-09-15T21:33:57.950 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.950 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977086791}) 2024-09-15T21:33:57.950 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291070242}) 2024-09-15T21:33:57.952 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.952 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380034060}) 2024-09-15T21:33:57.952 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3526155720}) 2024-09-15T21:33:57.952 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.952 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380034060}) 2024-09-15T21:33:57.952 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.949+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=908589084}) 2024-09-15T21:33:57.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3614187521}) 2024-09-15T21:33:57.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=633516286}) 2024-09-15T21:33:57.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3614187521}) 2024-09-15T21:33:57.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4075308422}) 2024-09-15T21:33:57.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=670107060}) 2024-09-15T21:33:57.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2694713517}) 2024-09-15T21:33:57.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=670107060}) 2024-09-15T21:33:57.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.953+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143018745}) 2024-09-15T21:33:57.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1053367365}) 2024-09-15T21:33:57.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3726511876}) 2024-09-15T21:33:57.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1053367365}) 2024-09-15T21:33:57.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3172434940}) 2024-09-15T21:33:57.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2320746887}) 2024-09-15T21:33:57.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3350744943}) 2024-09-15T21:33:57.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2320746887}) 2024-09-15T21:33:57.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.957+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=551066034}) 2024-09-15T21:33:57.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494080443}) 2024-09-15T21:33:57.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=206827853}) 2024-09-15T21:33:57.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494080443}) 2024-09-15T21:33:57.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3355132275}) 2024-09-15T21:33:57.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3644759364}) 2024-09-15T21:33:57.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3045012254}) 2024-09-15T21:33:57.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3644759364}) 2024-09-15T21:33:57.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.961+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3476395851}) 2024-09-15T21:33:57.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4196804245}) 2024-09-15T21:33:57.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502920318}) 2024-09-15T21:33:57.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4196804245}) 2024-09-15T21:33:57.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4104311030}) 2024-09-15T21:33:57.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4214689272}) 2024-09-15T21:33:57.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388242653}) 2024-09-15T21:33:57.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4214689272}) 2024-09-15T21:33:57.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.965+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2040990594}) 2024-09-15T21:33:57.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=910258921}) 2024-09-15T21:33:57.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3326256025}) 2024-09-15T21:33:57.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=910258921}) 2024-09-15T21:33:57.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2916389078}) 2024-09-15T21:33:57.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1621884073}) 2024-09-15T21:33:57.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=644461826}) 2024-09-15T21:33:57.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1621884073}) 2024-09-15T21:33:57.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.969+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=522185367}) 2024-09-15T21:33:57.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733364279}) 2024-09-15T21:33:57.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=697685740}) 2024-09-15T21:33:57.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733364279}) 2024-09-15T21:33:57.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2947009423}) 2024-09-15T21:33:57.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295657622}) 2024-09-15T21:33:57.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3157198867}) 2024-09-15T21:33:57.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295657622}) 2024-09-15T21:33:57.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.973+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3534782572}) 2024-09-15T21:33:57.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2268127924}) 2024-09-15T21:33:57.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3322926234}) 2024-09-15T21:33:57.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2268127924}) 2024-09-15T21:33:57.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3049165495}) 2024-09-15T21:33:57.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153688785}) 2024-09-15T21:33:57.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4016263883}) 2024-09-15T21:33:57.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153688785}) 2024-09-15T21:33:57.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.977+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1658891526}) 2024-09-15T21:33:57.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=698687608}) 2024-09-15T21:33:57.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1467722191}) 2024-09-15T21:33:57.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=698687608}) 2024-09-15T21:33:57.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3649588849}) 2024-09-15T21:33:57.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=222471726}) 2024-09-15T21:33:57.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086947085}) 2024-09-15T21:33:57.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=222471726}) 2024-09-15T21:33:57.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.981+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3951903878}) 2024-09-15T21:33:57.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776558473}) 2024-09-15T21:33:57.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2952073110}) 2024-09-15T21:33:57.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.987 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776558473}) 2024-09-15T21:33:57.987 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4106649472}) 2024-09-15T21:33:57.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=438171389}) 2024-09-15T21:33:57.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2741060016}) 2024-09-15T21:33:57.989 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.989 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=438171389}) 2024-09-15T21:33:57.989 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.985+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3771264768}) 2024-09-15T21:33:57.990 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.990 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1657221899}) 2024-09-15T21:33:57.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2154815064}) 2024-09-15T21:33:57.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1657221899}) 2024-09-15T21:33:57.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3523088493}) 2024-09-15T21:33:57.993 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.993 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102118405}) 2024-09-15T21:33:57.993 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4242490227}) 2024-09-15T21:33:57.993 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.993 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102118405}) 2024-09-15T21:33:57.993 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.989+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1596227215}) 2024-09-15T21:33:57.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4068169126}) 2024-09-15T21:33:57.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1418644041}) 2024-09-15T21:33:57.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4068169126}) 2024-09-15T21:33:57.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2974645482}) 2024-09-15T21:33:57.997 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.997 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=267910544}) 2024-09-15T21:33:57.997 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1853050501}) 2024-09-15T21:33:57.997 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.997 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=267910544}) 2024-09-15T21:33:57.997 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.993+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1907278425}) 2024-09-15T21:33:57.999 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.997+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:57.999 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.997+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133033627}) 2024-09-15T21:33:58.000 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.997+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978664980}) 2024-09-15T21:33:58.000 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.997+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:58.000 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.997+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133033627}) 2024-09-15T21:33:58.000 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:57.997+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=377852790}) 2024-09-15T21:33:58.002 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.001+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:58.003 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.001+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=45} crc {logm=1867306808,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3086264284,mon_config_key=2256006344,monmap=186996059,nvmeofgw=3140514009,osd_metadata=3068027012,osd_pg_creating=543505836,osd_snap=841190635,osdmap=744204856}) 2024-09-15T21:33:58.003 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.001+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=43} crc {logm=1867306808,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3086264284,mon_config_key=1677881253,monmap=186996059,nvmeofgw=2613832008,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1194980547}) 2024-09-15T21:33:58.003 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.001+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:58.003 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.001+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,nvmeofgw=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=45} crc {logm=1867306808,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3086264284,mon_config_key=2256006344,monmap=186996059,nvmeofgw=3140514009,osd_metadata=3068027012,osd_pg_creating=543505836,osd_snap=841190635,osdmap=744204856}) 2024-09-15T21:33:58.003 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.001+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=6,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=45} crc {logm=1867306808,mdsmap=173658045,mgr=2644266932,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3086264284,mon_config_key=2820227852,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3941347924}) 2024-09-15T21:33:58.015 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.013+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:58.015 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.013+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=7} crc {osdmap=3754327856}) 2024-09-15T21:33:58.015 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.013+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=9} crc {osdmap=1715420528}) 2024-09-15T21:33:58.015 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.013+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:33:58.015 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.013+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=7} crc {osdmap=3754327856}) 2024-09-15T21:33:58.015 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:33:58.013+0000 7f22c2a6e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=9} crc {osdmap=1715420528}) 2024-09-15T21:33:58.829 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:58.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient: get_monmap_and_config 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient: build_initial_monmap 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient: monmap: 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:58.960321+0000 2024-09-15T21:33:58.960 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:58.960321+0000 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:58.961 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 1 Processor -- start 2024-09-15T21:33:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 1 -- start start 2024-09-15T21:33:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient: init 2024-09-15T21:33:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:33:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1fe80a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1fe809ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe813e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient(hunting): picked mon.noname-b con 0x7f1fe80a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe80a3130 con 0x7f1fe80a0890 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe8073df0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe80a1bf0 con 0x7f1fe80a1380 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fefa6d640 10 monclient(hunting): authenticate will time out at 1962.355721s 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fee00a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe813e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe813e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35630/0 (socket says 172.21.15.17:35630) 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 1 -- 172.21.15.17:0/583497228 learned_addr learned my addr 172.21.15.17:0/583497228 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): get_auth_request con 0x7f1fe80a1380 auth_method 0 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.957+0000 7f1fed809640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1fe80a0890 0x7f1fe80a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44864/0 (socket says 172.21.15.17:44864) 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient(hunting): handle_auth_done global_id 164432 payload 274 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 1 -- 172.21.15.17:0/583497228 >> v1:172.21.15.119:6789/0 conn(0x7f1fe809ffc0 legacy=0x7f1fe813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 1 -- 172.21.15.17:0/583497228 >> v1:172.21.15.17:6789/0 conn(0x7f1fe80a0890 legacy=0x7f1fe80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fec807640 1 -- 172.21.15.17:0/583497228 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788184501 0 0) 0x7f1fe80a3130 con 0x7f1fe80a0890 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient: _finish_hunting 0 2024-09-15T21:33:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 1 monclient: found mon.noname-b 2024-09-15T21:33:58.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:58.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:58.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fee00a640 1 -- 172.21.15.17:0/583497228 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe809ccd0 con 0x7f1fe80a1380 2024-09-15T21:33:58.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.961+0000 7f1fec807640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:33:58.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fee00a640 1 --2- 172.21.15.17:0/583497228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe813e1f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009eb0 tx=0x7f1fe001c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5ddc2d64ce439c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 1 -- 172.21.15.17:0/583497228 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00a56c0 con 0x7f1fe80a1380 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: dump: 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:58.968 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: _finish_auth 0 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: _check_auth_tickets 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 5 monclient: authenticate success, global_id 164432 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 1 -- 172.21.15.17:0/583497228 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe00a5860 con 0x7f1fe80a1380 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 1 -- 172.21.15.17:0/583497228 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00a5c30 con 0x7f1fe80a1380 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 4 set_mon_vals no callback set 2024-09-15T21:33:58.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fec807640 10 monclient: dump: 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 10 monclient: shutdown 2024-09-15T21:33:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:58.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 -- 172.21.15.17:0/583497228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 msgr2=0x7f1fe813e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:58.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/583497228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe813e1f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009eb0 tx=0x7f1fe001c8f0 comp rx=0 tx=0).stop 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 -- 172.21.15.17:0/583497228 shutdown_connections 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/583497228 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe813e1f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 -- 172.21.15.17:0/583497228 >> 172.21.15.17:0/583497228 conn(0x7f1fe80a66a0 msgr2=0x7f1fe80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 -- 172.21.15.17:0/583497228 shutdown_connections 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 -- 172.21.15.17:0/583497228 wait complete. 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 10 monclient: build_initial_monmap 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 10 monclient: monmap: 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 Processor -- start 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 1 -- start start 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.965+0000 7f1fefa6d640 10 monclient: init 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 0x7f1fe809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): picked mon.noname-a con 0x7f1fe809ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe80a0890 0x7f1fe809d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): picked mon.noname-b con 0x7f1fe80a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe809de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): picked mon.noname-c con 0x7f1fe80a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81412e0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8141160 con 0x7f1fe80a0890 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe8141460 con 0x7f1fe80a1380 2024-09-15T21:33:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient(hunting): authenticate will time out at 1962.364173s 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 0x7f1fe809d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe809de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 0x7f1fe809d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58570/0 (socket says 172.21.15.17:58570) 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe809de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35636/0 (socket says 172.21.15.17:35636) 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 -- 172.21.15.17:0/446053994 learned_addr learned my addr 172.21.15.17:0/446053994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): get_auth_request con 0x7f1fe809ffc0 auth_method 0 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): get_auth_request con 0x7f1fe80a1380 auth_method 0 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe80a0890 0x7f1fe809d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 10 monclient(hunting): get_auth_request con 0x7f1fe80a0890 auth_method 0 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient(hunting): handle_auth_done global_id 174366 payload 824 2024-09-15T21:33:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 -- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 msgr2=0x7f1fe809de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe809de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 -- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe80a0890 msgr2=0x7f1fe809d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe80a0890 0x7f1fe809d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient: _finish_hunting 0 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 monclient: found mon.noname-a 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 -- 172.21.15.17:0/446053994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe8120fe0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fee00a640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe809de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed809640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 0x7f1fe809d400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4001230 tx=0x7f1fe400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40074e0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: dump: 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe4007680 con 0x7f1fe809ffc0 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4004a30 con 0x7f1fe809ffc0 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 5 monclient: authenticate success, global_id 174366 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _renew_subs 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe806f5f0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fdb7fe640 4 set_mon_vals no callback set 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 10 monclient: dump: 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _renew_subs 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe809f0c0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1fe4011070 con 0x7f1fe809ffc0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe80a0c70 con 0x7f1fe809ffc0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03b9c0 0x7f1fcc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f1fe4050340 con 0x7f1fe809ffc0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03b9c0 0x7f1fcc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 10 monclient: get_auth_request con 0x7f1fcc03b9c0 auth_method 0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.969+0000 7f1fed008640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03b9c0 0x7f1fcc03de80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1fe813d6b0 tx=0x7f1fdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.973+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1fe4012dd0 con 0x7f1fe809ffc0 2024-09-15T21:33:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.973+0000 7f1fda7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:58.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:58.973+0000 7f1fda7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:59.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:33:59.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:59.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:59.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1fe80008e0 con 0x7f1fe809ffc0 2024-09-15T21:33:59.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fda7fc640 1 -- 172.21.15.17:0/446053994 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f1fe40138e0 con 0x7f1fe809ffc0 2024-09-15T21:33:59.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fda7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:33:59.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fda7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:59.216 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:59.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03b9c0 msgr2=0x7f1fcc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03b9c0 0x7f1fcc03de80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1fe813d6b0 tx=0x7f1fdc009040 comp rx=0 tx=0).stop 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 10 monclient: shutdown 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 msgr2=0x7f1fe809d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.213+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 0x7f1fe809d400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4001230 tx=0x7f1fe400c8b0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 shutdown_connections 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1fcc03b9c0 0x7f1fcc03de80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1fe80a1380 0x7f1fe809de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fe80a0890 0x7f1fe809d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 --2- 172.21.15.17:0/446053994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fe809ffc0 0x7f1fe809d400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 >> 172.21.15.17:0/446053994 conn(0x7f1fe80a66a0 msgr2=0x7f1fe80a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 shutdown_connections 2024-09-15T21:33:59.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.217+0000 7f1fefa6d640 1 -- 172.21.15.17:0/446053994 wait complete. 2024-09-15T21:33:59.235 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:33:59.235 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:33:59.235 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:33:59.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient: get_monmap_and_config 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient: monmap: 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:59.354301+0000 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:59.354301+0000 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:59.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 Processor -- start 2024-09-15T21:33:59.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 -- start start 2024-09-15T21:33:59.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient: init 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 0x7f546c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): picked mon.noname-a con 0x7f546c1378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): picked mon.noname-b con 0x7f546c1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f546c0a5040 0x7f546c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): picked mon.noname-c con 0x7f546c0a5040 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f546c0a1d00 con 0x7f546c1378c0 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f546c0a1fe0 con 0x7f546c0a5040 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f546c0a22c0 con 0x7f546c1381e0 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f5472318640 10 monclient(hunting): authenticate will time out at 1962.749678s 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 0x7f546c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 0x7f546c137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58578/0 (socket says 172.21.15.17:58578) 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 1 -- 172.21.15.17:0/904413515 learned_addr learned my addr 172.21.15.17:0/904413515 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): get_auth_request con 0x7f546c1378c0 auth_method 0 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): get_auth_request con 0x7f546c1381e0 auth_method 0 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f54708b5640 1 --2- 172.21.15.17:0/904413515 >> v2:172.21.15.119:3300/0 conn(0x7f546c0a5040 0x7f546c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f54708b5640 10 monclient(hunting): get_auth_request con 0x7f546c0a5040 auth_method 0 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f54708b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f54708b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f54708b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient(hunting): handle_auth_done global_id 164438 payload 274 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 -- 172.21.15.17:0/904413515 >> v2:172.21.15.119:3300/0 conn(0x7f546c0a5040 msgr2=0x7f546c0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 --2- 172.21.15.17:0/904413515 >> v2:172.21.15.119:3300/0 conn(0x7f546c0a5040 0x7f546c0a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 -- 172.21.15.17:0/904413515 >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 msgr2=0x7f546c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 --2- 172.21.15.17:0/904413515 >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 0x7f546c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546b7fe640 1 -- 172.21.15.17:0/904413515 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c0a4490 con 0x7f546c1381e0 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.353+0000 7f546bfff640 1 --2- 172.21.15.17:0/904413515 >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 0x7f546c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:59.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 1 --2- 172.21.15.17:0/904413515 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c0a4b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f546c0a6150 tx=0x7f545c01c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa05687aea09e025 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 1 -- 172.21.15.17:0/904413515 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0b0070 con 0x7f546c1381e0 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: dump: 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.364 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 1 -- 172.21.15.17:0/904413515 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545c0ab600 con 0x7f546c1381e0 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 1 -- 172.21.15.17:0/904413515 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0ab9d0 con 0x7f546c1381e0 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 5 monclient: authenticate success, global_id 164438 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 4 set_mon_vals no callback set 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546affd640 10 monclient: dump: 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.366 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: shutdown 2024-09-15T21:33:59.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- 172.21.15.17:0/904413515 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 msgr2=0x7f546c0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- 172.21.15.17:0/904413515 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c0a4b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f546c0a6150 tx=0x7f545c01c340 comp rx=0 tx=0).stop 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- 172.21.15.17:0/904413515 shutdown_connections 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- 172.21.15.17:0/904413515 >> v2:172.21.15.119:3300/0 conn(0x7f546c0a5040 0x7f546c0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- 172.21.15.17:0/904413515 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c0a4b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- 172.21.15.17:0/904413515 >> v2:172.21.15.17:3300/0 conn(0x7f546c1378c0 0x7f546c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- 172.21.15.17:0/904413515 >> 172.21.15.17:0/904413515 conn(0x7f546c0aac40 msgr2=0x7f546c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- 172.21.15.17:0/904413515 shutdown_connections 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- 172.21.15.17:0/904413515 wait complete. 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: monmap: 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 Processor -- start 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- start start 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: init 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f546c0a5040 0x7f546c1cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): picked mon.noname-b con 0x7f546c0a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c1378c0 0x7f546c136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): picked mon.noname-a con 0x7f546c1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c13c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): picked mon.noname-c con 0x7f546c1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f546c141730 con 0x7f546c1378c0 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1415b0 con 0x7f546c0a5040 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f546c1418b0 con 0x7f546c1381e0 2024-09-15T21:33:59.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient(hunting): authenticate will time out at 1962.754974s 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c13c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c1378c0 0x7f546c136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c13c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35640/0 (socket says 172.21.15.17:35640) 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 -- 172.21.15.17:0/3481266224 learned_addr learned my addr 172.21.15.17:0/3481266224 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): get_auth_request con 0x7f546c1381e0 auth_method 0 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): get_auth_request con 0x7f546c1378c0 auth_method 0 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546bfff640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f546c0a5040 0x7f546c1cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546bfff640 10 monclient(hunting): get_auth_request con 0x7f546c0a5040 auth_method 0 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient(hunting): handle_auth_done global_id 164441 payload 824 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f546c0a5040 msgr2=0x7f546c1cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f546c0a5040 0x7f546c1cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c1378c0 msgr2=0x7f546c136de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c1378c0 0x7f546c136de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 monclient: found mon.noname-c 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c0a8140 con 0x7f546c1381e0 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f546b7fe640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c1378c0 0x7f546c136de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:59.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f54708b5640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c13c070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f546400d800 tx=0x7f546400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546400b6c0 con 0x7f546c1381e0 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: dump: 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.380 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 5 monclient: authenticate success, global_id 164441 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5464019070 con 0x7f546c1381e0 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546400baa0 con 0x7f546c1381e0 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: _renew_subs 2024-09-15T21:33:59.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c1227a0 con 0x7f546c1381e0 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5469ffb640 4 set_mon_vals no callback set 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.357+0000 7f5468ff9640 10 monclient: dump: 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.382 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 10 monclient: _renew_subs 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c0a12f0 con 0x7f546c1381e0 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 0x7f546c13c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f546c137ca0 con 0x7f546c137320 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 0x7f546c13c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient: get_auth_request con 0x7f546c137320 auth_method 0 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f546400bc40 con 0x7f546c1381e0 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 10 monclient(hunting): handle_auth_done global_id 174375 payload 824 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f546c120cf0 con 0x7f546c137320 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545003b9c0 0x7f545003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5464050a80 con 0x7f546c1381e0 2024-09-15T21:33:59.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546bfff640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 0x7f546c13c5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5454007d30 tx=0x7f5454010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545401a020 con 0x7f546c137320 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 10 monclient: dump: 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.385 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546b7fe640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545003b9c0 0x7f545003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546b7fe640 10 monclient: get_auth_request con 0x7f545003b9c0 auth_method 0 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5454015640 con 0x7f546c137320 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 msgr2=0x7f546c13c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 0x7f546c13c5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5454007d30 tx=0x7f5454010590 comp rx=0 tx=0).stop 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 msgr2=0x7f546c13c5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:59.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f5468ff9640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c137320 0x7f546c13c5b0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5454007d30 tx=0x7f5454010590 comp rx=0 tx=0).stop 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.361+0000 7f546b7fe640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545003b9c0 0x7f545003de80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f545c0ad000 tx=0x7f545c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.365+0000 7f5472318640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.365+0000 7f5472318640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 0x7f546c138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.365+0000 7f5472318640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.365+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f546c13cf00 con 0x7f546c13caf0 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 0x7f546c138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient: get_auth_request con 0x7f546c13caf0 auth_method 0 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 10 monclient(hunting): handle_auth_done global_id 174378 payload 824 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 1 -- 172.21.15.17:0/3481266224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f546c1d0b80 con 0x7f546c13caf0 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f546bfff640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 0x7f546c138d60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f545400b010 tx=0x7f5454004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545401a030 con 0x7f546c13caf0 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 10 monclient: dump: 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.388 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4715 (secure 0 0 0) 0x7f54540039a0 con 0x7f546c13caf0 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 msgr2=0x7f546c138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 0x7f546c138d60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f545400b010 tx=0x7f5454004a50 comp rx=0 tx=0).stop 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 msgr2=0x7f546c138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:59.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5468ff9640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f546c13caf0 0x7f546c138d60 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f545400b010 tx=0x7f5454004a50 comp rx=0 tx=0).stop 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 152, 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 129106, 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:59.390 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.391 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.392 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:59.393 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:59.394 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.395 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:59.398 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:59.399 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.401 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:59.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.402 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.403 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:59.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.403 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.403 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.403 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:59.404 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545003b9c0 msgr2=0x7f545003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f545003b9c0 0x7f545003de80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f545c0ad000 tx=0x7f545c0a6000 comp rx=0 tx=0).stop 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 10 monclient: shutdown 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 msgr2=0x7f546c13c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 --2- 172.21.15.17:0/3481266224 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f546c1381e0 0x7f546c13c070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f546400d800 tx=0x7f546400dcc0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f54708b5640 1 -- 172.21.15.17:0/3481266224 reap_dead start 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f54708b5640 1 -- 172.21.15.17:0/3481266224 reap_dead start 2024-09-15T21:33:59.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 shutdown_connections 2024-09-15T21:33:59.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 >> 172.21.15.17:0/3481266224 conn(0x7f546c0aac40 msgr2=0x7f546c13e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 shutdown_connections 2024-09-15T21:33:59.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.369+0000 7f5472318640 1 -- 172.21.15.17:0/3481266224 wait complete. 2024-09-15T21:33:59.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:59.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:33:59.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:33:59.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient: get_monmap_and_config 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient: monmap: 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:59.527057+0000 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:59.527057+0000 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:59.527 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:59.528 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:59.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 Processor -- start 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 -- start start 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient: init 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b4c09c6a0 0x7f7b4c09ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): picked mon.noname-c con 0x7f7b4c09c6a0 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): picked mon.noname-b con 0x7f7b4c137c60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7b4c0a1b60 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c074840 con 0x7f7b4c0a1b60 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4c0742d0 con 0x7f7b4c09c6a0 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c09a580 con 0x7f7b4c137c60 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b51b2b640 10 monclient(hunting): authenticate will time out at 1962.922768s 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c138040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35652/0 (socket says 172.21.15.17:35652) 2024-09-15T21:33:59.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 -- 172.21.15.17:0/3733440456 learned_addr learned my addr 172.21.15.17:0/3733440456 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): get_auth_request con 0x7f7b4c137c60 auth_method 0 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 1 --2- 172.21.15.17:0/3733440456 >> v2:172.21.15.119:3300/0 conn(0x7f7b4c09c6a0 0x7f7b4c09ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): get_auth_request con 0x7f7b4c09c6a0 auth_method 0 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient(hunting): handle_auth_done global_id 164444 payload 274 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 -- 172.21.15.17:0/3733440456 >> v2:172.21.15.119:3300/0 conn(0x7f7b4c09c6a0 msgr2=0x7f7b4c09ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 --2- 172.21.15.17:0/3733440456 >> v2:172.21.15.119:3300/0 conn(0x7f7b4c09c6a0 0x7f7b4c09ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 -- 172.21.15.17:0/3733440456 >> v1:172.21.15.119:6789/0 conn(0x7f7b4c0a1b60 legacy=0x7f7b4c0a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 monclient: found mon.noname-b 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.525+0000 7f7b4affd640 1 -- 172.21.15.17:0/3733440456 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c0a39a0 con 0x7f7b4c137c60 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4affd640 1 --2- 172.21.15.17:0/3733440456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c138040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009e50 tx=0x7f7b3401c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715696a0121d26c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 1 -- 172.21.15.17:0/3733440456 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340a5640 con 0x7f7b4c137c60 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: dump: 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.539 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 1 -- 172.21.15.17:0/3733440456 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b340a57e0 con 0x7f7b4c137c60 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 1 -- 172.21.15.17:0/3733440456 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340a5bb0 con 0x7f7b4c137c60 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 5 monclient: authenticate success, global_id 164444 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:59.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 4 set_mon_vals no callback set 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b4a7fc640 10 monclient: dump: 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 10 monclient: shutdown 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 -- 172.21.15.17:0/3733440456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 msgr2=0x7f7b4c138040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/3733440456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c138040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009e50 tx=0x7f7b3401c980 comp rx=0 tx=0).stop 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 -- 172.21.15.17:0/3733440456 shutdown_connections 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/3733440456 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c138040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/3733440456 >> v2:172.21.15.119:3300/0 conn(0x7f7b4c09c6a0 0x7f7b4c09ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 -- 172.21.15.17:0/3733440456 >> 172.21.15.17:0/3733440456 conn(0x7f7b4c0a2c50 msgr2=0x7f7b4c0a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 -- 172.21.15.17:0/3733440456 shutdown_connections 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 -- 172.21.15.17:0/3733440456 wait complete. 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 10 monclient: monmap: 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 Processor -- start 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 1 -- start start 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.529+0000 7f7b51b2b640 10 monclient: init 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:33:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4c09c6a0 0x7f7b4c1cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): picked mon.noname-b con 0x7f7b4c09c6a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b4c0a1b60 0x7f7b4c13ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): picked mon.noname-a con 0x7f7b4c0a1b60 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c13f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): picked mon.noname-c con 0x7f7b4c137c60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c142690 con 0x7f7b4c0a1b60 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c142510 con 0x7f7b4c09c6a0 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c142810 con 0x7f7b4c137c60 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): authenticate will time out at 1962.928193s 2024-09-15T21:33:59.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c13f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c13f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35654/0 (socket says 172.21.15.17:35654) 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 -- 172.21.15.17:0/843408032 learned_addr learned my addr 172.21.15.17:0/843408032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): get_auth_request con 0x7f7b4c137c60 auth_method 0 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b4c0a1b60 0x7f7b4c13ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): get_auth_request con 0x7f7b4c0a1b60 auth_method 0 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4c09c6a0 0x7f7b4c1cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): get_auth_request con 0x7f7b4c09c6a0 auth_method 0 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient(hunting): handle_auth_done global_id 164447 payload 824 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4c09c6a0 msgr2=0x7f7b4c1cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4c09c6a0 0x7f7b4c1cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b4c0a1b60 msgr2=0x7f7b4c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b4c0a1b60 0x7f7b4c13ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 monclient: found mon.noname-c 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c13b380 con 0x7f7b4c137c60 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b4c0a1b60 0x7f7b4c13ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4bfff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c13f150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c500 tx=0x7f7b4400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44017070 con 0x7f7b4c137c60 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: dump: 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 5 monclient: authenticate success, global_id 164447 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient: _renew_subs 2024-09-15T21:33:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b440052d0 con 0x7f7b4c137c60 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c13d6e0 con 0x7f7b4c137c60 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44003820 con 0x7f7b4c137c60 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: dump: 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b49ffb640 4 set_mon_vals no callback set 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient: _renew_subs 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c13bee0 con 0x7f7b4c137c60 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 0x7f7b4c139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c13f690 con 0x7f7b4c139940 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 0x7f7b4c139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient: get_auth_request con 0x7f7b4c139940 auth_method 0 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7b4401a070 con 0x7f7b4c137c60 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_done global_id 164450 payload 824 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b4c09ca80 con 0x7f7b4c139940 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b2c03b9c0 0x7f7b2c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7b4404f980 con 0x7f7b4c137c60 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4b7fe640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 0x7f7b4c139d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c011600 tx=0x7f7b3c011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c004230 con 0x7f7b4c139940 2024-09-15T21:33:59.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: dump: 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b2c03b9c0 0x7f7b2c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 10 monclient: get_auth_request con 0x7f7b2c03b9c0 auth_method 0 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7b3c004230 con 0x7f7b4c139940 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 msgr2=0x7f7b4c139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 0x7f7b4c139d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c011600 tx=0x7f7b3c011ac0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 msgr2=0x7f7b4c139d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b43fff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c139940 0x7f7b4c139d20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.533+0000 7f7b4affd640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b2c03b9c0 0x7f7b2c03de80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7b340ae000 tx=0x7f7b340a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 0x7f7b4c0972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c13cbc0 con 0x7f7b4c13a260 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 0x7f7b4c0972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient: get_auth_request con 0x7f7b4c13a260 auth_method 0 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 10 monclient(hunting): handle_auth_done global_id 164453 payload 824 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 1 -- 172.21.15.17:0/843408032 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7b4c09d340 con 0x7f7b4c13a260 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4b7fe640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 0x7f7b4c0972a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c011600 tx=0x7f7b3c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c003830 con 0x7f7b4c13a260 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 10 monclient: dump: 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.556 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4880 (secure 0 0 0) 0x7f7b3c003830 con 0x7f7b4c13a260 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 msgr2=0x7f7b4c0972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 0x7f7b4c0972a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c011600 tx=0x7f7b3c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 msgr2=0x7f7b4c0972a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b43fff640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c13a260 0x7f7b4c0972a0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c011600 tx=0x7f7b3c0046d0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 152, 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 188125, 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:59.558 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.559 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:59.571 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b2c03b9c0 msgr2=0x7f7b2c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7b2c03b9c0 0x7f7b2c03de80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7b340ae000 tx=0x7f7b340a7070 comp rx=0 tx=0).stop 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 10 monclient: shutdown 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 msgr2=0x7f7b4c13f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 --2- 172.21.15.17:0/843408032 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7b4c137c60 0x7f7b4c13f150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c500 tx=0x7f7b4400c9c0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4bfff640 1 -- 172.21.15.17:0/843408032 reap_dead start 2024-09-15T21:33:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b4bfff640 1 -- 172.21.15.17:0/843408032 reap_dead start 2024-09-15T21:33:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 shutdown_connections 2024-09-15T21:33:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 >> 172.21.15.17:0/843408032 conn(0x7f7b4c0a2c50 msgr2=0x7f7b4c09b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 shutdown_connections 2024-09-15T21:33:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.541+0000 7f7b51b2b640 1 -- 172.21.15.17:0/843408032 wait complete. 2024-09-15T21:33:59.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:59.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:33:59.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient: get_monmap_and_config 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient: monmap: 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:59.691099+0000 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:59.691099+0000 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:59.692 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 Processor -- start 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 -- start start 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient: init 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7aa009d400 0x7f7aa009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): picked mon.noname-a con 0x7f7aa009d400 addr v2:172.21.15.17:3300/0 2024-09-15T21:33:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7aa009dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7aa0134d70 0x7f7aa013cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): picked mon.noname-c con 0x7f7aa0134d70 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa00a3130 con 0x7f7aa009dd20 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa00a1bc0 con 0x7f7aa009d400 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa00a3490 con 0x7f7aa0134d70 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa6574640 10 monclient(hunting): authenticate will time out at 1963.087000s 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7aa009d400 0x7f7aa009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7aa009d400 0x7f7aa009d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58618/0 (socket says 172.21.15.17:58618) 2024-09-15T21:33:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 -- 172.21.15.17:0/3082748403 learned_addr learned my addr 172.21.15.17:0/3082748403 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): get_auth_request con 0x7f7aa009d400 auth_method 0 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa4b11640 1 --2- 172.21.15.17:0/3082748403 >> v2:172.21.15.119:3300/0 conn(0x7f7aa0134d70 0x7f7aa013cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa4b11640 10 monclient(hunting): get_auth_request con 0x7f7aa0134d70 auth_method 0 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa4b11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa4b11640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7aa4b11640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_done global_id 174384 payload 274 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 -- 172.21.15.17:0/3082748403 >> v2:172.21.15.119:3300/0 conn(0x7f7aa0134d70 msgr2=0x7f7aa013cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 --2- 172.21.15.17:0/3082748403 >> v2:172.21.15.119:3300/0 conn(0x7f7aa0134d70 0x7f7aa013cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 -- 172.21.15.17:0/3082748403 >> v1:172.21.15.17:6789/0 conn(0x7f7aa009dd20 legacy=0x7f7aa0134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 monclient: found mon.noname-a 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.689+0000 7f7a9ffff640 1 -- 172.21.15.17:0/3082748403 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa009e7a0 con 0x7f7aa009d400 2024-09-15T21:33:59.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9ffff640 1 --2- 172.21.15.17:0/3082748403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa009d400 0x7f7aa009d7e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7aa009f4f0 tx=0x7f7a9001eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a97d6b0a2c26829 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 1 -- 172.21.15.17:0/3082748403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900b4070 con 0x7f7aa009d400 2024-09-15T21:33:59.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:59.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: dump: 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 1 -- 172.21.15.17:0/3082748403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a900a7880 con 0x7f7aa009d400 2024-09-15T21:33:59.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 1 -- 172.21.15.17:0/3082748403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900a7c50 con 0x7f7aa009d400 2024-09-15T21:33:59.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:59.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7a9f7fe640 10 monclient: dump: 2024-09-15T21:33:59.709 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.709 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.709 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.710 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.710 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 5 monclient: authenticate success, global_id 174384 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 4 set_mon_vals no callback set 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 10 monclient: shutdown 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 -- 172.21.15.17:0/3082748403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa009d400 msgr2=0x7f7aa009d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 --2- 172.21.15.17:0/3082748403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa009d400 0x7f7aa009d7e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7aa009f4f0 tx=0x7f7a9001eab0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 -- 172.21.15.17:0/3082748403 shutdown_connections 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 --2- 172.21.15.17:0/3082748403 >> v2:172.21.15.119:3300/0 conn(0x7f7aa0134d70 0x7f7aa013cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 --2- 172.21.15.17:0/3082748403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa009d400 0x7f7aa009d7e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 -- 172.21.15.17:0/3082748403 >> 172.21.15.17:0/3082748403 conn(0x7f7aa00a66a0 msgr2=0x7f7aa00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 -- 172.21.15.17:0/3082748403 shutdown_connections 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 1 -- 172.21.15.17:0/3082748403 wait complete. 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.693+0000 7f7aa6574640 10 monclient: monmap: 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.712 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 Processor -- start 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- start start 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: init 2024-09-15T21:33:59.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7aa009d400 0x7f7aa013a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): picked mon.noname-c con 0x7f7aa009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa009dd20 0x7f7aa013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): picked mon.noname-b con 0x7f7aa009dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa0134d70 0x7f7aa013b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): picked mon.noname-a con 0x7f7aa0134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa009fba0 con 0x7f7aa0134d70 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa009fa20 con 0x7f7aa009dd20 2024-09-15T21:33:59.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa00a10a0 con 0x7f7aa009d400 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): authenticate will time out at 1963.093047s 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa0134d70 0x7f7aa013b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7aa009d400 0x7f7aa013a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa0134d70 0x7f7aa013b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58632/0 (socket says 172.21.15.17:58632) 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 -- 172.21.15.17:0/1635109518 learned_addr learned my addr 172.21.15.17:0/1635109518 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): get_auth_request con 0x7f7aa0134d70 auth_method 0 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): get_auth_request con 0x7f7aa009d400 auth_method 0 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa009dd20 0x7f7aa013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 10 monclient(hunting): get_auth_request con 0x7f7aa009dd20 auth_method 0 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient(hunting): handle_auth_done global_id 174387 payload 824 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7aa009d400 msgr2=0x7f7aa013a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f7aa009d400 0x7f7aa013a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa009dd20 msgr2=0x7f7aa013ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa009dd20 0x7f7aa013ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 monclient: found mon.noname-a 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa011e4d0 con 0x7f7aa0134d70 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa4b11640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa0134d70 0x7f7aa013b360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a98007e40 tx=0x7f7a9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98014620 con 0x7f7aa0134d70 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: dump: 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.718 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 5 monclient: authenticate success, global_id 174387 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a98019070 con 0x7f7aa0134d70 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: _renew_subs 2024-09-15T21:33:59.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98014a20 con 0x7f7aa0134d70 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa011e0b0 con 0x7f7aa0134d70 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9e7fc640 4 set_mon_vals no callback set 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 10 monclient: dump: 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: _renew_subs 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa013c3a0 con 0x7f7aa0134d70 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 0x7f7aa0063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa002e9b0 con 0x7f7aa002e5d0 2024-09-15T21:33:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f7a98014bc0 con 0x7f7aa0134d70 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7a8003b9c0 0x7f7a8003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 0x7f7aa0063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f7a98050e50 con 0x7f7aa0134d70 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient: get_auth_request con 0x7f7aa002e5d0 auth_method 0 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7a8003b9c0 0x7f7a8003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a97fff640 10 monclient: get_auth_request con 0x7f7a8003b9c0 auth_method 0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_done global_id 194146 payload 824 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.697+0000 7f7a9ffff640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7aa00a14b0 con 0x7f7aa002e5d0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9ffff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 0x7f7aa0063e00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0074430 tx=0x7f7a9001b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a97fff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7a8003b9c0 0x7f7a8003de80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7aa01342d0 tx=0x7f7a88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900c0070 con 0x7f7aa002e5d0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 10 monclient: dump: 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7a9000b730 con 0x7f7aa002e5d0 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 msgr2=0x7f7aa0063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 0x7f7aa0063e00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0074430 tx=0x7f7a9001b850 comp rx=0 tx=0).stop 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 msgr2=0x7f7aa0063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.701+0000 7f7a9d7fa640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa002e5d0 0x7f7aa0063e00 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0074430 tx=0x7f7a9001b850 comp rx=0 tx=0).stop 2024-09-15T21:33:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7aa6574640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:33:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7aa6574640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 0x7f7aa01d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7aa6574640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01371a0 con 0x7f7aa0064340 2024-09-15T21:33:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 0x7f7aa01d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient: get_auth_request con 0x7f7aa0064340 auth_method 0 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.705+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9ffff640 10 monclient(hunting): handle_auth_done global_id 194149 payload 824 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9ffff640 1 -- 172.21.15.17:0/1635109518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7aa006d020 con 0x7f7aa0064340 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9ffff640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 0x7f7aa01d5e00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7a9001b6c0 tx=0x7f7a9000b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900cc070 con 0x7f7aa0064340 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 10 monclient: dump: 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f7a900d0450 con 0x7f7aa0064340 2024-09-15T21:33:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 msgr2=0x7f7aa01d5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 0x7f7aa01d5e00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7a9001b6c0 tx=0x7f7a9000b450 comp rx=0 tx=0).stop 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 msgr2=0x7f7aa01d5e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7a9d7fa640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aa0064340 0x7f7aa01d5e00 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7a9001b6c0 tx=0x7f7a9000b450 comp rx=0 tx=0).stop 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 152, 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 6499, 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:33:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:33:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:33:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:33:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:33:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:33:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:33:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:33:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:33:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:33:59.742 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7a8003b9c0 msgr2=0x7f7a8003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f7a8003b9c0 0x7f7a8003de80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7aa01342d0 tx=0x7f7a88009040 comp rx=0 tx=0).stop 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 10 monclient: shutdown 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa0134d70 msgr2=0x7f7aa013b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 --2- 172.21.15.17:0/1635109518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7aa0134d70 0x7f7aa013b360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a98007e40 tx=0x7f7a9800e4b0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa4b11640 1 -- 172.21.15.17:0/1635109518 reap_dead start 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa4b11640 1 -- 172.21.15.17:0/1635109518 reap_dead start 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 shutdown_connections 2024-09-15T21:33:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 >> 172.21.15.17:0/1635109518 conn(0x7f7aa00a66a0 msgr2=0x7f7aa0072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 shutdown_connections 2024-09-15T21:33:59.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.709+0000 7f7aa6574640 1 -- 172.21.15.17:0/1635109518 wait complete. 2024-09-15T21:33:59.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:33:59.745 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-15T21:33:59.745 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:33:59.745 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T21:33:59.745 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T21:33:59.746 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:33:59.746 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:33:59.761 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:33:59.761 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:33:59.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:59.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient: get_monmap_and_config 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient: monmap: 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:33:59.915973+0000 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:33:59.915973+0000 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:33:59.916 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:33:59.917 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:33:59.917 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 1 Processor -- start 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 1 -- start start 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient: init 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f74f80a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74f80a5580 0x7f74f813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient(hunting): picked mon.noname-c con 0x7f74f80a5580 addr v2:172.21.15.119:3300/0 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f74f80a59d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f80a2c30 con 0x7f74f80a4a90 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f80a19b0 con 0x7f74f80a59d0 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f74f80a0f40 con 0x7f74f80a5580 2024-09-15T21:33:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fea21640 10 monclient(hunting): authenticate will time out at 1963.311386s 2024-09-15T21:33:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f7fff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f74f80a4a90 legacy=0x7f74f80a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:33:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74f80a5580 0x7f74f813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74f80a5580 0x7f74f813cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:48234/0 (socket says 172.21.15.17:48234) 2024-09-15T21:33:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 1 -- 172.21.15.17:0/56711972 learned_addr learned my addr 172.21.15.17:0/56711972 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 10 monclient(hunting): get_auth_request con 0x7f74f80a5580 auth_method 0 2024-09-15T21:33:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74fcfbe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f74f80a59d0 0x7f74f813f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:33980/0 (socket says 172.21.15.17:33980) 2024-09-15T21:33:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.913+0000 7f74f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 1 -- 172.21.15.17:0/56711972 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275735737 0 0) 0x7f74f80a19b0 con 0x7f74f80a59d0 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient(hunting): my global_id is 194155 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 1 -- 172.21.15.17:0/56711972 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d8003530 con 0x7f74f80a59d0 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 10 monclient(hunting): handle_auth_done global_id 194152 payload 274 2024-09-15T21:33:59.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 1 -- 172.21.15.17:0/56711972 >> v1:172.21.15.119:6789/0 conn(0x7f74f80a59d0 legacy=0x7f74f813f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 1 -- 172.21.15.17:0/56711972 >> v1:172.21.15.17:6789/0 conn(0x7f74f80a4a90 legacy=0x7f74f80a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:33:59.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 1 monclient: found mon.noname-c 2024-09-15T21:33:59.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:33:59.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 1 -- 172.21.15.17:0/56711972 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f80a1ce0 con 0x7f74f80a5580 2024-09-15T21:33:59.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f77fe640 1 --2- 172.21.15.17:0/56711972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a5580 0x7f74f813cd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74e00098a0 tx=0x7f74e001c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e32a0e4830f15d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 1 -- 172.21.15.17:0/56711972 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00b0070 con 0x7f74f80a5580 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: dump: 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 1 -- 172.21.15.17:0/56711972 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e00ab600 con 0x7f74f80a5580 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 1 -- 172.21.15.17:0/56711972 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00ab9b0 con 0x7f74f80a5580 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74f6ffd640 10 monclient: dump: 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.930 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 5 monclient: authenticate success, global_id 194152 2024-09-15T21:33:59.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 10 monclient: get_monmap_and_config success 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 4 set_mon_vals no callback set 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 10 monclient: shutdown 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 -- 172.21.15.17:0/56711972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a5580 msgr2=0x7f74f813cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 --2- 172.21.15.17:0/56711972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a5580 0x7f74f813cd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74e00098a0 tx=0x7f74e001c280 comp rx=0 tx=0).stop 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 -- 172.21.15.17:0/56711972 shutdown_connections 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 --2- 172.21.15.17:0/56711972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a5580 0x7f74f813cd10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 -- 172.21.15.17:0/56711972 >> 172.21.15.17:0/56711972 conn(0x7f74f80aabb0 msgr2=0x7f74f80a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 -- 172.21.15.17:0/56711972 shutdown_connections 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.917+0000 7f74fea21640 1 -- 172.21.15.17:0/56711972 wait complete. 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: build_initial_monmap 2024-09-15T21:33:59.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: monmap: 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 Processor -- start 2024-09-15T21:33:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- start start 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: init 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _reopen_session rank -1 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 0x7f74f81cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): picked mon.noname-c con 0x7f74f80a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f80a5580 0x7f74f81d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): picked mon.noname-a con 0x7f74f80a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a59d0 0x7f74f813ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): picked mon.noname-b con 0x7f74f80a59d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8143730 con 0x7f74f80a5580 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81435b0 con 0x7f74f80a59d0 2024-09-15T21:33:59.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): start opening mon connection 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f74f81438b0 con 0x7f74f80a4a90 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): _renew_subs 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient(hunting): authenticate will time out at 1963.317413s 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f77fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f80a5580 msgr2=0x7f74f81d0200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f80a5580 0x7f74f81d0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 0x7f74f81cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 0x7f74f81cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35700/0 (socket says 172.21.15.17:35700) 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 -- 172.21.15.17:0/112756883 learned_addr learned my addr 172.21.15.17:0/112756883 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): get_auth_request con 0x7f74f80a4a90 auth_method 0 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fcfbe640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a59d0 0x7f74f813ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient(hunting): handle_auth_done global_id 164459 payload 824 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 -- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a59d0 msgr2=0x7f74f813ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:33:59.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a59d0 0x7f74f813ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 -- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f80a5580 msgr2=0x7f74f81d0200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f80a5580 0x7f74f81d0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 monclient: found mon.noname-c 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 -- 172.21.15.17:0/112756883 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8120b60 con 0x7f74f80a4a90 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fcfbe640 10 monclient: get_auth_request con 0x7f74f80a59d0 auth_method 0 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fcfbe640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a59d0 0x7f74f813ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f7fff640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 0x7f74f81cfcc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b9e0 tx=0x7f74e800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e800cba0 con 0x7f74f80a4a90 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: dump: 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:33:59.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e8010070 con 0x7f74f80a4a90 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 5 monclient: authenticate success, global_id 164459 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _renew_subs 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f813fed0 con 0x7f74f80a4a90 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8015470 con 0x7f74f80a4a90 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 10 monclient: dump: 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.939 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f67fc640 4 set_mon_vals no callback set 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _renew_subs 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f813ed00 con 0x7f74f80a4a90 2024-09-15T21:33:59.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:33:59.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f80a4e70 con 0x7f74f80a4a90 2024-09-15T21:33:59.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f74e801e3f0 con 0x7f74f80a4a90 2024-09-15T21:33:59.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b970 0x7f74dc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:33:59.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f74e8050980 con 0x7f74f80a4a90 2024-09-15T21:33:59.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f77fe640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b970 0x7f74dc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:33:59.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.921+0000 7f74f77fe640 10 monclient: get_auth_request con 0x7f74dc03b970 auth_method 0 2024-09-15T21:33:59.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.925+0000 7f74f77fe640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b970 0x7f74dc03de30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f74e00ad000 tx=0x7f74e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:33:59.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.925+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f74e801d3a0 con 0x7f74f80a4a90 2024-09-15T21:33:59.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.925+0000 7f74f4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:33:59.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:33:59.925+0000 7f74f4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:00.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:00.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:00.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:00.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f74f8069200 con 0x7f74f80a4a90 2024-09-15T21:34:00.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74f4ff9640 1 -- 172.21.15.17:0/112756883 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f74e8014cf0 con 0x7f74f80a4a90 2024-09-15T21:34:00.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74f4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:00.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74f4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:00.163 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:00.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b970 msgr2=0x7f74dc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:00.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b970 0x7f74dc03de30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f74e00ad000 tx=0x7f74e0008040 comp rx=0 tx=0).stop 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 10 monclient: shutdown 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 msgr2=0x7f74f81cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 0x7f74f81cfcc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b9e0 tx=0x7f74e800bea0 comp rx=0 tx=0).stop 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 shutdown_connections 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f74dc03b970 0x7f74dc03de30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f80a59d0 0x7f74f813ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74f80a5580 0x7f74f81d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 --2- 172.21.15.17:0/112756883 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f74f80a4a90 0x7f74f81cfcc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 >> 172.21.15.17:0/112756883 conn(0x7f74f80aabb0 msgr2=0x7f74f80a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 shutdown_connections 2024-09-15T21:34:00.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:00.161+0000 7f74fea21640 1 -- 172.21.15.17:0/112756883 wait complete. 2024-09-15T21:34:03.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:03.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.313+0000 7f388b023640 10 monclient: get_monmap_and_config 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.313+0000 7f388b023640 10 monclient: build_initial_monmap 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.313+0000 7f388b023640 10 monclient: monmap: 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:03.316881+0000 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:03.316881+0000 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:03.317 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.313+0000 7f388b023640 1 Processor -- start 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.313+0000 7f388b023640 1 -- start start 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.313+0000 7f388b023640 10 monclient: init 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f388409ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): picked mon.noname-b con 0x7f38840a1250 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38840a1630 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f388413ccd0 0x7f388413f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): picked mon.noname-c con 0x7f388413ccd0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3884075220 con 0x7f38840a1630 2024-09-15T21:34:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3884074da0 con 0x7f388413ccd0 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f388409dbc0 con 0x7f38840a1250 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient(hunting): authenticate will time out at 1966.712230s 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3883fff640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f38840a1630 legacy=0x7f38840a1a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f388409ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f388409ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35710/0 (socket says 172.21.15.17:35710) 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 -- 172.21.15.17:0/1675704616 learned_addr learned my addr 172.21.15.17:0/1675704616 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): get_auth_request con 0x7f38840a1250 auth_method 0 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38895c0640 1 --2- 172.21.15.17:0/1675704616 >> v2:172.21.15.119:3300/0 conn(0x7f388413ccd0 0x7f388413f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38895c0640 10 monclient(hunting): get_auth_request con 0x7f388413ccd0 auth_method 0 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38895c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38895c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38895c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient(hunting): handle_auth_done global_id 164462 payload 274 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 -- 172.21.15.17:0/1675704616 >> v2:172.21.15.119:3300/0 conn(0x7f388413ccd0 msgr2=0x7f388413f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 --2- 172.21.15.17:0/1675704616 >> v2:172.21.15.119:3300/0 conn(0x7f388413ccd0 0x7f388413f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 -- 172.21.15.17:0/1675704616 >> v1:172.21.15.17:6789/0 conn(0x7f38840a1630 legacy=0x7f38840a1a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient: _finish_hunting 0 2024-09-15T21:34:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 monclient: found mon.noname-b 2024-09-15T21:34:03.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:03.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:03.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 -- 172.21.15.17:0/1675704616 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388409f380 con 0x7f38840a1250 2024-09-15T21:34:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f3888dbf640 1 --2- 172.21.15.17:0/1675704616 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f388409ee40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f387c009e50 tx=0x7f387c01c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2983b9606bc794 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 1 -- 172.21.15.17:0/1675704616 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0a5530 con 0x7f38840a1250 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: dump: 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 1 -- 172.21.15.17:0/1675704616 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c0a56d0 con 0x7f38840a1250 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 1 -- 172.21.15.17:0/1675704616 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0a5aa0 con 0x7f38840a1250 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 5 monclient: authenticate success, global_id 164462 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 4 set_mon_vals no callback set 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f38837fe640 10 monclient: dump: 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 10 monclient: shutdown 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- 172.21.15.17:0/1675704616 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 msgr2=0x7f388409ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 --2- 172.21.15.17:0/1675704616 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f388409ee40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f387c009e50 tx=0x7f387c01c8f0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- 172.21.15.17:0/1675704616 shutdown_connections 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 --2- 172.21.15.17:0/1675704616 >> v2:172.21.15.119:3300/0 conn(0x7f388413ccd0 0x7f388413f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 --2- 172.21.15.17:0/1675704616 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f388409ee40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- 172.21.15.17:0/1675704616 >> 172.21.15.17:0/1675704616 conn(0x7f38840a6820 msgr2=0x7f38840a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.317+0000 7f388b023640 1 -- 172.21.15.17:0/1675704616 shutdown_connections 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- 172.21.15.17:0/1675704616 wait complete. 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: build_initial_monmap 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: monmap: 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 Processor -- start 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- start start 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: init 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f38841390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): picked mon.noname-c con 0x7f38840a1250 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38840a1630 0x7f3884139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): picked mon.noname-a con 0x7f38840a1630 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f388413ccd0 0x7f3884139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): picked mon.noname-b con 0x7f388413ccd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f38841421b0 con 0x7f38840a1630 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3884142030 con 0x7f388413ccd0 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3884142330 con 0x7f38840a1250 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient(hunting): authenticate will time out at 1966.717597s 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3883fff640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38840a1630 msgr2=0x7f3884139610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3883fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38840a1630 0x7f3884139610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f38841390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f38841390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35726/0 (socket says 172.21.15.17:35726) 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 -- 172.21.15.17:0/4171067374 learned_addr learned my addr 172.21.15.17:0/4171067374 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): get_auth_request con 0x7f38840a1250 auth_method 0 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38895c0640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f388413ccd0 0x7f3884139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient(hunting): handle_auth_done global_id 164465 payload 824 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 -- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f388413ccd0 msgr2=0x7f3884139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f388413ccd0 0x7f3884139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 -- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38840a1630 msgr2=0x7f3884139610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38840a1630 0x7f3884139610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient: _finish_hunting 0 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 monclient: found mon.noname-c 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 -- 172.21.15.17:0/4171067374 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388412b6e0 con 0x7f38840a1250 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38895c0640 10 monclient: get_auth_request con 0x7f388413ccd0 auth_method 0 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38895c0640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f388413ccd0 0x7f3884139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3888dbf640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f38841390d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f387c01d190 tx=0x7f387c019690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:03.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0bc020 con 0x7f38840a1250 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: dump: 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:03.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: _finish_auth 0 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: _check_auth_tickets 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c0b8880 con 0x7f38840a1250 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0a5530 con 0x7f38840a1250 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 10 monclient: dump: 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:03.337 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 5 monclient: authenticate success, global_id 164465 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _renew_subs 2024-09-15T21:34:03.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388412bb10 con 0x7f38840a1250 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f3882ffd640 4 set_mon_vals no callback set 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _renew_subs 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f388413a900 con 0x7f38840a1250 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f387c0bf020 con 0x7f38840a1250 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388409ee40 con 0x7f38840a1250 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f387003b970 0x7f387003de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.321+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f387c0eb590 con 0x7f38840a1250 2024-09-15T21:34:03.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.325+0000 7f3883fff640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f387003b970 0x7f387003de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:03.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.325+0000 7f3883fff640 10 monclient: get_auth_request con 0x7f387003b970 auth_method 0 2024-09-15T21:34:03.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.325+0000 7f3883fff640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f387003b970 0x7f387003de30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f388409d7b0 tx=0x7f3878006d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:03.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.329+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f387c0c6070 con 0x7f38840a1250 2024-09-15T21:34:03.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.329+0000 7f38817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:03.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.329+0000 7f38817fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:03.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f388b023640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:03.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f388b023640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:03.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f388b023640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:03.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f388412c160 con 0x7f38840a1250 2024-09-15T21:34:03.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f38817fa640 1 -- 172.21.15.17:0/4171067374 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f387c016600 con 0x7f38840a1250 2024-09-15T21:34:03.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f38817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:03.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.561+0000 7f38817fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:03.565 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f387003b970 msgr2=0x7f387003de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f387003b970 0x7f387003de30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f388409d7b0 tx=0x7f3878006d10 comp rx=0 tx=0).stop 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 10 monclient: shutdown 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 msgr2=0x7f38841390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f38841390d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f387c01d190 tx=0x7f387c019690 comp rx=0 tx=0).stop 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 shutdown_connections 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f387003b970 0x7f387003de30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f388413ccd0 0x7f3884139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38840a1630 0x7f3884139610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 --2- 172.21.15.17:0/4171067374 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f38840a1250 0x7f38841390d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:03.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 >> 172.21.15.17:0/4171067374 conn(0x7f38840a6820 msgr2=0x7f38840a4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 shutdown_connections 2024-09-15T21:34:03.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:03.565+0000 7f388b023640 1 -- 172.21.15.17:0/4171067374 wait complete. 2024-09-15T21:34:06.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:06.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.713+0000 7f7463789640 10 monclient: get_monmap_and_config 2024-09-15T21:34:06.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.713+0000 7f7463789640 10 monclient: build_initial_monmap 2024-09-15T21:34:06.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.713+0000 7f7463789640 10 monclient: monmap: 2024-09-15T21:34:06.717 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:06.717 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:06.717482+0000 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:06.717482+0000 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:06.718 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:06.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 1 Processor -- start 2024-09-15T21:34:06.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 1 -- start start 2024-09-15T21:34:06.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient: init 2024-09-15T21:34:06.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f745c09d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f745c09def0 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c13d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient(hunting): picked mon.noname-b con 0x7f745c13afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c0a3490 con 0x7f745c09def0 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c0a0320 con 0x7f745c09d400 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7460d24640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f745c09def0 legacy=0x7f745c1398a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f745c0a1bc0 con 0x7f745c13afc0 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7463789640 10 monclient(hunting): authenticate will time out at 1970.113054s 2024-09-15T21:34:06.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c13d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c13d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35732/0 (socket says 172.21.15.17:35732) 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 -- 172.21.15.17:0/1786604951 learned_addr learned my addr 172.21.15.17:0/1786604951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): get_auth_request con 0x7f745c13afc0 auth_method 0 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient(hunting): handle_auth_done global_id 164468 payload 274 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 -- 172.21.15.17:0/1786604951 >> v1:172.21.15.119:6789/0 conn(0x7f745c09d400 legacy=0x7f745c09d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 -- 172.21.15.17:0/1786604951 >> v1:172.21.15.17:6789/0 conn(0x7f745c09def0 legacy=0x7f745c1398a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient: _finish_hunting 0 2024-09-15T21:34:06.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 monclient: found mon.noname-b 2024-09-15T21:34:06.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:06.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:06.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.717+0000 7f7461d26640 1 -- 172.21.15.17:0/1786604951 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c09e7a0 con 0x7f745c13afc0 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 --2- 172.21.15.17:0/1786604951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c13d3b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f745c09d0a0 tx=0x7f745401ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb261afd18d9ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 1 -- 172.21.15.17:0/1786604951 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540ab5f0 con 0x7f745c13afc0 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: dump: 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: _finish_auth 0 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 1 -- 172.21.15.17:0/1786604951 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74540b0070 con 0x7f745c13afc0 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 5 monclient: authenticate success, global_id 164468 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:06.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 4 set_mon_vals no callback set 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 1 -- 172.21.15.17:0/1786604951 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540ab9b0 con 0x7f745c13afc0 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f745bfff640 10 monclient: dump: 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:06.730 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: shutdown 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- 172.21.15.17:0/1786604951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 msgr2=0x7f745c13d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 --2- 172.21.15.17:0/1786604951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c13d3b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f745c09d0a0 tx=0x7f745401ce70 comp rx=0 tx=0).stop 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- 172.21.15.17:0/1786604951 shutdown_connections 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 --2- 172.21.15.17:0/1786604951 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c13d3b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- 172.21.15.17:0/1786604951 >> 172.21.15.17:0/1786604951 conn(0x7f745c0a66a0 msgr2=0x7f745c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- 172.21.15.17:0/1786604951 shutdown_connections 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- 172.21.15.17:0/1786604951 wait complete. 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: build_initial_monmap 2024-09-15T21:34:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: monmap: 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 Processor -- start 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- start start 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: init 2024-09-15T21:34:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f745c09d400 0x7f745c130a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): picked mon.noname-b con 0x7f745c09d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c09def0 0x7f745c1cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): picked mon.noname-a con 0x7f745c09def0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c1cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): picked mon.noname-c con 0x7f745c13afc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f745c09ce50 con 0x7f745c09def0 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f745c09ccd0 con 0x7f745c09d400 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:06.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f745c0a0890 con 0x7f745c13afc0 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7463789640 10 monclient(hunting): authenticate will time out at 1970.119259s 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7460d24640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c09def0 msgr2=0x7f745c1cba30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7460d24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c09def0 0x7f745c1cba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c1cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c1cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:35738/0 (socket says 172.21.15.17:35738) 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 -- 172.21.15.17:0/743800926 learned_addr learned my addr 172.21.15.17:0/743800926 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): get_auth_request con 0x7f745c13afc0 auth_method 0 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:06.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461525640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f745c09d400 0x7f745c130a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461525640 10 monclient(hunting): get_auth_request con 0x7f745c09d400 auth_method 0 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461525640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461525640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461525640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient(hunting): handle_auth_done global_id 164471 payload 824 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 -- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f745c09d400 msgr2=0x7f745c130a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f745c09d400 0x7f745c130a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 -- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c09def0 msgr2=0x7f745c1cba30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c09def0 0x7f745c1cba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient: _finish_hunting 0 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 monclient: found mon.noname-c 2024-09-15T21:34:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461d26640 1 -- 172.21.15.17:0/743800926 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c072110 con 0x7f745c13afc0 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.721+0000 7f7461525640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f745c09d400 0x7f745c130a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7461d26640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c1cbf70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f745401ccb0 tx=0x7f74540191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540bf610 con 0x7f745c13afc0 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: dump: 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74540bf7b0 con 0x7f745c13afc0 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 5 monclient: authenticate success, global_id 164471 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 monclient: _renew_subs 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:06.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c070bf0 con 0x7f745c13afc0 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540bfba0 con 0x7f745c13afc0 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 10 monclient: dump: 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:06.738 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f745b7fe640 4 set_mon_vals no callback set 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 monclient: _renew_subs 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c071e30 con 0x7f745c13afc0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c09d7e0 con 0x7f745c13afc0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f74540bd020 con 0x7f745c13afc0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f744803b970 0x7f744803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f74540eae10 con 0x7f745c13afc0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7461525640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f744803b970 0x7f744803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:06.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7461525640 10 monclient: get_auth_request con 0x7f744803b970 auth_method 0 2024-09-15T21:34:06.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.725+0000 7f7461525640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f744803b970 0x7f744803de30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f744c0012f0 tx=0x7f744c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:06.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.729+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f7454016150 con 0x7f745c13afc0 2024-09-15T21:34:06.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.729+0000 7f7459ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:06.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.729+0000 7f7459ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:06.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:06.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:06.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:06.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f745c070930 con 0x7f745c13afc0 2024-09-15T21:34:06.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7459ffb640 1 -- 172.21.15.17:0/743800926 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f74540bfd40 con 0x7f745c13afc0 2024-09-15T21:34:06.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7459ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:06.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7459ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:06.960 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f744803b970 msgr2=0x7f744803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f744803b970 0x7f744803de30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f744c0012f0 tx=0x7f744c009040 comp rx=0 tx=0).stop 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 10 monclient: shutdown 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 msgr2=0x7f745c1cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.957+0000 7f7463789640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c1cbf70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f745401ccb0 tx=0x7f74540191b0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 shutdown_connections 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f744803b970 0x7f744803de30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f745c13afc0 0x7f745c1cbf70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c09def0 0x7f745c1cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 --2- 172.21.15.17:0/743800926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f745c09d400 0x7f745c130a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:06.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 >> 172.21.15.17:0/743800926 conn(0x7f745c0a66a0 msgr2=0x7f745c133890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:06.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 shutdown_connections 2024-09-15T21:34:06.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:06.961+0000 7f7463789640 1 -- 172.21.15.17:0/743800926 wait complete. 2024-09-15T21:34:09.979 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient: get_monmap_and_config 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient: build_initial_monmap 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient: monmap: 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:10.111767+0000 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:10.111767+0000 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 Processor -- start 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 -- start start 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient: init 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f600c0a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): picked mon.noname-b con 0x7f600c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 0x7f600c13d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): picked mon.noname-a con 0x7f600c0a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f600c0a3130 con 0x7f600c0a0890 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f600c073df0 con 0x7f600c0a1380 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f600c0a1bf0 con 0x7f600c09ffc0 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f601214f640 10 monclient(hunting): authenticate will time out at 1973.507328s 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600bfff640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 msgr2=0x7f600c13d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 0x7f600c13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600b7fe640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f600c0a0890 legacy=0x7f600c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:43256/0 (socket says 172.21.15.17:43256) 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 learned_addr learned my addr 172.21.15.17:0/2482049599 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 10 monclient(hunting): get_auth_request con 0x7f600c09ffc0 auth_method 0 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.109+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:10.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:10.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 10 monclient(hunting): get_auth_request con 0x7f600c09ffc0 auth_method 0 2024-09-15T21:34:10.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:10.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:10.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:10.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:10.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:10.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:34:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600bfff640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 msgr2=0x7f600c13d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600bfff640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 0x7f600c13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:34:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.313+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v1:172.21.15.17:6789/0 conn(0x7f600c0a0890 legacy=0x7f600c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600bfff640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 msgr2=0x7f600c13d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600bfff640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 0x7f600c13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:34:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 10 monclient(hunting): get_auth_request con 0x7f600c09ffc0 auth_method 0 2024-09-15T21:34:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v1:172.21.15.17:6789/0 conn(0x7f600c0a0890 legacy=0x7f600c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:10.717+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:34:11.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v1:172.21.15.17:6789/0 conn(0x7f600c0a0890 legacy=0x7f600c0a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:11.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:11.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 10 monclient(hunting): get_auth_request con 0x7f600c09ffc0 auth_method 0 2024-09-15T21:34:11.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:11.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:11.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:11.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:11.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:11.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:11.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:34:11.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600bfff640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 msgr2=0x7f600c13d010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:11.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:11.521+0000 7f600bfff640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 0x7f600c13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:34:13.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): tick 2024-09-15T21:34:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:34:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 monclient(hunting): continuing hunt 2024-09-15T21:34:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:34:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 msgr2=0x7f600c13aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:13.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f600c09ffc0 0x7f600c13aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 msgr2=0x7f600c13d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.17:3300/0 conn(0x7f600c0a1380 0x7f600c13d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 -- 172.21.15.17:0/2482049599 >> v1:172.21.15.17:6789/0 conn(0x7f600c0a0890 legacy=0x7f600c0a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient: _add_conns ranks=[4,1] 2024-09-15T21:34:13.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): picked mon.noname-c con 0x7f5fec0036b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5fec006220 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 -- 172.21.15.17:0/2482049599 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f600c09ccd0 con 0x7f5fec006220 2024-09-15T21:34:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 1 -- 172.21.15.17:0/2482049599 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f600c073df0 con 0x7f5fec0036b0 2024-09-15T21:34:13.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.109+0000 7f60097fa640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 10 monclient(hunting): get_auth_request con 0x7f5fec0036b0 auth_method 0 2024-09-15T21:34:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:13.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:13.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.113+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 10 monclient(hunting): get_auth_request con 0x7f5fec0036b0 auth_method 0 2024-09-15T21:34:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.317+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 10 monclient(hunting): get_auth_request con 0x7f5fec0036b0 auth_method 0 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:13.721+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 10 monclient(hunting): get_auth_request con 0x7f5fec0036b0 auth_method 0 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:14.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.525+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:34:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.761+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121191122 0 0) 0x7f600c09ccd0 con 0x7f5fec006220 2024-09-15T21:34:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.761+0000 7f600a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:34:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.761+0000 7f600a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.761+0000 7f600a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.761+0000 7f600a7fc640 10 monclient(hunting): my global_id is 194164 2024-09-15T21:34:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.761+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ff8003530 con 0x7f5fec006220 2024-09-15T21:34:14.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3283817002 0 0) 0x7f5ff8003530 con 0x7f5fec006220 2024-09-15T21:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 msgr2=0x7f5fec005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 --2- 172.21.15.17:0/2482049599 >> v2:172.21.15.119:3300/0 conn(0x7f5fec0036b0 0x7f5fec005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c0a3130 con 0x7f5fec006220 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f5ffc0075f0 con 0x7f5fec006220 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: dump: 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 >> v1:172.21.15.119:6789/0 conn(0x7f5fec006220 legacy=0x7f5fec0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600bfff640 1 -- 172.21.15.17:0/2482049599 reap_dead start 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff80039c0 0x7f5ff8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): picked mon.c con 0x7f5ff80039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff8004810 0x7f5ff8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): picked mon.b con 0x7f5ff8004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80052a0 0x7f5ff800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): picked mon.a con 0x7f5ff80052a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f600c073df0 con 0x7f5ff80052a0 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff800c1f0 con 0x7f5ff8004810 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff800c3f0 con 0x7f5ff80039c0 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600bfff640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80052a0 msgr2=0x7f5ff800b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff80039c0 0x7f5ff8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600bfff640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80052a0 0x7f5ff800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600b7fe640 10 monclient(hunting): get_auth_request con 0x7f5ff80039c0 auth_method 0 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.769+0000 7f600affd640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff8004810 0x7f5ff8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): get_auth_request con 0x7f5ff8004810 auth_method 0 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient(hunting): handle_auth_done global_id 194164 payload 293 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff8004810 msgr2=0x7f5ff8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff8004810 0x7f5ff8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80052a0 msgr2=0x7f5ff800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80052a0 0x7f5ff800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 1 monclient: found mon.c 2024-09-15T21:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 1 -- 172.21.15.17:0/2482049599 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff8003530 con 0x7f5ff80039c0 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f600b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f601214f640 5 monclient: authenticate success, global_id 194164 2024-09-15T21:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.773+0000 7f601214f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600b7fe640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff80039c0 0x7f5ff8003dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009e00 tx=0x7f5ff401d470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e763a164de7b57f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40b4070 con 0x7f5ff80039c0 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: dump: 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f601214f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff40ab050 con 0x7f5ff80039c0 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f601214f640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f601214f640 4 set_mon_vals no callback set 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 1 -- 172.21.15.17:0/2482049599 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40a9030 con 0x7f5ff80039c0 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.777+0000 7f600a7fc640 10 monclient: dump: 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:14.786 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 10 monclient: shutdown 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 -- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff80039c0 msgr2=0x7f5ff8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff80039c0 0x7f5ff8003dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009e00 tx=0x7f5ff401d470 comp rx=0 tx=0).stop 2024-09-15T21:34:14.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 -- 172.21.15.17:0/2482049599 shutdown_connections 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80052a0 0x7f5ff800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff8004810 0x7f5ff8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 --2- 172.21.15.17:0/2482049599 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff80039c0 0x7f5ff8003dc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 -- 172.21.15.17:0/2482049599 >> 172.21.15.17:0/2482049599 conn(0x7f600c0a66a0 msgr2=0x7f600c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 -- 172.21.15.17:0/2482049599 shutdown_connections 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 -- 172.21.15.17:0/2482049599 wait complete. 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 10 monclient: build_initial_monmap 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 10 monclient: monmap: 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:14.788 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:14.789 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:14.789 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:14.789 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:14.789 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:14.789 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 Processor -- start 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 1 -- start start 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.781+0000 7f601214f640 10 monclient: init 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80039c0 0x7f600c1ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): picked mon.noname-a con 0x7f5ff80039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 0x7f600c133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): picked mon.noname-c con 0x7f5ff8004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600b7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80039c0 msgr2=0x7f600c1ca180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff80052a0 0x7f600c11d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80039c0 0x7f600c1ca180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): picked mon.noname-b con 0x7f5ff80052a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0a1000 con 0x7f5ff80039c0 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0a0e80 con 0x7f5ff80052a0 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f600c0a1180 con 0x7f5ff8004810 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient(hunting): authenticate will time out at 1978.181206s 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 0x7f600c133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 0x7f600c133b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:43298/0 (socket says 172.21.15.17:43298) 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 -- 172.21.15.17:0/3484009162 learned_addr learned my addr 172.21.15.17:0/3484009162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): get_auth_request con 0x7f5ff8004810 auth_method 0 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600bfff640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff80052a0 0x7f600c11d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600bfff640 10 monclient(hunting): get_auth_request con 0x7f5ff80052a0 auth_method 0 2024-09-15T21:34:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient(hunting): handle_auth_done global_id 164474 payload 824 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 -- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff80052a0 msgr2=0x7f600c11d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff80052a0 0x7f600c11d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 -- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80039c0 msgr2=0x7f600c1ca180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80039c0 0x7f600c1ca180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient: _finish_hunting 0 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 monclient: found mon.noname-c 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 -- 172.21.15.17:0/3484009162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c121220 con 0x7f5ff8004810 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f600affd640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 0x7f600c133b20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc017e20 tx=0x7f5ffc0156a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc019840 con 0x7f5ff8004810 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: dump: 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: _finish_auth 0 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ffc019a00 con 0x7f5ff8004810 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc019df0 con 0x7f5ff8004810 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 10 monclient: dump: 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 5 monclient: authenticate success, global_id 164474 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _renew_subs 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f60097fa640 4 set_mon_vals no callback set 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c121500 con 0x7f5ff8004810 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _renew_subs 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c09d0f0 con 0x7f5ff8004810 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5ffc021420 con 0x7f5ff8004810 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fec03bfc0 0x7f5fec03e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5ffc054d60 con 0x7f5ff8004810 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.785+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600c135670 con 0x7f5ff8004810 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.789+0000 7f600b7fe640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fec03bfc0 0x7f5fec03e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.789+0000 7f600b7fe640 10 monclient: get_auth_request con 0x7f5fec03bfc0 auth_method 0 2024-09-15T21:34:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.789+0000 7f600b7fe640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fec03bfc0 0x7f5fec03e480 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40b1000 tx=0x7f5ff4005dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:14.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.793+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f5ffc015e10 con 0x7f5ff8004810 2024-09-15T21:34:14.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.793+0000 7f6003fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:14.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:14.793+0000 7f6003fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f601214f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f601214f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f601214f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f600c134440 con 0x7f5ff8004810 2024-09-15T21:34:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f6003fff640 1 -- 172.21.15.17:0/3484009162 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f5ffc0046b0 con 0x7f5ff8004810 2024-09-15T21:34:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f6003fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.025+0000 7f6003fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:15.030 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fec03bfc0 msgr2=0x7f5fec03e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fec03bfc0 0x7f5fec03e480 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40b1000 tx=0x7f5ff4005dc0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 10 monclient: shutdown 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 msgr2=0x7f600c133b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 0x7f600c133b20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc017e20 tx=0x7f5ffc0156a0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 shutdown_connections 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5fec03bfc0 0x7f5fec03e480 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff80052a0 0x7f600c11d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5ff8004810 0x7f600c133b20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 --2- 172.21.15.17:0/3484009162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ff80039c0 0x7f600c1ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 >> 172.21.15.17:0/3484009162 conn(0x7f600c0a66a0 msgr2=0x7f600c0a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:15.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 shutdown_connections 2024-09-15T21:34:15.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.029+0000 7f601214f640 1 -- 172.21.15.17:0/3484009162 wait complete. 2024-09-15T21:34:15.047 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:34:15.047 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:34:15.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:34:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 10 monclient: get_monmap_and_config 2024-09-15T21:34:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 10 monclient: build_initial_monmap 2024-09-15T21:34:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 10 monclient: monmap: 2024-09-15T21:34:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:15.156544+0000 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:15.156544+0000 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:15.157 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 1 Processor -- start 2024-09-15T21:34:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 1 -- start start 2024-09-15T21:34:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 10 monclient: init 2024-09-15T21:34:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.153+0000 7fa5a5bae640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5a00a4a90 0x7fa5a00a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 10 monclient(hunting): picked mon.noname-a con 0x7fa5a00a4a90 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa5a00a53b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa5a00a5800 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a00a2c30 con 0x7fa5a00a53b0 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a0138670 con 0x7fa5a00a5800 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a00a0f00 con 0x7fa5a00a4a90 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 10 monclient(hunting): authenticate will time out at 1978.552122s 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7fa5a00a4a90 msgr2=0x7fa5a00a4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5a00a4a90 0x7fa5a00a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7fa5a00a53b0 legacy=0x7fa5a0137850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa5a00a5800 0x7fa5a0137f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.17:57508/0 (socket says 172.21.15.17:57508) 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59ffff640 1 -- 172.21.15.17:0/2630980289 learned_addr learned my addr 172.21.15.17:0/2630980289 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301764449 0 0) 0x7fa5a0138670 con 0x7fa5a00a5800 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): my global_id is 194170 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa588003530 con 0x7fa5a00a5800 2024-09-15T21:34:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2205969871 0 0) 0x7fa588003530 con 0x7fa5a00a5800 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 >> v2:172.21.15.17:3300/0 conn(0x7fa5a00a4a90 msgr2=0x7fa5a00a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 --2- 172.21.15.17:0/2630980289 >> v2:172.21.15.17:3300/0 conn(0x7fa5a00a4a90 0x7fa5a00a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 >> v1:172.21.15.17:6789/0 conn(0x7fa5a00a53b0 legacy=0x7fa5a0137850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a00a19b0 con 0x7fa5a00a5800 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa598001a80 con 0x7fa5a00a5800 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: dump: 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:34:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 >> v1:172.21.15.119:6789/0 conn(0x7fa5a00a5800 legacy=0x7fa5a0137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5880039c0 0x7fa588003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): picked mon.c con 0x7fa5880039c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa588004810 0x7fa588004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): picked mon.b con 0x7fa588004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880053d0 0x7fa58800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): picked mon.a con 0x7fa5880053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00a0f00 con 0x7fa5880053d0 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa58800c270 con 0x7fa588004810 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa58800c470 con 0x7fa5880039c0 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59ffff640 1 -- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880053d0 msgr2=0x7fa58800b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59ffff640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880053d0 0x7fa58800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5880039c0 0x7fa588003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa588004810 0x7fa588004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): get_auth_request con 0x7fa5880039c0 auth_method 0 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): get_auth_request con 0x7fa588004810 auth_method 0 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_done global_id 194170 payload 293 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 -- 172.21.15.17:0/2630980289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa588004810 msgr2=0x7fa588004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa588004810 0x7fa588004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 -- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880053d0 msgr2=0x7fa58800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880053d0 0x7fa58800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 monclient: found mon.c 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 1 -- 172.21.15.17:0/2630980289 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa588003530 con 0x7fa5880039c0 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59ffff640 1 -- 172.21.15.17:0/2630980289 reap_dead start 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa59f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 5 monclient: authenticate success, global_id 194170 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.157+0000 7fa5a5bae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5880039c0 0x7fa588003dc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa590003990 tx=0x7fa59001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f634747182c38bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590009520 con 0x7fa5880039c0 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: dump: 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.172 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590003bc0 con 0x7fa5880039c0 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 4 set_mon_vals no callback set 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 1 -- 172.21.15.17:0/2630980289 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590005070 con 0x7fa5880039c0 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59e7fc640 10 monclient: dump: 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: shutdown 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5880039c0 msgr2=0x7fa588003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5880039c0 0x7fa588003dc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa590003990 tx=0x7fa59001c420 comp rx=0 tx=0).stop 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/2630980289 shutdown_connections 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/2630980289 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5880039c0 0x7fa588003dc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/2630980289 >> 172.21.15.17:0/2630980289 conn(0x7fa5a00aabb0 msgr2=0x7fa5a00a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/2630980289 shutdown_connections 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/2630980289 wait complete. 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: build_initial_monmap 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: monmap: 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:15.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 Processor -- start 2024-09-15T21:34:15.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- start start 2024-09-15T21:34:15.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: init 2024-09-15T21:34:15.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:15.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:34:15.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880039c0 0x7fa5a01ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): picked mon.noname-a con 0x7fa5880039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5a01cd290 0x7fa5a0125270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): picked mon.noname-b con 0x7fa5a01cd290 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 0x7fa5a01257b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): picked mon.noname-c con 0x7fa5a01cd6f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00a4420 con 0x7fa5880039c0 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00a1f70 con 0x7fa5a01cd290 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a0128c20 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient(hunting): authenticate will time out at 1978.558411s 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59f7fe640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880039c0 msgr2=0x7fa5a01ccd50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880039c0 0x7fa5a01ccd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 0x7fa5a01257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 0x7fa5a01257b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:43318/0 (socket says 172.21.15.17:43318) 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 -- 172.21.15.17:0/3536837056 learned_addr learned my addr 172.21.15.17:0/3536837056 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): get_auth_request con 0x7fa5a01cd6f0 auth_method 0 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59effd640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5a01cd290 0x7fa5a0125270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59effd640 10 monclient(hunting): get_auth_request con 0x7fa5a01cd290 auth_method 0 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient(hunting): handle_auth_done global_id 164477 payload 824 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5a01cd290 msgr2=0x7fa5a0125270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5a01cd290 0x7fa5a0125270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880039c0 msgr2=0x7fa5a01ccd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5880039c0 0x7fa5a01ccd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 monclient: found mon.noname-c 2024-09-15T21:34:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a0128f00 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59effd640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5a01cd290 0x7fa5a0125270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59ffff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 0x7fa5a01257b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00a2420 tx=0x7fa59800ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800dae0 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: dump: 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: _finish_auth 0 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59800dca0 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 5 monclient: authenticate success, global_id 164477 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: _renew_subs 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a0126ba0 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800a310 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa587fff640 10 monclient: dump: 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa59dffb640 4 set_mon_vals no callback set 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: _renew_subs 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.161+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a0127700 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa5a5bae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 0x7fa5a01260d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa5a5bae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a00a45a0 con 0x7fa5a0125cf0 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 0x7fa5a01260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient: get_auth_request con 0x7fa5a0125cf0 auth_method 0 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa598014070 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_done global_id 164480 payload 824 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5a00a1280 con 0x7fa5a0125cf0 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa58c03b970 0x7fa58c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 0x7fa5a01260d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa590006fd0 tx=0x7fa588005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa5980532b0 con 0x7fa5a01cd6f0 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590006de0 con 0x7fa5a0125cf0 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 10 monclient: dump: 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa5900050c0 con 0x7fa5a0125cf0 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 msgr2=0x7fa5a01260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 0x7fa5a01260d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa590006fd0 tx=0x7fa588005840 comp rx=0 tx=0).stop 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 msgr2=0x7fa5a01260d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa587fff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a0125cf0 0x7fa5a01260d0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa590006fd0 tx=0x7fa588005840 comp rx=0 tx=0).stop 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59effd640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa58c03b970 0x7fa58c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59effd640 10 monclient: get_auth_request con 0x7fa58c03b970 auth_method 0 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.165+0000 7fa59effd640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa58c03b970 0x7fa58c03de30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa5940038a0 tx=0x7fa594008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa5a5bae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 0x7fa5a0121b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa5a5bae640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a01288a0 con 0x7fa5a01216f0 2024-09-15T21:34:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 0x7fa5a0121b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient: get_auth_request con 0x7fa5a01216f0 auth_method 0 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 10 monclient(hunting): handle_auth_done global_id 164483 payload 824 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 1 -- 172.21.15.17:0/3536837056 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa5a01384c0 con 0x7fa5a01216f0 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa59f7fe640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 0x7fa5a0121b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa59001c250 tx=0x7fa5900090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590005840 con 0x7fa5a01216f0 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa587fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa587fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa587fff640 10 monclient: dump: 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.169+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4869 (secure 0 0 0) 0x7fa5900ae480 con 0x7fa5a01216f0 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa587fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa587fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 msgr2=0x7fa5a0121b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa587fff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 0x7fa5a0121b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa59001c250 tx=0x7fa5900090e0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa587fff640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 msgr2=0x7fa5a0121b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa587fff640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01216f0 0x7fa5a0121b00 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa59001c250 tx=0x7fa5900090e0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 154, 2024-09-15T21:34:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 203753, 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:15.208 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa58c03b970 msgr2=0x7fa58c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa58c03b970 0x7fa58c03de30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa5940038a0 tx=0x7fa594008040 comp rx=0 tx=0).stop 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 10 monclient: shutdown 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 msgr2=0x7fa5a01257b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 0x7fa5a01257b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00a2420 tx=0x7fa59800ff00 comp rx=0 tx=0).stop 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa59ffff640 1 -- 172.21.15.17:0/3536837056 reap_dead start 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 shutdown_connections 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 --2- 172.21.15.17:0/3536837056 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5a01cd6f0 0x7fa5a01257b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 >> 172.21.15.17:0/3536837056 conn(0x7fa5a00aabb0 msgr2=0x7fa5a009e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:15.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 shutdown_connections 2024-09-15T21:34:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.173+0000 7fa5a5bae640 1 -- 172.21.15.17:0/3536837056 wait complete. 2024-09-15T21:34:15.211 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:34:15.211 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:34:15.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient: get_monmap_and_config 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient: build_initial_monmap 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient: monmap: 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:15.326351+0000 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:15.326351+0000 2024-09-15T21:34:15.326 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:15.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 Processor -- start 2024-09-15T21:34:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 -- start start 2024-09-15T21:34:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient: init 2024-09-15T21:34:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2024-09-15T21:34:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34901378c0 0x7f3490137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): picked mon.noname-a con 0x7f34901378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f34901381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 0x7f34900a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): picked mon.noname-b con 0x7f34900a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34900a1d00 con 0x7f34901381e0 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f34900a1fe0 con 0x7f34901378c0 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f34900a22c0 con 0x7f34900a5210 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f349575d640 10 monclient(hunting): authenticate will time out at 1978.721750s 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348effd640 1 -- >> v2:172.21.15.17:3300/0 conn(0x7f34901378c0 msgr2=0x7f3490137ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348e7fc640 1 -- >> v1:172.21.15.17:6789/0 conn(0x7f34901381e0 legacy=0x7f34900a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.17:6789/0 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34901378c0 0x7f3490137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 0x7f34900a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 0x7f34900a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:43354/0 (socket says 172.21.15.17:43354) 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 -- 172.21.15.17:0/4147135179 learned_addr learned my addr 172.21.15.17:0/4147135179 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): get_auth_request con 0x7f34900a5210 auth_method 0 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient(hunting): handle_auth_done global_id 164486 payload 274 2024-09-15T21:34:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 -- 172.21.15.17:0/4147135179 >> v2:172.21.15.17:3300/0 conn(0x7f34901378c0 msgr2=0x7f3490137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 --2- 172.21.15.17:0/4147135179 >> v2:172.21.15.17:3300/0 conn(0x7f34901378c0 0x7f3490137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 -- 172.21.15.17:0/4147135179 >> v1:172.21.15.17:6789/0 conn(0x7f34901381e0 legacy=0x7f34900a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.325+0000 7f348f7fe640 1 -- 172.21.15.17:0/4147135179 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34900a4490 con 0x7f34900a5210 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 --2- 172.21.15.17:0/4147135179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 0x7f34900a56a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34800098a0 tx=0x7f348001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa5528e7f0e2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 1 -- 172.21.15.17:0/4147135179 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800b1070 con 0x7f34900a5210 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: dump: 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 1 -- 172.21.15.17:0/4147135179 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34800ac620 con 0x7f34900a5210 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 1 -- 172.21.15.17:0/4147135179 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800ac9f0 con 0x7f34900a5210 2024-09-15T21:34:15.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 5 monclient: authenticate success, global_id 164486 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 4 set_mon_vals no callback set 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348dffb640 10 monclient: dump: 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: shutdown 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/4147135179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 msgr2=0x7f34900a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 --2- 172.21.15.17:0/4147135179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 0x7f34900a56a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34800098a0 tx=0x7f348001c420 comp rx=0 tx=0).stop 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/4147135179 shutdown_connections 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 --2- 172.21.15.17:0/4147135179 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34900a5210 0x7f34900a56a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 --2- 172.21.15.17:0/4147135179 >> v2:172.21.15.17:3300/0 conn(0x7f34901378c0 0x7f3490137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/4147135179 >> 172.21.15.17:0/4147135179 conn(0x7f34900aac40 msgr2=0x7f34900a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/4147135179 shutdown_connections 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/4147135179 wait complete. 2024-09-15T21:34:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: build_initial_monmap 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: monmap: 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 Processor -- start 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- start start 2024-09-15T21:34:15.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: init 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34900a5210 0x7f34901cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): picked mon.noname-b con 0x7f34900a5210 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34901378c0 0x7f34901d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): picked mon.noname-a con 0x7f34901378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34901381e0 0x7f349013cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): picked mon.noname-c con 0x7f34901381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f34901425c0 con 0x7f34901378c0 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3490138700 con 0x7f34900a5210 2024-09-15T21:34:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3490142740 con 0x7f34901381e0 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient(hunting): authenticate will time out at 1978.726670s 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348e7fc640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34901378c0 msgr2=0x7f34901d0440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34901378c0 0x7f34901d0440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34901381e0 0x7f349013cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34901381e0 0x7f349013cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:43366/0 (socket says 172.21.15.17:43366) 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 -- 172.21.15.17:0/1724414890 learned_addr learned my addr 172.21.15.17:0/1724414890 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): get_auth_request con 0x7f34901381e0 auth_method 0 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient(hunting): handle_auth_done global_id 164489 payload 824 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348effd640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34900a5210 0x7f34901cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34900a5210 msgr2=0x7f34901cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34900a5210 0x7f34901cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34901378c0 msgr2=0x7f34901d0440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34901378c0 0x7f34901d0440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 monclient: found mon.noname-c 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34900a6fe0 con 0x7f34901381e0 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348effd640 10 monclient: get_auth_request con 0x7f34900a5210 auth_method 0 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348effd640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34900a5210 0x7f34901cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348f7fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34901381e0 0x7f349013cec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f348001ccc0 tx=0x7f348001ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800bb020 con 0x7f34901381e0 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: dump: 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34800b7620 con 0x7f34901381e0 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 5 monclient: authenticate success, global_id 164489 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: _renew_subs 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349013e980 con 0x7f34901381e0 2024-09-15T21:34:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800b1040 con 0x7f34901381e0 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f34777fe640 10 monclient: dump: 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: _renew_subs 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f348cff9640 4 set_mon_vals no callback set 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.329+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3490121120 con 0x7f34901381e0 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f349575d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f349575d640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 0x7f349013d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f34900a56a0 con 0x7f34901d0980 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 0x7f349013d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient: get_auth_request con 0x7f34901d0980 auth_method 0 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f34800be020 con 0x7f34901381e0 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f347c03b970 0x7f347c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 10 monclient(hunting): handle_auth_done global_id 194176 payload 824 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34901212f0 con 0x7f34901d0980 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f34800eab10 con 0x7f34901381e0 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348effd640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 0x7f349013d400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f348800ccd0 tx=0x7f348800a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348e7fc640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f347c03b970 0x7f347c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348e7fc640 10 monclient: get_auth_request con 0x7f347c03b970 auth_method 0 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488013070 con 0x7f34901d0980 2024-09-15T21:34:15.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 10 monclient: dump: 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f348e7fc640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f347c03b970 0x7f347c03de30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3484006fd0 tx=0x7f3484006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f348800ae90 con 0x7f34901d0980 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 msgr2=0x7f349013d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 0x7f349013d400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f348800ccd0 tx=0x7f348800a5f0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 msgr2=0x7f349013d400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:15.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.333+0000 7f34777fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34901d0980 0x7f349013d400 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 0x7f3490138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f349013dd50 con 0x7f349013d940 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 0x7f3490138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient: get_auth_request con 0x7f349013d940 auth_method 0 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:15.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 10 monclient(hunting): handle_auth_done global_id 194179 payload 824 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 1 -- 172.21.15.17:0/1724414890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3490069270 con 0x7f349013d940 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348effd640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 0x7f3490138d60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f348800ccd0 tx=0x7f3488005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488019ad0 con 0x7f349013d940 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 10 monclient: dump: 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.355 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4536 (secure 0 0 0) 0x7f34880233a0 con 0x7f349013d940 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 msgr2=0x7f3490138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 0x7f3490138d60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f348800ccd0 tx=0x7f3488005d60 comp rx=0 tx=0).stop 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 msgr2=0x7f3490138d60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:15.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f34777fe640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f349013d940 0x7f3490138d60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 154, 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22131, 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:15.357 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:15.358 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:15.359 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:15.360 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:15.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.362 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:15.365 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.366 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.366 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:15.370 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:15.370 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:15.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:15.373 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:15.374 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:34:15.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:15.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:15.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:15.374 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:15.374 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:15.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f347c03b970 msgr2=0x7f347c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f347c03b970 0x7f347c03de30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3484006fd0 tx=0x7f3484006d40 comp rx=0 tx=0).stop 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 10 monclient: shutdown 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34901381e0 msgr2=0x7f349013cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 --2- 172.21.15.17:0/1724414890 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f34901381e0 0x7f349013cec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f348001ccc0 tx=0x7f348001ac60 comp rx=0 tx=0).stop 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348f7fe640 1 -- 172.21.15.17:0/1724414890 reap_dead start 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f348f7fe640 1 -- 172.21.15.17:0/1724414890 reap_dead start 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 shutdown_connections 2024-09-15T21:34:15.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 >> 172.21.15.17:0/1724414890 conn(0x7f34900aac40 msgr2=0x7f34900a8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:15.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 shutdown_connections 2024-09-15T21:34:15.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:15.341+0000 7f349575d640 1 -- 172.21.15.17:0/1724414890 wait complete. 2024-09-15T21:34:15.377 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:34:15.377 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:34:35.378 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T21:34:35.378 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:34:35.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:34:35.381 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:34:35.381 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:34:35.382 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient: get_monmap_and_config 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient: build_initial_monmap 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient: monmap: 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:35.503125+0000 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:35.503125+0000 2024-09-15T21:34:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:35.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 Processor -- start 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 -- start start 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient: init 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): picked mon.noname-a con 0x7f23b80a4a90 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b80a53b0 0x7f23b80a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): picked mon.noname-b con 0x7f23b80a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:35.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f23b8137850 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23b80a2c30 con 0x7f23b8137850 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f23b80a0f00 con 0x7f23b80a4a90 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f23b80a19b0 con 0x7f23b80a53b0 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bea81640 10 monclient(hunting): authenticate will time out at 1998.898125s 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33986/0 (socket says 172.21.15.17:33986) 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 -- 172.21.15.17:0/304760884 learned_addr learned my addr 172.21.15.17:0/304760884 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:35.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 10 monclient(hunting): get_auth_request con 0x7f23b80a4a90 auth_method 0 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b80a53b0 0x7f23b80a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): get_auth_request con 0x7f23b80a53b0 auth_method 0 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 -- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 msgr2=0x7f23b80a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 -- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 msgr2=0x7f23b80a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 --2- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23bc81d640 1 --2- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:35.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient(hunting): handle_auth_done global_id 164492 payload 274 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 1 -- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 msgr2=0x7f23b80a4e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 1 --2- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 1 -- 172.21.15.17:0/304760884 >> v1:172.21.15.119:6789/0 conn(0x7f23b8137850 legacy=0x7f23b8137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 1 monclient: found mon.noname-b 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.501+0000 7f23b7fff640 1 -- 172.21.15.17:0/304760884 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b80a1c90 con 0x7f23b80a53b0 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 --2- 172.21.15.17:0/304760884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b80a53b0 0x7f23b80a57d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097c0 tx=0x7f23ac01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f569b4a2081cde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:35.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 1 -- 172.21.15.17:0/304760884 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0b0070 con 0x7f23b80a53b0 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: dump: 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:35.516 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 1 -- 172.21.15.17:0/304760884 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ac0ab600 con 0x7f23b80a53b0 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 1 -- 172.21.15.17:0/304760884 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0ab9d0 con 0x7f23b80a53b0 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 5 monclient: authenticate success, global_id 164492 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 4 set_mon_vals no callback set 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b77fe640 10 monclient: dump: 2024-09-15T21:34:35.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: shutdown 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/304760884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b80a53b0 msgr2=0x7f23b80a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:35.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 --2- 172.21.15.17:0/304760884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b80a53b0 0x7f23b80a57d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097c0 tx=0x7f23ac01c330 comp rx=0 tx=0).stop 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/304760884 shutdown_connections 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 --2- 172.21.15.17:0/304760884 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b80a53b0 0x7f23b80a57d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 --2- 172.21.15.17:0/304760884 >> v2:172.21.15.17:3300/0 conn(0x7f23b80a4a90 0x7f23b80a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/304760884 >> 172.21.15.17:0/304760884 conn(0x7f23b80aabb0 msgr2=0x7f23b80a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/304760884 shutdown_connections 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/304760884 wait complete. 2024-09-15T21:34:35.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: build_initial_monmap 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: monmap: 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 Processor -- start 2024-09-15T21:34:35.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- start start 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: init 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b80a4a90 0x7f23b81cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): picked mon.noname-b con 0x7f23b80a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): picked mon.noname-a con 0x7f23b80a53b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b8137850 0x7f23b813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): picked mon.noname-c con 0x7f23b8137850 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81426d0 con 0x7f23b80a53b0 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:35.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8142550 con 0x7f23b80a4a90 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f23b8142850 con 0x7f23b8137850 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient(hunting): authenticate will time out at 1998.902969s 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b8137850 0x7f23b813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33994/0 (socket says 172.21.15.17:33994) 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 -- 172.21.15.17:0/3103254390 learned_addr learned my addr 172.21.15.17:0/3103254390 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 10 monclient(hunting): get_auth_request con 0x7f23b80a53b0 auth_method 0 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:35.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): get_auth_request con 0x7f23b8137850 auth_method 0 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 -- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 msgr2=0x7f23b81369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 -- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 msgr2=0x7f23b81369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:35.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b7fff640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bc81d640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b80a4a90 0x7f23b81cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bc81d640 10 monclient(hunting): get_auth_request con 0x7f23b80a4a90 auth_method 0 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bc81d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bc81d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bc81d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient(hunting): handle_auth_done global_id 164495 payload 824 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 -- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b80a4a90 msgr2=0x7f23b81cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b80a4a90 0x7f23b81cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 -- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 msgr2=0x7f23b81369d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient: _finish_hunting 0 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 monclient: found mon.noname-c 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 -- 172.21.15.17:0/3103254390 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81d0a10 con 0x7f23b8137850 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bd01e640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b8137850 0x7f23b813cb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23b000c700 tx=0x7f23b000cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0003b80 con 0x7f23b8137850 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:35.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: dump: 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:35.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: _finish_auth 0 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b0003d20 con 0x7f23b8137850 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 5 monclient: authenticate success, global_id 164495 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: _renew_subs 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b80a1280 con 0x7f23b8137850 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b000aa50 con 0x7f23b8137850 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b57fa640 10 monclient: dump: 2024-09-15T21:34:35.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: _renew_subs 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23b6ffd640 4 set_mon_vals no callback set 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.505+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b8121080 con 0x7f23b8137850 2024-09-15T21:34:35.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bea81640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bea81640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bea81640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b80a4e70 con 0x7f23b8137850 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f23b000abf0 con 0x7f23b8137850 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23b57fa640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f23a403b9c0 0x7f23a403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f23b00504a0 con 0x7f23b8137850 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bc81d640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f23a403b9c0 0x7f23a403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bc81d640 10 monclient: get_auth_request con 0x7f23a403b9c0 auth_method 0 2024-09-15T21:34:35.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.509+0000 7f23bc81d640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f23a403b9c0 0x7f23a403de80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009100 tx=0x7f23a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:35.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.513+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f23b81d0a10 con 0x7f23b8137850 2024-09-15T21:34:35.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.513+0000 7f23b57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:35.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.513+0000 7f23b57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:35.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:35.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f23b8120ce0 con 0x7f23b8137850 2024-09-15T21:34:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23b57fa640 1 -- 172.21.15.17:0/3103254390 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f23b80a2c30 con 0x7f23b8137850 2024-09-15T21:34:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23b57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23b57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:35.759 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:35.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f23a403b9c0 msgr2=0x7f23a403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f23a403b9c0 0x7f23a403de80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009100 tx=0x7f23a8008040 comp rx=0 tx=0).stop 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 10 monclient: shutdown 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b8137850 msgr2=0x7f23b813cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b8137850 0x7f23b813cb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23b000c700 tx=0x7f23b000cbc0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 shutdown_connections 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f23a403b9c0 0x7f23a403de80 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23b8137850 0x7f23b813cb40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23b80a53b0 0x7f23b81369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 --2- 172.21.15.17:0/3103254390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b80a4a90 0x7f23b81cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 >> 172.21.15.17:0/3103254390 conn(0x7f23b80aabb0 msgr2=0x7f23b80a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 shutdown_connections 2024-09-15T21:34:35.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:35.757+0000 7f23bea81640 1 -- 172.21.15.17:0/3103254390 wait complete. 2024-09-15T21:34:38.779 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient: get_monmap_and_config 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient: build_initial_monmap 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient: monmap: 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:38.911728+0000 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:38.911728+0000 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:38.913 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 Processor -- start 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 -- start start 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient: init 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb6c0a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb6c09ffc0 0x7ffb6c13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): picked mon.noname-c con 0x7ffb6c09ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 0x7ffb6c13d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): picked mon.noname-b con 0x7ffb6c0a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb6c0a3130 con 0x7ffb6c0a0890 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffb6c073df0 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6c0a1bf0 con 0x7ffb6c0a1380 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb70843640 10 monclient(hunting): authenticate will time out at 2002.307343s 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 0x7ffb6c13d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 0x7ffb6c13d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:39996/0 (socket says 172.21.15.17:39996) 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 -- 172.21.15.17:0/4247826949 learned_addr learned my addr 172.21.15.17:0/4247826949 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): get_auth_request con 0x7ffb6c0a1380 auth_method 0 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb69d9b640 1 --2- 172.21.15.17:0/4247826949 >> v2:172.21.15.119:3300/0 conn(0x7ffb6c09ffc0 0x7ffb6c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb69d9b640 10 monclient(hunting): get_auth_request con 0x7ffb6c09ffc0 auth_method 0 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient(hunting): handle_auth_done global_id 164498 payload 274 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 -- 172.21.15.17:0/4247826949 >> v2:172.21.15.119:3300/0 conn(0x7ffb6c09ffc0 msgr2=0x7ffb6c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 --2- 172.21.15.17:0/4247826949 >> v2:172.21.15.119:3300/0 conn(0x7ffb6c09ffc0 0x7ffb6c13aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 -- 172.21.15.17:0/4247826949 >> v1:172.21.15.17:6789/0 conn(0x7ffb6c0a0890 legacy=0x7ffb6c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient: _finish_hunting 0 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 monclient: found mon.noname-b 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.909+0000 7ffb6ad9d640 1 -- 172.21.15.17:0/4247826949 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c09ccd0 con 0x7ffb6c0a1380 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6ad9d640 1 --2- 172.21.15.17:0/4247826949 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 0x7ffb6c13d010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009e80 tx=0x7ffb6401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9199d7d14feaf7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 1 -- 172.21.15.17:0/4247826949 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640a56a0 con 0x7ffb6c0a1380 2024-09-15T21:34:38.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: dump: 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:38.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: _finish_auth 0 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: _check_auth_tickets 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 1 -- 172.21.15.17:0/4247826949 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb640a5840 con 0x7ffb6c0a1380 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 1 -- 172.21.15.17:0/4247826949 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640a5c10 con 0x7ffb6c0a1380 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 5 monclient: authenticate success, global_id 164498 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 4 set_mon_vals no callback set 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb6959a640 10 monclient: dump: 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:38.922 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 10 monclient: shutdown 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 -- 172.21.15.17:0/4247826949 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 msgr2=0x7ffb6c13d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 --2- 172.21.15.17:0/4247826949 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 0x7ffb6c13d010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009e80 tx=0x7ffb6401c8d0 comp rx=0 tx=0).stop 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 -- 172.21.15.17:0/4247826949 shutdown_connections 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 --2- 172.21.15.17:0/4247826949 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c0a1380 0x7ffb6c13d010 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 --2- 172.21.15.17:0/4247826949 >> v2:172.21.15.119:3300/0 conn(0x7ffb6c09ffc0 0x7ffb6c13aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:38.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 -- 172.21.15.17:0/4247826949 >> 172.21.15.17:0/4247826949 conn(0x7ffb6c0a66a0 msgr2=0x7ffb6c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 -- 172.21.15.17:0/4247826949 shutdown_connections 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 1 -- 172.21.15.17:0/4247826949 wait complete. 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 10 monclient: build_initial_monmap 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.913+0000 7ffb70843640 10 monclient: monmap: 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:38.927 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 Processor -- start 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- start start 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: init 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 0x7ffb6c11e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): picked mon.noname-c con 0x7ffb6c09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6c0a0890 0x7ffb6c135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): picked mon.noname-b con 0x7ffb6c0a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 0x7ffb6c11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): picked mon.noname-a con 0x7ffb6c0a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:38.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c140100 con 0x7ffb6c0a1380 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c13ff80 con 0x7ffb6c0a0890 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6c140280 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient(hunting): authenticate will time out at 2002.312989s 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 0x7ffb6c11e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 0x7ffb6c11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 0x7ffb6c11e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40002/0 (socket says 172.21.15.17:40002) 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 10 monclient(hunting): get_auth_request con 0x7ffb6c0a1380 auth_method 0 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 -- 172.21.15.17:0/2873772978 learned_addr learned my addr 172.21.15.17:0/2873772978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:38.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): get_auth_request con 0x7ffb6c09ffc0 auth_method 0 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 1 -- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 msgr2=0x7ffb6c11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 1 -- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 msgr2=0x7ffb6c11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 0x7ffb6c11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:38.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6c0a0890 0x7ffb6c135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6ad9d640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 0x7ffb6c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): get_auth_request con 0x7ffb6c0a0890 auth_method 0 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient(hunting): handle_auth_done global_id 164501 payload 824 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 -- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6c0a0890 msgr2=0x7ffb6c135790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6c0a0890 0x7ffb6c135790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 -- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 msgr2=0x7ffb6c11c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 0x7ffb6c11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:38.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 monclient: found mon.noname-c 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 -- 172.21.15.17:0/2873772978 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c120120 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb6a59c640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 0x7ffb6c11e920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c530 tx=0x7ffb5c00c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c017070 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: dump: 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.932 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: _finish_auth 0 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: _check_auth_tickets 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 5 monclient: authenticate success, global_id 164501 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0052d0 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _renew_subs 2024-09-15T21:34:38.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c136430 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c012870 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 10 monclient: dump: 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:38.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb68d99640 4 set_mon_vals no callback set 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _renew_subs 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c11fe40 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6c0a0c70 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffb5c01a070 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb4c03b9c0 0x7ffb4c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ffb5c04f880 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb4c03b9c0 0x7ffb4c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:38.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.917+0000 7ffb69d9b640 10 monclient: get_auth_request con 0x7ffb4c03b9c0 auth_method 0 2024-09-15T21:34:38.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.921+0000 7ffb69d9b640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb4c03b9c0 0x7ffb4c03de80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c13ccf0 tx=0x7ffb60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:38.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.921+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffb5c003d60 con 0x7ffb6c09ffc0 2024-09-15T21:34:38.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.921+0000 7ffb5affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:38.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:38.921+0000 7ffb5affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:39.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:39.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:39.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:39.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ffb6c06d020 con 0x7ffb6c09ffc0 2024-09-15T21:34:39.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb5affd640 1 -- 172.21.15.17:0/2873772978 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7ffb5c019cd0 con 0x7ffb6c09ffc0 2024-09-15T21:34:39.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb5affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:39.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb5affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:39.160 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:39.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb4c03b9c0 msgr2=0x7ffb4c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb4c03b9c0 0x7ffb4c03de80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c13ccf0 tx=0x7ffb60008040 comp rx=0 tx=0).stop 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 10 monclient: shutdown 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 msgr2=0x7ffb6c11e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 0x7ffb6c11e920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c530 tx=0x7ffb5c00c9f0 comp rx=0 tx=0).stop 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 shutdown_connections 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb4c03b9c0 0x7ffb4c03de80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb6c0a1380 0x7ffb6c11c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb6c0a0890 0x7ffb6c135790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 --2- 172.21.15.17:0/2873772978 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffb6c09ffc0 0x7ffb6c11e920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 >> 172.21.15.17:0/2873772978 conn(0x7ffb6c0a66a0 msgr2=0x7ffb6c0a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:39.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.157+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 shutdown_connections 2024-09-15T21:34:39.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:39.161+0000 7ffb70843640 1 -- 172.21.15.17:0/2873772978 wait complete. 2024-09-15T21:34:42.179 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:42.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient: get_monmap_and_config 2024-09-15T21:34:42.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient: build_initial_monmap 2024-09-15T21:34:42.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient: monmap: 2024-09-15T21:34:42.312 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:42.312404+0000 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:42.312404+0000 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:42.313 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 Processor -- start 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 -- start start 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient: init 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1500a0890 0x7fd1500a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): picked mon.noname-c con 0x7fd1500a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd15009ffc0 0x7fd1500a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): picked mon.noname-a con 0x7fd15009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd150135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1500a3130 con 0x7fd150135670 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd1500a16f0 con 0x7fd15009ffc0 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd150073df0 con 0x7fd1500a0890 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.309+0000 7fd154cd8640 10 monclient(hunting): authenticate will time out at 2005.707819s 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd15009ffc0 0x7fd1500a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd15009ffc0 0x7fd1500a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40572/0 (socket says 172.21.15.17:40572) 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 -- 172.21.15.17:0/3339909734 learned_addr learned my addr 172.21.15.17:0/3339909734 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): get_auth_request con 0x7fd15009ffc0 auth_method 0 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14e59c640 1 --2- 172.21.15.17:0/3339909734 >> v2:172.21.15.119:3300/0 conn(0x7fd1500a0890 0x7fd1500a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_done global_id 204099 payload 274 2024-09-15T21:34:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 -- 172.21.15.17:0/3339909734 >> v2:172.21.15.119:3300/0 conn(0x7fd1500a0890 msgr2=0x7fd1500a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 --2- 172.21.15.17:0/3339909734 >> v2:172.21.15.119:3300/0 conn(0x7fd1500a0890 0x7fd1500a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 -- 172.21.15.17:0/3339909734 >> v1:172.21.15.119:6789/0 conn(0x7fd150135670 legacy=0x7fd150135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 -- 172.21.15.17:0/3339909734 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1500a1bf0 con 0x7fd15009ffc0 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14e59c640 10 monclient: get_auth_request con 0x7fd1500a0890 auth_method 0 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14e59c640 1 --2- 172.21.15.17:0/3339909734 >> v2:172.21.15.119:3300/0 conn(0x7fd1500a0890 0x7fd1500a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14dd9b640 1 --2- 172.21.15.17:0/3339909734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd15009ffc0 0x7fd1500a11b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd140009e50 tx=0x7fd14001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec1c9738e1d05ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 1 -- 172.21.15.17:0/3339909734 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400a5640 con 0x7fd15009ffc0 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: dump: 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.322 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: _finish_auth 0 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 1 -- 172.21.15.17:0/3339909734 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1400a57e0 con 0x7fd15009ffc0 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 1 -- 172.21.15.17:0/3339909734 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400a5bb0 con 0x7fd15009ffc0 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 5 monclient: authenticate success, global_id 204099 2024-09-15T21:34:42.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 4 set_mon_vals no callback set 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd14d59a640 10 monclient: dump: 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.324 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 10 monclient: shutdown 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 -- 172.21.15.17:0/3339909734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd15009ffc0 msgr2=0x7fd1500a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 --2- 172.21.15.17:0/3339909734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd15009ffc0 0x7fd1500a11b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd140009e50 tx=0x7fd14001c8b0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 -- 172.21.15.17:0/3339909734 shutdown_connections 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 --2- 172.21.15.17:0/3339909734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd15009ffc0 0x7fd1500a11b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 --2- 172.21.15.17:0/3339909734 >> v2:172.21.15.119:3300/0 conn(0x7fd1500a0890 0x7fd1500a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 -- 172.21.15.17:0/3339909734 >> 172.21.15.17:0/3339909734 conn(0x7fd1500a66a0 msgr2=0x7fd1500a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 -- 172.21.15.17:0/3339909734 shutdown_connections 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.313+0000 7fd154cd8640 1 -- 172.21.15.17:0/3339909734 wait complete. 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: build_initial_monmap 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: monmap: 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:42.326 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:42.327 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:42.327 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.327 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:42.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:42.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:42.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 Processor -- start 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- start start 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: init 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd15009ffc0 0x7fd150134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): picked mon.noname-b con 0x7fd15009ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 0x7fd15009d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): picked mon.noname-a con 0x7fd1500a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd150135670 0x7fd15009d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): picked mon.noname-c con 0x7fd150135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd150140690 con 0x7fd1500a0890 2024-09-15T21:34:42.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd150140510 con 0x7fd15009ffc0 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd150140810 con 0x7fd150135670 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient(hunting): authenticate will time out at 2005.713319s 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 0x7fd15009d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 0x7fd15009d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40578/0 (socket says 172.21.15.17:40578) 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 -- 172.21.15.17:0/2850393463 learned_addr learned my addr 172.21.15.17:0/2850393463 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): get_auth_request con 0x7fd1500a0890 auth_method 0 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd150135670 0x7fd15009d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): get_auth_request con 0x7fd150135670 auth_method 0 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd15009ffc0 0x7fd150134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 10 monclient(hunting): get_auth_request con 0x7fd15009ffc0 auth_method 0 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient(hunting): handle_auth_done global_id 204102 payload 824 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 -- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd150135670 msgr2=0x7fd15009d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd150135670 0x7fd15009d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 -- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd15009ffc0 msgr2=0x7fd150134880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd15009ffc0 0x7fd150134880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 -- 172.21.15.17:0/2850393463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15013d2c0 con 0x7fd1500a0890 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14dd9b640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 0x7fd15009d400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1400af040 tx=0x7fd140019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400bc020 con 0x7fd1500a0890 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: dump: 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: _finish_auth 0 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: _check_auth_tickets 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1400b88c0 con 0x7fd1500a0890 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 5 monclient: authenticate success, global_id 204102 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _renew_subs 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd150120990 con 0x7fd1500a0890 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140009510 con 0x7fd1500a0890 2024-09-15T21:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 10 monclient: dump: 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14cd99640 4 set_mon_vals no callback set 2024-09-15T21:34:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _renew_subs 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1501cfe00 con 0x7fd1500a0890 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd150135aa0 con 0x7fd1500a0890 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd1400bf020 con 0x7fd1500a0890 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd13003b9c0 0x7fd13003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd1400eb690 con 0x7fd1500a0890 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd13003b9c0 0x7fd13003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.317+0000 7fd14e59c640 10 monclient: get_auth_request con 0x7fd13003b9c0 auth_method 0 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.321+0000 7fd14e59c640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd13003b9c0 0x7fd13003de80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd14400a9a0 tx=0x7fd144008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.325+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd1400c6030 con 0x7fd1500a0890 2024-09-15T21:34:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.325+0000 7fd13effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.325+0000 7fd13effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:42.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd154cd8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:42.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd154cd8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd154cd8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd15009f320 con 0x7fd1500a0890 2024-09-15T21:34:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd13effd640 1 -- 172.21.15.17:0/2850393463 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fd1400165c0 con 0x7fd1500a0890 2024-09-15T21:34:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd13effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:42.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.561+0000 7fd13effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:42.565 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd13003b9c0 msgr2=0x7fd13003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd13003b9c0 0x7fd13003de80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd14400a9a0 tx=0x7fd144008040 comp rx=0 tx=0).stop 2024-09-15T21:34:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 10 monclient: shutdown 2024-09-15T21:34:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 msgr2=0x7fd15009d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 0x7fd15009d400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1400af040 tx=0x7fd140019650 comp rx=0 tx=0).stop 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 shutdown_connections 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd13003b9c0 0x7fd13003de80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd150135670 0x7fd15009d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1500a0890 0x7fd15009d400 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 --2- 172.21.15.17:0/2850393463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd15009ffc0 0x7fd150134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 >> 172.21.15.17:0/2850393463 conn(0x7fd1500a66a0 msgr2=0x7fd1500a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 shutdown_connections 2024-09-15T21:34:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.565+0000 7fd154cd8640 1 -- 172.21.15.17:0/2850393463 wait complete. 2024-09-15T21:34:42.585 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:34:42.585 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:34:42.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient: get_monmap_and_config 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient: build_initial_monmap 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient: monmap: 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:42.715032+0000 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:42.715032+0000 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:42.715 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:42.716 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:42.716 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 Processor -- start 2024-09-15T21:34:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 -- start start 2024-09-15T21:34:42.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient: init 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 0x7f4a44137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): picked mon.noname-b con 0x7f4a441378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a441381e0 0x7f4a440a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): picked mon.noname-a con 0x7f4a441381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:42.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a440a5040 0x7f4a440a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): picked mon.noname-c con 0x7f4a440a5040 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4a440a1d00 con 0x7f4a441381e0 2024-09-15T21:34:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4a440a1fe0 con 0x7f4a440a5040 2024-09-15T21:34:42.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a440a22c0 con 0x7f4a441378c0 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a4b9bd640 10 monclient(hunting): authenticate will time out at 2006.110405s 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 0x7f4a44137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 0x7f4a44137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40044/0 (socket says 172.21.15.17:40044) 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 -- 172.21.15.17:0/1877081146 learned_addr learned my addr 172.21.15.17:0/1877081146 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:42.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): get_auth_request con 0x7f4a441378c0 auth_method 0 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.17:3300/0 conn(0x7f4a441381e0 0x7f4a440a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): get_auth_request con 0x7f4a441381e0 auth_method 0 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49f5a640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.119:3300/0 conn(0x7f4a440a5040 0x7f4a440a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49f5a640 10 monclient(hunting): get_auth_request con 0x7f4a440a5040 auth_method 0 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49f5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49f5a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49f5a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient(hunting): handle_auth_done global_id 164507 payload 274 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 -- 172.21.15.17:0/1877081146 >> v2:172.21.15.119:3300/0 conn(0x7f4a440a5040 msgr2=0x7f4a440a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.119:3300/0 conn(0x7f4a440a5040 0x7f4a440a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 -- 172.21.15.17:0/1877081146 >> v2:172.21.15.17:3300/0 conn(0x7f4a441381e0 msgr2=0x7f4a440a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.17:3300/0 conn(0x7f4a441381e0 0x7f4a440a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient: _finish_hunting 0 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 monclient: found mon.noname-b 2024-09-15T21:34:42.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a49759640 1 -- 172.21.15.17:0/1877081146 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a440a4490 con 0x7f4a441378c0 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.713+0000 7f4a48f58640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.17:3300/0 conn(0x7f4a441381e0 0x7f4a440a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49759640 1 --2- 172.21.15.17:0/1877081146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 0x7f4a44137ca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a6150 tx=0x7f4a4001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3456004f232a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 1 -- 172.21.15.17:0/1877081146 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400a5540 con 0x7f4a441378c0 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: dump: 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: _finish_auth 0 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 1 -- 172.21.15.17:0/1877081146 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a400a56e0 con 0x7f4a441378c0 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 1 -- 172.21.15.17:0/1877081146 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400a5ab0 con 0x7f4a441378c0 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 5 monclient: authenticate success, global_id 164507 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 4 set_mon_vals no callback set 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a3bfff640 10 monclient: dump: 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: shutdown 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/1877081146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 msgr2=0x7f4a44137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/1877081146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 0x7f4a44137ca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a6150 tx=0x7f4a4001c8b0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/1877081146 shutdown_connections 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.119:3300/0 conn(0x7f4a440a5040 0x7f4a440a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/1877081146 >> v2:172.21.15.17:3300/0 conn(0x7f4a441381e0 0x7f4a440a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/1877081146 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441378c0 0x7f4a44137ca0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/1877081146 >> 172.21.15.17:0/1877081146 conn(0x7f4a440aac40 msgr2=0x7f4a440a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/1877081146 shutdown_connections 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/1877081146 wait complete. 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: build_initial_monmap 2024-09-15T21:34:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: monmap: 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 Processor -- start 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- start start 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: init 2024-09-15T21:34:42.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a440a5040 0x7f4a441cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): picked mon.noname-a con 0x7f4a440a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a441378c0 0x7f4a44136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): picked mon.noname-b con 0x7f4a441378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 0x7f4a441370e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): picked mon.noname-c con 0x7f4a441381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44141730 con 0x7f4a440a5040 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441415b0 con 0x7f4a441378c0 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a441418b0 con 0x7f4a441381e0 2024-09-15T21:34:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a4b9bd640 10 monclient(hunting): authenticate will time out at 2006.115708s 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 0x7f4a441370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 0x7f4a441370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40046/0 (socket says 172.21.15.17:40046) 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 1 -- 172.21.15.17:0/3384263017 learned_addr learned my addr 172.21.15.17:0/3384263017 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 10 monclient(hunting): get_auth_request con 0x7f4a441381e0 auth_method 0 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49759640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a440a5040 0x7f4a441cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.717+0000 7f4a49f5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): get_auth_request con 0x7f4a440a5040 auth_method 0 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a441378c0 0x7f4a44136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 10 monclient(hunting): get_auth_request con 0x7f4a441378c0 auth_method 0 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 10 monclient(hunting): handle_auth_done global_id 164510 payload 824 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a441378c0 msgr2=0x7f4a44136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a441378c0 0x7f4a44136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a440a5040 msgr2=0x7f4a441cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a440a5040 0x7f4a441cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 10 monclient: _finish_hunting 0 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 monclient: found mon.noname-c 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a44122560 con 0x7f4a441381e0 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a441378c0 0x7f4a44136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49f5a640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 0x7f4a441370e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00d800 tx=0x7f4a3c00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00b6c0 con 0x7f4a441381e0 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: dump: 2024-09-15T21:34:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3c019070 con 0x7f4a441381e0 2024-09-15T21:34:42.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 5 monclient: authenticate success, global_id 164510 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 10 monclient: _renew_subs 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a44120f90 con 0x7f4a441381e0 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00baa0 con 0x7f4a441381e0 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: dump: 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.742 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a3b7fe640 4 set_mon_vals no callback set 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 10 monclient: _renew_subs 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a44121330 con 0x7f4a441381e0 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 0x7f4a4413d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44137ca0 con 0x7f4a4413cdd0 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4a3c00bc40 con 0x7f4a441381e0 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 0x7f4a4413d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient: get_auth_request con 0x7f4a4413cdd0 auth_method 0 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2803b9f0 0x7f4a2803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4a3c050ae0 con 0x7f4a441381e0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 10 monclient(hunting): handle_auth_done global_id 204111 payload 824 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a4413e840 con 0x7f4a4413cdd0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a49759640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 0x7f4a4413d1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a7b50 tx=0x7f4a40019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2803b9f0 0x7f4a2803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 10 monclient: get_auth_request con 0x7f4a2803b9f0 auth_method 0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40009510 con 0x7f4a4413cdd0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: dump: 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4a400c0440 con 0x7f4a4413cdd0 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:42.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 msgr2=0x7f4a4413d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 0x7f4a4413d1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a7b50 tx=0x7f4a40019650 comp rx=0 tx=0).stop 2024-09-15T21:34:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 msgr2=0x7f4a4413d1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a39ffb640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413cdd0 0x7f4a4413d1b0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a7b50 tx=0x7f4a40019650 comp rx=0 tx=0).stop 2024-09-15T21:34:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.721+0000 7f4a48f58640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2803b9f0 0x7f4a2803deb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a44121ba0 tx=0x7f4a34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:42.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 0x7f4a44138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44069270 con 0x7f4a4413d6f0 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 0x7f4a44138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient: get_auth_request con 0x7f4a4413d6f0 auth_method 0 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 10 monclient(hunting): handle_auth_done global_id 204114 payload 824 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 1 -- 172.21.15.17:0/3384263017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a4413db00 con 0x7f4a4413d6f0 2024-09-15T21:34:42.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49759640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 0x7f4a44138d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001d150 tx=0x7f4a400a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40004250 con 0x7f4a4413d6f0 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 10 monclient: dump: 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.750 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7f4a40004250 con 0x7f4a4413d6f0 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 msgr2=0x7f4a44138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 0x7f4a44138d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001d150 tx=0x7f4a400a7000 comp rx=0 tx=0).stop 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 msgr2=0x7f4a44138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a39ffb640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a4413d6f0 0x7f4a44138d60 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001d150 tx=0x7f4a400a7000 comp rx=0 tx=0).stop 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:42.751 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 158, 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7263, 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:42.753 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:42.754 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:42.755 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:42.756 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:42.757 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.758 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:42.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:42.760 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:42.763 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:42.764 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.764 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.764 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:42.764 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2803b9f0 msgr2=0x7f4a2803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a2803b9f0 0x7f4a2803deb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a44121ba0 tx=0x7f4a34008040 comp rx=0 tx=0).stop 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 10 monclient: shutdown 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 msgr2=0x7f4a441370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 0x7f4a441370e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00d800 tx=0x7f4a3c00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a49f5a640 1 -- 172.21.15.17:0/3384263017 reap_dead start 2024-09-15T21:34:42.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 shutdown_connections 2024-09-15T21:34:42.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 --2- 172.21.15.17:0/3384263017 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a441381e0 0x7f4a441370e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 >> 172.21.15.17:0/3384263017 conn(0x7f4a440aac40 msgr2=0x7f4a4406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:42.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 shutdown_connections 2024-09-15T21:34:42.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.729+0000 7f4a4b9bd640 1 -- 172.21.15.17:0/3384263017 wait complete. 2024-09-15T21:34:42.766 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:34:42.766 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:34:42.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:34:42.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient: get_monmap_and_config 2024-09-15T21:34:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient: build_initial_monmap 2024-09-15T21:34:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient: monmap: 2024-09-15T21:34:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.888 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:42.887823+0000 2024-09-15T21:34:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:42.887823+0000 2024-09-15T21:34:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:42.889 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:42.890 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:42.890 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 Processor -- start 2024-09-15T21:34:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 -- start start 2024-09-15T21:34:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient: init 2024-09-15T21:34:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2024-09-15T21:34:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd1e00a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1e00a5580 0x7fd1e013cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): picked mon.noname-a con 0x7fd1e00a5580 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1e00a59d0 0x7fd1e013f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): picked mon.noname-c con 0x7fd1e00a59d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e00a2c30 con 0x7fd1e00a4a90 2024-09-15T21:34:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00a0f40 con 0x7fd1e00a5580 2024-09-15T21:34:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00a19b0 con 0x7fd1e00a59d0 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.885+0000 7fd1e7938640 10 monclient(hunting): authenticate will time out at 2006.283843s 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1e00a5580 0x7fd1e013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1e00a5580 0x7fd1e013cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40630/0 (socket says 172.21.15.17:40630) 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 -- 172.21.15.17:0/3366704626 learned_addr learned my addr 172.21.15.17:0/3366704626 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): get_auth_request con 0x7fd1e00a5580 auth_method 0 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e5ed5640 1 --2- 172.21.15.17:0/3366704626 >> v2:172.21.15.119:3300/0 conn(0x7fd1e00a59d0 0x7fd1e013f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e5ed5640 10 monclient(hunting): get_auth_request con 0x7fd1e00a59d0 auth_method 0 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e5ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e5ed5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e5ed5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_done global_id 204117 payload 274 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 -- 172.21.15.17:0/3366704626 >> v2:172.21.15.119:3300/0 conn(0x7fd1e00a59d0 msgr2=0x7fd1e013f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/3366704626 >> v2:172.21.15.119:3300/0 conn(0x7fd1e00a59d0 0x7fd1e013f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 -- 172.21.15.17:0/3366704626 >> v1:172.21.15.119:6789/0 conn(0x7fd1e00a4a90 legacy=0x7fd1e00a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient: _finish_hunting 0 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 monclient: found mon.noname-a 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 -- 172.21.15.17:0/3366704626 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e00a1c90 con 0x7fd1e00a5580 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/3366704626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a5580 0x7fd1e013cd10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009870 tx=0x7fd1d001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa5907edece2a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 1 -- 172.21.15.17:0/3366704626 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00b2070 con 0x7fd1e00a5580 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: dump: 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: _finish_auth 0 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 5 monclient: authenticate success, global_id 204117 2024-09-15T21:34:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 1 -- 172.21.15.17:0/3366704626 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d00a9070 con 0x7fd1e00a5580 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 4 set_mon_vals no callback set 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 1 -- 172.21.15.17:0/3366704626 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00ad880 con 0x7fd1e00a5580 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1d7fff640 10 monclient: dump: 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 10 monclient: shutdown 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 -- 172.21.15.17:0/3366704626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a5580 msgr2=0x7fd1e013cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 --2- 172.21.15.17:0/3366704626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a5580 0x7fd1e013cd10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009870 tx=0x7fd1d001c440 comp rx=0 tx=0).stop 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 -- 172.21.15.17:0/3366704626 shutdown_connections 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 --2- 172.21.15.17:0/3366704626 >> v2:172.21.15.119:3300/0 conn(0x7fd1e00a59d0 0x7fd1e013f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 --2- 172.21.15.17:0/3366704626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a5580 0x7fd1e013cd10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 -- 172.21.15.17:0/3366704626 >> 172.21.15.17:0/3366704626 conn(0x7fd1e00aabb0 msgr2=0x7fd1e00a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 -- 172.21.15.17:0/3366704626 shutdown_connections 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.889+0000 7fd1e7938640 1 -- 172.21.15.17:0/3366704626 wait complete. 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: build_initial_monmap 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: monmap: 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 Processor -- start 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- start start 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: init 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a4a90 0x7fd1e01cf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): picked mon.noname-a con 0x7fd1e00a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e00a5580 0x7fd1e01cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): picked mon.noname-b con 0x7fd1e00a5580 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e00a59d0 0x7fd1e01d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): picked mon.noname-c con 0x7fd1e00a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0142140 con 0x7fd1e00a4a90 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0141fc0 con 0x7fd1e00a5580 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e01422c0 con 0x7fd1e00a59d0 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): authenticate will time out at 2006.289258s 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a4a90 0x7fd1e01cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e00a59d0 0x7fd1e01d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a4a90 0x7fd1e01cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40634/0 (socket says 172.21.15.17:40634) 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 -- 172.21.15.17:0/1686379878 learned_addr learned my addr 172.21.15.17:0/1686379878 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): get_auth_request con 0x7fd1e00a4a90 auth_method 0 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): get_auth_request con 0x7fd1e00a59d0 auth_method 0 2024-09-15T21:34:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e00a5580 0x7fd1e01cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): get_auth_request con 0x7fd1e00a5580 auth_method 0 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient(hunting): handle_auth_done global_id 204120 payload 824 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e00a59d0 msgr2=0x7fd1e01d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e00a59d0 0x7fd1e01d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e00a5580 msgr2=0x7fd1e01cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e00a5580 0x7fd1e01cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient: _finish_hunting 0 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 monclient: found mon.noname-a 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e013dc70 con 0x7fd1e00a4a90 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e00a59d0 0x7fd1e01d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e56d4640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a4a90 0x7fd1e01cf640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c530 tx=0x7fd1dc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc017070 con 0x7fd1e00a4a90 2024-09-15T21:34:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: dump: 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:42.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0052d0 con 0x7fd1e00a4a90 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 5 monclient: authenticate success, global_id 204120 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: _renew_subs 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e013f3d0 con 0x7fd1e00a4a90 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc012870 con 0x7fd1e00a4a90 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: dump: 2024-09-15T21:34:42.927 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: _renew_subs 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d77fe640 4 set_mon_vals no callback set 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e013d990 con 0x7fd1e00a4a90 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 0x7fd1e0134c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e013d7f0 con 0x7fd1e0134820 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 0x7fd1e0134c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient: get_auth_request con 0x7fd1e0134820 auth_method 0 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd1dc01a070 con 0x7fd1e00a4a90 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_done global_id 164516 payload 824 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1e00a4e70 con 0x7fd1e0134820 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd1c403b9c0 0x7fd1c403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd1dc04f880 con 0x7fd1e00a4a90 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 0x7fd1e0134c00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00a7900 tx=0x7fd1d001ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd1c403b9c0 0x7fd1c403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00b2040 con 0x7fd1e0134820 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1d5ffb640 10 monclient: dump: 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.893+0000 7fd1e5ed5640 10 monclient: get_auth_request con 0x7fd1c403b9c0 auth_method 0 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd1d0009490 con 0x7fd1e0134820 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 msgr2=0x7fd1e0134c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 0x7fd1e0134c00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00a7900 tx=0x7fd1d001ac80 comp rx=0 tx=0).stop 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 msgr2=0x7fd1e0134c00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1d5ffb640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0134820 0x7fd1e0134c00 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00a7900 tx=0x7fd1d001ac80 comp rx=0 tx=0).stop 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.897+0000 7fd1e5ed5640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd1c403b9c0 0x7fd1c403de80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80077c0 tx=0x7fd1d8009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 0x7fd1e009d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e013c470 con 0x7fd1e0135140 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 0x7fd1e009d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient: get_auth_request con 0x7fd1e0135140 auth_method 0 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 10 monclient(hunting): handle_auth_done global_id 164519 payload 824 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 1 -- 172.21.15.17:0/1686379878 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd1e0069200 con 0x7fd1e0135140 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e4ed3640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 0x7fd1e009d590 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c10 tx=0x7fd1d00191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00c5070 con 0x7fd1e0135140 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 10 monclient: dump: 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7fd1d00c17f0 con 0x7fd1e0135140 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 msgr2=0x7fd1e009d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 0x7fd1e009d590 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c10 tx=0x7fd1d00191e0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 msgr2=0x7fd1e009d590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1d5ffb640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd1e0135140 0x7fd1e009d590 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c10 tx=0x7fd1d00191e0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 158, 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 231485, 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:42.935 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:42.936 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:42.937 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:42.938 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:42.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:42.940 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.941 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:42.942 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:42.943 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:42.947 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:42.948 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.948 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:34:42.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.948 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:42.948 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:42.949 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:34:42.949 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.949 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:42.949 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:42.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:42.949 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:42.950 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd1c403b9c0 msgr2=0x7fd1c403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd1c403b9c0 0x7fd1c403de80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80077c0 tx=0x7fd1d8009820 comp rx=0 tx=0).stop 2024-09-15T21:34:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 10 monclient: shutdown 2024-09-15T21:34:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a4a90 msgr2=0x7fd1e01cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 --2- 172.21.15.17:0/1686379878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00a4a90 0x7fd1e01cf640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c530 tx=0x7fd1dc00c9f0 comp rx=0 tx=0).stop 2024-09-15T21:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e5ed5640 1 -- 172.21.15.17:0/1686379878 reap_dead start 2024-09-15T21:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e5ed5640 1 -- 172.21.15.17:0/1686379878 reap_dead start 2024-09-15T21:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 shutdown_connections 2024-09-15T21:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.901+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 >> 172.21.15.17:0/1686379878 conn(0x7fd1e00aabb0 msgr2=0x7fd1e0137a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.905+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 shutdown_connections 2024-09-15T21:34:42.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:42.905+0000 7fd1e7938640 1 -- 172.21.15.17:0/1686379878 wait complete. 2024-09-15T21:34:42.955 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:34:42.955 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:34:42.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient: get_monmap_and_config 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient: build_initial_monmap 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient: monmap: 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:43.079158+0000 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:43.079158+0000 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:43.080 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 Processor -- start 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 -- start start 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient: init 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faff41378c0 0x7faff4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): picked mon.noname-a con 0x7faff41378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff40a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): picked mon.noname-b con 0x7faff41381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faff40a5040 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff40a1d00 con 0x7faff40a5040 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7faff40a1fe0 con 0x7faff41378c0 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faff40a22c0 con 0x7faff41381e0 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff933b640 10 monclient(hunting): authenticate will time out at 2006.474608s 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faff41378c0 0x7faff4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faff41378c0 0x7faff4137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40638/0 (socket says 172.21.15.17:40638) 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 -- 172.21.15.17:0/409248769 learned_addr learned my addr 172.21.15.17:0/409248769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): get_auth_request con 0x7faff41378c0 auth_method 0 2024-09-15T21:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff40a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): get_auth_request con 0x7faff41381e0 auth_method 0 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient(hunting): handle_auth_done global_id 204123 payload 274 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 -- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 msgr2=0x7faff40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff40a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 -- 172.21.15.17:0/409248769 >> v1:172.21.15.17:6789/0 conn(0x7faff40a5040 legacy=0x7faff40a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 monclient: found mon.noname-a 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff2ffd640 1 -- 172.21.15.17:0/409248769 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff40a5c00 con 0x7faff41378c0 2024-09-15T21:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.077+0000 7faff27fc640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff40a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff2ffd640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff41378c0 0x7faff4137ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafec009e50 tx=0x7fafec01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206ae4ee920cdcfe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 1 -- 172.21.15.17:0/409248769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0a5740 con 0x7faff41378c0 2024-09-15T21:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: dump: 2024-09-15T21:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 1 -- 172.21.15.17:0/409248769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0a58e0 con 0x7faff41378c0 2024-09-15T21:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 1 -- 172.21.15.17:0/409248769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0a5cb0 con 0x7faff41378c0 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 5 monclient: authenticate success, global_id 204123 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 4 set_mon_vals no callback set 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff1ffb640 10 monclient: dump: 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 10 monclient: shutdown 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 -- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff41378c0 msgr2=0x7faff4137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff41378c0 0x7faff4137ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafec009e50 tx=0x7fafec01c8f0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 -- 172.21.15.17:0/409248769 shutdown_connections 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff40a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 --2- 172.21.15.17:0/409248769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff41378c0 0x7faff4137ca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 -- 172.21.15.17:0/409248769 >> 172.21.15.17:0/409248769 conn(0x7faff40aac40 msgr2=0x7faff40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 -- 172.21.15.17:0/409248769 shutdown_connections 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 -- 172.21.15.17:0/409248769 wait complete. 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 10 monclient: build_initial_monmap 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 10 monclient: monmap: 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:43.096 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 Processor -- start 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 1 -- start start 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 10 monclient: init 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.081+0000 7faff933b640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff40a5040 0x7faff41cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): picked mon.noname-a con 0x7faff40a5040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff41378c0 0x7faff4136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): picked mon.noname-b con 0x7faff41378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): picked mon.noname-c con 0x7faff41381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faff406f340 con 0x7faff40a5040 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faff406f1c0 con 0x7faff41378c0 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7faff41425c0 con 0x7faff41381e0 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): authenticate will time out at 2006.480079s 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff40a5040 0x7faff41cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff40a5040 0x7faff41cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40640/0 (socket says 172.21.15.17:40640) 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 -- 172.21.15.17:0/3059163040 learned_addr learned my addr 172.21.15.17:0/3059163040 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): get_auth_request con 0x7faff40a5040 auth_method 0 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): get_auth_request con 0x7faff41381e0 auth_method 0 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient(hunting): handle_auth_done global_id 204129 payload 824 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 msgr2=0x7faff413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7faff41381e0 0x7faff413cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff41378c0 0x7faff4136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff41378c0 msgr2=0x7faff4136a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff41378c0 0x7faff4136a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 monclient: found mon.noname-a 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff4120dd0 con 0x7faff40a5040 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient: get_auth_request con 0x7faff41378c0 auth_method 0 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff41378c0 0x7faff4136a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff2ffd640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff40a5040 0x7faff41cf7f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fafec01a520 tx=0x7fafec019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0bc020 con 0x7faff40a5040 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: dump: 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0b8670 con 0x7faff40a5040 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0b8a40 con 0x7faff40a5040 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 5 monclient: authenticate success, global_id 204129 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient: _renew_subs 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff41210b0 con 0x7faff40a5040 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff17fa640 4 set_mon_vals no callback set 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: dump: 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient: _renew_subs 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff40a8a50 con 0x7faff40a5040 2024-09-15T21:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 0x7faff413d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faff4137ca0 con 0x7faff4136f40 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fafec0bf020 con 0x7faff40a5040 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafd403b9c0 0x7fafd403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 0x7faff413d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fafec0eb4d0 con 0x7faff40a5040 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient: get_auth_request con 0x7faff4136f40 auth_method 0 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafd403b9c0 0x7fafd403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 10 monclient: get_auth_request con 0x7fafd403b9c0 auth_method 0 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 10 monclient(hunting): handle_auth_done global_id 194212 payload 824 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faff41212f0 con 0x7faff4136f40 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff27fc640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 0x7faff413d100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faff40a87c0 tx=0x7fafdc010c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7faff37fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafd403b9c0 0x7fafd403de80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faff413f050 tx=0x7fafe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc00f040 con 0x7faff4136f40 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.085+0000 7fafeb7fe640 10 monclient: dump: 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fafdc010e70 con 0x7faff4136f40 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 msgr2=0x7faff413d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 0x7faff413d100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faff40a87c0 tx=0x7fafdc010c20 comp rx=0 tx=0).stop 2024-09-15T21:34:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 msgr2=0x7faff413d100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.089+0000 7fafeb7fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff4136f40 0x7faff413d100 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faff40a87c0 tx=0x7fafdc010c20 comp rx=0 tx=0).stop 2024-09-15T21:34:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff933b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff933b640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 0x7faff4138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff933b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faff413da50 con 0x7faff413d640 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 0x7faff4138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient: get_auth_request con 0x7faff413d640 auth_method 0 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 10 monclient(hunting): handle_auth_done global_id 194215 payload 824 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 1 -- 172.21.15.17:0/3059163040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faff41d0670 con 0x7faff413d640 2024-09-15T21:34:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7faff27fc640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 0x7faff4138d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafdc008010 tx=0x7fafdc0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc006450 con 0x7faff413d640 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7fafeb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7fafeb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.093+0000 7fafeb7fe640 10 monclient: dump: 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7fafdc006450 con 0x7faff413d640 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 msgr2=0x7faff4138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 0x7faff4138d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafdc008010 tx=0x7fafdc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 msgr2=0x7faff4138d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7fafeb7fe640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff413d640 0x7faff4138d60 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafdc008010 tx=0x7fafdc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 158, 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 49882, 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:43.113 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:43.114 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:43.115 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:43.116 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:43.117 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:43.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:43.119 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:43.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.123 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:43.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:43.126 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafd403b9c0 msgr2=0x7fafd403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fafd403b9c0 0x7fafd403de80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faff413f050 tx=0x7fafe4009040 comp rx=0 tx=0).stop 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 10 monclient: shutdown 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff40a5040 msgr2=0x7faff41cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 --2- 172.21.15.17:0/3059163040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff40a5040 0x7faff41cf7f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fafec01a520 tx=0x7fafec019690 comp rx=0 tx=0).stop 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff37fe640 1 -- 172.21.15.17:0/3059163040 reap_dead start 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff37fe640 1 -- 172.21.15.17:0/3059163040 reap_dead start 2024-09-15T21:34:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 shutdown_connections 2024-09-15T21:34:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 >> 172.21.15.17:0/3059163040 conn(0x7faff40aac40 msgr2=0x7faff406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 shutdown_connections 2024-09-15T21:34:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.097+0000 7faff933b640 1 -- 172.21.15.17:0/3059163040 wait complete. 2024-09-15T21:34:43.129 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:34:43.129 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:34:43.129 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.249+0000 7ffbb6a6b640 10 monclient: get_monmap_and_config 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.249+0000 7ffbb6a6b640 10 monclient: build_initial_monmap 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.249+0000 7ffbb6a6b640 10 monclient: monmap: 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:43.252990+0000 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:43.252990+0000 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.254 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:43.254 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:43.254 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:43.254 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:43.254 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:43.254 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.249+0000 7ffbb6a6b640 1 Processor -- start 2024-09-15T21:34:43.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.249+0000 7ffbb6a6b640 1 -- start start 2024-09-15T21:34:43.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.249+0000 7ffbb6a6b640 10 monclient: init 2024-09-15T21:34:43.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:43.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2024-09-15T21:34:43.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbb00a0890 0x7ffbb00a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): picked mon.noname-c con 0x7ffbb00a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffbb009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 0x7ffbb0135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): picked mon.noname-b con 0x7ffbb00a11b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb00a3130 con 0x7ffbb009ffc0 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb0073df0 con 0x7ffbb00a0890 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb00a1bf0 con 0x7ffbb00a11b0 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient(hunting): authenticate will time out at 2006.648431s 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 0x7ffbb0135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 0x7ffbb0135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40098/0 (socket says 172.21.15.17:40098) 2024-09-15T21:34:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 -- 172.21.15.17:0/416188170 learned_addr learned my addr 172.21.15.17:0/416188170 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): get_auth_request con 0x7ffbb00a11b0 auth_method 0 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb4807640 1 --2- 172.21.15.17:0/416188170 >> v2:172.21.15.119:3300/0 conn(0x7ffbb00a0890 0x7ffbb00a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb4807640 10 monclient(hunting): get_auth_request con 0x7ffbb00a0890 auth_method 0 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb4807640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb4807640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb4807640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient(hunting): handle_auth_done global_id 164528 payload 274 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 -- 172.21.15.17:0/416188170 >> v2:172.21.15.119:3300/0 conn(0x7ffbb00a0890 msgr2=0x7ffbb00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 --2- 172.21.15.17:0/416188170 >> v2:172.21.15.119:3300/0 conn(0x7ffbb00a0890 0x7ffbb00a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 -- 172.21.15.17:0/416188170 >> v1:172.21.15.119:6789/0 conn(0x7ffbb009ffc0 legacy=0x7ffbb0135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient: _finish_hunting 0 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 monclient: found mon.noname-b 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 -- 172.21.15.17:0/416188170 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb00a15e0 con 0x7ffbb00a11b0 2024-09-15T21:34:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb5008640 1 --2- 172.21.15.17:0/416188170 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 0x7ffbb0135d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098a0 tx=0x7ffba001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e191173072d1fd62 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 1 -- 172.21.15.17:0/416188170 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00b0070 con 0x7ffbb00a11b0 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: dump: 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.265 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 1 -- 172.21.15.17:0/416188170 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba00ab600 con 0x7ffbb00a11b0 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 1 -- 172.21.15.17:0/416188170 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00ab9d0 con 0x7ffbb00a11b0 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 5 monclient: authenticate success, global_id 164528 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbb6a6b640 4 set_mon_vals no callback set 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.253+0000 7ffbaf7fe640 10 monclient: dump: 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: shutdown 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/416188170 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 msgr2=0x7ffbb0135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/416188170 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 0x7ffbb0135d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098a0 tx=0x7ffba001c400 comp rx=0 tx=0).stop 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/416188170 shutdown_connections 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/416188170 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a11b0 0x7ffbb0135d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/416188170 >> v2:172.21.15.119:3300/0 conn(0x7ffbb00a0890 0x7ffbb00a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/416188170 >> 172.21.15.17:0/416188170 conn(0x7ffbb00a66a0 msgr2=0x7ffbb00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/416188170 shutdown_connections 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/416188170 wait complete. 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: build_initial_monmap 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: monmap: 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 Processor -- start 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- start start 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: init 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:34:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb009ffc0 0x7ffbb011e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): picked mon.noname-b con 0x7ffbb009ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 0x7ffbb0135170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): picked mon.noname-c con 0x7ffbb00a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 0x7ffbb011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): picked mon.noname-a con 0x7ffbb00a11b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0140510 con 0x7ffbb00a11b0 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb01364d0 con 0x7ffbb009ffc0 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb0140690 con 0x7ffbb00a0890 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient(hunting): authenticate will time out at 2006.654742s 2024-09-15T21:34:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 0x7ffbb0135170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 0x7ffbb011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 0x7ffbb0135170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40114/0 (socket says 172.21.15.17:40114) 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 0x7ffbb011c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40648/0 (socket says 172.21.15.17:40648) 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 -- 172.21.15.17:0/3716345102 learned_addr learned my addr 172.21.15.17:0/3716345102 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): get_auth_request con 0x7ffbb00a0890 auth_method 0 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): get_auth_request con 0x7ffbb00a11b0 auth_method 0 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb4807640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb009ffc0 0x7ffbb011e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb4807640 10 monclient(hunting): get_auth_request con 0x7ffbb009ffc0 auth_method 0 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb4807640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb4807640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb4807640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient(hunting): handle_auth_done global_id 164531 payload 824 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 -- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb009ffc0 msgr2=0x7ffbb011e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb009ffc0 0x7ffbb011e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 -- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 msgr2=0x7ffbb011c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 0x7ffbb011c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient: _finish_hunting 0 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 monclient: found mon.noname-c 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 -- 172.21.15.17:0/3716345102 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb01ce3b0 con 0x7ffbb00a0890 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb4807640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb009ffc0 0x7ffbb011e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb5008640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 0x7ffbb011c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaffff640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 0x7ffbb0135170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800c7f0 tx=0x7ffb9800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98016020 con 0x7ffbb00a0890 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: dump: 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.275 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: _finish_auth 0 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 5 monclient: authenticate success, global_id 164531 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: _renew_subs 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9800aa50 con 0x7ffbb00a0890 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb009d0f0 con 0x7ffbb00a0890 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800ae40 con 0x7ffbb00a0890 2024-09-15T21:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbad7fa640 10 monclient: dump: 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbaeffd640 4 set_mon_vals no callback set 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: _renew_subs 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.257+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb011f820 con 0x7ffbb00a0890 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb6a6b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb6a6b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb6a6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ffb98019020 con 0x7ffbb00a0890 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb00a0c70 con 0x7ffbb00a0890 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbad7fa640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb9003b9c0 0x7ffb9003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ffb98050590 con 0x7ffbb00a0890 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb4807640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb9003b9c0 0x7ffb9003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb4807640 10 monclient: get_auth_request con 0x7ffb9003b9c0 auth_method 0 2024-09-15T21:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.261+0000 7ffbb4807640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb9003b9c0 0x7ffb9003de80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ffba8007a70 tx=0x7ffba8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.265+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ffb98020030 con 0x7ffbb00a0890 2024-09-15T21:34:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.265+0000 7ffbad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.265+0000 7ffbad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.497+0000 7ffbb6a6b640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:34:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.497+0000 7ffbb6a6b640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:34:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.497+0000 7ffbb6a6b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.497+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ffbb011fa60 con 0x7ffbb00a0890 2024-09-15T21:34:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.501+0000 7ffbad7fa640 1 -- 172.21.15.17:0/3716345102 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7ffb98010cf0 con 0x7ffbb00a0890 2024-09-15T21:34:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.501+0000 7ffbad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:34:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.501+0000 7ffbad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:43.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.501+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.501+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=11,kv=11,logm=47} crc {auth=2701047679,config=3541143443,health=426306039,kv=2762157375,logm=2193716070}) 2024-09-15T21:34:43.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.501+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=11,kv=11,logm=47} crc {auth=60585595,config=3541143443,health=3138022320,kv=1867416868,logm=542172053}) 2024-09-15T21:34:43.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.501+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.501+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=11,kv=11,logm=47} crc {auth=2701047679,config=3541143443,health=426306039,kv=2762157375,logm=2193716070}) 2024-09-15T21:34:43.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.501+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=30,config=2,health=11,kv=12,logm=45} crc {auth=524007357,config=3541143443,health=1933779806,kv=3079713637,logm=2793199935}) 2024-09-15T21:34:43.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb9003b9c0 msgr2=0x7ffb9003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb9003b9c0 0x7ffb9003de80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ffba8007a70 tx=0x7ffba8008040 comp rx=0 tx=0).stop 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 10 monclient: shutdown 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 msgr2=0x7ffbb0135170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 0x7ffbb0135170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800c7f0 tx=0x7ffb9800ccb0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 shutdown_connections 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ffb9003b9c0 0x7ffb9003de80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb00a11b0 0x7ffbb011c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ffbb00a0890 0x7ffbb0135170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 --2- 172.21.15.17:0/3716345102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb009ffc0 0x7ffbb011e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 >> 172.21.15.17:0/3716345102 conn(0x7ffbb00a66a0 msgr2=0x7ffbb0072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 shutdown_connections 2024-09-15T21:34:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:43.505+0000 7ffbb6a6b640 1 -- 172.21.15.17:0/3716345102 wait complete. 2024-09-15T21:34:43.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.505+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.505+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827820082}) 2024-09-15T21:34:43.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.505+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2704949668}) 2024-09-15T21:34:43.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.505+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.505+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827820082}) 2024-09-15T21:34:43.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.505+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3774173701}) 2024-09-15T21:34:43.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1500359750}) 2024-09-15T21:34:43.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2001560177}) 2024-09-15T21:34:43.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1500359750}) 2024-09-15T21:34:43.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1731737542}) 2024-09-15T21:34:43.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1511359251}) 2024-09-15T21:34:43.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=95659049}) 2024-09-15T21:34:43.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1511359251}) 2024-09-15T21:34:43.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.509+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3814101390}) 2024-09-15T21:34:43.515 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.513+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.513+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1868845391}) 2024-09-15T21:34:43.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.513+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=370356691}) 2024-09-15T21:34:43.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.513+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.513+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1868845391}) 2024-09-15T21:34:43.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.513+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=387015106}) 2024-09-15T21:34:43.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328767380}) 2024-09-15T21:34:43.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=335485856}) 2024-09-15T21:34:43.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328767380}) 2024-09-15T21:34:43.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1548487839}) 2024-09-15T21:34:43.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991028470}) 2024-09-15T21:34:43.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1758544913}) 2024-09-15T21:34:43.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991028470}) 2024-09-15T21:34:43.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.517+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2192712623}) 2024-09-15T21:34:43.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586652656}) 2024-09-15T21:34:43.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4216529139}) 2024-09-15T21:34:43.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586652656}) 2024-09-15T21:34:43.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1881085457}) 2024-09-15T21:34:43.524 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:34:43.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993954849}) 2024-09-15T21:34:43.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3237379335}) 2024-09-15T21:34:43.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993954849}) 2024-09-15T21:34:43.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.521+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2488379686}) 2024-09-15T21:34:43.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131692631}) 2024-09-15T21:34:43.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=239927596}) 2024-09-15T21:34:43.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131692631}) 2024-09-15T21:34:43.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1981805018}) 2024-09-15T21:34:43.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366467337}) 2024-09-15T21:34:43.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2868344145}) 2024-09-15T21:34:43.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.525+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366467337}) 2024-09-15T21:34:43.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2598572886}) 2024-09-15T21:34:43.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2073872080}) 2024-09-15T21:34:43.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2154094058}) 2024-09-15T21:34:43.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.533 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2073872080}) 2024-09-15T21:34:43.533 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.529+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=966241878}) 2024-09-15T21:34:43.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4045498973}) 2024-09-15T21:34:43.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3368143770}) 2024-09-15T21:34:43.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4045498973}) 2024-09-15T21:34:43.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=323732203}) 2024-09-15T21:34:43.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1953057769}) 2024-09-15T21:34:43.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1055365931}) 2024-09-15T21:34:43.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1953057769}) 2024-09-15T21:34:43.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.533+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3506656055}) 2024-09-15T21:34:43.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100716514}) 2024-09-15T21:34:43.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907518022}) 2024-09-15T21:34:43.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100716514}) 2024-09-15T21:34:43.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835429991}) 2024-09-15T21:34:43.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3022054193}) 2024-09-15T21:34:43.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452742043}) 2024-09-15T21:34:43.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3022054193}) 2024-09-15T21:34:43.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.537+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4008897982}) 2024-09-15T21:34:43.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3043303786}) 2024-09-15T21:34:43.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=60386794}) 2024-09-15T21:34:43.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3043303786}) 2024-09-15T21:34:43.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=509853232}) 2024-09-15T21:34:43.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=399126288}) 2024-09-15T21:34:43.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.541+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=988924505}) 2024-09-15T21:34:43.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=399126288}) 2024-09-15T21:34:43.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858570172}) 2024-09-15T21:34:43.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127988433}) 2024-09-15T21:34:43.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2255073995}) 2024-09-15T21:34:43.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127988433}) 2024-09-15T21:34:43.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.545+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2163066909}) 2024-09-15T21:34:43.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1880486483}) 2024-09-15T21:34:43.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3956251390}) 2024-09-15T21:34:43.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1880486483}) 2024-09-15T21:34:43.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=485441291}) 2024-09-15T21:34:43.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4164442018}) 2024-09-15T21:34:43.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2766353251}) 2024-09-15T21:34:43.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4164442018}) 2024-09-15T21:34:43.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.549+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=804363092}) 2024-09-15T21:34:43.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=708882346}) 2024-09-15T21:34:43.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=327510404}) 2024-09-15T21:34:43.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=708882346}) 2024-09-15T21:34:43.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1854749145}) 2024-09-15T21:34:43.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2219817707}) 2024-09-15T21:34:43.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1394315210}) 2024-09-15T21:34:43.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2219817707}) 2024-09-15T21:34:43.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.553+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3495380651}) 2024-09-15T21:34:43.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=837442728}) 2024-09-15T21:34:43.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1187561225}) 2024-09-15T21:34:43.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=837442728}) 2024-09-15T21:34:43.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2741938181}) 2024-09-15T21:34:43.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157646106}) 2024-09-15T21:34:43.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3022758894}) 2024-09-15T21:34:43.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157646106}) 2024-09-15T21:34:43.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.557+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1821717224}) 2024-09-15T21:34:43.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3021315120}) 2024-09-15T21:34:43.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3579668032}) 2024-09-15T21:34:43.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3021315120}) 2024-09-15T21:34:43.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1758442336}) 2024-09-15T21:34:43.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3615569457}) 2024-09-15T21:34:43.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2108736435}) 2024-09-15T21:34:43.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.561+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3615569457}) 2024-09-15T21:34:43.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2145601747}) 2024-09-15T21:34:43.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2904525634}) 2024-09-15T21:34:43.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2955856062}) 2024-09-15T21:34:43.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2904525634}) 2024-09-15T21:34:43.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.565+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3587733070}) 2024-09-15T21:34:43.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3440647540}) 2024-09-15T21:34:43.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4030083222}) 2024-09-15T21:34:43.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3440647540}) 2024-09-15T21:34:43.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=747920030}) 2024-09-15T21:34:43.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153591698}) 2024-09-15T21:34:43.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=908719637}) 2024-09-15T21:34:43.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.573 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153591698}) 2024-09-15T21:34:43.573 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.569+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1958579323}) 2024-09-15T21:34:43.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1496713301}) 2024-09-15T21:34:43.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=352021830}) 2024-09-15T21:34:43.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.575 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1496713301}) 2024-09-15T21:34:43.575 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=38752230}) 2024-09-15T21:34:43.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493316815}) 2024-09-15T21:34:43.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2499382248}) 2024-09-15T21:34:43.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.577 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493316815}) 2024-09-15T21:34:43.577 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.573+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=267774524}) 2024-09-15T21:34:43.579 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.579 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=893532213}) 2024-09-15T21:34:43.579 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=678946839}) 2024-09-15T21:34:43.579 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.579 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=893532213}) 2024-09-15T21:34:43.579 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1653577467}) 2024-09-15T21:34:43.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2815276447}) 2024-09-15T21:34:43.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4270909422}) 2024-09-15T21:34:43.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2815276447}) 2024-09-15T21:34:43.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.577+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2773990476}) 2024-09-15T21:34:43.583 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.583 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2362599962}) 2024-09-15T21:34:43.583 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1230148490}) 2024-09-15T21:34:43.583 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.583 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2362599962}) 2024-09-15T21:34:43.583 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1441587534}) 2024-09-15T21:34:43.585 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2347537644}) 2024-09-15T21:34:43.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2295530067}) 2024-09-15T21:34:43.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2347537644}) 2024-09-15T21:34:43.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.581+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1580295872}) 2024-09-15T21:34:43.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.585+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.585+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3718035267}) 2024-09-15T21:34:43.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.585+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1343183227}) 2024-09-15T21:34:43.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.585+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.585+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3718035267}) 2024-09-15T21:34:43.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.585+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3081641278}) 2024-09-15T21:34:43.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414588583}) 2024-09-15T21:34:43.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=54627258}) 2024-09-15T21:34:43.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414588583}) 2024-09-15T21:34:43.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3353938483}) 2024-09-15T21:34:43.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998542956}) 2024-09-15T21:34:43.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2068312868}) 2024-09-15T21:34:43.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998542956}) 2024-09-15T21:34:43.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.589+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3690640860}) 2024-09-15T21:34:43.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335270731}) 2024-09-15T21:34:43.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=566010178}) 2024-09-15T21:34:43.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335270731}) 2024-09-15T21:34:43.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1528017732}) 2024-09-15T21:34:43.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3582784021}) 2024-09-15T21:34:43.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2358991945}) 2024-09-15T21:34:43.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3582784021}) 2024-09-15T21:34:43.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.593+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2474890528}) 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=109817890}) 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=809247868}) 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=109817890}) 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2135107420}) 2024-09-15T21:34:43.601 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614714570}) 2024-09-15T21:34:43.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3328488963}) 2024-09-15T21:34:43.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614714570}) 2024-09-15T21:34:43.602 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.597+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=448841115}) 2024-09-15T21:34:43.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3267118830}) 2024-09-15T21:34:43.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632310512}) 2024-09-15T21:34:43.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.603 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3267118830}) 2024-09-15T21:34:43.604 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=621352085}) 2024-09-15T21:34:43.605 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2347161006}) 2024-09-15T21:34:43.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=672360562}) 2024-09-15T21:34:43.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2347161006}) 2024-09-15T21:34:43.606 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.601+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3783794491}) 2024-09-15T21:34:43.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=810082027}) 2024-09-15T21:34:43.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4277825949}) 2024-09-15T21:34:43.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.608 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=810082027}) 2024-09-15T21:34:43.608 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207519694}) 2024-09-15T21:34:43.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863515468}) 2024-09-15T21:34:43.609 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3417974845}) 2024-09-15T21:34:43.610 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.610 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863515468}) 2024-09-15T21:34:43.610 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.605+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1100319739}) 2024-09-15T21:34:43.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493403313}) 2024-09-15T21:34:43.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=739471177}) 2024-09-15T21:34:43.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493403313}) 2024-09-15T21:34:43.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1685361765}) 2024-09-15T21:34:43.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123982103}) 2024-09-15T21:34:43.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.609+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=329796358}) 2024-09-15T21:34:43.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123982103}) 2024-09-15T21:34:43.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3007167056}) 2024-09-15T21:34:43.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1816465804}) 2024-09-15T21:34:43.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3248386414}) 2024-09-15T21:34:43.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.616 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1816465804}) 2024-09-15T21:34:43.617 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.613+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1077174559}) 2024-09-15T21:34:43.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133061861}) 2024-09-15T21:34:43.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1594597179}) 2024-09-15T21:34:43.618 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.619 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133061861}) 2024-09-15T21:34:43.619 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1885506146}) 2024-09-15T21:34:43.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397673272}) 2024-09-15T21:34:43.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1067480833}) 2024-09-15T21:34:43.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397673272}) 2024-09-15T21:34:43.620 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.617+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4031572433}) 2024-09-15T21:34:43.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3748397310}) 2024-09-15T21:34:43.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1435465657}) 2024-09-15T21:34:43.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3748397310}) 2024-09-15T21:34:43.622 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721221753}) 2024-09-15T21:34:43.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3598723194}) 2024-09-15T21:34:43.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3382932504}) 2024-09-15T21:34:43.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.624 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3598723194}) 2024-09-15T21:34:43.625 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.621+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=981287904}) 2024-09-15T21:34:43.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=806082584}) 2024-09-15T21:34:43.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=866252794}) 2024-09-15T21:34:43.626 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.627 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=806082584}) 2024-09-15T21:34:43.627 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129558997}) 2024-09-15T21:34:43.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.628 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479515156}) 2024-09-15T21:34:43.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4013964159}) 2024-09-15T21:34:43.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479515156}) 2024-09-15T21:34:43.629 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.625+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4049942197}) 2024-09-15T21:34:43.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440079845}) 2024-09-15T21:34:43.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=940569916}) 2024-09-15T21:34:43.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440079845}) 2024-09-15T21:34:43.631 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1976668989}) 2024-09-15T21:34:43.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3715660872}) 2024-09-15T21:34:43.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3591291894}) 2024-09-15T21:34:43.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3715660872}) 2024-09-15T21:34:43.633 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.629+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1566921114}) 2024-09-15T21:34:43.634 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.635 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2077535651}) 2024-09-15T21:34:43.635 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2923125653}) 2024-09-15T21:34:43.635 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.635 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2077535651}) 2024-09-15T21:34:43.635 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2918760154}) 2024-09-15T21:34:43.637 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.637 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=427872407}) 2024-09-15T21:34:43.637 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1601142880}) 2024-09-15T21:34:43.637 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.637 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=427872407}) 2024-09-15T21:34:43.637 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.633+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2624635729}) 2024-09-15T21:34:43.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1494608882}) 2024-09-15T21:34:43.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4262961495}) 2024-09-15T21:34:43.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1494608882}) 2024-09-15T21:34:43.639 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4166305828}) 2024-09-15T21:34:43.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2307457160}) 2024-09-15T21:34:43.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=339666058}) 2024-09-15T21:34:43.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2307457160}) 2024-09-15T21:34:43.641 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.637+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3509552275}) 2024-09-15T21:34:43.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=509346737}) 2024-09-15T21:34:43.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1047349876}) 2024-09-15T21:34:43.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=509346737}) 2024-09-15T21:34:43.643 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=144167186}) 2024-09-15T21:34:43.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=887913271}) 2024-09-15T21:34:43.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4285954342}) 2024-09-15T21:34:43.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=887913271}) 2024-09-15T21:34:43.645 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.641+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2000271262}) 2024-09-15T21:34:43.647 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.647 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1470663748}) 2024-09-15T21:34:43.647 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2645588793}) 2024-09-15T21:34:43.647 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1470663748}) 2024-09-15T21:34:43.648 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1328312222}) 2024-09-15T21:34:43.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775579794}) 2024-09-15T21:34:43.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4134449673}) 2024-09-15T21:34:43.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775579794}) 2024-09-15T21:34:43.649 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.645+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1201343974}) 2024-09-15T21:34:43.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2074079688}) 2024-09-15T21:34:43.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1630409865}) 2024-09-15T21:34:43.651 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2074079688}) 2024-09-15T21:34:43.652 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3046117490}) 2024-09-15T21:34:43.653 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.653 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3640962379}) 2024-09-15T21:34:43.653 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2977697545}) 2024-09-15T21:34:43.653 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.653 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3640962379}) 2024-09-15T21:34:43.653 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.649+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3481574877}) 2024-09-15T21:34:43.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2165035799}) 2024-09-15T21:34:43.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=459110511}) 2024-09-15T21:34:43.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2165035799}) 2024-09-15T21:34:43.655 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1892826758}) 2024-09-15T21:34:43.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423793412}) 2024-09-15T21:34:43.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3137518326}) 2024-09-15T21:34:43.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423793412}) 2024-09-15T21:34:43.657 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.653+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=5355862}) 2024-09-15T21:34:43.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971426281}) 2024-09-15T21:34:43.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2199511976}) 2024-09-15T21:34:43.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971426281}) 2024-09-15T21:34:43.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3532773050}) 2024-09-15T21:34:43.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2904986503}) 2024-09-15T21:34:43.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3051885693}) 2024-09-15T21:34:43.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2904986503}) 2024-09-15T21:34:43.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.657+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3050210908}) 2024-09-15T21:34:43.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2475437263}) 2024-09-15T21:34:43.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=24372840}) 2024-09-15T21:34:43.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2475437263}) 2024-09-15T21:34:43.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3733863726}) 2024-09-15T21:34:43.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278216882}) 2024-09-15T21:34:43.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=576753288}) 2024-09-15T21:34:43.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278216882}) 2024-09-15T21:34:43.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.661+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1196772385}) 2024-09-15T21:34:43.666 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=621546130}) 2024-09-15T21:34:43.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=178840218}) 2024-09-15T21:34:43.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=621546130}) 2024-09-15T21:34:43.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2670420918}) 2024-09-15T21:34:43.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730720353}) 2024-09-15T21:34:43.668 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246799035}) 2024-09-15T21:34:43.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730720353}) 2024-09-15T21:34:43.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.665+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=680989408}) 2024-09-15T21:34:43.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4041288246}) 2024-09-15T21:34:43.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3562778515}) 2024-09-15T21:34:43.670 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4041288246}) 2024-09-15T21:34:43.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1054319813}) 2024-09-15T21:34:43.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1632345871}) 2024-09-15T21:34:43.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2861841848}) 2024-09-15T21:34:43.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1632345871}) 2024-09-15T21:34:43.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.669+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=861275110}) 2024-09-15T21:34:43.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1705740722}) 2024-09-15T21:34:43.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2790104743}) 2024-09-15T21:34:43.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1705740722}) 2024-09-15T21:34:43.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=645195140}) 2024-09-15T21:34:43.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2914573360}) 2024-09-15T21:34:43.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2501583794}) 2024-09-15T21:34:43.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2914573360}) 2024-09-15T21:34:43.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1412786769}) 2024-09-15T21:34:43.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075695023}) 2024-09-15T21:34:43.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3552324492}) 2024-09-15T21:34:43.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075695023}) 2024-09-15T21:34:43.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.673+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2219986218}) 2024-09-15T21:34:43.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297215356}) 2024-09-15T21:34:43.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3825439779}) 2024-09-15T21:34:43.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297215356}) 2024-09-15T21:34:43.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3678612682}) 2024-09-15T21:34:43.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222861542}) 2024-09-15T21:34:43.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=33098044}) 2024-09-15T21:34:43.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222861542}) 2024-09-15T21:34:43.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.677+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3902410187}) 2024-09-15T21:34:43.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3201980000}) 2024-09-15T21:34:43.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=218939041}) 2024-09-15T21:34:43.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3201980000}) 2024-09-15T21:34:43.684 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=552886057}) 2024-09-15T21:34:43.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1883484076}) 2024-09-15T21:34:43.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=564920246}) 2024-09-15T21:34:43.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1883484076}) 2024-09-15T21:34:43.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.681+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4111858192}) 2024-09-15T21:34:43.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564285161}) 2024-09-15T21:34:43.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1047944265}) 2024-09-15T21:34:43.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.687 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564285161}) 2024-09-15T21:34:43.688 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2829815897}) 2024-09-15T21:34:43.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2239881477}) 2024-09-15T21:34:43.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2150220895}) 2024-09-15T21:34:43.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2239881477}) 2024-09-15T21:34:43.689 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.685+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4234444546}) 2024-09-15T21:34:43.690 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351181645}) 2024-09-15T21:34:43.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2257716469}) 2024-09-15T21:34:43.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351181645}) 2024-09-15T21:34:43.691 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1658890221}) 2024-09-15T21:34:43.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2085270740}) 2024-09-15T21:34:43.692 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2399590988}) 2024-09-15T21:34:43.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2085270740}) 2024-09-15T21:34:43.693 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.689+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1616952324}) 2024-09-15T21:34:43.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1665544974}) 2024-09-15T21:34:43.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457372741}) 2024-09-15T21:34:43.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.694 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1665544974}) 2024-09-15T21:34:43.695 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757078021}) 2024-09-15T21:34:43.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3951054887}) 2024-09-15T21:34:43.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3730977323}) 2024-09-15T21:34:43.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3951054887}) 2024-09-15T21:34:43.696 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.693+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=610406785}) 2024-09-15T21:34:43.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2170642125}) 2024-09-15T21:34:43.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3136131000}) 2024-09-15T21:34:43.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2170642125}) 2024-09-15T21:34:43.698 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2601028617}) 2024-09-15T21:34:43.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186661325}) 2024-09-15T21:34:43.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3803055211}) 2024-09-15T21:34:43.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186661325}) 2024-09-15T21:34:43.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.697+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3940562311}) 2024-09-15T21:34:43.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543266936}) 2024-09-15T21:34:43.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1586937232}) 2024-09-15T21:34:43.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543266936}) 2024-09-15T21:34:43.702 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2456825793}) 2024-09-15T21:34:43.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.704 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3021415409}) 2024-09-15T21:34:43.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4001116816}) 2024-09-15T21:34:43.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3021415409}) 2024-09-15T21:34:43.705 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.701+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1841209985}) 2024-09-15T21:34:43.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=6601425}) 2024-09-15T21:34:43.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=17157962}) 2024-09-15T21:34:43.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=6601425}) 2024-09-15T21:34:43.707 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1666356087}) 2024-09-15T21:34:43.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3428829990}) 2024-09-15T21:34:43.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=851782159}) 2024-09-15T21:34:43.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3428829990}) 2024-09-15T21:34:43.709 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.705+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717591304}) 2024-09-15T21:34:43.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2270814671}) 2024-09-15T21:34:43.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4004491573}) 2024-09-15T21:34:43.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2270814671}) 2024-09-15T21:34:43.711 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713066942}) 2024-09-15T21:34:43.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=734110325}) 2024-09-15T21:34:43.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1372285972}) 2024-09-15T21:34:43.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=734110325}) 2024-09-15T21:34:43.714 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.709+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2901430778}) 2024-09-15T21:34:43.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=216957931}) 2024-09-15T21:34:43.715 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255954125}) 2024-09-15T21:34:43.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=216957931}) 2024-09-15T21:34:43.716 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=533834109}) 2024-09-15T21:34:43.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1533427362}) 2024-09-15T21:34:43.717 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3681799801}) 2024-09-15T21:34:43.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1533427362}) 2024-09-15T21:34:43.718 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.713+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=59867857}) 2024-09-15T21:34:43.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.717+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.717+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7} crc {logm=1376114169,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3421867097,mon_config_key=3516532442}) 2024-09-15T21:34:43.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.717+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8} crc {logm=3983647252,mdsmap=173658045,mgr=3162623449,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1885949827,mon_config_key=3829959883}) 2024-09-15T21:34:43.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.717+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.717+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7} crc {logm=1376114169,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3421867097,mon_config_key=3516532442}) 2024-09-15T21:34:43.720 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.717+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=71,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=6} crc {logm=1069384672,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1095795938,mon_config_key=1950766801}) 2024-09-15T21:34:43.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.733+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.733+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=2644563206}) 2024-09-15T21:34:43.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.733+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=4,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=50} crc {mon_config_key=3115115494,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=2007190259}) 2024-09-15T21:34:43.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.733+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:34:43.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.733+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=2644563206}) 2024-09-15T21:34:43.734 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:34:43.733+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {mon_config_key=612054786,monmap=186996059,nvmeofgw=598337686,osd_metadata=166226148,osd_pg_creating=543505836,osd_snap=841190635,osdmap=151281891}) 2024-09-15T21:34:44.525 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:34:44.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient: get_monmap_and_config 2024-09-15T21:34:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient: build_initial_monmap 2024-09-15T21:34:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient: monmap: 2024-09-15T21:34:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:44.656388+0000 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:44.656388+0000 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:44.657 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 1 Processor -- start 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 1 -- start start 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient: init 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9da81379c0 0x7f9da806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient(hunting): picked mon.noname-a con 0x7f9da81379c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9da80a63b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9da81384b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da809c7f0 con 0x7f9da81384b0 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da809c510 con 0x7f9da80a63b0 2024-09-15T21:34:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9da809d1d0 con 0x7f9da81379c0 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9db027c640 10 monclient(hunting): authenticate will time out at 2008.051644s 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9da81379c0 0x7f9da806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9da81379c0 0x7f9da806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40652/0 (socket says 172.21.15.17:40652) 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 1 -- 172.21.15.17:0/3313275949 learned_addr learned my addr 172.21.15.17:0/3313275949 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): get_auth_request con 0x7f9da81379c0 auth_method 0 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.653+0000 7f9dae018640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 10 monclient(hunting): handle_auth_done global_id 204135 payload 274 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 1 -- 172.21.15.17:0/3313275949 >> v1:172.21.15.119:6789/0 conn(0x7f9da80a63b0 legacy=0x7f9da8137da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 1 -- 172.21.15.17:0/3313275949 >> v1:172.21.15.17:6789/0 conn(0x7f9da81384b0 legacy=0x7f9da813ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 1 -- 172.21.15.17:0/3313275949 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520583854 0 0) 0x7f9da809c7f0 con 0x7f9da81384b0 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 10 monclient: _finish_hunting 0 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 1 monclient: found mon.noname-a 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 1 -- 172.21.15.17:0/3313275949 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da80a5c10 con 0x7f9da81379c0 2024-09-15T21:34:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dae018640 1 --2- 172.21.15.17:0/3313275949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da81379c0 0x7f9da806f5e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9da40098a0 tx=0x7f9da401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9302b3ade0bc5ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 1 -- 172.21.15.17:0/3313275949 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40b0070 con 0x7f9da81379c0 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: dump: 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: _finish_auth 0 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: _check_auth_tickets 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 1 -- 172.21.15.17:0/3313275949 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da40ab600 con 0x7f9da81379c0 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 5 monclient: authenticate success, global_id 204135 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 1 -- 172.21.15.17:0/3313275949 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40ab9d0 con 0x7f9da81379c0 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 4 set_mon_vals no callback set 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9dad016640 10 monclient: dump: 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 10 monclient: shutdown 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 -- 172.21.15.17:0/3313275949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da81379c0 msgr2=0x7f9da806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 --2- 172.21.15.17:0/3313275949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da81379c0 0x7f9da806f5e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9da40098a0 tx=0x7f9da401c3d0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 -- 172.21.15.17:0/3313275949 shutdown_connections 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 --2- 172.21.15.17:0/3313275949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da81379c0 0x7f9da806f5e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 -- 172.21.15.17:0/3313275949 >> 172.21.15.17:0/3313275949 conn(0x7f9da80a7100 msgr2=0x7f9da80a7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 -- 172.21.15.17:0/3313275949 shutdown_connections 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 1 -- 172.21.15.17:0/3313275949 wait complete. 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 10 monclient: build_initial_monmap 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.657+0000 7f9db027c640 10 monclient: monmap: 2024-09-15T21:34:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 Processor -- start 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- start start 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: init 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 0x7f9da813bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): picked mon.noname-a con 0x7f9da80a63b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9da81379c0 0x7f9da813c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): picked mon.noname-b con 0x7f9da81379c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9da81384b0 0x7f9da813c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): picked mon.noname-c con 0x7f9da81384b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81432b0 con 0x7f9da80a63b0 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8143130 con 0x7f9da81379c0 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9da8143430 con 0x7f9da81384b0 2024-09-15T21:34:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient(hunting): authenticate will time out at 2008.056730s 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 0x7f9da813bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 0x7f9da813bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40654/0 (socket says 172.21.15.17:40654) 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 -- 172.21.15.17:0/1403468475 learned_addr learned my addr 172.21.15.17:0/1403468475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): get_auth_request con 0x7f9da80a63b0 auth_method 0 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9da81384b0 0x7f9da813c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): get_auth_request con 0x7f9da81384b0 auth_method 0 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae819640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient(hunting): handle_auth_done global_id 204141 payload 824 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dad817640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9da81379c0 0x7f9da813c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 -- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9da81384b0 msgr2=0x7f9da813c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9da81384b0 0x7f9da813c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 -- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9da81379c0 msgr2=0x7f9da813c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9da81379c0 0x7f9da813c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient: _finish_hunting 0 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 monclient: found mon.noname-a 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 -- 172.21.15.17:0/1403468475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da813eed0 con 0x7f9da80a63b0 2024-09-15T21:34:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dad817640 10 monclient: get_auth_request con 0x7f9da81379c0 auth_method 0 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dad817640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9da81379c0 0x7f9da813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dae018640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 0x7f9da813bf60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ae040 tx=0x7f9da4019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40b0040 con 0x7f9da80a63b0 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: dump: 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: _finish_auth 0 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: _check_auth_tickets 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da40b9070 con 0x7f9da80a63b0 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 5 monclient: authenticate success, global_id 204141 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _renew_subs 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da813fd40 con 0x7f9da80a63b0 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40aab90 con 0x7f9da80a63b0 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 10 monclient: dump: 2024-09-15T21:34:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:44.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dac815640 4 set_mon_vals no callback set 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _renew_subs 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da813d9e0 con 0x7f9da80a63b0 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da806f5e0 con 0x7f9da80a63b0 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9da40ab4c0 con 0x7f9da80a63b0 2024-09-15T21:34:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d8c03b9c0 0x7f9d8c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9da40ea880 con 0x7f9da80a63b0 2024-09-15T21:34:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dad817640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d8c03b9c0 0x7f9d8c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.661+0000 7f9dad817640 10 monclient: get_auth_request con 0x7f9d8c03b9c0 auth_method 0 2024-09-15T21:34:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.665+0000 7f9dad817640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d8c03b9c0 0x7f9d8c03de80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9da81376e0 tx=0x7f9d98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.665+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9da4016110 con 0x7f9da80a63b0 2024-09-15T21:34:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.665+0000 7f9d9effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.665+0000 7f9d9effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9db027c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9db027c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9db027c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9da80692d0 con 0x7f9da80a63b0 2024-09-15T21:34:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9d9effd640 1 -- 172.21.15.17:0/1403468475 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f9da40bec10 con 0x7f9da80a63b0 2024-09-15T21:34:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9d9effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.905+0000 7f9d9effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:44.910 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d8c03b9c0 msgr2=0x7f9d8c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d8c03b9c0 0x7f9d8c03de80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9da81376e0 tx=0x7f9d98009040 comp rx=0 tx=0).stop 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 10 monclient: shutdown 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 msgr2=0x7f9da813bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 0x7f9da813bf60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ae040 tx=0x7f9da4019170 comp rx=0 tx=0).stop 2024-09-15T21:34:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 shutdown_connections 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9d8c03b9c0 0x7f9d8c03de80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9da81384b0 0x7f9da813c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9da81379c0 0x7f9da813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 --2- 172.21.15.17:0/1403468475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9da80a63b0 0x7f9da813bf60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 >> 172.21.15.17:0/1403468475 conn(0x7f9da80a7100 msgr2=0x7f9da80a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 shutdown_connections 2024-09-15T21:34:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:44.909+0000 7f9db027c640 1 -- 172.21.15.17:0/1403468475 wait complete. 2024-09-15T21:34:44.929 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:34:44.929 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:34:44.929 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:34:44.929 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:34:45.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient: get_monmap_and_config 2024-09-15T21:34:45.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient: monmap: 2024-09-15T21:34:45.051 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.051 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.051 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:45.050427+0000 2024-09-15T21:34:45.051 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:45.050427+0000 2024-09-15T21:34:45.051 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.052 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.052 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:45.052 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:45.052 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:45.052 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:45.053 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:45.053 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 Processor -- start 2024-09-15T21:34:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 -- start start 2024-09-15T21:34:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient: init 2024-09-15T21:34:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): picked mon.noname-b con 0x7fca2c0a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fca2c0a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca2c0a57d0 0x7fca2c137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): picked mon.noname-c con 0x7fca2c0a57d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2c0a2c30 con 0x7fca2c0a53b0 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c0a5c80 con 0x7fca2c0a57d0 2024-09-15T21:34:45.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c0a0f50 con 0x7fca2c0a4a90 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca3140f640 10 monclient(hunting): authenticate will time out at 2008.445873s 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40132/0 (socket says 172.21.15.17:40132) 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 -- 172.21.15.17:0/1641104034 learned_addr learned my addr 172.21.15.17:0/1641104034 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): get_auth_request con 0x7fca2c0a4a90 auth_method 0 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2b7fe640 1 --2- 172.21.15.17:0/1641104034 >> v2:172.21.15.119:3300/0 conn(0x7fca2c0a57d0 0x7fca2c137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2b7fe640 10 monclient(hunting): get_auth_request con 0x7fca2c0a57d0 auth_method 0 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient(hunting): handle_auth_done global_id 164537 payload 274 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 -- 172.21.15.17:0/1641104034 >> v2:172.21.15.119:3300/0 conn(0x7fca2c0a57d0 msgr2=0x7fca2c137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 --2- 172.21.15.17:0/1641104034 >> v2:172.21.15.119:3300/0 conn(0x7fca2c0a57d0 0x7fca2c137f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 -- 172.21.15.17:0/1641104034 >> v1:172.21.15.119:6789/0 conn(0x7fca2c0a53b0 legacy=0x7fca2c137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 monclient: found mon.noname-b 2024-09-15T21:34:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2affd640 1 -- 172.21.15.17:0/1641104034 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c0a1a00 con 0x7fca2c0a4a90 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.049+0000 7fca2b7fe640 1 --2- 172.21.15.17:0/1641104034 >> v2:172.21.15.119:3300/0 conn(0x7fca2c0a57d0 0x7fca2c137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 1 --2- 172.21.15.17:0/1641104034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c0a4e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca24009870 tx=0x7fca2401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=549bfe66c8479c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 1 -- 172.21.15.17:0/1641104034 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240ab680 con 0x7fca2c0a4a90 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: dump: 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 1 -- 172.21.15.17:0/1641104034 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca240b0070 con 0x7fca2c0a4a90 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 1 -- 172.21.15.17:0/1641104034 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240aba40 con 0x7fca2c0a4a90 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 5 monclient: authenticate success, global_id 164537 2024-09-15T21:34:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 4 set_mon_vals no callback set 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca29ffb640 10 monclient: dump: 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: shutdown 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- 172.21.15.17:0/1641104034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 msgr2=0x7fca2c0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 --2- 172.21.15.17:0/1641104034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c0a4e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca24009870 tx=0x7fca2401c480 comp rx=0 tx=0).stop 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- 172.21.15.17:0/1641104034 shutdown_connections 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 --2- 172.21.15.17:0/1641104034 >> v2:172.21.15.119:3300/0 conn(0x7fca2c0a57d0 0x7fca2c137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 --2- 172.21.15.17:0/1641104034 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c0a4e70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- 172.21.15.17:0/1641104034 >> 172.21.15.17:0/1641104034 conn(0x7fca2c0aabb0 msgr2=0x7fca2c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- 172.21.15.17:0/1641104034 shutdown_connections 2024-09-15T21:34:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- 172.21.15.17:0/1641104034 wait complete. 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: monmap: 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:45.081 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 Processor -- start 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- start start 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: init 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c1cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): picked mon.noname-c con 0x7fca2c0a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2c0a53b0 0x7fca2c1376c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): picked mon.noname-b con 0x7fca2c0a53b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c0a57d0 0x7fca2c13d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): picked mon.noname-a con 0x7fca2c0a57d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c1384a0 con 0x7fca2c0a57d0 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c0a4560 con 0x7fca2c0a53b0 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c138620 con 0x7fca2c0a4a90 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca3140f640 10 monclient(hunting): authenticate will time out at 2008.451464s 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c1cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c0a57d0 0x7fca2c13d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c1cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40144/0 (socket says 172.21.15.17:40144) 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 1 -- 172.21.15.17:0/3983427814 learned_addr learned my addr 172.21.15.17:0/3983427814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): get_auth_request con 0x7fca2c0a4a90 auth_method 0 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): get_auth_request con 0x7fca2c0a57d0 auth_method 0 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2a7fc640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2c0a53b0 0x7fca2c1376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2a7fc640 10 monclient(hunting): get_auth_request con 0x7fca2c0a53b0 auth_method 0 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.053+0000 7fca2b7fe640 10 monclient(hunting): handle_auth_done global_id 204144 payload 824 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 msgr2=0x7fca2c1cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c1cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2c0a53b0 msgr2=0x7fca2c1376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2c0a53b0 0x7fca2c1376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 monclient: found mon.noname-a 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c13f340 con 0x7fca2c0a57d0 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca2c0a4a90 0x7fca2c1cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2a7fc640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2c0a53b0 0x7fca2c1376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2b7fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c0a57d0 0x7fca2c13d880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00e630 tx=0x7fca1c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c017070 con 0x7fca2c0a57d0 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: dump: 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.086 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c00c620 con 0x7fca2c0a57d0 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00f040 con 0x7fca2c0a57d0 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: dump: 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.088 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca28ff9640 4 set_mon_vals no callback set 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 5 monclient: authenticate success, global_id 204144 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 10 monclient: _renew_subs 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c1d0600 con 0x7fca2c0a57d0 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 10 monclient: _renew_subs 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c0a9860 con 0x7fca2c0a57d0 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 0x7fca2c13ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c09fe40 con 0x7fca2c137c00 2024-09-15T21:34:45.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 0x7fca2c13ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient: get_auth_request con 0x7fca2c137c00 auth_method 0 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fca1c01a070 con 0x7fca2c0a57d0 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 10 monclient(hunting): handle_auth_done global_id 204147 payload 824 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca2c0a4e70 con 0x7fca2c137c00 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca0c03b9c0 0x7fca0c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fca1c04fed0 con 0x7fca2c0a57d0 2024-09-15T21:34:45.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2affd640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 0x7fca2c13ddc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca2401a0b0 tx=0x7fca24019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240be5f0 con 0x7fca2c137c00 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: dump: 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.091 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2a7fc640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca0c03b9c0 0x7fca0c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2a7fc640 10 monclient: get_auth_request con 0x7fca0c03b9c0 auth_method 0 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fca240b0040 con 0x7fca2c137c00 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 msgr2=0x7fca2c13ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 0x7fca2c13ddc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca2401a0b0 tx=0x7fca24019220 comp rx=0 tx=0).stop 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 msgr2=0x7fca2c13ddc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca237fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c137c00 0x7fca2c13ddc0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca2401a0b0 tx=0x7fca24019220 comp rx=0 tx=0).stop 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.057+0000 7fca2a7fc640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca0c03b9c0 0x7fca0c03de80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fca2c13eec0 tx=0x7fca14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 0x7fca2c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c13e6e0 con 0x7fca2c13e300 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 0x7fca2c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient: get_auth_request con 0x7fca2c13e300 auth_method 0 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 10 monclient(hunting): handle_auth_done global_id 204150 payload 824 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 1 -- 172.21.15.17:0/3983427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fca2c120a60 con 0x7fca2c13e300 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2affd640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 0x7fca2c138cf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca240bc000 tx=0x7fca24009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240c5670 con 0x7fca2c13e300 2024-09-15T21:34:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 10 monclient: dump: 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4546 (secure 0 0 0) 0x7fca240a7070 con 0x7fca2c13e300 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 msgr2=0x7fca2c138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 0x7fca2c138cf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca240bc000 tx=0x7fca24009240 comp rx=0 tx=0).stop 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 msgr2=0x7fca2c138cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca237fe640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c13e300 0x7fca2c138cf0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca240bc000 tx=0x7fca24009240 comp rx=0 tx=0).stop 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:45.095 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 158, 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9599, 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:45.107 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca0c03b9c0 msgr2=0x7fca0c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca0c03b9c0 0x7fca0c03de80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fca2c13eec0 tx=0x7fca14009040 comp rx=0 tx=0).stop 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 10 monclient: shutdown 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c0a57d0 msgr2=0x7fca2c13d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 --2- 172.21.15.17:0/3983427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca2c0a57d0 0x7fca2c13d880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00e630 tx=0x7fca1c00eaf0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2b7fe640 1 -- 172.21.15.17:0/3983427814 reap_dead start 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca2b7fe640 1 -- 172.21.15.17:0/3983427814 reap_dead start 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 shutdown_connections 2024-09-15T21:34:45.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 >> 172.21.15.17:0/3983427814 conn(0x7fca2c0aabb0 msgr2=0x7fca2c0a0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 shutdown_connections 2024-09-15T21:34:45.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.065+0000 7fca3140f640 1 -- 172.21.15.17:0/3983427814 wait complete. 2024-09-15T21:34:45.110 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:34:45.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:34:45.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient: get_monmap_and_config 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient: monmap: 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:45.227367+0000 2024-09-15T21:34:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:45.227367+0000 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:45.228 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 Processor -- start 2024-09-15T21:34:45.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 -- start start 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient: init 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 0x7f9f6c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): picked mon.noname-a con 0x7f9f6c0a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): picked mon.noname-b con 0x7f9f6c09e8c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9f6c13be60 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6c0a31c0 con 0x7f9f6c13be60 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6c0a1ac0 con 0x7f9f6c0a0890 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c0a16f0 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f70be1640 10 monclient(hunting): authenticate will time out at 2008.622981s 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c0a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 0x7f9f6c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c0a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40156/0 (socket says 172.21.15.17:40156) 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 -- 172.21.15.17:0/4290743624 learned_addr learned my addr 172.21.15.17:0/4290743624 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): get_auth_request con 0x7f9f6c09e8c0 auth_method 0 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 0x7f9f6c0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40700/0 (socket says 172.21.15.17:40700) 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): get_auth_request con 0x7f9f6c0a0890 auth_method 0 2024-09-15T21:34:45.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_done global_id 164543 payload 274 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 -- 172.21.15.17:0/4290743624 >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 msgr2=0x7f9f6c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/4290743624 >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 0x7f9f6c0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 -- 172.21.15.17:0/4290743624 >> v1:172.21.15.17:6789/0 conn(0x7f9f6c13be60 legacy=0x7f9f6c13c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 monclient: found mon.noname-b 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f69d9b640 1 -- 172.21.15.17:0/4290743624 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c09d640 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.225+0000 7f9f6a59c640 1 --2- 172.21.15.17:0/4290743624 >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 0x7f9f6c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/4290743624 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c0a11b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0a1c40 tx=0x7f9f5c01c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c8be27eb557892e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 1 -- 172.21.15.17:0/4290743624 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0a5760 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: dump: 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.243 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 1 -- 172.21.15.17:0/4290743624 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c0a5900 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 5 monclient: authenticate success, global_id 164543 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:45.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 4 set_mon_vals no callback set 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 1 -- 172.21.15.17:0/4290743624 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0a5cd0 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f6959a640 10 monclient: dump: 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.245 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 10 monclient: shutdown 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 -- 172.21.15.17:0/4290743624 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 msgr2=0x7f9f6c0a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 --2- 172.21.15.17:0/4290743624 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c0a11b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0a1c40 tx=0x7f9f5c01c8a0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 -- 172.21.15.17:0/4290743624 shutdown_connections 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 --2- 172.21.15.17:0/4290743624 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c0a11b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 --2- 172.21.15.17:0/4290743624 >> v2:172.21.15.17:3300/0 conn(0x7f9f6c0a0890 0x7f9f6c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 -- 172.21.15.17:0/4290743624 >> 172.21.15.17:0/4290743624 conn(0x7f9f6c0a66a0 msgr2=0x7f9f6c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 -- 172.21.15.17:0/4290743624 shutdown_connections 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 -- 172.21.15.17:0/4290743624 wait complete. 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 10 monclient: monmap: 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:45.247 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 Processor -- start 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 1 -- start start 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.229+0000 7f9f70be1640 10 monclient: init 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c0a0890 0x7f9f6c1d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): picked mon.noname-c con 0x7f9f6c0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c13be60 0x7f9f6c13b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): picked mon.noname-b con 0x7f9f6c13be60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f6c141040 0x7f9f6c141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): picked mon.noname-a con 0x7f9f6c141040 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c09ce50 con 0x7f9f6c141040 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c09ccd0 con 0x7f9f6c13be60 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c1468e0 con 0x7f9f6c0a0890 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): authenticate will time out at 2008.628317s 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c0a0890 0x7f9f6c1d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f6c141040 0x7f9f6c141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c0a0890 0x7f9f6c1d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40158/0 (socket says 172.21.15.17:40158) 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 -- 172.21.15.17:0/958600669 learned_addr learned my addr 172.21.15.17:0/958600669 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f6c141040 0x7f9f6c141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40708/0 (socket says 172.21.15.17:40708) 2024-09-15T21:34:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): get_auth_request con 0x7f9f6c0a0890 auth_method 0 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9f6c141040 auth_method 0 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c13be60 0x7f9f6c13b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): get_auth_request con 0x7f9f6c13be60 auth_method 0 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient(hunting): handle_auth_done global_id 164546 payload 824 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c13be60 msgr2=0x7f9f6c13b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c13be60 0x7f9f6c13b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f6c141040 msgr2=0x7f9f6c141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f6c141040 0x7f9f6c141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 monclient: found mon.noname-c 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c143660 con 0x7f9f6c0a0890 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f6c141040 0x7f9f6c141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6a59c640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c0a0890 0x7f9f6c1d3d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400ebe0 tx=0x7f9f5400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54017070 con 0x7f9f6c0a0890 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5400cc30 con 0x7f9f6c0a0890 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 5 monclient: authenticate success, global_id 164546 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient: _renew_subs 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c09d0f0 con 0x7f9f6c0a0890 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54013470 con 0x7f9f6c0a0890 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f63fff640 4 set_mon_vals no callback set 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient: _renew_subs 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c125ac0 con 0x7f9f6c0a0890 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c02e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c125670 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c02e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient: get_auth_request con 0x7f9f6c09e8c0 auth_method 0 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9f5401a070 con 0x7f9f6c0a0890 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_done global_id 164549 payload 824 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f6c13c2c0 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4c03b9c0 0x7f9f4c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9f540505c0 con 0x7f9f6c0a0890 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c02e5d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0a01e0 tx=0x7f9f5c019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c009510 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4c03b9c0 0x7f9f4c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 10 monclient: get_auth_request con 0x7f9f4c03b9c0 auth_method 0 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9f5c0b6620 con 0x7f9f6c09e8c0 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 msgr2=0x7f9f6c02e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c02e5d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0a01e0 tx=0x7f9f5c019640 comp rx=0 tx=0).stop 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 msgr2=0x7f9f6c02e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c09e8c0 0x7f9f6c02e5d0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0a01e0 tx=0x7f9f5c019640 comp rx=0 tx=0).stop 2024-09-15T21:34:45.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.233+0000 7f9f6ad9d640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4c03b9c0 0x7f9f4c03de80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f64007fd0 tx=0x7f9f64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:45.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 0x7f9f6c063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c073080 con 0x7f9f6c1419b0 2024-09-15T21:34:45.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 0x7f9f6c063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient: get_auth_request con 0x7f9f6c1419b0 auth_method 0 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 10 monclient(hunting): handle_auth_done global_id 164552 payload 824 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 1 -- 172.21.15.17:0/958600669 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9f6c0a5730 con 0x7f9f6c1419b0 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f69d9b640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 0x7f9f6c063e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01a4d0 tx=0x7f9f5c0a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0c6070 con 0x7f9f6c1419b0 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 10 monclient: dump: 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f9f5c0c2cc0 con 0x7f9f6c1419b0 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 msgr2=0x7f9f6c063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 0x7f9f6c063e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01a4d0 tx=0x7f9f5c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 msgr2=0x7f9f6c063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f62ffd640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c1419b0 0x7f9f6c063e00 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01a4d0 tx=0x7f9f5c0a7000 comp rx=0 tx=0).stop 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:45.263 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 158, 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 233825, 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:45.264 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.265 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.266 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:45.267 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:45.268 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.269 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:45.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.271 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:45.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:45.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:45.277 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:45.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4c03b9c0 msgr2=0x7f9f4c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f4c03b9c0 0x7f9f4c03de80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f64007fd0 tx=0x7f9f64009040 comp rx=0 tx=0).stop 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 10 monclient: shutdown 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c0a0890 msgr2=0x7f9f6c1d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 --2- 172.21.15.17:0/958600669 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f6c0a0890 0x7f9f6c1d3d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400ebe0 tx=0x7f9f5400c520 comp rx=0 tx=0).stop 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f6ad9d640 1 -- 172.21.15.17:0/958600669 reap_dead start 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f6ad9d640 1 -- 172.21.15.17:0/958600669 reap_dead start 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 shutdown_connections 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 >> 172.21.15.17:0/958600669 conn(0x7f9f6c0a66a0 msgr2=0x7f9f6c0a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 shutdown_connections 2024-09-15T21:34:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.241+0000 7f9f70be1640 1 -- 172.21.15.17:0/958600669 wait complete. 2024-09-15T21:34:45.280 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:34:45.280 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:34:45.280 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:34:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient: get_monmap_and_config 2024-09-15T21:34:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient: monmap: 2024-09-15T21:34:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:45.398578+0000 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:45.398578+0000 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:45.399 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 Processor -- start 2024-09-15T21:34:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 -- start start 2024-09-15T21:34:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient: init 2024-09-15T21:34:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f163c1359e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f163c1364d0 0x7f163c13aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): picked mon.noname-c con 0x7f163c1364d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 0x7f163c13d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): picked mon.noname-b con 0x7f163c13b020 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f163c069f30 con 0x7f163c1359e0 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f163c135830 con 0x7f163c1364d0 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f163c0a1660 con 0x7f163c13b020 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1643465640 10 monclient(hunting): authenticate will time out at 2008.794177s 2024-09-15T21:34:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 0x7f163c13d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 0x7f163c13d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40176/0 (socket says 172.21.15.17:40176) 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 -- 172.21.15.17:0/1316753802 learned_addr learned my addr 172.21.15.17:0/1316753802 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): get_auth_request con 0x7f163c13b020 auth_method 0 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1640a00640 1 --2- 172.21.15.17:0/1316753802 >> v2:172.21.15.119:3300/0 conn(0x7f163c1364d0 0x7f163c13aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1640a00640 10 monclient(hunting): get_auth_request con 0x7f163c1364d0 auth_method 0 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1640a00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1640a00640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1640a00640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient(hunting): handle_auth_done global_id 164555 payload 274 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 -- 172.21.15.17:0/1316753802 >> v2:172.21.15.119:3300/0 conn(0x7f163c1364d0 msgr2=0x7f163c13aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 --2- 172.21.15.17:0/1316753802 >> v2:172.21.15.119:3300/0 conn(0x7f163c1364d0 0x7f163c13aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 -- 172.21.15.17:0/1316753802 >> v1:172.21.15.119:6789/0 conn(0x7f163c1359e0 legacy=0x7f163c135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 monclient: found mon.noname-b 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.397+0000 7f1641a02640 1 -- 172.21.15.17:0/1316753802 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c0a1940 con 0x7f163c13b020 2024-09-15T21:34:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641a02640 1 --2- 172.21.15.17:0/1316753802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 0x7f163c13d410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f162c009e50 tx=0x7f162c01c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c4a30ac24929d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 1 -- 172.21.15.17:0/1316753802 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0a4c20 con 0x7f163c13b020 2024-09-15T21:34:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:34:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: dump: 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 1 -- 172.21.15.17:0/1316753802 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162c0a4dc0 con 0x7f163c13b020 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 5 monclient: authenticate success, global_id 164555 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 1 -- 172.21.15.17:0/1316753802 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0ac610 con 0x7f163c13b020 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f163bfff640 10 monclient: dump: 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 4 set_mon_vals no callback set 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: shutdown 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- 172.21.15.17:0/1316753802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 msgr2=0x7f163c13d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 --2- 172.21.15.17:0/1316753802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 0x7f163c13d410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f162c009e50 tx=0x7f162c01c5d0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- 172.21.15.17:0/1316753802 shutdown_connections 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 --2- 172.21.15.17:0/1316753802 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c13b020 0x7f163c13d410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 --2- 172.21.15.17:0/1316753802 >> v2:172.21.15.119:3300/0 conn(0x7f163c1364d0 0x7f163c13aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- 172.21.15.17:0/1316753802 >> 172.21.15.17:0/1316753802 conn(0x7f163c0a4c60 msgr2=0x7f163c0a5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- 172.21.15.17:0/1316753802 shutdown_connections 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- 172.21.15.17:0/1316753802 wait complete. 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: monmap: 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 Processor -- start 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- start start 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: init 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1359e0 0x7f163c09c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): picked mon.noname-a con 0x7f163c1359e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c1364d0 0x7f163c09cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): picked mon.noname-c con 0x7f163c1364d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c13b020 0x7f163c09d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): picked mon.noname-b con 0x7f163c13b020 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f163c140e70 con 0x7f163c1359e0 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f163c140cf0 con 0x7f163c13b020 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f163c140ff0 con 0x7f163c1364d0 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1643465640 10 monclient(hunting): authenticate will time out at 2008.799601s 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c1364d0 0x7f163c09cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1359e0 0x7f163c09c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c1364d0 0x7f163c09cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:40182/0 (socket says 172.21.15.17:40182) 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1359e0 0x7f163c09c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40718/0 (socket says 172.21.15.17:40718) 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 1 -- 172.21.15.17:0/2648428185 learned_addr learned my addr 172.21.15.17:0/2648428185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): get_auth_request con 0x7f163c1364d0 auth_method 0 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): get_auth_request con 0x7f163c1359e0 auth_method 0 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1640a00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.401+0000 7f1641a02640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c13b020 0x7f163c09d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 10 monclient(hunting): get_auth_request con 0x7f163c13b020 auth_method 0 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 10 monclient(hunting): handle_auth_done global_id 164558 payload 824 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c13b020 msgr2=0x7f163c09d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c13b020 0x7f163c09d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1359e0 msgr2=0x7f163c09c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1359e0 0x7f163c09c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 monclient: found mon.noname-c 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c13acc0 con 0x7f163c1364d0 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1359e0 0x7f163c09c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1640a00640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c1364d0 0x7f163c09cda0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f163000d800 tx=0x7f163000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000b6c0 con 0x7f163c1364d0 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: dump: 2024-09-15T21:34:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1630019070 con 0x7f163c1364d0 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 5 monclient: authenticate success, global_id 164558 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 10 monclient: _renew_subs 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c09f180 con 0x7f163c1364d0 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000ba60 con 0x7f163c1364d0 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: dump: 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 10 monclient: _renew_subs 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c13a390 con 0x7f163c1364d0 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f163b7fe640 4 set_mon_vals no callback set 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 0x7f163c1ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f163c135dc0 con 0x7f163c1ce980 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f163000bc00 con 0x7f163c1364d0 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f162803b9c0 0x7f162803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 0x7f163c1ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient: get_auth_request con 0x7f163c1ce980 auth_method 0 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f1630050a30 con 0x7f163c1364d0 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f162803b9c0 0x7f162803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 10 monclient: get_auth_request con 0x7f162803b9c0 auth_method 0 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 10 monclient(hunting): handle_auth_done global_id 194245 payload 824 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f163c0008e0 con 0x7f163c1ce980 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641201640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 0x7f163c1ced60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1634000f70 tx=0x7f163400ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1641a02640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f162803b9c0 0x7f162803de80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ad000 tx=0x7f162c0a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340052e0 con 0x7f163c1ce980 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: dump: 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1634018070 con 0x7f163c1ce980 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 msgr2=0x7f163c1ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 0x7f163c1ced60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1634000f70 tx=0x7f163400ff70 comp rx=0 tx=0).stop 2024-09-15T21:34:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 msgr2=0x7f163c1ced60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.405+0000 7f1639ffb640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c1ce980 0x7f163c1ced60 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1634000f70 tx=0x7f163400ff70 comp rx=0 tx=0).stop 2024-09-15T21:34:45.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1643465640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:34:45.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1643465640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 0x7f163c02d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1643465640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f163c072a80 con 0x7f163c02d450 2024-09-15T21:34:45.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 0x7f163c02d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient: get_auth_request con 0x7f163c02d450 auth_method 0 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 10 monclient(hunting): handle_auth_done global_id 194248 payload 824 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 1 -- 172.21.15.17:0/2648428185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f163c136c80 con 0x7f163c02d450 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1641201640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 0x7f163c02d860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1634000f70 tx=0x7f1634001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163400dac0 con 0x7f163c02d450 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1639ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1639ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.413+0000 7f1639ffb640 10 monclient: dump: 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f163400dac0 con 0x7f163c02d450 2024-09-15T21:34:45.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 msgr2=0x7f163c02d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 0x7f163c02d860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1634000f70 tx=0x7f1634001130 comp rx=0 tx=0).stop 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 msgr2=0x7f163c02d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1639ffb640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f163c02d450 0x7f163c02d860 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1634000f70 tx=0x7f1634001130 comp rx=0 tx=0).stop 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:45.430 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 158, 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 52206, 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.431 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:34:45.432 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:34:45.434 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:34:45.435 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.437 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:34:45.440 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.441 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:34:45.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f162803b9c0 msgr2=0x7f162803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f162803b9c0 0x7f162803de80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ad000 tx=0x7f162c0a6070 comp rx=0 tx=0).stop 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 10 monclient: shutdown 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c1364d0 msgr2=0x7f163c09cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 --2- 172.21.15.17:0/2648428185 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f163c1364d0 0x7f163c09cda0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f163000d800 tx=0x7f163000dcc0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1641a02640 1 -- 172.21.15.17:0/2648428185 reap_dead start 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1641a02640 1 -- 172.21.15.17:0/2648428185 reap_dead start 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 shutdown_connections 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 >> 172.21.15.17:0/2648428185 conn(0x7f163c0a4c60 msgr2=0x7f163c134940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 shutdown_connections 2024-09-15T21:34:45.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.417+0000 7f1643465640 1 -- 172.21.15.17:0/2648428185 wait complete. 2024-09-15T21:34:45.446 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:34:45.446 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T21:34:45.446 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:34:45.446 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T21:34:45.446 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T21:34:45.446 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:34:45.446 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:45.470 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:34:45.470 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:34:45.471 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient: get_monmap_and_config 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient: monmap: 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:45.635331+0000 2024-09-15T21:34:45.635 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:45.635331+0000 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:45.636 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 1 Processor -- start 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 1 -- start start 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient: init 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fca4409d400 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fca4409def0 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:45.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca4413b1d0 0x7fca4413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient(hunting): picked mon.noname-c con 0x7fca4413b1d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca440a3130 con 0x7fca4409def0 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4409e7a0 con 0x7fca4409d400 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fca440a1bc0 con 0x7fca4413b1d0 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 10 monclient(hunting): authenticate will time out at 2009.030611s 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fca4409def0 0x7fca44139ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47674/0 (socket says 172.21.15.17:47674) 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 -- 172.21.15.17:0/2278024695 learned_addr learned my addr 172.21.15.17:0/2278024695 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 1 --2- 172.21.15.17:0/2278024695 >> v2:172.21.15.119:3300/0 conn(0x7fca4413b1d0 0x7fca4413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3074470094 0 0) 0x7fca440a3130 con 0x7fca4409def0 2024-09-15T21:34:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): get_auth_request con 0x7fca4413b1d0 auth_method 0 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): _negotiate 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): my global_id is 204165 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca30005740 con 0x7fca4409def0 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 838477700 0 0) 0x7fca30005740 con 0x7fca4409def0 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 >> v2:172.21.15.119:3300/0 conn(0x7fca4413b1d0 msgr2=0x7fca4413d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:45.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 --2- 172.21.15.17:0/2278024695 >> v2:172.21.15.119:3300/0 conn(0x7fca4413b1d0 0x7fca4413d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 >> v1:172.21.15.119:6789/0 conn(0x7fca4409d400 legacy=0x7fca4409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca440a0320 con 0x7fca4409def0 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fca38001700 con 0x7fca4409def0 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 5 monclient: authenticate success, global_id 204165 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: dump: 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:34:45.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 >> v1:172.21.15.17:6789/0 conn(0x7fca4409def0 legacy=0x7fca44139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca30005c70 0x7fca30006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): picked mon.c con 0x7fca30005c70 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca30006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): picked mon.a con 0x7fca30006a20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca300075e0 0x7fca3000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): picked mon.b con 0x7fca300075e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca3000e420 con 0x7fca30006a20 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca3000e5d0 con 0x7fca300075e0 2024-09-15T21:34:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fca3000e7d0 con 0x7fca30005c70 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca48e27640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4c08d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49e29640 1 -- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca30005c70 msgr2=0x7fca30006090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49e29640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca30005c70 0x7fca30006090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca30006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): get_auth_request con 0x7fca30006a20 auth_method 0 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient(hunting): handle_auth_done global_id 204165 payload 293 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 -- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca30005c70 msgr2=0x7fca30006090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca30005c70 0x7fca30006090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 -- 172.21.15.17:0/2278024695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca300075e0 msgr2=0x7fca3000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca300075e0 0x7fca3000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 monclient: found mon.a 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 1 -- 172.21.15.17:0/2278024695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca30005740 con 0x7fca30006a20 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca4a62a640 1 -- 172.21.15.17:0/2278024695 reap_dead start 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.633+0000 7fca49628640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca49628640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca30006ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c210 tx=0x7fca3801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7533138203c0ea0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38007d20 con 0x7fca30006a20 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: dump: 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca380a9020 con 0x7fca30006a20 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 1 -- 172.21.15.17:0/2278024695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38005b20 con 0x7fca30006a20 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 4 set_mon_vals no callback set 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca48e27640 10 monclient: dump: 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.657 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 10 monclient: shutdown 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 -- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 msgr2=0x7fca30006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca30006ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c210 tx=0x7fca3801cba0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 -- 172.21.15.17:0/2278024695 shutdown_connections 2024-09-15T21:34:45.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 --2- 172.21.15.17:0/2278024695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca30006ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 -- 172.21.15.17:0/2278024695 >> 172.21.15.17:0/2278024695 conn(0x7fca440a66a0 msgr2=0x7fca440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 -- 172.21.15.17:0/2278024695 shutdown_connections 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 -- 172.21.15.17:0/2278024695 wait complete. 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 10 monclient: build_initial_monmap 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 10 monclient: monmap: 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:45.659 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 Processor -- start 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 1 -- start start 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.637+0000 7fca4c08d640 10 monclient: init 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca441cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): picked mon.noname-a con 0x7fca30006a20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44133a20 0x7fca44133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): picked mon.noname-b con 0x7fca44133a20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca4411d6f0 0x7fca4411dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): picked mon.noname-c con 0x7fca4411d6f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca440a3490 con 0x7fca30006a20 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca44134e70 con 0x7fca44133a20 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fca44134ff0 con 0x7fca4411d6f0 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient(hunting): authenticate will time out at 2009.036264s 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4a62a640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca4411d6f0 msgr2=0x7fca4411dba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca441cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4a62a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca4411d6f0 0x7fca4411dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca441cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40764/0 (socket says 172.21.15.17:40764) 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 -- 172.21.15.17:0/643466562 learned_addr learned my addr 172.21.15.17:0/643466562 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:45.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): get_auth_request con 0x7fca30006a20 auth_method 0 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44133a20 0x7fca44133ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): get_auth_request con 0x7fca44133a20 auth_method 0 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:45.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient(hunting): handle_auth_done global_id 204168 payload 824 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 -- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca4411d6f0 msgr2=0x7fca4411dba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca4411d6f0 0x7fca4411dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 -- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44133a20 msgr2=0x7fca44133ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44133a20 0x7fca44133ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient: _finish_hunting 0 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 monclient: found mon.noname-a 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 -- 172.21.15.17:0/643466562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca441343e0 con 0x7fca30006a20 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49e29640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca441cfd30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c530 tx=0x7fca4000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40007480 con 0x7fca30006a20 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: dump: 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.664 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: _finish_auth 0 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: _check_auth_tickets 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca40007620 con 0x7fca30006a20 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 5 monclient: authenticate success, global_id 204168 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _renew_subs 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca44120b40 con 0x7fca30006a20 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400059f0 con 0x7fca30006a20 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:45.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 10 monclient: dump: 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca37fff640 4 set_mon_vals no callback set 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _renew_subs 2024-09-15T21:34:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4411f080 con 0x7fca30006a20 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca44135320 con 0x7fca30006a20 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fca40005b90 con 0x7fca30006a20 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca2803b970 0x7fca2803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fca400501a0 con 0x7fca30006a20 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca2803b970 0x7fca2803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 10 monclient: get_auth_request con 0x7fca2803b970 auth_method 0 2024-09-15T21:34:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.641+0000 7fca49628640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca2803b970 0x7fca2803de30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fca380aa000 tx=0x7fca38001a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:45.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.645+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fca44120b40 con 0x7fca30006a20 2024-09-15T21:34:45.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.645+0000 7fca367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:45.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.645+0000 7fca367fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fca441214a0 con 0x7fca30006a20 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca367fc640 1 -- 172.21.15.17:0/643466562 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fca40010d10 con 0x7fca30006a20 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:45.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca367fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:45.884 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca2803b970 msgr2=0x7fca2803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca2803b970 0x7fca2803de30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fca380aa000 tx=0x7fca38001a00 comp rx=0 tx=0).stop 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 10 monclient: shutdown 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 msgr2=0x7fca441cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca441cfd30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c530 tx=0x7fca4000c9f0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 shutdown_connections 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fca2803b970 0x7fca2803de30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca4411d6f0 0x7fca4411dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44133a20 0x7fca44133ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 --2- 172.21.15.17:0/643466562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca30006a20 0x7fca441cfd30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:45.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 >> 172.21.15.17:0/643466562 conn(0x7fca440a66a0 msgr2=0x7fca440a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:45.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 shutdown_connections 2024-09-15T21:34:45.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:45.881+0000 7fca4c08d640 1 -- 172.21.15.17:0/643466562 wait complete. 2024-09-15T21:34:48.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient: get_monmap_and_config 2024-09-15T21:34:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient: build_initial_monmap 2024-09-15T21:34:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient: monmap: 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:49.035615+0000 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:49.035615+0000 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 1 Processor -- start 2024-09-15T21:34:49.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 1 -- start start 2024-09-15T21:34:49.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient: init 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 0x7fc12409d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient(hunting): picked mon.noname-b con 0x7fc12409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc12409dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc124134d70 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1240a3490 con 0x7fc12409dd20 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1240a0320 con 0x7fc124134d70 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc1240a1bc0 con 0x7fc12409d400 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc128fbf640 10 monclient(hunting): authenticate will time out at 2012.431002s 2024-09-15T21:34:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12259c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 msgr2=0x7fc12409d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 0x7fc12409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc121d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc12409dd20 0x7fc124134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44048/0 (socket says 172.21.15.17:44048) 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc121d9b640 1 -- 172.21.15.17:0/2542726104 learned_addr learned my addr 172.21.15.17:0/2542726104 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806110652 0 0) 0x7fc1240a3490 con 0x7fc12409dd20 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient(hunting): _negotiate 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient(hunting): my global_id is 204171 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc10c003530 con 0x7fc12409dd20 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3916693802 0 0) 0x7fc10c003530 con 0x7fc12409dd20 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 msgr2=0x7fc12409d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 0x7fc12409d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 >> v1:172.21.15.119:6789/0 conn(0x7fc124134d70 legacy=0x7fc12413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: _finish_hunting 0 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12409e7a0 con 0x7fc12409dd20 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: _finish_auth 0 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: _check_auth_tickets 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fc114001700 con 0x7fc12409dd20 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:34:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: dump: 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:34:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 >> v1:172.21.15.17:6789/0 conn(0x7fc12409dd20 legacy=0x7fc124134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.033+0000 7fc12159a640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c0039f0 0x7fc10c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): picked mon.b con 0x7fc10c0039f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc10c004810 0x7fc10c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): picked mon.a con 0x7fc10c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 0x7fc10c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): picked mon.c con 0x7fc12409d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240a1bc0 con 0x7fc10c004810 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0056a0 con 0x7fc10c0039f0 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c00bf90 con 0x7fc12409d400 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc122d9d640 1 -- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 msgr2=0x7fc10c00b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc122d9d640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 0x7fc10c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc10c004810 0x7fc10c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): get_auth_request con 0x7fc10c004810 auth_method 0 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12259c640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c0039f0 0x7fc10c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12259c640 10 monclient(hunting): get_auth_request con 0x7fc10c0039f0 auth_method 0 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient(hunting): handle_auth_done global_id 204171 payload 293 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 -- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 msgr2=0x7fc10c00b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12409d400 0x7fc10c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 -- 172.21.15.17:0/2542726104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c0039f0 msgr2=0x7fc10c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c0039f0 0x7fc10c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 monclient: found mon.a 2024-09-15T21:34:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 -- 172.21.15.17:0/2542726104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c003530 con 0x7fc10c004810 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient: _finish_auth 0 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12259c640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c0039f0 0x7fc10c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc128fbf640 5 monclient: authenticate success, global_id 204171 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc128fbf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc121d9b640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc10c004810 0x7fc10c004c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c240 tx=0x7fc11401d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6df5e53e9d9ad2d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114007cf0 con 0x7fc10c004810 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: dump: 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140a9020 con 0x7fc10c004810 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 1 -- 172.21.15.17:0/2542726104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140a5b30 con 0x7fc10c004810 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc128fbf640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc128fbf640 4 set_mon_vals no callback set 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.037+0000 7fc12159a640 10 monclient: dump: 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:49.052 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: shutdown 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc10c004810 msgr2=0x7fc10c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 --2- 172.21.15.17:0/2542726104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc10c004810 0x7fc10c004c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c240 tx=0x7fc11401d5b0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc122d9d640 1 -- 172.21.15.17:0/2542726104 reap_dead start 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/2542726104 shutdown_connections 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/2542726104 >> 172.21.15.17:0/2542726104 conn(0x7fc1240a66a0 msgr2=0x7fc1240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/2542726104 shutdown_connections 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/2542726104 wait complete. 2024-09-15T21:34:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: build_initial_monmap 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: monmap: 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:49.054 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 Processor -- start 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- start start 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: init 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c004810 0x7fc1240a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): picked mon.noname-b con 0x7fc10c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 0x7fc12413a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): picked mon.noname-a con 0x7fc124139c40 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12413a600 0x7fc1241d6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): picked mon.noname-c con 0x7fc12413a600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240a32b0 con 0x7fc124139c40 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240a3130 con 0x7fc10c004810 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc12409ccd0 con 0x7fc12413a600 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient(hunting): authenticate will time out at 2012.437904s 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 0x7fc12413a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 0x7fc12413a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45218/0 (socket says 172.21.15.17:45218) 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 -- 172.21.15.17:0/1303836762 learned_addr learned my addr 172.21.15.17:0/1303836762 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): get_auth_request con 0x7fc124139c40 auth_method 0 2024-09-15T21:34:49.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc122d9d640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12413a600 msgr2=0x7fc1241d6600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc122d9d640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12413a600 0x7fc1241d6600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient(hunting): handle_auth_done global_id 204174 payload 824 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 -- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12413a600 msgr2=0x7fc1241d6600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12413a600 0x7fc1241d6600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 -- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c004810 msgr2=0x7fc1240a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:49.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c004810 0x7fc1240a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 monclient: found mon.noname-a 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 -- 172.21.15.17:0/1303836762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12413c980 con 0x7fc124139c40 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc121d9b640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 0x7fc12413a0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc114000c10 tx=0x7fc10c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140a5af0 con 0x7fc124139c40 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: dump: 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:49.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: _finish_auth 0 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140a5c90 con 0x7fc124139c40 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 5 monclient: authenticate success, global_id 204174 2024-09-15T21:34:49.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _renew_subs 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12406f410 con 0x7fc124139c40 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140049f0 con 0x7fc124139c40 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 10 monclient: dump: 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _renew_subs 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12413c6a0 con 0x7fc124139c40 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc113fff640 4 set_mon_vals no callback set 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc114004b90 con 0x7fc124139c40 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124134660 con 0x7fc124139c40 2024-09-15T21:34:49.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.041+0000 7fc112ffd640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10403b970 0x7fc10403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.045+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc1140ea790 con 0x7fc124139c40 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.045+0000 7fc12259c640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10403b970 0x7fc10403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.045+0000 7fc12259c640 10 monclient: get_auth_request con 0x7fc10403b970 auth_method 0 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.045+0000 7fc12259c640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10403b970 0x7fc10403de30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc1180038a0 tx=0x7fc118008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.049+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc114016930 con 0x7fc124139c40 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.049+0000 7fc112ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:49.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.049+0000 7fc112ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc128fbf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc128fbf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc128fbf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc124071f40 con 0x7fc124139c40 2024-09-15T21:34:49.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc112ffd640 1 -- 172.21.15.17:0/1303836762 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fc1140c2e30 con 0x7fc124139c40 2024-09-15T21:34:49.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc112ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:49.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.285+0000 7fc112ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:49.288 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10403b970 msgr2=0x7fc10403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10403b970 0x7fc10403de30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc1180038a0 tx=0x7fc118008040 comp rx=0 tx=0).stop 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 10 monclient: shutdown 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 msgr2=0x7fc12413a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 0x7fc12413a0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc114000c10 tx=0x7fc10c005840 comp rx=0 tx=0).stop 2024-09-15T21:34:49.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 shutdown_connections 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc10403b970 0x7fc10403de30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc12413a600 0x7fc1241d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc124139c40 0x7fc12413a0c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 --2- 172.21.15.17:0/1303836762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc10c004810 0x7fc1240a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 >> 172.21.15.17:0/1303836762 conn(0x7fc1240a66a0 msgr2=0x7fc1240a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 shutdown_connections 2024-09-15T21:34:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:49.289+0000 7fc128fbf640 1 -- 172.21.15.17:0/1303836762 wait complete. 2024-09-15T21:34:52.309 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:52.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient: get_monmap_and_config 2024-09-15T21:34:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient: build_initial_monmap 2024-09-15T21:34:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient: monmap: 2024-09-15T21:34:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:52.443686+0000 2024-09-15T21:34:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:52.443686+0000 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:52.445 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 Processor -- start 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 -- start start 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient: init 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc0a0890 0x7f17cc0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): picked mon.noname-c con 0x7f17cc0a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc09ffc0 0x7f17cc0a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): picked mon.noname-b con 0x7f17cc09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f17cc135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17cc0a3130 con 0x7f17cc135670 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f17cc0a16f0 con 0x7f17cc0a0890 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc073df0 con 0x7f17cc09ffc0 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17d1f19640 10 monclient(hunting): authenticate will time out at 2015.839606s 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17caffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc09ffc0 msgr2=0x7f17cc0a11b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.441+0000 7f17caffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc09ffc0 0x7f17cc0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc0a0890 0x7f17cc0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc0a0890 0x7f17cc0a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:43204/0 (socket says 172.21.15.17:43204) 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 -- 172.21.15.17:0/130657935 learned_addr learned my addr 172.21.15.17:0/130657935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request con 0x7f17cc0a0890 auth_method 0 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_done global_id 194257 payload 274 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 -- 172.21.15.17:0/130657935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc09ffc0 msgr2=0x7f17cc0a11b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 --2- 172.21.15.17:0/130657935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc09ffc0 0x7f17cc0a11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 -- 172.21.15.17:0/130657935 >> v1:172.21.15.119:6789/0 conn(0x7f17cc135670 legacy=0x7f17cc135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 monclient: found mon.noname-c 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 -- 172.21.15.17:0/130657935 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc0a1bf0 con 0x7f17cc0a0890 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17cb7fe640 1 --2- 172.21.15.17:0/130657935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc0a0890 0x7f17cc0a0c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0a1870 tx=0x7f17bc01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bfbc851ee4d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 1 -- 172.21.15.17:0/130657935 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0b0070 con 0x7f17cc0a0890 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:34:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: dump: 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: _finish_auth 0 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 1 -- 172.21.15.17:0/130657935 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc0ab600 con 0x7f17cc0a0890 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 5 monclient: authenticate success, global_id 194257 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 10 monclient: get_monmap_and_config success 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 4 set_mon_vals no callback set 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 1 -- 172.21.15.17:0/130657935 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0ab9d0 con 0x7f17cc0a0890 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17ca7fc640 10 monclient: dump: 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 10 monclient: shutdown 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 1 -- 172.21.15.17:0/130657935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc0a0890 msgr2=0x7f17cc0a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 1 --2- 172.21.15.17:0/130657935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc0a0890 0x7f17cc0a0c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0a1870 tx=0x7f17bc01c420 comp rx=0 tx=0).stop 2024-09-15T21:34:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 1 -- 172.21.15.17:0/130657935 shutdown_connections 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 1 --2- 172.21.15.17:0/130657935 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc09ffc0 0x7f17cc0a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 1 --2- 172.21.15.17:0/130657935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc0a0890 0x7f17cc0a0c70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.445+0000 7f17d1f19640 1 -- 172.21.15.17:0/130657935 >> 172.21.15.17:0/130657935 conn(0x7f17cc0a66a0 msgr2=0x7f17cc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- 172.21.15.17:0/130657935 shutdown_connections 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- 172.21.15.17:0/130657935 wait complete. 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: build_initial_monmap 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: monmap: 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:52.457 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:52.458 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:34:52.458 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:34:52.458 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:34:52.458 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 Processor -- start 2024-09-15T21:34:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- start start 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: init 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc09ffc0 0x7f17cc1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): picked mon.noname-b con 0x7f17cc09ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc0a0890 0x7f17cc134830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): picked mon.noname-c con 0x7f17cc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 0x7f17cc11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): picked mon.noname-a con 0x7f17cc135670 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc140690 con 0x7f17cc135670 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc140510 con 0x7f17cc09ffc0 2024-09-15T21:34:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc140810 con 0x7f17cc0a0890 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient(hunting): authenticate will time out at 2015.845522s 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17caffd640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc0a0890 msgr2=0x7f17cc134830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17caffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc0a0890 0x7f17cc134830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 0x7f17cc11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 0x7f17cc11c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45224/0 (socket says 172.21.15.17:45224) 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 -- 172.21.15.17:0/22321597 learned_addr learned my addr 172.21.15.17:0/22321597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): get_auth_request con 0x7f17cc135670 auth_method 0 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc09ffc0 0x7f17cc1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request con 0x7f17cc09ffc0 auth_method 0 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient(hunting): handle_auth_done global_id 204177 payload 824 2024-09-15T21:34:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 -- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc0a0890 msgr2=0x7f17cc134830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc0a0890 0x7f17cc134830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 -- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc09ffc0 msgr2=0x7f17cc1342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc09ffc0 0x7f17cc1342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 monclient: found mon.noname-a 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 -- 172.21.15.17:0/22321597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc11ed70 con 0x7f17cc135670 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17cbfff640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 0x7f17cc11c4e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17c400c7a0 tx=0x7f17c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4003b80 con 0x7f17cc135670 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: dump: 2024-09-15T21:34:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: _finish_auth 0 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:34:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 5 monclient: authenticate success, global_id 204177 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4003d40 con 0x7f17cc135670 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _renew_subs 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c400aa90 con 0x7f17cc135670 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc11f050 con 0x7f17cc135670 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17c9ffb640 4 set_mon_vals no callback set 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 10 monclient: dump: 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _renew_subs 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc11db20 con 0x7f17cc135670 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc135aa0 con 0x7f17cc135670 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f17c400ac30 con 0x7f17cc135670 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17b803b970 0x7f17b803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.449+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f17c4050400 con 0x7f17cc135670 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.453+0000 7f17cb7fe640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17b803b970 0x7f17b803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.453+0000 7f17cb7fe640 10 monclient: get_auth_request con 0x7f17b803b970 auth_method 0 2024-09-15T21:34:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.453+0000 7f17cb7fe640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17b803b970 0x7f17b803de30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0ad000 tx=0x7f17bc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:34:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.457+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f17c4012420 con 0x7f17cc135670 2024-09-15T21:34:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.457+0000 7f17b3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:34:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.457+0000 7f17b3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:34:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17d1f19640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:34:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17d1f19640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17d1f19640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:34:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f17cc06d020 con 0x7f17cc135670 2024-09-15T21:34:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17b3fff640 1 -- 172.21.15.17:0/22321597 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f17c4005500 con 0x7f17cc135670 2024-09-15T21:34:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17b3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:34:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.693+0000 7f17b3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:34:52.696 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:52.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17b803b970 msgr2=0x7f17b803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17b803b970 0x7f17b803de30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0ad000 tx=0x7f17bc0a6000 comp rx=0 tx=0).stop 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 10 monclient: shutdown 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 msgr2=0x7f17cc11c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 0x7f17cc11c4e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17c400c7a0 tx=0x7f17c400cc60 comp rx=0 tx=0).stop 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 shutdown_connections 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f17b803b970 0x7f17b803de30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17cc135670 0x7f17cc11c4e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f17cc0a0890 0x7f17cc134830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 --2- 172.21.15.17:0/22321597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc09ffc0 0x7f17cc1342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 >> 172.21.15.17:0/22321597 conn(0x7f17cc0a66a0 msgr2=0x7f17cc136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:34:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 shutdown_connections 2024-09-15T21:34:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:52.697+0000 7f17d1f19640 1 -- 172.21.15.17:0/22321597 wait complete. 2024-09-15T21:34:55.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient: get_monmap_and_config 2024-09-15T21:34:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient: build_initial_monmap 2024-09-15T21:34:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient: monmap: 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:34:55.851402+0000 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:34:55.851402+0000 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:34:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:34:55.853 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:34:55.853 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:34:55.853 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:34:55.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 Processor -- start 2024-09-15T21:34:55.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 -- start start 2024-09-15T21:34:55.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient: init 2024-09-15T21:34:55.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient: _reopen_session rank -1 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3ca809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): picked mon.noname-b con 0x7f3ca809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): picked mon.noname-a con 0x7f3ca809dd20 addr v2:172.21.15.17:3300/0 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): picked mon.noname-c con 0x7f3ca8134ba0 addr v2:172.21.15.119:3300/0 2024-09-15T21:34:55.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca809f0b0 con 0x7f3ca809dd20 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca80a1bc0 con 0x7f3ca8134ba0 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): start opening mon connection 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca80a3490 con 0x7f3ca809d400 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): _renew_subs 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3cacf96640 10 monclient(hunting): authenticate will time out at 2019.247178s 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca659c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3ca809d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3ca809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:55.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45234/0 (socket says 172.21.15.17:45234) 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 learned_addr learned my addr 172.21.15.17:0/1833376570 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request con 0x7f3ca809dd20 auth_method 0 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:55.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request con 0x7f3ca8134ba0 auth_method 0 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:55.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:55.849+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:34:56.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca659c640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3ca809d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:56.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3ca809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request con 0x7f3ca809dd20 auth_method 0 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:56.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request con 0x7f3ca8134ba0 auth_method 0 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:56.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.053+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:34:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca659c640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3ca809d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3ca809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:34:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request con 0x7f3ca809dd20 auth_method 0 2024-09-15T21:34:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:34:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request con 0x7f3ca8134ba0 auth_method 0 2024-09-15T21:34:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:34:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:56.457+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request con 0x7f3ca809dd20 auth_method 0 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca659c640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3ca809d7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3ca809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request con 0x7f3ca8134ba0 auth_method 0 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:34:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:57.261+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:34:58.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient(hunting): tick 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 monclient(hunting): continuing hunt 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3ca809d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3ca809d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 msgr2=0x7f3ca8135050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.119:3300/0 conn(0x7f3ca8134ba0 0x7f3ca8135050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 -- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 msgr2=0x7f3ca8134660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 --2- 172.21.15.17:0/1833376570 >> v2:172.21.15.17:3300/0 conn(0x7f3ca809dd20 0x7f3ca8134660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:34:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient: _add_conns ranks=[1,0] 2024-09-15T21:34:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3c880058d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:34:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c880095e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:34:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 -- 172.21.15.17:0/1833376570 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca80a0320 con 0x7f3c880095e0 2024-09-15T21:34:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 1 -- 172.21.15.17:0/1833376570 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c8800e320 con 0x7f3c880058d0 2024-09-15T21:34:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:34:58.853+0000 7f3c97fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.025+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178161246 0 0) 0x7f3ca80a0320 con 0x7f3c880095e0 2024-09-15T21:35:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.025+0000 7f3ca559a640 10 monclient(hunting): _negotiate 2024-09-15T21:35:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.025+0000 7f3ca559a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.025+0000 7f3ca559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.025+0000 7f3ca559a640 10 monclient(hunting): my global_id is 204180 2024-09-15T21:35:00.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.025+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c90003530 con 0x7f3c880095e0 2024-09-15T21:35:00.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471689302 0 0) 0x7f3c8800e320 con 0x7f3c880058d0 2024-09-15T21:35:00.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): _negotiate 2024-09-15T21:35:00.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): my global_id is 194263 2024-09-15T21:35:00.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca80a0320 con 0x7f3c880058d0 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3528929135 0 0) 0x7f3ca80a0320 con 0x7f3c880058d0 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 >> v1:172.21.15.17:6789/0 conn(0x7f3c880095e0 legacy=0x7f3c8800baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 monclient: found mon.noname-c-legacy 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.119:6789/0 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8800f520 con 0x7f3c880058d0 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f3c9c004290 con 0x7f3c880058d0 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: dump: 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] but i'm connected to v1:172.21.15.119:6789/0, reconnecting 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 >> v1:172.21.15.119:6789/0 conn(0x7f3c880058d0 legacy=0x7f3c88007d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90003ba0 0x7f3c90003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): picked mon.a con 0x7f3c90003ba0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c90004810 0x7f3c90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): picked mon.b con 0x7f3c90004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3c9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): picked mon.c con 0x7f3ca809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90005650 con 0x7f3c90003ba0 2024-09-15T21:35:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9000bf10 con 0x7f3c90004810 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9000c110 con 0x7f3ca809d400 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca559a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 reap_dead start 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca6d9d640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3c9000b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3c9000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90003ba0 0x7f3c90003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): get_auth_request con 0x7f3c90003ba0 auth_method 0 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient(hunting): handle_auth_done global_id 194263 payload 293 2024-09-15T21:35:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 msgr2=0x7f3c9000b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3c9000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c90004810 msgr2=0x7f3c90004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c90004810 0x7f3c90004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 monclient: found mon.a 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 1 -- 172.21.15.17:0/1833376570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca80a0320 con 0x7f3c90003ba0 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.029+0000 7f3ca659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.033+0000 7f3cacf96640 5 monclient: authenticate success, global_id 194263 2024-09-15T21:35:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.033+0000 7f3cacf96640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.037+0000 7f3ca659c640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90003ba0 0x7f3c90003f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00c490 tx=0x7f3c9c01cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bfc97d358ce9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c00c0e0 con 0x7f3c90003ba0 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: dump: 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c0a9020 con 0x7f3c90003ba0 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 1 -- 172.21.15.17:0/1833376570 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0a5b40 con 0x7f3c90003ba0 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3ca559a640 10 monclient: dump: 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 4 set_mon_vals no callback set 2024-09-15T21:35:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 10 monclient: shutdown 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 -- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90003ba0 msgr2=0x7f3c90003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90003ba0 0x7f3c90003f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00c490 tx=0x7f3c9c01cd90 comp rx=0 tx=0).stop 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 -- 172.21.15.17:0/1833376570 shutdown_connections 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c90004810 0x7f3c90004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90003ba0 0x7f3c90003f80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 --2- 172.21.15.17:0/1833376570 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3ca809d400 0x7f3c9000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 -- 172.21.15.17:0/1833376570 >> 172.21.15.17:0/1833376570 conn(0x7f3ca80a66a0 msgr2=0x7f3ca80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 -- 172.21.15.17:0/1833376570 shutdown_connections 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.045+0000 7f3cacf96640 1 -- 172.21.15.17:0/1833376570 wait complete. 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient: build_initial_monmap 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient: monmap: 2024-09-15T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:00.052 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 Processor -- start 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 -- start start 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient: init 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c90003ba0 0x7f3ca811ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): picked mon.noname-c con 0x7f3c90003ba0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 0x7f3ca811f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): picked mon.noname-a con 0x7f3c90004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca809d400 0x7f3ca811f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): picked mon.noname-b con 0x7f3ca809d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca659c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c90003ba0 msgr2=0x7f3ca811ee40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca809e100 con 0x7f3c90004810 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca80a3130 con 0x7f3ca809d400 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca80a5290 con 0x7f3c90003ba0 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c90003ba0 0x7f3ca811ee40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3cacf96640 10 monclient(hunting): authenticate will time out at 2023.446945s 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 0x7f3ca811f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 0x7f3ca811f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53064/0 (socket says 172.21.15.17:53064) 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/3166805823 learned_addr learned my addr 172.21.15.17:0/3166805823 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request con 0x7f3c90004810 auth_method 0 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca6d9d640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca809d400 0x7f3ca811f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request con 0x7f3ca809d400 auth_method 0 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient(hunting): handle_auth_done global_id 204183 payload 824 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c90003ba0 msgr2=0x7f3ca811ee40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c90003ba0 0x7f3ca811ee40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca809d400 msgr2=0x7f3ca811f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca809d400 0x7f3ca811f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 monclient: found mon.noname-a 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.049+0000 7f3ca5d9b640 1 -- 172.21.15.17:0/3166805823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca81228f0 con 0x7f3c90004810 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3ca5d9b640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 0x7f3ca811f380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c98013700 tx=0x7f3c98013bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9801c070 con 0x7f3c90004810 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: dump: 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 5 monclient: authenticate success, global_id 204183 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 monclient: _renew_subs 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c98004220 con 0x7f3c90004810 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca8122bd0 con 0x7f3c90004810 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800f040 con 0x7f3c90004810 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 10 monclient: dump: 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3ca4d99640 4 set_mon_vals no callback set 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 monclient: _renew_subs 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca8122ff0 con 0x7f3c90004810 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3c980107d0 con 0x7f3c90004810 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca809cd60 con 0x7f3c90004810 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803e970 0x7f3c88040e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3c98050aa0 con 0x7f3c90004810 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3ca659c640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803e970 0x7f3c88040e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3ca659c640 10 monclient: get_auth_request con 0x7f3c8803e970 auth_method 0 2024-09-15T21:35:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.053+0000 7f3ca659c640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803e970 0x7f3c88040e30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0aa000 tx=0x7f3c9c0054b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.057+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3c98003e40 con 0x7f3c90004810 2024-09-15T21:35:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.057+0000 7f3c96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.057+0000 7f3c96ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:00.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.305+0000 7f3cacf96640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.305+0000 7f3cacf96640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.305+0000 7f3cacf96640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.305+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3ca809d0f0 con 0x7f3c90004810 2024-09-15T21:35:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3c96ffd640 1 -- 172.21.15.17:0/3166805823 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f3c98003620 con 0x7f3c90004810 2024-09-15T21:35:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3c96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3c96ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:00.311 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":162,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:00.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803e970 msgr2=0x7f3c88040e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803e970 0x7f3c88040e30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0aa000 tx=0x7f3c9c0054b0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 10 monclient: shutdown 2024-09-15T21:35:00.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:00.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 msgr2=0x7f3ca811f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 0x7f3ca811f380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c98013700 tx=0x7f3c98013bc0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 shutdown_connections 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3c8803e970 0x7f3c88040e30 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca809d400 0x7f3ca811f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c90004810 0x7f3ca811f380 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 --2- 172.21.15.17:0/3166805823 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3c90003ba0 0x7f3ca811ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 >> 172.21.15.17:0/3166805823 conn(0x7f3ca80a66a0 msgr2=0x7f3ca80a3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 shutdown_connections 2024-09-15T21:35:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.309+0000 7f3cacf96640 1 -- 172.21.15.17:0/3166805823 wait complete. 2024-09-15T21:35:00.331 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:35:00.331 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:35:00.331 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:35:00.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient: get_monmap_and_config 2024-09-15T21:35:00.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient: build_initial_monmap 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient: monmap: 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:00.450694+0000 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:00.450694+0000 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:00.451 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 Processor -- start 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 -- start start 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient: init 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 0x7fbffc09ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): picked mon.noname-b con 0x7fbffc099600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbffc0999e0 0x7fbffc0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): picked mon.noname-c con 0x7fbffc0999e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbffc099de0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc075200 con 0x7fbffc099de0 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0748c0 con 0x7fbffc0999e0 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc074d80 con 0x7fbffc099600 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fc000970640 10 monclient(hunting): authenticate will time out at 2023.846794s 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffa59c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 msgr2=0x7fbffc09ee60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 0x7fbffc09ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbffc099de0 0x7fbffc0a1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37966/0 (socket says 172.21.15.17:37966) 2024-09-15T21:35:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffad9d640 1 -- 172.21.15.17:0/1652419117 learned_addr learned my addr 172.21.15.17:0/1652419117 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/1652419117 >> v2:172.21.15.119:3300/0 conn(0x7fbffc0999e0 0x7fbffc0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request con 0x7fbffc0999e0 auth_method 0 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850918901 0 0) 0x7fbffc075200 con 0x7fbffc099de0 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): _negotiate 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): my global_id is 204186 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe8003530 con 0x7fbffc099de0 2024-09-15T21:35:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 846654657 0 0) 0x7fbfe8003530 con 0x7fbffc099de0 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 msgr2=0x7fbffc09ee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 0x7fbffc09ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 >> v2:172.21.15.119:3300/0 conn(0x7fbffc0999e0 msgr2=0x7fbffc0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 --2- 172.21.15.17:0/1652419117 >> v2:172.21.15.119:3300/0 conn(0x7fbffc0999e0 0x7fbffc0a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:35:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc09dbc0 con 0x7fbffc099de0 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fbff4001a80 con 0x7fbffc099de0 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: dump: 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 >> v1:172.21.15.17:6789/0 conn(0x7fbffc099de0 legacy=0x7fbffc0a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe80039c0 0x7fbfe8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): picked mon.b con 0x7fbfe80039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbfe8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): picked mon.a con 0x7fbfe8004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 0x7fbfe800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): picked mon.c con 0x7fbffc099600 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0748c0 con 0x7fbfe8004810 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc074d80 con 0x7fbfe80039c0 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe80055d0 con 0x7fbffc099600 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff959a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbfe8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffad9d640 1 -- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 msgr2=0x7fbfe800b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request con 0x7fbfe8004810 auth_method 0 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffad9d640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 0x7fbfe800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.449+0000 7fbffa59c640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe80039c0 0x7fbfe8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbffa59c640 10 monclient(hunting): get_auth_request con 0x7fbfe80039c0 auth_method 0 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbffa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbffa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbffa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_done global_id 204186 payload 293 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 -- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 msgr2=0x7fbfe800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc099600 0x7fbfe800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 -- 172.21.15.17:0/1652419117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe80039c0 msgr2=0x7fbfe8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfe80039c0 0x7fbfe8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 monclient: found mon.a 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 -- 172.21.15.17:0/1652419117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe8003530 con 0x7fbfe8004810 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 5 monclient: authenticate success, global_id 204186 2024-09-15T21:35:00.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbfe8004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009ab0 tx=0x7fbfec01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c115941b5b09f759 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b2070 con 0x7fbfe8004810 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: dump: 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec0a5630 con 0x7fbfe8004810 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 1 -- 172.21.15.17:0/1652419117 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0096b0 con 0x7fbfe8004810 2024-09-15T21:35:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 4 set_mon_vals no callback set 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbff959a640 10 monclient: dump: 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.473 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 10 monclient: shutdown 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 1 -- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 msgr2=0x7fbfe8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 1 --2- 172.21.15.17:0/1652419117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbfe8004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009ab0 tx=0x7fbfec01ca40 comp rx=0 tx=0).stop 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fbffad9d640 1 -- 172.21.15.17:0/1652419117 reap_dead start 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 1 -- 172.21.15.17:0/1652419117 shutdown_connections 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 1 -- 172.21.15.17:0/1652419117 >> 172.21.15.17:0/1652419117 conn(0x7fbffc0a6820 msgr2=0x7fbffc0a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 1 -- 172.21.15.17:0/1652419117 shutdown_connections 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 1 -- 172.21.15.17:0/1652419117 wait complete. 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 10 monclient: build_initial_monmap 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.453+0000 7fc000970640 10 monclient: monmap: 2024-09-15T21:35:00.474 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 Processor -- start 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- start start 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: init 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbffc08d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): picked mon.noname-a con 0x7fbfe8004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbffc08dea0 0x7fbffc08e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): picked mon.noname-b con 0x7fbffc08dea0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc1d4450 0x7fbffc1d6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): picked mon.noname-c con 0x7fbffc1d4450 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0a20c0 con 0x7fbfe8004810 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0a1f40 con 0x7fbffc08dea0 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc1402f0 con 0x7fbffc1d4450 2024-09-15T21:35:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): authenticate will time out at 2023.852609s 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffad9d640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc1d4450 msgr2=0x7fbffc1d6840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc1d4450 0x7fbffc1d6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbffc08d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbffc08d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53080/0 (socket says 172.21.15.17:53080) 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 -- 172.21.15.17:0/75594313 learned_addr learned my addr 172.21.15.17:0/75594313 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): get_auth_request con 0x7fbfe8004810 auth_method 0 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbffc08dea0 0x7fbffc08e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request con 0x7fbffc08dea0 auth_method 0 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient(hunting): handle_auth_done global_id 204189 payload 824 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc1d4450 msgr2=0x7fbffc1d6840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbffc1d4450 0x7fbffc1d6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbffc08dea0 msgr2=0x7fbffc08e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbffc08dea0 0x7fbffc08e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 monclient: found mon.noname-a 2024-09-15T21:35:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc08f3d0 con 0x7fbfe8004810 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbffc08dea0 0x7fbffc08e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffa59c640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbffc08d960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400d7d0 tx=0x7fbfe400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4016030 con 0x7fbfe8004810 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: dump: 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe40039c0 con 0x7fbfe8004810 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 5 monclient: authenticate success, global_id 204189 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: _renew_subs 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc091b50 con 0x7fbfe8004810 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4003b90 con 0x7fbfe8004810 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: dump: 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff3fff640 4 set_mon_vals no callback set 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: _renew_subs 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc090e90 con 0x7fbfe8004810 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 0x7fbffc063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc09d810 con 0x7fbffc02e5d0 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 0x7fbffc063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient: get_auth_request con 0x7fbffc02e5d0 auth_method 0 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbfe4019070 con 0x7fbfe8004810 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_done global_id 204192 payload 824 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbffc02e9b0 con 0x7fbffc02e5d0 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfdc03b970 0x7fbfdc03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fbfe4050180 con 0x7fbfe8004810 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 0x7fbffc063e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbffc06dc80 tx=0x7fbfe8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b2040 con 0x7fbffc02e5d0 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: dump: 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffad9d640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfdc03b970 0x7fbfdc03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbffad9d640 10 monclient: get_auth_request con 0x7fbfdc03b970 auth_method 0 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbfec0b9460 con 0x7fbffc02e5d0 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 msgr2=0x7fbffc063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 0x7fbffc063e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbffc06dc80 tx=0x7fbfe8005840 comp rx=0 tx=0).stop 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 msgr2=0x7fbffc063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.457+0000 7fbff2ffd640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc02e5d0 0x7fbffc063e00 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbffc06dc80 tx=0x7fbfe8005840 comp rx=0 tx=0).stop 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.461+0000 7fbffad9d640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfdc03b970 0x7fbfdc03de30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0a08a0 tx=0x7fbff4001490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 0x7fbffc1dada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0a12f0 con 0x7fbffc064340 2024-09-15T21:35:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 0x7fbffc1dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient: get_auth_request con 0x7fbffc064340 auth_method 0 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 10 monclient(hunting): handle_auth_done global_id 204195 payload 824 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 1 -- 172.21.15.17:0/75594313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbffc13c1c0 con 0x7fbffc064340 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff9d9b640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 0x7fbffc1dada0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009ab0 tx=0x7fbfec0a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0c8070 con 0x7fbffc064340 2024-09-15T21:35:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 10 monclient: dump: 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4703 (secure 0 0 0) 0x7fbfec0c6070 con 0x7fbffc064340 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 msgr2=0x7fbffc1dada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 0x7fbffc1dada0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009ab0 tx=0x7fbfec0a7070 comp rx=0 tx=0).stop 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 msgr2=0x7fbffc1dada0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbff2ffd640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbffc064340 0x7fbffc1dada0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009ab0 tx=0x7fbfec0a7070 comp rx=0 tx=0).stop 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:00.488 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 162, 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 24999, 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:00.489 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.490 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:00.492 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:00.493 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:00.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:00.495 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:00.496 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:00.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:00.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:00.499 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.500 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:00.501 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfdc03b970 msgr2=0x7fbfdc03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbfdc03b970 0x7fbfdc03de30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0a08a0 tx=0x7fbff4001490 comp rx=0 tx=0).stop 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 10 monclient: shutdown 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 msgr2=0x7fbffc08d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 --2- 172.21.15.17:0/75594313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe8004810 0x7fbffc08d960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400d7d0 tx=0x7fbfe400dc90 comp rx=0 tx=0).stop 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbffad9d640 1 -- 172.21.15.17:0/75594313 reap_dead start 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fbffad9d640 1 -- 172.21.15.17:0/75594313 reap_dead start 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 shutdown_connections 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 >> 172.21.15.17:0/75594313 conn(0x7fbffc0a6820 msgr2=0x7fbffc09ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 shutdown_connections 2024-09-15T21:35:00.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.465+0000 7fc000970640 1 -- 172.21.15.17:0/75594313 wait complete. 2024-09-15T21:35:00.504 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:35:00.504 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:35:00.504 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient: get_monmap_and_config 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient: build_initial_monmap 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient: monmap: 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:00.614344+0000 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:00.614344+0000 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:00.614 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.615 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:00.615 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:00.615 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:00.615 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:00.615 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:00.615 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 1 Processor -- start 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 1 -- start start 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient: init 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2de40745d0 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2de413ab60 0x7f2de413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient(hunting): picked mon.noname-c con 0x7f2de413ab60 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2de413d490 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de4069f30 con 0x7f2de413d490 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de406ede0 con 0x7f2de40745d0 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2de409fa10 con 0x7f2de413ab60 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 10 monclient(hunting): authenticate will time out at 2024.009887s 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2de413d490 0x7f2de413f880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37974/0 (socket says 172.21.15.17:37974) 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 1 -- 172.21.15.17:0/3408125583 learned_addr learned my addr 172.21.15.17:0/3408125583 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 --2- 172.21.15.17:0/3408125583 >> v2:172.21.15.119:3300/0 conn(0x7f2de413ab60 0x7f2de413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): get_auth_request con 0x7f2de413ab60 auth_method 0 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971901441 0 0) 0x7f2de4069f30 con 0x7f2de413d490 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _negotiate 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): my global_id is 204198 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd0003530 con 0x7f2de413d490 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066089306 0 0) 0x7f2ddc0014a0 con 0x7f2de40745d0 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _negotiate 2024-09-15T21:35:00.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): my global_id is 194278 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2de4069f30 con 0x7f2de40745d0 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 63202994 0 0) 0x7f2dd0003530 con 0x7f2de413d490 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 >> v2:172.21.15.119:3300/0 conn(0x7f2de413ab60 msgr2=0x7f2de413cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 --2- 172.21.15.17:0/3408125583 >> v2:172.21.15.119:3300/0 conn(0x7f2de413ab60 0x7f2de413cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 >> v1:172.21.15.119:6789/0 conn(0x7f2de40745d0 legacy=0x7f2de40749b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de40a4440 con 0x7f2de413d490 2024-09-15T21:35:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f2dd8001700 con 0x7f2de413d490 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 5 monclient: authenticate success, global_id 204198 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: dump: 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.630 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 >> v1:172.21.15.17:6789/0 conn(0x7f2de413d490 legacy=0x7f2de413f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2dd0003c40 0x7f2dd0004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): picked mon.c con 0x7f2dd0003c40 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dd0004810 0x7f2dd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): picked mon.a con 0x7f2dd0004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd00053a0 0x7f2dd000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): picked mon.b con 0x7f2dd00053a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2de409fa10 con 0x7f2dd0004810 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd000c290 con 0x7f2dd00053a0 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd000c490 con 0x7f2dd0003c40 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2dd7fff640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2deb4e9640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9285640 1 -- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2dd0003c40 msgr2=0x7f2dd0004040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9285640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2dd0003c40 0x7f2dd0004040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dd0004810 0x7f2dd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): get_auth_request con 0x7f2dd0004810 auth_method 0 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd00053a0 0x7f2dd000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 10 monclient(hunting): get_auth_request con 0x7f2dd00053a0 auth_method 0 2024-09-15T21:35:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient(hunting): handle_auth_done global_id 204198 payload 293 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 -- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2dd0003c40 msgr2=0x7f2dd0004040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2dd0003c40 0x7f2dd0004040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 -- 172.21.15.17:0/3408125583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd00053a0 msgr2=0x7f2dd000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd00053a0 0x7f2dd000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 monclient: found mon.a 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 1 -- 172.21.15.17:0/3408125583 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd0003530 con 0x7f2dd0004810 2024-09-15T21:35:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de9a86640 1 -- 172.21.15.17:0/3408125583 reap_dead start 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.613+0000 7f2de8a84640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2de8a84640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dd0004810 0x7f2dd0004c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009ae0 tx=0x7f2de001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceffcb30baef3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de00b2070 con 0x7f2dd0004810 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: dump: 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.635 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de00a55e0 con 0x7f2dd0004810 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 1 -- 172.21.15.17:0/3408125583 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de00096b0 con 0x7f2dd0004810 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2dd7fff640 10 monclient: dump: 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.636 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 4 set_mon_vals no callback set 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: shutdown 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dd0004810 msgr2=0x7f2dd0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dd0004810 0x7f2dd0004c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009ae0 tx=0x7f2de001ca50 comp rx=0 tx=0).stop 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- 172.21.15.17:0/3408125583 shutdown_connections 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 --2- 172.21.15.17:0/3408125583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dd0004810 0x7f2dd0004c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- 172.21.15.17:0/3408125583 >> 172.21.15.17:0/3408125583 conn(0x7f2de40a4c60 msgr2=0x7f2de40a5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- 172.21.15.17:0/3408125583 shutdown_connections 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- 172.21.15.17:0/3408125583 wait complete. 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: build_initial_monmap 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: monmap: 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 Processor -- start 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- start start 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: init 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd0004810 0x7f2de409da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): picked mon.noname-b con 0x7f2dd0004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2de4137af0 0x7f2de4137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): picked mon.noname-c con 0x7f2de4137af0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de41cfae0 0x7f2de41cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): picked mon.noname-a con 0x7f2de41cfae0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4074f40 con 0x7f2de41cfae0 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4074dc0 con 0x7f2dd0004810 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2de40a3c00 con 0x7f2de4137af0 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2deb4e9640 10 monclient(hunting): authenticate will time out at 2024.015912s 2024-09-15T21:35:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2de8a84640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2de4137af0 msgr2=0x7f2de4137ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.617+0000 7f2de8a84640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2de4137af0 0x7f2de4137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de41cfae0 0x7f2de41cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de41cfae0 0x7f2de41cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53120/0 (socket says 172.21.15.17:53120) 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 -- 172.21.15.17:0/4154550229 learned_addr learned my addr 172.21.15.17:0/4154550229 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): get_auth_request con 0x7f2de41cfae0 auth_method 0 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd0004810 0x7f2de409da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): get_auth_request con 0x7f2dd0004810 auth_method 0 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient(hunting): handle_auth_done global_id 204201 payload 824 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2de4137af0 msgr2=0x7f2de4137ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f2de4137af0 0x7f2de4137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd0004810 msgr2=0x7f2de409da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd0004810 0x7f2de409da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:00.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient: _finish_hunting 0 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 monclient: found mon.noname-a 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de409e4f0 con 0x7f2de41cfae0 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9a86640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de41cfae0 0x7f2de41cff50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8000f70 tx=0x7f2dd800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8019070 con 0x7f2de41cfae0 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: dump: 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.643 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd8010510 con 0x7f2de41cfae0 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:00.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8010900 con 0x7f2de41cfae0 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: dump: 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 5 monclient: authenticate success, global_id 204201 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 10 monclient: _renew_subs 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd77fe640 4 set_mon_vals no callback set 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de41255b0 con 0x7f2de41cfae0 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 10 monclient: _renew_subs 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de41384f0 con 0x7f2de41cfae0 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 0x7f2de41d0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2de40a6c20 con 0x7f2de402e5d0 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f2dd801c070 con 0x7f2de41cfae0 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2dc803b9a0 0x7f2dc803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 0x7f2de41d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient: get_auth_request con 0x7f2de402e5d0 auth_method 0 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f2dd804fa50 con 0x7f2de41cfae0 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de8a84640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2dc803b9a0 0x7f2dc803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de8a84640 10 monclient: get_auth_request con 0x7f2dc803b9a0 auth_method 0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 10 monclient(hunting): handle_auth_done global_id 194284 payload 824 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2de402e9b0 con 0x7f2de402e5d0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de9285640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 0x7f2de41d0490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001640 tx=0x7f2ddc007a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2de8a84640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2dc803b9a0 0x7f2dc803de60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2de00af000 tx=0x7f2de0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc003980 con 0x7f2de402e5d0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.621+0000 7f2dd5ffb640 10 monclient: dump: 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.648 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2ddc003980 con 0x7f2de402e5d0 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 msgr2=0x7f2de41d0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 0x7f2de41d0490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001640 tx=0x7f2ddc007a70 comp rx=0 tx=0).stop 2024-09-15T21:35:00.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 msgr2=0x7f2de41d0490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:00.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.625+0000 7f2dd5ffb640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de402e5d0 0x7f2de41d0490 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001640 tx=0x7f2ddc007a70 comp rx=0 tx=0).stop 2024-09-15T21:35:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2deb4e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2deb4e9640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 0x7f2de4064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2deb4e9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2de406f050 con 0x7f2de4063e00 2024-09-15T21:35:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 0x7f2de4064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient: get_auth_request con 0x7f2de4063e00 auth_method 0 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 10 monclient(hunting): handle_auth_done global_id 194287 payload 824 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 1 -- 172.21.15.17:0/4154550229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2de409f180 con 0x7f2de4063e00 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2de9285640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 0x7f2de4064210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001640 tx=0x7f2ddc001260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.629+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc025410 con 0x7f2de4063e00 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:35:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 10 monclient: dump: 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7f2ddc0133f0 con 0x7f2de4063e00 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 msgr2=0x7f2de4064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 0x7f2de4064210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001640 tx=0x7f2ddc001260 comp rx=0 tx=0).stop 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 msgr2=0x7f2de4064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2dd5ffb640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4063e00 0x7f2de4064210 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001640 tx=0x7f2ddc001260 comp rx=0 tx=0).stop 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:35:00.654 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 162, 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 67426, 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:00.655 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:00.656 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:00.657 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:00.658 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:00.659 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:00.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:00.661 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:00.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:00.666 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:00.667 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2dc803b9a0 msgr2=0x7f2dc803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f2dc803b9a0 0x7f2dc803de60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2de00af000 tx=0x7f2de0008040 comp rx=0 tx=0).stop 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 10 monclient: shutdown 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de41cfae0 msgr2=0x7f2de41cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 --2- 172.21.15.17:0/4154550229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de41cfae0 0x7f2de41cff50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8000f70 tx=0x7f2dd800e960 comp rx=0 tx=0).stop 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2de9a86640 1 -- 172.21.15.17:0/4154550229 reap_dead start 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2de9a86640 1 -- 172.21.15.17:0/4154550229 reap_dead start 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 shutdown_connections 2024-09-15T21:35:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 >> 172.21.15.17:0/4154550229 conn(0x7f2de40a4c60 msgr2=0x7f2de40a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:00.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 shutdown_connections 2024-09-15T21:35:00.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:00.633+0000 7f2deb4e9640 1 -- 172.21.15.17:0/4154550229 wait complete. 2024-09-15T21:35:00.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:35:00.670 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:35:20.671 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T21:35:20.671 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:35:20.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:35:20.674 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:35:20.675 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:35:20.675 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:20.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient: get_monmap_and_config 2024-09-15T21:35:20.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient: build_initial_monmap 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient: monmap: 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:20.808662+0000 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:20.808662+0000 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:20.809 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:20.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 Processor -- start 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 -- start start 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient: init 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7001378c0 0x7fd700137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): picked mon.noname-c con 0x7fd7001378c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd7001381e0 0x7fd7000a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): picked mon.noname-a con 0x7fd7001381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd7000a5040 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7000a1d00 con 0x7fd7000a5040 2024-09-15T21:35:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd7000a1fe0 con 0x7fd7001381e0 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd7000a22c0 con 0x7fd7001378c0 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd7079c0640 10 monclient(hunting): authenticate will time out at 2044.203800s 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.804+0000 7fd704f5b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd7001381e0 0x7fd7000a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd7001381e0 0x7fd7000a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48872/0 (socket says 172.21.15.17:48872) 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 -- 172.21.15.17:0/4146275961 learned_addr learned my addr 172.21.15.17:0/4146275961 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): get_auth_request con 0x7fd7001381e0 auth_method 0 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:20.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd70575c640 1 --2- 172.21.15.17:0/4146275961 >> v2:172.21.15.119:3300/0 conn(0x7fd7001378c0 0x7fd700137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd70575c640 10 monclient(hunting): get_auth_request con 0x7fd7001378c0 auth_method 0 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd70575c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd70575c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd70575c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient(hunting): handle_auth_done global_id 204204 payload 274 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 -- 172.21.15.17:0/4146275961 >> v2:172.21.15.119:3300/0 conn(0x7fd7001378c0 msgr2=0x7fd700137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 --2- 172.21.15.17:0/4146275961 >> v2:172.21.15.119:3300/0 conn(0x7fd7001378c0 0x7fd700137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 -- 172.21.15.17:0/4146275961 >> v1:172.21.15.119:6789/0 conn(0x7fd7000a5040 legacy=0x7fd7000a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient: _finish_hunting 0 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 monclient: found mon.noname-a 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:20.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 -- 172.21.15.17:0/4146275961 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7000a4490 con 0x7fd7001381e0 2024-09-15T21:35:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd704f5b640 1 --2- 172.21.15.17:0/4146275961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd7000a4b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009870 tx=0x7fd6f001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31354b427f26380 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 1 -- 172.21.15.17:0/4146275961 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00b1070 con 0x7fd7001381e0 2024-09-15T21:35:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: dump: 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: _finish_auth 0 2024-09-15T21:35:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 1 -- 172.21.15.17:0/4146275961 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f00ac620 con 0x7fd7001381e0 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 5 monclient: authenticate success, global_id 204204 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 1 -- 172.21.15.17:0/4146275961 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00ac9f0 con 0x7fd7001381e0 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 4 set_mon_vals no callback set 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd6f7fff640 10 monclient: dump: 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 10 monclient: shutdown 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 -- 172.21.15.17:0/4146275961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 msgr2=0x7fd7000a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 --2- 172.21.15.17:0/4146275961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd7000a4b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009870 tx=0x7fd6f001c3e0 comp rx=0 tx=0).stop 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 -- 172.21.15.17:0/4146275961 shutdown_connections 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 --2- 172.21.15.17:0/4146275961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd7000a4b00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 --2- 172.21.15.17:0/4146275961 >> v2:172.21.15.119:3300/0 conn(0x7fd7001378c0 0x7fd700137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 -- 172.21.15.17:0/4146275961 >> 172.21.15.17:0/4146275961 conn(0x7fd7000aac40 msgr2=0x7fd7000a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 -- 172.21.15.17:0/4146275961 shutdown_connections 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 1 -- 172.21.15.17:0/4146275961 wait complete. 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 10 monclient: build_initial_monmap 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.808+0000 7fd7079c0640 10 monclient: monmap: 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:20.823 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 Processor -- start 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- start start 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: init 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7000a5040 0x7fd7001cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): picked mon.noname-b con 0x7fd7000a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 0x7fd7001d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): picked mon.noname-c con 0x7fd7001378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd70013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:20.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): picked mon.noname-a con 0x7fd7001381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd70006f340 con 0x7fd7001381e0 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd70006f1c0 con 0x7fd7000a5040 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd7001425c0 con 0x7fd7001378c0 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient(hunting): authenticate will time out at 2044.208752s 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd70013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 0x7fd7001d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd70013cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48876/0 (socket says 172.21.15.17:48876) 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 -- 172.21.15.17:0/282799486 learned_addr learned my addr 172.21.15.17:0/282799486 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): get_auth_request con 0x7fd7001381e0 auth_method 0 2024-09-15T21:35:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 10 monclient(hunting): get_auth_request con 0x7fd7001378c0 auth_method 0 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7000a5040 0x7fd7001cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 10 monclient(hunting): get_auth_request con 0x7fd7000a5040 auth_method 0 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 1 -- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 msgr2=0x7fd7001d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 1 -- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 msgr2=0x7fd7001d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 0x7fd7001d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd704f5b640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 0x7fd7001d0140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient(hunting): handle_auth_done global_id 204207 payload 824 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 -- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 msgr2=0x7fd7001d0140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 0x7fd7001d0140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 -- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7000a5040 msgr2=0x7fd7001cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7000a5040 0x7fd7001cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient: _finish_hunting 0 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 monclient: found mon.noname-a 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:20.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 -- 172.21.15.17:0/282799486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7001215c0 con 0x7fd7001381e0 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7000a5040 0x7fd7001cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd705f5d640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd70013cdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c780 tx=0x7fd6f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8004b70 con 0x7fd7001381e0 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: dump: 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:20.828 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f8004d10 con 0x7fd7001381e0 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 5 monclient: authenticate success, global_id 204207 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _renew_subs 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70009ff20 con 0x7fd7001381e0 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800aa90 con 0x7fd7001381e0 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 10 monclient: dump: 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:20.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _renew_subs 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f77fe640 4 set_mon_vals no callback set 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd700121000 con 0x7fd7001381e0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd700137ca0 con 0x7fd7001381e0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd6f800ac30 con 0x7fd7001381e0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd6f80508a0 con 0x7fd7001381e0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:20.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.812+0000 7fd70575c640 10 monclient: get_auth_request con 0x7fd6e403b9f0 auth_method 0 2024-09-15T21:35:20.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.816+0000 7fd70575c640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc007a70 tx=0x7fd6fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:20.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.816+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd6f8020070 con 0x7fd7001381e0 2024-09-15T21:35:20.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.816+0000 7fd6f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:20.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:20.816+0000 7fd6f5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:21.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd7079c0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:21.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd7079c0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:21.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd7079c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:21.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd700137270 con 0x7fd7001381e0 2024-09-15T21:35:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd6f5ffb640 1 -- 172.21.15.17:0/282799486 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fd6f8005210 con 0x7fd7001381e0 2024-09-15T21:35:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd6f5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.052+0000 7fd6f5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:21.056 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":162,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 msgr2=0x7fd6e403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc007a70 tx=0x7fd6fc008040 comp rx=0 tx=0).stop 2024-09-15T21:35:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 10 monclient: shutdown 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 msgr2=0x7fd70013cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd70013cdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c780 tx=0x7fd6f800cc40 comp rx=0 tx=0).stop 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 shutdown_connections 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6e403b9f0 0x7fd6e403deb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001381e0 0x7fd70013cdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd7001378c0 0x7fd7001d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 --2- 172.21.15.17:0/282799486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7000a5040 0x7fd7001cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 >> 172.21.15.17:0/282799486 conn(0x7fd7000aac40 msgr2=0x7fd70006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 shutdown_connections 2024-09-15T21:35:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:21.056+0000 7fd7079c0640 1 -- 172.21.15.17:0/282799486 wait complete. 2024-09-15T21:35:24.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient: get_monmap_and_config 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient: build_initial_monmap 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient: monmap: 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:24.207974+0000 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:24.207974+0000 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:24.208 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:24.209 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:24.209 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:24.209 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:24.209 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:24.209 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:24.209 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 Processor -- start 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 -- start start 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient: init 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0acc1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0acc1383b0 0x7f0acc13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): picked mon.noname-c con 0x7f0acc1383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0acc13d2c0 0x7f0acc13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): picked mon.noname-a con 0x7f0acc13d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0acc0a1a20 con 0x7f0acc1378c0 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0acc0a1d00 con 0x7f0acc13d2c0 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0acc0a1fe0 con 0x7f0acc1383b0 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0ad1d50640 10 monclient(hunting): authenticate will time out at 2047.603454s 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0acc1378c0 0x7f0acc137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38050/0 (socket says 172.21.15.17:38050) 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acb7fe640 1 -- 172.21.15.17:0/476177008 learned_addr learned my addr 172.21.15.17:0/476177008 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 1 --2- 172.21.15.17:0/476177008 >> v2:172.21.15.17:3300/0 conn(0x7f0acc13d2c0 0x7f0acc13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): get_auth_request con 0x7f0acc13d2c0 auth_method 0 2024-09-15T21:35:24.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acaffd640 1 --2- 172.21.15.17:0/476177008 >> v2:172.21.15.119:3300/0 conn(0x7f0acc1383b0 0x7f0acc13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acaffd640 10 monclient(hunting): get_auth_request con 0x7f0acc1383b0 auth_method 0 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.204+0000 7f0acaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 10 monclient(hunting): handle_auth_done global_id 204210 payload 274 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 1 -- 172.21.15.17:0/476177008 >> v2:172.21.15.119:3300/0 conn(0x7f0acc1383b0 msgr2=0x7f0acc13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 1 --2- 172.21.15.17:0/476177008 >> v2:172.21.15.119:3300/0 conn(0x7f0acc1383b0 0x7f0acc13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 1 -- 172.21.15.17:0/476177008 >> v1:172.21.15.17:6789/0 conn(0x7f0acc1378c0 legacy=0x7f0acc137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 1 monclient: found mon.noname-a 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 1 -- 172.21.15.17:0/476177008 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc0a22c0 con 0x7f0acc13d2c0 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 1 -- 172.21.15.17:0/476177008 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423866062 0 0) 0x7f0abc001120 con 0x7f0acc1378c0 2024-09-15T21:35:24.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0acbfff640 1 --2- 172.21.15.17:0/476177008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc13f6b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009d90 tx=0x7f0ac401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a029f2c9c37cf68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 1 -- 172.21.15.17:0/476177008 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40b1070 con 0x7f0acc13d2c0 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: dump: 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:24.219 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: _finish_auth 0 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 1 -- 172.21.15.17:0/476177008 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac40a4d90 con 0x7f0acc13d2c0 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 1 -- 172.21.15.17:0/476177008 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40ac660 con 0x7f0acc13d2c0 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 5 monclient: authenticate success, global_id 204210 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 4 set_mon_vals no callback set 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:24.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0aca7fc640 10 monclient: dump: 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:24.221 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 10 monclient: shutdown 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 -- 172.21.15.17:0/476177008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 msgr2=0x7f0acc13f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/476177008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc13f6b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009d90 tx=0x7f0ac401c4d0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 -- 172.21.15.17:0/476177008 shutdown_connections 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/476177008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc13f6b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/476177008 >> v2:172.21.15.119:3300/0 conn(0x7f0acc1383b0 0x7f0acc13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 -- 172.21.15.17:0/476177008 >> 172.21.15.17:0/476177008 conn(0x7f0acc0aac40 msgr2=0x7f0acc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 -- 172.21.15.17:0/476177008 shutdown_connections 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 1 -- 172.21.15.17:0/476177008 wait complete. 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 10 monclient: build_initial_monmap 2024-09-15T21:35:24.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.208+0000 7f0ad1d50640 10 monclient: monmap: 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 Processor -- start 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- start start 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: init 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): picked mon.noname-c con 0x7f0acc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0acc1383b0 0x7f0acc136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): picked mon.noname-b con 0x7f0acc1383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc09dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): picked mon.noname-a con 0x7f0acc13d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc143020 con 0x7f0acc13d2c0 2024-09-15T21:35:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc142ea0 con 0x7f0acc1383b0 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0acc1431a0 con 0x7f0acc1378c0 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient(hunting): authenticate will time out at 2047.608777s 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc09dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc09dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48912/0 (socket says 172.21.15.17:48912) 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:43064/0 (socket says 172.21.15.17:43064) 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 -- 172.21.15.17:0/3712650751 learned_addr learned my addr 172.21.15.17:0/3712650751 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): get_auth_request con 0x7f0acc13d2c0 auth_method 0 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 10 monclient(hunting): get_auth_request con 0x7f0acc1378c0 auth_method 0 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 -- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 msgr2=0x7f0acc1d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 -- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 msgr2=0x7f0acc1d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acaffd640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0acc1383b0 0x7f0acc136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acaffd640 10 monclient(hunting): get_auth_request con 0x7f0acc1383b0 auth_method 0 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient(hunting): handle_auth_done global_id 204216 payload 824 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 -- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 msgr2=0x7f0acc1d07c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 -- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0acc1383b0 msgr2=0x7f0acc136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0acc1383b0 0x7f0acc136ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 monclient: found mon.noname-a 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 -- 172.21.15.17:0/3712650751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc121670 con 0x7f0acc13d2c0 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acbfff640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc09dd00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40af040 tx=0x7f0ac4019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40b1050 con 0x7f0acc13d2c0 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: dump: 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:24.228 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: _finish_auth 0 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 5 monclient: authenticate success, global_id 204216 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac40bd070 con 0x7f0acc13d2c0 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _renew_subs 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc0a12f0 con 0x7f0acc13d2c0 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4009450 con 0x7f0acc13d2c0 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 10 monclient: dump: 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _renew_subs 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1d1640 con 0x7f0acc13d2c0 2024-09-15T21:35:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ac97fa640 4 set_mon_vals no callback set 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0ac40c2460 con 0x7f0acc13d2c0 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc137ca0 con 0x7f0acc13d2c0 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0ab803b9c0 0x7f0ab803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0ac40eabf0 con 0x7f0acc13d2c0 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0ab803b9c0 0x7f0ab803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.212+0000 7f0acb7fe640 10 monclient: get_auth_request con 0x7f0ab803b9c0 auth_method 0 2024-09-15T21:35:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.216+0000 7f0acb7fe640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0ab803b9c0 0x7f0ab803de80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007fd0 tx=0x7f0abc0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:24.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.216+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0ac4016500 con 0x7f0acc13d2c0 2024-09-15T21:35:24.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.216+0000 7f0ab3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:24.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.216+0000 7f0ab3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ad1d50640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ad1d50640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ad1d50640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0acc069270 con 0x7f0acc13d2c0 2024-09-15T21:35:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ab3fff640 1 -- 172.21.15.17:0/3712650751 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f0ac40aae00 con 0x7f0acc13d2c0 2024-09-15T21:35:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ab3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.452+0000 7f0ab3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:24.458 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":162,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0ab803b9c0 msgr2=0x7f0ab803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0ab803b9c0 0x7f0ab803de80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007fd0 tx=0x7f0abc0079c0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 10 monclient: shutdown 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 msgr2=0x7f0acc09dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc09dd00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40af040 tx=0x7f0ac4019590 comp rx=0 tx=0).stop 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 shutdown_connections 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0ab803b9c0 0x7f0ab803de80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0acc13d2c0 0x7f0acc09dd00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0acc1383b0 0x7f0acc136ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 --2- 172.21.15.17:0/3712650751 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0acc1378c0 0x7f0acc1d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 >> 172.21.15.17:0/3712650751 conn(0x7f0acc0aac40 msgr2=0x7f0acc0a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 shutdown_connections 2024-09-15T21:35:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:24.456+0000 7f0ad1d50640 1 -- 172.21.15.17:0/3712650751 wait complete. 2024-09-15T21:35:27.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient: get_monmap_and_config 2024-09-15T21:35:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient: build_initial_monmap 2024-09-15T21:35:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient: monmap: 2024-09-15T21:35:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:27.612090+0000 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:27.612090+0000 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:27.613 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 1 Processor -- start 2024-09-15T21:35:27.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 1 -- start start 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient: init 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc2901378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2901383b0 0x7fc29013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient(hunting): picked mon.noname-c con 0x7fc2901383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc29013d2c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2900a1a20 con 0x7fc29013d2c0 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2900a1d00 con 0x7fc2901378c0 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc2900a1fe0 con 0x7fc2901383b0 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc2980e8640 10 monclient(hunting): authenticate will time out at 2051.007493s 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc296685640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc29013d2c0 0x7fc29013f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38052/0 (socket says 172.21.15.17:38052) 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc296685640 1 -- 172.21.15.17:0/1141340905 learned_addr learned my addr 172.21.15.17:0/1141340905 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc295683640 1 --2- 172.21.15.17:0/1141340905 >> v2:172.21.15.119:3300/0 conn(0x7fc2901383b0 0x7fc29013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc295683640 10 monclient(hunting): get_auth_request con 0x7fc2901383b0 auth_method 0 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc295683640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc295683640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc295683640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.608+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051359271 0 0) 0x7fc280001120 con 0x7fc29013d2c0 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): _negotiate 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): my global_id is 204219 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc278003530 con 0x7fc29013d2c0 2024-09-15T21:35:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3931579628 0 0) 0x7fc278003530 con 0x7fc29013d2c0 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 >> v2:172.21.15.119:3300/0 conn(0x7fc2901383b0 msgr2=0x7fc29013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 --2- 172.21.15.17:0/1141340905 >> v2:172.21.15.119:3300/0 conn(0x7fc2901383b0 0x7fc29013cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 >> v1:172.21.15.119:6789/0 conn(0x7fc2901378c0 legacy=0x7fc290137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: _finish_hunting 0 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2900a22c0 con 0x7fc29013d2c0 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: _finish_auth 0 2024-09-15T21:35:27.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: _check_auth_tickets 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fc2800018d0 con 0x7fc29013d2c0 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: dump: 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 >> v1:172.21.15.17:6789/0 conn(0x7fc29013d2c0 legacy=0x7fc29013f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278003a90 0x7fc278003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): picked mon.b con 0x7fc278003a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc278004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): picked mon.a con 0x7fc278004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2780053d0 0x7fc27800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): picked mon.c con 0x7fc2780053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc2900a1fe0 con 0x7fc278004810 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc27800c270 con 0x7fc278003a90 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc27800c470 con 0x7fc2780053d0 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc278004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2780053d0 0x7fc27800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): get_auth_request con 0x7fc278004810 auth_method 0 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): get_auth_request con 0x7fc2780053d0 auth_method 0 2024-09-15T21:35:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278003a90 0x7fc278003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): get_auth_request con 0x7fc278003a90 auth_method 0 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295e84640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient(hunting): handle_auth_done global_id 204219 payload 293 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 -- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2780053d0 msgr2=0x7fc27800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc2780053d0 0x7fc27800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 -- 172.21.15.17:0/1141340905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278003a90 msgr2=0x7fc278003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278003a90 0x7fc278003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient: _finish_hunting 0 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 monclient: found mon.a 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 -- 172.21.15.17:0/1141340905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc278003530 con 0x7fc278004810 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient: _finish_auth 0 2024-09-15T21:35:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 10 monclient: _check_auth_tickets 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 5 monclient: authenticate success, global_id 204219 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc296685640 1 -- 172.21.15.17:0/1141340905 reap_dead start 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc295683640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc278004c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc288009fc0 tx=0x7fc28801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ab2560c199da5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2880b0070 con 0x7fc278004810 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: dump: 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2880ab600 con 0x7fc278004810 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 4 set_mon_vals no callback set 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 1 -- 172.21.15.17:0/1141340905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288009c10 con 0x7fc278004810 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc294e82640 10 monclient: dump: 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 10 monclient: shutdown 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 -- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 msgr2=0x7fc278004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc278004c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc288009fc0 tx=0x7fc28801c940 comp rx=0 tx=0).stop 2024-09-15T21:35:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 -- 172.21.15.17:0/1141340905 shutdown_connections 2024-09-15T21:35:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 --2- 172.21.15.17:0/1141340905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc278004c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 -- 172.21.15.17:0/1141340905 >> 172.21.15.17:0/1141340905 conn(0x7fc2900aac40 msgr2=0x7fc2900a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 -- 172.21.15.17:0/1141340905 shutdown_connections 2024-09-15T21:35:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.612+0000 7fc2980e8640 1 -- 172.21.15.17:0/1141340905 wait complete. 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: build_initial_monmap 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: monmap: 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 Processor -- start 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- start start 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: init 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc2901d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): picked mon.noname-a con 0x7fc278004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): picked mon.noname-c con 0x7fc290136ac0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2901246a0 0x7fc290124b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): picked mon.noname-b con 0x7fc2901246a0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc29013b580 con 0x7fc278004810 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2900a5940 con 0x7fc2901246a0 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc29013b700 con 0x7fc290136ac0 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient(hunting): authenticate will time out at 2051.013353s 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc2901d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60124/0 (socket says 172.21.15.17:60124) 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 -- 172.21.15.17:0/3652531656 learned_addr learned my addr 172.21.15.17:0/3652531656 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 10 monclient(hunting): get_auth_request con 0x7fc290136ac0 auth_method 0 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): get_auth_request con 0x7fc278004810 auth_method 0 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 -- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 msgr2=0x7fc290136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 -- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 msgr2=0x7fc290136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:27.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc296685640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2901246a0 0x7fc290124b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient(hunting): handle_auth_done global_id 204222 payload 824 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 -- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 msgr2=0x7fc290136f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 -- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2901246a0 msgr2=0x7fc290124b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2901246a0 0x7fc290124b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient: _finish_hunting 0 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 monclient: found mon.noname-a 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 -- 172.21.15.17:0/3652531656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2901281e0 con 0x7fc278004810 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc296685640 10 monclient: get_auth_request con 0x7fc2901246a0 auth_method 0 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc296685640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2901246a0 0x7fc290124b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:35:27.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295e84640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc2901d18e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc28400f5f0 tx=0x7fc28400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28401a070 con 0x7fc278004810 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: dump: 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:27.638 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc284005320 con 0x7fc278004810 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 5 monclient: authenticate success, global_id 204222 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _renew_subs 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2901284c0 con 0x7fc278004810 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284006a20 con 0x7fc278004810 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 10 monclient: dump: 2024-09-15T21:35:27.639 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _renew_subs 2024-09-15T21:35:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc290126160 con 0x7fc278004810 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28ffff640 4 set_mon_vals no callback set 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29013ba90 con 0x7fc278004810 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc28401d070 con 0x7fc278004810 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc27c03b9c0 0x7fc27c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc28404fcf0 con 0x7fc278004810 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc27c03b9c0 0x7fc27c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.616+0000 7fc295683640 10 monclient: get_auth_request con 0x7fc27c03b9c0 auth_method 0 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.620+0000 7fc295683640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc27c03b9c0 0x7fc27c03de80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc288000f90 tx=0x7fc288004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.620+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fc284007570 con 0x7fc278004810 2024-09-15T21:35:27.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.620+0000 7fc28e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.620+0000 7fc28e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc2980e8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc2980e8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc2980e8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc29013cf40 con 0x7fc278004810 2024-09-15T21:35:27.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc28e7fc640 1 -- 172.21.15.17:0/3652531656 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fc284007050 con 0x7fc278004810 2024-09-15T21:35:27.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc28e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:27.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.852+0000 7fc28e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:27.858 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:27.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc27c03b9c0 msgr2=0x7fc27c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:27.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc27c03b9c0 0x7fc27c03de80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc288000f90 tx=0x7fc288004ac0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 10 monclient: shutdown 2024-09-15T21:35:27.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:27.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 msgr2=0x7fc2901d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:27.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc2901d18e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc28400f5f0 tx=0x7fc28400fab0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 shutdown_connections 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc27c03b9c0 0x7fc27c03de80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2901246a0 0x7fc290124b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc290136ac0 0x7fc290136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 --2- 172.21.15.17:0/3652531656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc278004810 0x7fc2901d18e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 >> 172.21.15.17:0/3652531656 conn(0x7fc2900aac40 msgr2=0x7fc29006df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 shutdown_connections 2024-09-15T21:35:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.856+0000 7fc2980e8640 1 -- 172.21.15.17:0/3652531656 wait complete. 2024-09-15T21:35:27.876 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:35:27.876 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:35:27.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:35:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient: get_monmap_and_config 2024-09-15T21:35:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient: build_initial_monmap 2024-09-15T21:35:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient: monmap: 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:27.990489+0000 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:27.990489+0000 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:27.991 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 Processor -- start 2024-09-15T21:35:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 -- start start 2024-09-15T21:35:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient: init 2024-09-15T21:35:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11741378c0 0x7f1174137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): picked mon.noname-a con 0x7f11741378c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11741381e0 0x7f11740a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): picked mon.noname-c con 0x7f11741381e0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): picked mon.noname-b con 0x7f11740a5040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f11740a1d00 con 0x7f11741378c0 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f11740a1fe0 con 0x7f11741381e0 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11740a22c0 con 0x7f11740a5040 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f117a238640 10 monclient(hunting): authenticate will time out at 2051.385908s 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11741378c0 0x7f1174137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11741378c0 0x7f1174137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38336/0 (socket says 172.21.15.17:38336) 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60126/0 (socket says 172.21.15.17:60126) 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 -- 172.21.15.17:0/1426669986 learned_addr learned my addr 172.21.15.17:0/1426669986 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): get_auth_request con 0x7f11741378c0 auth_method 0 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 10 monclient(hunting): get_auth_request con 0x7f11740a5040 auth_method 0 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 1 -- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 msgr2=0x7f11740a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 1 -- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 msgr2=0x7f11740a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1172ffd640 1 --2- 172.21.15.17:0/1426669986 >> v2:172.21.15.119:3300/0 conn(0x7f11741381e0 0x7f11740a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1173fff640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1172ffd640 10 monclient(hunting): get_auth_request con 0x7f11741381e0 auth_method 0 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1172ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1172ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f1172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient(hunting): handle_auth_done global_id 204225 payload 274 2024-09-15T21:35:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 -- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 msgr2=0x7f11740a54d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 -- 172.21.15.17:0/1426669986 >> v2:172.21.15.119:3300/0 conn(0x7f11741381e0 msgr2=0x7f11740a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 --2- 172.21.15.17:0/1426669986 >> v2:172.21.15.119:3300/0 conn(0x7f11741381e0 0x7f11740a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient: _finish_hunting 0 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 monclient: found mon.noname-a 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.988+0000 7f11737fe640 1 -- 172.21.15.17:0/1426669986 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11740a4490 con 0x7f11741378c0 2024-09-15T21:35:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11737fe640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174137ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f116801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3b06ae077c4e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 1 -- 172.21.15.17:0/1426669986 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680b1070 con 0x7f11741378c0 2024-09-15T21:35:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: dump: 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: _finish_auth 0 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: _check_auth_tickets 2024-09-15T21:35:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 5 monclient: authenticate success, global_id 204225 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 1 -- 172.21.15.17:0/1426669986 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11680ac620 con 0x7f11741378c0 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 4 set_mon_vals no callback set 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 1 -- 172.21.15.17:0/1426669986 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680ac9f0 con 0x7f11741378c0 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11727fc640 10 monclient: dump: 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.004 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: shutdown 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 msgr2=0x7f1174137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174137ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f116801c3e0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1426669986 shutdown_connections 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- 172.21.15.17:0/1426669986 >> v2:172.21.15.119:3300/0 conn(0x7f11741381e0 0x7f11740a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11740a5040 0x7f11740a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- 172.21.15.17:0/1426669986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174137ca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1426669986 >> 172.21.15.17:0/1426669986 conn(0x7f11740aac40 msgr2=0x7f11740a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1426669986 shutdown_connections 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1426669986 wait complete. 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: build_initial_monmap 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: monmap: 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:28.006 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 Processor -- start 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- start start 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: init 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11740a5040 0x7f11741cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): picked mon.noname-b con 0x7f11740a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174136d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): picked mon.noname-a con 0x7f11741378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 0x7f117413bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): picked mon.noname-c con 0x7f11741381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1174141730 con 0x7f11741378c0 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11741415b0 con 0x7f11740a5040 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f11741418b0 con 0x7f11741381e0 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient(hunting): authenticate will time out at 2051.390915s 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 0x7f117413bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 0x7f117413bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60136/0 (socket says 172.21.15.17:60136) 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): get_auth_request con 0x7f11741378c0 auth_method 0 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 -- 172.21.15.17:0/1319938704 learned_addr learned my addr 172.21.15.17:0/1319938704 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 10 monclient(hunting): get_auth_request con 0x7f11741381e0 auth_method 0 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 msgr2=0x7f117413bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 msgr2=0x7f117413bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 0x7f117413bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1173fff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 0x7f117413bf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient(hunting): handle_auth_done global_id 204228 payload 824 2024-09-15T21:35:28.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f11737fe640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11740a5040 0x7f11741cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 msgr2=0x7f117413bf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f11741381e0 0x7f117413bf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11740a5040 msgr2=0x7f11741cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11740a5040 0x7f11741cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 monclient: found mon.noname-a 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117413da20 con 0x7f11741378c0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1172ffd640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174136d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f116400d620 tx=0x7f116400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400b620 con 0x7f11741378c0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:28.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: dump: 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: _finish_auth 0 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:28.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164019070 con 0x7f11741378c0 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400ba00 con 0x7f11741378c0 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f115bfff640 10 monclient: dump: 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.012 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 5 monclient: authenticate success, global_id 204228 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: _renew_subs 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f1171ffb640 4 set_mon_vals no callback set 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1174122730 con 0x7f11741378c0 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: _renew_subs 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.992+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1174122170 con 0x7f11741378c0 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f117a238640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f117a238640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 0x7f11741372d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11741d0360 con 0x7f1174120f30 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 0x7f11741372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient: get_auth_request con 0x7f1174120f30 auth_method 0 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 10 monclient(hunting): handle_auth_done global_id 204231 payload 824 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f116400bba0 con 0x7f11741378c0 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1174137ca0 con 0x7f1174120f30 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f11737fe640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 0x7f11741372d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11740ab150 tx=0x7f116801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f116003ba20 0x7f116003dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f11640509a0 con 0x7f11741378c0 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680bb030 con 0x7f1174120f30 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 10 monclient: dump: 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.015 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f11680b7420 con 0x7f1174120f30 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 msgr2=0x7f11741372d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 0x7f11741372d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11740ab150 tx=0x7f116801ac20 comp rx=0 tx=0).stop 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 msgr2=0x7f11741372d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f115bfff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1174120f30 0x7f11741372d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f1173fff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f116003ba20 0x7f116003dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f1173fff640 10 monclient: get_auth_request con 0x7f116003ba20 auth_method 0 2024-09-15T21:35:28.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:27.996+0000 7f1173fff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f116003ba20 0x7f116003dee0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f116c0095e0 tx=0x7f116c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f117a238640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f117a238640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 0x7f117413c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f117a238640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f117413cdf0 con 0x7f117413c4a0 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 0x7f117413c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient: get_auth_request con 0x7f117413c4a0 auth_method 0 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 10 monclient(hunting): handle_auth_done global_id 204234 payload 824 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 1 -- 172.21.15.17:0/1319938704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f117406cba0 con 0x7f117413c4a0 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f11737fe640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 0x7f117413c8b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f1168019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680c5070 con 0x7f117413c4a0 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f115bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f115bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.000+0000 7f115bfff640 10 monclient: dump: 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.019 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f11680c1a10 con 0x7f117413c4a0 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 msgr2=0x7f117413c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 0x7f117413c8b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f1168019180 comp rx=0 tx=0).stop 2024-09-15T21:35:28.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 msgr2=0x7f117413c8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f115bfff640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117413c4a0 0x7f117413c8b0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f1168019180 comp rx=0 tx=0).stop 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 166, 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 52535, 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:28.021 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:28.022 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:28.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:28.024 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:28.025 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.026 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:28.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:28.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:28.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:28.032 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:28.033 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f116003ba20 msgr2=0x7f116003dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f116003ba20 0x7f116003dee0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f116c0095e0 tx=0x7f116c00b040 comp rx=0 tx=0).stop 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 10 monclient: shutdown 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 msgr2=0x7f1174136d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 --2- 172.21.15.17:0/1319938704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11741378c0 0x7f1174136d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f116400d620 tx=0x7f116400dae0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f1173fff640 1 -- 172.21.15.17:0/1319938704 reap_dead start 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f1173fff640 1 -- 172.21.15.17:0/1319938704 reap_dead start 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 shutdown_connections 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 >> 172.21.15.17:0/1319938704 conn(0x7f11740aac40 msgr2=0x7f117413dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 shutdown_connections 2024-09-15T21:35:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.004+0000 7f117a238640 1 -- 172.21.15.17:0/1319938704 wait complete. 2024-09-15T21:35:28.036 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:35:28.036 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:35:28.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient: get_monmap_and_config 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient: build_initial_monmap 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient: monmap: 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:28.150271+0000 2024-09-15T21:35:28.150 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:28.150271+0000 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:28.151 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 1 Processor -- start 2024-09-15T21:35:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 1 -- start start 2024-09-15T21:35:28.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient: init 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb940a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient(hunting): picked mon.noname-b con 0x7feb940a28b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feb940a31d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7feb94135d80 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9409e410 con 0x7feb940a31d0 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb940a1330 con 0x7feb94135d80 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb9409ef80 con 0x7feb940a28b0 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 10 monclient(hunting): authenticate will time out at 2051.545826s 2024-09-15T21:35:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb940a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7feb940a31d0 0x7feb94135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58250/0 (socket says 172.21.15.17:58250) 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 -- 172.21.15.17:0/1563185458 learned_addr learned my addr 172.21.15.17:0/1563185458 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): get_auth_request con 0x7feb940a28b0 auth_method 0 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 1 -- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 msgr2=0x7feb940a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 1 -- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 msgr2=0x7feb940a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb940a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb940a2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799710897 0 0) 0x7feb9409e410 con 0x7feb940a31d0 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): _negotiate 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): my global_id is 204237 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb80003530 con 0x7feb940a31d0 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3781838725 0 0) 0x7feb80003530 con 0x7feb940a31d0 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 msgr2=0x7feb940a2c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb940a2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 >> v1:172.21.15.119:6789/0 conn(0x7feb94135d80 legacy=0x7feb9413dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: _finish_hunting 0 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9406d0d0 con 0x7feb940a31d0 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: _finish_auth 0 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:35:28.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 5 monclient: authenticate success, global_id 204237 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7feb7c001700 con 0x7feb940a31d0 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: dump: 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.166 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 >> v1:172.21.15.17:6789/0 conn(0x7feb940a31d0 legacy=0x7feb94135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb80003a40 0x7feb80003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): picked mon.b con 0x7feb80003a40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): picked mon.a con 0x7feb80004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): picked mon.c con 0x7feb940a28b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:28.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb9409ef80 con 0x7feb80004810 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb800055f0 con 0x7feb80003a40 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb8000bf10 con 0x7feb940a28b0 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb91d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb991b0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): get_auth_request con 0x7feb80004810 auth_method 0 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): get_auth_request con 0x7feb940a28b0 auth_method 0 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb80003a40 0x7feb80003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): get_auth_request con 0x7feb80003a40 auth_method 0 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:28.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb92d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient(hunting): handle_auth_done global_id 204237 payload 293 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 -- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 msgr2=0x7feb8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 -- 172.21.15.17:0/1563185458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb80003a40 msgr2=0x7feb80003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb80003a40 0x7feb80003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 monclient: found mon.a 2024-09-15T21:35:28.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 1 -- 172.21.15.17:0/1563185458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb80003530 con 0x7feb80004810 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient: _finish_auth 0 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 10 monclient: _check_auth_tickets 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.148+0000 7feb9359e640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb940a28b0 0x7feb8000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9259c640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb80004c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00c290 tx=0x7feb7c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c80f32999d8bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c007d60 con 0x7feb80004810 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: dump: 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.171 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c0a9020 con 0x7feb80004810 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 4 set_mon_vals no callback set 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 1 -- 172.21.15.17:0/1563185458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c005ae0 con 0x7feb80004810 2024-09-15T21:35:28.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb91d9b640 10 monclient: dump: 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: shutdown 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 msgr2=0x7feb80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 --2- 172.21.15.17:0/1563185458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb80004c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00c290 tx=0x7feb7c01cbf0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9359e640 1 -- 172.21.15.17:0/1563185458 reap_dead start 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- 172.21.15.17:0/1563185458 shutdown_connections 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- 172.21.15.17:0/1563185458 >> 172.21.15.17:0/1563185458 conn(0x7feb940a74a0 msgr2=0x7feb940a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- 172.21.15.17:0/1563185458 shutdown_connections 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- 172.21.15.17:0/1563185458 wait complete. 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: build_initial_monmap 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: monmap: 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:28.174 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 Processor -- start 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- start start 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: init 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb941cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): picked mon.noname-a con 0x7feb80004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 0x7feb941ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): picked mon.noname-c con 0x7feb941ce2f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb9413bc40 0x7feb9413c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): picked mon.noname-b con 0x7feb9413bc40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb940a35b0 con 0x7feb80004810 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb940a4310 con 0x7feb9413bc40 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb9411e2d0 con 0x7feb941ce2f0 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb991b0640 10 monclient(hunting): authenticate will time out at 2051.551796s 2024-09-15T21:35:28.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb941cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9259c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 0x7feb941ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb941cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38376/0 (socket says 172.21.15.17:38376) 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 1 -- 172.21.15.17:0/282329552 learned_addr learned my addr 172.21.15.17:0/282329552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 10 monclient(hunting): get_auth_request con 0x7feb80004810 auth_method 0 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb92d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9259c640 10 monclient(hunting): get_auth_request con 0x7feb941ce2f0 auth_method 0 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.152+0000 7feb9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:28.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 msgr2=0x7feb941ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 msgr2=0x7feb941ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 0x7feb941ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 0x7feb941ce770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb9413bc40 0x7feb9413c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 10 monclient(hunting): get_auth_request con 0x7feb9413bc40 auth_method 0 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 10 monclient(hunting): handle_auth_done global_id 204240 payload 824 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 msgr2=0x7feb941ce770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb941ce2f0 0x7feb941ce770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb9413bc40 msgr2=0x7feb9413c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb9413bc40 0x7feb9413c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 10 monclient: _finish_hunting 0 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 monclient: found mon.noname-a 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb940a5e40 con 0x7feb80004810 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb92d9d640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb941cdbc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb84011630 tx=0x7feb84011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8400f040 con 0x7feb80004810 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: dump: 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.179 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 5 monclient: authenticate success, global_id 204240 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb84004230 con 0x7feb80004810 2024-09-15T21:35:28.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8400f720 con 0x7feb80004810 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 10 monclient: dump: 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb90d99640 4 set_mon_vals no callback set 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 10 monclient: _renew_subs 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9409e100 con 0x7feb80004810 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 10 monclient: _renew_subs 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9413e210 con 0x7feb80004810 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb940a3730 con 0x7feb9402e5d0 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb9402e5d0 auth_method 0 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7feb8400f8c0 con 0x7feb80004810 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb7403b9c0 0x7feb7403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7feb84050a60 con 0x7feb80004810 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb7403b9c0 0x7feb7403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 10 monclient: get_auth_request con 0x7feb7403b9c0 auth_method 0 2024-09-15T21:35:28.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.156+0000 7feb9359e640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb7403b9c0 0x7feb7403de80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7feb9406dc60 tx=0x7feb8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb9402e5d0 auth_method 0 2024-09-15T21:35:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:35:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.360+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:35:28.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:28.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb9402e5d0 auth_method 0 2024-09-15T21:35:28.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:28.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:28.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:28.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:35:28.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:28.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:28.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:28.764+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:35:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb9402e5d0 auth_method 0 2024-09-15T21:35:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:35:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:29.568+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:35:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.156+0000 7feb8a7fc640 10 monclient: tick 2024-09-15T21:35:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.156+0000 7feb8a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:35:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.156+0000 7feb8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.156+0000 7feb8a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb9402e5d0 auth_method 0 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 10 monclient(hunting): handle_auth_done global_id 214100 payload 824 2024-09-15T21:35:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb9402e9b0 con 0x7feb9402e5d0 2024-09-15T21:35:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:35:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:35:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7feb940a7f20 tx=0x7feb80005840 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:31.172+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 10 monclient: tick 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 5 monclient: _check_tell_commands timeout tell command 1 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 0x7feb6c005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 msgr2=0x7feb9413c600 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb9402e5d0 0x7feb9413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8a7fc640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c0060b0 con 0x7feb6c0035a0 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 0x7feb6c005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb6c0035a0 auth_method 0 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 10 monclient(hunting): handle_auth_done global_id 214103 payload 824 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb6c006260 con 0x7feb6c0035a0 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 0x7feb6c005a30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01cc30 tx=0x7feb7c0a5f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c004d80 con 0x7feb6c0035a0 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 10 monclient: dump: 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7feb7c0a9040 con 0x7feb6c0035a0 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 msgr2=0x7feb6c005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 0x7feb6c005a30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01cc30 tx=0x7feb7c0a5f70 comp rx=0 tx=0).stop 2024-09-15T21:35:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 msgr2=0x7feb6c005a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.156+0000 7feb8b7fe640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb6c0035a0 0x7feb6c005a30 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01cc30 tx=0x7feb7c0a5f70 comp rx=0 tx=0).stop 2024-09-15T21:35:32.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb991b0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:32.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb991b0640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 0x7feb94064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb991b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7feb94135670 con 0x7feb94063e00 2024-09-15T21:35:32.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 0x7feb94064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient: get_auth_request con 0x7feb94063e00 auth_method 0 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 10 monclient(hunting): handle_auth_done global_id 214106 payload 824 2024-09-15T21:35:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.164+0000 7feb9259c640 1 -- 172.21.15.17:0/282329552 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feb94134f70 con 0x7feb94063e00 2024-09-15T21:35:32.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb9259c640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 0x7feb94064210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01b2f0 tx=0x7feb80005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0c66b0 con 0x7feb94063e00 2024-09-15T21:35:32.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 10 monclient: dump: 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.172 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7feb7c0ae430 con 0x7feb94063e00 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 msgr2=0x7feb94064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 0x7feb94064210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01b2f0 tx=0x7feb80005840 comp rx=0 tx=0).stop 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 msgr2=0x7feb94064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb8b7fe640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7feb94063e00 0x7feb94064210 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01b2f0 tx=0x7feb80005840 comp rx=0 tx=0).stop 2024-09-15T21:35:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb9359e640 1 -- 172.21.15.17:0/282329552 reap_dead start 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 166, 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11415, 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:32.174 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 6, 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:32.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:32.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.177 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:32.178 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:32.179 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:32.180 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:32.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:32.187 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:32.188 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb7403b9c0 msgr2=0x7feb7403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb7403b9c0 0x7feb7403de80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7feb9406dc60 tx=0x7feb8c009040 comp rx=0 tx=0).stop 2024-09-15T21:35:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 10 monclient: shutdown 2024-09-15T21:35:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 msgr2=0x7feb941cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb941cdbc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb84011630 tx=0x7feb84011af0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 shutdown_connections 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7feb7403b9c0 0x7feb7403de80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 --2- 172.21.15.17:0/282329552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80004810 0x7feb941cdbc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 >> 172.21.15.17:0/282329552 conn(0x7feb940a74a0 msgr2=0x7feb940a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 shutdown_connections 2024-09-15T21:35:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.168+0000 7feb991b0640 1 -- 172.21.15.17:0/282329552 wait complete. 2024-09-15T21:35:32.200 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:35:32.200 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:35:32.201 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:35:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient: get_monmap_and_config 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient: build_initial_monmap 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient: monmap: 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:32.334376+0000 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:32.334376+0000 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:32.336 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 Processor -- start 2024-09-15T21:35:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 -- start start 2024-09-15T21:35:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient: init 2024-09-15T21:35:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f97ac1378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97ac1383b0 0x7f97ac13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): picked mon.noname-c con 0x7f97ac1383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac13d2c0 0x7f97ac13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): picked mon.noname-b con 0x7f97ac13d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ac0a1a20 con 0x7f97ac1378c0 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f97ac0a1d00 con 0x7f97ac1383b0 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac0a1fe0 con 0x7f97ac13d2c0 2024-09-15T21:35:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97b19bb640 10 monclient(hunting): authenticate will time out at 2055.729901s 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97aaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f97ac1378c0 0x7f97ac137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58264/0 (socket says 172.21.15.17:58264) 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97aaffd640 1 -- 172.21.15.17:0/3048283264 learned_addr learned my addr 172.21.15.17:0/3048283264 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 1 --2- 172.21.15.17:0/3048283264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac13d2c0 0x7f97ac13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): get_auth_request con 0x7f97ac13d2c0 auth_method 0 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97aa7fc640 1 --2- 172.21.15.17:0/3048283264 >> v2:172.21.15.119:3300/0 conn(0x7f97ac1383b0 0x7f97ac13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_done global_id 214109 payload 274 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97a9ffb640 1 -- 172.21.15.17:0/3048283264 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683658750 0 0) 0x7f97a40014a0 con 0x7f97ac1378c0 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3048283264 >> v2:172.21.15.119:3300/0 conn(0x7f97ac1383b0 msgr2=0x7f97ac13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 1 --2- 172.21.15.17:0/3048283264 >> v2:172.21.15.119:3300/0 conn(0x7f97ac1383b0 0x7f97ac13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3048283264 >> v1:172.21.15.17:6789/0 conn(0x7f97ac1378c0 legacy=0x7f97ac137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 1 monclient: found mon.noname-b 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:35:32.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3048283264 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac0a22c0 con 0x7f97ac13d2c0 2024-09-15T21:35:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97a9ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:35:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97aa7fc640 10 monclient: get_auth_request con 0x7f97ac1383b0 auth_method 0 2024-09-15T21:35:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.332+0000 7f97aa7fc640 1 --2- 172.21.15.17:0/3048283264 >> v2:172.21.15.119:3300/0 conn(0x7f97ac1383b0 0x7f97ac13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:35:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 --2- 172.21.15.17:0/3048283264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac13d2c0 0x7f97ac13f6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097f0 tx=0x7f979c01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2306b4450e30122c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 1 -- 172.21.15.17:0/3048283264 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0b1070 con 0x7f97ac13d2c0 2024-09-15T21:35:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: dump: 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 5 monclient: authenticate success, global_id 214109 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 1 -- 172.21.15.17:0/3048283264 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c0a47f0 con 0x7f97ac13d2c0 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 4 set_mon_vals no callback set 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 1 -- 172.21.15.17:0/3048283264 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0a4bc0 con 0x7f97ac13d2c0 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a9ffb640 10 monclient: dump: 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: shutdown 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- 172.21.15.17:0/3048283264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac13d2c0 msgr2=0x7f97ac13f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3048283264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac13d2c0 0x7f97ac13f6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097f0 tx=0x7f979c01c040 comp rx=0 tx=0).stop 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- 172.21.15.17:0/3048283264 shutdown_connections 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3048283264 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac13d2c0 0x7f97ac13f6b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3048283264 >> v2:172.21.15.119:3300/0 conn(0x7f97ac1383b0 0x7f97ac13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- 172.21.15.17:0/3048283264 >> 172.21.15.17:0/3048283264 conn(0x7f97ac0aac40 msgr2=0x7f97ac0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- 172.21.15.17:0/3048283264 shutdown_connections 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- 172.21.15.17:0/3048283264 wait complete. 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: build_initial_monmap 2024-09-15T21:35:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: monmap: 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 Processor -- start 2024-09-15T21:35:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- start start 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: init 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac1378c0 0x7f97ac1d0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): picked mon.noname-c con 0x7f97ac1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac1383b0 0x7f97ac1369b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): picked mon.noname-b con 0x7f97ac1383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 0x7f97ac09d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): picked mon.noname-a con 0x7f97ac13d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac143130 con 0x7f97ac13d2c0 2024-09-15T21:35:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac142fb0 con 0x7f97ac1383b0 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac1432b0 con 0x7f97ac1378c0 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97b19bb640 10 monclient(hunting): authenticate will time out at 2055.735091s 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac1378c0 0x7f97ac1d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 0x7f97ac09d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 0x7f97ac09d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38386/0 (socket says 172.21.15.17:38386) 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac1378c0 0x7f97ac1d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60258/0 (socket says 172.21.15.17:60258) 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3322608772 learned_addr learned my addr 172.21.15.17:0/3322608772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): get_auth_request con 0x7f97ac13d2c0 auth_method 0 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): get_auth_request con 0x7f97ac1378c0 auth_method 0 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aa7fc640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac1383b0 0x7f97ac1369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient(hunting): handle_auth_done global_id 204246 payload 824 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac1378c0 msgr2=0x7f97ac1d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac1378c0 0x7f97ac1d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac1383b0 msgr2=0x7f97ac1369b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac1383b0 0x7f97ac1369b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 monclient: found mon.noname-a 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac121060 con 0x7f97ac13d2c0 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aa7fc640 10 monclient: get_auth_request con 0x7f97ac1383b0 auth_method 0 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aa7fc640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac1383b0 0x7f97ac1369b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97aaffd640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f97ac1378c0 0x7f97ac1d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97ab7fe640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 0x7f97ac09d960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f979c01a6f0 tx=0x7f979c019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.336+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0b1040 con 0x7f97ac13d2c0 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: dump: 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: _finish_auth 0 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c0ac640 con 0x7f97ac13d2c0 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 5 monclient: authenticate success, global_id 204246 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 10 monclient: _renew_subs 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac121340 con 0x7f97ac13d2c0 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0aca10 con 0x7f97ac13d2c0 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: dump: 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.362 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 10 monclient: _renew_subs 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a97fa640 4 set_mon_vals no callback set 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac09ff30 con 0x7f97ac13d2c0 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 0x7f97ac09dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac0a7f00 con 0x7f97ac136ef0 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f979c0acbb0 con 0x7f97ac13d2c0 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 0x7f97ac09dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient: get_auth_request con 0x7f97ac136ef0 auth_method 0 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f978c03ba20 0x7f978c03dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f979c0e9e70 con 0x7f97ac13d2c0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aa7fc640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f978c03ba20 0x7f978c03dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 10 monclient(hunting): handle_auth_done global_id 194317 payload 824 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97ac09f3d0 con 0x7f97ac136ef0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aa7fc640 10 monclient: get_auth_request con 0x7f978c03ba20 auth_method 0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aaffd640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 0x7f97ac09dea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a400f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a400fe80 con 0x7f97ac136ef0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: dump: 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97aa7fc640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f978c03ba20 0x7f978c03dee0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f97ac13f060 tx=0x7f9794008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f97a4018070 con 0x7f97ac136ef0 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 msgr2=0x7f97ac09dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 0x7f97ac09dea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a400f5a0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 msgr2=0x7f97ac09dea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:32.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.340+0000 7f97a3fff640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac136ef0 0x7f97ac09dea0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a400f5a0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97b19bb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 0x7f97ac13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97b19bb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac09e7f0 con 0x7f97ac09e3e0 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 0x7f97ac13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient: get_auth_request con 0x7f97ac09e3e0 auth_method 0 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 10 monclient(hunting): handle_auth_done global_id 194320 payload 824 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 1 -- 172.21.15.17:0/3322608772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f97ac0a1270 con 0x7f97ac09e3e0 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97aaffd640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 0x7f97ac13ad70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f97a4015040 tx=0x7f97a4001410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4013950 con 0x7f97ac09e3e0 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 10 monclient: dump: 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f97a4013950 con 0x7f97ac09e3e0 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 msgr2=0x7f97ac13ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 0x7f97ac13ad70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f97a4015040 tx=0x7f97a4001410 comp rx=0 tx=0).stop 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 msgr2=0x7f97ac13ad70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.348+0000 7f97a3fff640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac09e3e0 0x7f97ac13ad70 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f97a4015040 tx=0x7f97a4001410 comp rx=0 tx=0).stop 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 166, 2024-09-15T21:35:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 99142, 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 6, 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:32.382 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:32.384 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.385 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:32.386 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:32.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:32.387 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:32.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:32.387 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:32.387 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:32.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f978c03ba20 msgr2=0x7f978c03dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f978c03ba20 0x7f978c03dee0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f97ac13f060 tx=0x7f9794008040 comp rx=0 tx=0).stop 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 10 monclient: shutdown 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 msgr2=0x7f97ac09d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 0x7f97ac09d960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f979c01a6f0 tx=0x7f979c019100 comp rx=0 tx=0).stop 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97ab7fe640 1 -- 172.21.15.17:0/3322608772 reap_dead start 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 shutdown_connections 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 --2- 172.21.15.17:0/3322608772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac13d2c0 0x7f97ac09d960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 >> 172.21.15.17:0/3322608772 conn(0x7f97ac0aac40 msgr2=0x7f97ac0a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 shutdown_connections 2024-09-15T21:35:32.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.352+0000 7f97b19bb640 1 -- 172.21.15.17:0/3322608772 wait complete. 2024-09-15T21:35:32.390 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:35:32.390 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:35:32.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:35:32.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.500+0000 7f188f9d0640 10 monclient: get_monmap_and_config 2024-09-15T21:35:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.500+0000 7f188f9d0640 10 monclient: build_initial_monmap 2024-09-15T21:35:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.500+0000 7f188f9d0640 10 monclient: monmap: 2024-09-15T21:35:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:32.505529+0000 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:32.505529+0000 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:32.507 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 Processor -- start 2024-09-15T21:35:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 -- start start 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient: init 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f18880a0890 0x7f18880a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): picked mon.noname-a con 0x7f18880a0890 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188809ffc0 0x7f18880a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): picked mon.noname-b con 0x7f188809ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1888135670 0x7f1888135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): picked mon.noname-c con 0x7f1888135670 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f18880a16f0 con 0x7f18880a0890 2024-09-15T21:35:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f18880a3130 con 0x7f1888135670 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1888073df0 con 0x7f188809ffc0 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient(hunting): authenticate will time out at 2055.900970s 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f18880a0890 0x7f18880a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188809ffc0 0x7f18880a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f18880a0890 0x7f18880a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38388/0 (socket says 172.21.15.17:38388) 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 -- 172.21.15.17:0/493557710 learned_addr learned my addr 172.21.15.17:0/493557710 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): get_auth_request con 0x7f18880a0890 auth_method 0 2024-09-15T21:35:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): get_auth_request con 0x7f188809ffc0 auth_method 0 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188cf6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188df6d640 1 --2- 172.21.15.17:0/493557710 >> v2:172.21.15.119:3300/0 conn(0x7f1888135670 0x7f1888135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient(hunting): handle_auth_done global_id 204249 payload 274 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 -- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188809ffc0 msgr2=0x7f18880a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 --2- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188809ffc0 0x7f18880a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 -- 172.21.15.17:0/493557710 >> v2:172.21.15.119:3300/0 conn(0x7f1888135670 msgr2=0x7f1888135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 --2- 172.21.15.17:0/493557710 >> v2:172.21.15.119:3300/0 conn(0x7f1888135670 0x7f1888135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 monclient: found mon.noname-a 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 -- 172.21.15.17:0/493557710 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18880a1bf0 con 0x7f18880a0890 2024-09-15T21:35:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188df6d640 10 monclient: get_auth_request con 0x7f1888135670 auth_method 0 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188df6d640 1 --2- 172.21.15.17:0/493557710 >> v2:172.21.15.119:3300/0 conn(0x7f1888135670 0x7f1888135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188d76c640 1 --2- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18880a0c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f188806f1c0 tx=0x7f187801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed68cc0340b44786 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 1 -- 172.21.15.17:0/493557710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780a5640 con 0x7f18880a0890 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: dump: 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: _finish_auth 0 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 5 monclient: authenticate success, global_id 204249 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 1 -- 172.21.15.17:0/493557710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18780a57e0 con 0x7f18880a0890 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f188f9d0640 4 set_mon_vals no callback set 2024-09-15T21:35:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 1 -- 172.21.15.17:0/493557710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780a5bb0 con 0x7f18880a0890 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.504+0000 7f1887fff640 10 monclient: dump: 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: shutdown 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 msgr2=0x7f18880a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18880a0c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f188806f1c0 tx=0x7f187801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/493557710 shutdown_connections 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- 172.21.15.17:0/493557710 >> v2:172.21.15.119:3300/0 conn(0x7f1888135670 0x7f1888135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f188809ffc0 0x7f18880a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- 172.21.15.17:0/493557710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18880a0c70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/493557710 >> 172.21.15.17:0/493557710 conn(0x7f18880a66a0 msgr2=0x7f18880a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/493557710 shutdown_connections 2024-09-15T21:35:32.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/493557710 wait complete. 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: build_initial_monmap 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: monmap: 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 Processor -- start 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- start start 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: init 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f188809ffc0 0x7f18881cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): picked mon.noname-b con 0x7f188809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18881347f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): picked mon.noname-a con 0x7f18880a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1888135670 0x7f1888134d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): picked mon.noname-c con 0x7f1888135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f188813f680 con 0x7f18880a0890 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f188813f500 con 0x7f188809ffc0 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f188813f800 con 0x7f1888135670 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient(hunting): authenticate will time out at 2055.905937s 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18881347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1888135670 0x7f1888134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18881347f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38390/0 (socket says 172.21.15.17:38390) 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 -- 172.21.15.17:0/3572389708 learned_addr learned my addr 172.21.15.17:0/3572389708 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): get_auth_request con 0x7f18880a0890 auth_method 0 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): get_auth_request con 0x7f1888135670 auth_method 0 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188df6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188d76c640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f188809ffc0 0x7f18881cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient(hunting): handle_auth_done global_id 204252 payload 824 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 -- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1888135670 msgr2=0x7f1888134d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1888135670 0x7f1888134d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 -- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f188809ffc0 msgr2=0x7f18881cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f188809ffc0 0x7f18881cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient: _finish_hunting 0 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 monclient: found mon.noname-a 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 -- 172.21.15.17:0/3572389708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1888121ed0 con 0x7f18880a0890 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188d76c640 10 monclient: get_auth_request con 0x7f188809ffc0 auth_method 0 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188d76c640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f188809ffc0 0x7f18881cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188cf6b640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18881347f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f187c00e930 tx=0x7f187c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c017070 con 0x7f18880a0890 2024-09-15T21:35:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: dump: 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 5 monclient: authenticate success, global_id 204252 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f187c00ca00 con 0x7f18880a0890 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _renew_subs 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1888121bf0 con 0x7f18880a0890 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00cdb0 con 0x7f18880a0890 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 10 monclient: dump: 2024-09-15T21:35:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1886ffd640 4 set_mon_vals no callback set 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _renew_subs 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188811e5b0 con 0x7f18880a0890 2024-09-15T21:35:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f187c01a070 con 0x7f18880a0890 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.508+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188811fe50 con 0x7f18880a0890 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.512+0000 7f1885ffb640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f187403b9c0 0x7f187403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.512+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f187c050740 con 0x7f18880a0890 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.512+0000 7f188d76c640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f187403b9c0 0x7f187403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.512+0000 7f188d76c640 10 monclient: get_auth_request con 0x7f187403b9c0 auth_method 0 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.512+0000 7f188d76c640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f187403b9c0 0x7f187403de80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f18780ae000 tx=0x7f18780a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.516+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f188811fe50 con 0x7f18880a0890 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.516+0000 7f1885ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:32.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.516+0000 7f1885ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f188f9d0640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:35:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f188f9d0640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:35:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f188f9d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f1888135aa0 con 0x7f18880a0890 2024-09-15T21:35:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f1885ffb640 1 -- 172.21.15.17:0/3572389708 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f187c013c70 con 0x7f18880a0890 2024-09-15T21:35:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f1885ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:35:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.756+0000 7f1885ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:32.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.756+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.756+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=31,config=2,health=12,kv=12,logm=43} crc {auth=4239414722,config=3541143443,health=1350913232,kv=3079713637,logm=3467100752}) 2024-09-15T21:35:32.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.756+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=30,config=2,health=12,kv=11,logm=45} crc {auth=4289833601,config=3541143443,health=1350913232,kv=2435832620,logm=529688319}) 2024-09-15T21:35:32.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.756+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.756+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=31,config=2,health=12,kv=12,logm=43} crc {auth=4239414722,config=3541143443,health=1350913232,kv=3079713637,logm=3467100752}) 2024-09-15T21:35:32.762 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.756+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=11,kv=12,logm=46} crc {auth=2989398114,config=3541143443,health=1933779806,kv=3079713637,logm=1918425015}) 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f187403b9c0 msgr2=0x7f187403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f187403b9c0 0x7f187403de80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f18780ae000 tx=0x7f18780a7070 comp rx=0 tx=0).stop 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 10 monclient: shutdown 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 msgr2=0x7f18881347f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18881347f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f187c00e930 tx=0x7f187c00edf0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 shutdown_connections 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f187403b9c0 0x7f187403de80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1888135670 0x7f1888134d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18880a0890 0x7f18881347f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 --2- 172.21.15.17:0/3572389708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f188809ffc0 0x7f18881cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 >> 172.21.15.17:0/3572389708 conn(0x7f18880a66a0 msgr2=0x7f18880a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 shutdown_connections 2024-09-15T21:35:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:32.760+0000 7f188f9d0640 1 -- 172.21.15.17:0/3572389708 wait complete. 2024-09-15T21:35:32.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.760+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.760+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3174021944}) 2024-09-15T21:35:32.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.760+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2414231381}) 2024-09-15T21:35:32.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.760+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.766 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.760+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3174021944}) 2024-09-15T21:35:32.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.760+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021940940}) 2024-09-15T21:35:32.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.764+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.764+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3513222997}) 2024-09-15T21:35:32.767 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.764+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=783133651}) 2024-09-15T21:35:32.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.764+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.764+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3513222997}) 2024-09-15T21:35:32.768 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.764+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1672156549}) 2024-09-15T21:35:32.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3868758456}) 2024-09-15T21:35:32.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3109500012}) 2024-09-15T21:35:32.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3868758456}) 2024-09-15T21:35:32.770 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=612535483}) 2024-09-15T21:35:32.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.773 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4242773864}) 2024-09-15T21:35:32.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2585296950}) 2024-09-15T21:35:32.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4242773864}) 2024-09-15T21:35:32.774 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.768+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2459044796}) 2024-09-15T21:35:32.775 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269262886}) 2024-09-15T21:35:32.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4178190540}) 2024-09-15T21:35:32.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269262886}) 2024-09-15T21:35:32.776 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3271621450}) 2024-09-15T21:35:32.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219673533}) 2024-09-15T21:35:32.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3631031403}) 2024-09-15T21:35:32.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219673533}) 2024-09-15T21:35:32.778 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.772+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2040487207}) 2024-09-15T21:35:32.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.776+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.776+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=957052164}) 2024-09-15T21:35:32.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.776+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=213286944}) 2024-09-15T21:35:32.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.776+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.776+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=957052164}) 2024-09-15T21:35:32.780 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.776+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1579990887}) 2024-09-15T21:35:32.784 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:35:32.787 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.787 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135973212}) 2024-09-15T21:35:32.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3433133687}) 2024-09-15T21:35:32.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135973212}) 2024-09-15T21:35:32.788 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1851223713}) 2024-09-15T21:35:32.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164040268}) 2024-09-15T21:35:32.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1669510102}) 2024-09-15T21:35:32.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164040268}) 2024-09-15T21:35:32.789 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.784+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3620803331}) 2024-09-15T21:35:32.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4229405144}) 2024-09-15T21:35:32.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2776423380}) 2024-09-15T21:35:32.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4229405144}) 2024-09-15T21:35:32.791 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=340678598}) 2024-09-15T21:35:32.792 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578931292}) 2024-09-15T21:35:32.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=208188611}) 2024-09-15T21:35:32.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578931292}) 2024-09-15T21:35:32.793 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.788+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2484828434}) 2024-09-15T21:35:32.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167816445}) 2024-09-15T21:35:32.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1302994066}) 2024-09-15T21:35:32.794 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.795 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167816445}) 2024-09-15T21:35:32.795 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1345789429}) 2024-09-15T21:35:32.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1755705684}) 2024-09-15T21:35:32.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3532404064}) 2024-09-15T21:35:32.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1755705684}) 2024-09-15T21:35:32.796 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.792+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1743649115}) 2024-09-15T21:35:32.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1145392895}) 2024-09-15T21:35:32.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3941222777}) 2024-09-15T21:35:32.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1145392895}) 2024-09-15T21:35:32.798 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1554631429}) 2024-09-15T21:35:32.800 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.800 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56044193}) 2024-09-15T21:35:32.800 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1048579933}) 2024-09-15T21:35:32.800 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.800 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56044193}) 2024-09-15T21:35:32.800 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246154138}) 2024-09-15T21:35:32.801 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=523543675}) 2024-09-15T21:35:32.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1301630875}) 2024-09-15T21:35:32.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=523543675}) 2024-09-15T21:35:32.802 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.796+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858066387}) 2024-09-15T21:35:32.803 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.803 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624887767}) 2024-09-15T21:35:32.803 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3010223129}) 2024-09-15T21:35:32.803 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.804 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624887767}) 2024-09-15T21:35:32.804 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=343060635}) 2024-09-15T21:35:32.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3852837985}) 2024-09-15T21:35:32.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053360913}) 2024-09-15T21:35:32.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3852837985}) 2024-09-15T21:35:32.805 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.800+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=371537379}) 2024-09-15T21:35:32.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2922010595}) 2024-09-15T21:35:32.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2641095541}) 2024-09-15T21:35:32.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2922010595}) 2024-09-15T21:35:32.807 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4085774015}) 2024-09-15T21:35:32.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2327507453}) 2024-09-15T21:35:32.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3087793411}) 2024-09-15T21:35:32.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2327507453}) 2024-09-15T21:35:32.809 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.804+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403878904}) 2024-09-15T21:35:32.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583350895}) 2024-09-15T21:35:32.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3322125636}) 2024-09-15T21:35:32.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583350895}) 2024-09-15T21:35:32.812 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=148816364}) 2024-09-15T21:35:32.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2084581872}) 2024-09-15T21:35:32.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3291102180}) 2024-09-15T21:35:32.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2084581872}) 2024-09-15T21:35:32.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.808+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3103947008}) 2024-09-15T21:35:32.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=850581677}) 2024-09-15T21:35:32.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=107227632}) 2024-09-15T21:35:32.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=850581677}) 2024-09-15T21:35:32.816 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247357865}) 2024-09-15T21:35:32.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3431149200}) 2024-09-15T21:35:32.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.812+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2617167130}) 2024-09-15T21:35:32.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3431149200}) 2024-09-15T21:35:32.818 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1322644997}) 2024-09-15T21:35:32.820 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3608447271}) 2024-09-15T21:35:32.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081752693}) 2024-09-15T21:35:32.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3608447271}) 2024-09-15T21:35:32.821 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.816+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1292225650}) 2024-09-15T21:35:32.822 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2536602722}) 2024-09-15T21:35:32.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1037541118}) 2024-09-15T21:35:32.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2536602722}) 2024-09-15T21:35:32.823 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1608799177}) 2024-09-15T21:35:32.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=531559325}) 2024-09-15T21:35:32.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=30323626}) 2024-09-15T21:35:32.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=531559325}) 2024-09-15T21:35:32.825 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.820+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1758019053}) 2024-09-15T21:35:32.827 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.827 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3113944586}) 2024-09-15T21:35:32.827 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3713817639}) 2024-09-15T21:35:32.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3113944586}) 2024-09-15T21:35:32.828 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3608963707}) 2024-09-15T21:35:32.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.829 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707989922}) 2024-09-15T21:35:32.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1582084483}) 2024-09-15T21:35:32.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707989922}) 2024-09-15T21:35:32.830 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.824+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1811197411}) 2024-09-15T21:35:32.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.831 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3670335149}) 2024-09-15T21:35:32.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1451794905}) 2024-09-15T21:35:32.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3670335149}) 2024-09-15T21:35:32.832 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=274260017}) 2024-09-15T21:35:32.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2290673672}) 2024-09-15T21:35:32.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3113973510}) 2024-09-15T21:35:32.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2290673672}) 2024-09-15T21:35:32.834 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.828+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1292294122}) 2024-09-15T21:35:32.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1177505657}) 2024-09-15T21:35:32.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1912929177}) 2024-09-15T21:35:32.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1177505657}) 2024-09-15T21:35:32.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1769923680}) 2024-09-15T21:35:32.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2821423136}) 2024-09-15T21:35:32.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=569222006}) 2024-09-15T21:35:32.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.832+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2821423136}) 2024-09-15T21:35:32.838 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=91032116}) 2024-09-15T21:35:32.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030573356}) 2024-09-15T21:35:32.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3126706858}) 2024-09-15T21:35:32.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030573356}) 2024-09-15T21:35:32.840 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.836+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2864041732}) 2024-09-15T21:35:32.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3314602356}) 2024-09-15T21:35:32.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371165926}) 2024-09-15T21:35:32.842 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.843 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3314602356}) 2024-09-15T21:35:32.843 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1237988567}) 2024-09-15T21:35:32.844 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674393882}) 2024-09-15T21:35:32.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3155938215}) 2024-09-15T21:35:32.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674393882}) 2024-09-15T21:35:32.845 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.840+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1574149765}) 2024-09-15T21:35:32.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1525852642}) 2024-09-15T21:35:32.849 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=52171091}) 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1525852642}) 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1138470725}) 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006439027}) 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3910244126}) 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006439027}) 2024-09-15T21:35:32.850 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.844+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=607169435}) 2024-09-15T21:35:32.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1680808574}) 2024-09-15T21:35:32.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=40098985}) 2024-09-15T21:35:32.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1680808574}) 2024-09-15T21:35:32.851 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4263232422}) 2024-09-15T21:35:32.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=60029524}) 2024-09-15T21:35:32.853 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3947823210}) 2024-09-15T21:35:32.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=60029524}) 2024-09-15T21:35:32.854 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.848+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3379224096}) 2024-09-15T21:35:32.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480992525}) 2024-09-15T21:35:32.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1493128495}) 2024-09-15T21:35:32.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480992525}) 2024-09-15T21:35:32.856 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4140933667}) 2024-09-15T21:35:32.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.852+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082462676}) 2024-09-15T21:35:32.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=238407996}) 2024-09-15T21:35:32.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082462676}) 2024-09-15T21:35:32.858 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=160358261}) 2024-09-15T21:35:32.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=854877757}) 2024-09-15T21:35:32.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1503034006}) 2024-09-15T21:35:32.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.860 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=854877757}) 2024-09-15T21:35:32.861 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.856+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1250690815}) 2024-09-15T21:35:32.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1591411123}) 2024-09-15T21:35:32.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3708527010}) 2024-09-15T21:35:32.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1591411123}) 2024-09-15T21:35:32.863 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3438676533}) 2024-09-15T21:35:32.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366916990}) 2024-09-15T21:35:32.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3483536597}) 2024-09-15T21:35:32.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366916990}) 2024-09-15T21:35:32.865 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.860+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2772814679}) 2024-09-15T21:35:32.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1490224}) 2024-09-15T21:35:32.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2920190891}) 2024-09-15T21:35:32.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1490224}) 2024-09-15T21:35:32.867 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3169672542}) 2024-09-15T21:35:32.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2793334619}) 2024-09-15T21:35:32.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2620120156}) 2024-09-15T21:35:32.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.869 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2793334619}) 2024-09-15T21:35:32.870 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.864+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=29499904}) 2024-09-15T21:35:32.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4137782010}) 2024-09-15T21:35:32.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=936022220}) 2024-09-15T21:35:32.871 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4137782010}) 2024-09-15T21:35:32.872 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097199158}) 2024-09-15T21:35:32.873 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.873 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610068734}) 2024-09-15T21:35:32.873 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=242836602}) 2024-09-15T21:35:32.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610068734}) 2024-09-15T21:35:32.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.868+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1781527061}) 2024-09-15T21:35:32.875 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.875 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4274744494}) 2024-09-15T21:35:32.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1629386767}) 2024-09-15T21:35:32.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4274744494}) 2024-09-15T21:35:32.876 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2769970369}) 2024-09-15T21:35:32.877 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430827573}) 2024-09-15T21:35:32.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=286100170}) 2024-09-15T21:35:32.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430827573}) 2024-09-15T21:35:32.878 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.872+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3709851608}) 2024-09-15T21:35:32.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1633784780}) 2024-09-15T21:35:32.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2075448101}) 2024-09-15T21:35:32.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1633784780}) 2024-09-15T21:35:32.880 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2783521382}) 2024-09-15T21:35:32.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900296086}) 2024-09-15T21:35:32.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3231762573}) 2024-09-15T21:35:32.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.876+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900296086}) 2024-09-15T21:35:32.882 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1187089599}) 2024-09-15T21:35:32.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3106768470}) 2024-09-15T21:35:32.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2534167149}) 2024-09-15T21:35:32.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3106768470}) 2024-09-15T21:35:32.884 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.880+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=681475055}) 2024-09-15T21:35:32.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=563428033}) 2024-09-15T21:35:32.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1883237361}) 2024-09-15T21:35:32.886 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.887 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=563428033}) 2024-09-15T21:35:32.887 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2020077246}) 2024-09-15T21:35:32.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.888 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523746263}) 2024-09-15T21:35:32.889 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2065584051}) 2024-09-15T21:35:32.889 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.889 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523746263}) 2024-09-15T21:35:32.889 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.884+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818955870}) 2024-09-15T21:35:32.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722406513}) 2024-09-15T21:35:32.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=184112942}) 2024-09-15T21:35:32.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722406513}) 2024-09-15T21:35:32.891 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=121265176}) 2024-09-15T21:35:32.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078464208}) 2024-09-15T21:35:32.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4088564918}) 2024-09-15T21:35:32.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078464208}) 2024-09-15T21:35:32.893 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.888+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2859375195}) 2024-09-15T21:35:32.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=616347173}) 2024-09-15T21:35:32.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=55976369}) 2024-09-15T21:35:32.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=616347173}) 2024-09-15T21:35:32.895 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3980716609}) 2024-09-15T21:35:32.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162342531}) 2024-09-15T21:35:32.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3322452825}) 2024-09-15T21:35:32.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162342531}) 2024-09-15T21:35:32.897 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.892+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=255776623}) 2024-09-15T21:35:32.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2019704238}) 2024-09-15T21:35:32.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3728549682}) 2024-09-15T21:35:32.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2019704238}) 2024-09-15T21:35:32.899 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2598098921}) 2024-09-15T21:35:32.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2749505031}) 2024-09-15T21:35:32.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4252229395}) 2024-09-15T21:35:32.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2749505031}) 2024-09-15T21:35:32.901 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.896+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=382721101}) 2024-09-15T21:35:32.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505230193}) 2024-09-15T21:35:32.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=597584252}) 2024-09-15T21:35:32.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505230193}) 2024-09-15T21:35:32.903 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1290604391}) 2024-09-15T21:35:32.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2131748950}) 2024-09-15T21:35:32.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2539774744}) 2024-09-15T21:35:32.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2131748950}) 2024-09-15T21:35:32.905 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.900+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4152442219}) 2024-09-15T21:35:32.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=546701201}) 2024-09-15T21:35:32.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2807547933}) 2024-09-15T21:35:32.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=546701201}) 2024-09-15T21:35:32.907 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2684935172}) 2024-09-15T21:35:32.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890260324}) 2024-09-15T21:35:32.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698836564}) 2024-09-15T21:35:32.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890260324}) 2024-09-15T21:35:32.909 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.904+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4116401748}) 2024-09-15T21:35:32.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=915440569}) 2024-09-15T21:35:32.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=798025943}) 2024-09-15T21:35:32.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.911 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=915440569}) 2024-09-15T21:35:32.912 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1086752884}) 2024-09-15T21:35:32.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2585466878}) 2024-09-15T21:35:32.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=285589111}) 2024-09-15T21:35:32.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2585466878}) 2024-09-15T21:35:32.913 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.908+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=860482594}) 2024-09-15T21:35:32.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=706513915}) 2024-09-15T21:35:32.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3611115486}) 2024-09-15T21:35:32.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.915 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=706513915}) 2024-09-15T21:35:32.916 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3162063943}) 2024-09-15T21:35:32.917 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.917 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3903269135}) 2024-09-15T21:35:32.917 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=837956423}) 2024-09-15T21:35:32.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3903269135}) 2024-09-15T21:35:32.918 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.912+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=404580614}) 2024-09-15T21:35:32.919 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.919 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=354790807}) 2024-09-15T21:35:32.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1773624395}) 2024-09-15T21:35:32.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=354790807}) 2024-09-15T21:35:32.920 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4076504902}) 2024-09-15T21:35:32.921 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.921 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3696588594}) 2024-09-15T21:35:32.921 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4022588199}) 2024-09-15T21:35:32.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3696588594}) 2024-09-15T21:35:32.922 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.916+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2897738343}) 2024-09-15T21:35:32.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3468443487}) 2024-09-15T21:35:32.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4051218477}) 2024-09-15T21:35:32.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3468443487}) 2024-09-15T21:35:32.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2692771615}) 2024-09-15T21:35:32.925 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.925 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2030064552}) 2024-09-15T21:35:32.925 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1408585161}) 2024-09-15T21:35:32.925 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.926 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2030064552}) 2024-09-15T21:35:32.926 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.920+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908920748}) 2024-09-15T21:35:32.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1073557491}) 2024-09-15T21:35:32.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=700840325}) 2024-09-15T21:35:32.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1073557491}) 2024-09-15T21:35:32.927 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=323072575}) 2024-09-15T21:35:32.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2064101250}) 2024-09-15T21:35:32.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=568923863}) 2024-09-15T21:35:32.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.929 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2064101250}) 2024-09-15T21:35:32.930 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.924+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2081696576}) 2024-09-15T21:35:32.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1106599011}) 2024-09-15T21:35:32.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=991196974}) 2024-09-15T21:35:32.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1106599011}) 2024-09-15T21:35:32.931 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2818389631}) 2024-09-15T21:35:32.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1081455134}) 2024-09-15T21:35:32.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=454433517}) 2024-09-15T21:35:32.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1081455134}) 2024-09-15T21:35:32.933 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.928+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=838318755}) 2024-09-15T21:35:32.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561306905}) 2024-09-15T21:35:32.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387411171}) 2024-09-15T21:35:32.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561306905}) 2024-09-15T21:35:32.935 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3987370970}) 2024-09-15T21:35:32.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869752556}) 2024-09-15T21:35:32.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350639919}) 2024-09-15T21:35:32.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869752556}) 2024-09-15T21:35:32.937 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.932+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3696879129}) 2024-09-15T21:35:32.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=990720372}) 2024-09-15T21:35:32.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=548254995}) 2024-09-15T21:35:32.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=990720372}) 2024-09-15T21:35:32.939 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3166446112}) 2024-09-15T21:35:32.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632912341}) 2024-09-15T21:35:32.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4283369943}) 2024-09-15T21:35:32.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632912341}) 2024-09-15T21:35:32.941 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.936+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2738215726}) 2024-09-15T21:35:32.943 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.943 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=281021850}) 2024-09-15T21:35:32.943 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3626907620}) 2024-09-15T21:35:32.943 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.943 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=281021850}) 2024-09-15T21:35:32.943 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3613181234}) 2024-09-15T21:35:32.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=709089458}) 2024-09-15T21:35:32.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3478477040}) 2024-09-15T21:35:32.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=709089458}) 2024-09-15T21:35:32.945 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.940+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=613705650}) 2024-09-15T21:35:32.947 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.947 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597522473}) 2024-09-15T21:35:32.947 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2176774510}) 2024-09-15T21:35:32.947 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.947 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597522473}) 2024-09-15T21:35:32.947 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=722051799}) 2024-09-15T21:35:32.949 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.949 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=320587471}) 2024-09-15T21:35:32.949 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2786226753}) 2024-09-15T21:35:32.949 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.949 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=320587471}) 2024-09-15T21:35:32.949 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.944+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=923835732}) 2024-09-15T21:35:32.951 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.951 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2183912634}) 2024-09-15T21:35:32.951 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=849088678}) 2024-09-15T21:35:32.951 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.951 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2183912634}) 2024-09-15T21:35:32.951 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4166389727}) 2024-09-15T21:35:32.953 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.953 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4066660433}) 2024-09-15T21:35:32.953 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3979309995}) 2024-09-15T21:35:32.953 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.953 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4066660433}) 2024-09-15T21:35:32.953 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.948+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3348618429}) 2024-09-15T21:35:32.954 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.955 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2302187465}) 2024-09-15T21:35:32.955 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2103566585}) 2024-09-15T21:35:32.955 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.955 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2302187465}) 2024-09-15T21:35:32.955 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3960705238}) 2024-09-15T21:35:32.956 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.957 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321237450}) 2024-09-15T21:35:32.957 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3810732066}) 2024-09-15T21:35:32.957 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.957 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321237450}) 2024-09-15T21:35:32.957 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.952+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540642065}) 2024-09-15T21:35:32.958 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.959 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2293463926}) 2024-09-15T21:35:32.959 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2171100298}) 2024-09-15T21:35:32.959 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.959 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2293463926}) 2024-09-15T21:35:32.959 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=931825966}) 2024-09-15T21:35:32.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3884063387}) 2024-09-15T21:35:32.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=799663629}) 2024-09-15T21:35:32.960 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.961 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3884063387}) 2024-09-15T21:35:32.961 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.956+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1983413874}) 2024-09-15T21:35:32.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=291803701}) 2024-09-15T21:35:32.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1820296515}) 2024-09-15T21:35:32.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=291803701}) 2024-09-15T21:35:32.962 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2379811304}) 2024-09-15T21:35:32.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1364289362}) 2024-09-15T21:35:32.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2602055873}) 2024-09-15T21:35:32.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1364289362}) 2024-09-15T21:35:32.964 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.960+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3854437803}) 2024-09-15T21:35:32.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316539306}) 2024-09-15T21:35:32.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=573035737}) 2024-09-15T21:35:32.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316539306}) 2024-09-15T21:35:32.966 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4175414824}) 2024-09-15T21:35:32.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2376130268}) 2024-09-15T21:35:32.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342252495}) 2024-09-15T21:35:32.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2376130268}) 2024-09-15T21:35:32.968 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.964+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4287772319}) 2024-09-15T21:35:32.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.968+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.968+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2610550709}) 2024-09-15T21:35:32.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.968+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1859502012}) 2024-09-15T21:35:32.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.968+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.970 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.968+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2610550709}) 2024-09-15T21:35:32.971 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.968+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1613406792}) 2024-09-15T21:35:32.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025543491}) 2024-09-15T21:35:32.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2915706584}) 2024-09-15T21:35:32.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025543491}) 2024-09-15T21:35:32.974 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1678350291}) 2024-09-15T21:35:32.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3987661331}) 2024-09-15T21:35:32.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4097166877}) 2024-09-15T21:35:32.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3987661331}) 2024-09-15T21:35:32.976 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.972+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389283206}) 2024-09-15T21:35:32.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3241029345}) 2024-09-15T21:35:32.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1305749291}) 2024-09-15T21:35:32.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3241029345}) 2024-09-15T21:35:32.978 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1103123520}) 2024-09-15T21:35:32.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1429497974}) 2024-09-15T21:35:32.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1472662490}) 2024-09-15T21:35:32.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1429497974}) 2024-09-15T21:35:32.980 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=917891006}) 2024-09-15T21:35:32.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.976+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=676630922}) 2024-09-15T21:35:32.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1594711445}) 2024-09-15T21:35:32.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=676630922}) 2024-09-15T21:35:32.982 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2748472865}) 2024-09-15T21:35:32.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364871293}) 2024-09-15T21:35:32.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4067053145}) 2024-09-15T21:35:32.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364871293}) 2024-09-15T21:35:32.984 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=554968835}) 2024-09-15T21:35:32.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604069977}) 2024-09-15T21:35:32.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=543211168}) 2024-09-15T21:35:32.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604069977}) 2024-09-15T21:35:32.986 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.980+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=562656661}) 2024-09-15T21:35:32.987 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3134765816}) 2024-09-15T21:35:32.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1407114328}) 2024-09-15T21:35:32.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3134765816}) 2024-09-15T21:35:32.988 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1690424719}) 2024-09-15T21:35:32.989 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.989 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3171793806}) 2024-09-15T21:35:32.989 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1726658878}) 2024-09-15T21:35:32.990 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.990 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3171793806}) 2024-09-15T21:35:32.990 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.984+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3114180275}) 2024-09-15T21:35:32.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.988+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.988+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4211321842}) 2024-09-15T21:35:32.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.988+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790832746}) 2024-09-15T21:35:32.991 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.988+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.992 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.988+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4211321842}) 2024-09-15T21:35:32.992 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.988+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4133638615}) 2024-09-15T21:35:32.994 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.992+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.992+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=15} crc {logm=1318983818,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1172496742,mon_config_key=1438411635,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3550364910}) 2024-09-15T21:35:32.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.992+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=14} crc {logm=3845882991,mdsmap=3508357332,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3053705304,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=1154780841,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2410579}) 2024-09-15T21:35:32.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.992+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:32.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.992+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=15} crc {logm=1318983818,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1172496742,mon_config_key=1438411635,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3550364910}) 2024-09-15T21:35:32.995 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:32.992+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=13} crc {logm=1566832334,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=3053705304,mon_config_key=3076924734,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505837,osd_snap=3823785925,osdmap=1940270006}) 2024-09-15T21:35:33.009 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:33.004+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:33.009 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:33.004+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=40} crc {osdmap=3149327262}) 2024-09-15T21:35:33.009 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:33.004+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=39} crc {osdmap=55009350}) 2024-09-15T21:35:33.009 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:33.004+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:35:33.009 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:33.004+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=40} crc {osdmap=3149327262}) 2024-09-15T21:35:33.010 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:35:33.004+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=36} crc {osdmap=1255457127}) 2024-09-15T21:35:33.785 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:35:33.785 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient: get_monmap_and_config 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient: build_initial_monmap 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient: monmap: 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:33.919921+0000 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:33.919921+0000 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:33.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:33.921 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:33.921 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:33.921 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 Processor -- start 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 -- start start 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient: init 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f693c09d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c09def0 0x7f693c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): picked mon.noname-b con 0x7f693c09def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f693c139ff0 0x7f693c13c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): picked mon.noname-c con 0x7f693c139ff0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:33.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c0a3130 con 0x7f693c09d400 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f693c0a1bc0 con 0x7f693c139ff0 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f693c0a3490 con 0x7f693c09def0 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f694257a640 10 monclient(hunting): authenticate will time out at 2057.315207s 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f693c09d400 0x7f693c09d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58272/0 (socket says 172.21.15.17:58272) 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693bfff640 1 -- 172.21.15.17:0/1089852235 learned_addr learned my addr 172.21.15.17:0/1089852235 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 1 --2- 172.21.15.17:0/1089852235 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c09def0 0x7f693c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): get_auth_request con 0x7f693c09def0 auth_method 0 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:33.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f6940b17640 1 --2- 172.21.15.17:0/1089852235 >> v2:172.21.15.119:3300/0 conn(0x7f693c139ff0 0x7f693c13c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f6940b17640 10 monclient(hunting): get_auth_request con 0x7f693c139ff0 auth_method 0 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f6940b17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f6940b17640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f6940b17640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693affd640 1 -- 172.21.15.17:0/1089852235 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712751127 0 0) 0x7f693c0a3130 con 0x7f693c09d400 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693affd640 10 monclient(hunting): _negotiate 2024-09-15T21:35:33.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693affd640 10 monclient(hunting): my global_id is 204255 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693affd640 1 -- 172.21.15.17:0/1089852235 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6924003530 con 0x7f693c09d400 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient(hunting): handle_auth_done global_id 214121 payload 274 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 1 -- 172.21.15.17:0/1089852235 >> v2:172.21.15.119:3300/0 conn(0x7f693c139ff0 msgr2=0x7f693c13c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 1 --2- 172.21.15.17:0/1089852235 >> v2:172.21.15.119:3300/0 conn(0x7f693c139ff0 0x7f693c13c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 1 -- 172.21.15.17:0/1089852235 >> v1:172.21.15.17:6789/0 conn(0x7f693c09d400 legacy=0x7f693c09d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 1 monclient: found mon.noname-b 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:35:33.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.916+0000 7f693b7fe640 1 -- 172.21.15.17:0/1089852235 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c09e7a0 con 0x7f693c09def0 2024-09-15T21:35:33.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693b7fe640 1 --2- 172.21.15.17:0/1089852235 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c09def0 0x7f693c139ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69300097c0 tx=0x7f693001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5953e505c488e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:33.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 1 -- 172.21.15.17:0/1089852235 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69300b0070 con 0x7f693c09def0 2024-09-15T21:35:33.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: dump: 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:33.932 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: _finish_auth 0 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 1 -- 172.21.15.17:0/1089852235 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69300ab600 con 0x7f693c09def0 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 1 -- 172.21.15.17:0/1089852235 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69300ab9d0 con 0x7f693c09def0 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 5 monclient: authenticate success, global_id 214121 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 4 set_mon_vals no callback set 2024-09-15T21:35:33.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f693affd640 10 monclient: dump: 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 10 monclient: shutdown 2024-09-15T21:35:33.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 -- 172.21.15.17:0/1089852235 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c09def0 msgr2=0x7f693c139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 --2- 172.21.15.17:0/1089852235 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c09def0 0x7f693c139ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69300097c0 tx=0x7f693001c360 comp rx=0 tx=0).stop 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 -- 172.21.15.17:0/1089852235 shutdown_connections 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 --2- 172.21.15.17:0/1089852235 >> v2:172.21.15.119:3300/0 conn(0x7f693c139ff0 0x7f693c13c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 --2- 172.21.15.17:0/1089852235 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c09def0 0x7f693c139ab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 -- 172.21.15.17:0/1089852235 >> 172.21.15.17:0/1089852235 conn(0x7f693c0a66a0 msgr2=0x7f693c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 -- 172.21.15.17:0/1089852235 shutdown_connections 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 -- 172.21.15.17:0/1089852235 wait complete. 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 10 monclient: build_initial_monmap 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 10 monclient: monmap: 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:33.935 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 Processor -- start 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 1 -- start start 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.920+0000 7f694257a640 10 monclient: init 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:33.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f693c09d400 0x7f693c073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): picked mon.noname-b con 0x7f693c09d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f693c09def0 0x7f693c139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): picked mon.noname-a con 0x7f693c09def0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 0x7f693c1cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): picked mon.noname-c con 0x7f693c139ff0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0a0890 con 0x7f693c09def0 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0a0a10 con 0x7f693c09d400 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f693c0a0b90 con 0x7f693c139ff0 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient(hunting): authenticate will time out at 2057.320262s 2024-09-15T21:35:33.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 0x7f693c1cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f693c09def0 0x7f693c139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 0x7f693c1cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60274/0 (socket says 172.21.15.17:60274) 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 -- 172.21.15.17:0/1896045129 learned_addr learned my addr 172.21.15.17:0/1896045129 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): get_auth_request con 0x7f693c139ff0 auth_method 0 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): get_auth_request con 0x7f693c09def0 auth_method 0 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:33.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f693c09d400 0x7f693c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 10 monclient(hunting): get_auth_request con 0x7f693c09d400 auth_method 0 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient(hunting): handle_auth_done global_id 214124 payload 824 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 -- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f693c09d400 msgr2=0x7f693c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f693c09d400 0x7f693c073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 -- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f693c09def0 msgr2=0x7f693c139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f693c09def0 0x7f693c139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient: _finish_hunting 0 2024-09-15T21:35:33.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 monclient: found mon.noname-c 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 -- 172.21.15.17:0/1896045129 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1cc610 con 0x7f693c139ff0 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693b7fe640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f693c09def0 0x7f693c139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6940b17640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 0x7f693c1cf6d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6934007e40 tx=0x7f693400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934014620 con 0x7f693c139ff0 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: dump: 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.940 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: _finish_auth 0 2024-09-15T21:35:33.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 5 monclient: authenticate success, global_id 214124 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6934019070 con 0x7f693c139ff0 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934014a20 con 0x7f693c139ff0 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 10 monclient: dump: 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:33.942 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693a7fc640 4 set_mon_vals no callback set 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _renew_subs 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c136f20 con 0x7f693c139ff0 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _renew_subs 2024-09-15T21:35:33.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c0a5380 con 0x7f693c139ff0 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c09d0f0 con 0x7f693c139ff0 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6934014bc0 con 0x7f693c139ff0 2024-09-15T21:35:33.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f692803b9c0 0x7f692803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f6934050fe0 con 0x7f693c139ff0 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f692803b9c0 0x7f692803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 10 monclient: get_auth_request con 0x7f692803b9c0 auth_method 0 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.924+0000 7f693bfff640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f692803b9c0 0x7f692803de80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f692c007a70 tx=0x7f692c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.928+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f693c1cc610 con 0x7f693c139ff0 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.928+0000 7f6938ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:33.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:33.928+0000 7f6938ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:34.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f694257a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:34.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f694257a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:34.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f694257a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f693c09d7e0 con 0x7f693c139ff0 2024-09-15T21:35:34.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f6938ff9640 1 -- 172.21.15.17:0/1896045129 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f693404ee50 con 0x7f693c139ff0 2024-09-15T21:35:34.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f6938ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:34.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.164+0000 7f6938ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:34.169 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:34.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f692803b9c0 msgr2=0x7f692803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f692803b9c0 0x7f692803de80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f692c007a70 tx=0x7f692c009040 comp rx=0 tx=0).stop 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 10 monclient: shutdown 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 msgr2=0x7f693c1cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 0x7f693c1cf6d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6934007e40 tx=0x7f693400e4b0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 shutdown_connections 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f692803b9c0 0x7f692803de80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f693c139ff0 0x7f693c1cf6d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f693c09def0 0x7f693c139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 --2- 172.21.15.17:0/1896045129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f693c09d400 0x7f693c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 >> 172.21.15.17:0/1896045129 conn(0x7f693c0a66a0 msgr2=0x7f693c13c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 shutdown_connections 2024-09-15T21:35:34.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.168+0000 7f694257a640 1 -- 172.21.15.17:0/1896045129 wait complete. 2024-09-15T21:35:34.189 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:35:34.189 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:35:34.189 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:35:34.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:35:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient: get_monmap_and_config 2024-09-15T21:35:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient: monmap: 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:34.318492+0000 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:34.318492+0000 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 Processor -- start 2024-09-15T21:35:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 -- start start 2024-09-15T21:35:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient: init 2024-09-15T21:35:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-09-15T21:35:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 0x7f9dec0a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): picked mon.noname-b con 0x7f9dec0a1a20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9dec0a0360 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9dec0a2340 0x7f9dec0a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): picked mon.noname-a con 0x7f9dec0a2340 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec06f5a0 con 0x7f9dec0a0360 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec06ea00 con 0x7f9dec0a2340 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec09f980 con 0x7f9dec0a1a20 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9df087b640 10 monclient(hunting): authenticate will time out at 2057.713851s 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 0x7f9dec0a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9dec0a2340 0x7f9dec0a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 0x7f9dec0a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60288/0 (socket says 172.21.15.17:60288) 2024-09-15T21:35:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9dec0a2340 0x7f9dec0a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38406/0 (socket says 172.21.15.17:38406) 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 1 -- 172.21.15.17:0/1035590803 learned_addr learned my addr 172.21.15.17:0/1035590803 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): get_auth_request con 0x7f9dec0a1a20 auth_method 0 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): get_auth_request con 0x7f9dec0a2340 auth_method 0 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient(hunting): handle_auth_done global_id 204261 payload 274 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 -- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 msgr2=0x7f9dec0a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 --2- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 0x7f9dec0a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 -- 172.21.15.17:0/1035590803 >> v1:172.21.15.119:6789/0 conn(0x7f9dec0a0360 legacy=0x7f9dec0a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 monclient: found mon.noname-a 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dead9d640 1 -- 172.21.15.17:0/1035590803 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0a5810 con 0x7f9dec0a2340 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.316+0000 7f9dea59c640 1 --2- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 0x7f9dec0a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dead9d640 1 --2- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a2340 0x7f9dec0a4190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009d90 tx=0x7f9de401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d7b7076b04c728 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 1 -- 172.21.15.17:0/1035590803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a6710 con 0x7f9dec0a2340 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: dump: 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.328 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 5 monclient: authenticate success, global_id 204261 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 1 -- 172.21.15.17:0/1035590803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de40a68b0 con 0x7f9dec0a2340 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 1 -- 172.21.15.17:0/1035590803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a6c80 con 0x7f9dec0a2340 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de959a640 10 monclient: dump: 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.329 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 4 set_mon_vals no callback set 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: shutdown 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a2340 msgr2=0x7f9dec0a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 --2- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a2340 0x7f9dec0a4190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009d90 tx=0x7f9de401c8c0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- 172.21.15.17:0/1035590803 shutdown_connections 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 --2- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a2340 0x7f9dec0a4190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 --2- 172.21.15.17:0/1035590803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a1a20 0x7f9dec0a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- 172.21.15.17:0/1035590803 >> 172.21.15.17:0/1035590803 conn(0x7f9dec0a8980 msgr2=0x7f9dec0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- 172.21.15.17:0/1035590803 shutdown_connections 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- 172.21.15.17:0/1035590803 wait complete. 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: monmap: 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:34.331 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:34.332 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 Processor -- start 2024-09-15T21:35:34.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- start start 2024-09-15T21:35:34.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: init 2024-09-15T21:35:34.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 0x7f9dec128010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): picked mon.noname-a con 0x7f9dec0a1a20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a2340 0x7f9dec125eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): picked mon.noname-c con 0x7f9dec0a2340 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dec13ee80 0x7f9dec1cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): picked mon.noname-b con 0x7f9dec13ee80 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1426a0 con 0x7f9dec0a1a20 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0a48e0 con 0x7f9dec13ee80 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec142820 con 0x7f9dec0a2340 2024-09-15T21:35:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9df087b640 10 monclient(hunting): authenticate will time out at 2057.719662s 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 0x7f9dec128010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a2340 0x7f9dec125eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a2340 0x7f9dec125eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60298/0 (socket says 172.21.15.17:60298) 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 1 -- 172.21.15.17:0/2257799107 learned_addr learned my addr 172.21.15.17:0/2257799107 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 0x7f9dec128010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38422/0 (socket says 172.21.15.17:38422) 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 10 monclient(hunting): get_auth_request con 0x7f9dec0a2340 auth_method 0 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): get_auth_request con 0x7f9dec0a1a20 auth_method 0 2024-09-15T21:35:34.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9dea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.320+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dec13ee80 0x7f9dec1cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 10 monclient(hunting): get_auth_request con 0x7f9dec13ee80 auth_method 0 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 10 monclient(hunting): handle_auth_done global_id 204264 payload 824 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a2340 msgr2=0x7f9dec125eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9dec0a2340 0x7f9dec125eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dec13ee80 msgr2=0x7f9dec1cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dec13ee80 0x7f9dec1cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 monclient: found mon.noname-a 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec1274b0 con 0x7f9dec0a1a20 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dea59c640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 0x7f9dec128010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00e910 tx=0x7f9ddc00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00ca00 con 0x7f9dec0a1a20 2024-09-15T21:35:34.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: dump: 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc00cba0 con 0x7f9dec0a1a20 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc004030 con 0x7f9dec0a1a20 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: dump: 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 5 monclient: authenticate success, global_id 204264 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddbfff640 4 set_mon_vals no callback set 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 10 monclient: _renew_subs 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec0a7280 con 0x7f9dec0a1a20 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 10 monclient: _renew_subs 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec129810 con 0x7f9dec0a1a20 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 0x7f9dec13f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec126f60 con 0x7f9dec0a0360 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 0x7f9dec13f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient: get_auth_request con 0x7f9dec0a0360 auth_method 0 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9ddc0041d0 con 0x7f9dec0a1a20 2024-09-15T21:35:34.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_done global_id 204267 payload 824 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dec0a1e00 con 0x7f9dec0a0360 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dcc03b9f0 0x7f9dcc03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9ddc0505d0 con 0x7f9dec0a1a20 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9de9d9b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 0x7f9dec13f290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9dec127fa0 tx=0x7f9de000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00067e0 con 0x7f9dec0a0360 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: dump: 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dcc03b9f0 0x7f9dcc03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 10 monclient: get_auth_request con 0x7f9dcc03b9f0 auth_method 0 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9de00067e0 con 0x7f9dec0a0360 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:34.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 msgr2=0x7f9dec13f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 0x7f9dec13f290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9dec127fa0 tx=0x7f9de000fc00 comp rx=0 tx=0).stop 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 msgr2=0x7f9dec13f290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9ddaffd640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a0360 0x7f9dec13f290 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9dec127fa0 tx=0x7f9de000fc00 comp rx=0 tx=0).stop 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.324+0000 7f9dead9d640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dcc03b9f0 0x7f9dcc03deb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9de40af000 tx=0x7f9de40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 0x7f9dec02d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0739b0 con 0x7f9dec02d450 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 0x7f9dec02d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient: get_auth_request con 0x7f9dec02d450 auth_method 0 2024-09-15T21:35:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 10 monclient(hunting): handle_auth_done global_id 204270 payload 824 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 1 -- 172.21.15.17:0/2257799107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9dec0694c0 con 0x7f9dec02d450 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9de9d9b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 0x7f9dec02d860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9dec127fa0 tx=0x7f9de0003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0007590 con 0x7f9dec02d450 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 10 monclient: dump: 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4714 (secure 0 0 0) 0x7f9de0007590 con 0x7f9dec02d450 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 msgr2=0x7f9dec02d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 0x7f9dec02d860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9dec127fa0 tx=0x7f9de0003770 comp rx=0 tx=0).stop 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 msgr2=0x7f9dec02d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9ddaffd640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec02d450 0x7f9dec02d860 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9dec127fa0 tx=0x7f9de0003770 comp rx=0 tx=0).stop 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 166, 2024-09-15T21:35:34.352 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 58867, 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 8, 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.353 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.354 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:34.355 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:34.356 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.357 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:34.362 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:34.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.364 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:34.365 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dcc03b9f0 msgr2=0x7f9dcc03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9dcc03b9f0 0x7f9dcc03deb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9de40af000 tx=0x7f9de40a7000 comp rx=0 tx=0).stop 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 10 monclient: shutdown 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 msgr2=0x7f9dec128010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9dead9d640 1 -- 172.21.15.17:0/2257799107 reap_dead start 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 0x7f9dec128010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00e910 tx=0x7f9ddc00edd0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 shutdown_connections 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 --2- 172.21.15.17:0/2257799107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0a1a20 0x7f9dec128010 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 >> 172.21.15.17:0/2257799107 conn(0x7f9dec0a8980 msgr2=0x7f9dec0a1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 shutdown_connections 2024-09-15T21:35:34.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.332+0000 7f9df087b640 1 -- 172.21.15.17:0/2257799107 wait complete. 2024-09-15T21:35:34.368 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:35:34.368 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:35:34.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:35:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient: get_monmap_and_config 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient: monmap: 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:34.486752+0000 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:34.486752+0000 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:34.488 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:34.488 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:34.488 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:34.488 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 1 Processor -- start 2024-09-15T21:35:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 1 -- start start 2024-09-15T21:35:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient: init 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fac78073640 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fac78074130 0x7fac7813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient(hunting): picked mon.noname-c con 0x7fac78074130 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fac7813d140 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac780a0890 con 0x7fac78073640 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac780746e0 con 0x7fac7813d140 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fac780a1640 con 0x7fac78074130 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 10 monclient(hunting): authenticate will time out at 2057.882178s 2024-09-15T21:35:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac78073640 0x7fac78073a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58288/0 (socket says 172.21.15.17:58288) 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 1 -- 172.21.15.17:0/302326403 learned_addr learned my addr 172.21.15.17:0/302326403 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 --2- 172.21.15.17:0/302326403 >> v2:172.21.15.119:3300/0 conn(0x7fac78074130 0x7fac7813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): get_auth_request con 0x7fac78074130 auth_method 0 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594736955 0 0) 0x7fac780a0890 con 0x7fac78073640 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): _negotiate 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): my global_id is 204273 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac60003530 con 0x7fac78073640 2024-09-15T21:35:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 958778933 0 0) 0x7fac60003530 con 0x7fac78073640 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 >> v2:172.21.15.119:3300/0 conn(0x7fac78074130 msgr2=0x7fac7813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 --2- 172.21.15.17:0/302326403 >> v2:172.21.15.119:3300/0 conn(0x7fac78074130 0x7fac7813cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 >> v1:172.21.15.119:6789/0 conn(0x7fac7813d140 legacy=0x7fac7813f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac780a1920 con 0x7fac78073640 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fac70001a80 con 0x7fac78073640 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: dump: 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 >> v1:172.21.15.17:6789/0 conn(0x7fac78073640 legacy=0x7fac78073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac60003a60 0x7fac60003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): picked mon.b con 0x7fac60003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac60004810 0x7fac60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): picked mon.c con 0x7fac60004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac600053d0 0x7fac6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): picked mon.a con 0x7fac600053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fac780a1640 con 0x7fac600053d0 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac6000c240 con 0x7fac60003a60 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fac6000c440 con 0x7fac60004810 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7d31b640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac60004810 0x7fac60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): get_auth_request con 0x7fac60004810 auth_method 0 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac600053d0 0x7fac6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac60003a60 0x7fac60003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): get_auth_request con 0x7fac600053d0 auth_method 0 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 10 monclient(hunting): get_auth_request con 0x7fac60003a60 auth_method 0 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient(hunting): handle_auth_done global_id 204273 payload 293 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 -- 172.21.15.17:0/302326403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac60003a60 msgr2=0x7fac60003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac60003a60 0x7fac60003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 -- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac600053d0 msgr2=0x7fac6000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac600053d0 0x7fac6000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 monclient: found mon.c 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 1 -- 172.21.15.17:0/302326403 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac60003530 con 0x7fac60004810 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7db1c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7eb1e640 1 -- 172.21.15.17:0/302326403 reap_dead start 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac7e31d640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac60003a60 0x7fac60003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 5 monclient: authenticate success, global_id 204273 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.484+0000 7fac80581640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7db1c640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac60004810 0x7fac60004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009ae0 tx=0x7fac6c01ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa879c05d04290d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0b2070 con 0x7fac60004810 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: dump: 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c0a5550 con 0x7fac60004810 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 1 -- 172.21.15.17:0/302326403 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0096b0 con 0x7fac60004810 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:34.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 4 set_mon_vals no callback set 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac7d31b640 10 monclient: dump: 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 10 monclient: shutdown 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 -- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac60004810 msgr2=0x7fac60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac60004810 0x7fac60004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009ae0 tx=0x7fac6c01ca50 comp rx=0 tx=0).stop 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 -- 172.21.15.17:0/302326403 shutdown_connections 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 --2- 172.21.15.17:0/302326403 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac60004810 0x7fac60004c90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 -- 172.21.15.17:0/302326403 >> 172.21.15.17:0/302326403 conn(0x7fac780a66a0 msgr2=0x7fac780a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 -- 172.21.15.17:0/302326403 shutdown_connections 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 -- 172.21.15.17:0/302326403 wait complete. 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 10 monclient: monmap: 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 Processor -- start 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 1 -- start start 2024-09-15T21:35:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.488+0000 7fac80581640 10 monclient: init 2024-09-15T21:35:34.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:35:34.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac60004810 0x7fac78133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): picked mon.noname-a con 0x7fac60004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac781342b0 0x7fac78134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): picked mon.noname-c con 0x7fac781342b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac78120f40 0x7fac78121340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): picked mon.noname-b con 0x7fac78120f40 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fac780a0e80 con 0x7fac60004810 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac780a0d00 con 0x7fac78120f40 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fac78137ea0 con 0x7fac781342b0 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): authenticate will time out at 2057.888152s 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac60004810 0x7fac78133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac781342b0 0x7fac78134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac60004810 0x7fac78133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38452/0 (socket says 172.21.15.17:38452) 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 1 -- 172.21.15.17:0/701077660 learned_addr learned my addr 172.21.15.17:0/701077660 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): get_auth_request con 0x7fac60004810 auth_method 0 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): get_auth_request con 0x7fac781342b0 auth_method 0 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac78120f40 0x7fac78121340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 10 monclient(hunting): get_auth_request con 0x7fac78120f40 auth_method 0 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient(hunting): handle_auth_done global_id 214133 payload 824 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac78120f40 msgr2=0x7fac78121340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac78120f40 0x7fac78121340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac60004810 msgr2=0x7fac78133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac60004810 0x7fac78133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 monclient: found mon.noname-c 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac78124a80 con 0x7fac781342b0 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7db1c640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac781342b0 0x7fac78134730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0ac000 tx=0x7fac6c0197f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0b2040 con 0x7fac781342b0 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: dump: 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 5 monclient: authenticate success, global_id 214133 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient: _renew_subs 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c0a5e00 con 0x7fac781342b0 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac78124170 con 0x7fac781342b0 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c004020 con 0x7fac781342b0 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: dump: 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient: _renew_subs 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac77fff640 4 set_mon_vals no callback set 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac78135330 con 0x7fac781342b0 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 0x7fac78121c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fac7809d0f0 con 0x7fac78121880 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 0x7fac78121c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fac6c0bf030 con 0x7fac781342b0 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient: get_auth_request con 0x7fac78121880 auth_method 0 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 10 monclient(hunting): handle_auth_done global_id 214136 payload 824 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac6403b9c0 0x7fac6403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac780a0890 con 0x7fac78121880 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fac6c0ea0a0 con 0x7fac781342b0 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7e31d640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 0x7fac78121c60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac700079d0 tx=0x7fac70011620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac6403b9c0 0x7fac6403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 10 monclient: get_auth_request con 0x7fac6403b9c0 auth_method 0 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70019020 con 0x7fac78121880 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: dump: 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fac70015470 con 0x7fac78121880 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 msgr2=0x7fac78121c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 0x7fac78121c60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac700079d0 tx=0x7fac70011620 comp rx=0 tx=0).stop 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 msgr2=0x7fac78121c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac76ffd640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac78121880 0x7fac78121c60 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac700079d0 tx=0x7fac70011620 comp rx=0 tx=0).stop 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.492+0000 7fac7eb1e640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac6403b9c0 0x7fac6403de80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac780a3530 tx=0x7fac68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 0x7fac7811c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fac78135650 con 0x7fac7811c4e0 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 0x7fac7811c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient: get_auth_request con 0x7fac7811c4e0 auth_method 0 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 10 monclient(hunting): handle_auth_done global_id 214139 payload 824 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 1 -- 172.21.15.17:0/701077660 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fac7806d020 con 0x7fac7811c4e0 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7e31d640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 0x7fac7811c8f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac700079d0 tx=0x7fac7806ccf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70019030 con 0x7fac7811c4e0 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 10 monclient: dump: 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.523 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7fac70015ba0 con 0x7fac7811c4e0 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 msgr2=0x7fac7811c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 0x7fac7811c8f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac700079d0 tx=0x7fac7806ccf0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 msgr2=0x7fac7811c8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac76ffd640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac7811c4e0 0x7fac7811c8f0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac700079d0 tx=0x7fac7806ccf0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:35:34.525 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 166, 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 13747, 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 8, 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.526 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.527 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:34.528 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:34.538 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac6403b9c0 msgr2=0x7fac6403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fac6403b9c0 0x7fac6403de80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac780a3530 tx=0x7fac68009040 comp rx=0 tx=0).stop 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 10 monclient: shutdown 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac781342b0 msgr2=0x7fac78134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 --2- 172.21.15.17:0/701077660 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fac781342b0 0x7fac78134730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0ac000 tx=0x7fac6c0197f0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7eb1e640 1 -- 172.21.15.17:0/701077660 reap_dead start 2024-09-15T21:35:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac7eb1e640 1 -- 172.21.15.17:0/701077660 reap_dead start 2024-09-15T21:35:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 shutdown_connections 2024-09-15T21:35:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 >> 172.21.15.17:0/701077660 conn(0x7fac780a66a0 msgr2=0x7fac780a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 shutdown_connections 2024-09-15T21:35:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.500+0000 7fac80581640 1 -- 172.21.15.17:0/701077660 wait complete. 2024-09-15T21:35:34.540 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:35:34.540 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:35:34.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:35:34.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.652+0000 7f0147739640 10 monclient: get_monmap_and_config 2024-09-15T21:35:34.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.652+0000 7f0147739640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.652+0000 7f0147739640 10 monclient: monmap: 2024-09-15T21:35:34.657 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:34.657325+0000 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:34.657325+0000 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:34.658 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.652+0000 7f0147739640 1 Processor -- start 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.652+0000 7f0147739640 1 -- start start 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient: init 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01400a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f014009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient(hunting): picked mon.noname-b con 0x7f01400a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01400a3130 con 0x7f01400a0890 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0140073df0 con 0x7f014009ffc0 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f01400a1bf0 con 0x7f01400a1380 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient(hunting): authenticate will time out at 2058.052700s 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f01454d5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01400a0890 0x7f01400a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58292/0 (socket says 172.21.15.17:58292) 2024-09-15T21:35:34.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f01454d5640 1 -- 172.21.15.17:0/2143769301 learned_addr learned my addr 172.21.15.17:0/2143769301 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 1 --2- 172.21.15.17:0/2143769301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): get_auth_request con 0x7f01400a1380 auth_method 0 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 1 -- 172.21.15.17:0/2143769301 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3385764509 0 0) 0x7f01400a3130 con 0x7f01400a0890 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient(hunting): _negotiate 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient(hunting): my global_id is 204279 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 1 -- 172.21.15.17:0/2143769301 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f012c003530 con 0x7f01400a0890 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient(hunting): handle_auth_done global_id 214142 payload 274 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 1 -- 172.21.15.17:0/2143769301 >> v1:172.21.15.119:6789/0 conn(0x7f014009ffc0 legacy=0x7f014013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 1 -- 172.21.15.17:0/2143769301 >> v1:172.21.15.17:6789/0 conn(0x7f01400a0890 legacy=0x7f01400a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 1 monclient: found mon.noname-b 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 1 -- 172.21.15.17:0/2143769301 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014009ccd0 con 0x7f01400a1380 2024-09-15T21:35:34.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0145cd6640 1 --2- 172.21.15.17:0/2143769301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014013e1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0138009df0 tx=0x7f013801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850ec55f0546fe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 1 -- 172.21.15.17:0/2143769301 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380b2070 con 0x7f01400a1380 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: dump: 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.670 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 5 monclient: authenticate success, global_id 214142 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 1 -- 172.21.15.17:0/2143769301 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01380a5820 con 0x7f01400a1380 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0147739640 4 set_mon_vals no callback set 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 1 -- 172.21.15.17:0/2143769301 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380a5bd0 con 0x7f01400a1380 2024-09-15T21:35:34.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.656+0000 7f0137fff640 10 monclient: dump: 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: shutdown 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/2143769301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 msgr2=0x7f014013e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 --2- 172.21.15.17:0/2143769301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014013e1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0138009df0 tx=0x7f013801c850 comp rx=0 tx=0).stop 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/2143769301 shutdown_connections 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 --2- 172.21.15.17:0/2143769301 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014013e1f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/2143769301 >> 172.21.15.17:0/2143769301 conn(0x7f01400a66a0 msgr2=0x7f01400a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/2143769301 shutdown_connections 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/2143769301 wait complete. 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: monmap: 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:34.673 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 Processor -- start 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- start start 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: init 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:35:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014009ffc0 0x7f0140135370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): picked mon.noname-a con 0x7f014009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01400a0890 0x7f01401358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): picked mon.noname-b con 0x7f01400a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): picked mon.noname-c con 0x7f01400a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f01401412e0 con 0x7f014009ffc0 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0140141160 con 0x7f01400a0890 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0140141460 con 0x7f01400a1380 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient(hunting): authenticate will time out at 2058.058130s 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014011c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60330/0 (socket says 172.21.15.17:60330) 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 -- 172.21.15.17:0/224817740 learned_addr learned my addr 172.21.15.17:0/224817740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): get_auth_request con 0x7f01400a1380 auth_method 0 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014009ffc0 0x7f0140135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): get_auth_request con 0x7f014009ffc0 auth_method 0 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient(hunting): handle_auth_done global_id 214145 payload 824 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0144cd4640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01400a0890 0x7f01401358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01400a0890 msgr2=0x7f01401358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01400a0890 0x7f01401358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014009ffc0 msgr2=0x7f0140135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014009ffc0 0x7f0140135370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 monclient: found mon.noname-c 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014011ee50 con 0x7f01400a1380 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01454d5640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014009ffc0 0x7f0140135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0144cd4640 10 monclient: get_auth_request con 0x7f01400a0890 auth_method 0 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0144cd4640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01400a0890 0x7f01401358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0145cd6640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014011c4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01380b0040 tx=0x7f01380195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380bd020 con 0x7f01400a1380 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: dump: 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.678 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01380b3040 con 0x7f01400a1380 2024-09-15T21:35:34.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380b2050 con 0x7f01400a1380 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0135ffb640 10 monclient: dump: 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.680 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f01377fe640 4 set_mon_vals no callback set 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 5 monclient: authenticate success, global_id 214145 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: _renew_subs 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014011f130 con 0x7f01400a1380 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: _renew_subs 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.660+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f014006f790 con 0x7f01400a1380 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0147739640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0147739640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 0x7f014011ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0147739640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f014009d0f0 con 0x7f014011ea60 2024-09-15T21:35:34.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f01380039a0 con 0x7f01400a1380 2024-09-15T21:35:34.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f012403b9c0 0x7f012403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f01380eb2c0 con 0x7f01400a1380 2024-09-15T21:35:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 0x7f014011ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient: get_auth_request con 0x7f014011ea60 auth_method 0 2024-09-15T21:35:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0144cd4640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f012403b9c0 0x7f012403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0144cd4640 10 monclient: get_auth_request con 0x7f012403b9c0 auth_method 0 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 10 monclient(hunting): handle_auth_done global_id 194344 payload 824 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01400a0c70 con 0x7f014011ea60 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f01454d5640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 0x7f014011ca20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0130005890 tx=0x7f0130007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0144cd4640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f012403b9c0 0x7f012403de80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f014013d9d0 tx=0x7f013c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130005ad0 con 0x7f014011ea60 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 10 monclient: dump: 2024-09-15T21:35:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0130018070 con 0x7f014011ea60 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:34.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 msgr2=0x7f014011ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 0x7f014011ca20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0130005890 tx=0x7f0130007770 comp rx=0 tx=0).stop 2024-09-15T21:35:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 msgr2=0x7f014011ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:34.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.664+0000 7f0135ffb640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011ea60 0x7f014011ca20 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0130005890 tx=0x7f0130007770 comp rx=0 tx=0).stop 2024-09-15T21:35:34.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:34.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 0x7f0140137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f014011d340 con 0x7f014011cf60 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 0x7f0140137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient: get_auth_request con 0x7f014011cf60 auth_method 0 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 10 monclient(hunting): handle_auth_done global_id 194347 payload 824 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 1 -- 172.21.15.17:0/224817740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0140070500 con 0x7f014011cf60 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f01454d5640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 0x7f0140137ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0130005890 tx=0x7f0130005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013000fa90 con 0x7f014011cf60 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 10 monclient: dump: 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.691 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7f013000fa90 con 0x7f014011cf60 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 msgr2=0x7f0140137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 0x7f0140137ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0130005890 tx=0x7f0130005800 comp rx=0 tx=0).stop 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 msgr2=0x7f0140137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0135ffb640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014011cf60 0x7f0140137ab0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0130005890 tx=0x7f0130005800 comp rx=0 tx=0).stop 2024-09-15T21:35:34.692 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 166, 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 101465, 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 8, 2024-09-15T21:35:34.693 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.694 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.695 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:34.696 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:34.697 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:34.699 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:34.702 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:34.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:34.704 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:34.705 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f012403b9c0 msgr2=0x7f012403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f012403b9c0 0x7f012403de80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f014013d9d0 tx=0x7f013c009040 comp rx=0 tx=0).stop 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 10 monclient: shutdown 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 msgr2=0x7f014011c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 --2- 172.21.15.17:0/224817740 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f01400a1380 0x7f014011c4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01380b0040 tx=0x7f01380195f0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0145cd6640 1 -- 172.21.15.17:0/224817740 reap_dead start 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0145cd6640 1 -- 172.21.15.17:0/224817740 reap_dead start 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 shutdown_connections 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 >> 172.21.15.17:0/224817740 conn(0x7f01400a66a0 msgr2=0x7f014006fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 shutdown_connections 2024-09-15T21:35:34.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.672+0000 7f0147739640 1 -- 172.21.15.17:0/224817740 wait complete. 2024-09-15T21:35:34.708 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:35:34.708 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-15T21:35:34.708 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T21:35:34.708 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T21:35:34.708 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T21:35:34.708 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:35:34.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:34.738 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:35:34.738 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:35:34.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 10 monclient: get_monmap_and_config 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 10 monclient: monmap: 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:34.851989+0000 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:34.851989+0000 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 1 Processor -- start 2024-09-15T21:35:34.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 1 -- start start 2024-09-15T21:35:34.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 10 monclient: init 2024-09-15T21:35:34.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.848+0000 7f48ce6db640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4c00 0x7f48c809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient(hunting): picked mon.noname-b con 0x7f48c80a4c00 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f48c80a4fe0 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f48c813ce10 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c809c700 con 0x7f48c813ce10 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c80a5d10 con 0x7f48c80a4fe0 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f48c806f820 con 0x7f48c80a4c00 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient(hunting): authenticate will time out at 2058.248144s 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4c00 0x7f48c809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ccc78640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f48c813ce10 0x7f48c813f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58308/0 (socket says 172.21.15.17:58308) 2024-09-15T21:35:34.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ccc78640 1 -- 172.21.15.17:0/3990934467 learned_addr learned my addr 172.21.15.17:0/3990934467 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48bffff640 1 -- 172.21.15.17:0/3990934467 >> v1:172.21.15.119:6789/0 conn(0x7f48c80a4fe0 legacy=0x7f48c80a53e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): get_auth_request con 0x7f48c80a4c00 auth_method 0 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 1 -- 172.21.15.17:0/3990934467 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682900451 0 0) 0x7f48c809c700 con 0x7f48c813ce10 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient(hunting): _negotiate 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient(hunting): my global_id is 204285 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 1 -- 172.21.15.17:0/3990934467 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48b4003530 con 0x7f48c813ce10 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient(hunting): handle_auth_done global_id 214148 payload 274 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 1 -- 172.21.15.17:0/3990934467 >> v1:172.21.15.119:6789/0 conn(0x7f48c80a4fe0 legacy=0x7f48c80a53e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 1 -- 172.21.15.17:0/3990934467 >> v1:172.21.15.17:6789/0 conn(0x7f48c813ce10 legacy=0x7f48c813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 1 monclient: found mon.noname-b 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 1 -- 172.21.15.17:0/3990934467 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c806f200 con 0x7f48c80a4c00 2024-09-15T21:35:34.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c7fff640 1 --2- 172.21.15.17:0/3990934467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4c00 0x7f48c809cb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48b80097f0 tx=0x7f48b801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbd0a18a0d2d60e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 1 -- 172.21.15.17:0/3990934467 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b80b0070 con 0x7f48c80a4c00 2024-09-15T21:35:34.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:35:34.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: dump: 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 1 -- 172.21.15.17:0/3990934467 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b80ab600 con 0x7f48c80a4c00 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 1 -- 172.21.15.17:0/3990934467 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b80ab9b0 con 0x7f48c80a4c00 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 5 monclient: authenticate success, global_id 214148 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 4 set_mon_vals no callback set 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48c77fe640 10 monclient: dump: 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.865 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 10 monclient: shutdown 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- 172.21.15.17:0/3990934467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4c00 msgr2=0x7f48c809cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3990934467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4c00 0x7f48c809cb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48b80097f0 tx=0x7f48b801c430 comp rx=0 tx=0).stop 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- 172.21.15.17:0/3990934467 shutdown_connections 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3990934467 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4c00 0x7f48c809cb20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- 172.21.15.17:0/3990934467 >> 172.21.15.17:0/3990934467 conn(0x7f48c80aaeb0 msgr2=0x7f48c80a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- 172.21.15.17:0/3990934467 shutdown_connections 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.852+0000 7f48ce6db640 1 -- 172.21.15.17:0/3990934467 wait complete. 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: build_initial_monmap 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: monmap: 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:34.867 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 Processor -- start 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- start start 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: init 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80a4c00 0x7f48c8137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): picked mon.noname-a con 0x7f48c80a4c00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 0x7f48c8137e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): picked mon.noname-c con 0x7f48c80a4fe0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48c813ce10 0x7f48c8139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): picked mon.noname-b con 0x7f48c813ce10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:34.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8143760 con 0x7f48c80a4c00 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81438e0 con 0x7f48c813ce10 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f48c8143a60 con 0x7f48c80a4fe0 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient(hunting): authenticate will time out at 2058.253193s 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 0x7f48c8137e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 0x7f48c8137e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60348/0 (socket says 172.21.15.17:60348) 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 -- 172.21.15.17:0/3714146719 learned_addr learned my addr 172.21.15.17:0/3714146719 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): get_auth_request con 0x7f48c80a4fe0 auth_method 0 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ccc78640 1 -- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48c813ce10 msgr2=0x7f48c8139bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ccc78640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48c813ce10 0x7f48c8139bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80a4c00 0x7f48c8137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): get_auth_request con 0x7f48c80a4c00 auth_method 0 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient(hunting): handle_auth_done global_id 214151 payload 824 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 -- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48c813ce10 msgr2=0x7f48c8139bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48c813ce10 0x7f48c8139bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 -- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80a4c00 msgr2=0x7f48c8137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80a4c00 0x7f48c8137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient: _finish_hunting 0 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 monclient: found mon.noname-c 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 -- 172.21.15.17:0/3714146719 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c8121460 con 0x7f48c80a4fe0 2024-09-15T21:35:34.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48bffff640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 0x7f48c8137e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48b000b510 tx=0x7f48b000b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0003d80 con 0x7f48c80a4fe0 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: dump: 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: _finish_auth 0 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b0015070 con 0x7f48c80a4fe0 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00052b0 con 0x7f48c80a4fe0 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 10 monclient: dump: 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:34.873 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 5 monclient: authenticate success, global_id 214151 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _renew_subs 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c81405b0 con 0x7f48c80a4fe0 2024-09-15T21:35:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c6ffd640 4 set_mon_vals no callback set 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _renew_subs 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c8121180 con 0x7f48c80a4fe0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f48b0005450 con 0x7f48c80a4fe0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c809cb20 con 0x7f48c80a4fe0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f48a803b9a0 0x7f48a803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f48b00503c0 con 0x7f48c80a4fe0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f48a803b9a0 0x7f48a803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.856+0000 7f48c7fff640 10 monclient: get_auth_request con 0x7f48a803b9a0 auth_method 0 2024-09-15T21:35:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.860+0000 7f48c7fff640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f48a803b9a0 0x7f48a803de60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f48b80ad000 tx=0x7f48b80a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:34.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.860+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f48b0010c30 con 0x7f48c80a4fe0 2024-09-15T21:35:34.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.860+0000 7f48c57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:34.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:34.860+0000 7f48c57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.100+0000 7f48ce6db640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.100+0000 7f48ce6db640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.100+0000 7f48ce6db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.100+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f48c8120e00 con 0x7f48c80a4fe0 2024-09-15T21:35:35.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48c57fa640 1 -- 172.21.15.17:0/3714146719 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f48b000d360 con 0x7f48c80a4fe0 2024-09-15T21:35:35.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48c57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:35.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48c57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:35.107 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f48a803b9a0 msgr2=0x7f48a803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f48a803b9a0 0x7f48a803de60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f48b80ad000 tx=0x7f48b80a4f90 comp rx=0 tx=0).stop 2024-09-15T21:35:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 10 monclient: shutdown 2024-09-15T21:35:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 msgr2=0x7f48c8137e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 0x7f48c8137e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48b000b510 tx=0x7f48b000b9d0 comp rx=0 tx=0).stop 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 shutdown_connections 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f48a803b9a0 0x7f48a803de60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48c813ce10 0x7f48c8139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f48c80a4fe0 0x7f48c8137e40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 --2- 172.21.15.17:0/3714146719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80a4c00 0x7f48c8137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 >> 172.21.15.17:0/3714146719 conn(0x7f48c80aaeb0 msgr2=0x7f48c80a2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 shutdown_connections 2024-09-15T21:35:35.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:35.104+0000 7f48ce6db640 1 -- 172.21.15.17:0/3714146719 wait complete. 2024-09-15T21:35:38.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:38.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.256+0000 7f23b0235640 10 monclient: get_monmap_and_config 2024-09-15T21:35:38.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.256+0000 7f23b0235640 10 monclient: build_initial_monmap 2024-09-15T21:35:38.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.256+0000 7f23b0235640 10 monclient: monmap: 2024-09-15T21:35:38.261 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:38.261 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:38.261201+0000 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:38.261201+0000 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:38.262 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.256+0000 7f23b0235640 1 Processor -- start 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.256+0000 7f23b0235640 1 -- start start 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.256+0000 7f23b0235640 10 monclient: init 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f23a80a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:38.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23a809ffc0 0x7f23a813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient(hunting): picked mon.noname-a con 0x7f23a809ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f23a80a1380 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a80a3130 con 0x7f23a80a1380 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a8073df0 con 0x7f23a80a0890 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f23a80a1bf0 con 0x7f23a809ffc0 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient(hunting): authenticate will time out at 2061.656620s 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ae7d2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f23a80a1380 0x7f23a813d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50640/0 (socket says 172.21.15.17:50640) 2024-09-15T21:35:38.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ae7d2640 1 -- 172.21.15.17:0/1658984968 learned_addr learned my addr 172.21.15.17:0/1658984968 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23adfd1640 1 -- 172.21.15.17:0/1658984968 >> v1:172.21.15.119:6789/0 conn(0x7f23a80a0890 legacy=0x7f23a80a0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 1 --2- 172.21.15.17:0/1658984968 >> v2:172.21.15.17:3300/0 conn(0x7f23a809ffc0 0x7f23a813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): get_auth_request con 0x7f23a809ffc0 auth_method 0 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 1 -- 172.21.15.17:0/1658984968 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1098876837 0 0) 0x7f23a80a3130 con 0x7f23a80a1380 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient(hunting): _negotiate 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient(hunting): my global_id is 204291 2024-09-15T21:35:38.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 1 -- 172.21.15.17:0/1658984968 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2394003530 con 0x7f23a80a1380 2024-09-15T21:35:38.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:38.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:38.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:38.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_done global_id 204294 payload 274 2024-09-15T21:35:38.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 1 -- 172.21.15.17:0/1658984968 >> v1:172.21.15.119:6789/0 conn(0x7f23a80a0890 legacy=0x7f23a80a0c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:38.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 1 -- 172.21.15.17:0/1658984968 >> v1:172.21.15.17:6789/0 conn(0x7f23a80a1380 legacy=0x7f23a813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:38.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient: _finish_hunting 0 2024-09-15T21:35:38.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 1 monclient: found mon.noname-a 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 1 -- 172.21.15.17:0/1658984968 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3524418474 0 0) 0x7f2394003530 con 0x7f23a80a1380 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 1 -- 172.21.15.17:0/1658984968 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a809ccd0 con 0x7f23a809ffc0 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23ad7d0640 1 --2- 172.21.15.17:0/1658984968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a809ffc0 0x7f23a813aac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f23a40097f0 tx=0x7f23a401c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b3533fc9708ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 1 -- 172.21.15.17:0/1658984968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b0070 con 0x7f23a809ffc0 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: dump: 2024-09-15T21:35:38.274 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: _finish_auth 0 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: _check_auth_tickets 2024-09-15T21:35:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 1 -- 172.21.15.17:0/1658984968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a40ab400 con 0x7f23a809ffc0 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 5 monclient: authenticate success, global_id 204294 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23b0235640 4 set_mon_vals no callback set 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 1 -- 172.21.15.17:0/1658984968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0001a80 con 0x7f23a809ffc0 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.260+0000 7f23acfcf640 10 monclient: dump: 2024-09-15T21:35:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: shutdown 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/1658984968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a809ffc0 msgr2=0x7f23a813aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 --2- 172.21.15.17:0/1658984968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a809ffc0 0x7f23a813aac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f23a40097f0 tx=0x7f23a401c280 comp rx=0 tx=0).stop 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/1658984968 shutdown_connections 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 --2- 172.21.15.17:0/1658984968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a809ffc0 0x7f23a813aac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/1658984968 >> 172.21.15.17:0/1658984968 conn(0x7f23a80a66a0 msgr2=0x7f23a80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/1658984968 shutdown_connections 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/1658984968 wait complete. 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: build_initial_monmap 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: monmap: 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:38.278 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 Processor -- start 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- start start 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: init 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a809ffc0 0x7f23a8135810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): picked mon.noname-b con 0x7f23a809ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 0x7f23a811d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): picked mon.noname-a con 0x7f23a80a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23a80a1380 0x7f23a811d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): picked mon.noname-c con 0x7f23a80a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81412e0 con 0x7f23a80a0890 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8141160 con 0x7f23a809ffc0 2024-09-15T21:35:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f23a8141460 con 0x7f23a80a1380 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient(hunting): authenticate will time out at 2061.662021s 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 0x7f23a811d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23adfd1640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a809ffc0 msgr2=0x7f23a8135810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 0x7f23a811d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56496/0 (socket says 172.21.15.17:56496) 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 -- 172.21.15.17:0/3235724359 learned_addr learned my addr 172.21.15.17:0/3235724359 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): get_auth_request con 0x7f23a80a0890 auth_method 0 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23adfd1640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a809ffc0 0x7f23a8135810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23a80a1380 0x7f23a811d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): get_auth_request con 0x7f23a80a1380 auth_method 0 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ae7d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient(hunting): handle_auth_done global_id 204297 payload 824 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 -- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23a80a1380 msgr2=0x7f23a811d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23a80a1380 0x7f23a811d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 -- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a809ffc0 msgr2=0x7f23a8135810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a809ffc0 0x7f23a8135810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient: _finish_hunting 0 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 monclient: found mon.noname-a 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 -- 172.21.15.17:0/3235724359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a8136450 con 0x7f23a80a0890 2024-09-15T21:35:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23ad7d0640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 0x7f23a811d270 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ae040 tx=0x7f23a401aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40bb020 con 0x7f23a80a0890 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: dump: 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a40b1040 con 0x7f23a80a0890 2024-09-15T21:35:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b0050 con 0x7f23a80a0890 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 5 monclient: authenticate success, global_id 204297 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: _renew_subs 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a811ecf0 con 0x7f23a80a0890 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239e7fc640 10 monclient: dump: 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f239ffff640 4 set_mon_vals no callback set 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: _renew_subs 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.264+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a81207b0 con 0x7f23a80a0890 2024-09-15T21:35:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23b0235640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23b0235640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23b0235640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a80a0c70 con 0x7f23a80a0890 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f23a40be070 con 0x7f23a80a0890 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f239e7fc640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f238c03b970 0x7f238c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f23a40ea0b0 con 0x7f23a80a0890 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23adfd1640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f238c03b970 0x7f238c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23adfd1640 10 monclient: get_auth_request con 0x7f238c03b970 auth_method 0 2024-09-15T21:35:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.268+0000 7f23adfd1640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f238c03b970 0x7f238c03de30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2398006fd0 tx=0x7f2398006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.272+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f23a4015fd0 con 0x7f23a80a0890 2024-09-15T21:35:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.272+0000 7f239e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.272+0000 7f239e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:38.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f23b0235640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f23b0235640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f23b0235640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f23a8136620 con 0x7f23a80a0890 2024-09-15T21:35:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f239e7fc640 1 -- 172.21.15.17:0/3235724359 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f23a4004460 con 0x7f23a80a0890 2024-09-15T21:35:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f239e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.512+0000 7f239e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:38.517 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:38.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f238c03b970 msgr2=0x7f238c03de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:38.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f238c03b970 0x7f238c03de30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2398006fd0 tx=0x7f2398006d40 comp rx=0 tx=0).stop 2024-09-15T21:35:38.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 10 monclient: shutdown 2024-09-15T21:35:38.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 msgr2=0x7f23a811d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 0x7f23a811d270 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ae040 tx=0x7f23a401aac0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 shutdown_connections 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f238c03b970 0x7f238c03de30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f23a80a1380 0x7f23a811d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a80a0890 0x7f23a811d270 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 --2- 172.21.15.17:0/3235724359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a809ffc0 0x7f23a8135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 >> 172.21.15.17:0/3235724359 conn(0x7f23a80a66a0 msgr2=0x7f23a809d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 shutdown_connections 2024-09-15T21:35:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:38.516+0000 7f23b0235640 1 -- 172.21.15.17:0/3235724359 wait complete. 2024-09-15T21:35:41.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.664+0000 7fba1d9bd640 10 monclient: get_monmap_and_config 2024-09-15T21:35:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.664+0000 7fba1d9bd640 10 monclient: build_initial_monmap 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.664+0000 7fba1d9bd640 10 monclient: monmap: 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:41.668477+0000 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:41.668477+0000 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:41.670 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:41.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.664+0000 7fba1d9bd640 1 Processor -- start 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.664+0000 7fba1d9bd640 1 -- start start 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.664+0000 7fba1d9bd640 10 monclient: init 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fba181378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba181383b0 0x7fba1813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): picked mon.noname-b con 0x7fba181383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fba1813d2c0 0x7fba1813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): picked mon.noname-a con 0x7fba1813d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba180a1a20 con 0x7fba181378c0 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fba180a1d00 con 0x7fba1813d2c0 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fba180a1fe0 con 0x7fba181383b0 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient(hunting): authenticate will time out at 2065.064428s 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba16ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fba181378c0 0x7fba18137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50644/0 (socket says 172.21.15.17:50644) 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba16ffd640 1 -- 172.21.15.17:0/3345239562 learned_addr learned my addr 172.21.15.17:0/3345239562 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 --2- 172.21.15.17:0/3345239562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba181383b0 0x7fba1813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): get_auth_request con 0x7fba181383b0 auth_method 0 2024-09-15T21:35:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 1 --2- 172.21.15.17:0/3345239562 >> v2:172.21.15.17:3300/0 conn(0x7fba1813d2c0 0x7fba1813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): get_auth_request con 0x7fba1813d2c0 auth_method 0 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:41.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient(hunting): handle_auth_done global_id 214157 payload 274 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 -- 172.21.15.17:0/3345239562 >> v2:172.21.15.17:3300/0 conn(0x7fba1813d2c0 msgr2=0x7fba1813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 --2- 172.21.15.17:0/3345239562 >> v2:172.21.15.17:3300/0 conn(0x7fba1813d2c0 0x7fba1813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 -- 172.21.15.17:0/3345239562 >> v1:172.21.15.17:6789/0 conn(0x7fba181378c0 legacy=0x7fba18137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient: _finish_hunting 0 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 monclient: found mon.noname-b 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:35:41.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 -- 172.21.15.17:0/3345239562 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba180a22c0 con 0x7fba181383b0 2024-09-15T21:35:41.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba167fc640 1 --2- 172.21.15.17:0/3345239562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba181383b0 0x7fba1813cd80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba000097c0 tx=0x7fba0001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b418b49ab43a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 1 -- 172.21.15.17:0/3345239562 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b1070 con 0x7fba181383b0 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: dump: 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:41.682 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 1 -- 172.21.15.17:0/3345239562 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba000a47f0 con 0x7fba181383b0 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 1 -- 172.21.15.17:0/3345239562 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000a4bc0 con 0x7fba181383b0 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 5 monclient: authenticate success, global_id 214157 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:41.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 4 set_mon_vals no callback set 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba15ffb640 10 monclient: dump: 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:41.684 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 10 monclient: shutdown 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3345239562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba181383b0 msgr2=0x7fba1813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.668+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3345239562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba181383b0 0x7fba1813cd80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba000097c0 tx=0x7fba0001c040 comp rx=0 tx=0).stop 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3345239562 shutdown_connections 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3345239562 >> v2:172.21.15.17:3300/0 conn(0x7fba1813d2c0 0x7fba1813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3345239562 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba181383b0 0x7fba1813cd80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3345239562 >> 172.21.15.17:0/3345239562 conn(0x7fba180aac40 msgr2=0x7fba180a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3345239562 shutdown_connections 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3345239562 wait complete. 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: build_initial_monmap 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: monmap: 2024-09-15T21:35:41.685 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 Processor -- start 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- start start 2024-09-15T21:35:41.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: init 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba181378c0 0x7fba180a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): picked mon.noname-b con 0x7fba181378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba181383b0 0x7fba181d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): picked mon.noname-a con 0x7fba181383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 0x7fba18136d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): picked mon.noname-c con 0x7fba1813d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:41.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba18143040 con 0x7fba181383b0 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fba18142ec0 con 0x7fba181378c0 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fba181431c0 con 0x7fba1813d2c0 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient(hunting): authenticate will time out at 2065.069663s 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 0x7fba18136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba181383b0 0x7fba181d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 0x7fba18136d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:38476/0 (socket says 172.21.15.17:38476) 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 -- 172.21.15.17:0/3297306803 learned_addr learned my addr 172.21.15.17:0/3297306803 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): get_auth_request con 0x7fba1813d2c0 auth_method 0 2024-09-15T21:35:41.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba16ffd640 1 -- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba181378c0 msgr2=0x7fba180a03a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): get_auth_request con 0x7fba181383b0 auth_method 0 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba16ffd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba181378c0 0x7fba180a03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:41.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient(hunting): handle_auth_done global_id 214160 payload 824 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 -- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba181378c0 msgr2=0x7fba180a03a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba181378c0 0x7fba180a03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 -- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba181383b0 msgr2=0x7fba181d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba181383b0 0x7fba181d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient: _finish_hunting 0 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 monclient: found mon.noname-c 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 -- 172.21.15.17:0/3297306803 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba180a3c50 con 0x7fba1813d2c0 2024-09-15T21:35:41.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba177fe640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 0x7fba18136d70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba1000efc0 tx=0x7fba1000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10018020 con 0x7fba1813d2c0 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: dump: 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:41.691 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: _finish_auth 0 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba10014480 con 0x7fba1813d2c0 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10014870 con 0x7fba1813d2c0 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 5 monclient: authenticate success, global_id 214160 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:41.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 10 monclient: dump: 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:41.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba14ff9640 4 set_mon_vals no callback set 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _renew_subs 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba180a12f0 con 0x7fba1813d2c0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _renew_subs 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba18121800 con 0x7fba1813d2c0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fba1001b020 con 0x7fba1813d2c0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba18137ca0 con 0x7fba1813d2c0 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9f803b9a0 0x7fb9f803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:41.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.672+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fba10051690 con 0x7fba1813d2c0 2024-09-15T21:35:41.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.676+0000 7fba16ffd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9f803b9a0 0x7fb9f803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:41.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.676+0000 7fba16ffd640 10 monclient: get_auth_request con 0x7fb9f803b9a0 auth_method 0 2024-09-15T21:35:41.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.676+0000 7fba16ffd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9f803b9a0 0x7fb9f803de60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fba08004820 tx=0x7fba08007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:41.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.680+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fba10022020 con 0x7fba1813d2c0 2024-09-15T21:35:41.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.680+0000 7fba0ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:41.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.680+0000 7fba0ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:41.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.916+0000 7fba1d9bd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:41.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.916+0000 7fba1d9bd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:41.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.916+0000 7fba1d9bd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:41.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.916+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fba18069270 con 0x7fba1813d2c0 2024-09-15T21:35:41.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba0ffff640 1 -- 172.21.15.17:0/3297306803 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fba10010040 con 0x7fba1813d2c0 2024-09-15T21:35:41.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba0ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:41.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba0ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:41.922 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9f803b9a0 msgr2=0x7fb9f803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9f803b9a0 0x7fb9f803de60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fba08004820 tx=0x7fba08007b50 comp rx=0 tx=0).stop 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 10 monclient: shutdown 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 msgr2=0x7fba18136d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 0x7fba18136d70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba1000efc0 tx=0x7fba1000c490 comp rx=0 tx=0).stop 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 shutdown_connections 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fb9f803b9a0 0x7fb9f803de60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fba1813d2c0 0x7fba18136d70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba181383b0 0x7fba181d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 --2- 172.21.15.17:0/3297306803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba181378c0 0x7fba180a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:41.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 >> 172.21.15.17:0/3297306803 conn(0x7fba180aac40 msgr2=0x7fba180a8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:41.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 shutdown_connections 2024-09-15T21:35:41.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:41.920+0000 7fba1d9bd640 1 -- 172.21.15.17:0/3297306803 wait complete. 2024-09-15T21:35:44.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient: get_monmap_and_config 2024-09-15T21:35:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient: build_initial_monmap 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient: monmap: 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:45.075753+0000 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:45.075753+0000 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:45.076 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 1 Processor -- start 2024-09-15T21:35:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 1 -- start start 2024-09-15T21:35:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient: init 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff9a40a4a90 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient(hunting): picked mon.noname-b con 0x7ff9a40a5580 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff9a40a59a0 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a40a2c30 con 0x7ff9a40a4a90 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a40a19b0 con 0x7ff9a40a59a0 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a40a1c90 con 0x7ff9a40a5580 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a93e4640 10 monclient(hunting): authenticate will time out at 2068.471148s 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff9a40a4a90 0x7ff9a40a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50648/0 (socket says 172.21.15.17:50648) 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 learned_addr learned my addr 172.21.15.17:0/3838533332 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:45.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3838533332 >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a59a0 legacy=0x7ff9a413f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request con 0x7ff9a40a5580 auth_method 0 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:45.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.072+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3838533332 >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a59a0 legacy=0x7ff9a413f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request con 0x7ff9a40a5580 auth_method 0 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.276+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request con 0x7ff9a40a5580 auth_method 0 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:35:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:45.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:45.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:35:45.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:45.680+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3838533332 >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a59a0 legacy=0x7ff9a413f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3838533332 >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a59a0 legacy=0x7ff9a413f260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request con 0x7ff9a40a5580 auth_method 0 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:35:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:46.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:46.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:46.484+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:35:48.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): tick 2024-09-15T21:35:48.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:35:48.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 monclient(hunting): continuing hunt 2024-09-15T21:35:48.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:35:48.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 msgr2=0x7ff9a413cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9a40a5580 0x7ff9a413cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 -- 172.21.15.17:0/3838533332 >> v1:172.21.15.119:6789/0 conn(0x7ff9a40a59a0 legacy=0x7ff9a413f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 -- 172.21.15.17:0/3838533332 >> v1:172.21.15.17:6789/0 conn(0x7ff9a40a4a90 legacy=0x7ff9a40a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient: _add_conns ranks=[3,4] 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): picked mon.noname-a con 0x7ff9900036e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 0x7ff990008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): picked mon.noname-c con 0x7ff990006250 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 -- 172.21.15.17:0/3838533332 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a40a1c90 con 0x7ff9900036e0 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:48.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 1 -- 172.21.15.17:0/3838533332 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff990008dd0 con 0x7ff990006250 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a0ff9640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 msgr2=0x7ff990008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 0x7ff990008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request con 0x7ff9900036e0 auth_method 0 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.072+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.076+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 msgr2=0x7ff990005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.076+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 msgr2=0x7ff990005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.076+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:48.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.076+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.276+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 msgr2=0x7ff990008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.276+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 0x7ff990008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:35:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request con 0x7ff9900036e0 auth_method 0 2024-09-15T21:35:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 msgr2=0x7ff990005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:35:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 msgr2=0x7ff990005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.280+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:35:48.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.680+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 msgr2=0x7ff990008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:48.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.680+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 0x7ff990008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request con 0x7ff9900036e0 auth_method 0 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 msgr2=0x7ff990005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 msgr2=0x7ff990005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:35:48.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:48.684+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:35:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.484+0000 7ff9a27fc640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 msgr2=0x7ff990008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.484+0000 7ff9a27fc640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 0x7ff990008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:35:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.17:3300/0 conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request con 0x7ff9900036e0 auth_method 0 2024-09-15T21:35:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:35:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_done global_id 204306 payload 274 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 msgr2=0x7ff990008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> v2:172.21.15.119:3300/0 conn(0x7ff990006250 0x7ff990008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 1 monclient: found mon.noname-a 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.488+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3838533332 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a40a1f70 con 0x7ff9900036e0 2024-09-15T21:35:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.492+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9900036e0 0x7ff990005b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff990006220 tx=0x7ff99401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75d611c69b304ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 1 -- 172.21.15.17:0/3838533332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a66e0 con 0x7ff9900036e0 2024-09-15T21:35:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: dump: 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: _finish_auth 0 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 1 -- 172.21.15.17:0/3838533332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9940a6880 con 0x7ff9900036e0 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 5 monclient: authenticate success, global_id 204306 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 1 -- 172.21.15.17:0/3838533332 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a6c70 con 0x7ff9900036e0 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 4 set_mon_vals no callback set 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a1ffb640 10 monclient: dump: 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.501 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 10 monclient: shutdown 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9900036e0 msgr2=0x7ff990005b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9900036e0 0x7ff990005b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff990006220 tx=0x7ff99401c910 comp rx=0 tx=0).stop 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3838533332 reap_dead start 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3838533332 shutdown_connections 2024-09-15T21:35:49.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3838533332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9900036e0 0x7ff990005b60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3838533332 >> 172.21.15.17:0/3838533332 conn(0x7ff9a40aabb0 msgr2=0x7ff9a40a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3838533332 shutdown_connections 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3838533332 wait complete. 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 10 monclient: build_initial_monmap 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.496+0000 7ff9a93e4640 10 monclient: monmap: 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:49.504 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:49.504 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.504 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:49.504 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:49.504 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:49.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 Processor -- start 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 -- start start 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: init 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 0x7ff9a41377c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): picked mon.noname-c con 0x7ff9900036e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4137d00 0x7ff9a413cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): picked mon.noname-b con 0x7ff9a4137d00 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a413d1a0 0x7ff9a413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): picked mon.noname-a con 0x7ff9a413d1a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4143940 con 0x7ff9a413d1a0 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41437c0 con 0x7ff9a4137d00 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a4143ac0 con 0x7ff9900036e0 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient(hunting): authenticate will time out at 2072.898321s 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 0x7ff9a41377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a27fc640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4137d00 msgr2=0x7ff9a413cc60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a413d1a0 0x7ff9a413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4137d00 0x7ff9a413cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 0x7ff9a41377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:55226/0 (socket says 172.21.15.17:55226) 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 1 -- 172.21.15.17:0/3802282814 learned_addr learned my addr 172.21.15.17:0/3802282814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request con 0x7ff9900036e0 auth_method 0 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): get_auth_request con 0x7ff9a413d1a0 auth_method 0 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient(hunting): handle_auth_done global_id 204309 payload 824 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 msgr2=0x7ff9a41377c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 0x7ff9a41377c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4137d00 msgr2=0x7ff9a413cc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4137d00 0x7ff9a413cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient: _finish_hunting 0 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 monclient: found mon.noname-a 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 -- 172.21.15.17:0/3802282814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a413fe60 con 0x7ff9a413d1a0 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 0x7ff9a41377c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a37fe640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a413d1a0 0x7ff9a413d600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40a15d0 tx=0x7ff99c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c00cdd0 con 0x7ff9a413d1a0 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: dump: 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.511 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c010030 con 0x7ff9a413d1a0 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c016670 con 0x7ff9a413d1a0 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff98b7fe640 10 monclient: dump: 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 5 monclient: authenticate success, global_id 204309 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: _renew_subs 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a17fa640 4 set_mon_vals no callback set 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a4120f60 con 0x7ff9a413d1a0 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: _renew_subs 2024-09-15T21:35:49.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.500+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a40a1280 con 0x7ff9a413d1a0 2024-09-15T21:35:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a93e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a93e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a93e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a40a4e70 con 0x7ff9a413d1a0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff99c0153e0 con 0x7ff9a413d1a0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff98b7fe640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff99003c760 0x7ff99003ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff99c050620 con 0x7ff9a413d1a0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff99003c760 0x7ff99003ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a2ffd640 10 monclient: get_auth_request con 0x7ff99003c760 auth_method 0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.504+0000 7ff9a2ffd640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff99003c760 0x7ff99003ec20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff99401a540 tx=0x7ff994010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.508+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff9a413fe60 con 0x7ff9a413d1a0 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.508+0000 7ff98b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.508+0000 7ff98b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff9a93e4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:35:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff9a93e4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff9a93e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:35:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff9a41206e0 con 0x7ff9a413d1a0 2024-09-15T21:35:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff98b7fe640 1 -- 172.21.15.17:0/3802282814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7ff99c01e350 con 0x7ff9a413d1a0 2024-09-15T21:35:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff98b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:35:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.748+0000 7ff98b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":170,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff99003c760 msgr2=0x7ff99003ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff99003c760 0x7ff99003ec20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff99401a540 tx=0x7ff994010040 comp rx=0 tx=0).stop 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 10 monclient: shutdown 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a413d1a0 msgr2=0x7ff9a413d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a413d1a0 0x7ff9a413d600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40a15d0 tx=0x7ff99c00be60 comp rx=0 tx=0).stop 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 shutdown_connections 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff99003c760 0x7ff99003ec20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9a413d1a0 0x7ff9a413d600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9a4137d00 0x7ff9a413cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 --2- 172.21.15.17:0/3802282814 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff9900036e0 0x7ff9a41377c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 >> 172.21.15.17:0/3802282814 conn(0x7ff9a40aabb0 msgr2=0x7ff9a409e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 shutdown_connections 2024-09-15T21:35:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.752+0000 7ff9a93e4640 1 -- 172.21.15.17:0/3802282814 wait complete. 2024-09-15T21:35:49.772 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:35:49.772 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:35:49.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:35:49.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient: get_monmap_and_config 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient: build_initial_monmap 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient: monmap: 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:49.902668+0000 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:49.902668+0000 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:49.904 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:49.904 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 Processor -- start 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 -- start start 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient: init 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82701340c0 0x7f82701344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): picked mon.noname-c con 0x7f82701340c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f827009d6c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82700a0fa0 0x7f827013be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): picked mon.noname-b con 0x7f82700a0fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82701349e0 con 0x7f827009d6c0 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f827009d520 con 0x7f82701340c0 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f827009e1b0 con 0x7f82700a0fa0 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f827831f640 10 monclient(hunting): authenticate will time out at 2073.298588s 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82758ba640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f827009d6c0 0x7f82700a0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36328/0 (socket says 172.21.15.17:36328) 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82700a0fa0 0x7f827013be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82760bb640 1 -- >> v2:172.21.15.119:3300/0 conn(0x7f82701340c0 msgr2=0x7f82701344a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82758ba640 1 -- 172.21.15.17:0/4243886171 learned_addr learned my addr 172.21.15.17:0/4243886171 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82760bb640 1 --2- 172.21.15.17:0/4243886171 >> v2:172.21.15.119:3300/0 conn(0x7f82701340c0 0x7f82701344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): get_auth_request con 0x7f82700a0fa0 auth_method 0 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82750b9640 1 -- 172.21.15.17:0/4243886171 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121348988 0 0) 0x7f82701349e0 con 0x7f827009d6c0 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82750b9640 10 monclient(hunting): _negotiate 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82750b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82750b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82750b9640 10 monclient(hunting): my global_id is 204312 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82750b9640 1 -- 172.21.15.17:0/4243886171 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f825c003530 con 0x7f827009d6c0 2024-09-15T21:35:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient(hunting): handle_auth_done global_id 214166 payload 274 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 1 -- 172.21.15.17:0/4243886171 >> v2:172.21.15.119:3300/0 conn(0x7f82701340c0 msgr2=0x7f82701344a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 1 --2- 172.21.15.17:0/4243886171 >> v2:172.21.15.119:3300/0 conn(0x7f82701340c0 0x7f82701344a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 1 -- 172.21.15.17:0/4243886171 >> v1:172.21.15.17:6789/0 conn(0x7f827009d6c0 legacy=0x7f82700a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient: _finish_hunting 0 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 1 monclient: found mon.noname-b 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:35:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.900+0000 7f82768bc640 1 -- 172.21.15.17:0/4243886171 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827009e490 con 0x7f82700a0fa0 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 1 --2- 172.21.15.17:0/4243886171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82700a0fa0 0x7f827013be70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8268009d90 tx=0x7f826801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648d97dd05a5ac45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 1 -- 172.21.15.17:0/4243886171 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680b2070 con 0x7f82700a0fa0 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: dump: 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.915 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: _finish_auth 0 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: _check_auth_tickets 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 1 -- 172.21.15.17:0/4243886171 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82680a57a0 con 0x7f82700a0fa0 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 5 monclient: authenticate success, global_id 214166 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:49.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 1 -- 172.21.15.17:0/4243886171 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680a5b70 con 0x7f82700a0fa0 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 4 set_mon_vals no callback set 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82750b9640 10 monclient: dump: 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: shutdown 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- 172.21.15.17:0/4243886171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82700a0fa0 msgr2=0x7f827013be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 --2- 172.21.15.17:0/4243886171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82700a0fa0 0x7f827013be70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8268009d90 tx=0x7f826801c850 comp rx=0 tx=0).stop 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- 172.21.15.17:0/4243886171 shutdown_connections 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 --2- 172.21.15.17:0/4243886171 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82700a0fa0 0x7f827013be70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 --2- 172.21.15.17:0/4243886171 >> v2:172.21.15.119:3300/0 conn(0x7f82701340c0 0x7f82701344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- 172.21.15.17:0/4243886171 >> 172.21.15.17:0/4243886171 conn(0x7f82700a66a0 msgr2=0x7f82700a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- 172.21.15.17:0/4243886171 shutdown_connections 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- 172.21.15.17:0/4243886171 wait complete. 2024-09-15T21:35:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: build_initial_monmap 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: monmap: 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 Processor -- start 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- start start 2024-09-15T21:35:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: init 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f827009d6c0 0x7f82701cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): picked mon.noname-c con 0x7f827009d6c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700a0fa0 0x7f82701cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): picked mon.noname-b con 0x7f82700a0fa0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82701340c0 0x7f82701d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): picked mon.noname-a con 0x7f82701340c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f827013e3c0 con 0x7f82701340c0 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82700a27a0 con 0x7f82700a0fa0 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f827013e540 con 0x7f827009d6c0 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f827831f640 10 monclient(hunting): authenticate will time out at 2073.303776s 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82701340c0 0x7f82701d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82760bb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f827009d6c0 0x7f82701cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82701340c0 0x7f82701d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58682/0 (socket says 172.21.15.17:58682) 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 1 -- 172.21.15.17:0/710140445 learned_addr learned my addr 172.21.15.17:0/710140445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 10 monclient(hunting): get_auth_request con 0x7f82701340c0 auth_method 0 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82768bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82758ba640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700a0fa0 msgr2=0x7f82701cff00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82758ba640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700a0fa0 0x7f82701cff00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82760bb640 10 monclient(hunting): get_auth_request con 0x7f827009d6c0 auth_method 0 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82760bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82760bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.904+0000 7f82760bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82768bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82768bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82768bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 10 monclient(hunting): handle_auth_done global_id 214169 payload 824 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700a0fa0 msgr2=0x7f82701cff00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700a0fa0 0x7f82701cff00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82701340c0 msgr2=0x7f82701d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82701340c0 0x7f82701d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 10 monclient: _finish_hunting 0 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 monclient: found mon.noname-c 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827013b740 con 0x7f827009d6c0 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82760bb640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f827009d6c0 0x7f82701cf9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b470 tx=0x7f826c00b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c003dd0 con 0x7f827009d6c0 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: dump: 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: _finish_auth 0 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 5 monclient: authenticate success, global_id 214169 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 10 monclient: _renew_subs 2024-09-15T21:35:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c015070 con 0x7f827009d6c0 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82700a4ca0 con 0x7f827009d6c0 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0052b0 con 0x7f827009d6c0 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: dump: 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82748b8640 4 set_mon_vals no callback set 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 10 monclient: _renew_subs 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f827013bd00 con 0x7f827009d6c0 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 0x7f827012d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82701344a0 con 0x7f827012d1b0 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 0x7f827012d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient: get_auth_request con 0x7f827012d1b0 auth_method 0 2024-09-15T21:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f826c00f070 con 0x7f827009d6c0 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 10 monclient(hunting): handle_auth_done global_id 204318 payload 824 2024-09-15T21:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82700008e0 con 0x7f827012d1b0 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f825403b970 0x7f825403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f826c04fdc0 con 0x7f827009d6c0 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82758ba640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 0x7f827012d590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f827013b950 tx=0x7f826000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260005bc0 con 0x7f827012d1b0 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: dump: 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82768bc640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f825403b970 0x7f825403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82768bc640 10 monclient: get_auth_request con 0x7f825403b970 auth_method 0 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f826000ce50 con 0x7f827012d1b0 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 msgr2=0x7f827012d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 0x7f827012d590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f827013b950 tx=0x7f826000cb10 comp rx=0 tx=0).stop 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 msgr2=0x7f827012d590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f8266ffd640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012d1b0 0x7f827012d590 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.908+0000 7f82768bc640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f825403b970 0x7f825403de30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f826801c890 tx=0x7f82680a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 0x7f827006f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f827012deb0 con 0x7f827012dad0 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 0x7f827006f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient: get_auth_request con 0x7f827012dad0 auth_method 0 2024-09-15T21:35:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 10 monclient(hunting): handle_auth_done global_id 204321 payload 824 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 1 -- 172.21.15.17:0/710140445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f82700734c0 con 0x7f827012dad0 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82758ba640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 0x7f827006f230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8270134620 tx=0x7f826000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82600070d0 con 0x7f827012dad0 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 10 monclient: dump: 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4703 (secure 0 0 0) 0x7f82600070d0 con 0x7f827012dad0 2024-09-15T21:35:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 msgr2=0x7f827006f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 0x7f827006f230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8270134620 tx=0x7f826000f000 comp rx=0 tx=0).stop 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 msgr2=0x7f827006f230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f8266ffd640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827012dad0 0x7f827006f230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:49.934 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 170, 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 74450, 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:49.935 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:49.936 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:49.937 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:49.938 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:49.939 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:49.940 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:49.941 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:49.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:49.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:49.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:35:49.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:49.947 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:49.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f825403b970 msgr2=0x7f825403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f825403b970 0x7f825403de30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f826801c890 tx=0x7f82680a7070 comp rx=0 tx=0).stop 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 10 monclient: shutdown 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f827009d6c0 msgr2=0x7f82701cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 --2- 172.21.15.17:0/710140445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f827009d6c0 0x7f82701cf9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b470 tx=0x7f826c00b930 comp rx=0 tx=0).stop 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82768bc640 1 -- 172.21.15.17:0/710140445 reap_dead start 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f82768bc640 1 -- 172.21.15.17:0/710140445 reap_dead start 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 shutdown_connections 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 >> 172.21.15.17:0/710140445 conn(0x7f82700a66a0 msgr2=0x7f8270132a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 shutdown_connections 2024-09-15T21:35:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:49.916+0000 7f827831f640 1 -- 172.21.15.17:0/710140445 wait complete. 2024-09-15T21:35:49.950 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:35:49.950 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-15T21:35:49.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:35:50.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient: get_monmap_and_config 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient: build_initial_monmap 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient: monmap: 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:35:50.067644+0000 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:35:50.067644+0000 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:35:50.068 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 1 Processor -- start 2024-09-15T21:35:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 1 -- start start 2024-09-15T21:35:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient: init 2024-09-15T21:35:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2024-09-15T21:35:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fee281378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee281383b0 0x7fee2813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient(hunting): picked mon.noname-a con 0x7fee281383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fee2813d2c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee280a1a20 con 0x7fee281378c0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee280a1d00 con 0x7fee2813d2c0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fee280a1fe0 con 0x7fee281383b0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee302e4640 10 monclient(hunting): authenticate will time out at 2073.463081s 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2e881640 1 -- >> v1:172.21.15.119:6789/0 conn(0x7fee2813d2c0 legacy=0x7fee2813f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.119:6789/0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2e080640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fee281378c0 0x7fee28137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36342/0 (socket says 172.21.15.17:36342) 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee281383b0 0x7fee2813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2e080640 1 -- 172.21.15.17:0/4245849410 learned_addr learned my addr 172.21.15.17:0/4245849410 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:35:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): get_auth_request con 0x7fee281383b0 auth_method 0 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d07e640 1 -- 172.21.15.17:0/4245849410 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168208680 0 0) 0x7fee18001120 con 0x7fee281378c0 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d07e640 10 monclient(hunting): _negotiate 2024-09-15T21:35:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d07e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d07e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d07e640 10 monclient(hunting): my global_id is 204327 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d07e640 1 -- 172.21.15.17:0/4245849410 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee14003530 con 0x7fee281378c0 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient(hunting): handle_auth_done global_id 204324 payload 274 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 1 -- 172.21.15.17:0/4245849410 >> v1:172.21.15.119:6789/0 conn(0x7fee2813d2c0 legacy=0x7fee2813f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 1 -- 172.21.15.17:0/4245849410 >> v1:172.21.15.17:6789/0 conn(0x7fee281378c0 legacy=0x7fee28137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient: _finish_hunting 0 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 1 monclient: found mon.noname-a 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:35:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.064+0000 7fee2d87f640 1 -- 172.21.15.17:0/4245849410 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee280a22c0 con 0x7fee281383b0 2024-09-15T21:35:50.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d87f640 1 --2- 172.21.15.17:0/4245849410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee281383b0 0x7fee2813cd80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee10009dc0 tx=0x7fee1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5913be4701484 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:50.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 1 -- 172.21.15.17:0/4245849410 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100b1070 con 0x7fee281383b0 2024-09-15T21:35:50.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:50.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:35:50.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: dump: 2024-09-15T21:35:50.079 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: _finish_auth 0 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: _check_auth_tickets 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:50.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 5 monclient: authenticate success, global_id 204324 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 1 -- 172.21.15.17:0/4245849410 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee100a4db0 con 0x7fee281383b0 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 1 -- 172.21.15.17:0/4245849410 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100ac640 con 0x7fee281383b0 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee2d07e640 10 monclient: dump: 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:50.081 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 10 monclient: get_monmap_and_config success 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 4 set_mon_vals no callback set 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 10 monclient: shutdown 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 -- 172.21.15.17:0/4245849410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee281383b0 msgr2=0x7fee2813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 --2- 172.21.15.17:0/4245849410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee281383b0 0x7fee2813cd80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee10009dc0 tx=0x7fee1001c4f0 comp rx=0 tx=0).stop 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 -- 172.21.15.17:0/4245849410 shutdown_connections 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 --2- 172.21.15.17:0/4245849410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee281383b0 0x7fee2813cd80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 -- 172.21.15.17:0/4245849410 >> 172.21.15.17:0/4245849410 conn(0x7fee280aac40 msgr2=0x7fee280a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:50.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 -- 172.21.15.17:0/4245849410 shutdown_connections 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 -- 172.21.15.17:0/4245849410 wait complete. 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 10 monclient: build_initial_monmap 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 10 monclient: monmap: 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:35:50.083 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 Processor -- start 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 1 -- start start 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.068+0000 7fee302e4640 10 monclient: init 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient: _reopen_session rank -1 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee281378c0 0x7fee281d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): picked mon.noname-b con 0x7fee281378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee281383b0 0x7fee28136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): picked mon.noname-c con 0x7fee281383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee2813d2c0 0x7fee28137210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): picked mon.noname-a con 0x7fee2813d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:50.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fee28144230 con 0x7fee2813d2c0 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee281440b0 con 0x7fee281378c0 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fee281443b0 con 0x7fee281383b0 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): _renew_subs 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): authenticate will time out at 2073.468310s 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 1 -- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee281378c0 msgr2=0x7fee281d0970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:3300/0 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee2813d2c0 0x7fee28137210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee281378c0 0x7fee281d0970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee281383b0 0x7fee28136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee2813d2c0 0x7fee28137210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58720/0 (socket says 172.21.15.17:58720) 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 1 -- 172.21.15.17:0/264104392 learned_addr learned my addr 172.21.15.17:0/264104392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:35:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): get_auth_request con 0x7fee281383b0 auth_method 0 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): get_auth_request con 0x7fee2813d2c0 auth_method 0 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:50.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient(hunting): handle_auth_done global_id 214172 payload 824 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee281378c0 msgr2=0x7fee281d0970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee281378c0 0x7fee281d0970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee2813d2c0 msgr2=0x7fee28137210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee2813d2c0 0x7fee28137210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient: _finish_hunting 0 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 monclient: found mon.noname-c 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee280a0450 con 0x7fee281383b0 2024-09-15T21:35:50.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee2813d2c0 0x7fee28137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2d87f640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee281383b0 0x7fee28136cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee100ab000 tx=0x7fee100195b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100bb020 con 0x7fee281383b0 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: dump: 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:50.088 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: _finish_auth 0 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 5 monclient: authenticate success, global_id 214172 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee10004040 con 0x7fee281383b0 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100b1040 con 0x7fee281383b0 2024-09-15T21:35:50.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient: _renew_subs 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee280a0730 con 0x7fee281383b0 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2c87d640 4 set_mon_vals no callback set 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: dump: 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:50.090 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient: _renew_subs 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee280a8220 con 0x7fee281383b0 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 0x7fee2809d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fee28137ca0 con 0x7fee2809f210 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fee100be020 con 0x7fee281383b0 2024-09-15T21:35:50.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 0x7fee2809d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient: get_auth_request con 0x7fee2809f210 auth_method 0 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fee0803b9a0 0x7fee0803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fee100ea6b0 con 0x7fee281383b0 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 10 monclient(hunting): handle_auth_done global_id 214175 payload 824 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee280a12f0 con 0x7fee2809f210 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fee0803b9a0 0x7fee0803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:50.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e080640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 0x7fee2809d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee18004820 tx=0x7fee1800cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 10 monclient: get_auth_request con 0x7fee0803b9a0 auth_method 0 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18017070 con 0x7fee2809f210 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: dump: 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fee1800ce50 con 0x7fee2809f210 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 msgr2=0x7fee2809d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 0x7fee2809d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee18004820 tx=0x7fee1800cb10 comp rx=0 tx=0).stop 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 msgr2=0x7fee2809d4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee26ffd640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2809f210 0x7fee2809d4e0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee18004820 tx=0x7fee1800cb10 comp rx=0 tx=0).stop 2024-09-15T21:35:50.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.072+0000 7fee2e881640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fee0803b9a0 0x7fee0803de60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fee2809ee40 tx=0x7fee20009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 0x7fee2802d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fee2813a470 con 0x7fee2802d450 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 0x7fee2802d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient: get_auth_request con 0x7fee2802d450 auth_method 0 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:35:50.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 10 monclient(hunting): handle_auth_done global_id 214178 payload 824 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 1 -- 172.21.15.17:0/264104392 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fee28069270 con 0x7fee2802d450 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e080640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 0x7fee2802d860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee180012c0 tx=0x7fee1800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18007500 con 0x7fee2802d450 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 10 monclient: dump: 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:35:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7fee18007500 con 0x7fee2802d450 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 msgr2=0x7fee2802d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 0x7fee2802d860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee180012c0 tx=0x7fee1800f000 comp rx=0 tx=0).stop 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 msgr2=0x7fee2802d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee26ffd640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee2802d450 0x7fee2802d860 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee180012c0 tx=0x7fee1800f000 comp rx=0 tx=0).stop 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 170, 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 29327, 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:35:50.099 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:50.100 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:50.101 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:50.102 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:35:50.103 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.104 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:50.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:50.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:35:50.107 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:50.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:35:50.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:50.110 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:35:50.111 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:35:50.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fee0803b9a0 msgr2=0x7fee0803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fee0803b9a0 0x7fee0803de60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fee2809ee40 tx=0x7fee20009110 comp rx=0 tx=0).stop 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 10 monclient: shutdown 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee281383b0 msgr2=0x7fee28136cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 --2- 172.21.15.17:0/264104392 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fee281383b0 0x7fee28136cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee100ab000 tx=0x7fee100195b0 comp rx=0 tx=0).stop 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e881640 1 -- 172.21.15.17:0/264104392 reap_dead start 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee2e881640 1 -- 172.21.15.17:0/264104392 reap_dead start 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 shutdown_connections 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 >> 172.21.15.17:0/264104392 conn(0x7fee280aac40 msgr2=0x7fee2813faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 shutdown_connections 2024-09-15T21:35:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:35:50.080+0000 7fee302e4640 1 -- 172.21.15.17:0/264104392 wait complete. 2024-09-15T21:35:50.114 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:35:50.114 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:36:10.115 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T21:36:10.115 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:36:10.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:36:10.119 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:36:10.119 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:36:10.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient: get_monmap_and_config 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient: build_initial_monmap 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient: monmap: 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:10.251838+0000 2024-09-15T21:36:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:10.251838+0000 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:10.253 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:10.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 1 Processor -- start 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 1 -- start start 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient: init 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f88bc0a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc09ffc0 0x7f88bc13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient(hunting): picked mon.noname-b con 0x7f88bc09ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f88bc0a1380 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88bc0a3130 con 0x7f88bc0a0890 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88bc073df0 con 0x7f88bc0a1380 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc0a1bf0 con 0x7f88bc09ffc0 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c32f0640 10 monclient(hunting): authenticate will time out at 2093.647228s 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c108c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f88bc0a0890 0x7f88bc0a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43034/0 (socket says 172.21.15.17:43034) 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c108c640 1 -- 172.21.15.17:0/1280818397 learned_addr learned my addr 172.21.15.17:0/1280818397 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 1 --2- 172.21.15.17:0/1280818397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc09ffc0 0x7f88bc13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): get_auth_request con 0x7f88bc09ffc0 auth_method 0 2024-09-15T21:36:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88bbfff640 1 -- 172.21.15.17:0/1280818397 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390021876 0 0) 0x7f88bc0a3130 con 0x7f88bc0a0890 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88bbfff640 10 monclient(hunting): _negotiate 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88bbfff640 10 monclient(hunting): my global_id is 204333 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88bbfff640 1 -- 172.21.15.17:0/1280818397 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a8003530 con 0x7f88bc0a0890 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient(hunting): handle_auth_done global_id 214181 payload 274 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 1 -- 172.21.15.17:0/1280818397 >> v1:172.21.15.119:6789/0 conn(0x7f88bc0a1380 legacy=0x7f88bc13d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 1 -- 172.21.15.17:0/1280818397 >> v1:172.21.15.17:6789/0 conn(0x7f88bc0a0890 legacy=0x7f88bc0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient: _finish_hunting 0 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 1 monclient: found mon.noname-b 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.247+0000 7f88c088b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:36:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c088b640 1 -- 172.21.15.17:0/1280818397 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc09ccd0 con 0x7f88bc09ffc0 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c088b640 1 --2- 172.21.15.17:0/1280818397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc09ffc0 0x7f88bc13aac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009dc0 tx=0x7f88a401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31256a8c5a58b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 1 -- 172.21.15.17:0/1280818397 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40b2070 con 0x7f88bc09ffc0 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: dump: 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: _finish_auth 0 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 5 monclient: authenticate success, global_id 214181 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 1 -- 172.21.15.17:0/1280818397 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a40a5820 con 0x7f88bc09ffc0 2024-09-15T21:36:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 1 -- 172.21.15.17:0/1280818397 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40a5bd0 con 0x7f88bc09ffc0 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88bbfff640 10 monclient: dump: 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:10.268 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 4 set_mon_vals no callback set 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 10 monclient: shutdown 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 -- 172.21.15.17:0/1280818397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc09ffc0 msgr2=0x7f88bc13aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 --2- 172.21.15.17:0/1280818397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc09ffc0 0x7f88bc13aac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009dc0 tx=0x7f88a401c850 comp rx=0 tx=0).stop 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 -- 172.21.15.17:0/1280818397 shutdown_connections 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 --2- 172.21.15.17:0/1280818397 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc09ffc0 0x7f88bc13aac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 -- 172.21.15.17:0/1280818397 >> 172.21.15.17:0/1280818397 conn(0x7f88bc0a66a0 msgr2=0x7f88bc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 -- 172.21.15.17:0/1280818397 shutdown_connections 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 -- 172.21.15.17:0/1280818397 wait complete. 2024-09-15T21:36:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 10 monclient: build_initial_monmap 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 10 monclient: monmap: 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.251+0000 7f88c32f0640 1 Processor -- start 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- start start 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: init 2024-09-15T21:36:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 0x7f88bc11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): picked mon.noname-b con 0x7f88bc09ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc0a0890 0x7f88bc11ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): picked mon.noname-c con 0x7f88bc0a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0a1380 0x7f88bc11cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): picked mon.noname-a con 0x7f88bc0a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1412e0 con 0x7f88bc0a1380 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc141160 con 0x7f88bc09ffc0 2024-09-15T21:36:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc141460 con 0x7f88bc0a0890 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient(hunting): authenticate will time out at 2093.652532s 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc0a0890 0x7f88bc11ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0a1380 0x7f88bc11cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc0a0890 0x7f88bc11ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57140/0 (socket says 172.21.15.17:57140) 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 1 -- 172.21.15.17:0/3828760747 learned_addr learned my addr 172.21.15.17:0/3828760747 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): get_auth_request con 0x7f88bc0a1380 auth_method 0 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): get_auth_request con 0x7f88bc0a0890 auth_method 0 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 0x7f88bc11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c088b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 10 monclient(hunting): get_auth_request con 0x7f88bc09ffc0 auth_method 0 2024-09-15T21:36:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient(hunting): handle_auth_done global_id 204336 payload 824 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 -- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 msgr2=0x7f88bc11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 -- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 msgr2=0x7f88bc11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 0x7f88bc11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 -- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc0a0890 msgr2=0x7f88bc11ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc0a0890 0x7f88bc11ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 -- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 msgr2=0x7f88bc11c4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 0x7f88bc11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 0x7f88bc11c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient: _finish_hunting 0 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 monclient: found mon.noname-a 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 -- 172.21.15.17:0/3828760747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc11e930 con 0x7f88bc0a1380 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c188d640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0a1380 0x7f88bc11cf60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c3c0 tx=0x7f88b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4017070 con 0x7f88bc0a1380 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: dump: 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b40052d0 con 0x7f88bc0a1380 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4003840 con 0x7f88bc0a1380 2024-09-15T21:36:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 5 monclient: authenticate success, global_id 204336 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 10 monclient: dump: 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88bb7fe640 4 set_mon_vals no callback set 2024-09-15T21:36:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _renew_subs 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc06f7d0 con 0x7f88bc0a1380 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _renew_subs 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc11eef0 con 0x7f88bc0a1380 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc11da20 con 0x7f88bc0a1380 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f88b401a070 con 0x7f88bc0a1380 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f889c03b9c0 0x7f889c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f88b404f840 con 0x7f88bc0a1380 2024-09-15T21:36:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f889c03b9c0 0x7f889c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 10 monclient: get_auth_request con 0x7f889c03b9c0 auth_method 0 2024-09-15T21:36:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.255+0000 7f88c108c640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f889c03b9c0 0x7f889c03de80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f88ac007a70 tx=0x7f88ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.259+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f88bc11da20 con 0x7f88bc0a1380 2024-09-15T21:36:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.259+0000 7f88b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.259+0000 7f88b9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:10.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88c32f0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:36:10.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88c32f0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:10.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88c32f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:10.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f88bc0a0c70 con 0x7f88bc0a1380 2024-09-15T21:36:10.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88b9ffb640 1 -- 172.21.15.17:0/3828760747 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f88b4005480 con 0x7f88bc0a1380 2024-09-15T21:36:10.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:10.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.499+0000 7f88b9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:10.505 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":170,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:36:10.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f889c03b9c0 msgr2=0x7f889c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f889c03b9c0 0x7f889c03de80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f88ac007a70 tx=0x7f88ac009040 comp rx=0 tx=0).stop 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 10 monclient: shutdown 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0a1380 msgr2=0x7f88bc11cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0a1380 0x7f88bc11cf60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c3c0 tx=0x7f88b400c880 comp rx=0 tx=0).stop 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 shutdown_connections 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f889c03b9c0 0x7f889c03de80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0a1380 0x7f88bc11cf60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88bc0a0890 0x7f88bc11ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 --2- 172.21.15.17:0/3828760747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88bc09ffc0 0x7f88bc11c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 >> 172.21.15.17:0/3828760747 conn(0x7f88bc0a66a0 msgr2=0x7f88bc0a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 shutdown_connections 2024-09-15T21:36:10.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:10.503+0000 7f88c32f0640 1 -- 172.21.15.17:0/3828760747 wait complete. 2024-09-15T21:36:13.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:36:13.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient: get_monmap_and_config 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient: build_initial_monmap 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient: monmap: 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:13.667561+0000 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:13.667561+0000 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:13.668 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:13.669 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:13.669 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:13.669 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 1 Processor -- start 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 1 -- start start 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient: init 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac073640 0x7fe2ac073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient(hunting): picked mon.noname-b con 0x7fe2ac073640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe2ac073f60 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe2ac137da0 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac0a0890 con 0x7fe2ac137da0 2024-09-15T21:36:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac0746e0 con 0x7fe2ac073f60 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0a1640 con 0x7fe2ac073640 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2b20f3640 10 monclient(hunting): authenticate will time out at 2097.062935s 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2abfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe2ac137da0 0x7fe2ac13fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43038/0 (socket says 172.21.15.17:43038) 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2abfff640 1 -- 172.21.15.17:0/1275435559 learned_addr learned my addr 172.21.15.17:0/1275435559 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 1 --2- 172.21.15.17:0/1275435559 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac073640 0x7fe2ac073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): get_auth_request con 0x7fe2ac073640 auth_method 0 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_done global_id 214187 payload 274 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 1 -- 172.21.15.17:0/1275435559 >> v1:172.21.15.119:6789/0 conn(0x7fe2ac073f60 legacy=0x7fe2ac137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2aa7fc640 1 -- 172.21.15.17:0/1275435559 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065137309 0 0) 0x7fe2ac0a0890 con 0x7fe2ac137da0 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 1 -- 172.21.15.17:0/1275435559 >> v1:172.21.15.17:6789/0 conn(0x7fe2ac137da0 legacy=0x7fe2ac13fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 1 monclient: found mon.noname-b 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2ab7fe640 1 -- 172.21.15.17:0/1275435559 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac0a1920 con 0x7fe2ac073640 2024-09-15T21:36:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.663+0000 7fe2aa7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:36:13.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2ab7fe640 1 --2- 172.21.15.17:0/1275435559 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac073640 0x7fe2ac073a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00097f0 tx=0x7fe2a001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc24f923f1b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:13.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 1 -- 172.21.15.17:0/1275435559 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b0070 con 0x7fe2ac073640 2024-09-15T21:36:13.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:13.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: dump: 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:13.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: _finish_auth 0 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 1 -- 172.21.15.17:0/1275435559 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a00ab600 con 0x7fe2ac073640 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 5 monclient: authenticate success, global_id 214187 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 4 set_mon_vals no callback set 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 1 -- 172.21.15.17:0/1275435559 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00ab9d0 con 0x7fe2ac073640 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2aa7fc640 10 monclient: dump: 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:13.680 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 10 monclient: shutdown 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 -- 172.21.15.17:0/1275435559 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac073640 msgr2=0x7fe2ac073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/1275435559 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac073640 0x7fe2ac073a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00097f0 tx=0x7fe2a001c430 comp rx=0 tx=0).stop 2024-09-15T21:36:13.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 -- 172.21.15.17:0/1275435559 shutdown_connections 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/1275435559 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac073640 0x7fe2ac073a20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 -- 172.21.15.17:0/1275435559 >> 172.21.15.17:0/1275435559 conn(0x7fe2ac0a66a0 msgr2=0x7fe2ac0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 -- 172.21.15.17:0/1275435559 shutdown_connections 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 -- 172.21.15.17:0/1275435559 wait complete. 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 10 monclient: build_initial_monmap 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 10 monclient: monmap: 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:13.682 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 Processor -- start 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 1 -- start start 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.667+0000 7fe2b20f3640 10 monclient: init 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2ac073640 0x7fe2ac11e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): picked mon.noname-a con 0x7fe2ac073640 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac073f60 0x7fe2ac1d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): picked mon.noname-b con 0x7fe2ac073f60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac137da0 0x7fe2ac11c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:13.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): picked mon.noname-c con 0x7fe2ac137da0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0a0e80 con 0x7fe2ac073640 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0a0d00 con 0x7fe2ac073f60 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac143b60 con 0x7fe2ac137da0 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient(hunting): authenticate will time out at 2097.068091s 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2ac073640 0x7fe2ac11e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac137da0 0x7fe2ac11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2ac073640 0x7fe2ac11e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37378/0 (socket says 172.21.15.17:37378) 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 1 -- 172.21.15.17:0/2642223819 learned_addr learned my addr 172.21.15.17:0/2642223819 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): get_auth_request con 0x7fe2ac073640 auth_method 0 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): get_auth_request con 0x7fe2ac137da0 auth_method 0 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:13.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2aaffd640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac073f60 0x7fe2ac1d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2aaffd640 10 monclient(hunting): get_auth_request con 0x7fe2ac073f60 auth_method 0 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient(hunting): handle_auth_done global_id 214190 payload 824 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 -- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac073f60 msgr2=0x7fe2ac1d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac073f60 0x7fe2ac1d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 -- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2ac073640 msgr2=0x7fe2ac11e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2ac073640 0x7fe2ac11e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient: _finish_hunting 0 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 monclient: found mon.noname-c 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:36:13.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 -- 172.21.15.17:0/2642223819 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac11fb10 con 0x7fe2ac137da0 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2abfff640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac137da0 0x7fe2ac11c4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400c690 tx=0x7fe2a400cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4018070 con 0x7fe2ac137da0 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: dump: 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:13.687 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: _finish_auth 0 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4004d90 con 0x7fe2ac137da0 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 5 monclient: authenticate success, global_id 214190 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _renew_subs 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac11fdf0 con 0x7fe2ac137da0 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400a880 con 0x7fe2ac137da0 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:13.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 10 monclient: dump: 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2a9ffb640 4 set_mon_vals no callback set 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _renew_subs 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:13.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1200d0 con 0x7fe2ac137da0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac073a20 con 0x7fe2ac137da0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe2a401b070 con 0x7fe2ac137da0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29803b9c0 0x7fe29803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe2a404fac0 con 0x7fe2ac137da0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29803b9c0 0x7fe29803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 10 monclient: get_auth_request con 0x7fe29803b9c0 auth_method 0 2024-09-15T21:36:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.671+0000 7fe2ab7fe640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29803b9c0 0x7fe29803de80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe2a001ccd0 tx=0x7fe2a00a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.675+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe2a4011360 con 0x7fe2ac137da0 2024-09-15T21:36:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.675+0000 7fe293fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.675+0000 7fe293fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:13.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:36:13.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:13.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:13.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe2ac06d020 con 0x7fe2ac137da0 2024-09-15T21:36:13.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe293fff640 1 -- 172.21.15.17:0/2642223819 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fe2a40070d0 con 0x7fe2ac137da0 2024-09-15T21:36:13.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe293fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:13.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe293fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:13.913 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":170,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:36:13.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29803b9c0 msgr2=0x7fe29803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:13.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29803b9c0 0x7fe29803de80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe2a001ccd0 tx=0x7fe2a00a4f90 comp rx=0 tx=0).stop 2024-09-15T21:36:13.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 10 monclient: shutdown 2024-09-15T21:36:13.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:13.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac137da0 msgr2=0x7fe2ac11c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:13.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.907+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac137da0 0x7fe2ac11c4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400c690 tx=0x7fe2a400cb50 comp rx=0 tx=0).stop 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 shutdown_connections 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe29803b9c0 0x7fe29803de80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe2ac137da0 0x7fe2ac11c4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac073f60 0x7fe2ac1d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 --2- 172.21.15.17:0/2642223819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2ac073640 0x7fe2ac11e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 >> 172.21.15.17:0/2642223819 conn(0x7fe2ac0a66a0 msgr2=0x7fe2ac09ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 shutdown_connections 2024-09-15T21:36:13.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:13.911+0000 7fe2b20f3640 1 -- 172.21.15.17:0/2642223819 wait complete. 2024-09-15T21:36:16.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:36:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient: get_monmap_and_config 2024-09-15T21:36:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient: monmap: 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:17.060221+0000 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:17.060221+0000 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:17.062 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 1 Processor -- start 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 1 -- start start 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient: init 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f00e80a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00e809ffc0 0x7f00e813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient(hunting): picked mon.noname-c con 0x7f00e809ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f00e80a1380 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80a3130 con 0x7f00e80a1380 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e8073df0 con 0x7f00e80a0890 2024-09-15T21:36:17.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f00e80a1bf0 con 0x7f00e809ffc0 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00f0223640 10 monclient(hunting): authenticate will time out at 2100.455725s 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00ee7c0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f00e80a1380 0x7f00e813d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43044/0 (socket says 172.21.15.17:43044) 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.055+0000 7f00ee7c0640 1 -- 172.21.15.17:0/3630451929 learned_addr learned my addr 172.21.15.17:0/3630451929 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 --2- 172.21.15.17:0/3630451929 >> v2:172.21.15.119:3300/0 conn(0x7f00e809ffc0 0x7f00e813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325569059 0 0) 0x7f00e80a3130 con 0x7f00e80a1380 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): _negotiate 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): my global_id is 204345 2024-09-15T21:36:17.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00d4003530 con 0x7f00e80a1380 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): get_auth_request con 0x7f00e809ffc0 auth_method 0 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4022618141 0 0) 0x7f00d4003530 con 0x7f00e80a1380 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 >> v2:172.21.15.119:3300/0 conn(0x7f00e809ffc0 msgr2=0x7f00e813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 --2- 172.21.15.17:0/3630451929 >> v2:172.21.15.119:3300/0 conn(0x7f00e809ffc0 0x7f00e813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 >> v1:172.21.15.119:6789/0 conn(0x7f00e80a0890 legacy=0x7f00e80a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e809ccd0 con 0x7f00e80a1380 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00f0223640 5 monclient: authenticate success, global_id 204345 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00f0223640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f00e0001a80 con 0x7f00e80a1380 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:36:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: dump: 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 >> v1:172.21.15.17:6789/0 conn(0x7f00e80a1380 legacy=0x7f00e813d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4003ae0 0x7f00d4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): picked mon.a con 0x7f00d4003ae0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00d4004810 0x7f00d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): picked mon.c con 0x7f00d4004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4005380 0x7f00d400b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): picked mon.b con 0x7f00d4005380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80a1bf0 con 0x7f00d4003ae0 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00d400c2d0 con 0x7f00d4005380 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f00d400c4d0 con 0x7f00d4004810 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00f0223640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4003ae0 0x7f00d4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00d4004810 0x7f00d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): get_auth_request con 0x7f00d4003ae0 auth_method 0 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): get_auth_request con 0x7f00d4004810 auth_method 0 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient(hunting): handle_auth_done global_id 204345 payload 293 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 -- 172.21.15.17:0/3630451929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4005380 msgr2=0x7f00d400b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4005380 0x7f00d400b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 -- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4003ae0 msgr2=0x7f00d4003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4003ae0 0x7f00d4003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 monclient: found mon.c 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 -- 172.21.15.17:0/3630451929 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d4003530 con 0x7f00d4004810 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ee7c0640 1 -- 172.21.15.17:0/3630451929 reap_dead start 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00edfbf640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4003ae0 0x7f00d4003f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ed7be640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00d4004810 0x7f00d4004c90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009ae0 tx=0x7f00dc01ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e302c2be0b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0096b0 con 0x7f00d4004810 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: dump: 2024-09-15T21:36:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0a6430 con 0x7f00d4004810 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 1 -- 172.21.15.17:0/3630451929 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0a6820 con 0x7f00d4004810 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00f0223640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00f0223640 4 set_mon_vals no callback set 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.059+0000 7f00ecfbd640 10 monclient: dump: 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: shutdown 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00d4004810 msgr2=0x7f00d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00d4004810 0x7f00d4004c90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009ae0 tx=0x7f00dc01ca50 comp rx=0 tx=0).stop 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/3630451929 shutdown_connections 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 --2- 172.21.15.17:0/3630451929 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00d4004810 0x7f00d4004c90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/3630451929 >> 172.21.15.17:0/3630451929 conn(0x7f00e80a66a0 msgr2=0x7f00e80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/3630451929 shutdown_connections 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/3630451929 wait complete. 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: monmap: 2024-09-15T21:36:17.078 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:17.079 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 Processor -- start 2024-09-15T21:36:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- start start 2024-09-15T21:36:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: init 2024-09-15T21:36:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 0x7f00e81cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): picked mon.noname-a con 0x7f00d4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e81ce270 0x7f00e8123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): picked mon.noname-b con 0x7f00e81ce270 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00e81ce6f0 0x7f00e8123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): picked mon.noname-c con 0x7f00e81ce6f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8136210 con 0x7f00d4004810 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8136090 con 0x7f00e81ce270 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f00e8136390 con 0x7f00e81ce6f0 2024-09-15T21:36:17.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient(hunting): authenticate will time out at 2100.461710s 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 0x7f00e81cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00e81ce6f0 0x7f00e8123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 0x7f00e81cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37398/0 (socket says 172.21.15.17:37398) 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 -- 172.21.15.17:0/2655122241 learned_addr learned my addr 172.21.15.17:0/2655122241 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): get_auth_request con 0x7f00d4004810 auth_method 0 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): get_auth_request con 0x7f00e81ce6f0 auth_method 0 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ed7be640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e81ce270 0x7f00e8123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ed7be640 10 monclient(hunting): get_auth_request con 0x7f00e81ce270 auth_method 0 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ed7be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ed7be640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ed7be640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient(hunting): handle_auth_done global_id 204348 payload 824 2024-09-15T21:36:17.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 -- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00e81ce6f0 msgr2=0x7f00e8123570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00e81ce6f0 0x7f00e8123570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 -- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e81ce270 msgr2=0x7f00e8123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e81ce270 0x7f00e8123030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 monclient: found mon.noname-a 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 -- 172.21.15.17:0/2655122241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e81265b0 con 0x7f00d4004810 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00ee7c0640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00e81ce6f0 0x7f00e8123570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00edfbf640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 0x7f00e81cdd30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00e400f7a0 tx=0x7f00e400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40013e0 con 0x7f00d4004810 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: dump: 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e4018070 con 0x7f00d4004810 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e400d730 con 0x7f00d4004810 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 5 monclient: authenticate success, global_id 204348 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _renew_subs 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e8135400 con 0x7f00d4004810 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00dbfff640 4 set_mon_vals no callback set 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 10 monclient: dump: 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _renew_subs 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e8124af0 con 0x7f00d4004810 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f00e400d8d0 con 0x7f00d4004810 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8138ab0 con 0x7f00d4004810 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f00cc03b9c0 0x7f00cc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.063+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f00e4050330 con 0x7f00d4004810 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.067+0000 7f00ed7be640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f00cc03b9c0 0x7f00cc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.067+0000 7f00ed7be640 10 monclient: get_auth_request con 0x7f00cc03b9c0 auth_method 0 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.067+0000 7f00ed7be640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f00cc03b9c0 0x7f00cc03de80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0ad000 tx=0x7f00dc0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.071+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f00e81265b0 con 0x7f00d4004810 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.071+0000 7f00da7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.071+0000 7f00da7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00f0223640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:36:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00f0223640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00f0223640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f00e806d020 con 0x7f00d4004810 2024-09-15T21:36:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00da7fc640 1 -- 172.21.15.17:0/2655122241 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f00e404f080 con 0x7f00d4004810 2024-09-15T21:36:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00da7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:17.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.303+0000 7f00da7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:17.309 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:36:17.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f00cc03b9c0 msgr2=0x7f00cc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f00cc03b9c0 0x7f00cc03de80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0ad000 tx=0x7f00dc0b2040 comp rx=0 tx=0).stop 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 10 monclient: shutdown 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 msgr2=0x7f00e81cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 0x7f00e81cdd30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00e400f7a0 tx=0x7f00e400fc60 comp rx=0 tx=0).stop 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 shutdown_connections 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f00cc03b9c0 0x7f00cc03de80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f00e81ce6f0 0x7f00e8123570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e81ce270 0x7f00e8123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 --2- 172.21.15.17:0/2655122241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d4004810 0x7f00e81cdd30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 >> 172.21.15.17:0/2655122241 conn(0x7f00e80a66a0 msgr2=0x7f00e8072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 shutdown_connections 2024-09-15T21:36:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.307+0000 7f00f0223640 1 -- 172.21.15.17:0/2655122241 wait complete. 2024-09-15T21:36:17.327 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:36:17.327 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:36:17.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:36:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.431+0000 7f6a4982c640 10 monclient: get_monmap_and_config 2024-09-15T21:36:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.431+0000 7f6a4982c640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.431+0000 7f6a4982c640 10 monclient: monmap: 2024-09-15T21:36:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:17.438138+0000 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:17.438138+0000 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:17.439 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 1 Processor -- start 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 1 -- start start 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient: init 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6a441340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a4409d6c0 0x7f6a44138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient(hunting): picked mon.noname-c con 0x7f6a4409d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6a44139390 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4409e1b0 con 0x7f6a44139390 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4409e490 con 0x7f6a441340c0 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6a4409d520 con 0x7f6a4409d6c0 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 10 monclient(hunting): authenticate will time out at 2100.833649s 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6a44139390 0x7f6a4413b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43054/0 (socket says 172.21.15.17:43054) 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 -- 172.21.15.17:0/2671165010 learned_addr learned my addr 172.21.15.17:0/2671165010 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 1 --2- 172.21.15.17:0/2671165010 >> v2:172.21.15.119:3300/0 conn(0x7f6a4409d6c0 0x7f6a44138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): get_auth_request con 0x7f6a4409d6c0 auth_method 0 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094114879 0 0) 0x7f6a4409e1b0 con 0x7f6a44139390 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): my global_id is 204351 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a2c003530 con 0x7f6a44139390 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 945548635 0 0) 0x7f6a2c003530 con 0x7f6a44139390 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 >> v2:172.21.15.119:3300/0 conn(0x7f6a4409d6c0 msgr2=0x7f6a44138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 --2- 172.21.15.17:0/2671165010 >> v2:172.21.15.119:3300/0 conn(0x7f6a4409d6c0 0x7f6a44138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 >> v1:172.21.15.119:6789/0 conn(0x7f6a441340c0 legacy=0x7f6a441344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:36:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4409e770 con 0x7f6a44139390 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f6a3c001a80 con 0x7f6a44139390 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: dump: 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 >> v1:172.21.15.17:6789/0 conn(0x7f6a44139390 legacy=0x7f6a4413b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a2c003ab0 0x7f6a2c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): picked mon.c con 0x7f6a2c003ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c004810 0x7f6a2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): picked mon.b con 0x7f6a2c004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a2c0053d0 0x7f6a2c00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): picked mon.a con 0x7f6a2c0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4409d520 con 0x7f6a2c0053d0 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c00d3a0 con 0x7f6a2c004810 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c00d5a0 con 0x7f6a2c003ab0 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a41ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a2c0053d0 0x7f6a2c00cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a2c003ab0 0x7f6a2c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): get_auth_request con 0x7f6a2c0053d0 auth_method 0 2024-09-15T21:36:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): get_auth_request con 0x7f6a2c003ab0 auth_method 0 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient(hunting): handle_auth_done global_id 204351 payload 293 2024-09-15T21:36:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 -- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a2c003ab0 msgr2=0x7f6a2c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a2c003ab0 0x7f6a2c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 -- 172.21.15.17:0/2671165010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c004810 msgr2=0x7f6a2c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c004810 0x7f6a2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 monclient: found mon.a 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 1 -- 172.21.15.17:0/2671165010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c003530 con 0x7f6a2c0053d0 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a2c003ab0 0x7f6a2c003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 5 monclient: authenticate success, global_id 204351 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.435+0000 7f6a4982c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 1 -- 172.21.15.17:0/2671165010 reap_dead start 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a2c0053d0 0x7f6a2c00cae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00a7b0 tx=0x7f6a3c01d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1291a9d1ebdf7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a340013e0 con 0x7f6a2c0053d0 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: dump: 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0ac640 con 0x7f6a2c0053d0 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 4 set_mon_vals no callback set 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 1 -- 172.21.15.17:0/2671165010 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00a3d0 con 0x7f6a2c0053d0 2024-09-15T21:36:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a41ffb640 10 monclient: dump: 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: shutdown 2024-09-15T21:36:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a2c0053d0 msgr2=0x7f6a2c00cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a2c0053d0 0x7f6a2c00cae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00a7b0 tx=0x7f6a3c01d220 comp rx=0 tx=0).stop 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- 172.21.15.17:0/2671165010 shutdown_connections 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 --2- 172.21.15.17:0/2671165010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a2c0053d0 0x7f6a2c00cae0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- 172.21.15.17:0/2671165010 >> 172.21.15.17:0/2671165010 conn(0x7f6a440a66a0 msgr2=0x7f6a440a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- 172.21.15.17:0/2671165010 shutdown_connections 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- 172.21.15.17:0/2671165010 wait complete. 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: monmap: 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 Processor -- start 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- start start 2024-09-15T21:36:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: init 2024-09-15T21:36:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:36:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c0053d0 0x7f6a44072c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): picked mon.noname-b con 0x7f6a2c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a44073170 0x7f6a440735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): picked mon.noname-c con 0x7f6a44073170 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a441d1c80 0x7f6a441d4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): picked mon.noname-a con 0x7f6a441d1c80 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44134830 con 0x7f6a441d1c80 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441346b0 con 0x7f6a2c0053d0 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6a441349b0 con 0x7f6a44073170 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient(hunting): authenticate will time out at 2100.839154s 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a44073170 0x7f6a440735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a44073170 0x7f6a440735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57188/0 (socket says 172.21.15.17:57188) 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 -- 172.21.15.17:0/1611328252 learned_addr learned my addr 172.21.15.17:0/1611328252 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): get_auth_request con 0x7f6a44073170 auth_method 0 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a441d1c80 0x7f6a441d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): get_auth_request con 0x7f6a441d1c80 auth_method 0 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient(hunting): handle_auth_done global_id 214196 payload 824 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c0053d0 0x7f6a44072c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c0053d0 msgr2=0x7f6a44072c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a2c0053d0 0x7f6a44072c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a441d1c80 msgr2=0x7f6a441d4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a441d1c80 0x7f6a441d4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 monclient: found mon.noname-c 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4411eb20 con 0x7f6a44073170 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a427fc640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a44073170 0x7f6a440735d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800e7c0 tx=0x7f6a3800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38017070 con 0x7f6a44073170 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 10 monclient: dump: 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 5 monclient: authenticate success, global_id 214196 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3800ee70 con 0x7f6a44073170 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: _renew_subs 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4409d0f0 con 0x7f6a44073170 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.439+0000 7f6a2bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800cb40 con 0x7f6a44073170 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: dump: 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 10 monclient: _renew_subs 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4411e840 con 0x7f6a44073170 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a417fa640 4 set_mon_vals no callback set 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 0x7f6a4406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4413c6e0 con 0x7f6a4406f230 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 0x7f6a4406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient: get_auth_request con 0x7f6a4406f230 auth_method 0 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f6a3801a070 con 0x7f6a44073170 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_done global_id 204357 payload 824 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a440a09b0 con 0x7f6a4406f230 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6a3003b9c0 0x7f6a3003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f6a380504b0 con 0x7f6a44073170 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 0x7f6a4406f610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a34003d90 tx=0x7f6a34001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34001d50 con 0x7f6a4406f230 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: dump: 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a437fe640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6a3003b9c0 0x7f6a3003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a437fe640 10 monclient: get_auth_request con 0x7f6a3003b9c0 auth_method 0 2024-09-15T21:36:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a34013570 con 0x7f6a4406f230 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 msgr2=0x7f6a4406f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 0x7f6a4406f610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a34003d90 tx=0x7f6a34001670 comp rx=0 tx=0).stop 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 msgr2=0x7f6a4406f610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a2bfff640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406f230 0x7f6a4406f610 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a34003d90 tx=0x7f6a34001670 comp rx=0 tx=0).stop 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.443+0000 7f6a437fe640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6a3003b9c0 0x7f6a3003de80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00d370 tx=0x7f6a3c0b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a4982c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a4982c640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 0x7f6a441d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a4982c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4411e410 con 0x7f6a4406fb50 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 0x7f6a441d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient: get_auth_request con 0x7f6a4406fb50 auth_method 0 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.447+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a42ffd640 10 monclient(hunting): handle_auth_done global_id 204360 payload 824 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a42ffd640 1 -- 172.21.15.17:0/1611328252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6a4406d020 con 0x7f6a4406fb50 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a42ffd640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 0x7f6a441d85d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a34003d90 tx=0x7f6a34001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34013c80 con 0x7f6a4406fb50 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 10 monclient: dump: 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4715 (secure 0 0 0) 0x7f6a34013c80 con 0x7f6a4406fb50 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:17.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 msgr2=0x7f6a441d85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 0x7f6a441d85d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a34003d90 tx=0x7f6a34001130 comp rx=0 tx=0).stop 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 msgr2=0x7f6a441d85d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a2bfff640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a4406fb50 0x7f6a441d85d0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a34003d90 tx=0x7f6a34001130 comp rx=0 tx=0).stop 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-15T21:36:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 101986, 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:36:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:17.489 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6a3003b9c0 msgr2=0x7f6a3003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f6a3003b9c0 0x7f6a3003de80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00d370 tx=0x7f6a3c0b0040 comp rx=0 tx=0).stop 2024-09-15T21:36:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 10 monclient: shutdown 2024-09-15T21:36:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a44073170 msgr2=0x7f6a440735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 --2- 172.21.15.17:0/1611328252 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6a44073170 0x7f6a440735d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800e7c0 tx=0x7f6a3800ec80 comp rx=0 tx=0).stop 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a437fe640 1 -- 172.21.15.17:0/1611328252 reap_dead start 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a437fe640 1 -- 172.21.15.17:0/1611328252 reap_dead start 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 shutdown_connections 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 >> 172.21.15.17:0/1611328252 conn(0x7f6a440a66a0 msgr2=0x7f6a441384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 shutdown_connections 2024-09-15T21:36:17.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.451+0000 7f6a4982c640 1 -- 172.21.15.17:0/1611328252 wait complete. 2024-09-15T21:36:17.491 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:36:17.492 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:36:17.492 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:36:17.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient: get_monmap_and_config 2024-09-15T21:36:17.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient: monmap: 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:17.604412+0000 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:17.604412+0000 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:17.605 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 Processor -- start 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 -- start start 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient: init 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf409c6a0 0x7efcf409ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): picked mon.noname-a con 0x7efcf409c6a0 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efcf4137c60 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf40a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): picked mon.noname-b con 0x7efcf4138040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf4074840 con 0x7efcf4137c60 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7efcf409a860 con 0x7efcf409c6a0 2024-09-15T21:36:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efcf40742d0 con 0x7efcf4138040 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfc2d7640 10 monclient(hunting): authenticate will time out at 2100.999730s 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf409c6a0 0x7efcf409ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa874640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf40a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf409c6a0 0x7efcf409ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37466/0 (socket says 172.21.15.17:37466) 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 1 -- 172.21.15.17:0/3714304223 learned_addr learned my addr 172.21.15.17:0/3714304223 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa874640 10 monclient(hunting): get_auth_request con 0x7efcf4138040 auth_method 0 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa874640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa874640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa874640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 10 monclient(hunting): get_auth_request con 0x7efcf409c6a0 auth_method 0 2024-09-15T21:36:17.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.599+0000 7efcfa073640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa874640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa874640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa874640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 10 monclient(hunting): handle_auth_done global_id 204363 payload 274 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 1 -- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 msgr2=0x7efcf40a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 1 --2- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf40a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 1 -- 172.21.15.17:0/3714304223 >> v1:172.21.15.119:6789/0 conn(0x7efcf4137c60 legacy=0x7efcf40a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 1 monclient: found mon.noname-a 2024-09-15T21:36:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 1 -- 172.21.15.17:0/3714304223 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf409a580 con 0x7efcf409c6a0 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfa073640 1 --2- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf409ca80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efcf409a9e0 tx=0x7efcf001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bc4cbc59b2fae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 1 -- 172.21.15.17:0/3714304223 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00b1070 con 0x7efcf409c6a0 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: dump: 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.617 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 1 -- 172.21.15.17:0/3714304223 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00ac640 con 0x7efcf409c6a0 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 1 -- 172.21.15.17:0/3714304223 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00aca10 con 0x7efcf409c6a0 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 5 monclient: authenticate success, global_id 204363 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 4 set_mon_vals no callback set 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcf9071640 10 monclient: dump: 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.619 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 10 monclient: shutdown 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 -- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 msgr2=0x7efcf409ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf409ca80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efcf409a9e0 tx=0x7efcf001c400 comp rx=0 tx=0).stop 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 -- 172.21.15.17:0/3714304223 shutdown_connections 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf40a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/3714304223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf409ca80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 -- 172.21.15.17:0/3714304223 >> 172.21.15.17:0/3714304223 conn(0x7efcf40a2c50 msgr2=0x7efcf40a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 -- 172.21.15.17:0/3714304223 shutdown_connections 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 1 -- 172.21.15.17:0/3714304223 wait complete. 2024-09-15T21:36:17.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.603+0000 7efcfc2d7640 10 monclient: monmap: 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 Processor -- start 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- start start 2024-09-15T21:36:17.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: init 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf41cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): picked mon.noname-a con 0x7efcf409c6a0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4137c60 0x7efcf413ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): picked mon.noname-b con 0x7efcf4137c60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): picked mon.noname-c con 0x7efcf4138040 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4142300 con 0x7efcf409c6a0 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efcf40a29c0 con 0x7efcf4137c60 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efcf4142480 con 0x7efcf4138040 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): authenticate will time out at 2101.004788s 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf41cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf41cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37478/0 (socket says 172.21.15.17:37478) 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 -- 172.21.15.17:0/1980336913 learned_addr learned my addr 172.21.15.17:0/1980336913 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): get_auth_request con 0x7efcf409c6a0 auth_method 0 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): get_auth_request con 0x7efcf4138040 auth_method 0 2024-09-15T21:36:17.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4137c60 0x7efcf413ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): get_auth_request con 0x7efcf4137c60 auth_method 0 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient(hunting): handle_auth_done global_id 204366 payload 824 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 msgr2=0x7efcf413f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf413f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4137c60 msgr2=0x7efcf413ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4137c60 0x7efcf413ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 monclient: found mon.noname-a 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf413d750 con 0x7efcf409c6a0 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf4138040 0x7efcf413f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa073640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf41cf8c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efcf00af040 tx=0x7efcf001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00bb020 con 0x7efcf409c6a0 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: dump: 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00b7420 con 0x7efcf409c6a0 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00b1040 con 0x7efcf409c6a0 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: dump: 2024-09-15T21:36:17.627 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf8870640 4 set_mon_vals no callback set 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 5 monclient: authenticate success, global_id 204366 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: _renew_subs 2024-09-15T21:36:17.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf409d340 con 0x7efcf409c6a0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: _renew_subs 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf413ceb0 con 0x7efcf409c6a0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 0x7efcf4139af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efcf409ca80 con 0x7efcf413d0f0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7efcf00be020 con 0x7efcf409c6a0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 0x7efcf4139af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient: get_auth_request con 0x7efcf413d0f0 auth_method 0 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efcd803b9c0 0x7efcd803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 10 monclient(hunting): handle_auth_done global_id 214205 payload 824 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcf413bf50 con 0x7efcf413d0f0 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7efcf00ea550 con 0x7efcf409c6a0 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcf9872640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 0x7efcf4139af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efcf413bee0 tx=0x7efce400e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4019070 con 0x7efcf413d0f0 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efceaffd640 10 monclient: dump: 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.630 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.607+0000 7efcfa874640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efcd803b9c0 0x7efcd803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efcfa874640 10 monclient: get_auth_request con 0x7efcd803b9c0 auth_method 0 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efce40052e0 con 0x7efcf413d0f0 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:17.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 msgr2=0x7efcf4139af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 0x7efcf4139af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efcf413bee0 tx=0x7efce400e960 comp rx=0 tx=0).stop 2024-09-15T21:36:17.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 msgr2=0x7efcf4139af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:17.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efceaffd640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413d0f0 0x7efcf4139af0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efcf413bee0 tx=0x7efce400e960 comp rx=0 tx=0).stop 2024-09-15T21:36:17.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.611+0000 7efcfa874640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efcd803b9c0 0x7efcd803de80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efcec00e740 tx=0x7efcec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:17.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 0x7efcf413a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7efcf413a950 con 0x7efcf413a030 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 0x7efcf413a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient: get_auth_request con 0x7efcf413a030 auth_method 0 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 10 monclient(hunting): handle_auth_done global_id 214208 payload 824 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 1 -- 172.21.15.17:0/1980336913 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efcf406d420 con 0x7efcf413a030 2024-09-15T21:36:17.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcf9872640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 0x7efcf413a410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efce4016000 tx=0x7efce4000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4019040 con 0x7efcf413a030 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 10 monclient: dump: 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.635 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 56863, 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:36:17.636 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.637 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.638 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.640 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:17.641 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:17.643 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:17.644 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:17.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:36:17.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:17.648 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7efce4007050 con 0x7efcf413a030 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 msgr2=0x7efcf413a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 0x7efcf413a410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efce4016000 tx=0x7efce4000ee0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 msgr2=0x7efcf413a410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efceaffd640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7efcf413a030 0x7efcf413a410 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efce4016000 tx=0x7efce4000ee0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efcd803b9c0 msgr2=0x7efcd803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7efcd803b9c0 0x7efcd803de80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efcec00e740 tx=0x7efcec00b040 comp rx=0 tx=0).stop 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 10 monclient: shutdown 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 msgr2=0x7efcf41cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 --2- 172.21.15.17:0/1980336913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf409c6a0 0x7efcf41cf8c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efcf00af040 tx=0x7efcf001ac40 comp rx=0 tx=0).stop 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfa874640 1 -- 172.21.15.17:0/1980336913 reap_dead start 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfa874640 1 -- 172.21.15.17:0/1980336913 reap_dead start 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 shutdown_connections 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 >> 172.21.15.17:0/1980336913 conn(0x7efcf40a2c50 msgr2=0x7efcf40992b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 shutdown_connections 2024-09-15T21:36:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.615+0000 7efcfc2d7640 1 -- 172.21.15.17:0/1980336913 wait complete. 2024-09-15T21:36:17.651 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:36:17.652 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:36:17.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient: get_monmap_and_config 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient: monmap: 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:17.767649+0000 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:17.767649+0000 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.769 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:17.769 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:17.769 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:17.769 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:17.769 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:17.769 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 1 Processor -- start 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 1 -- start start 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient: init 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa87c1340c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa87c09d6c0 0x7fa87c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient(hunting): picked mon.noname-c con 0x7fa87c09d6c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa87c139390 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c09e1b0 con 0x7fa87c139390 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c09e490 con 0x7fa87c1340c0 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa87c09d520 con 0x7fa87c09d6c0 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 10 monclient(hunting): authenticate will time out at 2101.162811s 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa87c139390 0x7fa87c13b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39450/0 (socket says 172.21.15.17:39450) 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87bfff640 1 -- 172.21.15.17:0/1348598918 learned_addr learned my addr 172.21.15.17:0/1348598918 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 1 --2- 172.21.15.17:0/1348598918 >> v2:172.21.15.119:3300/0 conn(0x7fa87c09d6c0 0x7fa87c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): get_auth_request con 0x7fa87c09d6c0 auth_method 0 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829339217 0 0) 0x7fa87c09e1b0 con 0x7fa87c139390 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): _negotiate 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): my global_id is 204369 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa864003530 con 0x7fa87c139390 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 712012399 0 0) 0x7fa864003530 con 0x7fa87c139390 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 >> v2:172.21.15.119:3300/0 conn(0x7fa87c09d6c0 msgr2=0x7fa87c138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 --2- 172.21.15.17:0/1348598918 >> v2:172.21.15.119:3300/0 conn(0x7fa87c09d6c0 0x7fa87c138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 >> v1:172.21.15.119:6789/0 conn(0x7fa87c1340c0 legacy=0x7fa87c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c09e770 con 0x7fa87c139390 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fa874001a80 con 0x7fa87c139390 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 5 monclient: authenticate success, global_id 204369 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: dump: 2024-09-15T21:36:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 >> v1:172.21.15.17:6789/0 conn(0x7fa87c139390 legacy=0x7fa87c13b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:36:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa864003ab0 0x7fa864003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): picked mon.c con 0x7fa864003ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa864004810 0x7fa864004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): picked mon.b con 0x7fa864004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640053d0 0x7fa86400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): picked mon.a con 0x7fa8640053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c09d520 con 0x7fa8640053d0 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400c290 con 0x7fa864004810 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa86400c490 con 0x7fa864003ab0 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87a7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa881b6d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87b7fe640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa864003ab0 0x7fa864003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87bfff640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640053d0 0x7fa86400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87b7fe640 10 monclient(hunting): get_auth_request con 0x7fa864003ab0 auth_method 0 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.763+0000 7fa87b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): get_auth_request con 0x7fa8640053d0 auth_method 0 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient(hunting): handle_auth_done global_id 204369 payload 293 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87affd640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa864004810 0x7fa864004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 -- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa864003ab0 msgr2=0x7fa864003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa864003ab0 0x7fa864003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 -- 172.21.15.17:0/1348598918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa864004810 msgr2=0x7fa864004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa864004810 0x7fa864004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 monclient: found mon.a 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 -- 172.21.15.17:0/1348598918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa864003530 con 0x7fa8640053d0 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87affd640 10 monclient: get_auth_request con 0x7fa864004810 auth_method 0 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87b7fe640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa864003ab0 0x7fa864003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87affd640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa864004810 0x7fa864004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 -- 172.21.15.17:0/1348598918 reap_dead start 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87bfff640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640053d0 0x7fa86400b9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa87400a7b0 tx=0x7fa87401d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7664c586ca36efb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8700013e0 con 0x7fa8640053d0 2024-09-15T21:36:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: dump: 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8740ac640 con 0x7fa8640053d0 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 4 set_mon_vals no callback set 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 1 -- 172.21.15.17:0/1348598918 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87400a3d0 con 0x7fa8640053d0 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa87a7fc640 10 monclient: dump: 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 10 monclient: shutdown 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 -- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640053d0 msgr2=0x7fa86400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640053d0 0x7fa86400b9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa87400a7b0 tx=0x7fa87401d220 comp rx=0 tx=0).stop 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 -- 172.21.15.17:0/1348598918 shutdown_connections 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 --2- 172.21.15.17:0/1348598918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640053d0 0x7fa86400b9d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 -- 172.21.15.17:0/1348598918 >> 172.21.15.17:0/1348598918 conn(0x7fa87c0a66a0 msgr2=0x7fa87c0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 -- 172.21.15.17:0/1348598918 shutdown_connections 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 1 -- 172.21.15.17:0/1348598918 wait complete. 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 10 monclient: build_initial_monmap 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.767+0000 7fa881b6d640 10 monclient: monmap: 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 Processor -- start 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- start start 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: init 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8640053d0 0x7fa87c070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): picked mon.noname-c con 0x7fa8640053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa87c06f230 0x7fa87c06f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): picked mon.noname-a con 0x7fa87c06f230 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c06fbd0 0x7fa87c1d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): picked mon.noname-b con 0x7fa87c06fbd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c134830 con 0x7fa87c06f230 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1346b0 con 0x7fa87c06fbd0 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c1349b0 con 0x7fa8640053d0 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): authenticate will time out at 2101.168806s 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8640053d0 0x7fa87c070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa87c06f230 0x7fa87c06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa87c06f230 0x7fa87c06f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39692/0 (socket says 172.21.15.17:39692) 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8640053d0 0x7fa87c070e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33582/0 (socket says 172.21.15.17:33582) 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 -- 172.21.15.17:0/2697377644 learned_addr learned my addr 172.21.15.17:0/2697377644 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): get_auth_request con 0x7fa87c06f230 auth_method 0 2024-09-15T21:36:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): get_auth_request con 0x7fa8640053d0 auth_method 0 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87bfff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c06fbd0 0x7fa87c1d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient(hunting): handle_auth_done global_id 214211 payload 824 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c06fbd0 msgr2=0x7fa87c1d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c06fbd0 0x7fa87c1d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa87c06f230 msgr2=0x7fa87c06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa87c06f230 0x7fa87c06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 monclient: found mon.noname-c 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c072670 con 0x7fa8640053d0 2024-09-15T21:36:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87bfff640 10 monclient: get_auth_request con 0x7fa87c06fbd0 auth_method 0 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87bfff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c06fbd0 0x7fa87c1d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87b7fe640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8640053d0 0x7fa87c070e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa870001ec0 tx=0x7fa87000f980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8700052e0 con 0x7fa8640053d0 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: dump: 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: _finish_auth 0 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870005480 con 0x7fa8640053d0 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 5 monclient: authenticate success, global_id 214211 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: _renew_subs 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c1334f0 con 0x7fa8640053d0 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870007030 con 0x7fa8640053d0 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 10 monclient: dump: 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: _renew_subs 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c072390 con 0x7fa8640053d0 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa879ffb640 4 set_mon_vals no callback set 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0a0920 con 0x7fa87c134b30 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa87001b020 con 0x7fa8640053d0 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient: get_auth_request con 0x7fa87c134b30 auth_method 0 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa86803b9c0 0x7fa86803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa87004fc40 con 0x7fa8640053d0 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 msgr2=0x7fa87c134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 msgr2=0x7fa87c134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87bfff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa86803b9c0 0x7fa86803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.771+0000 7fa87bfff640 10 monclient: get_auth_request con 0x7fa86803b9c0 auth_method 0 2024-09-15T21:36:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.775+0000 7fa87bfff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa86803b9c0 0x7fa86803de80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c260 tx=0x7fa8740b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient: get_auth_request con 0x7fa87c134b30 auth_method 0 2024-09-15T21:36:17.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:17.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 10 monclient(hunting): handle_auth_done global_id 224125 payload 824 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa87c0008e0 con 0x7fa87c134b30 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa86c016970 tx=0x7fa86c016e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c00f040 con 0x7fa87c134b30 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 10 monclient: dump: 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.982 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa86c0149a0 con 0x7fa87c134b30 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 msgr2=0x7fa87c134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa86c016970 tx=0x7fa86c016e30 comp rx=0 tx=0).stop 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 msgr2=0x7fa87c134f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.975+0000 7fa863fff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c134b30 0x7fa87c134f10 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa86c016970 tx=0x7fa86c016e30 comp rx=0 tx=0).stop 2024-09-15T21:36:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa881b6d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa881b6d640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 0x7fa87c1d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa881b6d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c06ffe0 con 0x7fa87c135450 2024-09-15T21:36:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 0x7fa87c1d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient: get_auth_request con 0x7fa87c135450 auth_method 0 2024-09-15T21:36:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:17.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:17.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:17.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:17.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:17.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 10 monclient(hunting): handle_auth_done global_id 224128 payload 824 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.983+0000 7fa87affd640 1 -- 172.21.15.17:0/2697377644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa87c0731b0 con 0x7fa87c135450 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa87affd640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 0x7fa87c1d61d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa86c008010 tx=0x7fa86c004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c003620 con 0x7fa87c135450 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 10 monclient: dump: 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:17.991 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7fa86c003620 con 0x7fa87c135450 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:17.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 msgr2=0x7fa87c1d61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 0x7fa87c1d61d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa86c008010 tx=0x7fa86c004740 comp rx=0 tx=0).stop 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 msgr2=0x7fa87c1d61d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa863fff640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c135450 0x7fa87c1d61d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-15T21:36:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7787, 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:18.005 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa86803b9c0 msgr2=0x7fa86803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa86803b9c0 0x7fa86803de80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c260 tx=0x7fa8740b0040 comp rx=0 tx=0).stop 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 10 monclient: shutdown 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8640053d0 msgr2=0x7fa87c070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 --2- 172.21.15.17:0/2697377644 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa8640053d0 0x7fa87c070e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa870001ec0 tx=0x7fa87000f980 comp rx=0 tx=0).stop 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa87bfff640 1 -- 172.21.15.17:0/2697377644 reap_dead start 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa87bfff640 1 -- 172.21.15.17:0/2697377644 reap_dead start 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 shutdown_connections 2024-09-15T21:36:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 >> 172.21.15.17:0/2697377644 conn(0x7fa87c0a66a0 msgr2=0x7fa87c09f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 shutdown_connections 2024-09-15T21:36:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:17.987+0000 7fa881b6d640 1 -- 172.21.15.17:0/2697377644 wait complete. 2024-09-15T21:36:18.011 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:36:18.011 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:36:18.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:36:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.131+0000 7f30eba44640 10 monclient: get_monmap_and_config 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.131+0000 7f30eba44640 10 monclient: build_initial_monmap 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.131+0000 7f30eba44640 10 monclient: monmap: 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:18.137646+0000 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:18.137646+0000 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:18.139 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:18.139 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:18.139 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 Processor -- start 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 -- start start 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient: init 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f30e40a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30e409ffc0 0x7f30e413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): picked mon.noname-a con 0x7f30e409ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e40a1380 0x7f30e413d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): picked mon.noname-b con 0x7f30e40a1380 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e40a3130 con 0x7f30e40a0890 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f30e4073df0 con 0x7f30e409ffc0 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f30e40a1bf0 con 0x7f30e40a1380 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient(hunting): authenticate will time out at 2101.533016s 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30e409ffc0 0x7f30e413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e40a1380 0x7f30e413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e97e0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f30e40a0890 0x7f30e40a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39466/0 (socket says 172.21.15.17:39466) 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e97e0640 1 -- 172.21.15.17:0/262453670 learned_addr learned my addr 172.21.15.17:0/262453670 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): get_auth_request con 0x7f30e409ffc0 auth_method 0 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): get_auth_request con 0x7f30e40a1380 auth_method 0 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e9fe1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_done global_id 204375 payload 274 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 -- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e40a1380 msgr2=0x7f30e413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 --2- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e40a1380 0x7f30e413d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 -- 172.21.15.17:0/262453670 >> v1:172.21.15.17:6789/0 conn(0x7f30e40a0890 legacy=0x7f30e40a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient: _finish_hunting 0 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 monclient: found mon.noname-a 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:18.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:18.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 -- 172.21.15.17:0/262453670 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e409ccd0 con 0x7f30e409ffc0 2024-09-15T21:36:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30e8fdf640 1 --2- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e409ffc0 0x7f30e413aac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30e00097f0 tx=0x7f30e001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2b3416e0279171 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 1 -- 172.21.15.17:0/262453670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e00b0070 con 0x7f30e409ffc0 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: dump: 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: _finish_auth 0 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 1 -- 172.21.15.17:0/262453670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e00ab600 con 0x7f30e409ffc0 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 5 monclient: authenticate success, global_id 204375 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 1 -- 172.21.15.17:0/262453670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e00ab9d0 con 0x7f30e409ffc0 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30eba44640 4 set_mon_vals no callback set 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.135+0000 7f30dbfff640 10 monclient: dump: 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: shutdown 2024-09-15T21:36:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e409ffc0 msgr2=0x7f30e413aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 --2- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e409ffc0 0x7f30e413aac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30e00097f0 tx=0x7f30e001c360 comp rx=0 tx=0).stop 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/262453670 shutdown_connections 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 --2- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e40a1380 0x7f30e413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 --2- 172.21.15.17:0/262453670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e409ffc0 0x7f30e413aac0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/262453670 >> 172.21.15.17:0/262453670 conn(0x7f30e40a66a0 msgr2=0x7f30e40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/262453670 shutdown_connections 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/262453670 wait complete. 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: build_initial_monmap 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: monmap: 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 Processor -- start 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- start start 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: init 2024-09-15T21:36:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e409ffc0 0x7f30e4132f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): picked mon.noname-c con 0x7f30e409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 0x7f30e409d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): picked mon.noname-a con 0x7f30e40a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e40a1380 0x7f30e409d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): picked mon.noname-b con 0x7f30e40a1380 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4140100 con 0x7f30e40a0890 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f30e413ff80 con 0x7f30e40a1380 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f30e4140280 con 0x7f30e409ffc0 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient(hunting): authenticate will time out at 2101.538119s 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 0x7f30e409d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e409ffc0 0x7f30e4132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 0x7f30e409d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39702/0 (socket says 172.21.15.17:39702) 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 -- 172.21.15.17:0/174313453 learned_addr learned my addr 172.21.15.17:0/174313453 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): get_auth_request con 0x7f30e40a0890 auth_method 0 2024-09-15T21:36:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): get_auth_request con 0x7f30e409ffc0 auth_method 0 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e97e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e9fe1640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e40a1380 0x7f30e409d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient(hunting): handle_auth_done global_id 204381 payload 824 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 -- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e409ffc0 msgr2=0x7f30e4132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e409ffc0 0x7f30e4132f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 -- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e40a1380 msgr2=0x7f30e409d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e40a1380 0x7f30e409d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient: _finish_hunting 0 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 monclient: found mon.noname-a 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 -- 172.21.15.17:0/174313453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e413b2e0 con 0x7f30e40a0890 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e9fe1640 10 monclient: get_auth_request con 0x7f30e40a1380 auth_method 0 2024-09-15T21:36:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e9fe1640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e40a1380 0x7f30e409d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30e8fdf640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 0x7f30e409d430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30e00ae040 tx=0x7f30e0019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e00b8020 con 0x7f30e40a0890 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: dump: 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: _finish_auth 0 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e00b4610 con 0x7f30e40a0890 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 5 monclient: authenticate success, global_id 204381 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: _renew_subs 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e413b5c0 con 0x7f30e40a0890 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e00b0040 con 0x7f30e40a0890 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:18.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30d9ffb640 10 monclient: dump: 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30db7fe640 4 set_mon_vals no callback set 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: _renew_subs 2024-09-15T21:36:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.139+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e413d0b0 con 0x7f30e40a0890 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30eba44640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30eba44640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30eba44640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e409d0f0 con 0x7f30e40a0890 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f30e00bb020 con 0x7f30e40a0890 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30d9ffb640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30c803b9f0 0x7f30c803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f30e00ea3b0 con 0x7f30e40a0890 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30e97e0640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30c803b9f0 0x7f30c803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30e97e0640 10 monclient: get_auth_request con 0x7f30c803b9f0 auth_method 0 2024-09-15T21:36:18.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.143+0000 7f30e97e0640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30c803b9f0 0x7f30c803deb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f30d4000f70 tx=0x7f30d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.147+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f30e00c2030 con 0x7f30e40a0890 2024-09-15T21:36:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.147+0000 7f30d9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:18.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.147+0000 7f30d9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:18.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.383+0000 7f30eba44640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:36:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.383+0000 7f30eba44640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:36:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.383+0000 7f30eba44640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.383+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f30e40a0c70 con 0x7f30e40a0890 2024-09-15T21:36:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30d9ffb640 1 -- 172.21.15.17:0/174313453 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f30e0016070 con 0x7f30e40a0890 2024-09-15T21:36:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30d9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:36:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30d9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:18.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.387+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.387+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=26,config=2,health=11,kv=12,logm=49} crc {auth=264890498,config=3541143443,health=3885141702,kv=3079713637,logm=3002213916}) 2024-09-15T21:36:18.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.387+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=937362030,config=3541143443,health=4127369059,kv=3858736194,logm=2756889605}) 2024-09-15T21:36:18.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.387+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.387+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=26,config=2,health=11,kv=12,logm=49} crc {auth=264890498,config=3541143443,health=3885141702,kv=3079713637,logm=3002213916}) 2024-09-15T21:36:18.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.387+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=927778126,config=3541143443,health=4127369059,kv=3079713637,logm=3184908694}) 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30c803b9f0 msgr2=0x7f30c803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30c803b9f0 0x7f30c803deb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f30d4000f70 tx=0x7f30d4009040 comp rx=0 tx=0).stop 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 10 monclient: shutdown 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 msgr2=0x7f30e409d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 0x7f30e409d430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30e00ae040 tx=0x7f30e0019100 comp rx=0 tx=0).stop 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 shutdown_connections 2024-09-15T21:36:18.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f30c803b9f0 0x7f30c803deb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e40a1380 0x7f30e409d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e40a0890 0x7f30e409d430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 --2- 172.21.15.17:0/174313453 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f30e409ffc0 0x7f30e4132f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 >> 172.21.15.17:0/174313453 conn(0x7f30e40a66a0 msgr2=0x7f30e4135870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 shutdown_connections 2024-09-15T21:36:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:18.387+0000 7f30eba44640 1 -- 172.21.15.17:0/174313453 wait complete. 2024-09-15T21:36:18.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=468597397}) 2024-09-15T21:36:18.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4290966542}) 2024-09-15T21:36:18.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=468597397}) 2024-09-15T21:36:18.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3458598821}) 2024-09-15T21:36:18.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=373261805}) 2024-09-15T21:36:18.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2999431451}) 2024-09-15T21:36:18.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=373261805}) 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.391+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=404484024}) 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.395+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.395+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=266764730}) 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.395+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2514544788}) 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.395+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.395+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=266764730}) 2024-09-15T21:36:18.402 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.395+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=593823881}) 2024-09-15T21:36:18.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3082564087}) 2024-09-15T21:36:18.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4259865607}) 2024-09-15T21:36:18.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3082564087}) 2024-09-15T21:36:18.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2610738297}) 2024-09-15T21:36:18.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=128641095}) 2024-09-15T21:36:18.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2988370152}) 2024-09-15T21:36:18.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=128641095}) 2024-09-15T21:36:18.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.399+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1499307003}) 2024-09-15T21:36:18.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890995860}) 2024-09-15T21:36:18.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4177216511}) 2024-09-15T21:36:18.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890995860}) 2024-09-15T21:36:18.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=52561480}) 2024-09-15T21:36:18.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727883482}) 2024-09-15T21:36:18.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=94342573}) 2024-09-15T21:36:18.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727883482}) 2024-09-15T21:36:18.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.403+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1190496702}) 2024-09-15T21:36:18.411 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:36:18.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.407+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.407+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2653126603}) 2024-09-15T21:36:18.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.407+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3102105843}) 2024-09-15T21:36:18.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.407+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.407+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2653126603}) 2024-09-15T21:36:18.413 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.407+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2829465738}) 2024-09-15T21:36:18.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=283506415}) 2024-09-15T21:36:18.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371559715}) 2024-09-15T21:36:18.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=283506415}) 2024-09-15T21:36:18.415 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=36485547}) 2024-09-15T21:36:18.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3498004937}) 2024-09-15T21:36:18.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=582016015}) 2024-09-15T21:36:18.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3498004937}) 2024-09-15T21:36:18.417 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.411+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=171940805}) 2024-09-15T21:36:18.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758255275}) 2024-09-15T21:36:18.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3230459470}) 2024-09-15T21:36:18.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758255275}) 2024-09-15T21:36:18.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3623609245}) 2024-09-15T21:36:18.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434234490}) 2024-09-15T21:36:18.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2045725679}) 2024-09-15T21:36:18.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434234490}) 2024-09-15T21:36:18.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.415+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3821356215}) 2024-09-15T21:36:18.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1309991062}) 2024-09-15T21:36:18.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1850265024}) 2024-09-15T21:36:18.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1309991062}) 2024-09-15T21:36:18.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=348938039}) 2024-09-15T21:36:18.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1698943235}) 2024-09-15T21:36:18.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1762092436}) 2024-09-15T21:36:18.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1698943235}) 2024-09-15T21:36:18.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4040571466}) 2024-09-15T21:36:18.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2699900712}) 2024-09-15T21:36:18.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3597357927}) 2024-09-15T21:36:18.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.427 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.419+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2699900712}) 2024-09-15T21:36:18.427 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=328682546}) 2024-09-15T21:36:18.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3315289222}) 2024-09-15T21:36:18.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4093182074}) 2024-09-15T21:36:18.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3315289222}) 2024-09-15T21:36:18.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3533511005}) 2024-09-15T21:36:18.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901297133}) 2024-09-15T21:36:18.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3062324877}) 2024-09-15T21:36:18.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901297133}) 2024-09-15T21:36:18.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.423+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1456996907}) 2024-09-15T21:36:18.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1048498206}) 2024-09-15T21:36:18.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=685725719}) 2024-09-15T21:36:18.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1048498206}) 2024-09-15T21:36:18.432 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=780830178}) 2024-09-15T21:36:18.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1949353252}) 2024-09-15T21:36:18.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1336383211}) 2024-09-15T21:36:18.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1949353252}) 2024-09-15T21:36:18.434 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.427+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=754343358}) 2024-09-15T21:36:18.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4010262713}) 2024-09-15T21:36:18.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2183542058}) 2024-09-15T21:36:18.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4010262713}) 2024-09-15T21:36:18.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=411156382}) 2024-09-15T21:36:18.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279388207}) 2024-09-15T21:36:18.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1418280194}) 2024-09-15T21:36:18.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2279388207}) 2024-09-15T21:36:18.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.431+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4020806020}) 2024-09-15T21:36:18.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1793622900}) 2024-09-15T21:36:18.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2209672786}) 2024-09-15T21:36:18.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1793622900}) 2024-09-15T21:36:18.439 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1064189614}) 2024-09-15T21:36:18.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3565054294}) 2024-09-15T21:36:18.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811026173}) 2024-09-15T21:36:18.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3565054294}) 2024-09-15T21:36:18.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1560657700}) 2024-09-15T21:36:18.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2156344702}) 2024-09-15T21:36:18.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1579678939}) 2024-09-15T21:36:18.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2156344702}) 2024-09-15T21:36:18.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.435+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=51090246}) 2024-09-15T21:36:18.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=735310124}) 2024-09-15T21:36:18.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=375592290}) 2024-09-15T21:36:18.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=735310124}) 2024-09-15T21:36:18.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1172447794}) 2024-09-15T21:36:18.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065412886}) 2024-09-15T21:36:18.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1923305413}) 2024-09-15T21:36:18.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065412886}) 2024-09-15T21:36:18.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.439+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939940469}) 2024-09-15T21:36:18.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=973719615}) 2024-09-15T21:36:18.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2971755686}) 2024-09-15T21:36:18.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=973719615}) 2024-09-15T21:36:18.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1049050998}) 2024-09-15T21:36:18.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=509108337}) 2024-09-15T21:36:18.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=976532523}) 2024-09-15T21:36:18.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=509108337}) 2024-09-15T21:36:18.451 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.443+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=494982372}) 2024-09-15T21:36:18.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247759983}) 2024-09-15T21:36:18.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=676446945}) 2024-09-15T21:36:18.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247759983}) 2024-09-15T21:36:18.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3242345275}) 2024-09-15T21:36:18.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2672946080}) 2024-09-15T21:36:18.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2337056712}) 2024-09-15T21:36:18.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2672946080}) 2024-09-15T21:36:18.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.447+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2955864445}) 2024-09-15T21:36:18.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2370618358}) 2024-09-15T21:36:18.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3380760872}) 2024-09-15T21:36:18.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.457 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2370618358}) 2024-09-15T21:36:18.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341385818}) 2024-09-15T21:36:18.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3730526777}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1857895665}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3730526777}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.451+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3161830042}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=892285227}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3346029482}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=892285227}) 2024-09-15T21:36:18.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=117402615}) 2024-09-15T21:36:18.462 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.462 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=787446050}) 2024-09-15T21:36:18.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3441598129}) 2024-09-15T21:36:18.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=787446050}) 2024-09-15T21:36:18.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.455+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1713984773}) 2024-09-15T21:36:18.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750322745}) 2024-09-15T21:36:18.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4161338768}) 2024-09-15T21:36:18.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750322745}) 2024-09-15T21:36:18.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2483953549}) 2024-09-15T21:36:18.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1703717805}) 2024-09-15T21:36:18.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2951589149}) 2024-09-15T21:36:18.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1703717805}) 2024-09-15T21:36:18.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.459+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3570336049}) 2024-09-15T21:36:18.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2459235831}) 2024-09-15T21:36:18.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=395442146}) 2024-09-15T21:36:18.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2459235831}) 2024-09-15T21:36:18.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3137784552}) 2024-09-15T21:36:18.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137187936}) 2024-09-15T21:36:18.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=826805206}) 2024-09-15T21:36:18.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.469 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137187936}) 2024-09-15T21:36:18.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.463+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2118334363}) 2024-09-15T21:36:18.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1508528839}) 2024-09-15T21:36:18.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4272646813}) 2024-09-15T21:36:18.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1508528839}) 2024-09-15T21:36:18.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1179349948}) 2024-09-15T21:36:18.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=846316956}) 2024-09-15T21:36:18.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1298298915}) 2024-09-15T21:36:18.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=846316956}) 2024-09-15T21:36:18.473 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.467+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3415802000}) 2024-09-15T21:36:18.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2310514663}) 2024-09-15T21:36:18.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2101859118}) 2024-09-15T21:36:18.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2310514663}) 2024-09-15T21:36:18.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3128849290}) 2024-09-15T21:36:18.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703615927}) 2024-09-15T21:36:18.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3823521499}) 2024-09-15T21:36:18.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703615927}) 2024-09-15T21:36:18.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.471+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=266685033}) 2024-09-15T21:36:18.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3158108151}) 2024-09-15T21:36:18.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2723588851}) 2024-09-15T21:36:18.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3158108151}) 2024-09-15T21:36:18.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3740095415}) 2024-09-15T21:36:18.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3529929091}) 2024-09-15T21:36:18.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3343893920}) 2024-09-15T21:36:18.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3529929091}) 2024-09-15T21:36:18.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1347867912}) 2024-09-15T21:36:18.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758792306}) 2024-09-15T21:36:18.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=660328362}) 2024-09-15T21:36:18.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.475+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758792306}) 2024-09-15T21:36:18.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524015624}) 2024-09-15T21:36:18.484 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.484 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=126693887}) 2024-09-15T21:36:18.484 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4197658551}) 2024-09-15T21:36:18.484 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.484 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=126693887}) 2024-09-15T21:36:18.484 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2844429374}) 2024-09-15T21:36:18.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2419836446}) 2024-09-15T21:36:18.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3199585425}) 2024-09-15T21:36:18.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2419836446}) 2024-09-15T21:36:18.486 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.479+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1106879710}) 2024-09-15T21:36:18.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805150155}) 2024-09-15T21:36:18.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1436501144}) 2024-09-15T21:36:18.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805150155}) 2024-09-15T21:36:18.488 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1769992381}) 2024-09-15T21:36:18.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889898957}) 2024-09-15T21:36:18.490 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1439585626}) 2024-09-15T21:36:18.490 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.490 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889898957}) 2024-09-15T21:36:18.490 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.483+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1644024194}) 2024-09-15T21:36:18.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2621287667}) 2024-09-15T21:36:18.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2569814454}) 2024-09-15T21:36:18.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2621287667}) 2024-09-15T21:36:18.492 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1039098113}) 2024-09-15T21:36:18.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2938750484}) 2024-09-15T21:36:18.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4293043345}) 2024-09-15T21:36:18.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2938750484}) 2024-09-15T21:36:18.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.487+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078294847}) 2024-09-15T21:36:18.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=747840817}) 2024-09-15T21:36:18.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1941868962}) 2024-09-15T21:36:18.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=747840817}) 2024-09-15T21:36:18.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1946692581}) 2024-09-15T21:36:18.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255642462}) 2024-09-15T21:36:18.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1911148634}) 2024-09-15T21:36:18.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255642462}) 2024-09-15T21:36:18.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.491+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=298938643}) 2024-09-15T21:36:18.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916314908}) 2024-09-15T21:36:18.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2543597915}) 2024-09-15T21:36:18.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916314908}) 2024-09-15T21:36:18.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1137751377}) 2024-09-15T21:36:18.501 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.501 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551594815}) 2024-09-15T21:36:18.501 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1915882089}) 2024-09-15T21:36:18.501 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.501 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551594815}) 2024-09-15T21:36:18.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.495+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=402681427}) 2024-09-15T21:36:18.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1940843757}) 2024-09-15T21:36:18.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2758860764}) 2024-09-15T21:36:18.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1940843757}) 2024-09-15T21:36:18.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3669748391}) 2024-09-15T21:36:18.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859023578}) 2024-09-15T21:36:18.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4245245438}) 2024-09-15T21:36:18.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859023578}) 2024-09-15T21:36:18.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.499+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4254208281}) 2024-09-15T21:36:18.506 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3017046846}) 2024-09-15T21:36:18.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3838971504}) 2024-09-15T21:36:18.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3017046846}) 2024-09-15T21:36:18.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3288367661}) 2024-09-15T21:36:18.508 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986169400}) 2024-09-15T21:36:18.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079676764}) 2024-09-15T21:36:18.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986169400}) 2024-09-15T21:36:18.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.503+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4210191682}) 2024-09-15T21:36:18.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141919295}) 2024-09-15T21:36:18.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=332239403}) 2024-09-15T21:36:18.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141919295}) 2024-09-15T21:36:18.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3932608178}) 2024-09-15T21:36:18.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809345489}) 2024-09-15T21:36:18.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=62509620}) 2024-09-15T21:36:18.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809345489}) 2024-09-15T21:36:18.513 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1425471843}) 2024-09-15T21:36:18.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2362710547}) 2024-09-15T21:36:18.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3437049027}) 2024-09-15T21:36:18.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2362710547}) 2024-09-15T21:36:18.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.507+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2949728128}) 2024-09-15T21:36:18.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231790069}) 2024-09-15T21:36:18.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878244509}) 2024-09-15T21:36:18.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231790069}) 2024-09-15T21:36:18.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1257708518}) 2024-09-15T21:36:18.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1504118502}) 2024-09-15T21:36:18.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=486340663}) 2024-09-15T21:36:18.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1504118502}) 2024-09-15T21:36:18.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.511+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1972745504}) 2024-09-15T21:36:18.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3027513433}) 2024-09-15T21:36:18.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3102714150}) 2024-09-15T21:36:18.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3027513433}) 2024-09-15T21:36:18.520 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3111920233}) 2024-09-15T21:36:18.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1037767059}) 2024-09-15T21:36:18.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=874330446}) 2024-09-15T21:36:18.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1037767059}) 2024-09-15T21:36:18.522 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.515+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=244723755}) 2024-09-15T21:36:18.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197423461}) 2024-09-15T21:36:18.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2022608969}) 2024-09-15T21:36:18.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197423461}) 2024-09-15T21:36:18.524 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379682092}) 2024-09-15T21:36:18.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3091758487}) 2024-09-15T21:36:18.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3984051990}) 2024-09-15T21:36:18.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3091758487}) 2024-09-15T21:36:18.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.519+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3529573483}) 2024-09-15T21:36:18.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2254902896}) 2024-09-15T21:36:18.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1423402933}) 2024-09-15T21:36:18.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2254902896}) 2024-09-15T21:36:18.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4249932737}) 2024-09-15T21:36:18.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=70889846}) 2024-09-15T21:36:18.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3889157788}) 2024-09-15T21:36:18.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.523+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=70889846}) 2024-09-15T21:36:18.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=210639817}) 2024-09-15T21:36:18.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1424720912}) 2024-09-15T21:36:18.533 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=394023187}) 2024-09-15T21:36:18.533 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.533 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1424720912}) 2024-09-15T21:36:18.533 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.527+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2682152445}) 2024-09-15T21:36:18.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727707955}) 2024-09-15T21:36:18.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2669312716}) 2024-09-15T21:36:18.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727707955}) 2024-09-15T21:36:18.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3697713685}) 2024-09-15T21:36:18.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3980474963}) 2024-09-15T21:36:18.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1095807962}) 2024-09-15T21:36:18.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3980474963}) 2024-09-15T21:36:18.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.531+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2236868213}) 2024-09-15T21:36:18.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.535+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.535+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3107396719}) 2024-09-15T21:36:18.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.535+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3295465095}) 2024-09-15T21:36:18.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.535+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.535+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3107396719}) 2024-09-15T21:36:18.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.535+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=206929516}) 2024-09-15T21:36:18.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=587761326}) 2024-09-15T21:36:18.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902752696}) 2024-09-15T21:36:18.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=587761326}) 2024-09-15T21:36:18.542 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3236710704}) 2024-09-15T21:36:18.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1930311231}) 2024-09-15T21:36:18.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2955749559}) 2024-09-15T21:36:18.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1930311231}) 2024-09-15T21:36:18.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.539+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=210429045}) 2024-09-15T21:36:18.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803897558}) 2024-09-15T21:36:18.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2304602021}) 2024-09-15T21:36:18.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803897558}) 2024-09-15T21:36:18.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1412729466}) 2024-09-15T21:36:18.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2837967205}) 2024-09-15T21:36:18.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1219337886}) 2024-09-15T21:36:18.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2837967205}) 2024-09-15T21:36:18.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.543+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4237276260}) 2024-09-15T21:36:18.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341592349}) 2024-09-15T21:36:18.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=857720570}) 2024-09-15T21:36:18.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341592349}) 2024-09-15T21:36:18.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=180839766}) 2024-09-15T21:36:18.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326097361}) 2024-09-15T21:36:18.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2225647714}) 2024-09-15T21:36:18.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326097361}) 2024-09-15T21:36:18.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.547+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2481299220}) 2024-09-15T21:36:18.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.555 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3250840036}) 2024-09-15T21:36:18.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4079975090}) 2024-09-15T21:36:18.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3250840036}) 2024-09-15T21:36:18.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1440747265}) 2024-09-15T21:36:18.557 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=893187801}) 2024-09-15T21:36:18.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603610860}) 2024-09-15T21:36:18.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=893187801}) 2024-09-15T21:36:18.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.551+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2312979308}) 2024-09-15T21:36:18.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=556320135}) 2024-09-15T21:36:18.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1740074511}) 2024-09-15T21:36:18.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=556320135}) 2024-09-15T21:36:18.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2342960929}) 2024-09-15T21:36:18.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1086860989}) 2024-09-15T21:36:18.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2003454926}) 2024-09-15T21:36:18.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1086860989}) 2024-09-15T21:36:18.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.555+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2823593493}) 2024-09-15T21:36:18.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2914777191}) 2024-09-15T21:36:18.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=753112235}) 2024-09-15T21:36:18.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2914777191}) 2024-09-15T21:36:18.564 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=467370427}) 2024-09-15T21:36:18.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=226287782}) 2024-09-15T21:36:18.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3000223219}) 2024-09-15T21:36:18.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=226287782}) 2024-09-15T21:36:18.566 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.559+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2638163491}) 2024-09-15T21:36:18.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2910425762}) 2024-09-15T21:36:18.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1016230662}) 2024-09-15T21:36:18.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2910425762}) 2024-09-15T21:36:18.568 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=452824020}) 2024-09-15T21:36:18.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266269238}) 2024-09-15T21:36:18.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=253832624}) 2024-09-15T21:36:18.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266269238}) 2024-09-15T21:36:18.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.563+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3758255219}) 2024-09-15T21:36:18.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034946813}) 2024-09-15T21:36:18.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2700695432}) 2024-09-15T21:36:18.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034946813}) 2024-09-15T21:36:18.572 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1587500989}) 2024-09-15T21:36:18.573 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=767663676}) 2024-09-15T21:36:18.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1145002365}) 2024-09-15T21:36:18.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=767663676}) 2024-09-15T21:36:18.574 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.567+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3135614069}) 2024-09-15T21:36:18.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554119529}) 2024-09-15T21:36:18.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3886532895}) 2024-09-15T21:36:18.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554119529}) 2024-09-15T21:36:18.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1872002634}) 2024-09-15T21:36:18.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4136493178}) 2024-09-15T21:36:18.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=138685394}) 2024-09-15T21:36:18.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4136493178}) 2024-09-15T21:36:18.578 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.571+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=625728967}) 2024-09-15T21:36:18.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1474876108}) 2024-09-15T21:36:18.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3363452112}) 2024-09-15T21:36:18.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1474876108}) 2024-09-15T21:36:18.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=113479990}) 2024-09-15T21:36:18.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266415945}) 2024-09-15T21:36:18.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2032057119}) 2024-09-15T21:36:18.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4266415945}) 2024-09-15T21:36:18.582 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.575+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2646499782}) 2024-09-15T21:36:18.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=237529201}) 2024-09-15T21:36:18.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580324596}) 2024-09-15T21:36:18.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=237529201}) 2024-09-15T21:36:18.584 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258870385}) 2024-09-15T21:36:18.585 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1719510012}) 2024-09-15T21:36:18.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=663866948}) 2024-09-15T21:36:18.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1719510012}) 2024-09-15T21:36:18.586 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.579+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1386164320}) 2024-09-15T21:36:18.587 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.587 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1955552623}) 2024-09-15T21:36:18.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2557827197}) 2024-09-15T21:36:18.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1955552623}) 2024-09-15T21:36:18.588 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196776865}) 2024-09-15T21:36:18.589 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2344028666}) 2024-09-15T21:36:18.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483624596}) 2024-09-15T21:36:18.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2344028666}) 2024-09-15T21:36:18.590 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.583+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2794792133}) 2024-09-15T21:36:18.591 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1005884726}) 2024-09-15T21:36:18.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4206266949}) 2024-09-15T21:36:18.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1005884726}) 2024-09-15T21:36:18.592 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416735366}) 2024-09-15T21:36:18.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130330000}) 2024-09-15T21:36:18.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=641175255}) 2024-09-15T21:36:18.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130330000}) 2024-09-15T21:36:18.594 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.587+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2066166179}) 2024-09-15T21:36:18.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.595 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3887112089}) 2024-09-15T21:36:18.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1407714401}) 2024-09-15T21:36:18.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3887112089}) 2024-09-15T21:36:18.596 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=332747472}) 2024-09-15T21:36:18.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1262022017}) 2024-09-15T21:36:18.597 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3097867809}) 2024-09-15T21:36:18.598 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.598 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1262022017}) 2024-09-15T21:36:18.598 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.591+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3706965386}) 2024-09-15T21:36:18.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.595+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.595+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2549724083}) 2024-09-15T21:36:18.599 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.595+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=315006624}) 2024-09-15T21:36:18.600 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.595+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.600 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.595+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2549724083}) 2024-09-15T21:36:18.600 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.595+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1768340639}) 2024-09-15T21:36:18.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.603+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.603+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2724363309}) 2024-09-15T21:36:18.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.603+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1773903893}) 2024-09-15T21:36:18.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.603+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.603+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2724363309}) 2024-09-15T21:36:18.607 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.603+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2020705622}) 2024-09-15T21:36:18.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.607+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.607+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948244996}) 2024-09-15T21:36:18.612 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.607+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=486796692}) 2024-09-15T21:36:18.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.607+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.607+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948244996}) 2024-09-15T21:36:18.613 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.607+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072817469}) 2024-09-15T21:36:18.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.659 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2011745655}) 2024-09-15T21:36:18.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=553029944}) 2024-09-15T21:36:18.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2011745655}) 2024-09-15T21:36:18.660 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=606708155}) 2024-09-15T21:36:18.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4109108698}) 2024-09-15T21:36:18.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1013898852}) 2024-09-15T21:36:18.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4109108698}) 2024-09-15T21:36:18.661 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.655+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1649874854}) 2024-09-15T21:36:18.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3012689171}) 2024-09-15T21:36:18.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3716400611}) 2024-09-15T21:36:18.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3012689171}) 2024-09-15T21:36:18.663 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933314254}) 2024-09-15T21:36:18.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=84014086}) 2024-09-15T21:36:18.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1555008716}) 2024-09-15T21:36:18.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=84014086}) 2024-09-15T21:36:18.665 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.659+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3403234838}) 2024-09-15T21:36:18.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705107058}) 2024-09-15T21:36:18.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2720341725}) 2024-09-15T21:36:18.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705107058}) 2024-09-15T21:36:18.667 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1255485403}) 2024-09-15T21:36:18.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802930775}) 2024-09-15T21:36:18.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642322177}) 2024-09-15T21:36:18.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802930775}) 2024-09-15T21:36:18.669 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.663+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364062327}) 2024-09-15T21:36:18.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4026768463}) 2024-09-15T21:36:18.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3057584291}) 2024-09-15T21:36:18.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4026768463}) 2024-09-15T21:36:18.671 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1133896234}) 2024-09-15T21:36:18.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.672 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533671852}) 2024-09-15T21:36:18.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=747276922}) 2024-09-15T21:36:18.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533671852}) 2024-09-15T21:36:18.673 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.667+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2771190987}) 2024-09-15T21:36:18.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=773079157}) 2024-09-15T21:36:18.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4009747803}) 2024-09-15T21:36:18.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=773079157}) 2024-09-15T21:36:18.675 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=479925113}) 2024-09-15T21:36:18.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3318663786}) 2024-09-15T21:36:18.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1440390448}) 2024-09-15T21:36:18.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.676 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3318663786}) 2024-09-15T21:36:18.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3983673033}) 2024-09-15T21:36:18.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=944334368}) 2024-09-15T21:36:18.677 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3376138254}) 2024-09-15T21:36:18.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=944334368}) 2024-09-15T21:36:18.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.671+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2010118399}) 2024-09-15T21:36:18.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.678 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1107816601}) 2024-09-15T21:36:18.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3233027084}) 2024-09-15T21:36:18.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1107816601}) 2024-09-15T21:36:18.679 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1335380132}) 2024-09-15T21:36:18.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307892629}) 2024-09-15T21:36:18.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3923376969}) 2024-09-15T21:36:18.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307892629}) 2024-09-15T21:36:18.680 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=864270625}) 2024-09-15T21:36:18.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3575439677}) 2024-09-15T21:36:18.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1026488020}) 2024-09-15T21:36:18.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3575439677}) 2024-09-15T21:36:18.681 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.675+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=223131275}) 2024-09-15T21:36:18.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2783966490}) 2024-09-15T21:36:18.682 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2526594433}) 2024-09-15T21:36:18.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2783966490}) 2024-09-15T21:36:18.683 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2974078237}) 2024-09-15T21:36:18.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.685 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=20} crc {logm=4202987715,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2955735567,mon_config_key=4092048061,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=1810338860,osdmap=2981043933}) 2024-09-15T21:36:18.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=26,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=22} crc {logm=1502047459,mdsmap=3009587246,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=532350818,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1558429179}) 2024-09-15T21:36:18.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=12,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=20} crc {logm=4202987715,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2955735567,mon_config_key=4092048061,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=1810338860,osdmap=2981043933}) 2024-09-15T21:36:18.686 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.679+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=26,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=14,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=24} crc {logm=3387652470,mdsmap=3009587245,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=2215352900,mon_config_key=521969046,monmap=186996059,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=2950480066}) 2024-09-15T21:36:18.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.695+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.695+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=30} crc {osdmap=3288885529}) 2024-09-15T21:36:18.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.695+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=33} crc {osdmap=1594257787}) 2024-09-15T21:36:18.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.695+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:36:18.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.695+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=30} crc {osdmap=3288885529}) 2024-09-15T21:36:18.700 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:18.695+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=33} crc {osdmap=3324766863}) 2024-09-15T21:36:19.413 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:36:19.413 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:36:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient: get_monmap_and_config 2024-09-15T21:36:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient: build_initial_monmap 2024-09-15T21:36:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient: monmap: 2024-09-15T21:36:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:19.544091+0000 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:19.544091+0000 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:19.545 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 Processor -- start 2024-09-15T21:36:19.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 -- start start 2024-09-15T21:36:19.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient: init 2024-09-15T21:36:19.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:19.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa5300a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): picked mon.noname-b con 0x7fa5300a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa53009ffc0 0x7fa5300a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): picked mon.noname-a con 0x7fa53009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa530135670 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5300a3130 con 0x7fa530135670 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa5300a16f0 con 0x7fa53009ffc0 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa530073df0 con 0x7fa5300a0890 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa535dd2640 10 monclient(hunting): authenticate will time out at 2102.939496s 2024-09-15T21:36:19.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa53009ffc0 0x7fa5300a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa5300a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa530135670 0x7fa530135aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39476/0 (socket says 172.21.15.17:39476) 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52ffff640 1 -- 172.21.15.17:0/1226413743 learned_addr learned my addr 172.21.15.17:0/1226413743 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): get_auth_request con 0x7fa5300a0890 auth_method 0 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): get_auth_request con 0x7fa53009ffc0 auth_method 0 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.539+0000 7fa52effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 1 -- 172.21.15.17:0/1226413743 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507607760 0 0) 0x7fa5300a3130 con 0x7fa530135670 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_done global_id 214220 payload 274 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 1 -- 172.21.15.17:0/1226413743 >> v2:172.21.15.17:3300/0 conn(0x7fa53009ffc0 msgr2=0x7fa5300a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 1 --2- 172.21.15.17:0/1226413743 >> v2:172.21.15.17:3300/0 conn(0x7fa53009ffc0 0x7fa5300a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 1 -- 172.21.15.17:0/1226413743 >> v1:172.21.15.17:6789/0 conn(0x7fa530135670 legacy=0x7fa530135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 1 monclient: found mon.noname-b 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:36:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 1 -- 172.21.15.17:0/1226413743 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5300a1bf0 con 0x7fa5300a0890 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52effd640 1 --2- 172.21.15.17:0/1226413743 >> v2:172.21.15.17:3300/0 conn(0x7fa53009ffc0 0x7fa5300a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52f7fe640 1 --2- 172.21.15.17:0/1226413743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa5300a0c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5300a1870 tx=0x7fa52001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24fce6129ad6a216 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 1 -- 172.21.15.17:0/1226413743 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200b0070 con 0x7fa5300a0890 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: dump: 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.552 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: _finish_auth 0 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 1 -- 172.21.15.17:0/1226413743 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5200ab600 con 0x7fa5300a0890 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 5 monclient: authenticate success, global_id 214220 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 4 set_mon_vals no callback set 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 1 -- 172.21.15.17:0/1226413743 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200ab9d0 con 0x7fa5300a0890 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa52e7fc640 10 monclient: dump: 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.554 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 10 monclient: shutdown 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 -- 172.21.15.17:0/1226413743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 msgr2=0x7fa5300a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 --2- 172.21.15.17:0/1226413743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa5300a0c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5300a1870 tx=0x7fa52001c360 comp rx=0 tx=0).stop 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 -- 172.21.15.17:0/1226413743 shutdown_connections 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 --2- 172.21.15.17:0/1226413743 >> v2:172.21.15.17:3300/0 conn(0x7fa53009ffc0 0x7fa5300a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 --2- 172.21.15.17:0/1226413743 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa5300a0c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 -- 172.21.15.17:0/1226413743 >> 172.21.15.17:0/1226413743 conn(0x7fa5300a66a0 msgr2=0x7fa5300a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 -- 172.21.15.17:0/1226413743 shutdown_connections 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 -- 172.21.15.17:0/1226413743 wait complete. 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 10 monclient: build_initial_monmap 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 10 monclient: monmap: 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.561 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 Processor -- start 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 1 -- start start 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.543+0000 7fa535dd2640 10 monclient: init 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 0x7fa53013c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): picked mon.noname-a con 0x7fa53009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa53013c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): picked mon.noname-c con 0x7fa5300a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530135670 0x7fa53013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): picked mon.noname-b con 0x7fa530135670 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa530140690 con 0x7fa53009ffc0 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa530140510 con 0x7fa530135670 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa530140810 con 0x7fa5300a0890 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient(hunting): authenticate will time out at 2102.944472s 2024-09-15T21:36:19.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 0x7fa53013c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 0x7fa53013c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39710/0 (socket says 172.21.15.17:39710) 2024-09-15T21:36:19.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 -- 172.21.15.17:0/2484887078 learned_addr learned my addr 172.21.15.17:0/2484887078 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:19.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): get_auth_request con 0x7fa53009ffc0 auth_method 0 2024-09-15T21:36:19.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa53013c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): get_auth_request con 0x7fa5300a0890 auth_method 0 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient(hunting): handle_auth_done global_id 204390 payload 824 2024-09-15T21:36:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 -- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 msgr2=0x7fa53013c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa53013c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 -- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530135670 msgr2=0x7fa53013ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52ffff640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530135670 0x7fa53013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530135670 0x7fa53013ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 monclient: found mon.noname-a 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 -- 172.21.15.17:0/2484887078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53011ddf0 con 0x7fa53009ffc0 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52ffff640 10 monclient: get_auth_request con 0x7fa530135670 auth_method 0 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52ffff640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530135670 0x7fa53013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52f7fe640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 0x7fa53013c430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa5200ae040 tx=0x7fa520019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200b8020 con 0x7fa53009ffc0 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: dump: 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: _finish_auth 0 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5200b4630 con 0x7fa53009ffc0 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200b0040 con 0x7fa53009ffc0 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 10 monclient: dump: 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.568 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 5 monclient: authenticate success, global_id 204390 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _renew_subs 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52dffb640 4 set_mon_vals no callback set 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5300a55d0 con 0x7fa53009ffc0 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _renew_subs 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53011e980 con 0x7fa53009ffc0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530135aa0 con 0x7fa53009ffc0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa5200bb020 con 0x7fa53009ffc0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa51403b9c0 0x7fa51403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa5200ea3d0 con 0x7fa53009ffc0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa51403b9c0 0x7fa51403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 10 monclient: get_auth_request con 0x7fa51403b9c0 auth_method 0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.547+0000 7fa52effd640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa51403b9c0 0x7fa51403de80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa53013dab0 tx=0x7fa51800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.551+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa5200c2030 con 0x7fa53009ffc0 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.551+0000 7fa527fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:19.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.551+0000 7fa527fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa535dd2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:36:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa535dd2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa535dd2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa53006d020 con 0x7fa53009ffc0 2024-09-15T21:36:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa527fff640 1 -- 172.21.15.17:0/2484887078 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fa520016070 con 0x7fa53009ffc0 2024-09-15T21:36:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa527fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.795+0000 7fa527fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:19.800 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa51403b9c0 msgr2=0x7fa51403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa51403b9c0 0x7fa51403de80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa53013dab0 tx=0x7fa51800b040 comp rx=0 tx=0).stop 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 10 monclient: shutdown 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 msgr2=0x7fa53013c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 0x7fa53013c430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa5200ae040 tx=0x7fa520019100 comp rx=0 tx=0).stop 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 shutdown_connections 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa51403b9c0 0x7fa51403de80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530135670 0x7fa53013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa5300a0890 0x7fa53013c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 --2- 172.21.15.17:0/2484887078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53009ffc0 0x7fa53013c430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 >> 172.21.15.17:0/2484887078 conn(0x7fa5300a66a0 msgr2=0x7fa530072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 shutdown_connections 2024-09-15T21:36:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.799+0000 7fa535dd2640 1 -- 172.21.15.17:0/2484887078 wait complete. 2024-09-15T21:36:19.820 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:36:19.820 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T21:36:19.820 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:36:19.820 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.935+0000 7ff119fd0640 10 monclient: get_monmap_and_config 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.935+0000 7ff119fd0640 10 monclient: build_initial_monmap 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.935+0000 7ff119fd0640 10 monclient: monmap: 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:19.940911+0000 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:19.940911+0000 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:19.942 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.935+0000 7ff119fd0640 1 Processor -- start 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.935+0000 7ff119fd0640 1 -- start start 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.935+0000 7ff119fd0640 10 monclient: init 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff1141378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff1141383b0 0x7ff11413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): picked mon.noname-a con 0x7ff1141383b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff11413d2c0 0x7ff11413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): picked mon.noname-c con 0x7ff11413d2c0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1140a1a20 con 0x7ff1141378c0 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff1140a1d00 con 0x7ff1141383b0 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff1140a1fe0 con 0x7ff11413d2c0 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient(hunting): authenticate will time out at 2103.336611s 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff1141383b0 0x7ff11413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff1141383b0 0x7ff11413cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39724/0 (socket says 172.21.15.17:39724) 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 -- 172.21.15.17:0/2191616989 learned_addr learned my addr 172.21.15.17:0/2191616989 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): get_auth_request con 0x7ff1141383b0 auth_method 0 2024-09-15T21:36:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff113fff640 1 --2- 172.21.15.17:0/2191616989 >> v2:172.21.15.119:3300/0 conn(0x7ff11413d2c0 0x7ff11413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient(hunting): handle_auth_done global_id 204393 payload 274 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 -- 172.21.15.17:0/2191616989 >> v2:172.21.15.119:3300/0 conn(0x7ff11413d2c0 msgr2=0x7ff11413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 --2- 172.21.15.17:0/2191616989 >> v2:172.21.15.119:3300/0 conn(0x7ff11413d2c0 0x7ff11413f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 -- 172.21.15.17:0/2191616989 >> v1:172.21.15.119:6789/0 conn(0x7ff1141378c0 legacy=0x7ff114137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient: _finish_hunting 0 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 monclient: found mon.noname-a 2024-09-15T21:36:19.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 -- 172.21.15.17:0/2191616989 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1140a22c0 con 0x7ff1141383b0 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff10affd640 1 --2- 172.21.15.17:0/2191616989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141383b0 0x7ff11413cd80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009870 tx=0x7ff0fc01bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaf277d97dab7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 1 -- 172.21.15.17:0/2191616989 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0ab070 con 0x7ff1141383b0 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: dump: 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: _finish_auth 0 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 1 -- 172.21.15.17:0/2191616989 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc0a5660 con 0x7ff1141383b0 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 1 -- 172.21.15.17:0/2191616989 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0a5a30 con 0x7ff1141383b0 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 5 monclient: authenticate success, global_id 204393 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 4 set_mon_vals no callback set 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff112ffd640 10 monclient: dump: 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 10 monclient: shutdown 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 -- 172.21.15.17:0/2191616989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141383b0 msgr2=0x7ff11413cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.939+0000 7ff119fd0640 1 --2- 172.21.15.17:0/2191616989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141383b0 0x7ff11413cd80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009870 tx=0x7ff0fc01bee0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/2191616989 shutdown_connections 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 --2- 172.21.15.17:0/2191616989 >> v2:172.21.15.119:3300/0 conn(0x7ff11413d2c0 0x7ff11413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 --2- 172.21.15.17:0/2191616989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141383b0 0x7ff11413cd80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/2191616989 >> 172.21.15.17:0/2191616989 conn(0x7ff1140aac40 msgr2=0x7ff1140a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/2191616989 shutdown_connections 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/2191616989 wait complete. 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: build_initial_monmap 2024-09-15T21:36:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: monmap: 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 Processor -- start 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- start start 2024-09-15T21:36:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: init 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1141378c0 0x7ff11413c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): picked mon.noname-b con 0x7ff1141378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1141383b0 0x7ff1141d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): picked mon.noname-c con 0x7ff1141383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff11413d2c0 0x7ff1141d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): picked mon.noname-a con 0x7ff11413d2c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff114143020 con 0x7ff11413d2c0 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff114142ea0 con 0x7ff1141378c0 2024-09-15T21:36:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff1141431a0 con 0x7ff1141383b0 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): authenticate will time out at 2103.341643s 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1141383b0 0x7ff1141d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff11413d2c0 0x7ff1141d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1141383b0 0x7ff1141d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33620/0 (socket says 172.21.15.17:33620) 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff11413d2c0 0x7ff1141d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39728/0 (socket says 172.21.15.17:39728) 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 -- 172.21.15.17:0/1123089027 learned_addr learned my addr 172.21.15.17:0/1123089027 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): get_auth_request con 0x7ff1141383b0 auth_method 0 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): get_auth_request con 0x7ff11413d2c0 auth_method 0 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff113fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1141378c0 0x7ff11413c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): get_auth_request con 0x7ff1141378c0 auth_method 0 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient(hunting): handle_auth_done global_id 214226 payload 824 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1141378c0 msgr2=0x7ff11413c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1141378c0 0x7ff11413c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff11413d2c0 msgr2=0x7ff1141d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff11413d2c0 0x7ff1141d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient: _finish_hunting 0 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 monclient: found mon.noname-c 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11409d940 con 0x7ff1141383b0 2024-09-15T21:36:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff10affd640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1141383b0 0x7ff1141d0ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc01c780 tx=0x7ff0fc018fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0ab050 con 0x7ff1141383b0 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: dump: 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: _finish_auth 0 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc0b0650 con 0x7ff1141383b0 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0b0a20 con 0x7ff1141383b0 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 10 monclient: dump: 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 5 monclient: authenticate success, global_id 214226 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: _renew_subs 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1127fc640 4 set_mon_vals no callback set 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1141d1950 con 0x7ff1141383b0 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: _renew_subs 2024-09-15T21:36:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11409e460 con 0x7ff1141383b0 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 0x7ff11402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff11409ed60 con 0x7ff1141cdc70 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 0x7ff11402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient: get_auth_request con 0x7ff1141cdc70 auth_method 0 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff0fc0ae070 con 0x7ff1141383b0 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 10 monclient(hunting): handle_auth_done global_id 204399 payload 824 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff114137ca0 con 0x7ff1141cdc70 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0f403ba20 0x7ff0f403dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff0fc0e9aa0 con 0x7ff1141383b0 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.943+0000 7ff1137fe640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 0x7ff11402d450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e450 tx=0x7ff10400e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040052e0 con 0x7ff1141cdc70 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 10 monclient: dump: 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff113fff640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0f403ba20 0x7ff0f403dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff113fff640 10 monclient: get_auth_request con 0x7ff0f403ba20 auth_method 0 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff1040052e0 con 0x7ff1141cdc70 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 msgr2=0x7ff11402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 0x7ff11402d450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e450 tx=0x7ff10400e910 comp rx=0 tx=0).stop 2024-09-15T21:36:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 msgr2=0x7ff11402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff110ff9640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141cdc70 0x7ff11402d450 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e450 tx=0x7ff10400e910 comp rx=0 tx=0).stop 2024-09-15T21:36:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.947+0000 7ff113fff640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0f403ba20 0x7ff0f403dee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff11413ce30 tx=0x7ff10c0076f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.951+0000 7ff119fd0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.951+0000 7ff119fd0640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 0x7ff11402d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.951+0000 7ff119fd0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.951+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff11409e0d0 con 0x7ff1141ce050 2024-09-15T21:36:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 0x7ff11402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient: get_auth_request con 0x7ff1141ce050 auth_method 0 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 10 monclient(hunting): handle_auth_done global_id 204402 payload 824 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 1 -- 172.21.15.17:0/1123089027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff11413d0a0 con 0x7ff1141ce050 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff1137fe640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 0x7ff11402d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e450 tx=0x7ff104001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104003810 con 0x7ff1141ce050 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 10 monclient: dump: 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4715 (secure 0 0 0) 0x7ff104003810 con 0x7ff1141ce050 2024-09-15T21:36:19.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 msgr2=0x7ff11402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 0x7ff11402d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e450 tx=0x7ff104001030 comp rx=0 tx=0).stop 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 msgr2=0x7ff11402d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff110ff9640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1141ce050 0x7ff11402d990 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e450 tx=0x7ff104001030 comp rx=0 tx=0).stop 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:36:19.972 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 104490, 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:19.973 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:19.974 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:19.975 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:19.976 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:19.977 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:19.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:19.980 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.981 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:19.982 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:19.983 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:19.984 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:19.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0f403ba20 msgr2=0x7ff0f403dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff0f403ba20 0x7ff0f403dee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff11413ce30 tx=0x7ff10c0076f0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 10 monclient: shutdown 2024-09-15T21:36:19.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1141383b0 msgr2=0x7ff1141d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 --2- 172.21.15.17:0/1123089027 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff1141383b0 0x7ff1141d0ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc01c780 tx=0x7ff0fc018fa0 comp rx=0 tx=0).stop 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff113fff640 1 -- 172.21.15.17:0/1123089027 reap_dead start 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff113fff640 1 -- 172.21.15.17:0/1123089027 reap_dead start 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 shutdown_connections 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 >> 172.21.15.17:0/1123089027 conn(0x7ff1140aac40 msgr2=0x7ff114136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 shutdown_connections 2024-09-15T21:36:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:19.955+0000 7ff119fd0640 1 -- 172.21.15.17:0/1123089027 wait complete. 2024-09-15T21:36:19.986 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:36:19.986 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:36:19.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:36:20.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient: get_monmap_and_config 2024-09-15T21:36:20.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient: build_initial_monmap 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient: monmap: 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:20.102299+0000 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:20.102299+0000 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:20.103 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 Processor -- start 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 -- start start 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient: init 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 0x7f4ecc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): picked mon.noname-b con 0x7f4ecc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ecc1381e0 0x7f4ecc0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): picked mon.noname-a con 0x7f4ecc1381e0 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4ecc0a5040 0x7f4ecc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): picked mon.noname-c con 0x7f4ecc0a5040 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4ecc0a1d00 con 0x7f4ecc1381e0 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4ecc0a1fe0 con 0x7f4ecc0a5040 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0a22c0 con 0x7f4ecc1378c0 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed2acc640 10 monclient(hunting): authenticate will time out at 2103.497607s 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 0x7f4ecc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 0x7f4ecc137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33634/0 (socket says 172.21.15.17:33634) 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 -- 172.21.15.17:0/1802495942 learned_addr learned my addr 172.21.15.17:0/1802495942 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): get_auth_request con 0x7f4ecc1378c0 auth_method 0 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ecbfff640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.17:3300/0 conn(0x7f4ecc1381e0 0x7f4ecc0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient(hunting): handle_auth_done global_id 214229 payload 274 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed1069640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.119:3300/0 conn(0x7f4ecc0a5040 0x7f4ecc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 -- 172.21.15.17:0/1802495942 >> v2:172.21.15.119:3300/0 conn(0x7f4ecc0a5040 msgr2=0x7f4ecc0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.119:3300/0 conn(0x7f4ecc0a5040 0x7f4ecc0a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 -- 172.21.15.17:0/1802495942 >> v2:172.21.15.17:3300/0 conn(0x7f4ecc1381e0 msgr2=0x7f4ecc0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.17:3300/0 conn(0x7f4ecc1381e0 0x7f4ecc0a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient: _finish_hunting 0 2024-09-15T21:36:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 monclient: found mon.noname-b 2024-09-15T21:36:20.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:20.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ed0868640 1 -- 172.21.15.17:0/1802495942 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc0a4490 con 0x7f4ecc1378c0 2024-09-15T21:36:20.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ecbfff640 10 monclient: get_auth_request con 0x7f4ecc1381e0 auth_method 0 2024-09-15T21:36:20.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.099+0000 7f4ecbfff640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.17:3300/0 conn(0x7f4ecc1381e0 0x7f4ecc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1802495942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 0x7f4ecc137ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a6150 tx=0x7f4ec401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd8e13087ef54ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 1 -- 172.21.15.17:0/1802495942 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40b0070 con 0x7f4ecc1378c0 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: dump: 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 1 -- 172.21.15.17:0/1802495942 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec40ab600 con 0x7f4ecc1378c0 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 1 -- 172.21.15.17:0/1802495942 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40ab9d0 con 0x7f4ecc1378c0 2024-09-15T21:36:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 5 monclient: authenticate success, global_id 214229 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 4 set_mon_vals no callback set 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecb7fe640 10 monclient: dump: 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: shutdown 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1802495942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 msgr2=0x7f4ecc137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1802495942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 0x7f4ecc137ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a6150 tx=0x7f4ec401c3e0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1802495942 shutdown_connections 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.119:3300/0 conn(0x7f4ecc0a5040 0x7f4ecc0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1802495942 >> v2:172.21.15.17:3300/0 conn(0x7f4ecc1381e0 0x7f4ecc0a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1802495942 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1378c0 0x7f4ecc137ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1802495942 >> 172.21.15.17:0/1802495942 conn(0x7f4ecc0aac40 msgr2=0x7f4ecc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1802495942 shutdown_connections 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1802495942 wait complete. 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: build_initial_monmap 2024-09-15T21:36:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: monmap: 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 Processor -- start 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- start start 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: init 2024-09-15T21:36:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0a5040 0x7f4ecc1cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): picked mon.noname-b con 0x7f4ecc0a5040 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc1378c0 0x7f4ecc136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): picked mon.noname-a con 0x7f4ecc1378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1381e0 0x7f4ecc13cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): picked mon.noname-c con 0x7f4ecc1381e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc141730 con 0x7f4ecc1378c0 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc1415b0 con 0x7f4ecc0a5040 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc1418b0 con 0x7f4ecc1381e0 2024-09-15T21:36:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed2acc640 10 monclient(hunting): authenticate will time out at 2103.503418s 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1381e0 0x7f4ecc13cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc1378c0 0x7f4ecc136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1381e0 0x7f4ecc13cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33642/0 (socket says 172.21.15.17:33642) 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc1378c0 0x7f4ecc136c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39756/0 (socket says 172.21.15.17:39756) 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 -- 172.21.15.17:0/1071463946 learned_addr learned my addr 172.21.15.17:0/1071463946 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): get_auth_request con 0x7f4ecc1381e0 auth_method 0 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): get_auth_request con 0x7f4ecc1378c0 auth_method 0 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0a5040 0x7f4ecc1cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed0868640 10 monclient(hunting): get_auth_request con 0x7f4ecc0a5040 auth_method 0 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed0868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed0868640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed0868640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient(hunting): handle_auth_done global_id 214232 payload 824 2024-09-15T21:36:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0a5040 msgr2=0x7f4ecc1cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0a5040 0x7f4ecc1cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc1378c0 msgr2=0x7f4ecc136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc1378c0 0x7f4ecc136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient: _finish_hunting 0 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 monclient: found mon.noname-c 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ed1069640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc122330 con 0x7f4ecc1381e0 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.103+0000 7f4ecbfff640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc1378c0 0x7f4ecc136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed1069640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1381e0 0x7f4ecc13cec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c820 tx=0x7f4eb400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4015070 con 0x7f4ecc1381e0 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: dump: 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: _finish_auth 0 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4004220 con 0x7f4ecc1381e0 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 5 monclient: authenticate success, global_id 214232 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 10 monclient: _renew_subs 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc0a8890 con 0x7f4ecc1381e0 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400ce10 con 0x7f4ecc1381e0 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: dump: 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.125 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 10 monclient: _renew_subs 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc0a0800 con 0x7f4ecc1381e0 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4eca7fc640 4 set_mon_vals no callback set 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 0x7f4ecc13d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc137ca0 con 0x7f4ecc120e10 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 0x7f4ecc13d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient: get_auth_request con 0x7f4ecc120e10 auth_method 0 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4eb4007830 con 0x7f4ecc1381e0 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 10 monclient(hunting): handle_auth_done global_id 214235 payload 824 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ecc0008e0 con 0x7f4ecc120e10 2024-09-15T21:36:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4eac03b9c0 0x7f4eac03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4eb4050640 con 0x7f4ecc1381e0 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 0x7f4ecc13d400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a7b50 tx=0x7f4ec401ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40b0040 con 0x7f4ecc120e10 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: dump: 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ecbfff640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4eac03b9c0 0x7f4eac03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ecbfff640 10 monclient: get_auth_request con 0x7f4eac03b9c0 auth_method 0 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4ec40abdc0 con 0x7f4ecc120e10 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 msgr2=0x7f4ecc13d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 0x7f4ecc13d400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a7b50 tx=0x7f4ec401ac20 comp rx=0 tx=0).stop 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 msgr2=0x7f4ecc13d400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ec97fa640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc120e10 0x7f4ecc13d400 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a7b50 tx=0x7f4ec401ac20 comp rx=0 tx=0).stop 2024-09-15T21:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.107+0000 7f4ecbfff640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4eac03b9c0 0x7f4eac03de80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc121610 tx=0x7f4ebc007810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 0x7f4ecc13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc06cba0 con 0x7f4ecc1371b0 2024-09-15T21:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 0x7f4ecc13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient: get_auth_request con 0x7f4ecc1371b0 auth_method 0 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 10 monclient(hunting): handle_auth_done global_id 214238 payload 824 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 1 -- 172.21.15.17:0/1071463946 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4ecc1dafa0 con 0x7f4ecc1371b0 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed0868640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 0x7f4ecc13d940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4ec401c420 tx=0x7f4ec4019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40b0040 con 0x7f4ecc1371b0 2024-09-15T21:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 10 monclient: dump: 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4712 (secure 0 0 0) 0x7f4ec40bf9f0 con 0x7f4ecc1371b0 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 msgr2=0x7f4ecc13d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 0x7f4ecc13d940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4ec401c420 tx=0x7f4ec4019180 comp rx=0 tx=0).stop 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 msgr2=0x7f4ecc13d940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ec97fa640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1371b0 0x7f4ecc13d940 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4ec401c420 tx=0x7f4ec4019180 comp rx=0 tx=0).stop 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:20.134 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 59363, 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:20.135 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:20.136 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:20.137 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:20.138 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:20.139 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:20.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:20.141 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:20.143 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.146 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4eac03b9c0 msgr2=0x7f4eac03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4eac03b9c0 0x7f4eac03de80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc121610 tx=0x7f4ebc007810 comp rx=0 tx=0).stop 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 10 monclient: shutdown 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1381e0 msgr2=0x7f4ecc13cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 --2- 172.21.15.17:0/1071463946 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ecc1381e0 0x7f4ecc13cec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c820 tx=0x7f4eb400cce0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed1069640 1 -- 172.21.15.17:0/1071463946 reap_dead start 2024-09-15T21:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed1069640 1 -- 172.21.15.17:0/1071463946 reap_dead start 2024-09-15T21:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 shutdown_connections 2024-09-15T21:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 >> 172.21.15.17:0/1071463946 conn(0x7f4ecc0aac40 msgr2=0x7f4ecc06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 shutdown_connections 2024-09-15T21:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.115+0000 7f4ed2acc640 1 -- 172.21.15.17:0/1071463946 wait complete. 2024-09-15T21:36:20.149 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:36:20.149 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:36:20.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.251+0000 7f57e75ea640 10 monclient: get_monmap_and_config 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.251+0000 7f57e75ea640 10 monclient: build_initial_monmap 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.251+0000 7f57e75ea640 10 monclient: monmap: 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:20.257939+0000 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:20.257939+0000 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:20.259 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 1 Processor -- start 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 1 -- start start 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient: init 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57e00a0890 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57e009ffc0 0x7f57e013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient(hunting): picked mon.noname-a con 0x7f57e009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f57e00a1380 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e00a3130 con 0x7f57e00a0890 2024-09-15T21:36:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e0073df0 con 0x7f57e00a1380 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f57e00a1bf0 con 0x7f57e009ffc0 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e75ea640 10 monclient(hunting): authenticate will time out at 2103.653324s 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57e009ffc0 0x7f57e013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57e009ffc0 0x7f57e013aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39760/0 (socket says 172.21.15.17:39760) 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e5386640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57e00a0890 0x7f57e00a0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39480/0 (socket says 172.21.15.17:39480) 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 -- 172.21.15.17:0/4125283031 learned_addr learned my addr 172.21.15.17:0/4125283031 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): get_auth_request con 0x7f57e009ffc0 auth_method 0 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient(hunting): handle_auth_done global_id 204408 payload 274 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 -- 172.21.15.17:0/4125283031 >> v1:172.21.15.119:6789/0 conn(0x7f57e00a1380 legacy=0x7f57e013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 -- 172.21.15.17:0/4125283031 >> v1:172.21.15.17:6789/0 conn(0x7f57e00a0890 legacy=0x7f57e00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient: _finish_hunting 0 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 monclient: found mon.noname-a 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57d7fff640 1 -- 172.21.15.17:0/4125283031 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 823215581 0 0) 0x7f57e00a3130 con 0x7f57e00a0890 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57e4b85640 1 -- 172.21.15.17:0/4125283031 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e009ccd0 con 0x7f57e009ffc0 2024-09-15T21:36:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.255+0000 7f57d7fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:36:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e4b85640 1 --2- 172.21.15.17:0/4125283031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e009ffc0 0x7f57e013aac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0098a0 tx=0x7f57dc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bce0558365f9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 1 -- 172.21.15.17:0/4125283031 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0b1070 con 0x7f57e009ffc0 2024-09-15T21:36:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: dump: 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: _finish_auth 0 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 5 monclient: authenticate success, global_id 204408 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 1 -- 172.21.15.17:0/4125283031 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0ac640 con 0x7f57e009ffc0 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 1 -- 172.21.15.17:0/4125283031 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0aca10 con 0x7f57e009ffc0 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d7fff640 10 monclient: dump: 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 4 set_mon_vals no callback set 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: shutdown 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/4125283031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e009ffc0 msgr2=0x7f57e013aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 --2- 172.21.15.17:0/4125283031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e009ffc0 0x7f57e013aac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0098a0 tx=0x7f57dc01c400 comp rx=0 tx=0).stop 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/4125283031 shutdown_connections 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 --2- 172.21.15.17:0/4125283031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e009ffc0 0x7f57e013aac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/4125283031 >> 172.21.15.17:0/4125283031 conn(0x7f57e00a66a0 msgr2=0x7f57e00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/4125283031 shutdown_connections 2024-09-15T21:36:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/4125283031 wait complete. 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: build_initial_monmap 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: monmap: 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 Processor -- start 2024-09-15T21:36:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- start start 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: init 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57e009ffc0 0x7f57e01cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): picked mon.noname-c con 0x7f57e009ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e00a0890 0x7f57e01cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): picked mon.noname-b con 0x7f57e00a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e00a1380 0x7f57e01cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): picked mon.noname-a con 0x7f57e00a1380 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01412e0 con 0x7f57e00a1380 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0141160 con 0x7f57e00a0890 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f57e0141460 con 0x7f57e009ffc0 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient(hunting): authenticate will time out at 2103.658829s 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57e009ffc0 0x7f57e01cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e00a1380 0x7f57e01cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57e009ffc0 0x7f57e01cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:33672/0 (socket says 172.21.15.17:33672) 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 -- 172.21.15.17:0/3735796200 learned_addr learned my addr 172.21.15.17:0/3735796200 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): get_auth_request con 0x7f57e00a1380 auth_method 0 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): get_auth_request con 0x7f57e009ffc0 auth_method 0 2024-09-15T21:36:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5b87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e4b85640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e00a0890 0x7f57e01cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient(hunting): handle_auth_done global_id 214241 payload 824 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e00a0890 msgr2=0x7f57e01cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e00a0890 0x7f57e01cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e00a1380 msgr2=0x7f57e01cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e00a1380 0x7f57e01cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient: _finish_hunting 0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 monclient: found mon.noname-c 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e006f700 con 0x7f57e009ffc0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e4b85640 10 monclient: get_auth_request con 0x7f57e00a0890 auth_method 0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e4b85640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e00a0890 0x7f57e01cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e5386640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57e009ffc0 0x7f57e01cd360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f57d000c490 tx=0x7f57d000c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0018070 con 0x7f57e009ffc0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: dump: 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:36:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 5 monclient: authenticate success, global_id 214241 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d0007500 con 0x7f57e009ffc0 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: _renew_subs 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e011edd0 con 0x7f57e009ffc0 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0004a30 con 0x7f57e009ffc0 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d5ffb640 10 monclient: dump: 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57d77fe640 4 set_mon_vals no callback set 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: _renew_subs 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.259+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e011df60 con 0x7f57e009ffc0 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e75ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e75ea640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 0x7f57e0138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e75ea640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57e011e6c0 con 0x7f57e013c930 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f57d0004bd0 con 0x7f57e009ffc0 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57c403b9c0 0x7f57c403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f57d004f9a0 con 0x7f57e009ffc0 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 0x7f57e0138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient: get_auth_request con 0x7f57e013c930 auth_method 0 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e5b87640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57c403b9c0 0x7f57c403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e5b87640 10 monclient: get_auth_request con 0x7f57c403b9c0 auth_method 0 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 10 monclient(hunting): handle_auth_done global_id 224137 payload 824 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57e00a0c70 con 0x7f57e013c930 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e5b87640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57c403b9c0 0x7f57c403de80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f57e013d980 tx=0x7f57d8003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57e4b85640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 0x7f57e0138ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f57e009d0a0 tx=0x7f57dc01ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0bb070 con 0x7f57e013c930 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 10 monclient: dump: 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f57dc0ace10 con 0x7f57e013c930 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 msgr2=0x7f57e0138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 0x7f57e0138ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f57e009d0a0 tx=0x7f57dc01ac40 comp rx=0 tx=0).stop 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 msgr2=0x7f57e0138ab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.263+0000 7f57d5ffb640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e013c930 0x7f57e0138ab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 0x7f57e0139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57e006d020 con 0x7f57e0138ff0 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 0x7f57e0139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient: get_auth_request con 0x7f57e0138ff0 auth_method 0 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 10 monclient(hunting): handle_auth_done global_id 224140 payload 824 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 1 -- 172.21.15.17:0/3735796200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f57e0139940 con 0x7f57e0138ff0 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e4b85640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 0x7f57e0139400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57dc000c10 tx=0x7f57dc0191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0bb040 con 0x7f57e0138ff0 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 10 monclient: dump: 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f57dc003d60 con 0x7f57e0138ff0 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 msgr2=0x7f57e0139400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 0x7f57e0139400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57dc000c10 tx=0x7f57dc0191a0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 msgr2=0x7f57e0139400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57d5ffb640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57e0138ff0 0x7f57e0139400 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10071, 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:36:20.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:20.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:20.290 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:20.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:20.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:20.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:20.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:20.296 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:20.299 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:20.300 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:20.300 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:20.300 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:20.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:20.300 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:20.300 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57c403b9c0 msgr2=0x7f57c403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f57c403b9c0 0x7f57c403de80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f57e013d980 tx=0x7f57d8003d50 comp rx=0 tx=0).stop 2024-09-15T21:36:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 10 monclient: shutdown 2024-09-15T21:36:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57e009ffc0 msgr2=0x7f57e01cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 --2- 172.21.15.17:0/3735796200 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f57e009ffc0 0x7f57e01cd360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f57d000c490 tx=0x7f57d000c950 comp rx=0 tx=0).stop 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e5b87640 1 -- 172.21.15.17:0/3735796200 reap_dead start 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e5b87640 1 -- 172.21.15.17:0/3735796200 reap_dead start 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 shutdown_connections 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 >> 172.21.15.17:0/3735796200 conn(0x7f57e00a66a0 msgr2=0x7f57e006fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 shutdown_connections 2024-09-15T21:36:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:20.271+0000 7f57e75ea640 1 -- 172.21.15.17:0/3735796200 wait complete. 2024-09-15T21:36:20.302 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:36:20.303 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T21:36:20.303 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-15T21:36:20.303 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T21:36:20.303 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T21:36:20.303 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:36:20.303 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:36:20.315 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:36:20.315 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T21:36:20.315 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T21:36:20.315 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:36:32.517 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=110 2024-09-15T21:36:32.517 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 110 -ne 110 ]] 2024-09-15T21:36:32.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:36:32.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ export CEPH_ARGS= 2024-09-15T21:36:32.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ CEPH_ARGS= 2024-09-15T21:36:32.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph auth del client.bar 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient: get_monmap_and_config 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient: build_initial_monmap 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient: monmap: 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:36:32.634172+0000 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:36:32.634172+0000 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:32.628 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:32.629 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:32.629 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 Processor -- start 2024-09-15T21:36:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 -- start start 2024-09-15T21:36:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient: init 2024-09-15T21:36:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:32.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): picked mon.noname-a con 0x7fa448134b10 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 0x7fa448063a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): picked mon.noname-b con 0x7fa448063640 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa448063fd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4480a47d0 con 0x7fa448063fd0 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa448073b50 con 0x7fa448134b10 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480a1b50 con 0x7fa448063640 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:32.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44d122640 10 monclient(hunting): authenticate will time out at 2111.431610s 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44659c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 msgr2=0x7fa448063a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa44659c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 0x7fa448063a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa446d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa446d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.119:42006/0 (socket says 172.21.15.119:42006) 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 learned_addr learned my addr 172.21.15.119:0/599675246 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.629+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448134b10 auth_method 0 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:32.633 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.633+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:32.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.833+0000 7fa44659c640 1 -- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 msgr2=0x7fa448063a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:32.831 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.833+0000 7fa44659c640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 0x7fa448063a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448134b10 auth_method 0 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:32.836 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:32.837 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:32.837+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:33.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.237+0000 7fa44659c640 1 -- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 msgr2=0x7fa448063a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:33.235 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.237+0000 7fa44659c640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 0x7fa448063a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:33.239 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448134b10 auth_method 0 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:33.240 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:33.241+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.041+0000 7fa44659c640 1 -- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 msgr2=0x7fa448063a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:34.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.041+0000 7fa44659c640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 0x7fa448063a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448134b10 auth_method 0 2024-09-15T21:36:34.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:36:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:34.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:34.045+0000 7fa446d9d640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:35.630 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): tick 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 monclient(hunting): continuing hunt 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 msgr2=0x7fa448063a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa448063640 0x7fa448063a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 msgr2=0x7fa448134ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.17:3300/0 conn(0x7fa448134b10 0x7fa448134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 >> v1:172.21.15.119:6789/0 conn(0x7fa448063fd0 legacy=0x7fa44813eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient: _add_conns ranks=[4,0] 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.119:3300/0 conn(0x7fa4340036d0 0x7fa434005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): picked mon.noname-c con 0x7fa4340036d0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa434006220 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4480a1e30 con 0x7fa434006220 2024-09-15T21:36:35.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:35.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa448073b50 con 0x7fa4340036d0 2024-09-15T21:36:35.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:35.629+0000 7fa444d99640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.629+0000 7fa444d99640 10 monclient(hunting): tick 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.629+0000 7fa444d99640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.629+0000 7fa444d99640 1 monclient(hunting): continuing hunt 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.629+0000 7fa444d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.629+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 >> v2:172.21.15.119:3300/0 conn(0x7fa4340036d0 msgr2=0x7fa434005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.629+0000 7fa444d99640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.119:3300/0 conn(0x7fa4340036d0 0x7fa434005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 >> v1:172.21.15.17:6789/0 conn(0x7fa434006220 legacy=0x7fa4340086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:38.631 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa448063fd0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa44759e640 1 -- 172.21.15.119:0/599675246 reap_dead start 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.119:3300/0 conn(0x7fa43400a800 0x7fa434010180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 10 monclient(hunting): picked mon.noname-c con 0x7fa43400a800 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa4340106c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4480a47d0 con 0x7fa4340106c0 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa434004b60 con 0x7fa448063fd0 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 1 -- 172.21.15.119:0/599675246 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa448073b50 con 0x7fa43400a800 2024-09-15T21:36:38.632 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:38.633+0000 7fa444d99640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:40.316 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T21:36:40.316 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T21:36:40.317 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-15T21:36:40.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:36:40.318 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 1 --2- 172.21.15.119:0/599675246 >> v2:172.21.15.119:3300/0 conn(0x7fa43400a800 0x7fa434010180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa43400a800 auth_method 0 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.317+0000 7fa44659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:40.319 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 10 monclient(hunting): handle_auth_done global_id 224143 payload 274 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 1 -- 172.21.15.119:0/599675246 >> v1:172.21.15.119:6789/0 conn(0x7fa448063fd0 legacy=0x7fa43400a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 1 -- 172.21.15.119:0/599675246 >> v1:172.21.15.17:6789/0 conn(0x7fa4340106c0 legacy=0x7fa4340086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 10 monclient: _finish_hunting 0 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 1 monclient: found mon.noname-c 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:36:40.320 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.321+0000 7fa44659c640 1 -- 172.21.15.119:0/599675246 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa434005de0 con 0x7fa43400a800 2024-09-15T21:36:40.325 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.325+0000 7fa44659c640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43400a800 0x7fa434010180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa434009ec0 tx=0x7fa43c01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c71d56ff9226e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 1 -- 172.21.15.119:0/599675246 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0a5a30 con 0x7fa43400a800 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: dump: 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:40.341 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: _finish_auth 0 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 1 -- 172.21.15.119:0/599675246 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c0a9070 con 0x7fa43400a800 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 1 -- 172.21.15.119:0/599675246 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0a5e10 con 0x7fa43400a800 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 5 monclient: authenticate success, global_id 224143 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 4 set_mon_vals no callback set 2024-09-15T21:36:40.342 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa445d9b640 10 monclient: dump: 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:40.343 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 10 monclient: shutdown 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 -- 172.21.15.119:0/599675246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43400a800 msgr2=0x7fa434010180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43400a800 0x7fa434010180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa434009ec0 tx=0x7fa43c01ca10 comp rx=0 tx=0).stop 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 -- 172.21.15.119:0/599675246 shutdown_connections 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 --2- 172.21.15.119:0/599675246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43400a800 0x7fa434010180 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 -- 172.21.15.119:0/599675246 >> 172.21.15.119:0/599675246 conn(0x7fa4480a7690 msgr2=0x7fa4480a3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 -- 172.21.15.119:0/599675246 shutdown_connections 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.341+0000 7fa44d122640 1 -- 172.21.15.119:0/599675246 wait complete. 2024-09-15T21:36:40.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient: build_initial_monmap 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient: monmap: 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 Processor -- start 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 -- start start 2024-09-15T21:36:40.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient: init 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa448121de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): picked mon.noname-c con 0x7fa43400a800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): picked mon.noname-a con 0x7fa4340106c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): picked mon.noname-b con 0x7fa448063fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa44809dc90 con 0x7fa4340106c0 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480a2c00 con 0x7fa448063fd0 2024-09-15T21:36:40.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa44809de10 con 0x7fa43400a800 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44d122640 10 monclient(hunting): authenticate will time out at 2119.145918s 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa446d9d640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa448121de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa446d9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa448121de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58470/0 (socket says 172.21.15.119:58470) 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 learned_addr learned my addr 172.21.15.119:0/4150192280 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:40.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.345+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:40.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient: get_monmap_and_config 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient: build_initial_monmap 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient: monmap: 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:40.451533+0000 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:40.451533+0000 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:40.453 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:40.453 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 Processor -- start 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 -- start start 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient: init 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f35700a0890 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): picked mon.noname-c con 0x7f357009ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): picked mon.noname-a con 0x7f35700a1380 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35700a3130 con 0x7f35700a0890 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3570073df0 con 0x7f35700a1380 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f35700a1bf0 con 0x7f357009ffc0 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f3577260640 10 monclient(hunting): authenticate will time out at 2123.846943s 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35402/0 (socket says 172.21.15.17:35402) 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 learned_addr learned my addr 172.21.15.17:0/1517879514 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 10 monclient(hunting): get_auth_request con 0x7f35700a1380 auth_method 0 2024-09-15T21:36:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 10 monclient(hunting): get_auth_request con 0x7f357009ffc0 auth_method 0 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:36:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.447+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:40.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa448121de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:40.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa448121de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:40.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:40.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:40.549 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:40.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.549+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 10 monclient(hunting): get_auth_request con 0x7f35700a1380 auth_method 0 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 10 monclient(hunting): get_auth_request con 0x7f357009ffc0 auth_method 0 2024-09-15T21:36:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:40.651+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:40.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa448121de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:40.948 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa448121de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:40.950 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:40.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:40.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:40.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:40.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:40.951 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:40.953+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:41.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 10 monclient(hunting): get_auth_request con 0x7f35700a1380 auth_method 0 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:41.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 10 monclient(hunting): get_auth_request con 0x7f357009ffc0 auth_method 0 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:41.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.055+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa448121de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa448121de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:41.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:41.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:41.757+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:41.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 10 monclient(hunting): get_auth_request con 0x7f35700a1380 auth_method 0 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 10 monclient(hunting): get_auth_request con 0x7f357009ffc0 auth_method 0 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:41.859+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:43.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): tick 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa448121de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa448121de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa448122860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa448122860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa448122320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa448122320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:36:43.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7fa448063fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): picked mon.noname-a con 0x7fa4340106c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7fa43400a800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa44809dc90 con 0x7fa4340106c0 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480a2c00 con 0x7fa448063fd0 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:43.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa44809de10 con 0x7fa43400a800 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa42e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:43.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:43.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.345+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:43.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): tick 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 msgr2=0x7f357013aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.119:3300/0 conn(0x7f357009ffc0 0x7f357013aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 -- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 msgr2=0x7f357013d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 --2- 172.21.15.17:0/1517879514 >> v2:172.21.15.17:3300/0 conn(0x7f35700a1380 0x7f357013d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 -- 172.21.15.17:0/1517879514 >> v1:172.21.15.119:6789/0 conn(0x7f35700a0890 legacy=0x7f35700a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient: _add_conns ranks=[2,0] 2024-09-15T21:36:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 0x7f3550005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7f35500036b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3550006230 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 -- 172.21.15.17:0/1517879514 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f357009ccd0 con 0x7f3550006230 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 1 -- 172.21.15.17:0/1517879514 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3570073df0 con 0x7f35500036b0 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f356e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f3574ffc640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 msgr2=0x7f3550005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:43.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.451+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 0x7f3550005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:43.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:43.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.549+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.655+0000 7f3574ffc640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 msgr2=0x7f3550005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:43.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:43.655+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 0x7f3550005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:43.952 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:43.953 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:43.953+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:44.059+0000 7f3574ffc640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 msgr2=0x7f3550005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:44.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:44.059+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 0x7f3550005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:44.756 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:36:44.757 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:44.757+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:44.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:44.863+0000 7f3574ffc640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 msgr2=0x7f3550005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:44.863+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 0x7f3550005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:36:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.371+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285158167 0 0) 0x7f357009ccd0 con 0x7f3550006230 2024-09-15T21:36:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.371+0000 7f356f7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:36:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.371+0000 7f356f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.371+0000 7f356f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.371+0000 7f356f7fe640 10 monclient(hunting): my global_id is 204417 2024-09-15T21:36:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.371+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f355c003530 con 0x7f3550006230 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.375+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798264305 0 0) 0x7f355c003530 con 0x7f3550006230 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.375+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 msgr2=0x7f3550005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.375+0000 7f356f7fe640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f35500036b0 0x7f3550005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.375+0000 7f356f7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.375+0000 7f356f7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.375+0000 7f356f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355000b040 con 0x7f3550006230 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f355800f040 con 0x7f3550006230 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3577260640 5 monclient: authenticate success, global_id 204417 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3577260640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: dump: 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 >> v1:172.21.15.17:6789/0 conn(0x7f3550006230 legacy=0x7f35500086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0039c0 0x7f355c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): picked mon.b con 0x7f355c0039c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c004810 0x7f355c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 reap_dead start 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): picked mon.a con 0x7f355c004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c0053d0 0x7f355c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): picked mon.c con 0x7f355c0053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3570073df0 con 0x7f355c004810 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f355c00c270 con 0x7f355c0039c0 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f355c00c470 con 0x7f355c0053d0 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f35757fd640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c0053d0 msgr2=0x7f355c00b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3577260640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c004810 0x7f355c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f35757fd640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c0053d0 0x7f355c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): get_auth_request con 0x7f355c004810 auth_method 0 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0039c0 0x7f355c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): get_auth_request con 0x7f355c0039c0 auth_method 0 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f3574ffc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient(hunting): handle_auth_done global_id 204417 payload 293 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c0053d0 msgr2=0x7f355c00b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c0053d0 0x7f355c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0039c0 msgr2=0x7f355c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0039c0 0x7f355c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 monclient: found mon.a 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 1 -- 172.21.15.17:0/1517879514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c003530 con 0x7f355c004810 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.379+0000 7f356ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356ffff640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c004810 0x7f355c004c90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f355800ef20 tx=0x7f355801d690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d400f02bc333f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580b4070 con 0x7f355c004810 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: dump: 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35580ab050 con 0x7f355c004810 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 1 -- 172.21.15.17:0/1517879514 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580a9030 con 0x7f355c004810 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 4 set_mon_vals no callback set 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f356f7fe640 10 monclient: dump: 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 10 monclient: shutdown 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 -- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c004810 msgr2=0x7f355c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c004810 0x7f355c004c90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f355800ef20 tx=0x7f355801d690 comp rx=0 tx=0).stop 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 -- 172.21.15.17:0/1517879514 shutdown_connections 2024-09-15T21:36:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c0053d0 0x7f355c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c004810 0x7f355c004c90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 --2- 172.21.15.17:0/1517879514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0039c0 0x7f355c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 -- 172.21.15.17:0/1517879514 >> 172.21.15.17:0/1517879514 conn(0x7f35700a66a0 msgr2=0x7f35700a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 -- 172.21.15.17:0/1517879514 shutdown_connections 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.383+0000 7f3577260640 1 -- 172.21.15.17:0/1517879514 wait complete. 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: build_initial_monmap 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: monmap: 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 Processor -- start 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 -- start start 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: init 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 0x7f3570134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): picked mon.noname-a con 0x7f355c0039c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c004810 0x7f3570134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): picked mon.noname-c con 0x7f355c004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0053d0 0x7f357011e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): picked mon.noname-b con 0x7f355c0053d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3570136210 con 0x7f355c0039c0 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3570136090 con 0x7f355c0053d0 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3570136390 con 0x7f355c004810 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient(hunting): authenticate will time out at 2128.786089s 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356ffff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c004810 msgr2=0x7f3570134dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356ffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c004810 0x7f3570134dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 0x7f3570134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 0x7f3570134890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35482/0 (socket says 172.21.15.17:35482) 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 -- 172.21.15.17:0/1497409766 learned_addr learned my addr 172.21.15.17:0/1497409766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): get_auth_request con 0x7f355c0039c0 auth_method 0 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f35757fd640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0053d0 0x7f357011e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f35757fd640 10 monclient(hunting): get_auth_request con 0x7f355c0053d0 auth_method 0 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f35757fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f35757fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f35757fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient(hunting): handle_auth_done global_id 204420 payload 824 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 -- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c004810 msgr2=0x7f3570134dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c004810 0x7f3570134dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 -- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0053d0 msgr2=0x7f357011e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0053d0 0x7f357011e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 monclient: found mon.noname-a 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 -- 172.21.15.17:0/1497409766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357011fef0 con 0x7f355c0039c0 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3574ffc640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 0x7f3570134890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3568011b50 tx=0x7f3568007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680020b0 con 0x7f355c0039c0 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: dump: 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 5 monclient: authenticate success, global_id 204420 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: _renew_subs 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357006f5c0 con 0x7f355c0039c0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: _renew_subs 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3570120a50 con 0x7f355c0039c0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356801b420 con 0x7f355c0039c0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356801b7d0 con 0x7f355c0039c0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356d7fa640 10 monclient: dump: 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.387+0000 7f356effd640 4 set_mon_vals no callback set 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f3577260640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f3577260640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f3577260640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3554005090 con 0x7f355c0039c0 2024-09-15T21:36:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f356801b970 con 0x7f355c0039c0 2024-09-15T21:36:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f356d7fa640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f355003c340 0x7f355003e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3568054b00 con 0x7f355c0039c0 2024-09-15T21:36:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f356ffff640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f355003c340 0x7f355003e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f356ffff640 10 monclient: get_auth_request con 0x7f355003c340 auth_method 0 2024-09-15T21:36:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.391+0000 7f356ffff640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f355003c340 0x7f355003e800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f35580b1000 tx=0x7f3558010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.395+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3568019320 con 0x7f355c0039c0 2024-09-15T21:36:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.395+0000 7f356d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.395+0000 7f356d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.659+0000 7f3577260640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:36:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.659+0000 7f3577260640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.659+0000 7f3577260640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.659+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3554005380 con 0x7f355c0039c0 2024-09-15T21:36:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f356d7fa640 1 -- 172.21.15.17:0/1497409766 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1555 (secure 0 0 0) 0x7f3568024dc0 con 0x7f355c0039c0 2024-09-15T21:36:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f356d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:36:45.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f356d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:45.667 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":174,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:36:45.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f355003c340 msgr2=0x7f355003e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f355003c340 0x7f355003e800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f35580b1000 tx=0x7f3558010040 comp rx=0 tx=0).stop 2024-09-15T21:36:45.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 10 monclient: shutdown 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 msgr2=0x7f3570134890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 0x7f3570134890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3568011b50 tx=0x7f3568007410 comp rx=0 tx=0).stop 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 shutdown_connections 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f355003c340 0x7f355003e800 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f355c0053d0 0x7f357011e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f355c004810 0x7f3570134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 --2- 172.21.15.17:0/1497409766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c0039c0 0x7f3570134890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 >> 172.21.15.17:0/1497409766 conn(0x7f35700a66a0 msgr2=0x7f35701c0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 shutdown_connections 2024-09-15T21:36:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.663+0000 7f3577260640 1 -- 172.21.15.17:0/1497409766 wait complete. 2024-09-15T21:36:45.687 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-15T21:36:45.687 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-15T21:36:45.687 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:36:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.807+0000 7fd6e2da8640 10 monclient: get_monmap_and_config 2024-09-15T21:36:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.807+0000 7fd6e2da8640 10 monclient: build_initial_monmap 2024-09-15T21:36:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.807+0000 7fd6e2da8640 10 monclient: monmap: 2024-09-15T21:36:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.814 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:45.813915+0000 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:45.813915+0000 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:45.815 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 1 Processor -- start 2024-09-15T21:36:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 1 -- start start 2024-09-15T21:36:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient: init 2024-09-15T21:36:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2024-09-15T21:36:45.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd6dc0a28b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 0x7fd6dc09cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient(hunting): picked mon.noname-b con 0x7fd6dc0a33a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd6dc13cc50 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc06cdc0 con 0x7fd6dc13cc50 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc074940 con 0x7fd6dc0a28b0 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc06d040 con 0x7fd6dc0a33a0 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 10 monclient(hunting): authenticate will time out at 2129.209345s 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 msgr2=0x7fd6dc09cb60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 0x7fd6dc09cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e1345640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd6dc13cc50 0x7fd6dc13f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49348/0 (socket says 172.21.15.17:49348) 2024-09-15T21:36:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e1345640 1 -- 172.21.15.17:0/2342988724 learned_addr learned my addr 172.21.15.17:0/2342988724 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707023793 0 0) 0x7fd6dc06cdc0 con 0x7fd6dc13cc50 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): _negotiate 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): my global_id is 204423 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6c4003530 con 0x7fd6dc13cc50 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2121359312 0 0) 0x7fd6c4003530 con 0x7fd6dc13cc50 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 msgr2=0x7fd6dc09cb60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 0x7fd6dc09cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 >> v1:172.21.15.119:6789/0 conn(0x7fd6dc0a28b0 legacy=0x7fd6dc0a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc09c740 con 0x7fd6dc13cc50 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7fd6d4001a80 con 0x7fd6dc13cc50 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 5 monclient: authenticate success, global_id 204423 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: dump: 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.821 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 >> v1:172.21.15.17:6789/0 conn(0x7fd6dc13cc50 legacy=0x7fd6dc13f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c40039f0 0x7fd6c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): picked mon.b con 0x7fd6c40039f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:45.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6c4004810 0x7fd6c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): picked mon.a con 0x7fd6c4004810 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 0x7fd6c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): picked mon.c con 0x7fd6dc0a33a0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc06d040 con 0x7fd6c4004810 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c40056a0 con 0x7fd6c40039f0 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c400bf90 con 0x7fd6dc0a33a0 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6db7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e2da8640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6c4004810 0x7fd6c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e1345640 1 -- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 msgr2=0x7fd6c400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e1345640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 0x7fd6c400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): get_auth_request con 0x7fd6c4004810 auth_method 0 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c40039f0 0x7fd6c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_done global_id 204423 payload 293 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 -- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 msgr2=0x7fd6c400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc0a33a0 0x7fd6c400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 -- 172.21.15.17:0/2342988724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c40039f0 msgr2=0x7fd6c4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c40039f0 0x7fd6c4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 monclient: found mon.a 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 1 -- 172.21.15.17:0/2342988724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c4003530 con 0x7fd6c4004810 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e0b44640 10 monclient: get_auth_request con 0x7fd6c40039f0 auth_method 0 2024-09-15T21:36:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.811+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c40039f0 0x7fd6c4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:45.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6c4004810 0x7fd6c4004c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0003990 tx=0x7fd6d001ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1546b33ad4275788 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0009520 con 0x7fd6c4004810 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: dump: 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0003bc0 con 0x7fd6c4004810 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 4 set_mon_vals no callback set 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 1 -- 172.21.15.17:0/2342988724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00050d0 con 0x7fd6c4004810 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6db7fe640 10 monclient: dump: 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.835 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: shutdown 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6c4004810 msgr2=0x7fd6c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 --2- 172.21.15.17:0/2342988724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6c4004810 0x7fd6c4004c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0003990 tx=0x7fd6d001ce50 comp rx=0 tx=0).stop 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e1345640 1 -- 172.21.15.17:0/2342988724 reap_dead start 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- 172.21.15.17:0/2342988724 shutdown_connections 2024-09-15T21:36:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- 172.21.15.17:0/2342988724 >> 172.21.15.17:0/2342988724 conn(0x7fd6dc0a74a0 msgr2=0x7fd6dc0a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- 172.21.15.17:0/2342988724 shutdown_connections 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- 172.21.15.17:0/2342988724 wait complete. 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: build_initial_monmap 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: monmap: 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:45.837 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 Processor -- start 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- start start 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: init 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4004810 0x7fd6dc13d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): picked mon.noname-b con 0x7fd6c4004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 0x7fd6dc13e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): picked mon.noname-a con 0x7fd6dc13dd70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc13e730 0x7fd6dc1d6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): picked mon.noname-c con 0x7fd6dc13e730 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1410f0 con 0x7fd6dc13dd70 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc140f70 con 0x7fd6c4004810 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc141270 con 0x7fd6dc13e730 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient(hunting): authenticate will time out at 2129.215123s 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e1345640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc13e730 msgr2=0x7fd6dc1d6420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e1345640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc13e730 0x7fd6dc1d6420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 0x7fd6dc13e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 0x7fd6dc13e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35506/0 (socket says 172.21.15.17:35506) 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 -- 172.21.15.17:0/3386023369 learned_addr learned my addr 172.21.15.17:0/3386023369 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): get_auth_request con 0x7fd6dc13dd70 auth_method 0 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4004810 0x7fd6dc13d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): get_auth_request con 0x7fd6c4004810 auth_method 0 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient(hunting): handle_auth_done global_id 204426 payload 824 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc13e730 msgr2=0x7fd6dc1d6420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd6dc13e730 0x7fd6dc1d6420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4004810 msgr2=0x7fd6dc13d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6c4004810 0x7fd6dc13d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 monclient: found mon.noname-a 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc13f7e0 con 0x7fd6dc13dd70 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6dbfff640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 0x7fd6dc13e1f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0097e00 tx=0x7fd6c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00bc070 con 0x7fd6dc13dd70 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 10 monclient: dump: 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 5 monclient: authenticate success, global_id 204426 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: _renew_subs 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0003d70 con 0x7fd6dc13dd70 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.815+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc11d9b0 con 0x7fd6dc13dd70 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0005310 con 0x7fd6dc13dd70 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: dump: 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 10 monclient: _renew_subs 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6da7fc640 4 set_mon_vals no callback set 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc0a0f50 con 0x7fd6dc13dd70 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 0x7fd6dc136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc0a0a90 con 0x7fd6dc11e1b0 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 0x7fd6dc136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd6d0003a30 con 0x7fd6dc13dd70 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient: get_auth_request con 0x7fd6dc11e1b0 auth_method 0 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6c803b9a0 0x7fd6c803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_done global_id 204429 payload 824 2024-09-15T21:36:45.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6dc09e100 con 0x7fd6dc11e1b0 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd6d00e9580 con 0x7fd6dc13dd70 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 0x7fd6dc136de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0107e0 tx=0x7fd6cc010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e1345640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6c803b9a0 0x7fd6c803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0052d0 con 0x7fd6dc11e1b0 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: dump: 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e1345640 10 monclient: get_auth_request con 0x7fd6c803b9a0 auth_method 0 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd6cc0052d0 con 0x7fd6dc11e1b0 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 msgr2=0x7fd6dc136de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 0x7fd6dc136de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0107e0 tx=0x7fd6cc010ca0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 msgr2=0x7fd6dc136de0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6d97fa640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc11e1b0 0x7fd6dc136de0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0107e0 tx=0x7fd6cc010ca0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.819+0000 7fd6e1345640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6c803b9a0 0x7fd6c803de60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0a3780 tx=0x7fd6d4001260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.823+0000 7fd6e2da8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:45.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.823+0000 7fd6e2da8640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 0x7fd6dc137730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.823+0000 7fd6e2da8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.823+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc136970 con 0x7fd6dc137320 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 0x7fd6dc137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient: get_auth_request con 0x7fd6dc137320 auth_method 0 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 10 monclient(hunting): handle_auth_done global_id 204432 payload 824 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 1 -- 172.21.15.17:0/3386023369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd6dc06e030 con 0x7fd6dc137320 2024-09-15T21:36:45.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e0b44640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 0x7fd6dc137730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000fc0 tx=0x7fd6cc001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc01ee50 con 0x7fd6dc137320 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 10 monclient: dump: 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.849 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4871 (secure 0 0 0) 0x7fd6cc005480 con 0x7fd6dc137320 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 msgr2=0x7fd6dc137730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 0x7fd6dc137730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000fc0 tx=0x7fd6cc001130 comp rx=0 tx=0).stop 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 msgr2=0x7fd6dc137730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6d97fa640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc137320 0x7fd6dc137730 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000fc0 tx=0x7fd6cc001130 comp rx=0 tx=0).stop 2024-09-15T21:36:45.850 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 174, 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 130362, 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:45.851 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:45.852 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:45.853 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:45.854 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:45.855 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:45.857 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.859 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:45.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:45.861 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:45.862 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:45.863 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:45.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:45.864 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:45.864 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6c803b9a0 msgr2=0x7fd6c803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd6c803b9a0 0x7fd6c803de60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0a3780 tx=0x7fd6d4001260 comp rx=0 tx=0).stop 2024-09-15T21:36:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 10 monclient: shutdown 2024-09-15T21:36:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 msgr2=0x7fd6dc13e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 0x7fd6dc13e1f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0097e00 tx=0x7fd6c4005840 comp rx=0 tx=0).stop 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e1345640 1 -- 172.21.15.17:0/3386023369 reap_dead start 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 shutdown_connections 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 --2- 172.21.15.17:0/3386023369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6dc13dd70 0x7fd6dc13e1f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 >> 172.21.15.17:0/3386023369 conn(0x7fd6dc0a74a0 msgr2=0x7fd6dc135aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 shutdown_connections 2024-09-15T21:36:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.827+0000 7fd6e2da8640 1 -- 172.21.15.17:0/3386023369 wait complete. 2024-09-15T21:36:45.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:36:45.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-15T21:36:45.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:36:45.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient: get_monmap_and_config 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient: build_initial_monmap 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient: monmap: 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:36:45.983546+0000 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:36:45.983546+0000 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:45.985 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:45.985 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:45.985 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:45.985 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 Processor -- start 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 -- start start 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient: init 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 0x7f26b4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): picked mon.noname-b con 0x7f26b41378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f26b41381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f26b40a5210 0x7f26b40a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): picked mon.noname-c con 0x7f26b40a5210 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b40a1d00 con 0x7f26b41381e0 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f26b40a1fe0 con 0x7f26b40a5210 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f26b40a22c0 con 0x7f26b41378c0 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26bc1b0640 10 monclient(hunting): authenticate will time out at 2129.379628s 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26b9f4c640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 msgr2=0x7f26b4137ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26b9f4c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 0x7f26b4137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26b974b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f26b41381e0 0x7f26b40a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49358/0 (socket says 172.21.15.17:49358) 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.979+0000 7f26b974b640 1 -- 172.21.15.17:0/1360686909 learned_addr learned my addr 172.21.15.17:0/1360686909 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 1 --2- 172.21.15.17:0/1360686909 >> v2:172.21.15.119:3300/0 conn(0x7f26b40a5210 0x7f26b40a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): get_auth_request con 0x7f26b40a5210 auth_method 0 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2439410457 0 0) 0x7f26b40a1d00 con 0x7f26b41381e0 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): _negotiate 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): my global_id is 204435 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a0003530 con 0x7f26b41381e0 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2729344033 0 0) 0x7f26a0003530 con 0x7f26b41381e0 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 msgr2=0x7f26b4137ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 0x7f26b4137ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 >> v2:172.21.15.119:3300/0 conn(0x7f26b40a5210 msgr2=0x7f26b40a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 --2- 172.21.15.17:0/1360686909 >> v2:172.21.15.119:3300/0 conn(0x7f26b40a5210 0x7f26b40a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b40a5c00 con 0x7f26b41381e0 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f26a4001700 con 0x7f26b41381e0 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 5 monclient: authenticate success, global_id 204435 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: dump: 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:45.993 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 >> v1:172.21.15.17:6789/0 conn(0x7f26b41381e0 legacy=0x7f26b40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26a0003a10 0x7f26a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): picked mon.a con 0x7f26a0003a10 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0004810 0x7f26a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): picked mon.b con 0x7f26a0004810 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 0x7f26a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): picked mon.c con 0x7f26b41378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:45.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f26b40a1fe0 con 0x7f26a0003a10 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26b40a22c0 con 0x7f26a0004810 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f26a00055b0 con 0x7f26b41378c0 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 1 -- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 msgr2=0x7f26a000b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 0x7f26a000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26a0003a10 0x7f26a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): get_auth_request con 0x7f26a0003a10 auth_method 0 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0004810 0x7f26a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): get_auth_request con 0x7f26a0004810 auth_method 0 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b974b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:36:45.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_done global_id 204435 payload 293 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 -- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 msgr2=0x7f26a000b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b41378c0 0x7f26a000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 -- 172.21.15.17:0/1360686909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0004810 msgr2=0x7f26a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0004810 0x7f26a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient: _finish_hunting 0 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 monclient: found mon.a 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 -- 172.21.15.17:0/1360686909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a0003530 con 0x7f26a0003a10 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient: _finish_auth 0 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 10 monclient: _check_auth_tickets 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:45.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26a0003a10 0x7f26a0003e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26b0003990 tx=0x7f26b001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ea16ba774f27a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0009520 con 0x7f26a0003a10 2024-09-15T21:36:46.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:46.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: dump: 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b0003bc0 con 0x7f26a0003a10 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:46.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 4 set_mon_vals no callback set 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 1 -- 172.21.15.17:0/1360686909 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0005070 con 0x7f26a0003a10 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26b8f4a640 10 monclient: dump: 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.006 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 10 monclient: shutdown 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 1 -- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26a0003a10 msgr2=0x7f26a0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 1 --2- 172.21.15.17:0/1360686909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26a0003a10 0x7f26a0003e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26b0003990 tx=0x7f26b001c420 comp rx=0 tx=0).stop 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26ba74d640 1 -- 172.21.15.17:0/1360686909 reap_dead start 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 1 -- 172.21.15.17:0/1360686909 shutdown_connections 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 1 -- 172.21.15.17:0/1360686909 >> 172.21.15.17:0/1360686909 conn(0x7f26b40aac40 msgr2=0x7f26b40a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 1 -- 172.21.15.17:0/1360686909 shutdown_connections 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.983+0000 7f26bc1b0640 1 -- 172.21.15.17:0/1360686909 wait complete. 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: build_initial_monmap 2024-09-15T21:36:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: monmap: 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 Processor -- start 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- start start 2024-09-15T21:36:46.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: init 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0003a10 0x7f26b41cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): picked mon.noname-b con 0x7f26a0003a10 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b4135ec0 0x7f26b4136340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): picked mon.noname-c con 0x7f26b4135ec0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 0x7f26b41ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): picked mon.noname-a con 0x7f26b41cc790 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f26b406f340 con 0x7f26b41cc790 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26b406f1c0 con 0x7f26a0003a10 2024-09-15T21:36:46.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f26b41405b0 con 0x7f26b4135ec0 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): authenticate will time out at 2129.385400s 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b974b640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b4135ec0 msgr2=0x7f26b4136340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b974b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b4135ec0 0x7f26b4136340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 0x7f26b41ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 0x7f26b41ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35536/0 (socket says 172.21.15.17:35536) 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 -- 172.21.15.17:0/2179036391 learned_addr learned my addr 172.21.15.17:0/2179036391 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): get_auth_request con 0x7f26b41cc790 auth_method 0 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:46.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0003a10 0x7f26b41cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): get_auth_request con 0x7f26a0003a10 auth_method 0 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient(hunting): handle_auth_done global_id 204438 payload 824 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b4135ec0 msgr2=0x7f26b4136340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f26b4135ec0 0x7f26b4136340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0003a10 msgr2=0x7f26b41cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0003a10 0x7f26b41cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient: _finish_hunting 0 2024-09-15T21:36:46.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 monclient: found mon.noname-a 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b41223b0 con 0x7f26b41cc790 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a0003a10 0x7f26b41cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26ba74d640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 0x7f26b41ccc10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f26b406f5b0 tx=0x7f26ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac018020 con 0x7f26b41cc790 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: dump: 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.012 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: _finish_auth 0 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 5 monclient: authenticate success, global_id 204438 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ac014480 con 0x7f26b41cc790 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:46.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: _renew_subs 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00cea0 con 0x7f26b41cc790 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b4122690 con 0x7f26b41cc790 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 10 monclient: dump: 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26abfff640 4 set_mon_vals no callback set 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: _renew_subs 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b4121170 con 0x7f26b41cc790 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 0x7f26b413dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26b40a8ac0 con 0x7f26b41cd150 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f26ac01b020 con 0x7f26b41cc790 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 0x7f26b413dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient: get_auth_request con 0x7f26b41cd150 auth_method 0 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f269803b9a0 0x7f269803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f26ac051340 con 0x7f26b41cc790 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b974b640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f269803b9a0 0x7f269803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.987+0000 7f26b974b640 10 monclient: get_auth_request con 0x7f269803b9a0 auth_method 0 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_done global_id 224161 payload 824 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26b9f4c640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26b40a8750 con 0x7f26b41cd150 2024-09-15T21:36:46.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 0x7f26b413dd90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26b0006fd0 tx=0x7f26a0005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26b974b640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f269803b9a0 0x7f269803de60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f26b4120c80 tx=0x7f26a4001e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0005040 con 0x7f26b41cd150 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 10 monclient: dump: 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.017 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f26b00b9440 con 0x7f26b41cd150 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 msgr2=0x7f26b413dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 0x7f26b413dd90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26b0006fd0 tx=0x7f26a0005840 comp rx=0 tx=0).stop 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 msgr2=0x7f26b413dd90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.991+0000 7f26aa7fc640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b41cd150 0x7f26b413dd90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26bc1b0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26bc1b0640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 0x7f26b413e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26bc1b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41cd530 con 0x7f26b413e2d0 2024-09-15T21:36:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 0x7f26b413e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26b9f4c640 10 monclient: get_auth_request con 0x7f26b413e2d0 auth_method 0 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26b9f4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26b9f4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.995+0000 7f26b9f4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26b9f4c640 10 monclient(hunting): handle_auth_done global_id 224164 payload 824 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26b9f4c640 1 -- 172.21.15.17:0/2179036391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f26b4069270 con 0x7f26b413e2d0 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26b9f4c640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 0x7f26b413e6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26b001c460 tx=0x7f26b00090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b00b9b40 con 0x7f26b413e2d0 2024-09-15T21:36:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 10 monclient: dump: 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4701 (secure 0 0 0) 0x7f26b00be570 con 0x7f26b413e2d0 2024-09-15T21:36:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 msgr2=0x7f26b413e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 0x7f26b413e6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26b001c460 tx=0x7f26b00090e0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 msgr2=0x7f26b413e6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26aa7fc640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b413e2d0 0x7f26b413e6e0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26b001c460 tx=0x7f26b00090e0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 174, 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 35798, 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:36:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:36:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:36:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:36:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:36:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:36:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:36:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:36:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:36:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:36:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:36:46.034 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:36:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f269803b9a0 msgr2=0x7f269803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f269803b9a0 0x7f269803de60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f26b4120c80 tx=0x7f26a4001e40 comp rx=0 tx=0).stop 2024-09-15T21:36:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 10 monclient: shutdown 2024-09-15T21:36:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 msgr2=0x7f26b41ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 0x7f26b41ccc10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f26b406f5b0 tx=0x7f26ac00c490 comp rx=0 tx=0).stop 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26ba74d640 1 -- 172.21.15.17:0/2179036391 reap_dead start 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 shutdown_connections 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 --2- 172.21.15.17:0/2179036391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b41cc790 0x7f26b41ccc10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 >> 172.21.15.17:0/2179036391 conn(0x7f26b40aac40 msgr2=0x7f26b40a9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 shutdown_connections 2024-09-15T21:36:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:36:45.999+0000 7f26bc1b0640 1 -- 172.21.15.17:0/2179036391 wait complete. 2024-09-15T21:36:46.037 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-15T21:36:46.037 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-15T21:36:46.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): tick 2024-09-15T21:36:46.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-15T21:36:46.346 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 monclient(hunting): continuing hunt 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): picked mon.noname-a con 0x7fa4340106c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7fa43400a800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:46.347 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7fa448063fd0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa44809dc90 con 0x7fa4340106c0 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480a2c00 con 0x7fa448063fd0 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fa44809de10 con 0x7fa43400a800 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44659c640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44659c640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.348 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): get_auth_request con 0x7fa448063fd0 auth_method 0 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): get_auth_request con 0x7fa4340106c0 auth_method 0 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:46.349 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa446d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient(hunting): handle_auth_done global_id 224167 payload 824 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 msgr2=0x7fa42800a280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 msgr2=0x7fa428007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient: _finish_hunting 0 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 monclient: found mon.noname-b 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:36:46.350 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42800a7e0 con 0x7fa448063fd0 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44759e640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa440000f90 tx=0x7fa440014a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44001b030 con 0x7fa448063fd0 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: dump: 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.352 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: _finish_auth 0 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440016690 con 0x7fa448063fd0 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:46.353 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400168c0 con 0x7fa448063fd0 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 5 monclient: authenticate success, global_id 224167 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 monclient: _renew_subs 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa448125920 con 0x7fa448063fd0 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa444d99640 4 set_mon_vals no callback set 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 10 monclient: dump: 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.354 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 monclient: _renew_subs 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4480a0ce0 con 0x7fa448063fd0 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:46.355 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44809e0f0 con 0x7fa448063fd0 2024-09-15T21:36:46.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fa44001e030 con 0x7fa448063fd0 2024-09-15T21:36:46.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.348+0000 7fa42f7fe640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43403d260 0x7fa43403f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:46.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.352+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fa440050640 con 0x7fa448063fd0 2024-09-15T21:36:46.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.352+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43403d260 0x7fa43403f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:46.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.352+0000 7fa446d9d640 10 monclient: get_auth_request con 0x7fa43403d260 auth_method 0 2024-09-15T21:36:46.356 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.352+0000 7fa446d9d640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43403d260 0x7fa43403f720 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa438004660 tx=0x7fa43801b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:46.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.356+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fa440027d10 con 0x7fa448063fd0 2024-09-15T21:36:46.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.356+0000 7fa42f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:46.357 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.356+0000 7fa42f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:46.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.600+0000 7fa44d122640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T21:36:46.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.600+0000 7fa44d122640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T21:36:46.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.600+0000 7fa44d122640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:46.602 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.600+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bar"} v 0) -- 0x7fa448135300 con 0x7fa448063fd0 2024-09-15T21:36:46.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.608+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bar"}]=0 v33) ==== 80+0+0 (secure 0 0 0) 0x7fa440017460 con 0x7fa448063fd0 2024-09-15T21:36:46.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.608+0000 7fa42f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T21:36:46.610 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.608+0000 7fa42f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa44d122640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T21:36:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa44d122640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T21:36:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa44d122640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:46.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bar"} v 0) -- 0x7fa4481255a0 con 0x7fa448063fd0 2024-09-15T21:36:46.772 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa42f7fe640 1 -- 172.21.15.119:0/4150192280 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bar"}]=0 v33) ==== 80+0+0 (secure 0 0 0) 0x7fa440016a60 con 0x7fa448063fd0 2024-09-15T21:36:46.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa42f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T21:36:46.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.772+0000 7fa42f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43403d260 msgr2=0x7fa43403f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43403d260 0x7fa43403f720 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa438004660 tx=0x7fa43801b040 comp rx=0 tx=0).stop 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 10 monclient: shutdown 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 msgr2=0x7fa428005700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa440000f90 tx=0x7fa440014a70 comp rx=0 tx=0).stop 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 shutdown_connections 2024-09-15T21:36:46.775 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fa43403d260 0x7fa43403f720 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa448063fd0 0x7fa428005700 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fa43400a800 0x7fa42800a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 --2- 172.21.15.119:0/4150192280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4340106c0 0x7fa428007c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:46.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 >> 172.21.15.119:0/4150192280 conn(0x7fa4480a7690 msgr2=0x7fa4481380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:46.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 shutdown_connections 2024-09-15T21:36:46.776 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:46.776+0000 7fa44d122640 1 -- 172.21.15.119:0/4150192280 wait complete. 2024-09-15T21:36:46.789 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 13 2024-09-15T21:36:46.789 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 2024-09-15T21:36:46.789 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=13 2024-09-15T21:36:46.789 INFO:tasks.workunit.client.0.smithi119.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T21:36:46.790 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T21:36:46.790 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T21:36:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=13 2024-09-15T21:36:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T21:36:46.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:36:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:+ rm /tmp/cephtest-mon-caps-madness.bazar.keyring /tmp/cephtest-mon-caps-madness.foo.keyring /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T21:36:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:+ cat 2024-09-15T21:36:46.903 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph auth import -i - 2024-09-15T21:36:47.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient: get_monmap_and_config 2024-09-15T21:36:47.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient: build_initial_monmap 2024-09-15T21:36:47.016 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient: monmap: 2024-09-15T21:36:47.016 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:36:47.022215+0000 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:36:47.022215+0000 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:47.017 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 Processor -- start 2024-09-15T21:36:47.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 -- start start 2024-09-15T21:36:47.021 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient: init 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdf80a1890 0x7fbdf809faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): picked mon.noname-c con 0x7fbdf80a1890 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbdf80a1c70 0x7fbdf80a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): picked mon.noname-a con 0x7fbdf80a1c70 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf813c030 0x7fbdf813c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): picked mon.noname-b con 0x7fbdf813c030 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:47.022 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fbdf809ead0 con 0x7fbdf80a1c70 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbdf809f090 con 0x7fbdf80a1890 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf809d4d0 con 0x7fbdf813c030 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfe800640 10 monclient(hunting): authenticate will time out at 2125.818865s 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdf80a1890 0x7fbdf809faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdf80a1890 0x7fbdf809faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41138/0 (socket says 172.21.15.119:41138) 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 -- 172.21.15.119:0/4218158445 learned_addr learned my addr 172.21.15.119:0/4218158445 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): get_auth_request con 0x7fbdf80a1890 auth_method 0 2024-09-15T21:36:47.023 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfcd9d640 1 -- 172.21.15.119:0/4218158445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf813c030 msgr2=0x7fbdf813c480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdfcd9d640 1 --2- 172.21.15.119:0/4218158445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf813c030 0x7fbdf813c480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf77fe640 1 --2- 172.21.15.119:0/4218158445 >> v2:172.21.15.17:3300/0 conn(0x7fbdf80a1c70 0x7fbdf80a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf77fe640 10 monclient(hunting): get_auth_request con 0x7fbdf80a1c70 auth_method 0 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.024 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient(hunting): handle_auth_done global_id 224173 payload 274 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 -- 172.21.15.119:0/4218158445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf813c030 msgr2=0x7fbdf813c480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 --2- 172.21.15.119:0/4218158445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf813c030 0x7fbdf813c480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 -- 172.21.15.119:0/4218158445 >> v2:172.21.15.17:3300/0 conn(0x7fbdf80a1c70 msgr2=0x7fbdf80a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 --2- 172.21.15.119:0/4218158445 >> v2:172.21.15.17:3300/0 conn(0x7fbdf80a1c70 0x7fbdf80a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient: _finish_hunting 0 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 monclient: found mon.noname-c 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.016+0000 7fbdf7fff640 1 -- 172.21.15.119:0/4218158445 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf809e7f0 con 0x7fbdf80a1890 2024-09-15T21:36:47.025 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf7fff640 1 --2- 172.21.15.119:0/4218158445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1890 0x7fbdf809faa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80a6d30 tx=0x7fbdec01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3ffca902c9ba792 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 1 -- 172.21.15.119:0/4218158445 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0b2070 con 0x7fbdf80a1890 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: dump: 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.026 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: _finish_auth 0 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 1 -- 172.21.15.119:0/4218158445 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdec0ad680 con 0x7fbdf80a1890 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 5 monclient: authenticate success, global_id 224173 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 1 -- 172.21.15.119:0/4218158445 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0ada50 con 0x7fbdf80a1890 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 4 set_mon_vals no callback set 2024-09-15T21:36:47.027 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdf6ffd640 10 monclient: dump: 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: shutdown 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:47.028 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 -- 172.21.15.119:0/4218158445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1890 msgr2=0x7fbdf809faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4218158445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1890 0x7fbdf809faa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80a6d30 tx=0x7fbdec01c400 comp rx=0 tx=0).stop 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 -- 172.21.15.119:0/4218158445 shutdown_connections 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4218158445 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf813c030 0x7fbdf813c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4218158445 >> v2:172.21.15.17:3300/0 conn(0x7fbdf80a1c70 0x7fbdf80a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4218158445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1890 0x7fbdf809faa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 -- 172.21.15.119:0/4218158445 >> 172.21.15.119:0/4218158445 conn(0x7fbdf80a6480 msgr2=0x7fbdf80a2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 -- 172.21.15.119:0/4218158445 shutdown_connections 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 -- 172.21.15.119:0/4218158445 wait complete. 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: build_initial_monmap 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: monmap: 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:47.029 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:47.030 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 Processor -- start 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 -- start start 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: init 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.020+0000 7fbdfe800640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf80a1890 0x7fbdf806e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): picked mon.noname-c con 0x7fbdf80a1890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 0x7fbdf806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): picked mon.noname-b con 0x7fbdf80a1c70 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdf813c030 0x7fbdf806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): picked mon.noname-a con 0x7fbdf813c030 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf8145b10 con 0x7fbdf813c030 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80a3430 con 0x7fbdf80a1c70 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf8145c90 con 0x7fbdf80a1890 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient(hunting): authenticate will time out at 2125.824070s 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 0x7fbdf806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 0x7fbdf806ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41146/0 (socket says 172.21.15.119:41146) 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 -- 172.21.15.119:0/4040499202 learned_addr learned my addr 172.21.15.119:0/4040499202 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:47.036 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): get_auth_request con 0x7fbdf80a1c70 auth_method 0 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf7fff640 1 -- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf80a1890 msgr2=0x7fbdf806e750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf7fff640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf80a1890 0x7fbdf806e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfcd9d640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdf813c030 0x7fbdf806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfcd9d640 10 monclient(hunting): get_auth_request con 0x7fbdf813c030 auth_method 0 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfcd9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfcd9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.037 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfcd9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient(hunting): handle_auth_done global_id 224176 payload 824 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 -- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf80a1890 msgr2=0x7fbdf806e750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf80a1890 0x7fbdf806e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 -- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdf813c030 msgr2=0x7fbdf806f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdf813c030 0x7fbdf806f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient: _finish_hunting 0 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 monclient: found mon.noname-b 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 -- 172.21.15.119:0/4040499202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf8072450 con 0x7fbdf80a1c70 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf77fe640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 0x7fbdf806ec90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbde800d800 tx=0x7fbde800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde800b6c0 con 0x7fbdf80a1c70 2024-09-15T21:36:47.038 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: dump: 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.039 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: _finish_auth 0 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 5 monclient: authenticate success, global_id 224176 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde8019070 con 0x7fbdf80a1c70 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient: _renew_subs 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde800ba60 con 0x7fbdf80a1c70 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf809d070 con 0x7fbdf80a1c70 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:47.040 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 10 monclient: dump: 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf67fc640 4 set_mon_vals no callback set 2024-09-15T21:36:47.041 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient: _renew_subs 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf80a5820 con 0x7fbdf80a1c70 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf806c7d0 con 0x7fbdf80a1c70 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fbde800bc00 con 0x7fbdf80a1c70 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbde403b970 0x7fbde403de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fbde8050a90 con 0x7fbdf80a1c70 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf7fff640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbde403b970 0x7fbde403de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf7fff640 10 monclient: get_auth_request con 0x7fbde403b970 auth_method 0 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.024+0000 7fbdf7fff640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbde403b970 0x7fbde403de30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0af000 tx=0x7fbdec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:47.042 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.028+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fbde801cd70 con 0x7fbdf80a1c70 2024-09-15T21:36:47.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.028+0000 7fbdf4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:47.043 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.028+0000 7fbdf4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.244+0000 7fbdfe800640 10 start_mon_command cmd=[{"prefix": "auth import"}] 2024-09-15T21:36:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.244+0000 7fbdfe800640 10 monclient: _send_command 2 [{"prefix": "auth import"}] 2024-09-15T21:36:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.244+0000 7fbdfe800640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.244 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.244+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth import"} v 0) -- 0x7fbdf80a2080 con 0x7fbdf80a1c70 2024-09-15T21:36:47.246 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:36:47.239+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 auth entities have invalid capabilities (AUTH_BAD_CAPS) 2024-09-15T21:36:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.252+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth import"}]=0 v34) ==== 59+0+0 (secure 0 0 0) 0x7fbde8013cb0 con 0x7fbdf80a1c70 2024-09-15T21:36:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.252+0000 7fbdf4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth import"}] 2024-09-15T21:36:47.251 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.252+0000 7fbdf4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:47.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.428+0000 7fbdfe800640 10 start_mon_command cmd=[{"prefix": "auth import"}] 2024-09-15T21:36:47.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.428+0000 7fbdfe800640 10 monclient: _send_command 3 [{"prefix": "auth import"}] 2024-09-15T21:36:47.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.428+0000 7fbdfe800640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.430 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.428+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth import"} v 0) -- 0x7fbdf80a0350 con 0x7fbdf80a1c70 2024-09-15T21:36:47.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.436+0000 7fbdf4ff9640 1 -- 172.21.15.119:0/4040499202 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "auth import"}]=0 v35) ==== 59+0+0 (secure 0 0 0) 0x7fbde8013390 con 0x7fbdf80a1c70 2024-09-15T21:36:47.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.436+0000 7fbdf4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth import"}] 2024-09-15T21:36:47.438 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.436+0000 7fbdf4ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbde403b970 msgr2=0x7fbde403de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbde403b970 0x7fbde403de30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0af000 tx=0x7fbdec008040 comp rx=0 tx=0).stop 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 10 monclient: shutdown 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 msgr2=0x7fbdf806ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 0x7fbdf806ec90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbde800d800 tx=0x7fbde800dcc0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 shutdown_connections 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fbde403b970 0x7fbde403de30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdf813c030 0x7fbdf806f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.441 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf80a1c70 0x7fbdf806ec90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 --2- 172.21.15.119:0/4040499202 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fbdf80a1890 0x7fbdf806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 >> 172.21.15.119:0/4040499202 conn(0x7fbdf80a6480 msgr2=0x7fbdf809f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:47.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 shutdown_connections 2024-09-15T21:36:47.442 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.440+0000 7fbdfe800640 1 -- 172.21.15.119:0/4040499202 wait complete. 2024-09-15T21:36:47.456 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph health 2024-09-15T21:36:47.457 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep AUTH_BAD_CAP 2024-09-15T21:36:47.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient: get_monmap_and_config 2024-09-15T21:36:47.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient: build_initial_monmap 2024-09-15T21:36:47.564 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient: monmap: 2024-09-15T21:36:47.564 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:36:47.570376+0000 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:36:47.570376+0000 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:47.565 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 Processor -- start 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 -- start start 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient: init 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7964135c50 0x7f7964136030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): picked mon.noname-a con 0x7f7964135c50 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:47.568 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f796409fac0 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): picked mon.noname-b con 0x7f79640a1fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796409ee20 con 0x7f796409fac0 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7964074b00 con 0x7f7964135c50 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f796409d750 con 0x7f79640a1fa0 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796c4eb640 10 monclient(hunting): authenticate will time out at 2126.367221s 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796aa88640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 msgr2=0x7f796413d8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796aa88640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969a86640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f796409fac0 0x7f79640a1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51174/0 (socket says 172.21.15.119:51174) 2024-09-15T21:36:47.569 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969a86640 1 -- 172.21.15.119:0/1695518636 learned_addr learned my addr 172.21.15.119:0/1695518636 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 1 --2- 172.21.15.119:0/1695518636 >> v2:172.21.15.17:3300/0 conn(0x7f7964135c50 0x7f7964136030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): get_auth_request con 0x7f7964135c50 auth_method 0 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f796a287640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969285640 1 -- 172.21.15.119:0/1695518636 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451699627 0 0) 0x7f796409ee20 con 0x7f796409fac0 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969285640 10 monclient(hunting): _negotiate 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969285640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969285640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969285640 10 monclient(hunting): my global_id is 224179 2024-09-15T21:36:47.570 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.564+0000 7f7969285640 1 -- 172.21.15.119:0/1695518636 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7950003530 con 0x7f796409fac0 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 10 monclient(hunting): handle_auth_done global_id 204456 payload 274 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 1 -- 172.21.15.119:0/1695518636 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2212519873 0 0) 0x7f7950003530 con 0x7f796409fac0 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 1 -- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 msgr2=0x7f796413d8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 1 --2- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 1 -- 172.21.15.119:0/1695518636 >> v1:172.21.15.119:6789/0 conn(0x7f796409fac0 legacy=0x7f79640a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 10 monclient: _finish_hunting 0 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 1 monclient: found mon.noname-a 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 1 -- 172.21.15.119:0/1695518636 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796409d4d0 con 0x7f7964135c50 2024-09-15T21:36:47.571 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796a287640 1 --2- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7964135c50 0x7f7964136030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79540097c0 tx=0x7f795401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8bd83cd0151e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 1 -- 172.21.15.119:0/1695518636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540b0070 con 0x7f7964135c50 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: dump: 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.576 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: _finish_auth 0 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: _check_auth_tickets 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 5 monclient: authenticate success, global_id 204456 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:47.577 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 1 -- 172.21.15.119:0/1695518636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79540ab400 con 0x7f7964135c50 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 4 set_mon_vals no callback set 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 1 -- 172.21.15.119:0/1695518636 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540a4e60 con 0x7f7964135c50 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f7969285640 10 monclient: dump: 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.578 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 10 monclient: shutdown 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 -- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7964135c50 msgr2=0x7f7964136030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 --2- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7964135c50 0x7f7964136030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79540097c0 tx=0x7f795401c270 comp rx=0 tx=0).stop 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 -- 172.21.15.119:0/1695518636 shutdown_connections 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 --2- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 --2- 172.21.15.119:0/1695518636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7964135c50 0x7f7964136030 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 -- 172.21.15.119:0/1695518636 >> 172.21.15.119:0/1695518636 conn(0x7f79640a6480 msgr2=0x7f79640a2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 -- 172.21.15.119:0/1695518636 shutdown_connections 2024-09-15T21:36:47.579 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.568+0000 7f796c4eb640 1 -- 172.21.15.119:0/1695518636 wait complete. 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: build_initial_monmap 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: monmap: 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 Processor -- start 2024-09-15T21:36:47.580 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- start start 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: init 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f796409fac0 0x7f79641cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): picked mon.noname-a con 0x7f796409fac0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): picked mon.noname-c con 0x7f79640a1fa0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 0x7f796413ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): picked mon.noname-b con 0x7f7964135c50 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7964140620 con 0x7f796409fac0 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79640a0010 con 0x7f7964135c50 2024-09-15T21:36:47.581 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f79641407a0 con 0x7f79640a1fa0 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient(hunting): authenticate will time out at 2126.373112s 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f7969a86640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 msgr2=0x7f796413c990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 0x7f796413ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f7969a86640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 0x7f796413ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41160/0 (socket says 172.21.15.119:41160) 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 -- 172.21.15.119:0/4146371564 learned_addr learned my addr 172.21.15.119:0/4146371564 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): get_auth_request con 0x7f7964135c50 auth_method 0 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:47.582 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f796409fac0 0x7f79641cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): get_auth_request con 0x7f796409fac0 auth_method 0 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient(hunting): handle_auth_done global_id 224182 payload 824 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 -- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 msgr2=0x7f796413c990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 -- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f796409fac0 msgr2=0x7f79641cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.583 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f796409fac0 0x7f79641cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient: _finish_hunting 0 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 monclient: found mon.noname-b 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 -- 172.21.15.119:0/4146371564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7964139100 con 0x7f7964135c50 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796aa88640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 0x7f796413ced0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f795c00b760 tx=0x7f795c00bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c00c8a0 con 0x7f7964135c50 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: dump: 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.584 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: _finish_auth 0 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: _check_auth_tickets 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c010070 con 0x7f7964135c50 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c00cd00 con 0x7f7964135c50 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:47.585 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 10 monclient: dump: 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 5 monclient: authenticate success, global_id 224182 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _renew_subs 2024-09-15T21:36:47.586 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f7968a84640 4 set_mon_vals no callback set 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796409d070 con 0x7f7964135c50 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _renew_subs 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796413b460 con 0x7f7964135c50 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f795c015470 con 0x7f7964135c50 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964136030 con 0x7f7964135c50 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f794803b9a0 0x7f794803de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f795c050590 con 0x7f7964135c50 2024-09-15T21:36:47.587 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f794803b9a0 0x7f794803de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:47.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 10 monclient: get_auth_request con 0x7f794803b9a0 auth_method 0 2024-09-15T21:36:47.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.572+0000 7f796a287640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f794803b9a0 0x7f794803de60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f79540ad000 tx=0x7f79540a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:47.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.576+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f795c01f3f0 con 0x7f7964135c50 2024-09-15T21:36:47.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.576+0000 7f795affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:47.588 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.576+0000 7f795affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:47.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.768+0000 7f796c4eb640 10 start_mon_command cmd=[{"prefix": "health"}] 2024-09-15T21:36:47.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.768+0000 7f796c4eb640 10 monclient: _send_command 2 [{"prefix": "health"}] 2024-09-15T21:36:47.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.768+0000 7f796c4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.770 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.768+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health"} v 0) -- 0x7f79640a0350 con 0x7f7964135c50 2024-09-15T21:36:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.772+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health"}]=0 v0) ==== 54+0+126 (secure 0 0 0) 0x7f795c013c90 con 0x7f7964135c50 2024-09-15T21:36:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.772+0000 7f795affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health"}] 2024-09-15T21:36:47.771 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.772+0000 7f795affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 10 start_mon_command cmd=[{"prefix": "health"}] 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 10 monclient: _send_command 3 [{"prefix": "health"}] 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health"} v 0) -- 0x7f7964139c60 con 0x7f7964135c50 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f795affd640 1 -- 172.21.15.119:0/4146371564 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "health"}]=0 v0) ==== 54+0+126 (secure 0 0 0) 0x7f795c0500d0 con 0x7f7964135c50 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f795affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "health"}] 2024-09-15T21:36:47.908 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f795affd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f794803b9a0 msgr2=0x7f794803de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f794803b9a0 0x7f794803de60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f79540ad000 tx=0x7f79540a6000 comp rx=0 tx=0).stop 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 10 monclient: shutdown 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 msgr2=0x7f796413ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 0x7f796413ced0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f795c00b760 tx=0x7f795c00bc20 comp rx=0 tx=0).stop 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 shutdown_connections 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f794803b9a0 0x7f794803de60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7964135c50 0x7f796413ced0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f79640a1fa0 0x7f796413c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 --2- 172.21.15.119:0/4146371564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f796409fac0 0x7f79641cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:47.911 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.908+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 >> 172.21.15.119:0/4146371564 conn(0x7f79640a6480 msgr2=0x7f79640725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:47.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.912+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 shutdown_connections 2024-09-15T21:36:47.912 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:47.912+0000 7f796c4eb640 1 -- 172.21.15.119:0/4146371564 wait complete. 2024-09-15T21:36:47.927 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph health detail 2024-09-15T21:36:47.927 INFO:tasks.workunit.client.0.smithi119.stderr:+ grep client.bad 2024-09-15T21:36:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.032+0000 7fe122ecc640 10 monclient: get_monmap_and_config 2024-09-15T21:36:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.032+0000 7fe122ecc640 10 monclient: build_initial_monmap 2024-09-15T21:36:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.032+0000 7fe122ecc640 10 monclient: monmap: 2024-09-15T21:36:48.034 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:48.035 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:36:48.040545+0000 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:36:48.040545+0000 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:48.037 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.044 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 Processor -- start 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 -- start start 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient: init 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe11c137cb0 0x7fe11c0a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): picked mon.noname-c con 0x7fe11c137cb0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe11c138090 0x7fe11c0a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): picked mon.noname-a con 0x7fe11c138090 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe11c138500 addr v1:172.21.15.119:6789/0 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe11c09dca0 con 0x7fe11c138500 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe11c0a4cb0 con 0x7fe11c138090 2024-09-15T21:36:48.045 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe11c0a4f90 con 0x7fe11c137cb0 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe122ecc640 10 monclient(hunting): authenticate will time out at 2126.837475s 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe11c137cb0 0x7fe11c0a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe11c137cb0 0x7fe11c0a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41164/0 (socket says 172.21.15.119:41164) 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 -- 172.21.15.119:0/977717014 learned_addr learned my addr 172.21.15.119:0/977717014 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): get_auth_request con 0x7fe11c137cb0 auth_method 0 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:48.046 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe11bfff640 1 --2- 172.21.15.119:0/977717014 >> v2:172.21.15.17:3300/0 conn(0x7fe11c138090 0x7fe11c0a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe11bfff640 10 monclient(hunting): get_auth_request con 0x7fe11c138090 auth_method 0 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe11bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe11bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe11bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient(hunting): handle_auth_done global_id 224185 payload 274 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 -- 172.21.15.119:0/977717014 >> v2:172.21.15.17:3300/0 conn(0x7fe11c138090 msgr2=0x7fe11c0a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 --2- 172.21.15.119:0/977717014 >> v2:172.21.15.17:3300/0 conn(0x7fe11c138090 0x7fe11c0a7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 -- 172.21.15.119:0/977717014 >> v1:172.21.15.119:6789/0 conn(0x7fe11c138500 legacy=0x7fe11c0a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient: _finish_hunting 0 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 monclient: found mon.noname-c 2024-09-15T21:36:48.047 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.036+0000 7fe120c68640 1 -- 172.21.15.119:0/977717014 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c0a5270 con 0x7fe11c137cb0 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 --2- 172.21.15.119:0/977717014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c0a5ca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe11c09d110 tx=0x7fe10c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3a5b82afb497a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 1 -- 172.21.15.119:0/977717014 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0b0070 con 0x7fe11c137cb0 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: dump: 2024-09-15T21:36:48.048 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 5 monclient: authenticate success, global_id 224185 2024-09-15T21:36:48.049 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 1 -- 172.21.15.119:0/977717014 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe10c0ab600 con 0x7fe11c137cb0 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 4 set_mon_vals no callback set 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 1 -- 172.21.15.119:0/977717014 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0ab9d0 con 0x7fe11c137cb0 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11b7fe640 10 monclient: dump: 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.050 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: shutdown 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/977717014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 msgr2=0x7fe11c0a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 --2- 172.21.15.119:0/977717014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c0a5ca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe11c09d110 tx=0x7fe10c01c3e0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/977717014 shutdown_connections 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 --2- 172.21.15.119:0/977717014 >> v2:172.21.15.17:3300/0 conn(0x7fe11c138090 0x7fe11c0a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 --2- 172.21.15.119:0/977717014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c0a5ca0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.051 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/977717014 >> 172.21.15.119:0/977717014 conn(0x7fe11c0aa2e0 msgr2=0x7fe11c0aa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/977717014 shutdown_connections 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/977717014 wait complete. 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: build_initial_monmap 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: monmap: 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:48.052 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 Processor -- start 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- start start 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: init 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c1cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): picked mon.noname-b con 0x7fe11c137cb0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe11c138090 0x7fe11c13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): picked mon.noname-c con 0x7fe11c138090 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe11c138500 0x7fe11c13f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): picked mon.noname-a con 0x7fe11c138500 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:48.053 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c0a8ce0 con 0x7fe11c138500 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c0a8b60 con 0x7fe11c137cb0 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe11c1423c0 con 0x7fe11c138090 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient(hunting): authenticate will time out at 2126.842363s 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11bfff640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe11c138090 msgr2=0x7fe11c13eba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c1cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe11c138090 0x7fe11c13eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c1cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41172/0 (socket says 172.21.15.119:41172) 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 -- 172.21.15.119:0/3438141081 learned_addr learned my addr 172.21.15.119:0/3438141081 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): get_auth_request con 0x7fe11c137cb0 auth_method 0 2024-09-15T21:36:48.054 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient(hunting): handle_auth_done global_id 224188 payload 824 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 -- 172.21.15.119:0/3438141081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe11c138090 msgr2=0x7fe11c13eba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe11c138090 0x7fe11c13eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 -- 172.21.15.119:0/3438141081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe11c138500 msgr2=0x7fe11c13f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe11c138500 0x7fe11c13f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient: _finish_hunting 0 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 monclient: found mon.noname-b 2024-09-15T21:36:48.055 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 -- 172.21.15.119:0/3438141081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c06d270 con 0x7fe11c137cb0 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe120c68640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c1cf500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe10c018ff0 tx=0x7fe10c019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0b9070 con 0x7fe11c137cb0 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: dump: 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.056 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: _finish_auth 0 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: _check_auth_tickets 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe10c0b4400 con 0x7fe11c137cb0 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 5 monclient: authenticate success, global_id 224188 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _renew_subs 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.057 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c13f620 con 0x7fe11c137cb0 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0b0040 con 0x7fe11c137cb0 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe1197fa640 10 monclient: dump: 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.058 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _renew_subs 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c13d670 con 0x7fe11c137cb0 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe11affd640 4 set_mon_vals no callback set 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.040+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11c0a5ca0 con 0x7fe11c137cb0 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.044+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe10c0bc070 con 0x7fe11c137cb0 2024-09-15T21:36:48.059 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.044+0000 7fe1197fa640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe10803b970 0x7fe10803de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.044+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe10c0e9900 con 0x7fe11c137cb0 2024-09-15T21:36:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.044+0000 7fe11bfff640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe10803b970 0x7fe10803de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.044+0000 7fe11bfff640 10 monclient: get_auth_request con 0x7fe10803b970 auth_method 0 2024-09-15T21:36:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.044+0000 7fe11bfff640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe10803b970 0x7fe10803de30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe110006e90 tx=0x7fe110006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:48.060 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.048+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe11c06d270 con 0x7fe11c137cb0 2024-09-15T21:36:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.048+0000 7fe1197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:48.061 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.048+0000 7fe1197fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:48.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe122ecc640 10 start_mon_command cmd=[{"prefix": "health", "detail": "detail"}] 2024-09-15T21:36:48.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe122ecc640 10 monclient: _send_command 2 [{"prefix": "health", "detail": "detail"}] 2024-09-15T21:36:48.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe122ecc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.261 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe11c13ce30 con 0x7fe11c137cb0 2024-09-15T21:36:48.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1206 (secure 0 0 0) 0x7fe10c0160f0 con 0x7fe11c137cb0 2024-09-15T21:36:48.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe1197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "detail": "detail"}] 2024-09-15T21:36:48.262 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.260+0000 7fe1197fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe122ecc640 10 start_mon_command cmd=[{"prefix": "health", "detail": "detail"}] 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe122ecc640 10 monclient: _send_command 3 [{"prefix": "health", "detail": "detail"}] 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe122ecc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe11c0008e0 con 0x7fe11c137cb0 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe1197fa640 1 -- 172.21.15.119:0/3438141081 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1206 (secure 0 0 0) 0x7fe10c0bdb60 con 0x7fe11c137cb0 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe1197fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "health", "detail": "detail"}] 2024-09-15T21:36:48.413 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.412+0000 7fe1197fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:36:48.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe10803b970 msgr2=0x7fe10803de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe10803b970 0x7fe10803de30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe110006e90 tx=0x7fe110006e20 comp rx=0 tx=0).stop 2024-09-15T21:36:48.415 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 10 monclient: shutdown 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 msgr2=0x7fe11c1cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c1cf500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe10c018ff0 tx=0x7fe10c019180 comp rx=0 tx=0).stop 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 shutdown_connections 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe10803b970 0x7fe10803de30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe11c138500 0x7fe11c13f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe11c138090 0x7fe11c13eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 --2- 172.21.15.119:0/3438141081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe11c137cb0 0x7fe11c1cf500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 >> 172.21.15.119:0/3438141081 conn(0x7fe11c0aa2e0 msgr2=0x7fe11c0a34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 shutdown_connections 2024-09-15T21:36:48.416 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.416+0000 7fe122ecc640 1 -- 172.21.15.119:0/3438141081 wait complete. 2024-09-15T21:36:48.428 INFO:tasks.workunit.client.0.smithi119.stdout: client.bad mds capability parse failed, stopped at 'also does not parse' of 'also does not parse' 2024-09-15T21:36:48.429 INFO:tasks.workunit.client.0.smithi119.stdout: client.bad mon capability parse failed, stopped at 'this is wrong' of 'this is wrong' 2024-09-15T21:36:48.429 INFO:tasks.workunit.client.0.smithi119.stdout: client.bad osd capability parse failed, stopped at 'does not parse' of 'does not parse' 2024-09-15T21:36:48.429 INFO:tasks.workunit.client.0.smithi119.stdout: client.bad mds capability parse failed, stopped at 'also does not parse' of 'also does not parse' 2024-09-15T21:36:48.429 INFO:tasks.workunit.client.0.smithi119.stdout: client.bad mon capability parse failed, stopped at 'this is wrong' of 'this is wrong' 2024-09-15T21:36:48.429 INFO:tasks.workunit.client.0.smithi119.stdout: client.bad osd capability parse failed, stopped at 'does not parse' of 'does not parse' 2024-09-15T21:36:48.429 INFO:tasks.workunit.client.0.smithi119.stderr:+ ceph auth rm client.bad 2024-09-15T21:36:48.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient: get_monmap_and_config 2024-09-15T21:36:48.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient: build_initial_monmap 2024-09-15T21:36:48.535 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient: monmap: 2024-09-15T21:36:48.535 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:48.535 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.535 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:36:48.541330+0000 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:36:48.541330+0000 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:36:48.536 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 Processor -- start 2024-09-15T21:36:48.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 -- start start 2024-09-15T21:36:48.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient: init 2024-09-15T21:36:48.538 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd19c09c9f0 0x7fd19c0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): picked mon.noname-c con 0x7fd19c09c9f0 addr v2:172.21.15.119:3300/0 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c09cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): picked mon.noname-b con 0x7fd19c0a90e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd19c09d310 0x7fd19c1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): picked mon.noname-a con 0x7fd19c09d310 addr v2:172.21.15.17:3300/0 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c0a4fa0 con 0x7fd19c09d310 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c0a9510 con 0x7fd19c09c9f0 2024-09-15T21:36:48.539 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c0a8c00 con 0x7fd19c0a90e0 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a436b640 10 monclient(hunting): authenticate will time out at 2127.337925s 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd19c09c9f0 0x7fd19c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a1906640 1 -- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 msgr2=0x7fd19c09cdd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd19c09c9f0 0x7fd19c0a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41188/0 (socket says 172.21.15.119:41188) 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 -- 172.21.15.119:0/580724526 learned_addr learned my addr 172.21.15.119:0/580724526 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): get_auth_request con 0x7fd19c09c9f0 auth_method 0 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a1906640 1 --2- 172.21.15.119:0/580724526 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c09cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:48.540 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2908640 1 --2- 172.21.15.119:0/580724526 >> v2:172.21.15.17:3300/0 conn(0x7fd19c09d310 0x7fd19c1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2908640 10 monclient(hunting): get_auth_request con 0x7fd19c09d310 auth_method 0 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2908640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2908640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2908640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient(hunting): handle_auth_done global_id 224191 payload 274 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 -- 172.21.15.119:0/580724526 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 msgr2=0x7fd19c09cdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 --2- 172.21.15.119:0/580724526 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c09cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 -- 172.21.15.119:0/580724526 >> v2:172.21.15.17:3300/0 conn(0x7fd19c09d310 msgr2=0x7fd19c1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 --2- 172.21.15.119:0/580724526 >> v2:172.21.15.17:3300/0 conn(0x7fd19c09d310 0x7fd19c1379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient: _finish_hunting 0 2024-09-15T21:36:48.541 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 monclient: found mon.noname-c 2024-09-15T21:36:48.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:48.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.542 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.536+0000 7fd1a2107640 1 -- 172.21.15.119:0/580724526 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c0a4cc0 con 0x7fd19c09c9f0 2024-09-15T21:36:48.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a2107640 1 --2- 172.21.15.119:0/580724526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09c9f0 0x7fd19c0a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd19c09dfd0 tx=0x7fd19801c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae58a131c05031a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:48.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 1 -- 172.21.15.119:0/580724526 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1980a6040 con 0x7fd19c09c9f0 2024-09-15T21:36:48.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.546 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: dump: 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: _finish_auth 0 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: _check_auth_tickets 2024-09-15T21:36:48.547 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 5 monclient: authenticate success, global_id 224191 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 1 -- 172.21.15.119:0/580724526 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1980af070 con 0x7fd19c09c9f0 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: get_monmap_and_config success 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 1 -- 172.21.15.119:0/580724526 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1980aa660 con 0x7fd19c09c9f0 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 4 set_mon_vals no callback set 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:48.548 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1105640 10 monclient: dump: 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.549 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.550 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: shutdown 2024-09-15T21:36:48.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:48.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/580724526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09c9f0 msgr2=0x7fd19c0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- 172.21.15.119:0/580724526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09c9f0 0x7fd19c0a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd19c09dfd0 tx=0x7fd19801c610 comp rx=0 tx=0).stop 2024-09-15T21:36:48.550 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/580724526 shutdown_connections 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- 172.21.15.119:0/580724526 >> v2:172.21.15.17:3300/0 conn(0x7fd19c09d310 0x7fd19c1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- 172.21.15.119:0/580724526 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c09cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- 172.21.15.119:0/580724526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09c9f0 0x7fd19c0a56a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/580724526 >> 172.21.15.119:0/580724526 conn(0x7fd19c0aa220 msgr2=0x7fd19c0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/580724526 shutdown_connections 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/580724526 wait complete. 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: build_initial_monmap 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: monmap: 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 0 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 0.000000 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:created 0.000000 2024-09-15T21:36:48.551 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 0 (unknown) 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 Processor -- start 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- start start 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: init 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: _reopen_session rank -1 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd19c09c9f0 0x7fd19c1d0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): picked mon.noname-a con 0x7fd19c09c9f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:36:48.552 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 0x7fd19c136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): picked mon.noname-b con 0x7fd19c09d310 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c13d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): picked mon.noname-c con 0x7fd19c0a90e0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1417e0 con 0x7fd19c09c9f0 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c141660 con 0x7fd19c09d310 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): start opening mon connection 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c141960 con 0x7fd19c0a90e0 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): _renew_subs 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient(hunting): authenticate will time out at 2127.342753s 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 0x7fd19c136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 0x7fd19c136f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41194/0 (socket says 172.21.15.119:41194) 2024-09-15T21:36:48.553 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 -- 172.21.15.119:0/40232529 learned_addr learned my addr 172.21.15.119:0/40232529 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): get_auth_request con 0x7fd19c09d310 auth_method 0 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a2908640 1 -- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 msgr2=0x7fd19c13d0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3301/0 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a2908640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c13d0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient(hunting): handle_auth_done global_id 224194 payload 824 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a2107640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd19c09c9f0 0x7fd19c1d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 -- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 msgr2=0x7fd19c13d0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:36:48.554 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c13d0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 -- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd19c09c9f0 msgr2=0x7fd19c1d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd19c09c9f0 0x7fd19c1d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient: _finish_hunting 0 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 monclient: found mon.noname-b 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 -- 172.21.15.119:0/40232529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c122930 con 0x7fd19c09d310 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a2107640 10 monclient: get_auth_request con 0x7fd19c09c9f0 auth_method 0 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a2107640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd19c09c9f0 0x7fd19c1d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a1906640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 0x7fd19c136f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b530 tx=0x7fd18c00b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c013020 con 0x7fd19c09d310 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:36:48.555 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: dump: 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: _finish_auth 0 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:36:48.556 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18c003df0 con 0x7fd19c09d310 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 5 monclient: authenticate success, global_id 224194 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: _renew_subs 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c0a77c0 con 0x7fd19c09d310 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0052b0 con 0x7fd19c09d310 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:36:48.557 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a0904640 4 set_mon_vals no callback set 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd192ffd640 10 monclient: dump: 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:epoch 1 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:min_mon_release 19 (squid) 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:election_strategy: 1 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: _renew_subs 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.558 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.540+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c13eb40 con 0x7fd19c09d310 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a436b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a436b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a436b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c1379e0 con 0x7fd19c09d310 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fd18c016020 con 0x7fd19c09d310 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd192ffd640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd18003b9a0 0x7fd18003de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fd18c050260 con 0x7fd19c09d310 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a2107640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd18003b9a0 0x7fd18003de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a2107640 10 monclient: get_auth_request con 0x7fd18003b9a0 auth_method 0 2024-09-15T21:36:48.559 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.544+0000 7fd1a2107640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd18003b9a0 0x7fd18003de60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd1980ae000 tx=0x7fd1980a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:36:48.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.548+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fd18c01d020 con 0x7fd19c09d310 2024-09-15T21:36:48.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.548+0000 7fd192ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:36:48.560 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.548+0000 7fd192ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:36:48.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.764+0000 7fd1a436b640 10 start_mon_command cmd=[{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T21:36:48.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.764+0000 7fd1a436b640 10 monclient: _send_command 2 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T21:36:48.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.764+0000 7fd1a436b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.765 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.764+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.bad"} v 0) -- 0x7fd19c121130 con 0x7fd19c09d310 2024-09-15T21:36:48.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.772+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.bad"}]=0 v36) ==== 79+0+0 (secure 0 0 0) 0x7fd18c00d380 con 0x7fd19c09d310 2024-09-15T21:36:48.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.772+0000 7fd192ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T21:36:48.773 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.772+0000 7fd192ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:36:48.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.932+0000 7fd1a436b640 10 start_mon_command cmd=[{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T21:36:48.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.932+0000 7fd1a436b640 10 monclient: _send_command 3 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T21:36:48.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.119:3300/0 2024-09-15T21:36:48.935 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.bad"} v 0) -- 0x7fd19c0008e0 con 0x7fd19c09d310 2024-09-15T21:36:48.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd192ffd640 1 -- 172.21.15.119:0/40232529 <== mon.1 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.bad"}]=0 v36) ==== 79+0+0 (secure 0 0 0) 0x7fd18c00fdf0 con 0x7fd19c09d310 2024-09-15T21:36:48.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd192ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T21:36:48.936 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd192ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T21:36:48.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd18003b9a0 msgr2=0x7fd18003de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd18003b9a0 0x7fd18003de60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd1980ae000 tx=0x7fd1980a7000 comp rx=0 tx=0).stop 2024-09-15T21:36:48.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 10 monclient: shutdown 2024-09-15T21:36:48.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:36:48.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 msgr2=0x7fd19c136f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:36:48.939 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.936+0000 7fd1a436b640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 0x7fd19c136f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b530 tx=0x7fd18c00b9f0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 shutdown_connections 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fd18003b9a0 0x7fd18003de60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fd19c0a90e0 0x7fd19c13d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd19c09d310 0x7fd19c136f70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 --2- 172.21.15.119:0/40232529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd19c09c9f0 0x7fd19c1d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 >> 172.21.15.119:0/40232529 conn(0x7fd19c0aa220 msgr2=0x7fd19c0748a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 shutdown_connections 2024-09-15T21:36:48.940 INFO:tasks.workunit.client.0.smithi119.stderr:2024-09-15T21:36:48.940+0000 7fd1a436b640 1 -- 172.21.15.119:0/40232529 wait complete. 2024-09-15T21:36:48.954 INFO:tasks.workunit.client.0.smithi119.stdout:ceph auth health | grep AUTH_BAD_CAP 2024-09-15T21:36:48.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect 'ceph auth health | grep AUTH_BAD_CAP' 1 2024-09-15T21:36:48.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ cmd='ceph auth health | grep AUTH_BAD_CAP' 2024-09-15T21:36:48.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ expected_ret=1 2024-09-15T21:36:48.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo ceph auth health '|' grep AUTH_BAD_CAP 2024-09-15T21:36:48.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ eval ceph auth health '|' grep AUTH_BAD_CAP 2024-09-15T21:36:49.301 INFO:tasks.workunit.client.0.smithi119.stderr:+ ret=1 2024-09-15T21:36:49.301 INFO:tasks.workunit.client.0.smithi119.stderr:+ [[ 1 -ne 1 ]] 2024-09-15T21:36:49.301 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-09-15T21:36:49.302 INFO:tasks.workunit.client.0.smithi119.stdout:OK 2024-09-15T21:36:49.302 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo OK 2024-09-15T21:36:49.303 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:36:49.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:36:49.358 INFO:tasks.workunit:Stopping ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh'] on client.0... 2024-09-15T21:36:49.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T21:36:49.700 DEBUG:teuthology.parallel:result is None 2024-09-15T21:36:49.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:36:49.710 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:36:49.710 DEBUG:teuthology.orchestra.run.smithi119:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:36:49.776 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:36:49.776 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-15T21:36:49.785 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-15T21:37:06.038 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T21:37:06.038 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:37:06.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:37:06.042 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:37:06.042 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T21:37:06.042 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T21:37:06.042 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs to unfreeze mons 2024-09-15T21:37:26.043 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T21:37:26.044 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T21:37:26.044 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:37:26.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient: get_monmap_and_config 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient: monmap: 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:26.180373+0000 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:26.180373+0000 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.181 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:26.182 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:26.182 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:26.182 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:26.182 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:26.182 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 Processor -- start 2024-09-15T21:37:26.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 -- start start 2024-09-15T21:37:26.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient: init 2024-09-15T21:37:26.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 0x7fe830137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): picked mon.noname-b con 0x7fe8301378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe8301381e0 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe8300a5210 0x7fe8300a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): picked mon.noname-c con 0x7fe8300a5210 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8300a1d00 con 0x7fe8301381e0 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe8300a1fe0 con 0x7fe8300a5210 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8300a22c0 con 0x7fe8301378c0 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.174+0000 7fe837698640 10 monclient(hunting): authenticate will time out at 2169.575818s 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8301381e0 0x7fe8300a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43506/0 (socket says 172.21.15.17:43506) 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 -- 172.21.15.17:0/1504251595 learned_addr learned my addr 172.21.15.17:0/1504251595 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 0x7fe830137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): get_auth_request con 0x7fe8301378c0 auth_method 0 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 1 --2- 172.21.15.17:0/1504251595 >> v2:172.21.15.119:3300/0 conn(0x7fe8300a5210 0x7fe8300a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): get_auth_request con 0x7fe8300a5210 auth_method 0 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1110897370 0 0) 0x7fe8300a1d00 con 0x7fe8301381e0 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): _negotiate 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): my global_id is 204477 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe818003530 con 0x7fe8301381e0 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835c35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient(hunting): handle_auth_done global_id 234101 payload 274 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 -- 172.21.15.17:0/1504251595 >> v2:172.21.15.119:3300/0 conn(0x7fe8300a5210 msgr2=0x7fe8300a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 --2- 172.21.15.17:0/1504251595 >> v2:172.21.15.119:3300/0 conn(0x7fe8300a5210 0x7fe8300a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 -- 172.21.15.17:0/1504251595 >> v1:172.21.15.17:6789/0 conn(0x7fe8301381e0 legacy=0x7fe8300a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 monclient: found mon.noname-b 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 -- 172.21.15.17:0/1504251595 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8300a5c00 con 0x7fe8301378c0 2024-09-15T21:37:26.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 0x7fe830137ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097c0 tx=0x7fe82001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f54c7f1b0d46ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 -- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 msgr2=0x7fe830137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 -- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 msgr2=0x7fe830137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 0x7fe830137ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097c0 tx=0x7fe82001c360 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe835434640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 0x7fe830137ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097c0 tx=0x7fe82001c360 comp rx=0 tx=0).stop 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 msgr2=0x7fe830137ca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe8301378c0 0x7fe830137ca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: _add_conns ranks=[1,3] 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe818003810 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 --2- 172.21.15.17:0/1504251595 >> v2:172.21.15.17:3300/0 conn(0x7fe8180079b0 0x7fe818009da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): picked mon.noname-a con 0x7fe8180079b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8300a1d00 con 0x7fe818003810 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe81800b5c0 con 0x7fe8180079b0 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 --2- 172.21.15.17:0/1504251595 >> v2:172.21.15.17:3300/0 conn(0x7fe8180079b0 0x7fe818009da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): get_auth_request con 0x7fe8180079b0 auth_method 0 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient(hunting): handle_auth_done global_id 234101 payload 293 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 -- 172.21.15.17:0/1504251595 >> v1:172.21.15.119:6789/0 conn(0x7fe818003810 legacy=0x7fe818003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 monclient: found mon.noname-a 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 -- 172.21.15.17:0/1504251595 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81800b830 con 0x7fe8180079b0 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe834c33640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe818009da0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8240019d0 tx=0x7fe820009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37aee37c972497b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8240016a0 con 0x7fe8180079b0 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: dump: 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe837698640 5 monclient: authenticate success, global_id 234101 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe837698640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe824004ab0 con 0x7fe8180079b0 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 1 -- 172.21.15.17:0/1504251595 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824004e60 con 0x7fe8180079b0 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe837698640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe837698640 4 set_mon_vals no callback set 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.178+0000 7fe82ffff640 10 monclient: dump: 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.202 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: shutdown 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 msgr2=0x7fe818009da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe818009da0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8240019d0 tx=0x7fe820009190 comp rx=0 tx=0).stop 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 1 -- 172.21.15.17:0/1504251595 reap_dead start 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/1504251595 shutdown_connections 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 --2- 172.21.15.17:0/1504251595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe818009da0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/1504251595 >> 172.21.15.17:0/1504251595 conn(0x7fe8300aac40 msgr2=0x7fe8300a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/1504251595 shutdown_connections 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/1504251595 wait complete. 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: monmap: 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.204 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 Processor -- start 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- start start 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: init 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe8301cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): picked mon.noname-a con 0x7fe8180079b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 0x7fe83013cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): picked mon.noname-c con 0x7fe830136ab0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe830136f30 0x7fe83013d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): picked mon.noname-b con 0x7fe830136f30 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe83006f340 con 0x7fe8180079b0 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe83006f1c0 con 0x7fe830136f30 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe8301425c0 con 0x7fe830136ab0 2024-09-15T21:37:26.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient(hunting): authenticate will time out at 2169.582207s 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 0x7fe83013cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe8301cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 0x7fe83013cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60752/0 (socket says 172.21.15.17:60752) 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): get_auth_request con 0x7fe8180079b0 auth_method 0 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 -- 172.21.15.17:0/3368834274 learned_addr learned my addr 172.21.15.17:0/3368834274 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): get_auth_request con 0x7fe830136ab0 auth_method 0 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe830136f30 0x7fe83013d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): get_auth_request con 0x7fe830136f30 auth_method 0 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835c35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient(hunting): handle_auth_done global_id 234104 payload 824 2024-09-15T21:37:26.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 -- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe830136f30 msgr2=0x7fe83013d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe830136f30 0x7fe83013d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 -- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 msgr2=0x7fe8301cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe8301cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 monclient: found mon.noname-c 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 -- 172.21.15.17:0/3368834274 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe830120e80 con 0x7fe830136ab0 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe835434640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe8301cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe834c33640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 0x7fe83013cc40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe81800b740 tx=0x7fe824001320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824023020 con 0x7fe830136ab0 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: dump: 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.210 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe82400ccd0 con 0x7fe830136ab0 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824007310 con 0x7fe830136ab0 2024-09-15T21:37:26.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82dffb640 10 monclient: dump: 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 5 monclient: authenticate success, global_id 234104 2024-09-15T21:37:26.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: _renew_subs 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83013e700 con 0x7fe830136ab0 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe82f7fe640 4 set_mon_vals no callback set 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: _renew_subs 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.182+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83013f260 con 0x7fe830136ab0 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe837698640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe837698640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe837698640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8300a56c0 con 0x7fe830136ab0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe824026020 con 0x7fe830136ab0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe82dffb640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe81c03b9f0 0x7fe81c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe824051c90 con 0x7fe830136ab0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe835434640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe81c03b9f0 0x7fe81c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe835434640 10 monclient: get_auth_request con 0x7fe81c03b9f0 auth_method 0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.186+0000 7fe835434640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe81c03b9f0 0x7fe81c03deb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe82001aa10 tx=0x7fe8200a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.190+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fe82402d070 con 0x7fe830136ab0 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.190+0000 7fe82dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.190+0000 7fe82dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe837698640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:37:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe837698640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:37:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe837698640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe83013fb80 con 0x7fe830136ab0 2024-09-15T21:37:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe82dffb640 1 -- 172.21.15.17:0/3368834274 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7fe830120e80 con 0x7fe830136ab0 2024-09-15T21:37:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe82dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:37:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.426+0000 7fe82dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:26.434 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":178,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:37:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe81c03b9f0 msgr2=0x7fe81c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe81c03b9f0 0x7fe81c03deb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe82001aa10 tx=0x7fe8200a6000 comp rx=0 tx=0).stop 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 10 monclient: shutdown 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 msgr2=0x7fe83013cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 0x7fe83013cc40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe81800b740 tx=0x7fe824001320 comp rx=0 tx=0).stop 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 shutdown_connections 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe81c03b9f0 0x7fe81c03deb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe830136f30 0x7fe83013d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe830136ab0 0x7fe83013cc40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 --2- 172.21.15.17:0/3368834274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8180079b0 0x7fe8301cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 >> 172.21.15.17:0/3368834274 conn(0x7fe8300aac40 msgr2=0x7fe83006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 shutdown_connections 2024-09-15T21:37:26.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.430+0000 7fe837698640 1 -- 172.21.15.17:0/3368834274 wait complete. 2024-09-15T21:37:26.453 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:37:26.453 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-15T21:37:26.453 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:37:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient: get_monmap_and_config 2024-09-15T21:37:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient: monmap: 2024-09-15T21:37:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:26.578952+0000 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:26.578952+0000 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:26.580 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 Processor -- start 2024-09-15T21:37:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 -- start start 2024-09-15T21:37:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient: init 2024-09-15T21:37:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0c00a0890 0x7fc0c00a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): picked mon.noname-c con 0x7fc0c00a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0c009ffc0 0x7fc0c00a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): picked mon.noname-a con 0x7fc0c009ffc0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc0c0135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c00a3130 con 0x7fc0c0135670 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c00a16f0 con 0x7fc0c009ffc0 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c0073df0 con 0x7fc0c00a0890 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c80c3640 10 monclient(hunting): authenticate will time out at 2169.974599s 2024-09-15T21:37:26.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0c009ffc0 0x7fc0c00a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0c009ffc0 0x7fc0c00a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51390/0 (socket says 172.21.15.17:51390) 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 -- 172.21.15.17:0/3340091951 learned_addr learned my addr 172.21.15.17:0/3340091951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): get_auth_request con 0x7fc0c009ffc0 auth_method 0 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/3340091951 >> v2:172.21.15.119:3300/0 conn(0x7fc0c00a0890 0x7fc0c00a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): get_auth_request con 0x7fc0c00a0890 auth_method 0 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient(hunting): handle_auth_done global_id 204483 payload 274 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 -- 172.21.15.17:0/3340091951 >> v2:172.21.15.119:3300/0 conn(0x7fc0c00a0890 msgr2=0x7fc0c00a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 --2- 172.21.15.17:0/3340091951 >> v2:172.21.15.119:3300/0 conn(0x7fc0c00a0890 0x7fc0c00a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 -- 172.21.15.17:0/3340091951 >> v1:172.21.15.119:6789/0 conn(0x7fc0c0135670 legacy=0x7fc0c0135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 monclient: found mon.noname-a 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.574+0000 7fc0c565e640 1 -- 172.21.15.17:0/3340091951 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c00a1bf0 con 0x7fc0c009ffc0 2024-09-15T21:37:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c565e640 1 --2- 172.21.15.17:0/3340091951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c009ffc0 0x7fc0c00a11b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009e50 tx=0x7fc0b801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175be57746fcd4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 1 -- 172.21.15.17:0/3340091951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80a5680 con 0x7fc0c009ffc0 2024-09-15T21:37:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: dump: 2024-09-15T21:37:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 1 -- 172.21.15.17:0/3340091951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80a5820 con 0x7fc0c009ffc0 2024-09-15T21:37:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 5 monclient: authenticate success, global_id 204483 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 1 -- 172.21.15.17:0/3340091951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80a5bf0 con 0x7fc0c009ffc0 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 4 set_mon_vals no callback set 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c4e5d640 10 monclient: dump: 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.593 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 10 monclient: shutdown 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 -- 172.21.15.17:0/3340091951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c009ffc0 msgr2=0x7fc0c00a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/3340091951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c009ffc0 0x7fc0c00a11b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009e50 tx=0x7fc0b801c8b0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 -- 172.21.15.17:0/3340091951 shutdown_connections 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/3340091951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c009ffc0 0x7fc0c00a11b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/3340091951 >> v2:172.21.15.119:3300/0 conn(0x7fc0c00a0890 0x7fc0c00a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 -- 172.21.15.17:0/3340091951 >> 172.21.15.17:0/3340091951 conn(0x7fc0c00a66a0 msgr2=0x7fc0c00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 -- 172.21.15.17:0/3340091951 shutdown_connections 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.578+0000 7fc0c80c3640 1 -- 172.21.15.17:0/3340091951 wait complete. 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: monmap: 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.595 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 Processor -- start 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- start start 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: init 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0c009ffc0 0x7fc0c013c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): picked mon.noname-b con 0x7fc0c009ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a0890 0x7fc0c0134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): picked mon.noname-a con 0x7fc0c00a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc0c0135670 0x7fc0c0134ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): picked mon.noname-c con 0x7fc0c0135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0140690 con 0x7fc0c00a0890 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0140510 con 0x7fc0c009ffc0 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c0140810 con 0x7fc0c0135670 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): authenticate will time out at 2169.980948s 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a0890 0x7fc0c0134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a0890 0x7fc0c0134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51400/0 (socket says 172.21.15.17:51400) 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc0c0135670 0x7fc0c0134ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 -- 172.21.15.17:0/1089106811 learned_addr learned my addr 172.21.15.17:0/1089106811 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): get_auth_request con 0x7fc0c00a0890 auth_method 0 2024-09-15T21:37:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): get_auth_request con 0x7fc0c0135670 auth_method 0 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0c009ffc0 0x7fc0c013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): get_auth_request con 0x7fc0c009ffc0 auth_method 0 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient(hunting): handle_auth_done global_id 204486 payload 824 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc0c0135670 msgr2=0x7fc0c0134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc0c0135670 0x7fc0c0134ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0c009ffc0 msgr2=0x7fc0c013c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0c009ffc0 0x7fc0c013c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 monclient: found mon.noname-a 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c013d0b0 con 0x7fc0c00a0890 2024-09-15T21:37:26.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0c009ffc0 0x7fc0c013c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c565e640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a0890 0x7fc0c0134660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80af040 tx=0x7fc0b8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80bc020 con 0x7fc0c00a0890 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: dump: 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.600 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80b88a0 con 0x7fc0c00a0890 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 5 monclient: authenticate success, global_id 204486 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: _renew_subs 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c0072800 con 0x7fc0c00a0890 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80a5680 con 0x7fc0c00a0890 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: dump: 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: _renew_subs 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b7fff640 4 set_mon_vals no callback set 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c009e640 con 0x7fc0c00a0890 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 0x7fc0c01cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0135aa0 con 0x7fc0c01cf7b0 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 0x7fc0c01cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc0b80bf020 con 0x7fc0c00a0890 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient: get_auth_request con 0x7fc0c01cf7b0 auth_method 0 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc0ac03b9f0 0x7fc0ac03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_done global_id 204489 payload 824 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0c00008e0 con 0x7fc0c01cf7b0 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc0b80eb6c0 con 0x7fc0c00a0890 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 0x7fc0c01cfb90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011830 tx=0x7fc0bc011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00fa30 con 0x7fc0c01cf7b0 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc0ac03b9f0 0x7fc0ac03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: dump: 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.604 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0c6660640 10 monclient: get_auth_request con 0x7fc0ac03b9f0 auth_method 0 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc0bc00fa30 con 0x7fc0c01cf7b0 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 msgr2=0x7fc0c01cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 0x7fc0c01cfb90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011830 tx=0x7fc0bc011cf0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 msgr2=0x7fc0c01cfb90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.582+0000 7fc0b67fc640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01cf7b0 0x7fc0c01cfb90 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011830 tx=0x7fc0bc011cf0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.586+0000 7fc0c6660640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc0ac03b9f0 0x7fc0ac03deb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007fd0 tx=0x7fc0b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:37:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 0x7fc0c0138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c01d04e0 con 0x7fc0c01d00d0 2024-09-15T21:37:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 0x7fc0c0138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient: get_auth_request con 0x7fc0c01d00d0 auth_method 0 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 10 monclient(hunting): handle_auth_done global_id 204492 payload 824 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 1 -- 172.21.15.17:0/1089106811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc0c009d8c0 con 0x7fc0c01d00d0 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c5e5f640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 0x7fc0c0138ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011830 tx=0x7fc0bc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc022770 con 0x7fc0c01d00d0 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 10 monclient: dump: 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4882 (secure 0 0 0) 0x7fc0bc02c430 con 0x7fc0c01d00d0 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 msgr2=0x7fc0c0138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 0x7fc0c0138ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011830 tx=0x7fc0bc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 msgr2=0x7fc0c0138ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0b67fc640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c01d00d0 0x7fc0c0138ab0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011830 tx=0x7fc0bc0046d0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 178, 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 171125, 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:37:26.610 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.611 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.612 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:37:26.613 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:37:26.614 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.615 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:37:26.617 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.618 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:37:26.620 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.621 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.622 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:37:26.623 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc0ac03b9f0 msgr2=0x7fc0ac03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc0ac03b9f0 0x7fc0ac03deb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007fd0 tx=0x7fc0b0009040 comp rx=0 tx=0).stop 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 10 monclient: shutdown 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a0890 msgr2=0x7fc0c0134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 --2- 172.21.15.17:0/1089106811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a0890 0x7fc0c0134660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80af040 tx=0x7fc0b8019650 comp rx=0 tx=0).stop 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c6660640 1 -- 172.21.15.17:0/1089106811 reap_dead start 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c6660640 1 -- 172.21.15.17:0/1089106811 reap_dead start 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 shutdown_connections 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 >> 172.21.15.17:0/1089106811 conn(0x7fc0c00a66a0 msgr2=0x7fc0c00a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 shutdown_connections 2024-09-15T21:37:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.590+0000 7fc0c80c3640 1 -- 172.21.15.17:0/1089106811 wait complete. 2024-09-15T21:37:26.627 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:37:26.628 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-15T21:37:26.628 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient: get_monmap_and_config 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient: monmap: 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:26.750461+0000 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:26.750461+0000 2024-09-15T21:37:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:26.752 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 Processor -- start 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 -- start start 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient: init 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44a40a4c00 0x7f44a409cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): picked mon.noname-a con 0x7f44a40a4c00 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f44a40a4fe0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44a413ce10 0x7f44a413f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): picked mon.noname-c con 0x7f44a413ce10 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a409c700 con 0x7f44a40a4fe0 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f44a406f820 con 0x7f44a40a4c00 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f44a40a5fc0 con 0x7f44a413ce10 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a8feb640 10 monclient(hunting): authenticate will time out at 2170.145636s 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44a40a4c00 0x7f44a409cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44a40a4c00 0x7f44a409cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51426/0 (socket says 172.21.15.17:51426) 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 -- 172.21.15.17:0/1117938498 learned_addr learned my addr 172.21.15.17:0/1117938498 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): get_auth_request con 0x7f44a40a4c00 auth_method 0 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a2d9d640 1 --2- 172.21.15.17:0/1117938498 >> v2:172.21.15.119:3300/0 conn(0x7f44a413ce10 0x7f44a413f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient(hunting): handle_auth_done global_id 204495 payload 274 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 -- 172.21.15.17:0/1117938498 >> v2:172.21.15.119:3300/0 conn(0x7f44a413ce10 msgr2=0x7f44a413f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 --2- 172.21.15.17:0/1117938498 >> v2:172.21.15.119:3300/0 conn(0x7f44a413ce10 0x7f44a413f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 -- 172.21.15.17:0/1117938498 >> v1:172.21.15.119:6789/0 conn(0x7f44a40a4fe0 legacy=0x7f44a40a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 monclient: found mon.noname-a 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a259c640 1 -- 172.21.15.17:0/1117938498 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a40a5d10 con 0x7f44a40a4c00 2024-09-15T21:37:26.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a2d9d640 10 monclient: get_auth_request con 0x7f44a413ce10 auth_method 0 2024-09-15T21:37:26.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.746+0000 7f44a2d9d640 1 --2- 172.21.15.17:0/1117938498 >> v2:172.21.15.119:3300/0 conn(0x7f44a413ce10 0x7f44a413f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 --2- 172.21.15.17:0/1117938498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a409cb20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44a409ba60 tx=0x7f449401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84ebfbb32e37b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 1 -- 172.21.15.17:0/1117938498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940b0070 con 0x7f44a40a4c00 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: dump: 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.764 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 1 -- 172.21.15.17:0/1117938498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44940ab600 con 0x7f44a40a4c00 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 1 -- 172.21.15.17:0/1117938498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940ab9d0 con 0x7f44a40a4c00 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a159a640 10 monclient: dump: 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 5 monclient: authenticate success, global_id 204495 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 4 set_mon_vals no callback set 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: shutdown 2024-09-15T21:37:26.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/1117938498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 msgr2=0x7f44a409cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 --2- 172.21.15.17:0/1117938498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a409cb20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44a409ba60 tx=0x7f449401c470 comp rx=0 tx=0).stop 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/1117938498 shutdown_connections 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 --2- 172.21.15.17:0/1117938498 >> v2:172.21.15.119:3300/0 conn(0x7f44a413ce10 0x7f44a413f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 --2- 172.21.15.17:0/1117938498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a409cb20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/1117938498 >> 172.21.15.17:0/1117938498 conn(0x7f44a40aaeb0 msgr2=0x7f44a40a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/1117938498 shutdown_connections 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/1117938498 wait complete. 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: monmap: 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:26.767 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 Processor -- start 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- start start 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: init 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a41cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): picked mon.noname-a con 0x7f44a40a4c00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:26.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a40a4fe0 0x7f44a41cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): picked mon.noname-b con 0x7f44a40a4fe0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a413ce10 0x7f44a41d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): picked mon.noname-c con 0x7f44a413ce10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41422f0 con 0x7f44a40a4c00 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4142170 con 0x7f44a40a4fe0 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f44a4142470 con 0x7f44a413ce10 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient(hunting): authenticate will time out at 2170.150806s 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a41cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a41cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51440/0 (socket says 172.21.15.17:51440) 2024-09-15T21:37:26.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 -- 172.21.15.17:0/3000135586 learned_addr learned my addr 172.21.15.17:0/3000135586 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): get_auth_request con 0x7f44a40a4c00 auth_method 0 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a413ce10 0x7f44a41d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): get_auth_request con 0x7f44a413ce10 auth_method 0 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a40a4fe0 0x7f44a41cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a1d9b640 10 monclient(hunting): get_auth_request con 0x7f44a40a4fe0 auth_method 0 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient(hunting): handle_auth_done global_id 204498 payload 824 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a413ce10 msgr2=0x7f44a41d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a413ce10 0x7f44a41d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a40a4fe0 msgr2=0x7f44a41cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a40a4fe0 0x7f44a41cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 monclient: found mon.noname-a 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a40a3460 con 0x7f44a40a4c00 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a259c640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a41cf5f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f449401cd10 tx=0x7f449401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940ba020 con 0x7f44a40a4c00 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: dump: 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44940aa740 con 0x7f44a40a4c00 2024-09-15T21:37:26.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 5 monclient: authenticate success, global_id 204498 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: _renew_subs 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a40a3740 con 0x7f44a40a4c00 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940b0040 con 0x7f44a40a4c00 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f4492ffd640 10 monclient: dump: 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.774 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a0d99640 4 set_mon_vals no callback set 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: _renew_subs 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.750+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a409f4d0 con 0x7f44a40a4c00 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a8feb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a8feb640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f44a406a080 con 0x7f44a40a2ea0 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient: get_auth_request con 0x7f44a40a2ea0 auth_method 0 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f44940bd020 con 0x7f44a40a4c00 2024-09-15T21:37:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_done global_id 234113 payload 824 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44a409cb20 con 0x7f44a40a2ea0 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f448403b9c0 0x7f448403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f449800e420 tx=0x7f449800e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f44940ead10 con 0x7f44a40a4c00 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980052e0 con 0x7f44a40a2ea0 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: dump: 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.777 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a2d9d640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f448403b9c0 0x7f448403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a2d9d640 10 monclient: get_auth_request con 0x7f448403b9c0 auth_method 0 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 msgr2=0x7f44a4131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 msgr2=0x7f44a4131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f449800e420 tx=0x7f449800e8e0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f449800e420 tx=0x7f449800e8e0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 msgr2=0x7f44a4131960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 0x7f44840468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 msgr2=0x7f44a4131960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a40a2ea0 0x7f44a4131960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4484046f00 con 0x7f44840444d0 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 0x7f44840468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient: get_auth_request con 0x7f44840444d0 auth_method 0 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_done global_id 234116 payload 824 2024-09-15T21:37:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4494001d40 con 0x7f44840444d0 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 0x7f44840468c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4498009010 tx=0x7f4498001030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f44a2d9d640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f448403b9c0 0x7f448403de80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f44a406f7b0 tx=0x7f449c007780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498007620 con 0x7f44840444d0 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: dump: 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4498023070 con 0x7f44840444d0 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 msgr2=0x7f44840468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 0x7f44840468c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4498009010 tx=0x7f4498001030 comp rx=0 tx=0).stop 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 msgr2=0x7f44840468c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.754+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44840444d0 0x7f44840468c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 0x7f44a41322b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f44a41327f0 con 0x7f44a4131ea0 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 0x7f44a41322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient: get_auth_request con 0x7f44a4131ea0 auth_method 0 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 10 monclient(hunting): handle_auth_done global_id 234119 payload 824 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 1 -- 172.21.15.17:0/3000135586 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f44a40a0f90 con 0x7f44a4131ea0 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a1d9b640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 0x7f44a41322b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4498000c10 tx=0x7f4498013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498003e10 con 0x7f44a4131ea0 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 10 monclient: dump: 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 <== mon.2 v2:172.21.15.17:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f4498003e10 con 0x7f44a4131ea0 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 msgr2=0x7f44a41322b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 0x7f44a41322b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4498000c10 tx=0x7f4498013000 comp rx=0 tx=0).stop 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 msgr2=0x7f44a41322b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f4492ffd640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f44a4131ea0 0x7f44a41322b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a2d9d640 1 -- 172.21.15.17:0/3000135586 reap_dead start 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 178, 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20635, 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:37:26.786 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.787 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.788 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:37:26.789 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.790 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.793 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:37:26.795 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.796 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:37:26.797 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:26.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f448403b9c0 msgr2=0x7f448403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f448403b9c0 0x7f448403de80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f44a406f7b0 tx=0x7f449c007780 comp rx=0 tx=0).stop 2024-09-15T21:37:26.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 10 monclient: shutdown 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 msgr2=0x7f44a41cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a41cf5f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f449401cd10 tx=0x7f449401acb0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 shutdown_connections 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f448403b9c0 0x7f448403de80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 --2- 172.21.15.17:0/3000135586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a40a4c00 0x7f44a41cf5f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 >> 172.21.15.17:0/3000135586 conn(0x7f44a40aaeb0 msgr2=0x7f44a413ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 shutdown_connections 2024-09-15T21:37:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.762+0000 7f44a8feb640 1 -- 172.21.15.17:0/3000135586 wait complete. 2024-09-15T21:37:26.799 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:37:26.799 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-15T21:37:26.800 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:37:26.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient: get_monmap_and_config 2024-09-15T21:37:26.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient: monmap: 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:26.919286+0000 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:26.919286+0000 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:26.920 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 Processor -- start 2024-09-15T21:37:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 -- start start 2024-09-15T21:37:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient: init 2024-09-15T21:37:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65140a0890 0x7f65140a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): picked mon.noname-c con 0x7f65140a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f651409ffc0 0x7f65140a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): picked mon.noname-b con 0x7f651409ffc0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6514135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65140a3130 con 0x7f6514135670 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f65140a16f0 con 0x7f65140a0890 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6514073df0 con 0x7f651409ffc0 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65197a7640 10 monclient(hunting): authenticate will time out at 2170.314676s 2024-09-15T21:37:26.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f651409ffc0 0x7f65140a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65140a0890 0x7f65140a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65140a0890 0x7f65140a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.17:45964/0 (socket says 172.21.15.17:45964) 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 -- 172.21.15.17:0/1135056424 learned_addr learned my addr 172.21.15.17:0/1135056424 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): get_auth_request con 0x7f65140a0890 auth_method 0 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): get_auth_request con 0x7f651409ffc0 auth_method 0 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f65127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient(hunting): handle_auth_done global_id 224218 payload 274 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 -- 172.21.15.17:0/1135056424 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f651409ffc0 msgr2=0x7f65140a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 --2- 172.21.15.17:0/1135056424 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f651409ffc0 0x7f65140a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 -- 172.21.15.17:0/1135056424 >> v1:172.21.15.119:6789/0 conn(0x7f6514135670 legacy=0x7f6514135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 monclient: found mon.noname-c 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.119:3300/0 2024-09-15T21:37:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.914+0000 7f6512ffd640 1 -- 172.21.15.17:0/1135056424 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65140a1bf0 con 0x7f65140a0890 2024-09-15T21:37:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6512ffd640 1 --2- 172.21.15.17:0/1135056424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65140a0890 0x7f65140a0c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65140a1870 tx=0x7f650401c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f4e3460718ecac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 1 -- 172.21.15.17:0/1135056424 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040b1070 con 0x7f65140a0890 2024-09-15T21:37:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: dump: 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 1 -- 172.21.15.17:0/1135056424 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65040ac640 con 0x7f65140a0890 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 5 monclient: authenticate success, global_id 224218 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 4 set_mon_vals no callback set 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 1 -- 172.21.15.17:0/1135056424 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040aca10 con 0x7f65140a0890 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:37:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f6511ffb640 10 monclient: dump: 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 10 monclient: shutdown 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 -- 172.21.15.17:0/1135056424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65140a0890 msgr2=0x7f65140a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 --2- 172.21.15.17:0/1135056424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65140a0890 0x7f65140a0c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65140a1870 tx=0x7f650401c400 comp rx=0 tx=0).stop 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 -- 172.21.15.17:0/1135056424 shutdown_connections 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 --2- 172.21.15.17:0/1135056424 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f651409ffc0 0x7f65140a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 --2- 172.21.15.17:0/1135056424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65140a0890 0x7f65140a0c70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 -- 172.21.15.17:0/1135056424 >> 172.21.15.17:0/1135056424 conn(0x7f65140a66a0 msgr2=0x7f65140a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 -- 172.21.15.17:0/1135056424 shutdown_connections 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 -- 172.21.15.17:0/1135056424 wait complete. 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 10 monclient: build_initial_monmap 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 10 monclient: monmap: 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 Processor -- start 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 1 -- start start 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.918+0000 7f65197a7640 10 monclient: init 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:37:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409ffc0 0x7f651409d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): picked mon.noname-b con 0x7f651409ffc0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65140a0890 0x7f651409d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): picked mon.noname-a con 0x7f65140a0890 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6514135670 0x7f651409de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): picked mon.noname-c con 0x7f6514135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6514140690 con 0x7f65140a0890 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6514140510 con 0x7f651409ffc0 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f6514140810 con 0x7f6514135670 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:26.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): authenticate will time out at 2170.320372s 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6514135670 0x7f651409de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65140a0890 0x7f651409d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6514135670 0x7f651409de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60836/0 (socket says 172.21.15.17:60836) 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 -- 172.21.15.17:0/4000063006 learned_addr learned my addr 172.21.15.17:0/4000063006 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): get_auth_request con 0x7f6514135670 auth_method 0 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): get_auth_request con 0x7f65140a0890 auth_method 0 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409ffc0 0x7f651409d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): get_auth_request con 0x7f651409ffc0 auth_method 0 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient(hunting): handle_auth_done global_id 234125 payload 824 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409ffc0 msgr2=0x7f651409d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409ffc0 0x7f651409d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65140a0890 msgr2=0x7f651409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65140a0890 0x7f651409d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient: _finish_hunting 0 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 monclient: found mon.noname-c 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6514121010 con 0x7f6514135670 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409ffc0 0x7f651409d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65137fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6514135670 0x7f651409de80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c7a0 tx=0x7f650c00cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c003b80 con 0x7f6514135670 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: dump: 2024-09-15T21:37:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: _finish_auth 0 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c003d40 con 0x7f6514135670 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 5 monclient: authenticate success, global_id 234125 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient: _renew_subs 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651413d300 con 0x7f6514135670 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00aa90 con 0x7f6514135670 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: dump: 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.943 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65117fa640 4 set_mon_vals no callback set 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient: _renew_subs 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f651413c8f0 con 0x7f6514135670 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 0x7f6514133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6514072800 con 0x7f651409e530 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f650c00ac30 con 0x7f6514135670 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 0x7f6514133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient: get_auth_request con 0x7f651409e530 auth_method 0 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f650003b9c0 0x7f650003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f650c0504a0 con 0x7f6514135670 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 10 monclient(hunting): handle_auth_done global_id 224224 payload 824 2024-09-15T21:37:26.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6514135aa0 con 0x7f651409e530 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f650003b9c0 0x7f650003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 10 monclient: get_auth_request con 0x7f650003b9c0 auth_method 0 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f6512ffd640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 0x7f6514133f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f651406f1c0 tx=0x7f650401ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040bb030 con 0x7f651409e530 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f64fb7fe640 10 monclient: dump: 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.946 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.922+0000 7f65127fc640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f650003b9c0 0x7f650003de80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f651409efb0 tx=0x7f6508009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6504009490 con 0x7f651409e530 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 msgr2=0x7f6514133f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 0x7f6514133f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f651406f1c0 tx=0x7f650401ac40 comp rx=0 tx=0).stop 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 msgr2=0x7f6514133f60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.926+0000 7f64fb7fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651409e530 0x7f6514133f60 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f651406f1c0 tx=0x7f650401ac40 comp rx=0 tx=0).stop 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f65197a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f65197a7640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 0x7f6514134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f65197a7640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6514134dc0 con 0x7f65141344a0 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 0x7f6514134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient: get_auth_request con 0x7f65141344a0 auth_method 0 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 10 monclient(hunting): handle_auth_done global_id 224227 payload 824 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 1 -- 172.21.15.17:0/4000063006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f65141dae10 con 0x7f65141344a0 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f6512ffd640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 0x7f6514134880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f650401a030 tx=0x7f65040191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040bb040 con 0x7f65141344a0 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f64fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f64fb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.930+0000 7f64fb7fe640 10 monclient: dump: 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:26.950 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 <== mon.1 v2:172.21.15.119:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4545 (secure 0 0 0) 0x7f6504003ce0 con 0x7f65141344a0 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 msgr2=0x7f6514134880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 0x7f6514134880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f650401a030 tx=0x7f65040191a0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 msgr2=0x7f6514134880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f64fb7fe640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65141344a0 0x7f6514134880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 178, 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 76733, 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-15T21:37:26.952 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.953 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.954 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "bbfd2e88-7522-4bd5-ae16-c665761d9528", 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:37:26.955 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-15T21:16:49.663269Z", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-15T21:37:26.956 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.957 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.958 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:3300", 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789", 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.119:6789/0", 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.119:6789/0", 2024-09-15T21:37:26.960 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3301", 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790", 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6790/0", 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6790/0", 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.962 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-15T21:37:26.963 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-15T21:37:26.964 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-15T21:37:26.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-15T21:37:26.964 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-15T21:37:26.964 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f650003b9c0 msgr2=0x7f650003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f650003b9c0 0x7f650003de80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f651409efb0 tx=0x7f6508009190 comp rx=0 tx=0).stop 2024-09-15T21:37:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 10 monclient: shutdown 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6514135670 msgr2=0x7f651409de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 --2- 172.21.15.17:0/4000063006 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f6514135670 0x7f651409de80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c7a0 tx=0x7f650c00cc60 comp rx=0 tx=0).stop 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65137fe640 1 -- 172.21.15.17:0/4000063006 reap_dead start 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65137fe640 1 -- 172.21.15.17:0/4000063006 reap_dead start 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 shutdown_connections 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 >> 172.21.15.17:0/4000063006 conn(0x7f65140a66a0 msgr2=0x7f6514072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 shutdown_connections 2024-09-15T21:37:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:26.934+0000 7f65197a7640 1 -- 172.21.15.17:0/4000063006 wait complete. 2024-09-15T21:37:26.966 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-15T21:37:26.966 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T21:37:26.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T21:37:27.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.082+0000 7f3f80996640 10 monclient: get_monmap_and_config 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.082+0000 7f3f80996640 10 monclient: build_initial_monmap 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.082+0000 7f3f80996640 10 monclient: monmap: 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:27.089541+0000 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:27.089541+0000 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:27.090 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:27.091 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:27.091 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 Processor -- start 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 -- start start 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient: init 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f7c0a4a90 0x7f3f7c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): picked mon.noname-a con 0x7f3f7c0a4a90 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3f7c0a53b0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 0x7f3f7c137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): picked mon.noname-b con 0x7f3f7c0a5800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f7c0a2c30 con 0x7f3f7c0a53b0 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:27.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7c0a0f00 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c0a19b0 con 0x7f3f7c0a5800 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient(hunting): authenticate will time out at 2170.484983s 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 0x7f3f7c137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f7c0a4a90 0x7f3f7c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 0x7f3f7c137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:60838/0 (socket says 172.21.15.17:60838) 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 1 -- 172.21.15.17:0/1122326762 learned_addr learned my addr 172.21.15.17:0/1122326762 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): get_auth_request con 0x7f3f7c0a5800 auth_method 0 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): get_auth_request con 0x7f3f7c0a4a90 auth_method 0 2024-09-15T21:37:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_done global_id 204504 payload 274 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 -- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 msgr2=0x7f3f7c137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 --2- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 0x7f3f7c137f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 -- 172.21.15.17:0/1122326762 >> v1:172.21.15.119:6789/0 conn(0x7f3f7c0a53b0 legacy=0x7f3f7c137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:27.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:37:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 monclient: found mon.noname-a 2024-09-15T21:37:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 -- 172.21.15.17:0/1122326762 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c0a1c90 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7ad9d640 1 --2- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 0x7f3f7c137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f7a59c640 1 --2- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a4a90 0x7f3f7c0a4e70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0097f0 tx=0x7f3f6c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ca1ec0702ccfe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 1 -- 172.21.15.17:0/1122326762 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b0070 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: dump: 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:27.101 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: _finish_auth 0 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: _check_auth_tickets 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 5 monclient: authenticate success, global_id 204504 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 1 -- 172.21.15.17:0/1122326762 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0ab400 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:27.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f80996640 4 set_mon_vals no callback set 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 1 -- 172.21.15.17:0/1122326762 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0ab7d0 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.086+0000 7f3f79d9b640 10 monclient: dump: 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:27.103 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: shutdown 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a4a90 msgr2=0x7f3f7c0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 --2- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a4a90 0x7f3f7c0a4e70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0097f0 tx=0x7f3f6c01c360 comp rx=0 tx=0).stop 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/1122326762 shutdown_connections 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 --2- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a5800 0x7f3f7c137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 --2- 172.21.15.17:0/1122326762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a4a90 0x7f3f7c0a4e70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/1122326762 >> 172.21.15.17:0/1122326762 conn(0x7f3f7c0aabb0 msgr2=0x7f3f7c0a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/1122326762 shutdown_connections 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/1122326762 wait complete. 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: build_initial_monmap 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: monmap: 2024-09-15T21:37:27.104 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 Processor -- start 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- start start 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: init 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:27.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a4a90 0x7f3f7c0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): picked mon.noname-c con 0x7f3f7c0a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 0x7f3f7c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): picked mon.noname-a con 0x7f3f7c0a53b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f7c0a5800 0x7f3f7c13f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): picked mon.noname-b con 0x7f3f7c0a5800 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c138620 con 0x7f3f7c0a53b0 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1384a0 con 0x7f3f7c0a5800 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c142550 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient(hunting): authenticate will time out at 2170.489954s 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 0x7f3f7c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a4a90 0x7f3f7c0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 0x7f3f7c13f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51470/0 (socket says 172.21.15.17:51470) 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 1 -- 172.21.15.17:0/2738124431 learned_addr learned my addr 172.21.15.17:0/2738124431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): get_auth_request con 0x7f3f7c0a4a90 auth_method 0 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): get_auth_request con 0x7f3f7c0a53b0 auth_method 0 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7ad9d640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f7c0a5800 0x7f3f7c13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7ad9d640 10 monclient(hunting): get_auth_request con 0x7f3f7c0a5800 auth_method 0 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient(hunting): handle_auth_done global_id 234131 payload 824 2024-09-15T21:37:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 -- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f7c0a5800 msgr2=0x7f3f7c13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f7c0a5800 0x7f3f7c13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 -- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 msgr2=0x7f3f7c13f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 0x7f3f7c13f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient: _finish_hunting 0 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 monclient: found mon.noname-c 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 -- 172.21.15.17:0/2738124431 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c13c430 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7ad9d640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f7c0a5800 0x7f3f7c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f73fff640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 0x7f3f7c13f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f7a59c640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a4a90 0x7f3f7c0733f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ab000 tx=0x7f3f6c019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b8020 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: dump: 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:27.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 5 monclient: authenticate success, global_id 234131 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0a4e00 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _renew_subs 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c13ca20 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b0040 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 10 monclient: dump: 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f78d99640 4 set_mon_vals no callback set 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _renew_subs 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c13a420 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3f6c0bb020 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.090+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c0a4e70 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.094+0000 7f3f72ffd640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f5c03b9c0 0x7f3f5c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.094+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3f6c0e9f50 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.094+0000 7f3f73fff640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f5c03b9c0 0x7f3f5c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.094+0000 7f3f73fff640 10 monclient: get_auth_request con 0x7f3f5c03b9c0 auth_method 0 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.094+0000 7f3f73fff640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f5c03b9c0 0x7f3f5c03de80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3f64001d00 tx=0x7f3f64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:27.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.098+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3f6c0c2030 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.098+0000 7f3f72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:27.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.098+0000 7f3f72ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:27.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f80996640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T21:37:27.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f80996640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T21:37:27.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f80996640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:27.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f3f7c069200 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f72ffd640 1 -- 172.21.15.17:0/2738124431 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f3f7c13ca20 con 0x7f3f7c0a4a90 2024-09-15T21:37:27.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f72ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T21:37:27.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.334+0000 7f3f72ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:27.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=11,kv=11,logm=51} crc {auth=4191888111,config=3541143443,health=3693454644,kv=3059833255,logm=761358156}) 2024-09-15T21:37:27.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=12,kv=10,logm=49} crc {auth=2805165922,config=3541143443,health=2247699913,kv=1657141003,logm=331745556}) 2024-09-15T21:37:27.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=11,kv=11,logm=51} crc {auth=4191888111,config=3541143443,health=3693454644,kv=3059833255,logm=761358156}) 2024-09-15T21:37:27.343 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=25,config=2,health=11,kv=12,logm=50} crc {auth=2270416308,config=3541143443,health=259001540,kv=3079713637,logm=322512163}) 2024-09-15T21:37:27.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f5c03b9c0 msgr2=0x7f3f5c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f5c03b9c0 0x7f3f5c03de80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3f64001d00 tx=0x7f3f64009040 comp rx=0 tx=0).stop 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 10 monclient: shutdown 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a4a90 msgr2=0x7f3f7c0733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a4a90 0x7f3f7c0733f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ab000 tx=0x7f3f6c019100 comp rx=0 tx=0).stop 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 shutdown_connections 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3f5c03b9c0 0x7f3f5c03de80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f7c0a5800 0x7f3f7c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f7c0a53b0 0x7f3f7c13f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 --2- 172.21.15.17:0/2738124431 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3f7c0a4a90 0x7f3f7c0733f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 >> 172.21.15.17:0/2738124431 conn(0x7f3f7c0aabb0 msgr2=0x7f3f7c0a02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 shutdown_connections 2024-09-15T21:37:27.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:27.338+0000 7f3f80996640 1 -- 172.21.15.17:0/2738124431 wait complete. 2024-09-15T21:37:27.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1498600353}) 2024-09-15T21:37:27.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3810751169}) 2024-09-15T21:37:27.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1498600353}) 2024-09-15T21:37:27.347 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.338+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=991673990}) 2024-09-15T21:37:27.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.342+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.342+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189601275}) 2024-09-15T21:37:27.348 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.342+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632409527}) 2024-09-15T21:37:27.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.342+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.342+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189601275}) 2024-09-15T21:37:27.349 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.342+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=139227063}) 2024-09-15T21:37:27.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=154694270}) 2024-09-15T21:37:27.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90178632}) 2024-09-15T21:37:27.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=154694270}) 2024-09-15T21:37:27.351 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299756305}) 2024-09-15T21:37:27.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650363171}) 2024-09-15T21:37:27.353 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2295439996}) 2024-09-15T21:37:27.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650363171}) 2024-09-15T21:37:27.354 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.346+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=132402293}) 2024-09-15T21:37:27.355 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.355 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1517530996}) 2024-09-15T21:37:27.355 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=292624120}) 2024-09-15T21:37:27.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1517530996}) 2024-09-15T21:37:27.356 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2549199329}) 2024-09-15T21:37:27.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1323405704}) 2024-09-15T21:37:27.357 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3450041385}) 2024-09-15T21:37:27.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1323405704}) 2024-09-15T21:37:27.358 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.350+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196993413}) 2024-09-15T21:37:27.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175643700}) 2024-09-15T21:37:27.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2032656314}) 2024-09-15T21:37:27.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175643700}) 2024-09-15T21:37:27.359 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3361460462}) 2024-09-15T21:37:27.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3854885405}) 2024-09-15T21:37:27.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712747132}) 2024-09-15T21:37:27.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3854885405}) 2024-09-15T21:37:27.361 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.354+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2022961884}) 2024-09-15T21:37:27.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4292465703}) 2024-09-15T21:37:27.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3960180854}) 2024-09-15T21:37:27.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4292465703}) 2024-09-15T21:37:27.363 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1832281255}) 2024-09-15T21:37:27.365 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T21:37:27.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3239524381}) 2024-09-15T21:37:27.365 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2906632282}) 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3239524381}) 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3311373319}) 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3464906679}) 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=618152177}) 2024-09-15T21:37:27.366 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3464906679}) 2024-09-15T21:37:27.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.358+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=859298710}) 2024-09-15T21:37:27.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.367 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1305018203}) 2024-09-15T21:37:27.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1952401323}) 2024-09-15T21:37:27.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1305018203}) 2024-09-15T21:37:27.368 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=289930269}) 2024-09-15T21:37:27.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2905084386}) 2024-09-15T21:37:27.369 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2194083076}) 2024-09-15T21:37:27.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2905084386}) 2024-09-15T21:37:27.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.362+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1458395075}) 2024-09-15T21:37:27.370 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=822661464}) 2024-09-15T21:37:27.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=35703285}) 2024-09-15T21:37:27.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=822661464}) 2024-09-15T21:37:27.371 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2896310900}) 2024-09-15T21:37:27.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=956358479}) 2024-09-15T21:37:27.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3592654941}) 2024-09-15T21:37:27.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=956358479}) 2024-09-15T21:37:27.372 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3244039986}) 2024-09-15T21:37:27.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=777577253}) 2024-09-15T21:37:27.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734987330}) 2024-09-15T21:37:27.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.374 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=777577253}) 2024-09-15T21:37:27.375 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.366+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1125325278}) 2024-09-15T21:37:27.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420303523}) 2024-09-15T21:37:27.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887231046}) 2024-09-15T21:37:27.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420303523}) 2024-09-15T21:37:27.376 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2894942254}) 2024-09-15T21:37:27.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802627441}) 2024-09-15T21:37:27.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=489753950}) 2024-09-15T21:37:27.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802627441}) 2024-09-15T21:37:27.378 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.370+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742996488}) 2024-09-15T21:37:27.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=255275468}) 2024-09-15T21:37:27.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1370526644}) 2024-09-15T21:37:27.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=255275468}) 2024-09-15T21:37:27.380 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1409159504}) 2024-09-15T21:37:27.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=33137220}) 2024-09-15T21:37:27.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537043578}) 2024-09-15T21:37:27.381 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=33137220}) 2024-09-15T21:37:27.382 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.374+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1223395792}) 2024-09-15T21:37:27.383 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435808454}) 2024-09-15T21:37:27.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1298494332}) 2024-09-15T21:37:27.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435808454}) 2024-09-15T21:37:27.384 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2052741944}) 2024-09-15T21:37:27.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3733345987}) 2024-09-15T21:37:27.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=536645210}) 2024-09-15T21:37:27.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3733345987}) 2024-09-15T21:37:27.386 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.378+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=241344959}) 2024-09-15T21:37:27.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2642347258}) 2024-09-15T21:37:27.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=103723523}) 2024-09-15T21:37:27.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2642347258}) 2024-09-15T21:37:27.387 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997880229}) 2024-09-15T21:37:27.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193803537}) 2024-09-15T21:37:27.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3448605042}) 2024-09-15T21:37:27.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193803537}) 2024-09-15T21:37:27.389 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.382+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2019165864}) 2024-09-15T21:37:27.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=525675640}) 2024-09-15T21:37:27.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1182575098}) 2024-09-15T21:37:27.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=525675640}) 2024-09-15T21:37:27.391 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4119299707}) 2024-09-15T21:37:27.392 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3942957025}) 2024-09-15T21:37:27.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2823293982}) 2024-09-15T21:37:27.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3942957025}) 2024-09-15T21:37:27.393 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1335310935}) 2024-09-15T21:37:27.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1272911228}) 2024-09-15T21:37:27.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.386+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3242902504}) 2024-09-15T21:37:27.394 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1272911228}) 2024-09-15T21:37:27.395 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3806273821}) 2024-09-15T21:37:27.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.396 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655597142}) 2024-09-15T21:37:27.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2784683632}) 2024-09-15T21:37:27.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655597142}) 2024-09-15T21:37:27.397 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.390+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4123610918}) 2024-09-15T21:37:27.398 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632576496}) 2024-09-15T21:37:27.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1812896256}) 2024-09-15T21:37:27.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=632576496}) 2024-09-15T21:37:27.399 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2632647074}) 2024-09-15T21:37:27.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974706747}) 2024-09-15T21:37:27.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409503678}) 2024-09-15T21:37:27.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974706747}) 2024-09-15T21:37:27.401 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.394+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=834928453}) 2024-09-15T21:37:27.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316204644}) 2024-09-15T21:37:27.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987134444}) 2024-09-15T21:37:27.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316204644}) 2024-09-15T21:37:27.403 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=29306154}) 2024-09-15T21:37:27.404 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3971066737}) 2024-09-15T21:37:27.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3158532694}) 2024-09-15T21:37:27.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3971066737}) 2024-09-15T21:37:27.405 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.398+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1702156951}) 2024-09-15T21:37:27.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1392069711}) 2024-09-15T21:37:27.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=369608108}) 2024-09-15T21:37:27.406 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1392069711}) 2024-09-15T21:37:27.407 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569924300}) 2024-09-15T21:37:27.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65720013}) 2024-09-15T21:37:27.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3536559561}) 2024-09-15T21:37:27.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65720013}) 2024-09-15T21:37:27.408 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1026678659}) 2024-09-15T21:37:27.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4199060733}) 2024-09-15T21:37:27.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2351159250}) 2024-09-15T21:37:27.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4199060733}) 2024-09-15T21:37:27.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.402+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=263821270}) 2024-09-15T21:37:27.411 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1325323021}) 2024-09-15T21:37:27.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1424970636}) 2024-09-15T21:37:27.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1325323021}) 2024-09-15T21:37:27.412 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1136376079}) 2024-09-15T21:37:27.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2739153917}) 2024-09-15T21:37:27.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3971189937}) 2024-09-15T21:37:27.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2739153917}) 2024-09-15T21:37:27.414 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.406+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2245040345}) 2024-09-15T21:37:27.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4205343962}) 2024-09-15T21:37:27.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3868540991}) 2024-09-15T21:37:27.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4205343962}) 2024-09-15T21:37:27.416 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1603732692}) 2024-09-15T21:37:27.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631843908}) 2024-09-15T21:37:27.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676403625}) 2024-09-15T21:37:27.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631843908}) 2024-09-15T21:37:27.418 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.410+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=554719619}) 2024-09-15T21:37:27.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2654324525}) 2024-09-15T21:37:27.419 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176357598}) 2024-09-15T21:37:27.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2654324525}) 2024-09-15T21:37:27.420 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=297334631}) 2024-09-15T21:37:27.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2022030913}) 2024-09-15T21:37:27.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1429391589}) 2024-09-15T21:37:27.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.421 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2022030913}) 2024-09-15T21:37:27.422 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.414+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=548759224}) 2024-09-15T21:37:27.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2356291556}) 2024-09-15T21:37:27.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=198214340}) 2024-09-15T21:37:27.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2356291556}) 2024-09-15T21:37:27.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3613252494}) 2024-09-15T21:37:27.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.424 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3016232330}) 2024-09-15T21:37:27.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198560775}) 2024-09-15T21:37:27.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3016232330}) 2024-09-15T21:37:27.425 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1037175376}) 2024-09-15T21:37:27.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379414294}) 2024-09-15T21:37:27.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3111203581}) 2024-09-15T21:37:27.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379414294}) 2024-09-15T21:37:27.426 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.418+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3086724018}) 2024-09-15T21:37:27.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=821846700}) 2024-09-15T21:37:27.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4289661833}) 2024-09-15T21:37:27.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=821846700}) 2024-09-15T21:37:27.428 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2632381242}) 2024-09-15T21:37:27.429 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.429 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674232473}) 2024-09-15T21:37:27.429 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3542071064}) 2024-09-15T21:37:27.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674232473}) 2024-09-15T21:37:27.430 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.422+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3544838078}) 2024-09-15T21:37:27.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707511206}) 2024-09-15T21:37:27.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=153967234}) 2024-09-15T21:37:27.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707511206}) 2024-09-15T21:37:27.431 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4006057036}) 2024-09-15T21:37:27.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=350447722}) 2024-09-15T21:37:27.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1720870524}) 2024-09-15T21:37:27.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=350447722}) 2024-09-15T21:37:27.433 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.426+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2553364661}) 2024-09-15T21:37:27.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1363355247}) 2024-09-15T21:37:27.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=218937851}) 2024-09-15T21:37:27.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1363355247}) 2024-09-15T21:37:27.435 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=851538832}) 2024-09-15T21:37:27.436 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797520000}) 2024-09-15T21:37:27.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120268196}) 2024-09-15T21:37:27.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797520000}) 2024-09-15T21:37:27.437 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=734444630}) 2024-09-15T21:37:27.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207630077}) 2024-09-15T21:37:27.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=910060429}) 2024-09-15T21:37:27.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207630077}) 2024-09-15T21:37:27.438 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.430+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2338607506}) 2024-09-15T21:37:27.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916149038}) 2024-09-15T21:37:27.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1836142557}) 2024-09-15T21:37:27.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916149038}) 2024-09-15T21:37:27.440 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=592448594}) 2024-09-15T21:37:27.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3960197455}) 2024-09-15T21:37:27.441 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=527329434}) 2024-09-15T21:37:27.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3960197455}) 2024-09-15T21:37:27.442 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.434+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814536013}) 2024-09-15T21:37:27.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3667106223}) 2024-09-15T21:37:27.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434139577}) 2024-09-15T21:37:27.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3667106223}) 2024-09-15T21:37:27.443 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119960939}) 2024-09-15T21:37:27.444 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=388566904}) 2024-09-15T21:37:27.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2292437962}) 2024-09-15T21:37:27.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=388566904}) 2024-09-15T21:37:27.445 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.438+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468964051}) 2024-09-15T21:37:27.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=512212443}) 2024-09-15T21:37:27.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=658216218}) 2024-09-15T21:37:27.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=512212443}) 2024-09-15T21:37:27.446 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4226961514}) 2024-09-15T21:37:27.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1267090386}) 2024-09-15T21:37:27.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3689905772}) 2024-09-15T21:37:27.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1267090386}) 2024-09-15T21:37:27.448 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=391671203}) 2024-09-15T21:37:27.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3005984565}) 2024-09-15T21:37:27.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1728503212}) 2024-09-15T21:37:27.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3005984565}) 2024-09-15T21:37:27.450 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.442+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1130517584}) 2024-09-15T21:37:27.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3970656409}) 2024-09-15T21:37:27.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2020769237}) 2024-09-15T21:37:27.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3970656409}) 2024-09-15T21:37:27.452 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1649998746}) 2024-09-15T21:37:27.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1553557845}) 2024-09-15T21:37:27.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2740911737}) 2024-09-15T21:37:27.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1553557845}) 2024-09-15T21:37:27.454 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.446+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3968555477}) 2024-09-15T21:37:27.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873562029}) 2024-09-15T21:37:27.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=755958503}) 2024-09-15T21:37:27.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873562029}) 2024-09-15T21:37:27.456 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=423158170}) 2024-09-15T21:37:27.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2368887309}) 2024-09-15T21:37:27.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3371323202}) 2024-09-15T21:37:27.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2368887309}) 2024-09-15T21:37:27.458 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.450+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2447928631}) 2024-09-15T21:37:27.459 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3402083352}) 2024-09-15T21:37:27.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1885163540}) 2024-09-15T21:37:27.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3402083352}) 2024-09-15T21:37:27.460 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614579671}) 2024-09-15T21:37:27.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130263721}) 2024-09-15T21:37:27.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=813122830}) 2024-09-15T21:37:27.461 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.462 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130263721}) 2024-09-15T21:37:27.462 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.454+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3707489050}) 2024-09-15T21:37:27.463 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2853513102}) 2024-09-15T21:37:27.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1539318456}) 2024-09-15T21:37:27.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2853513102}) 2024-09-15T21:37:27.464 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3091526452}) 2024-09-15T21:37:27.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82810880}) 2024-09-15T21:37:27.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3346574558}) 2024-09-15T21:37:27.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82810880}) 2024-09-15T21:37:27.465 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.458+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3981482396}) 2024-09-15T21:37:27.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1726664566}) 2024-09-15T21:37:27.466 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=490802556}) 2024-09-15T21:37:27.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1726664566}) 2024-09-15T21:37:27.467 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=195706688}) 2024-09-15T21:37:27.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2464108588}) 2024-09-15T21:37:27.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1634004895}) 2024-09-15T21:37:27.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2464108588}) 2024-09-15T21:37:27.468 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279770253}) 2024-09-15T21:37:27.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3191273553}) 2024-09-15T21:37:27.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=85866348}) 2024-09-15T21:37:27.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3191273553}) 2024-09-15T21:37:27.470 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.462+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3947135652}) 2024-09-15T21:37:27.471 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4282770119}) 2024-09-15T21:37:27.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3706913377}) 2024-09-15T21:37:27.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4282770119}) 2024-09-15T21:37:27.472 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323330464}) 2024-09-15T21:37:27.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2598119670}) 2024-09-15T21:37:27.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1980299811}) 2024-09-15T21:37:27.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2598119670}) 2024-09-15T21:37:27.474 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.466+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2601447202}) 2024-09-15T21:37:27.475 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3618825998}) 2024-09-15T21:37:27.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226645602}) 2024-09-15T21:37:27.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3618825998}) 2024-09-15T21:37:27.476 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4155361229}) 2024-09-15T21:37:27.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4069792837}) 2024-09-15T21:37:27.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=764846897}) 2024-09-15T21:37:27.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.477 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4069792837}) 2024-09-15T21:37:27.478 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.470+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3624288228}) 2024-09-15T21:37:27.479 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=565650098}) 2024-09-15T21:37:27.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1338347256}) 2024-09-15T21:37:27.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=565650098}) 2024-09-15T21:37:27.480 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129286663}) 2024-09-15T21:37:27.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=221368057}) 2024-09-15T21:37:27.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2570437479}) 2024-09-15T21:37:27.481 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=221368057}) 2024-09-15T21:37:27.482 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.474+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2668031762}) 2024-09-15T21:37:27.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3718082304}) 2024-09-15T21:37:27.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2331033341}) 2024-09-15T21:37:27.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3718082304}) 2024-09-15T21:37:27.483 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1760891798}) 2024-09-15T21:37:27.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=995178655}) 2024-09-15T21:37:27.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4247628736}) 2024-09-15T21:37:27.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=995178655}) 2024-09-15T21:37:27.485 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.478+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2464348067}) 2024-09-15T21:37:27.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1622049142}) 2024-09-15T21:37:27.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3098663275}) 2024-09-15T21:37:27.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1622049142}) 2024-09-15T21:37:27.487 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=52108232}) 2024-09-15T21:37:27.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2439579690}) 2024-09-15T21:37:27.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2971561446}) 2024-09-15T21:37:27.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.489 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2439579690}) 2024-09-15T21:37:27.490 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.482+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2476776492}) 2024-09-15T21:37:27.490 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3867339388}) 2024-09-15T21:37:27.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4163310182}) 2024-09-15T21:37:27.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3867339388}) 2024-09-15T21:37:27.491 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1282322656}) 2024-09-15T21:37:27.492 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.492 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969814446}) 2024-09-15T21:37:27.492 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1696283949}) 2024-09-15T21:37:27.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969814446}) 2024-09-15T21:37:27.493 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.486+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1159762746}) 2024-09-15T21:37:27.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=848157784}) 2024-09-15T21:37:27.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2911314677}) 2024-09-15T21:37:27.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=848157784}) 2024-09-15T21:37:27.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3820743117}) 2024-09-15T21:37:27.496 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.496 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=245858057}) 2024-09-15T21:37:27.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=188121587}) 2024-09-15T21:37:27.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=245858057}) 2024-09-15T21:37:27.497 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.490+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3412561255}) 2024-09-15T21:37:27.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.498 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3281091411}) 2024-09-15T21:37:27.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2862481718}) 2024-09-15T21:37:27.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3281091411}) 2024-09-15T21:37:27.499 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2282660600}) 2024-09-15T21:37:27.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=59886816}) 2024-09-15T21:37:27.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1059110093}) 2024-09-15T21:37:27.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=59886816}) 2024-09-15T21:37:27.500 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3350407393}) 2024-09-15T21:37:27.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2783502462}) 2024-09-15T21:37:27.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3041900489}) 2024-09-15T21:37:27.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2783502462}) 2024-09-15T21:37:27.502 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.494+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417817862}) 2024-09-15T21:37:27.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.503 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3943362975}) 2024-09-15T21:37:27.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=230524132}) 2024-09-15T21:37:27.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3943362975}) 2024-09-15T21:37:27.504 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=14087995}) 2024-09-15T21:37:27.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1755639803}) 2024-09-15T21:37:27.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282055695}) 2024-09-15T21:37:27.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1755639803}) 2024-09-15T21:37:27.505 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.498+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4257702511}) 2024-09-15T21:37:27.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2741864816}) 2024-09-15T21:37:27.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1939857995}) 2024-09-15T21:37:27.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2741864816}) 2024-09-15T21:37:27.507 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1975524514}) 2024-09-15T21:37:27.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=804591283}) 2024-09-15T21:37:27.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4040665087}) 2024-09-15T21:37:27.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=804591283}) 2024-09-15T21:37:27.509 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.502+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=52971291}) 2024-09-15T21:37:27.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.510 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284982488}) 2024-09-15T21:37:27.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1343206005}) 2024-09-15T21:37:27.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284982488}) 2024-09-15T21:37:27.511 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=173528902}) 2024-09-15T21:37:27.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4040505461}) 2024-09-15T21:37:27.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2010334046}) 2024-09-15T21:37:27.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4040505461}) 2024-09-15T21:37:27.512 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041510219}) 2024-09-15T21:37:27.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619604556}) 2024-09-15T21:37:27.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1258551408}) 2024-09-15T21:37:27.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619604556}) 2024-09-15T21:37:27.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.506+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=420265364}) 2024-09-15T21:37:27.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597033418}) 2024-09-15T21:37:27.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3990199829}) 2024-09-15T21:37:27.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597033418}) 2024-09-15T21:37:27.516 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4172535249}) 2024-09-15T21:37:27.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.517 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=221596435}) 2024-09-15T21:37:27.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1648555077}) 2024-09-15T21:37:27.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=221596435}) 2024-09-15T21:37:27.518 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.510+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2995663079}) 2024-09-15T21:37:27.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3858409602}) 2024-09-15T21:37:27.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1194608259}) 2024-09-15T21:37:27.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3858409602}) 2024-09-15T21:37:27.519 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4138009559}) 2024-09-15T21:37:27.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2077820030}) 2024-09-15T21:37:27.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2128606462}) 2024-09-15T21:37:27.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2077820030}) 2024-09-15T21:37:27.521 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.514+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=128214321}) 2024-09-15T21:37:27.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379499281}) 2024-09-15T21:37:27.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3934478889}) 2024-09-15T21:37:27.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379499281}) 2024-09-15T21:37:27.523 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422967357}) 2024-09-15T21:37:27.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2246222414}) 2024-09-15T21:37:27.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1872338272}) 2024-09-15T21:37:27.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2246222414}) 2024-09-15T21:37:27.525 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.518+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2161050401}) 2024-09-15T21:37:27.526 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684387643}) 2024-09-15T21:37:27.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1935415238}) 2024-09-15T21:37:27.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684387643}) 2024-09-15T21:37:27.527 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3839383780}) 2024-09-15T21:37:27.528 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.529 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2291823984}) 2024-09-15T21:37:27.529 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=147068033}) 2024-09-15T21:37:27.529 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.529 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2291823984}) 2024-09-15T21:37:27.529 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=477371236}) 2024-09-15T21:37:27.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.522+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3117924728}) 2024-09-15T21:37:27.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4024109160}) 2024-09-15T21:37:27.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3117924728}) 2024-09-15T21:37:27.531 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1332871632}) 2024-09-15T21:37:27.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=615961594}) 2024-09-15T21:37:27.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482938734}) 2024-09-15T21:37:27.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=615961594}) 2024-09-15T21:37:27.532 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=758744907}) 2024-09-15T21:37:27.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2433800566}) 2024-09-15T21:37:27.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2027137196}) 2024-09-15T21:37:27.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2433800566}) 2024-09-15T21:37:27.534 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.526+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4280063823}) 2024-09-15T21:37:27.535 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=910340628}) 2024-09-15T21:37:27.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3267059152}) 2024-09-15T21:37:27.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=910340628}) 2024-09-15T21:37:27.536 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2555493077}) 2024-09-15T21:37:27.537 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3616193289}) 2024-09-15T21:37:27.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=488156109}) 2024-09-15T21:37:27.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3616193289}) 2024-09-15T21:37:27.538 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.530+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3912883803}) 2024-09-15T21:37:27.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3892726627}) 2024-09-15T21:37:27.539 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3011358449}) 2024-09-15T21:37:27.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3892726627}) 2024-09-15T21:37:27.540 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2787536603}) 2024-09-15T21:37:27.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=778311019}) 2024-09-15T21:37:27.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4061603049}) 2024-09-15T21:37:27.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=778311019}) 2024-09-15T21:37:27.541 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.534+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2220845418}) 2024-09-15T21:37:27.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=174614473}) 2024-09-15T21:37:27.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1246026099}) 2024-09-15T21:37:27.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=174614473}) 2024-09-15T21:37:27.543 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3371282995}) 2024-09-15T21:37:27.544 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93637062}) 2024-09-15T21:37:27.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2548307347}) 2024-09-15T21:37:27.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93637062}) 2024-09-15T21:37:27.545 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.538+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2456279811}) 2024-09-15T21:37:27.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3354969615}) 2024-09-15T21:37:27.546 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=784099730}) 2024-09-15T21:37:27.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3354969615}) 2024-09-15T21:37:27.547 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=861986837}) 2024-09-15T21:37:27.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592536329}) 2024-09-15T21:37:27.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1407420647}) 2024-09-15T21:37:27.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.548 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592536329}) 2024-09-15T21:37:27.549 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113668398}) 2024-09-15T21:37:27.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2487836157}) 2024-09-15T21:37:27.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=990292141}) 2024-09-15T21:37:27.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.542+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.550 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2487836157}) 2024-09-15T21:37:27.551 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2868875295}) 2024-09-15T21:37:27.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3640569736}) 2024-09-15T21:37:27.552 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3150046049}) 2024-09-15T21:37:27.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3640569736}) 2024-09-15T21:37:27.553 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3297841665}) 2024-09-15T21:37:27.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=476078527}) 2024-09-15T21:37:27.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4027609473}) 2024-09-15T21:37:27.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=476078527}) 2024-09-15T21:37:27.554 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.546+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=642838716}) 2024-09-15T21:37:27.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1630720303}) 2024-09-15T21:37:27.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2627555621}) 2024-09-15T21:37:27.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1630720303}) 2024-09-15T21:37:27.556 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4195788299}) 2024-09-15T21:37:27.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2239413632}) 2024-09-15T21:37:27.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2216849671}) 2024-09-15T21:37:27.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2239413632}) 2024-09-15T21:37:27.558 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.550+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2319864181}) 2024-09-15T21:37:27.559 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1710145067}) 2024-09-15T21:37:27.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1244169538}) 2024-09-15T21:37:27.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1710145067}) 2024-09-15T21:37:27.560 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4040999256}) 2024-09-15T21:37:27.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4008250532}) 2024-09-15T21:37:27.561 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=837793694}) 2024-09-15T21:37:27.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4008250532}) 2024-09-15T21:37:27.562 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.554+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1820511820}) 2024-09-15T21:37:27.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3274435615}) 2024-09-15T21:37:27.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=56156996}) 2024-09-15T21:37:27.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3274435615}) 2024-09-15T21:37:27.563 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1031862999}) 2024-09-15T21:37:27.565 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.565 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=237071260}) 2024-09-15T21:37:27.565 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1037961368}) 2024-09-15T21:37:27.565 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.565 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=237071260}) 2024-09-15T21:37:27.565 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.558+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1729700416}) 2024-09-15T21:37:27.567 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.567 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430905006}) 2024-09-15T21:37:27.567 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1772735984}) 2024-09-15T21:37:27.567 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.567 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430905006}) 2024-09-15T21:37:27.567 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888065822}) 2024-09-15T21:37:27.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=2,mgr=7,mgr_metadata=1,mgrstat=10,mon_config_key=14,monmap=2} crc {logm=3652912754,mdsmap=3140514009,mgr=1715286761,mgr_metadata=240635875,mgrstat=1803280861,mon_config_key=521969046,monmap=2617021892}) 2024-09-15T21:37:27.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9} crc {logm=489018162,mdsmap=173658045,mgr=3916736423,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1038082621,mon_config_key=3541163686}) 2024-09-15T21:37:27.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.569 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=2,mgr=7,mgr_metadata=1,mgrstat=10,mon_config_key=14,monmap=2} crc {logm=3652912754,mdsmap=3140514009,mgr=1715286761,mgr_metadata=240635875,mgrstat=1803280861,mon_config_key=521969046,monmap=2617021892}) 2024-09-15T21:37:27.570 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.562+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=63,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13} crc {logm=1452698294,mdsmap=173658045,mgr=3654699112,mgr_command_descs=2551042219,mgr_metadata=240635875,mgrstat=1038082621,mon_config_key=1008760200}) 2024-09-15T21:37:27.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.574+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.574+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=1,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {monmap=1228700967,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3878507066}) 2024-09-15T21:37:27.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.574+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {monmap=1,nvmeofgw=3,osd_metadata=4,osd_pg_creating=1,osd_snap=2,osdmap=51} crc {monmap=1228700967,nvmeofgw=598337686,osd_metadata=3235920081,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3150460335}) 2024-09-15T21:37:27.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.574+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T21:37:27.580 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.574+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=1,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {monmap=1228700967,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=3878507066}) 2024-09-15T21:37:27.581 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-15T21:37:27.574+0000 7fc1aa6df640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {monmap=1,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=50} crc {monmap=1228700967,nvmeofgw=598337686,osd_metadata=2194678291,osd_pg_creating=543505836,osd_snap=841190635,osdmap=1104929832}) 2024-09-15T21:37:28.366 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-15T21:37:28.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient: get_monmap_and_config 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient: build_initial_monmap 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient: monmap: 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:28.499662+0000 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:28.499662+0000 2024-09-15T21:37:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:28.501 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 1 Processor -- start 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 1 -- start start 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient: init 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f71500a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71500a5580 0x7f715013cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient(hunting): picked mon.noname-a con 0x7f71500a5580 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f71500a59d0 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71500a2c30 con 0x7f71500a59d0 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71500a19b0 con 0x7f71500a4a90 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f71500a0f40 con 0x7f71500a5580 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f71549a3640 10 monclient(hunting): authenticate will time out at 2171.895109s 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71500a5580 0x7f715013cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714ed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f71500a59d0 0x7f715013f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45442/0 (socket says 172.21.15.17:45442) 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714ed9d640 1 -- 172.21.15.17:0/1090176662 learned_addr learned my addr 172.21.15.17:0/1090176662 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): get_auth_request con 0x7f71500a5580 auth_method 0 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient(hunting): handle_auth_done global_id 204510 payload 274 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 1 -- 172.21.15.17:0/1090176662 >> v1:172.21.15.119:6789/0 conn(0x7f71500a4a90 legacy=0x7f71500a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 1 -- 172.21.15.17:0/1090176662 >> v1:172.21.15.17:6789/0 conn(0x7f71500a59d0 legacy=0x7f715013f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.494+0000 7f714dd9b640 1 -- 172.21.15.17:0/1090176662 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71500a1ce0 con 0x7f71500a5580 2024-09-15T21:37:28.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714dd9b640 1 --2- 172.21.15.17:0/1090176662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f715013cd10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71440097f0 tx=0x7f714401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b123afb2773be3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:28.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 1 -- 172.21.15.17:0/1090176662 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440b0070 con 0x7f71500a5580 2024-09-15T21:37:28.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:28.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: dump: 2024-09-15T21:37:28.510 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: _finish_auth 0 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 1 -- 172.21.15.17:0/1090176662 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71440ab600 con 0x7f71500a5580 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 5 monclient: authenticate success, global_id 204510 2024-09-15T21:37:28.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 1 -- 172.21.15.17:0/1090176662 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440ab9d0 con 0x7f71500a5580 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f714d59a640 10 monclient: dump: 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.512 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 4 set_mon_vals no callback set 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 10 monclient: shutdown 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 -- 172.21.15.17:0/1090176662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 msgr2=0x7f715013cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 --2- 172.21.15.17:0/1090176662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f715013cd10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71440097f0 tx=0x7f714401c3c0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 -- 172.21.15.17:0/1090176662 shutdown_connections 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 --2- 172.21.15.17:0/1090176662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f715013cd10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 -- 172.21.15.17:0/1090176662 >> 172.21.15.17:0/1090176662 conn(0x7f71500aabb0 msgr2=0x7f71500a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 -- 172.21.15.17:0/1090176662 shutdown_connections 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 -- 172.21.15.17:0/1090176662 wait complete. 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 10 monclient: build_initial_monmap 2024-09-15T21:37:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 10 monclient: monmap: 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 Processor -- start 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 1 -- start start 2024-09-15T21:37:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.498+0000 7f71549a3640 10 monclient: init 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71500a4a90 0x7f71501373b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): picked mon.noname-b con 0x7f71500a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f71501378f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): picked mon.noname-a con 0x7f71500a5580 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71500a59d0 0x7f715013cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): picked mon.noname-c con 0x7f71500a59d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f71501437c0 con 0x7f71500a5580 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f715006f290 con 0x7f71500a4a90 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f7150143940 con 0x7f71500a59d0 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient(hunting): authenticate will time out at 2171.900511s 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f71501378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71500a59d0 0x7f715013cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f71501378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35404/0 (socket says 172.21.15.17:35404) 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 -- 172.21.15.17:0/3545020015 learned_addr learned my addr 172.21.15.17:0/3545020015 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): get_auth_request con 0x7f71500a5580 auth_method 0 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): get_auth_request con 0x7f71500a59d0 auth_method 0 2024-09-15T21:37:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71500a4a90 0x7f71501373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 10 monclient(hunting): get_auth_request con 0x7f71500a4a90 auth_method 0 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient(hunting): handle_auth_done global_id 204516 payload 824 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 -- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71500a59d0 msgr2=0x7f715013cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71500a59d0 0x7f715013cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 -- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71500a4a90 msgr2=0x7f71501373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71500a4a90 0x7f71501373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient: _finish_hunting 0 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 monclient: found mon.noname-a 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 -- 172.21.15.17:0/3545020015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715013fd80 con 0x7f71500a5580 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714ed9d640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71500a59d0 0x7f715013cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714dd9b640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f71501378f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7144018fd0 tx=0x7f7144019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440b8020 con 0x7f71500a5580 2024-09-15T21:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: dump: 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: _finish_auth 0 2024-09-15T21:37:28.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71440b4410 con 0x7f71500a5580 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 5 monclient: authenticate success, global_id 204516 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _renew_subs 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71501206e0 con 0x7f71500a5580 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440b0040 con 0x7f71500a5580 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 10 monclient: dump: 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.520 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714cd99640 4 set_mon_vals no callback set 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _renew_subs 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715013f390 con 0x7f71500a5580 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:28.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71500a4e70 con 0x7f71500a5580 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f71440bb020 con 0x7f71500a5580 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f713003b9c0 0x7f713003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f71440ea400 con 0x7f71500a5580 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f713003b9c0 0x7f713003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 10 monclient: get_auth_request con 0x7f713003b9c0 auth_method 0 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.502+0000 7f714e59c640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f713003b9c0 0x7f713003de80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7150138490 tx=0x7f7140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.506+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f71440c2030 con 0x7f71500a5580 2024-09-15T21:37:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.506+0000 7f713effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:28.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.506+0000 7f713effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.742+0000 7f71549a3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T21:37:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.742+0000 7f71549a3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T21:37:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.742+0000 7f71549a3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.742+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f71500a9860 con 0x7f71500a5580 2024-09-15T21:37:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f713effd640 1 -- 172.21.15.17:0/3545020015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f7144016070 con 0x7f71500a5580 2024-09-15T21:37:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f713effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T21:37:28.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f713effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:28.751 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":178,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","modified":"2024-09-15T21:16:49.663269Z","created":"2024-09-15T21:16:49.663269Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3301","nonce":0},{"type":"v1","addr":"172.21.15.17:6790","nonce":0}]},"addr":"172.21.15.17:6790/0","public_addr":"172.21.15.17:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:37:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f713003b9c0 msgr2=0x7f713003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f713003b9c0 0x7f713003de80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7150138490 tx=0x7f7140009040 comp rx=0 tx=0).stop 2024-09-15T21:37:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 10 monclient: shutdown 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 msgr2=0x7f71501378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f71501378f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7144018fd0 tx=0x7f7144019160 comp rx=0 tx=0).stop 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 shutdown_connections 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f713003b9c0 0x7f713003de80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f71500a59d0 0x7f715013cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71500a5580 0x7f71501378f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 --2- 172.21.15.17:0/3545020015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71500a4a90 0x7f71501373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 >> 172.21.15.17:0/3545020015 conn(0x7f71500aabb0 msgr2=0x7f71500a7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 shutdown_connections 2024-09-15T21:37:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.746+0000 7f71549a3640 1 -- 172.21.15.17:0/3545020015 wait complete. 2024-09-15T21:37:28.770 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-15T21:37:28.771 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:37:28.780 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:37:28.780 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:37:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.886+0000 7fca0f7f4640 10 monclient: get_monmap_and_config 2024-09-15T21:37:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.886+0000 7fca0f7f4640 10 monclient: build_initial_monmap 2024-09-15T21:37:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.886+0000 7fca0f7f4640 10 monclient: monmap: 2024-09-15T21:37:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:28.892909+0000 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:28.892909+0000 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:28.894 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.886+0000 7fca0f7f4640 1 Processor -- start 2024-09-15T21:37:28.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.886+0000 7fca0f7f4640 1 -- start start 2024-09-15T21:37:28.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.886+0000 7fca0f7f4640 10 monclient: init 2024-09-15T21:37:28.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:28.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fca080a4a90 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca080a5580 0x7fca0813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): picked mon.noname-c con 0x7fca080a5580 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca080a59d0 0x7fca0813f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): picked mon.noname-a con 0x7fca080a59d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca080a2c30 con 0x7fca080a4a90 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fca080a0f40 con 0x7fca080a59d0 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fca080a19b0 con 0x7fca080a5580 2024-09-15T21:37:28.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient(hunting): authenticate will time out at 2172.288269s 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca080a59d0 0x7fca0813f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca080a59d0 0x7fca0813f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35406/0 (socket says 172.21.15.17:35406) 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 -- 172.21.15.17:0/2803074584 learned_addr learned my addr 172.21.15.17:0/2803074584 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): get_auth_request con 0x7fca080a59d0 auth_method 0 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:28.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0cd8f640 1 --2- 172.21.15.17:0/2803074584 >> v2:172.21.15.119:3300/0 conn(0x7fca080a5580 0x7fca0813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0cd8f640 10 monclient(hunting): get_auth_request con 0x7fca080a5580 auth_method 0 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0cd8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0cd8f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0cd8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient(hunting): handle_auth_done global_id 204519 payload 274 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 -- 172.21.15.17:0/2803074584 >> v2:172.21.15.119:3300/0 conn(0x7fca080a5580 msgr2=0x7fca0813cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 --2- 172.21.15.17:0/2803074584 >> v2:172.21.15.119:3300/0 conn(0x7fca080a5580 0x7fca0813cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 -- 172.21.15.17:0/2803074584 >> v1:172.21.15.119:6789/0 conn(0x7fca080a4a90 legacy=0x7fca080a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient: _finish_hunting 0 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 monclient: found mon.noname-a 2024-09-15T21:37:28.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 -- 172.21.15.17:0/2803074584 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca080a1c90 con 0x7fca080a59d0 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0dd91640 1 --2- 172.21.15.17:0/2803074584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca0813f260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fca00009870 tx=0x7fca0001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29838f505289a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 1 -- 172.21.15.17:0/2803074584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000b0070 con 0x7fca080a59d0 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: dump: 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.907 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: _finish_auth 0 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: _check_auth_tickets 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 1 -- 172.21.15.17:0/2803074584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca000ab600 con 0x7fca080a59d0 2024-09-15T21:37:28.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 1 -- 172.21.15.17:0/2803074584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000ab9d0 con 0x7fca080a59d0 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fc9fffff640 10 monclient: dump: 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 5 monclient: authenticate success, global_id 204519 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 4 set_mon_vals no callback set 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 10 monclient: shutdown 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- 172.21.15.17:0/2803074584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 msgr2=0x7fca0813f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/2803074584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca0813f260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fca00009870 tx=0x7fca0001c260 comp rx=0 tx=0).stop 2024-09-15T21:37:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- 172.21.15.17:0/2803074584 shutdown_connections 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/2803074584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca0813f260 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/2803074584 >> v2:172.21.15.119:3300/0 conn(0x7fca080a5580 0x7fca0813cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- 172.21.15.17:0/2803074584 >> 172.21.15.17:0/2803074584 conn(0x7fca080aabb0 msgr2=0x7fca080a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- 172.21.15.17:0/2803074584 shutdown_connections 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.890+0000 7fca0f7f4640 1 -- 172.21.15.17:0/2803074584 wait complete. 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: build_initial_monmap 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: monmap: 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 Processor -- start 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- start start 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: init 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 0x7fca08134b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): picked mon.noname-c con 0x7fca080a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca080a5580 0x7fca081350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): picked mon.noname-b con 0x7fca080a5580 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca081cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): picked mon.noname-a con 0x7fca080a59d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fca081421d0 con 0x7fca080a59d0 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca0806f2a0 con 0x7fca080a5580 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fca08142350 con 0x7fca080a4a90 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient(hunting): authenticate will time out at 2172.293420s 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 0x7fca08134b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca081cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca081cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35422/0 (socket says 172.21.15.17:35422) 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 1 -- 172.21.15.17:0/4262193753 learned_addr learned my addr 172.21.15.17:0/4262193753 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 0x7fca08134b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57400/0 (socket says 172.21.15.17:57400) 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): get_auth_request con 0x7fca080a59d0 auth_method 0 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): get_auth_request con 0x7fca080a4a90 auth_method 0 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0cd8f640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca080a5580 0x7fca081350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0cd8f640 10 monclient(hunting): get_auth_request con 0x7fca080a5580 auth_method 0 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0cd8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0cd8f640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0cd8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:28.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient(hunting): handle_auth_done global_id 234137 payload 824 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 -- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca080a5580 msgr2=0x7fca081350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca080a5580 0x7fca081350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 -- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 msgr2=0x7fca081cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca081cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient: _finish_hunting 0 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 monclient: found mon.noname-c 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 -- 172.21.15.17:0/4262193753 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0813c630 con 0x7fca080a4a90 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0dd91640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca081cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0d590640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 0x7fca08134b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c780 tx=0x7fca0400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04017070 con 0x7fca080a4a90 2024-09-15T21:37:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: dump: 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 5 monclient: authenticate success, global_id 234137 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca04003db0 con 0x7fca080a4a90 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040052b0 con 0x7fca080a4a90 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _renew_subs 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0813da80 con 0x7fca080a4a90 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9ff7fe640 4 set_mon_vals no callback set 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 10 monclient: dump: 2024-09-15T21:37:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _renew_subs 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca080a1280 con 0x7fca080a4a90 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca080a4e70 con 0x7fca080a4a90 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fca0401a070 con 0x7fca080a4a90 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc9ec03b9c0 0x7fc9ec03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fca0404fbe0 con 0x7fca080a4a90 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.894+0000 7fca0cd8f640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc9ec03b9c0 0x7fc9ec03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.898+0000 7fca0cd8f640 10 monclient: get_auth_request con 0x7fc9ec03b9c0 auth_method 0 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.898+0000 7fca0cd8f640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc9ec03b9c0 0x7fc9ec03de80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fca080a3ae0 tx=0x7fc9f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.902+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fca0401a360 con 0x7fca080a4a90 2024-09-15T21:37:28.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.902+0000 7fc9fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:28.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:28.902+0000 7fc9fdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:29.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.066+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca0813c280 con 0x7fc9ec03b9c0 2024-09-15T21:37:29.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fc9fdffb640 1 -- 172.21.15.17:0/4262193753 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+77978 (secure 0 0 0) 0x7fca0813c280 con 0x7fc9ec03b9c0 2024-09-15T21:37:29.076 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:37:29.076 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc9ec03b9c0 msgr2=0x7fc9ec03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc9ec03b9c0 0x7fc9ec03de80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fca080a3ae0 tx=0x7fc9f8008040 comp rx=0 tx=0).stop 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 10 monclient: shutdown 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 msgr2=0x7fca08134b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 0x7fca08134b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c780 tx=0x7fca0400cc40 comp rx=0 tx=0).stop 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 shutdown_connections 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc9ec03b9c0 0x7fc9ec03de80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080a59d0 0x7fca081cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca080a5580 0x7fca081350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 --2- 172.21.15.17:0/4262193753 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fca080a4a90 0x7fca08134b70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 >> 172.21.15.17:0/4262193753 conn(0x7fca080aabb0 msgr2=0x7fca080a9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:29.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 shutdown_connections 2024-09-15T21:37:29.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.070+0000 7fca0f7f4640 1 -- 172.21.15.17:0/4262193753 wait complete. 2024-09-15T21:37:29.092 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":853,"stamp":"2024-09-15T21:37:27.529528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":215504,"kb_used_data":20056,"kb_used_omap":9,"kb_used_meta":195318,"kb_avail":566015536,"statfs":{"total":579820584960,"available":579599908864,"internally_reserved":0,"allocated":20537344,"data_stored":17290996,"data_compressed":1361822,"data_compressed_allocated":2695168,"data_compressed_original":6922240,"omap_allocated":9567,"internal_metadata":200006305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003545"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":302,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373667+0000","last_change":"2024-09-15T21:23:54.324070+0000","last_active":"2024-09-15T21:34:21.373667+0000","last_peered":"2024-09-15T21:34:21.373667+0000","last_clean":"2024-09-15T21:34:21.373667+0000","last_became_active":"2024-09-15T21:23:54.320552+0000","last_became_peered":"2024-09-15T21:23:54.320552+0000","last_unstale":"2024-09-15T21:34:21.373667+0000","last_undegraded":"2024-09-15T21:34:21.373667+0000","last_fullsized":"2024-09-15T21:34:21.373667+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:55.900283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256773+0000","last_change":"2024-09-15T21:23:54.301950+0000","last_active":"2024-09-15T21:34:46.256773+0000","last_peered":"2024-09-15T21:34:46.256773+0000","last_clean":"2024-09-15T21:34:46.256773+0000","last_became_active":"2024-09-15T21:23:54.301596+0000","last_became_peered":"2024-09-15T21:23:54.301596+0000","last_unstale":"2024-09-15T21:34:46.256773+0000","last_undegraded":"2024-09-15T21:34:46.256773+0000","last_fullsized":"2024-09-15T21:34:46.256773+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:07.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.251301+0000","last_change":"2024-09-15T21:23:54.314550+0000","last_active":"2024-09-15T21:34:25.251301+0000","last_peered":"2024-09-15T21:34:25.251301+0000","last_clean":"2024-09-15T21:34:25.251301+0000","last_became_active":"2024-09-15T21:23:54.313525+0000","last_became_peered":"2024-09-15T21:23:54.313525+0000","last_unstale":"2024-09-15T21:34:25.251301+0000","last_undegraded":"2024-09-15T21:34:25.251301+0000","last_fullsized":"2024-09-15T21:34:25.251301+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:21:52.088930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256725+0000","last_change":"2024-09-15T21:23:54.314288+0000","last_active":"2024-09-15T21:34:46.256725+0000","last_peered":"2024-09-15T21:34:46.256725+0000","last_clean":"2024-09-15T21:34:46.256725+0000","last_became_active":"2024-09-15T21:23:54.313839+0000","last_became_peered":"2024-09-15T21:23:54.313839+0000","last_unstale":"2024-09-15T21:34:46.256725+0000","last_undegraded":"2024-09-15T21:34:46.256725+0000","last_fullsized":"2024-09-15T21:34:46.256725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:09.847621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":231,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:51.257506+0000","last_change":"2024-09-15T21:23:55.902295+0000","last_active":"2024-09-15T21:34:51.257506+0000","last_peered":"2024-09-15T21:34:51.257506+0000","last_clean":"2024-09-15T21:34:51.257506+0000","last_became_active":"2024-09-15T21:23:55.901252+0000","last_became_peered":"2024-09-15T21:23:55.901252+0000","last_unstale":"2024-09-15T21:34:51.257506+0000","last_undegraded":"2024-09-15T21:34:51.257506+0000","last_fullsized":"2024-09-15T21:34:51.257506+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:55.224562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373691+0000","last_change":"2024-09-15T21:23:54.323472+0000","last_active":"2024-09-15T21:34:21.373691+0000","last_peered":"2024-09-15T21:34:21.373691+0000","last_clean":"2024-09-15T21:34:21.373691+0000","last_became_active":"2024-09-15T21:23:54.318732+0000","last_became_peered":"2024-09-15T21:23:54.318732+0000","last_unstale":"2024-09-15T21:34:21.373691+0000","last_undegraded":"2024-09-15T21:34:21.373691+0000","last_fullsized":"2024-09-15T21:34:21.373691+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:32:38.556252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258088+0000","last_change":"2024-09-15T21:23:54.326082+0000","last_active":"2024-09-15T21:34:46.258088+0000","last_peered":"2024-09-15T21:34:46.258088+0000","last_clean":"2024-09-15T21:34:46.258088+0000","last_became_active":"2024-09-15T21:23:54.325406+0000","last_became_peered":"2024-09-15T21:23:54.325406+0000","last_unstale":"2024-09-15T21:34:46.258088+0000","last_undegraded":"2024-09-15T21:34:46.258088+0000","last_fullsized":"2024-09-15T21:34:46.258088+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:51:57.618330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380779+0000","last_change":"2024-09-15T21:23:54.301728+0000","last_active":"2024-09-15T21:33:57.380779+0000","last_peered":"2024-09-15T21:33:57.380779+0000","last_clean":"2024-09-15T21:33:57.380779+0000","last_became_active":"2024-09-15T21:23:54.300781+0000","last_became_peered":"2024-09-15T21:23:54.300781+0000","last_unstale":"2024-09-15T21:33:57.380779+0000","last_undegraded":"2024-09-15T21:33:57.380779+0000","last_fullsized":"2024-09-15T21:33:57.380779+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:02.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:06.250727+0000","last_change":"2024-09-15T21:23:54.308826+0000","last_active":"2024-09-15T21:34:06.250727+0000","last_peered":"2024-09-15T21:34:06.250727+0000","last_clean":"2024-09-15T21:34:06.250727+0000","last_became_active":"2024-09-15T21:23:54.307998+0000","last_became_peered":"2024-09-15T21:23:54.307998+0000","last_unstale":"2024-09-15T21:34:06.250727+0000","last_undegraded":"2024-09-15T21:34:06.250727+0000","last_fullsized":"2024-09-15T21:34:06.250727+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:32.478200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":216,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:31.255681+0000","last_change":"2024-09-15T21:23:54.311062+0000","last_active":"2024-09-15T21:34:31.255681+0000","last_peered":"2024-09-15T21:34:31.255681+0000","last_clean":"2024-09-15T21:34:31.255681+0000","last_became_active":"2024-09-15T21:23:54.310738+0000","last_became_peered":"2024-09-15T21:23:54.310738+0000","last_unstale":"2024-09-15T21:34:31.255681+0000","last_undegraded":"2024-09-15T21:34:31.255681+0000","last_fullsized":"2024-09-15T21:34:31.255681+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:17.678319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":232,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.253623+0000","last_change":"2024-09-15T21:23:54.312726+0000","last_active":"2024-09-15T21:34:21.253623+0000","last_peered":"2024-09-15T21:34:21.253623+0000","last_clean":"2024-09-15T21:34:21.253623+0000","last_became_active":"2024-09-15T21:23:54.311958+0000","last_became_peered":"2024-09-15T21:23:54.311958+0000","last_unstale":"2024-09-15T21:34:21.253623+0000","last_undegraded":"2024-09-15T21:34:21.253623+0000","last_fullsized":"2024-09-15T21:34:21.253623+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:15.944561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384735+0000","last_change":"2024-09-15T21:23:54.310984+0000","last_active":"2024-09-15T21:34:22.384735+0000","last_peered":"2024-09-15T21:34:22.384735+0000","last_clean":"2024-09-15T21:34:22.384735+0000","last_became_active":"2024-09-15T21:23:54.309483+0000","last_became_peered":"2024-09-15T21:23:54.309483+0000","last_unstale":"2024-09-15T21:34:22.384735+0000","last_undegraded":"2024-09-15T21:34:22.384735+0000","last_fullsized":"2024-09-15T21:34:22.384735+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:26.416656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384680+0000","last_change":"2024-09-15T21:23:54.313402+0000","last_active":"2024-09-15T21:34:22.384680+0000","last_peered":"2024-09-15T21:34:22.384680+0000","last_clean":"2024-09-15T21:34:22.384680+0000","last_became_active":"2024-09-15T21:23:54.312375+0000","last_became_peered":"2024-09-15T21:23:54.312375+0000","last_unstale":"2024-09-15T21:34:22.384680+0000","last_undegraded":"2024-09-15T21:34:22.384680+0000","last_fullsized":"2024-09-15T21:34:22.384680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:29.568805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":304,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373629+0000","last_change":"2024-09-15T21:23:54.319437+0000","last_active":"2024-09-15T21:34:21.373629+0000","last_peered":"2024-09-15T21:34:21.373629+0000","last_clean":"2024-09-15T21:34:21.373629+0000","last_became_active":"2024-09-15T21:23:54.318494+0000","last_became_peered":"2024-09-15T21:23:54.318494+0000","last_unstale":"2024-09-15T21:34:21.373629+0000","last_undegraded":"2024-09-15T21:34:21.373629+0000","last_fullsized":"2024-09-15T21:34:21.373629+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:44:21.410329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250527+0000","last_change":"2024-09-15T21:23:55.306377+0000","last_active":"2024-09-15T21:34:20.250527+0000","last_peered":"2024-09-15T21:34:20.250527+0000","last_clean":"2024-09-15T21:34:20.250527+0000","last_became_active":"2024-09-15T21:23:55.305236+0000","last_became_peered":"2024-09-15T21:23:55.305236+0000","last_unstale":"2024-09-15T21:34:20.250527+0000","last_undegraded":"2024-09-15T21:34:20.250527+0000","last_fullsized":"2024-09-15T21:34:20.250527+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:30.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380708+0000","last_change":"2024-09-15T21:23:54.307012+0000","last_active":"2024-09-15T21:33:57.380708+0000","last_peered":"2024-09-15T21:33:57.380708+0000","last_clean":"2024-09-15T21:33:57.380708+0000","last_became_active":"2024-09-15T21:23:54.305792+0000","last_became_peered":"2024-09-15T21:23:54.305792+0000","last_unstale":"2024-09-15T21:33:57.380708+0000","last_undegraded":"2024-09-15T21:33:57.380708+0000","last_fullsized":"2024-09-15T21:33:57.380708+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:33:47.178490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":533,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:01.251383+0000","last_change":"2024-09-15T21:23:54.325895+0000","last_active":"2024-09-15T21:34:01.251383+0000","last_peered":"2024-09-15T21:34:01.251383+0000","last_clean":"2024-09-15T21:34:01.251383+0000","last_became_active":"2024-09-15T21:23:54.325552+0000","last_became_peered":"2024-09-15T21:23:54.325552+0000","last_unstale":"2024-09-15T21:34:01.251383+0000","last_undegraded":"2024-09-15T21:34:01.251383+0000","last_fullsized":"2024-09-15T21:34:01.251383+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:57.451100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":451,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252971+0000","last_change":"2024-09-15T21:23:54.326302+0000","last_active":"2024-09-15T21:34:21.252971+0000","last_peered":"2024-09-15T21:34:21.252971+0000","last_clean":"2024-09-15T21:34:21.252971+0000","last_became_active":"2024-09-15T21:23:54.316513+0000","last_became_peered":"2024-09-15T21:23:54.316513+0000","last_unstale":"2024-09-15T21:34:21.252971+0000","last_undegraded":"2024-09-15T21:34:21.252971+0000","last_fullsized":"2024-09-15T21:34:21.252971+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:21.773590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:51.249994+0000","last_change":"2024-09-15T21:23:54.308012+0000","last_active":"2024-09-15T21:33:51.249994+0000","last_peered":"2024-09-15T21:33:51.249994+0000","last_clean":"2024-09-15T21:33:51.249994+0000","last_became_active":"2024-09-15T21:23:54.307605+0000","last_became_peered":"2024-09-15T21:23:54.307605+0000","last_unstale":"2024-09-15T21:33:51.249994+0000","last_undegraded":"2024-09-15T21:33:51.249994+0000","last_fullsized":"2024-09-15T21:33:51.249994+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:36.197287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250456+0000","last_change":"2024-09-15T21:23:54.308207+0000","last_active":"2024-09-15T21:34:20.250456+0000","last_peered":"2024-09-15T21:34:20.250456+0000","last_clean":"2024-09-15T21:34:20.250456+0000","last_became_active":"2024-09-15T21:23:54.307364+0000","last_became_peered":"2024-09-15T21:23:54.307364+0000","last_unstale":"2024-09-15T21:34:20.250456+0000","last_undegraded":"2024-09-15T21:34:20.250456+0000","last_fullsized":"2024-09-15T21:34:20.250456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:31.761043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":461,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252911+0000","last_change":"2024-09-15T21:23:54.322590+0000","last_active":"2024-09-15T21:34:21.252911+0000","last_peered":"2024-09-15T21:34:21.252911+0000","last_clean":"2024-09-15T21:34:21.252911+0000","last_became_active":"2024-09-15T21:23:54.314394+0000","last_became_peered":"2024-09-15T21:23:54.314394+0000","last_unstale":"2024-09-15T21:34:21.252911+0000","last_undegraded":"2024-09-15T21:34:21.252911+0000","last_fullsized":"2024-09-15T21:34:21.252911+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:54.033640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":439,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384782+0000","last_change":"2024-09-15T21:23:55.901581+0000","last_active":"2024-09-15T21:34:22.384782+0000","last_peered":"2024-09-15T21:34:22.384782+0000","last_clean":"2024-09-15T21:34:22.384782+0000","last_became_active":"2024-09-15T21:23:55.900571+0000","last_became_peered":"2024-09-15T21:23:55.900571+0000","last_unstale":"2024-09-15T21:34:22.384782+0000","last_undegraded":"2024-09-15T21:34:22.384782+0000","last_fullsized":"2024-09-15T21:34:22.384782+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:22.834885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:26.254894+0000","last_change":"2024-09-15T21:23:54.311459+0000","last_active":"2024-09-15T21:34:26.254894+0000","last_peered":"2024-09-15T21:34:26.254894+0000","last_clean":"2024-09-15T21:34:26.254894+0000","last_became_active":"2024-09-15T21:23:54.310902+0000","last_became_peered":"2024-09-15T21:23:54.310902+0000","last_unstale":"2024-09-15T21:34:26.254894+0000","last_undegraded":"2024-09-15T21:34:26.254894+0000","last_fullsized":"2024-09-15T21:34:26.254894+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:32.505577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:47.388767+0000","last_change":"2024-09-15T21:23:54.295908+0000","last_active":"2024-09-15T21:34:47.388767+0000","last_peered":"2024-09-15T21:34:47.388767+0000","last_clean":"2024-09-15T21:34:47.388767+0000","last_became_active":"2024-09-15T21:23:54.294899+0000","last_became_peered":"2024-09-15T21:23:54.294899+0000","last_unstale":"2024-09-15T21:34:47.388767+0000","last_undegraded":"2024-09-15T21:34:47.388767+0000","last_fullsized":"2024-09-15T21:34:47.388767+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:02.582888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258125+0000","last_change":"2024-09-15T21:23:54.302593+0000","last_active":"2024-09-15T21:34:46.258125+0000","last_peered":"2024-09-15T21:34:46.258125+0000","last_clean":"2024-09-15T21:34:46.258125+0000","last_became_active":"2024-09-15T21:23:54.301715+0000","last_became_peered":"2024-09-15T21:23:54.301715+0000","last_unstale":"2024-09-15T21:34:46.258125+0000","last_undegraded":"2024-09-15T21:34:46.258125+0000","last_fullsized":"2024-09-15T21:34:46.258125+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:51.716170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607796,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38676,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333164,"statfs":{"total":96636764160,"available":96597159936,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34516,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337324,"statfs":{"total":96636764160,"available":96601419776,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640497,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34512,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337328,"statfs":{"total":96636764160,"available":96601423872,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640498,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38672,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333168,"statfs":{"total":96636764160,"available":96597164032,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640499,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:37:29.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:37:29.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.198+0000 7ff89bc3d640 10 monclient: get_monmap_and_config 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.198+0000 7ff89bc3d640 10 monclient: build_initial_monmap 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.198+0000 7ff89bc3d640 10 monclient: monmap: 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:29.205406+0000 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:29.205406+0000 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:29.207 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.198+0000 7ff89bc3d640 1 Processor -- start 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 1 -- start start 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient: init 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff89409d400 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff89409def0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff89413b1d0 0x7ff89413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient(hunting): picked mon.noname-a con 0x7ff89413b1d0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8940a3130 con 0x7ff89409d400 2024-09-15T21:37:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff89409e7a0 con 0x7ff89409def0 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff8940a1bc0 con 0x7ff89413b1d0 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient(hunting): authenticate will time out at 2172.600739s 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8999d9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff89409d400 0x7ff89409d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45444/0 (socket says 172.21.15.17:45444) 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8999d9640 1 -- 172.21.15.17:0/854394266 learned_addr learned my addr 172.21.15.17:0/854394266 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 1 --2- 172.21.15.17:0/854394266 >> v2:172.21.15.17:3300/0 conn(0x7ff89413b1d0 0x7ff89413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): get_auth_request con 0x7ff89413b1d0 auth_method 0 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient(hunting): handle_auth_done global_id 204525 payload 274 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 1 -- 172.21.15.17:0/854394266 >> v1:172.21.15.119:6789/0 conn(0x7ff89409def0 legacy=0x7ff894139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 1 -- 172.21.15.17:0/854394266 >> v1:172.21.15.17:6789/0 conn(0x7ff89409d400 legacy=0x7ff89409d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient: _finish_hunting 0 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 1 monclient: found mon.noname-a 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:29.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 1 -- 172.21.15.17:0/854394266 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8940a0320 con 0x7ff89413b1d0 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89a1da640 1 --2- 172.21.15.17:0/854394266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89413b1d0 0x7ff89413d5c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009d90 tx=0x7ff88c01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be26080e6d453bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 1 -- 172.21.15.17:0/854394266 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0a6700 con 0x7ff89413b1d0 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: dump: 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: _finish_auth 0 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: _check_auth_tickets 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 5 monclient: authenticate success, global_id 204525 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 1 -- 172.21.15.17:0/854394266 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c0a68a0 con 0x7ff89413b1d0 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff89bc3d640 4 set_mon_vals no callback set 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 1 -- 172.21.15.17:0/854394266 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0a6c70 con 0x7ff89413b1d0 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.202+0000 7ff8989d7640 10 monclient: dump: 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: shutdown 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/854394266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89413b1d0 msgr2=0x7ff89413d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/854394266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89413b1d0 0x7ff89413d5c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009d90 tx=0x7ff88c01c830 comp rx=0 tx=0).stop 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/854394266 shutdown_connections 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/854394266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89413b1d0 0x7ff89413d5c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/854394266 >> 172.21.15.17:0/854394266 conn(0x7ff8940a66a0 msgr2=0x7ff8940a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/854394266 shutdown_connections 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/854394266 wait complete. 2024-09-15T21:37:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: build_initial_monmap 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: monmap: 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 Processor -- start 2024-09-15T21:37:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- start start 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: init 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 0x7ff894134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): picked mon.noname-a con 0x7ff89409d400 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff89409def0 0x7ff894134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): picked mon.noname-c con 0x7ff89409def0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89413b1d0 0x7ff89406f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): picked mon.noname-b con 0x7ff89413b1d0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940a3490 con 0x7ff89409d400 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff894140940 con 0x7ff89413b1d0 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7ff894140ac0 con 0x7ff89409def0 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient(hunting): authenticate will time out at 2172.606027s 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 0x7ff894134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff89409def0 0x7ff894134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 0x7ff894134930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35436/0 (socket says 172.21.15.17:35436) 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 -- 172.21.15.17:0/1926676054 learned_addr learned my addr 172.21.15.17:0/1926676054 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): get_auth_request con 0x7ff89409d400 auth_method 0 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): get_auth_request con 0x7ff89409def0 auth_method 0 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89a1da640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89413b1d0 0x7ff89406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89a1da640 10 monclient(hunting): get_auth_request con 0x7ff89413b1d0 auth_method 0 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89a1da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89a1da640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89a1da640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient(hunting): handle_auth_done global_id 204531 payload 824 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 -- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff89409def0 msgr2=0x7ff894134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff89409def0 0x7ff894134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 -- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89413b1d0 msgr2=0x7ff89406f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89413b1d0 0x7ff89406f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient: _finish_hunting 0 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 monclient: found mon.noname-a 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 -- 172.21.15.17:0/1926676054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89413b960 con 0x7ff89409d400 2024-09-15T21:37:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8991d8640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff89409def0 0x7ff894134e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff8999d9640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 0x7ff894134930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff89000c4d0 tx=0x7ff89000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890017070 con 0x7ff89409d400 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: dump: 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff890005cf0 con 0x7ff89409d400 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 5 monclient: authenticate success, global_id 204531 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _renew_subs 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89413ae40 con 0x7ff89409d400 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890007270 con 0x7ff89409d400 2024-09-15T21:37:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 10 monclient: dump: 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88bfff640 4 set_mon_vals no callback set 2024-09-15T21:37:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _renew_subs 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89413a440 con 0x7ff89409d400 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7ff89001a070 con 0x7ff89409d400 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89409d7e0 con 0x7ff89409d400 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.206+0000 7ff88a7fc640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff87803b9f0 0x7ff87803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.210+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7ff89004f920 con 0x7ff89409d400 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.210+0000 7ff8991d8640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff87803b9f0 0x7ff87803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.210+0000 7ff8991d8640 10 monclient: get_auth_request con 0x7ff87803b9f0 auth_method 0 2024-09-15T21:37:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.210+0000 7ff8991d8640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff87803b9f0 0x7ff87803deb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff8940703d0 tx=0x7ff884009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.214+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7ff89413b960 con 0x7ff89409d400 2024-09-15T21:37:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.214+0000 7ff88a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.214+0000 7ff88a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.382+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff894070700 con 0x7ff87803b9f0 2024-09-15T21:37:29.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff88a7fc640 1 -- 172.21.15.17:0/1926676054 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+77978 (secure 0 0 0) 0x7ff894070700 con 0x7ff87803b9f0 2024-09-15T21:37:29.391 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:37:29.392 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:37:29.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff87803b9f0 msgr2=0x7ff87803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff87803b9f0 0x7ff87803deb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff8940703d0 tx=0x7ff884009040 comp rx=0 tx=0).stop 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 10 monclient: shutdown 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 msgr2=0x7ff894134930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 0x7ff894134930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff89000c4d0 tx=0x7ff89000c990 comp rx=0 tx=0).stop 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 shutdown_connections 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7ff87803b9f0 0x7ff87803deb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89413b1d0 0x7ff89406f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7ff89409def0 0x7ff894134e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 --2- 172.21.15.17:0/1926676054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff89409d400 0x7ff894134930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 >> 172.21.15.17:0/1926676054 conn(0x7ff8940a66a0 msgr2=0x7ff8940a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 shutdown_connections 2024-09-15T21:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.386+0000 7ff89bc3d640 1 -- 172.21.15.17:0/1926676054 wait complete. 2024-09-15T21:37:29.410 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":853,"stamp":"2024-09-15T21:37:27.529528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":215504,"kb_used_data":20056,"kb_used_omap":9,"kb_used_meta":195318,"kb_avail":566015536,"statfs":{"total":579820584960,"available":579599908864,"internally_reserved":0,"allocated":20537344,"data_stored":17290996,"data_compressed":1361822,"data_compressed_allocated":2695168,"data_compressed_original":6922240,"omap_allocated":9567,"internal_metadata":200006305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003545"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":302,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373667+0000","last_change":"2024-09-15T21:23:54.324070+0000","last_active":"2024-09-15T21:34:21.373667+0000","last_peered":"2024-09-15T21:34:21.373667+0000","last_clean":"2024-09-15T21:34:21.373667+0000","last_became_active":"2024-09-15T21:23:54.320552+0000","last_became_peered":"2024-09-15T21:23:54.320552+0000","last_unstale":"2024-09-15T21:34:21.373667+0000","last_undegraded":"2024-09-15T21:34:21.373667+0000","last_fullsized":"2024-09-15T21:34:21.373667+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:55.900283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256773+0000","last_change":"2024-09-15T21:23:54.301950+0000","last_active":"2024-09-15T21:34:46.256773+0000","last_peered":"2024-09-15T21:34:46.256773+0000","last_clean":"2024-09-15T21:34:46.256773+0000","last_became_active":"2024-09-15T21:23:54.301596+0000","last_became_peered":"2024-09-15T21:23:54.301596+0000","last_unstale":"2024-09-15T21:34:46.256773+0000","last_undegraded":"2024-09-15T21:34:46.256773+0000","last_fullsized":"2024-09-15T21:34:46.256773+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:07.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.251301+0000","last_change":"2024-09-15T21:23:54.314550+0000","last_active":"2024-09-15T21:34:25.251301+0000","last_peered":"2024-09-15T21:34:25.251301+0000","last_clean":"2024-09-15T21:34:25.251301+0000","last_became_active":"2024-09-15T21:23:54.313525+0000","last_became_peered":"2024-09-15T21:23:54.313525+0000","last_unstale":"2024-09-15T21:34:25.251301+0000","last_undegraded":"2024-09-15T21:34:25.251301+0000","last_fullsized":"2024-09-15T21:34:25.251301+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:21:52.088930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256725+0000","last_change":"2024-09-15T21:23:54.314288+0000","last_active":"2024-09-15T21:34:46.256725+0000","last_peered":"2024-09-15T21:34:46.256725+0000","last_clean":"2024-09-15T21:34:46.256725+0000","last_became_active":"2024-09-15T21:23:54.313839+0000","last_became_peered":"2024-09-15T21:23:54.313839+0000","last_unstale":"2024-09-15T21:34:46.256725+0000","last_undegraded":"2024-09-15T21:34:46.256725+0000","last_fullsized":"2024-09-15T21:34:46.256725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:09.847621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":231,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:51.257506+0000","last_change":"2024-09-15T21:23:55.902295+0000","last_active":"2024-09-15T21:34:51.257506+0000","last_peered":"2024-09-15T21:34:51.257506+0000","last_clean":"2024-09-15T21:34:51.257506+0000","last_became_active":"2024-09-15T21:23:55.901252+0000","last_became_peered":"2024-09-15T21:23:55.901252+0000","last_unstale":"2024-09-15T21:34:51.257506+0000","last_undegraded":"2024-09-15T21:34:51.257506+0000","last_fullsized":"2024-09-15T21:34:51.257506+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:55.224562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373691+0000","last_change":"2024-09-15T21:23:54.323472+0000","last_active":"2024-09-15T21:34:21.373691+0000","last_peered":"2024-09-15T21:34:21.373691+0000","last_clean":"2024-09-15T21:34:21.373691+0000","last_became_active":"2024-09-15T21:23:54.318732+0000","last_became_peered":"2024-09-15T21:23:54.318732+0000","last_unstale":"2024-09-15T21:34:21.373691+0000","last_undegraded":"2024-09-15T21:34:21.373691+0000","last_fullsized":"2024-09-15T21:34:21.373691+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:32:38.556252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258088+0000","last_change":"2024-09-15T21:23:54.326082+0000","last_active":"2024-09-15T21:34:46.258088+0000","last_peered":"2024-09-15T21:34:46.258088+0000","last_clean":"2024-09-15T21:34:46.258088+0000","last_became_active":"2024-09-15T21:23:54.325406+0000","last_became_peered":"2024-09-15T21:23:54.325406+0000","last_unstale":"2024-09-15T21:34:46.258088+0000","last_undegraded":"2024-09-15T21:34:46.258088+0000","last_fullsized":"2024-09-15T21:34:46.258088+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:51:57.618330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380779+0000","last_change":"2024-09-15T21:23:54.301728+0000","last_active":"2024-09-15T21:33:57.380779+0000","last_peered":"2024-09-15T21:33:57.380779+0000","last_clean":"2024-09-15T21:33:57.380779+0000","last_became_active":"2024-09-15T21:23:54.300781+0000","last_became_peered":"2024-09-15T21:23:54.300781+0000","last_unstale":"2024-09-15T21:33:57.380779+0000","last_undegraded":"2024-09-15T21:33:57.380779+0000","last_fullsized":"2024-09-15T21:33:57.380779+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:02.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:06.250727+0000","last_change":"2024-09-15T21:23:54.308826+0000","last_active":"2024-09-15T21:34:06.250727+0000","last_peered":"2024-09-15T21:34:06.250727+0000","last_clean":"2024-09-15T21:34:06.250727+0000","last_became_active":"2024-09-15T21:23:54.307998+0000","last_became_peered":"2024-09-15T21:23:54.307998+0000","last_unstale":"2024-09-15T21:34:06.250727+0000","last_undegraded":"2024-09-15T21:34:06.250727+0000","last_fullsized":"2024-09-15T21:34:06.250727+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:32.478200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":216,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:31.255681+0000","last_change":"2024-09-15T21:23:54.311062+0000","last_active":"2024-09-15T21:34:31.255681+0000","last_peered":"2024-09-15T21:34:31.255681+0000","last_clean":"2024-09-15T21:34:31.255681+0000","last_became_active":"2024-09-15T21:23:54.310738+0000","last_became_peered":"2024-09-15T21:23:54.310738+0000","last_unstale":"2024-09-15T21:34:31.255681+0000","last_undegraded":"2024-09-15T21:34:31.255681+0000","last_fullsized":"2024-09-15T21:34:31.255681+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:17.678319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":232,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.253623+0000","last_change":"2024-09-15T21:23:54.312726+0000","last_active":"2024-09-15T21:34:21.253623+0000","last_peered":"2024-09-15T21:34:21.253623+0000","last_clean":"2024-09-15T21:34:21.253623+0000","last_became_active":"2024-09-15T21:23:54.311958+0000","last_became_peered":"2024-09-15T21:23:54.311958+0000","last_unstale":"2024-09-15T21:34:21.253623+0000","last_undegraded":"2024-09-15T21:34:21.253623+0000","last_fullsized":"2024-09-15T21:34:21.253623+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:15.944561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384735+0000","last_change":"2024-09-15T21:23:54.310984+0000","last_active":"2024-09-15T21:34:22.384735+0000","last_peered":"2024-09-15T21:34:22.384735+0000","last_clean":"2024-09-15T21:34:22.384735+0000","last_became_active":"2024-09-15T21:23:54.309483+0000","last_became_peered":"2024-09-15T21:23:54.309483+0000","last_unstale":"2024-09-15T21:34:22.384735+0000","last_undegraded":"2024-09-15T21:34:22.384735+0000","last_fullsized":"2024-09-15T21:34:22.384735+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:26.416656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384680+0000","last_change":"2024-09-15T21:23:54.313402+0000","last_active":"2024-09-15T21:34:22.384680+0000","last_peered":"2024-09-15T21:34:22.384680+0000","last_clean":"2024-09-15T21:34:22.384680+0000","last_became_active":"2024-09-15T21:23:54.312375+0000","last_became_peered":"2024-09-15T21:23:54.312375+0000","last_unstale":"2024-09-15T21:34:22.384680+0000","last_undegraded":"2024-09-15T21:34:22.384680+0000","last_fullsized":"2024-09-15T21:34:22.384680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:29.568805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":304,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373629+0000","last_change":"2024-09-15T21:23:54.319437+0000","last_active":"2024-09-15T21:34:21.373629+0000","last_peered":"2024-09-15T21:34:21.373629+0000","last_clean":"2024-09-15T21:34:21.373629+0000","last_became_active":"2024-09-15T21:23:54.318494+0000","last_became_peered":"2024-09-15T21:23:54.318494+0000","last_unstale":"2024-09-15T21:34:21.373629+0000","last_undegraded":"2024-09-15T21:34:21.373629+0000","last_fullsized":"2024-09-15T21:34:21.373629+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:44:21.410329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250527+0000","last_change":"2024-09-15T21:23:55.306377+0000","last_active":"2024-09-15T21:34:20.250527+0000","last_peered":"2024-09-15T21:34:20.250527+0000","last_clean":"2024-09-15T21:34:20.250527+0000","last_became_active":"2024-09-15T21:23:55.305236+0000","last_became_peered":"2024-09-15T21:23:55.305236+0000","last_unstale":"2024-09-15T21:34:20.250527+0000","last_undegraded":"2024-09-15T21:34:20.250527+0000","last_fullsized":"2024-09-15T21:34:20.250527+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:30.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380708+0000","last_change":"2024-09-15T21:23:54.307012+0000","last_active":"2024-09-15T21:33:57.380708+0000","last_peered":"2024-09-15T21:33:57.380708+0000","last_clean":"2024-09-15T21:33:57.380708+0000","last_became_active":"2024-09-15T21:23:54.305792+0000","last_became_peered":"2024-09-15T21:23:54.305792+0000","last_unstale":"2024-09-15T21:33:57.380708+0000","last_undegraded":"2024-09-15T21:33:57.380708+0000","last_fullsized":"2024-09-15T21:33:57.380708+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:33:47.178490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":533,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:01.251383+0000","last_change":"2024-09-15T21:23:54.325895+0000","last_active":"2024-09-15T21:34:01.251383+0000","last_peered":"2024-09-15T21:34:01.251383+0000","last_clean":"2024-09-15T21:34:01.251383+0000","last_became_active":"2024-09-15T21:23:54.325552+0000","last_became_peered":"2024-09-15T21:23:54.325552+0000","last_unstale":"2024-09-15T21:34:01.251383+0000","last_undegraded":"2024-09-15T21:34:01.251383+0000","last_fullsized":"2024-09-15T21:34:01.251383+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:57.451100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":451,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252971+0000","last_change":"2024-09-15T21:23:54.326302+0000","last_active":"2024-09-15T21:34:21.252971+0000","last_peered":"2024-09-15T21:34:21.252971+0000","last_clean":"2024-09-15T21:34:21.252971+0000","last_became_active":"2024-09-15T21:23:54.316513+0000","last_became_peered":"2024-09-15T21:23:54.316513+0000","last_unstale":"2024-09-15T21:34:21.252971+0000","last_undegraded":"2024-09-15T21:34:21.252971+0000","last_fullsized":"2024-09-15T21:34:21.252971+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:21.773590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:51.249994+0000","last_change":"2024-09-15T21:23:54.308012+0000","last_active":"2024-09-15T21:33:51.249994+0000","last_peered":"2024-09-15T21:33:51.249994+0000","last_clean":"2024-09-15T21:33:51.249994+0000","last_became_active":"2024-09-15T21:23:54.307605+0000","last_became_peered":"2024-09-15T21:23:54.307605+0000","last_unstale":"2024-09-15T21:33:51.249994+0000","last_undegraded":"2024-09-15T21:33:51.249994+0000","last_fullsized":"2024-09-15T21:33:51.249994+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:36.197287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250456+0000","last_change":"2024-09-15T21:23:54.308207+0000","last_active":"2024-09-15T21:34:20.250456+0000","last_peered":"2024-09-15T21:34:20.250456+0000","last_clean":"2024-09-15T21:34:20.250456+0000","last_became_active":"2024-09-15T21:23:54.307364+0000","last_became_peered":"2024-09-15T21:23:54.307364+0000","last_unstale":"2024-09-15T21:34:20.250456+0000","last_undegraded":"2024-09-15T21:34:20.250456+0000","last_fullsized":"2024-09-15T21:34:20.250456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:31.761043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":461,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252911+0000","last_change":"2024-09-15T21:23:54.322590+0000","last_active":"2024-09-15T21:34:21.252911+0000","last_peered":"2024-09-15T21:34:21.252911+0000","last_clean":"2024-09-15T21:34:21.252911+0000","last_became_active":"2024-09-15T21:23:54.314394+0000","last_became_peered":"2024-09-15T21:23:54.314394+0000","last_unstale":"2024-09-15T21:34:21.252911+0000","last_undegraded":"2024-09-15T21:34:21.252911+0000","last_fullsized":"2024-09-15T21:34:21.252911+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:54.033640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":439,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384782+0000","last_change":"2024-09-15T21:23:55.901581+0000","last_active":"2024-09-15T21:34:22.384782+0000","last_peered":"2024-09-15T21:34:22.384782+0000","last_clean":"2024-09-15T21:34:22.384782+0000","last_became_active":"2024-09-15T21:23:55.900571+0000","last_became_peered":"2024-09-15T21:23:55.900571+0000","last_unstale":"2024-09-15T21:34:22.384782+0000","last_undegraded":"2024-09-15T21:34:22.384782+0000","last_fullsized":"2024-09-15T21:34:22.384782+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:22.834885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:26.254894+0000","last_change":"2024-09-15T21:23:54.311459+0000","last_active":"2024-09-15T21:34:26.254894+0000","last_peered":"2024-09-15T21:34:26.254894+0000","last_clean":"2024-09-15T21:34:26.254894+0000","last_became_active":"2024-09-15T21:23:54.310902+0000","last_became_peered":"2024-09-15T21:23:54.310902+0000","last_unstale":"2024-09-15T21:34:26.254894+0000","last_undegraded":"2024-09-15T21:34:26.254894+0000","last_fullsized":"2024-09-15T21:34:26.254894+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:32.505577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:47.388767+0000","last_change":"2024-09-15T21:23:54.295908+0000","last_active":"2024-09-15T21:34:47.388767+0000","last_peered":"2024-09-15T21:34:47.388767+0000","last_clean":"2024-09-15T21:34:47.388767+0000","last_became_active":"2024-09-15T21:23:54.294899+0000","last_became_peered":"2024-09-15T21:23:54.294899+0000","last_unstale":"2024-09-15T21:34:47.388767+0000","last_undegraded":"2024-09-15T21:34:47.388767+0000","last_fullsized":"2024-09-15T21:34:47.388767+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:02.582888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258125+0000","last_change":"2024-09-15T21:23:54.302593+0000","last_active":"2024-09-15T21:34:46.258125+0000","last_peered":"2024-09-15T21:34:46.258125+0000","last_clean":"2024-09-15T21:34:46.258125+0000","last_became_active":"2024-09-15T21:23:54.301715+0000","last_became_peered":"2024-09-15T21:23:54.301715+0000","last_unstale":"2024-09-15T21:34:46.258125+0000","last_undegraded":"2024-09-15T21:34:46.258125+0000","last_fullsized":"2024-09-15T21:34:46.258125+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:51.716170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607796,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38676,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333164,"statfs":{"total":96636764160,"available":96597159936,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34516,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337324,"statfs":{"total":96636764160,"available":96601419776,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640497,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34512,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337328,"statfs":{"total":96636764160,"available":96601423872,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640498,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38672,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333168,"statfs":{"total":96636764160,"available":96597164032,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640499,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:37:29.412 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:37:29.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:37:29.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.530+0000 7f88d712e640 10 monclient: get_monmap_and_config 2024-09-15T21:37:29.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.530+0000 7f88d712e640 10 monclient: build_initial_monmap 2024-09-15T21:37:29.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.530+0000 7f88d712e640 10 monclient: monmap: 2024-09-15T21:37:29.537 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:29.537072+0000 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:29.537072+0000 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:29.538 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.530+0000 7f88d712e640 1 Processor -- start 2024-09-15T21:37:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.530+0000 7f88d712e640 1 -- start start 2024-09-15T21:37:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.530+0000 7f88d712e640 10 monclient: init 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f88d01378c0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 0x7f88d013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): picked mon.noname-b con 0x7f88d01383b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88d013d2c0 0x7f88d013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): picked mon.noname-a con 0x7f88d013d2c0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d00a1a20 con 0x7f88d01378c0 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f88d00a1d00 con 0x7f88d013d2c0 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88d00a1fe0 con 0x7f88d01383b0 2024-09-15T21:37:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d712e640 10 monclient(hunting): authenticate will time out at 2172.932550s 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 0x7f88d013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88d013d2c0 0x7f88d013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 0x7f88d013cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57422/0 (socket says 172.21.15.17:57422) 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88d013d2c0 0x7f88d013f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35440/0 (socket says 172.21.15.17:35440) 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 1 -- 172.21.15.17:0/3312855855 learned_addr learned my addr 172.21.15.17:0/3312855855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): get_auth_request con 0x7f88d01383b0 auth_method 0 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): get_auth_request con 0x7f88d013d2c0 auth_method 0 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient(hunting): handle_auth_done global_id 204534 payload 274 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 -- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 msgr2=0x7f88d013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 --2- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 0x7f88d013cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 -- 172.21.15.17:0/3312855855 >> v1:172.21.15.119:6789/0 conn(0x7f88d01378c0 legacy=0x7f88d0137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient: _finish_hunting 0 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 monclient: found mon.noname-a 2024-09-15T21:37:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 -- 172.21.15.17:0/3312855855 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d00a22c0 con 0x7f88d013d2c0 2024-09-15T21:37:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88cffff640 1 --2- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 0x7f88d013cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.534+0000 7f88d56cb640 1 --2- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d013d2c0 0x7f88d013f6b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f88c00097c0 tx=0x7f88c001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4466f90a79df89 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 1 -- 172.21.15.17:0/3312855855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00b1070 con 0x7f88d013d2c0 2024-09-15T21:37:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: dump: 2024-09-15T21:37:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: _finish_auth 0 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 1 -- 172.21.15.17:0/3312855855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c00a47f0 con 0x7f88d013d2c0 2024-09-15T21:37:29.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 1 -- 172.21.15.17:0/3312855855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00a4bc0 con 0x7f88d013d2c0 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 5 monclient: authenticate success, global_id 204534 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 4 set_mon_vals no callback set 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cf7fe640 10 monclient: dump: 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: shutdown 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d013d2c0 msgr2=0x7f88d013f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 --2- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d013d2c0 0x7f88d013f6b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f88c00097c0 tx=0x7f88c001c040 comp rx=0 tx=0).stop 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/3312855855 shutdown_connections 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 --2- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d013d2c0 0x7f88d013f6b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 --2- 172.21.15.17:0/3312855855 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d01383b0 0x7f88d013cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/3312855855 >> 172.21.15.17:0/3312855855 conn(0x7f88d00aac40 msgr2=0x7f88d00a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/3312855855 shutdown_connections 2024-09-15T21:37:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/3312855855 wait complete. 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: build_initial_monmap 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: monmap: 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:29.553 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 Processor -- start 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- start start 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: init 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 0x7f88d013c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): picked mon.noname-a con 0x7f88d01378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d01383b0 0x7f88d009bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): picked mon.noname-b con 0x7f88d01383b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 0x7f88d009c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): picked mon.noname-c con 0x7f88d013d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0143070 con 0x7f88d01378c0 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0142ef0 con 0x7f88d01383b0 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f88d01431f0 con 0x7f88d013d2c0 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient(hunting): authenticate will time out at 2172.938256s 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 0x7f88d009c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 0x7f88d013c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 0x7f88d009c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57436/0 (socket says 172.21.15.17:57436) 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 -- 172.21.15.17:0/760936197 learned_addr learned my addr 172.21.15.17:0/760936197 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): get_auth_request con 0x7f88d013d2c0 auth_method 0 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 0x7f88d013c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35446/0 (socket says 172.21.15.17:35446) 2024-09-15T21:37:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): get_auth_request con 0x7f88d01378c0 auth_method 0 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient(hunting): handle_auth_done global_id 234146 payload 824 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cffff640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d01383b0 0x7f88d009bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 -- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d01383b0 msgr2=0x7f88d009bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d01383b0 0x7f88d009bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 -- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 msgr2=0x7f88d013c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 0x7f88d013c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient: _finish_hunting 0 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 monclient: found mon.noname-c 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 -- 172.21.15.17:0/760936197 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d013cc50 con 0x7f88d013d2c0 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cffff640 10 monclient: get_auth_request con 0x7f88d01383b0 auth_method 0 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cffff640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d01383b0 0x7f88d009bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:37:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d4eca640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 0x7f88d013c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d56cb640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 0x7f88d009c460 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88c0019e80 tx=0x7f88c0019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00b1040 con 0x7f88d013d2c0 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: dump: 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.558 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: _finish_auth 0 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: _check_auth_tickets 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 5 monclient: authenticate success, global_id 234146 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: _renew_subs 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c00ac660 con 0x7f88d013d2c0 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d009da40 con 0x7f88d013d2c0 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00aca50 con 0x7f88d013d2c0 2024-09-15T21:37:29.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88ceffd640 4 set_mon_vals no callback set 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88cd7fa640 10 monclient: dump: 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: _renew_subs 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.538+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d009c9a0 con 0x7f88d013d2c0 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f88c00a4670 con 0x7f88d013d2c0 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d712e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d712e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d712e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d0137ca0 con 0x7f88d013d2c0 2024-09-15T21:37:29.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88cd7fa640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b003b9c0 0x7f88b003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f88c00e9f00 con 0x7f88d013d2c0 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d4eca640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b003b9c0 0x7f88b003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d4eca640 10 monclient: get_auth_request con 0x7f88b003b9c0 auth_method 0 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.542+0000 7f88d4eca640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b003b9c0 0x7f88b003de80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f88c800f290 tx=0x7f88c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.546+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f88c0016070 con 0x7f88d013d2c0 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.546+0000 7f88cd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.546+0000 7f88cd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:29.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.714+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88d0069270 con 0x7f88b003b9c0 2024-09-15T21:37:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.718+0000 7f88cd7fa640 1 -- 172.21.15.17:0/760936197 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+77978 (secure 0 0 0) 0x7f88d0069270 con 0x7f88b003b9c0 2024-09-15T21:37:29.726 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:37:29.726 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b003b9c0 msgr2=0x7f88b003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b003b9c0 0x7f88b003de80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f88c800f290 tx=0x7f88c8009040 comp rx=0 tx=0).stop 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 10 monclient: shutdown 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 msgr2=0x7f88d009c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 0x7f88d009c460 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88c0019e80 tx=0x7f88c0019100 comp rx=0 tx=0).stop 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 shutdown_connections 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f88b003b9c0 0x7f88b003de80 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f88d013d2c0 0x7f88d009c460 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d01383b0 0x7f88d009bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 --2- 172.21.15.17:0/760936197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88d01378c0 0x7f88d013c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 >> 172.21.15.17:0/760936197 conn(0x7f88d00aac40 msgr2=0x7f88d00a9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 shutdown_connections 2024-09-15T21:37:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.722+0000 7f88d712e640 1 -- 172.21.15.17:0/760936197 wait complete. 2024-09-15T21:37:29.743 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":854,"stamp":"2024-09-15T21:37:29.530220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":215504,"kb_used_data":20056,"kb_used_omap":9,"kb_used_meta":195318,"kb_avail":566015536,"statfs":{"total":579820584960,"available":579599908864,"internally_reserved":0,"allocated":20537344,"data_stored":17290996,"data_compressed":1361822,"data_compressed_allocated":2695168,"data_compressed_original":6922240,"omap_allocated":9567,"internal_metadata":200006305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003604"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":302,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373667+0000","last_change":"2024-09-15T21:23:54.324070+0000","last_active":"2024-09-15T21:34:21.373667+0000","last_peered":"2024-09-15T21:34:21.373667+0000","last_clean":"2024-09-15T21:34:21.373667+0000","last_became_active":"2024-09-15T21:23:54.320552+0000","last_became_peered":"2024-09-15T21:23:54.320552+0000","last_unstale":"2024-09-15T21:34:21.373667+0000","last_undegraded":"2024-09-15T21:34:21.373667+0000","last_fullsized":"2024-09-15T21:34:21.373667+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:55.900283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256773+0000","last_change":"2024-09-15T21:23:54.301950+0000","last_active":"2024-09-15T21:34:46.256773+0000","last_peered":"2024-09-15T21:34:46.256773+0000","last_clean":"2024-09-15T21:34:46.256773+0000","last_became_active":"2024-09-15T21:23:54.301596+0000","last_became_peered":"2024-09-15T21:23:54.301596+0000","last_unstale":"2024-09-15T21:34:46.256773+0000","last_undegraded":"2024-09-15T21:34:46.256773+0000","last_fullsized":"2024-09-15T21:34:46.256773+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:07.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.251301+0000","last_change":"2024-09-15T21:23:54.314550+0000","last_active":"2024-09-15T21:34:25.251301+0000","last_peered":"2024-09-15T21:34:25.251301+0000","last_clean":"2024-09-15T21:34:25.251301+0000","last_became_active":"2024-09-15T21:23:54.313525+0000","last_became_peered":"2024-09-15T21:23:54.313525+0000","last_unstale":"2024-09-15T21:34:25.251301+0000","last_undegraded":"2024-09-15T21:34:25.251301+0000","last_fullsized":"2024-09-15T21:34:25.251301+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:21:52.088930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256725+0000","last_change":"2024-09-15T21:23:54.314288+0000","last_active":"2024-09-15T21:34:46.256725+0000","last_peered":"2024-09-15T21:34:46.256725+0000","last_clean":"2024-09-15T21:34:46.256725+0000","last_became_active":"2024-09-15T21:23:54.313839+0000","last_became_peered":"2024-09-15T21:23:54.313839+0000","last_unstale":"2024-09-15T21:34:46.256725+0000","last_undegraded":"2024-09-15T21:34:46.256725+0000","last_fullsized":"2024-09-15T21:34:46.256725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:09.847621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":231,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:51.257506+0000","last_change":"2024-09-15T21:23:55.902295+0000","last_active":"2024-09-15T21:34:51.257506+0000","last_peered":"2024-09-15T21:34:51.257506+0000","last_clean":"2024-09-15T21:34:51.257506+0000","last_became_active":"2024-09-15T21:23:55.901252+0000","last_became_peered":"2024-09-15T21:23:55.901252+0000","last_unstale":"2024-09-15T21:34:51.257506+0000","last_undegraded":"2024-09-15T21:34:51.257506+0000","last_fullsized":"2024-09-15T21:34:51.257506+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:55.224562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373691+0000","last_change":"2024-09-15T21:23:54.323472+0000","last_active":"2024-09-15T21:34:21.373691+0000","last_peered":"2024-09-15T21:34:21.373691+0000","last_clean":"2024-09-15T21:34:21.373691+0000","last_became_active":"2024-09-15T21:23:54.318732+0000","last_became_peered":"2024-09-15T21:23:54.318732+0000","last_unstale":"2024-09-15T21:34:21.373691+0000","last_undegraded":"2024-09-15T21:34:21.373691+0000","last_fullsized":"2024-09-15T21:34:21.373691+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:32:38.556252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258088+0000","last_change":"2024-09-15T21:23:54.326082+0000","last_active":"2024-09-15T21:34:46.258088+0000","last_peered":"2024-09-15T21:34:46.258088+0000","last_clean":"2024-09-15T21:34:46.258088+0000","last_became_active":"2024-09-15T21:23:54.325406+0000","last_became_peered":"2024-09-15T21:23:54.325406+0000","last_unstale":"2024-09-15T21:34:46.258088+0000","last_undegraded":"2024-09-15T21:34:46.258088+0000","last_fullsized":"2024-09-15T21:34:46.258088+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:51:57.618330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380779+0000","last_change":"2024-09-15T21:23:54.301728+0000","last_active":"2024-09-15T21:33:57.380779+0000","last_peered":"2024-09-15T21:33:57.380779+0000","last_clean":"2024-09-15T21:33:57.380779+0000","last_became_active":"2024-09-15T21:23:54.300781+0000","last_became_peered":"2024-09-15T21:23:54.300781+0000","last_unstale":"2024-09-15T21:33:57.380779+0000","last_undegraded":"2024-09-15T21:33:57.380779+0000","last_fullsized":"2024-09-15T21:33:57.380779+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:02.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:06.250727+0000","last_change":"2024-09-15T21:23:54.308826+0000","last_active":"2024-09-15T21:34:06.250727+0000","last_peered":"2024-09-15T21:34:06.250727+0000","last_clean":"2024-09-15T21:34:06.250727+0000","last_became_active":"2024-09-15T21:23:54.307998+0000","last_became_peered":"2024-09-15T21:23:54.307998+0000","last_unstale":"2024-09-15T21:34:06.250727+0000","last_undegraded":"2024-09-15T21:34:06.250727+0000","last_fullsized":"2024-09-15T21:34:06.250727+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:32.478200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":216,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:31.255681+0000","last_change":"2024-09-15T21:23:54.311062+0000","last_active":"2024-09-15T21:34:31.255681+0000","last_peered":"2024-09-15T21:34:31.255681+0000","last_clean":"2024-09-15T21:34:31.255681+0000","last_became_active":"2024-09-15T21:23:54.310738+0000","last_became_peered":"2024-09-15T21:23:54.310738+0000","last_unstale":"2024-09-15T21:34:31.255681+0000","last_undegraded":"2024-09-15T21:34:31.255681+0000","last_fullsized":"2024-09-15T21:34:31.255681+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:17.678319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":232,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.253623+0000","last_change":"2024-09-15T21:23:54.312726+0000","last_active":"2024-09-15T21:34:21.253623+0000","last_peered":"2024-09-15T21:34:21.253623+0000","last_clean":"2024-09-15T21:34:21.253623+0000","last_became_active":"2024-09-15T21:23:54.311958+0000","last_became_peered":"2024-09-15T21:23:54.311958+0000","last_unstale":"2024-09-15T21:34:21.253623+0000","last_undegraded":"2024-09-15T21:34:21.253623+0000","last_fullsized":"2024-09-15T21:34:21.253623+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:15.944561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384735+0000","last_change":"2024-09-15T21:23:54.310984+0000","last_active":"2024-09-15T21:34:22.384735+0000","last_peered":"2024-09-15T21:34:22.384735+0000","last_clean":"2024-09-15T21:34:22.384735+0000","last_became_active":"2024-09-15T21:23:54.309483+0000","last_became_peered":"2024-09-15T21:23:54.309483+0000","last_unstale":"2024-09-15T21:34:22.384735+0000","last_undegraded":"2024-09-15T21:34:22.384735+0000","last_fullsized":"2024-09-15T21:34:22.384735+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:26.416656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384680+0000","last_change":"2024-09-15T21:23:54.313402+0000","last_active":"2024-09-15T21:34:22.384680+0000","last_peered":"2024-09-15T21:34:22.384680+0000","last_clean":"2024-09-15T21:34:22.384680+0000","last_became_active":"2024-09-15T21:23:54.312375+0000","last_became_peered":"2024-09-15T21:23:54.312375+0000","last_unstale":"2024-09-15T21:34:22.384680+0000","last_undegraded":"2024-09-15T21:34:22.384680+0000","last_fullsized":"2024-09-15T21:34:22.384680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:29.568805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":304,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373629+0000","last_change":"2024-09-15T21:23:54.319437+0000","last_active":"2024-09-15T21:34:21.373629+0000","last_peered":"2024-09-15T21:34:21.373629+0000","last_clean":"2024-09-15T21:34:21.373629+0000","last_became_active":"2024-09-15T21:23:54.318494+0000","last_became_peered":"2024-09-15T21:23:54.318494+0000","last_unstale":"2024-09-15T21:34:21.373629+0000","last_undegraded":"2024-09-15T21:34:21.373629+0000","last_fullsized":"2024-09-15T21:34:21.373629+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:44:21.410329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250527+0000","last_change":"2024-09-15T21:23:55.306377+0000","last_active":"2024-09-15T21:34:20.250527+0000","last_peered":"2024-09-15T21:34:20.250527+0000","last_clean":"2024-09-15T21:34:20.250527+0000","last_became_active":"2024-09-15T21:23:55.305236+0000","last_became_peered":"2024-09-15T21:23:55.305236+0000","last_unstale":"2024-09-15T21:34:20.250527+0000","last_undegraded":"2024-09-15T21:34:20.250527+0000","last_fullsized":"2024-09-15T21:34:20.250527+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:30.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380708+0000","last_change":"2024-09-15T21:23:54.307012+0000","last_active":"2024-09-15T21:33:57.380708+0000","last_peered":"2024-09-15T21:33:57.380708+0000","last_clean":"2024-09-15T21:33:57.380708+0000","last_became_active":"2024-09-15T21:23:54.305792+0000","last_became_peered":"2024-09-15T21:23:54.305792+0000","last_unstale":"2024-09-15T21:33:57.380708+0000","last_undegraded":"2024-09-15T21:33:57.380708+0000","last_fullsized":"2024-09-15T21:33:57.380708+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:33:47.178490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":533,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:01.251383+0000","last_change":"2024-09-15T21:23:54.325895+0000","last_active":"2024-09-15T21:34:01.251383+0000","last_peered":"2024-09-15T21:34:01.251383+0000","last_clean":"2024-09-15T21:34:01.251383+0000","last_became_active":"2024-09-15T21:23:54.325552+0000","last_became_peered":"2024-09-15T21:23:54.325552+0000","last_unstale":"2024-09-15T21:34:01.251383+0000","last_undegraded":"2024-09-15T21:34:01.251383+0000","last_fullsized":"2024-09-15T21:34:01.251383+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:57.451100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":451,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252971+0000","last_change":"2024-09-15T21:23:54.326302+0000","last_active":"2024-09-15T21:34:21.252971+0000","last_peered":"2024-09-15T21:34:21.252971+0000","last_clean":"2024-09-15T21:34:21.252971+0000","last_became_active":"2024-09-15T21:23:54.316513+0000","last_became_peered":"2024-09-15T21:23:54.316513+0000","last_unstale":"2024-09-15T21:34:21.252971+0000","last_undegraded":"2024-09-15T21:34:21.252971+0000","last_fullsized":"2024-09-15T21:34:21.252971+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:21.773590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:51.249994+0000","last_change":"2024-09-15T21:23:54.308012+0000","last_active":"2024-09-15T21:33:51.249994+0000","last_peered":"2024-09-15T21:33:51.249994+0000","last_clean":"2024-09-15T21:33:51.249994+0000","last_became_active":"2024-09-15T21:23:54.307605+0000","last_became_peered":"2024-09-15T21:23:54.307605+0000","last_unstale":"2024-09-15T21:33:51.249994+0000","last_undegraded":"2024-09-15T21:33:51.249994+0000","last_fullsized":"2024-09-15T21:33:51.249994+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:36.197287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250456+0000","last_change":"2024-09-15T21:23:54.308207+0000","last_active":"2024-09-15T21:34:20.250456+0000","last_peered":"2024-09-15T21:34:20.250456+0000","last_clean":"2024-09-15T21:34:20.250456+0000","last_became_active":"2024-09-15T21:23:54.307364+0000","last_became_peered":"2024-09-15T21:23:54.307364+0000","last_unstale":"2024-09-15T21:34:20.250456+0000","last_undegraded":"2024-09-15T21:34:20.250456+0000","last_fullsized":"2024-09-15T21:34:20.250456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:31.761043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":461,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252911+0000","last_change":"2024-09-15T21:23:54.322590+0000","last_active":"2024-09-15T21:34:21.252911+0000","last_peered":"2024-09-15T21:34:21.252911+0000","last_clean":"2024-09-15T21:34:21.252911+0000","last_became_active":"2024-09-15T21:23:54.314394+0000","last_became_peered":"2024-09-15T21:23:54.314394+0000","last_unstale":"2024-09-15T21:34:21.252911+0000","last_undegraded":"2024-09-15T21:34:21.252911+0000","last_fullsized":"2024-09-15T21:34:21.252911+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:54.033640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":439,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384782+0000","last_change":"2024-09-15T21:23:55.901581+0000","last_active":"2024-09-15T21:34:22.384782+0000","last_peered":"2024-09-15T21:34:22.384782+0000","last_clean":"2024-09-15T21:34:22.384782+0000","last_became_active":"2024-09-15T21:23:55.900571+0000","last_became_peered":"2024-09-15T21:23:55.900571+0000","last_unstale":"2024-09-15T21:34:22.384782+0000","last_undegraded":"2024-09-15T21:34:22.384782+0000","last_fullsized":"2024-09-15T21:34:22.384782+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:22.834885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:26.254894+0000","last_change":"2024-09-15T21:23:54.311459+0000","last_active":"2024-09-15T21:34:26.254894+0000","last_peered":"2024-09-15T21:34:26.254894+0000","last_clean":"2024-09-15T21:34:26.254894+0000","last_became_active":"2024-09-15T21:23:54.310902+0000","last_became_peered":"2024-09-15T21:23:54.310902+0000","last_unstale":"2024-09-15T21:34:26.254894+0000","last_undegraded":"2024-09-15T21:34:26.254894+0000","last_fullsized":"2024-09-15T21:34:26.254894+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:32.505577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:47.388767+0000","last_change":"2024-09-15T21:23:54.295908+0000","last_active":"2024-09-15T21:34:47.388767+0000","last_peered":"2024-09-15T21:34:47.388767+0000","last_clean":"2024-09-15T21:34:47.388767+0000","last_became_active":"2024-09-15T21:23:54.294899+0000","last_became_peered":"2024-09-15T21:23:54.294899+0000","last_unstale":"2024-09-15T21:34:47.388767+0000","last_undegraded":"2024-09-15T21:34:47.388767+0000","last_fullsized":"2024-09-15T21:34:47.388767+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:02.582888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258125+0000","last_change":"2024-09-15T21:23:54.302593+0000","last_active":"2024-09-15T21:34:46.258125+0000","last_peered":"2024-09-15T21:34:46.258125+0000","last_clean":"2024-09-15T21:34:46.258125+0000","last_became_active":"2024-09-15T21:23:54.301715+0000","last_became_peered":"2024-09-15T21:23:54.301715+0000","last_unstale":"2024-09-15T21:34:46.258125+0000","last_undegraded":"2024-09-15T21:34:46.258125+0000","last_fullsized":"2024-09-15T21:34:46.258125+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:51.716170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607796,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38676,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333164,"statfs":{"total":96636764160,"available":96597159936,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34516,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337324,"statfs":{"total":96636764160,"available":96601419776,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640497,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34512,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337328,"statfs":{"total":96636764160,"available":96601423872,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640498,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38672,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333168,"statfs":{"total":96636764160,"available":96597164032,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640499,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:37:29.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:37:29.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.850+0000 7f8ddfd54640 10 monclient: get_monmap_and_config 2024-09-15T21:37:29.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.850+0000 7f8ddfd54640 10 monclient: build_initial_monmap 2024-09-15T21:37:29.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.850+0000 7f8ddfd54640 10 monclient: monmap: 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:29.857188+0000 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:29.857188+0000 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:29.858 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 Processor -- start 2024-09-15T21:37:29.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 -- start start 2024-09-15T21:37:29.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient: init 2024-09-15T21:37:29.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:29.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2024-09-15T21:37:29.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 0x7f8dd80a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): picked mon.noname-b con 0x7f8dd80a4a90 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8dd80a53b0 0x7f8dd80a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): picked mon.noname-a con 0x7f8dd80a53b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8dd8137850 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd80a2c30 con 0x7f8dd8137850 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd80a0f00 con 0x7f8dd80a53b0 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd80a19b0 con 0x7f8dd80a4a90 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddfd54640 10 monclient(hunting): authenticate will time out at 2173.253102s 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 0x7f8dd80a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dde2f1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8dd8137850 0x7f8dd8137ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45452/0 (socket says 172.21.15.17:45452) 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 0x7f8dd80a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57438/0 (socket says 172.21.15.17:57438) 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dde2f1640 1 -- 172.21.15.17:0/237242080 learned_addr learned my addr 172.21.15.17:0/237242080 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 1 --2- 172.21.15.17:0/237242080 >> v2:172.21.15.17:3300/0 conn(0x7f8dd80a53b0 0x7f8dd80a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): get_auth_request con 0x7f8dd80a4a90 auth_method 0 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): get_auth_request con 0x7f8dd80a53b0 auth_method 0 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_done global_id 234149 payload 274 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 -- 172.21.15.17:0/237242080 >> v2:172.21.15.17:3300/0 conn(0x7f8dd80a53b0 msgr2=0x7f8dd80a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 --2- 172.21.15.17:0/237242080 >> v2:172.21.15.17:3300/0 conn(0x7f8dd80a53b0 0x7f8dd80a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddcaee640 1 -- 172.21.15.17:0/237242080 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201702028 0 0) 0x7f8dd0001120 con 0x7f8dd8137850 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 -- 172.21.15.17:0/237242080 >> v1:172.21.15.17:6789/0 conn(0x7f8dd8137850 legacy=0x7f8dd8137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient: _finish_hunting 0 2024-09-15T21:37:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 monclient: found mon.noname-b 2024-09-15T21:37:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 -- 172.21.15.17:0/237242080 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80a1c90 con 0x7f8dd80a4a90 2024-09-15T21:37:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddcaee640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:37:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8ddd2ef640 1 --2- 172.21.15.17:0/237242080 >> v2:172.21.15.17:3300/0 conn(0x7f8dd80a53b0 0x7f8dd80a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.854+0000 7f8dddaf0640 1 --2- 172.21.15.17:0/237242080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 0x7f8dd80a4e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009d90 tx=0x7f8dd401c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e88a22e5e7d5ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 1 -- 172.21.15.17:0/237242080 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40b2070 con 0x7f8dd80a4a90 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: dump: 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: _finish_auth 0 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: _check_auth_tickets 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 1 -- 172.21.15.17:0/237242080 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd40a5790 con 0x7f8dd80a4a90 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 5 monclient: authenticate success, global_id 234149 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 4 set_mon_vals no callback set 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 1 -- 172.21.15.17:0/237242080 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40a5b60 con 0x7f8dd80a4a90 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddcaee640 10 monclient: dump: 2024-09-15T21:37:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: shutdown 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/237242080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 msgr2=0x7f8dd80a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/237242080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 0x7f8dd80a4e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009d90 tx=0x7f8dd401c8c0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/237242080 shutdown_connections 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/237242080 >> v2:172.21.15.17:3300/0 conn(0x7f8dd80a53b0 0x7f8dd80a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/237242080 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a4a90 0x7f8dd80a4e70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/237242080 >> 172.21.15.17:0/237242080 conn(0x7f8dd80aabb0 msgr2=0x7f8dd80a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/237242080 shutdown_connections 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/237242080 wait complete. 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: build_initial_monmap 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: monmap: 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.873 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 Processor -- start 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- start start 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: init 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dd80a4a90 0x7f8dd81cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): picked mon.noname-a con 0x7f8dd80a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 0x7f8dd8136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): picked mon.noname-c con 0x7f8dd80a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd8137850 0x7f8dd813cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): picked mon.noname-b con 0x7f8dd8137850 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd81426d0 con 0x7f8dd80a4a90 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8142550 con 0x7f8dd8137850 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd8142850 con 0x7f8dd80a53b0 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient(hunting): authenticate will time out at 2173.258082s 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 0x7f8dd8136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 0x7f8dd8136960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57454/0 (socket says 172.21.15.17:57454) 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dd80a4a90 0x7f8dd81cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 -- 172.21.15.17:0/2163337659 learned_addr learned my addr 172.21.15.17:0/2163337659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): get_auth_request con 0x7f8dd80a53b0 auth_method 0 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): get_auth_request con 0x7f8dd80a4a90 auth_method 0 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dddaf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient(hunting): handle_auth_done global_id 234152 payload 824 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dde2f1640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd8137850 0x7f8dd813cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 -- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd8137850 msgr2=0x7f8dd813cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd8137850 0x7f8dd813cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 -- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dd80a4a90 msgr2=0x7f8dd81cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dd80a4a90 0x7f8dd81cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient: _finish_hunting 0 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 monclient: found mon.noname-c 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 -- 172.21.15.17:0/2163337659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80a1280 con 0x7f8dd80a53b0 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddd2ef640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 0x7f8dd8136960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800ec10 tx=0x7f8dc800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8017070 con 0x7f8dd80a53b0 2024-09-15T21:37:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: dump: 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: _finish_auth 0 2024-09-15T21:37:29.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 5 monclient: authenticate success, global_id 234152 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _renew_subs 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc800cc30 con 0x7f8dd80a53b0 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd8120d30 con 0x7f8dd80a53b0 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8013470 con 0x7f8dd80a53b0 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 10 monclient: dump: 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:29.879 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dcffff640 4 set_mon_vals no callback set 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _renew_subs 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd813f140 con 0x7f8dd80a53b0 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f8dc801a070 con 0x7f8dd80a53b0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd80a85f0 con 0x7f8dd80a53b0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.858+0000 7f8dce7fc640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8dbc03b9f0 0x7f8dbc03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.862+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f8dc8050570 con 0x7f8dd80a53b0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.862+0000 7f8dddaf0640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8dbc03b9f0 0x7f8dbc03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.862+0000 7f8dddaf0640 10 monclient: get_auth_request con 0x7f8dbc03b9f0 auth_method 0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.862+0000 7f8dddaf0640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8dbc03b9f0 0x7f8dbc03deb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40af000 tx=0x7f8dd40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:29.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.866+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f8dd80a1280 con 0x7f8dd80a53b0 2024-09-15T21:37:29.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.866+0000 7f8dce7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:29.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:29.866+0000 7f8dce7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:30.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.034+0000 7f8ddfd54640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:37:30.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.034+0000 7f8ddfd54640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:37:30.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.034+0000 7f8ddfd54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:30.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.034+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dd80a4e70 con 0x7f8dd80a53b0 2024-09-15T21:37:30.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8dce7fc640 1 -- 172.21.15.17:0/2163337659 <== mon.2 v2:172.21.15.17:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v261) ==== 74+0+21979 (secure 0 0 0) 0x7f8dc8013ae0 con 0x7f8dd80a53b0 2024-09-15T21:37:30.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8dce7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T21:37:30.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8dce7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:30.043 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:37:30.043 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":261,"fsid":"bbfd2e88-7522-4bd5-ae16-c665761d9528","created":"2024-09-15T21:17:19.699639+0000","modified":"2024-09-15T21:26:16.265341+0000","last_up_change":"2024-09-15T21:17:31.232299+0000","last_in_change":"2024-09-15T21:21:19.678084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":183,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":15,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:17:31.217943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:17:37.652178+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fooset","create_time":"2024-09-15T21:21:47.302430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"barset","create_time":"2024-09-15T21:21:48.222766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccf68244-48f9-4748-b69c-942fb9065051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6809","nonce":3272576910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6811","nonce":3272576910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6815","nonce":3272576910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":3272576910},{"type":"v1","addr":"172.21.15.17:6813","nonce":3272576910}]},"public_addr":"172.21.15.17:6809/3272576910","cluster_addr":"172.21.15.17:6811/3272576910","heartbeat_back_addr":"172.21.15.17:6815/3272576910","heartbeat_front_addr":"172.21.15.17:6813/3272576910","state":["exists","up"]},{"osd":1,"uuid":"7cf58353-b02d-4f6d-9170-d06191c75695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6817","nonce":2555706928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6819","nonce":2555706928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6823","nonce":2555706928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2555706928},{"type":"v1","addr":"172.21.15.17:6821","nonce":2555706928}]},"public_addr":"172.21.15.17:6817/2555706928","cluster_addr":"172.21.15.17:6819/2555706928","heartbeat_back_addr":"172.21.15.17:6823/2555706928","heartbeat_front_addr":"172.21.15.17:6821/2555706928","state":["exists","up"]},{"osd":2,"uuid":"d6e7cff2-5a15-4a7e-9a15-ff26517c535a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6801","nonce":671096025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6803","nonce":671096025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6807","nonce":671096025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":671096025},{"type":"v1","addr":"172.21.15.17:6805","nonce":671096025}]},"public_addr":"172.21.15.17:6801/671096025","cluster_addr":"172.21.15.17:6803/671096025","heartbeat_back_addr":"172.21.15.17:6807/671096025","heartbeat_front_addr":"172.21.15.17:6805/671096025","state":["exists","up"]},{"osd":3,"uuid":"e95cbd18-104f-4bf8-8496-096427e7459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6801","nonce":3353277098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6803","nonce":3353277098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6807","nonce":3353277098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3353277098},{"type":"v1","addr":"172.21.15.119:6805","nonce":3353277098}]},"public_addr":"172.21.15.119:6801/3353277098","cluster_addr":"172.21.15.119:6803/3353277098","heartbeat_back_addr":"172.21.15.119:6807/3353277098","heartbeat_front_addr":"172.21.15.119:6805/3353277098","state":["exists","up"]},{"osd":4,"uuid":"79502179-8d0c-49fa-86e7-2afe7bfee508","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6817","nonce":2086984234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6819","nonce":2086984234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6823","nonce":2086984234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2086984234},{"type":"v1","addr":"172.21.15.119:6821","nonce":2086984234}]},"public_addr":"172.21.15.119:6817/2086984234","cluster_addr":"172.21.15.119:6819/2086984234","heartbeat_back_addr":"172.21.15.119:6823/2086984234","heartbeat_front_addr":"172.21.15.119:6821/2086984234","state":["exists","up"]},{"osd":5,"uuid":"10352789-0e34-46f3-be2f-a929f23a8799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6809","nonce":444741843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6811","nonce":444741843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6815","nonce":444741843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":444741843},{"type":"v1","addr":"172.21.15.119:6813","nonce":444741843}]},"public_addr":"172.21.15.119:6809/444741843","cluster_addr":"172.21.15.119:6811/444741843","heartbeat_back_addr":"172.21.15.119:6815/444741843","heartbeat_front_addr":"172.21.15.119:6813/444741843","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"79adef04-9b2e-463a-b4b9-a979dfe86845","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.768611+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.237742+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.809091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.353555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:29.041478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:17:28.487022+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"ec-foo-ssd":{"crush-device-class":"ssd","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:37:30.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8dbc03b9f0 msgr2=0x7f8dbc03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:30.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8dbc03b9f0 0x7f8dbc03deb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40af000 tx=0x7f8dd40a7070 comp rx=0 tx=0).stop 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 10 monclient: shutdown 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 msgr2=0x7f8dd8136960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 0x7f8dd8136960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800ec10 tx=0x7f8dc800c520 comp rx=0 tx=0).stop 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 shutdown_connections 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f8dbc03b9f0 0x7f8dbc03deb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd8137850 0x7f8dd813cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f8dd80a53b0 0x7f8dd8136960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 --2- 172.21.15.17:0/2163337659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dd80a4a90 0x7f8dd81cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 >> 172.21.15.17:0/2163337659 conn(0x7f8dd80aabb0 msgr2=0x7f8dd80a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 shutdown_connections 2024-09-15T21:37:30.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:30.038+0000 7f8ddfd54640 1 -- 172.21.15.17:0/2163337659 wait complete. 2024-09-15T21:37:31.062 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:37:31.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:37:31.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient: get_monmap_and_config 2024-09-15T21:37:31.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient: monmap: 2024-09-15T21:37:31.199 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:31.199061+0000 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:31.199061+0000 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:31.200 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 Processor -- start 2024-09-15T21:37:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 -- start start 2024-09-15T21:37:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient: init 2024-09-15T21:37:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc3b40a0800 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 0x7fc3b413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): picked mon.noname-b con 0x7fc3b40a12f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc3b413b0b0 0x7fc3b413d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): picked mon.noname-c con 0x7fc3b413b0b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b40a1d30 con 0x7fc3b40a0800 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b40a2010 con 0x7fc3b413b0b0 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40a25d0 con 0x7fc3b40a12f0 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3bb443640 10 monclient(hunting): authenticate will time out at 2174.594746s 2024-09-15T21:37:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 0x7fc3b413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 0x7fc3b413ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57468/0 (socket says 172.21.15.17:57468) 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 -- 172.21.15.17:0/4157218506 learned_addr learned my addr 172.21.15.17:0/4157218506 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): get_auth_request con 0x7fc3b40a12f0 auth_method 0 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b99e0640 1 --2- 172.21.15.17:0/4157218506 >> v2:172.21.15.119:3300/0 conn(0x7fc3b413b0b0 0x7fc3b413d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b99e0640 10 monclient(hunting): get_auth_request con 0x7fc3b413b0b0 auth_method 0 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b99e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b99e0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b99e0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient(hunting): handle_auth_done global_id 234155 payload 274 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 -- 172.21.15.17:0/4157218506 >> v2:172.21.15.119:3300/0 conn(0x7fc3b413b0b0 msgr2=0x7fc3b413d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 --2- 172.21.15.17:0/4157218506 >> v2:172.21.15.119:3300/0 conn(0x7fc3b413b0b0 0x7fc3b413d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 -- 172.21.15.17:0/4157218506 >> v1:172.21.15.119:6789/0 conn(0x7fc3b40a0800 legacy=0x7fc3b40a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 monclient: found mon.noname-b 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.194+0000 7fc3b89de640 1 -- 172.21.15.17:0/4157218506 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b406f410 con 0x7fc3b40a12f0 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3b89de640 1 --2- 172.21.15.17:0/4157218506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 0x7fc3b413ab70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098a0 tx=0x7fc3a801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea8572e61464fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 1 -- 172.21.15.17:0/4157218506 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80b0070 con 0x7fc3b40a12f0 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: dump: 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.211 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 1 -- 172.21.15.17:0/4157218506 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a80ab600 con 0x7fc3b40a12f0 2024-09-15T21:37:31.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 1 -- 172.21.15.17:0/4157218506 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80ab9d0 con 0x7fc3b40a12f0 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 5 monclient: authenticate success, global_id 234155 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 4 set_mon_vals no callback set 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3a7fff640 10 monclient: dump: 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.213 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 10 monclient: shutdown 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 -- 172.21.15.17:0/4157218506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 msgr2=0x7fc3b413ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 --2- 172.21.15.17:0/4157218506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 0x7fc3b413ab70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098a0 tx=0x7fc3a801c400 comp rx=0 tx=0).stop 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 -- 172.21.15.17:0/4157218506 shutdown_connections 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 --2- 172.21.15.17:0/4157218506 >> v2:172.21.15.119:3300/0 conn(0x7fc3b413b0b0 0x7fc3b413d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 --2- 172.21.15.17:0/4157218506 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a12f0 0x7fc3b413ab70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 -- 172.21.15.17:0/4157218506 >> 172.21.15.17:0/4157218506 conn(0x7fc3b40a6820 msgr2=0x7fc3b40a2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 -- 172.21.15.17:0/4157218506 shutdown_connections 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 -- 172.21.15.17:0/4157218506 wait complete. 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 10 monclient: monmap: 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 Processor -- start 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 1 -- start start 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.198+0000 7fc3bb443640 10 monclient: init 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a0800 0x7fc3b41cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): picked mon.noname-c con 0x7fc3b40a0800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b40a12f0 0x7fc3b41326b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): picked mon.noname-b con 0x7fc3b40a12f0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b413b0b0 0x7fc3b4137b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): picked mon.noname-a con 0x7fc3b413b0b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4140380 con 0x7fc3b413b0b0 2024-09-15T21:37:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4140200 con 0x7fc3b40a12f0 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b4140500 con 0x7fc3b40a0800 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient(hunting): authenticate will time out at 2174.600347s 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a0800 0x7fc3b41cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a0800 0x7fc3b41cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57472/0 (socket says 172.21.15.17:57472) 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 1 -- 172.21.15.17:0/2946449400 learned_addr learned my addr 172.21.15.17:0/2946449400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): get_auth_request con 0x7fc3b40a0800 auth_method 0 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b413b0b0 0x7fc3b4137b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): get_auth_request con 0x7fc3b413b0b0 auth_method 0 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b40a12f0 0x7fc3b41326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): get_auth_request con 0x7fc3b40a12f0 auth_method 0 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient(hunting): handle_auth_done global_id 204549 payload 824 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 -- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a0800 msgr2=0x7fc3b41cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fc3b40a0800 0x7fc3b41cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 -- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b40a12f0 msgr2=0x7fc3b41326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b40a12f0 0x7fc3b41326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 monclient: found mon.noname-a 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 -- 172.21.15.17:0/2946449400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b411cd60 con 0x7fc3b413b0b0 2024-09-15T21:37:31.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b99e0640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b413b0b0 0x7fc3b4137b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d530 tx=0x7fc3b000d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00125f0 con 0x7fc3b413b0b0 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: dump: 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.220 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b0019070 con 0x7fc3b413b0b0 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 5 monclient: authenticate success, global_id 204549 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _renew_subs 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b411d040 con 0x7fc3b413b0b0 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00129d0 con 0x7fc3b413b0b0 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:31.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: dump: 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a77fe640 4 set_mon_vals no callback set 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _renew_subs 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b413a180 con 0x7fc3b413b0b0 2024-09-15T21:37:31.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3b409d0f0 con 0x7fc3b413b0b0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fc3b0012b70 con 0x7fc3b413b0b0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc39803b9f0 0x7fc39803deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fc3b0050880 con 0x7fc3b413b0b0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fc398044950 0x7fc398046db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 --> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc398047480 con 0x7fc398044950 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7fc3b00552c0 con 0x7fc3b413b0b0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 10 monclient: handle_get_version_reply finishing 1 version 261 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fc398044950 0x7fc398046db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 10 monclient: get_auth_request con 0x7fc398044950 auth_method 0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc39803b9f0 0x7fc39803deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 10 monclient: get_auth_request con 0x7fc39803b9f0 auth_method 0 2024-09-15T21:37:31.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b89de640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fc398044950 0x7fc398046db0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3b91df640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc39803b9f0 0x7fc39803deb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac005740 tx=0x7fc3ac003e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.202+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== osd.0 v2:172.21.15.17:6808/3272576910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc398047480 con 0x7fc398044950 2024-09-15T21:37:31.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 --> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc3b413b790 con 0x7fc398044950 2024-09-15T21:37:31.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3a5ffb640 1 -- 172.21.15.17:0/2946449400 <== osd.0 v2:172.21.15.17:6808/3272576910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc3a80ab480 con 0x7fc398044950 2024-09-15T21:37:31.225 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:31.225 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:37:31.225 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fc398044950 msgr2=0x7fc398046db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:6808/3272576910,v1:172.21.15.17:6809/3272576910] conn(0x7fc398044950 0x7fc398046db0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc39803b9f0 msgr2=0x7fc39803deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fc39803b9f0 0x7fc39803deb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac005740 tx=0x7fc3ac003e70 comp rx=0 tx=0).stop 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 10 monclient: shutdown 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b413b0b0 msgr2=0x7fc3b4137b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 --2- 172.21.15.17:0/2946449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b413b0b0 0x7fc3b4137b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d530 tx=0x7fc3b000d9f0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3b99e0640 1 -- 172.21.15.17:0/2946449400 reap_dead start 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 shutdown_connections 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 >> 172.21.15.17:0/2946449400 conn(0x7fc3b40a6820 msgr2=0x7fc3b41357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 shutdown_connections 2024-09-15T21:37:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.218+0000 7fc3bb443640 1 -- 172.21.15.17:0/2946449400 wait complete. 2024-09-15T21:37:31.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:37:31.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient: get_monmap_and_config 2024-09-15T21:37:31.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient: monmap: 2024-09-15T21:37:31.355 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:31.354950+0000 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:31.354950+0000 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:31.356 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 Processor -- start 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 -- start start 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient: init 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 0x7f0dcc137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): picked mon.noname-b con 0x7f0dcc1378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0dcc1381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0dcc0a5210 0x7f0dcc0a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): picked mon.noname-a con 0x7f0dcc0a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc0a1d00 con 0x7f0dcc1381e0 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc0a1fe0 con 0x7f0dcc0a5210 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc0a22c0 con 0x7f0dcc1378c0 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd435e640 10 monclient(hunting): authenticate will time out at 2174.750713s 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0dcc0a5210 0x7f0dcc0a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 0x7f0dcc137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0dcc0a5210 0x7f0dcc0a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35484/0 (socket says 172.21.15.17:35484) 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 0x7f0dcc137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57488/0 (socket says 172.21.15.17:57488) 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 -- 172.21.15.17:0/1121761307 learned_addr learned my addr 172.21.15.17:0/1121761307 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): get_auth_request con 0x7f0dcc0a5210 auth_method 0 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): get_auth_request con 0x7f0dcc1378c0 auth_method 0 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_done global_id 204552 payload 274 2024-09-15T21:37:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 -- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 msgr2=0x7f0dcc137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 --2- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 0x7f0dcc137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 -- 172.21.15.17:0/1121761307 >> v1:172.21.15.119:6789/0 conn(0x7f0dcc1381e0 legacy=0x7f0dcc0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 monclient: found mon.noname-a 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd28fb640 1 -- 172.21.15.17:0/1121761307 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0a5c00 con 0x7f0dcc0a5210 2024-09-15T21:37:31.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.350+0000 7f0dd20fa640 1 --2- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 0x7f0dcc137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:31.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 1 --2- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc0a5210 0x7f0dcc0a56c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009e50 tx=0x7f0dc401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d68de4bea12eeda server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 1 -- 172.21.15.17:0/1121761307 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40a5630 con 0x7f0dcc0a5210 2024-09-15T21:37:31.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: dump: 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.368 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 5 monclient: authenticate success, global_id 204552 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 1 -- 172.21.15.17:0/1121761307 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc40a57d0 con 0x7f0dcc0a5210 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 4 set_mon_vals no callback set 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 1 -- 172.21.15.17:0/1121761307 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40a5ba0 con 0x7f0dcc0a5210 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd10f8640 10 monclient: dump: 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: shutdown 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc0a5210 msgr2=0x7f0dcc0a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 --2- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc0a5210 0x7f0dcc0a56c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009e50 tx=0x7f0dc401c8f0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- 172.21.15.17:0/1121761307 shutdown_connections 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 --2- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc0a5210 0x7f0dcc0a56c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 --2- 172.21.15.17:0/1121761307 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc1378c0 0x7f0dcc137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- 172.21.15.17:0/1121761307 >> 172.21.15.17:0/1121761307 conn(0x7f0dcc0aac40 msgr2=0x7f0dcc0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- 172.21.15.17:0/1121761307 shutdown_connections 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- 172.21.15.17:0/1121761307 wait complete. 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: monmap: 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 Processor -- start 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- start start 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: init 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 0x7f0dcc1cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): picked mon.noname-c con 0x7f0dcc0a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dcc1378c0 0x7f0dcc136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): picked mon.noname-b con 0x7f0dcc1378c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc1381e0 0x7f0dcc13cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): picked mon.noname-a con 0x7f0dcc1381e0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc06f340 con 0x7f0dcc1381e0 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc06f1c0 con 0x7f0dcc1378c0 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc1425c0 con 0x7f0dcc0a5210 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd435e640 10 monclient(hunting): authenticate will time out at 2174.755693s 2024-09-15T21:37:31.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 0x7f0dcc1cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 0x7f0dcc1cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57500/0 (socket says 172.21.15.17:57500) 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 1 -- 172.21.15.17:0/2101929924 learned_addr learned my addr 172.21.15.17:0/2101929924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): get_auth_request con 0x7f0dcc0a5210 auth_method 0 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc1381e0 0x7f0dcc13cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 10 monclient(hunting): get_auth_request con 0x7f0dcc1381e0 auth_method 0 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd18f9640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dcc1378c0 0x7f0dcc136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.354+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd28fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 10 monclient(hunting): get_auth_request con 0x7f0dcc1378c0 auth_method 0 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 10 monclient(hunting): handle_auth_done global_id 234164 payload 824 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 -- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dcc1378c0 msgr2=0x7f0dcc136b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dcc1378c0 0x7f0dcc136b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 -- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc1381e0 msgr2=0x7f0dcc13cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc1381e0 0x7f0dcc13cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 monclient: found mon.noname-c 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 -- 172.21.15.17:0/2101929924 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0a8a30 con 0x7f0dcc0a5210 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd20fa640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 0x7f0dcc1cf950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000ebe0 tx=0x7f0dc000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0017070 con 0x7f0dcc0a5210 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: dump: 2024-09-15T21:37:31.376 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc000cc30 con 0x7f0dcc0a5210 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0013470 con 0x7f0dcc0a5210 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 5 monclient: authenticate success, global_id 234164 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 monclient: _renew_subs 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc0a12f0 con 0x7f0dcc0a5210 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd08f7640 4 set_mon_vals no callback set 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 10 monclient: dump: 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.378 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 monclient: _renew_subs 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc120c10 con 0x7f0dcc0a5210 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:31.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc0a56c0 con 0x7f0dcc0a5210 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f0dc001a070 con 0x7f0dcc0a5210 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0db003b9c0 0x7f0db003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f0dc0050640 con 0x7f0dcc0a5210 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0db003b9c0 0x7f0db003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 10 monclient: get_auth_request con 0x7f0db003b9c0 auth_method 0 2024-09-15T21:37:31.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.358+0000 7f0dd18f9640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0db003b9c0 0x7f0db003de80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc13f260 tx=0x7f0dc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.362+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f0dcc0a8a30 con 0x7f0dcc0a5210 2024-09-15T21:37:31.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.362+0000 7f0dbeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:31.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.362+0000 7f0dbeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.526+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f0dcc1d07f0 con 0x7f0db003b9c0 2024-09-15T21:37:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dbeffd640 1 -- 172.21.15.17:0/2101929924 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0dcc1d07f0 con 0x7f0db003b9c0 2024-09-15T21:37:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:37:31.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0db003b9c0 msgr2=0x7f0db003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0db003b9c0 0x7f0db003de80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc13f260 tx=0x7f0dc8009040 comp rx=0 tx=0).stop 2024-09-15T21:37:31.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 10 monclient: shutdown 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 msgr2=0x7f0dcc1cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 0x7f0dcc1cf950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000ebe0 tx=0x7f0dc000c520 comp rx=0 tx=0).stop 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 shutdown_connections 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f0db003b9c0 0x7f0db003de80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0dcc1381e0 0x7f0dcc13cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dcc1378c0 0x7f0dcc136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 --2- 172.21.15.17:0/2101929924 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f0dcc0a5210 0x7f0dcc1cf950 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 >> 172.21.15.17:0/2101929924 conn(0x7f0dcc0aac40 msgr2=0x7f0dcc06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 shutdown_connections 2024-09-15T21:37:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.530+0000 7f0dd435e640 1 -- 172.21.15.17:0/2101929924 wait complete. 2024-09-15T21:37:31.553 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:37:31.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:37:31.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient: get_monmap_and_config 2024-09-15T21:37:31.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient: monmap: 2024-09-15T21:37:31.679 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:31.679033+0000 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:31.679033+0000 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:31.680 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 Processor -- start 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 -- start start 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient: init 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f819c0a4a90 0x7f819c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): picked mon.noname-c con 0x7f819c0a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f819c0a53b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): picked mon.noname-b con 0x7f819c137f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819c06fb70 con 0x7f819c0a53b0 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f819c0a1590 con 0x7f819c0a4a90 2024-09-15T21:37:31.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f819c0a5c60 con 0x7f819c137f60 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f81a1d0d640 10 monclient(hunting): authenticate will time out at 2175.074498s 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57516/0 (socket says 172.21.15.17:57516) 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 -- 172.21.15.17:0/3047651256 learned_addr learned my addr 172.21.15.17:0/3047651256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): get_auth_request con 0x7f819c137f60 auth_method 0 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819b7fe640 1 --2- 172.21.15.17:0/3047651256 >> v2:172.21.15.119:3300/0 conn(0x7f819c0a4a90 0x7f819c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819b7fe640 10 monclient(hunting): get_auth_request con 0x7f819c0a4a90 auth_method 0 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient(hunting): handle_auth_done global_id 234167 payload 274 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 -- 172.21.15.17:0/3047651256 >> v2:172.21.15.119:3300/0 conn(0x7f819c0a4a90 msgr2=0x7f819c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 --2- 172.21.15.17:0/3047651256 >> v2:172.21.15.119:3300/0 conn(0x7f819c0a4a90 0x7f819c0a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 -- 172.21.15.17:0/3047651256 >> v1:172.21.15.17:6789/0 conn(0x7f819c0a53b0 legacy=0x7f819c137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 monclient: found mon.noname-b 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.674+0000 7f819bfff640 1 -- 172.21.15.17:0/3047651256 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c0a1a40 con 0x7f819c137f60 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819bfff640 1 --2- 172.21.15.17:0/3047651256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13fd30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8194009e50 tx=0x7f819401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c77cbb288180fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 1 -- 172.21.15.17:0/3047651256 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81940a5600 con 0x7f819c137f60 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: dump: 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 5 monclient: authenticate success, global_id 234167 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 1 -- 172.21.15.17:0/3047651256 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81940a57a0 con 0x7f819c137f60 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 4 set_mon_vals no callback set 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 1 -- 172.21.15.17:0/3047651256 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81940a5b70 con 0x7f819c137f60 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f819affd640 10 monclient: dump: 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 10 monclient: shutdown 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 -- 172.21.15.17:0/3047651256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 msgr2=0x7f819c13fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 --2- 172.21.15.17:0/3047651256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13fd30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8194009e50 tx=0x7f819401c8f0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 -- 172.21.15.17:0/3047651256 shutdown_connections 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 --2- 172.21.15.17:0/3047651256 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13fd30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 --2- 172.21.15.17:0/3047651256 >> v2:172.21.15.119:3300/0 conn(0x7f819c0a4a90 0x7f819c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 -- 172.21.15.17:0/3047651256 >> 172.21.15.17:0/3047651256 conn(0x7f819c0a6f90 msgr2=0x7f819c0a73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 -- 172.21.15.17:0/3047651256 shutdown_connections 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 1 -- 172.21.15.17:0/3047651256 wait complete. 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.678+0000 7f81a1d0d640 10 monclient: monmap: 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 Processor -- start 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- start start 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: init 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f819c0a4a90 0x7f819c1cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): picked mon.noname-a con 0x7f819c0a4a90 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c0a53b0 0x7f819c13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): picked mon.noname-b con 0x7f819c0a53b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): picked mon.noname-c con 0x7f819c137f60 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f819c06f7e0 con 0x7f819c0a4a90 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f819c06f660 con 0x7f819c0a53b0 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f819c0a2c30 con 0x7f819c137f60 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient(hunting): authenticate will time out at 2175.080637s 2024-09-15T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f819c0a4a90 0x7f819c1cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f819c0a4a90 0x7f819c1cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35496/0 (socket says 172.21.15.17:35496) 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): get_auth_request con 0x7f819c137f60 auth_method 0 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 1 -- 172.21.15.17:0/4163413389 learned_addr learned my addr 172.21.15.17:0/4163413389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): get_auth_request con 0x7f819c0a4a90 auth_method 0 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c0a53b0 0x7f819c13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 10 monclient(hunting): get_auth_request con 0x7f819c0a53b0 auth_method 0 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient(hunting): handle_auth_done global_id 234170 payload 824 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 -- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c0a53b0 msgr2=0x7f819c13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c0a53b0 0x7f819c13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 -- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f819c0a4a90 msgr2=0x7f819c1cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f819c0a4a90 0x7f819c1cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 monclient: found mon.noname-c 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 -- 172.21.15.17:0/4163413389 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c0a01e0 con 0x7f819c137f60 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c0a53b0 0x7f819c13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819bfff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13f4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f81940ab000 tx=0x7f8194019690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81940bc020 con 0x7f819c137f60 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: dump: 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81940b8690 con 0x7f819c137f60 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 5 monclient: authenticate success, global_id 234170 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _renew_subs 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c13b230 con 0x7f819c137f60 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194009510 con 0x7f819c137f60 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: dump: 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8199ffb640 4 set_mon_vals no callback set 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _renew_subs 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c13c470 con 0x7f819c137f60 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f819c0a9210 con 0x7f819c137f60 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f81940bf020 con 0x7f819c137f60 2024-09-15T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f817c03b9c0 0x7f817c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f81940eb020 con 0x7f819c137f60 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f817c044920 0x7f817c046d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 --> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f817c047450 con 0x7f817c044920 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7f81940f2050 con 0x7f819c137f60 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8198ff9640 10 monclient: handle_get_version_reply finishing 1 version 261 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f817c044920 0x7f817c046d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 10 monclient: get_auth_request con 0x7f817c044920 auth_method 0 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f817c03b9c0 0x7f817c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 10 monclient: get_auth_request con 0x7f817c03b9c0 auth_method 0 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f8193fff640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f817c044920 0x7f817c046d80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.682+0000 7f819b7fe640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f817c03b9c0 0x7f817c03de80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f818c007b10 tx=0x7f818c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.686+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== osd.1 v2:172.21.15.17:6816/2555706928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f817c047450 con 0x7f817c044920 2024-09-15T21:37:31.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 --> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f819c134290 con 0x7f817c044920 2024-09-15T21:37:31.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f8198ff9640 1 -- 172.21.15.17:0/4163413389 <== osd.1 v2:172.21.15.17:6816/2555706928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f819c134290 con 0x7f817c044920 2024-09-15T21:37:31.705 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:31.705 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:37:31.705 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:31.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f817c044920 msgr2=0x7f817c046d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:6816/2555706928,v1:172.21.15.17:6817/2555706928] conn(0x7f817c044920 0x7f817c046d80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f817c03b9c0 msgr2=0x7f817c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f817c03b9c0 0x7f817c03de80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f818c007b10 tx=0x7f818c008040 comp rx=0 tx=0).stop 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 10 monclient: shutdown 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 msgr2=0x7f819c13f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 --2- 172.21.15.17:0/4163413389 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f819c137f60 0x7f819c13f4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f81940ab000 tx=0x7f8194019690 comp rx=0 tx=0).stop 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f819bfff640 1 -- 172.21.15.17:0/4163413389 reap_dead start 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 shutdown_connections 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 >> 172.21.15.17:0/4163413389 conn(0x7f819c0a6f90 msgr2=0x7f819c133930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 shutdown_connections 2024-09-15T21:37:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.698+0000 7f81a1d0d640 1 -- 172.21.15.17:0/4163413389 wait complete. 2024-09-15T21:37:31.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:37:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient: get_monmap_and_config 2024-09-15T21:37:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient: monmap: 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:31.831165+0000 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:31.831165+0000 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:31.832 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 Processor -- start 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 -- start start 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient: init 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): picked mon.noname-b con 0x7f82d809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f82d809dd20 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82d8134d70 0x7f82d813cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): picked mon.noname-a con 0x7f82d8134d70 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d80a3130 con 0x7f82d809dd20 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f82d80a1bc0 con 0x7f82d8134d70 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f82d80a3490 con 0x7f82d809d400 2024-09-15T21:37:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dea9a640 10 monclient(hunting): authenticate will time out at 2175.226743s 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d809d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57532/0 (socket says 172.21.15.17:57532) 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 -- 172.21.15.17:0/2372279766 learned_addr learned my addr 172.21.15.17:0/2372279766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): get_auth_request con 0x7f82d809d400 auth_method 0 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 1 --2- 172.21.15.17:0/2372279766 >> v2:172.21.15.17:3300/0 conn(0x7f82d8134d70 0x7f82d813cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): get_auth_request con 0x7f82d8134d70 auth_method 0 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient(hunting): handle_auth_done global_id 234173 payload 274 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 -- 172.21.15.17:0/2372279766 >> v2:172.21.15.17:3300/0 conn(0x7f82d8134d70 msgr2=0x7f82d813cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 --2- 172.21.15.17:0/2372279766 >> v2:172.21.15.17:3300/0 conn(0x7f82d8134d70 0x7f82d813cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 -- 172.21.15.17:0/2372279766 >> v1:172.21.15.119:6789/0 conn(0x7f82d809dd20 legacy=0x7f82d8134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 monclient: found mon.noname-b 2024-09-15T21:37:31.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dc836640 1 -- 172.21.15.17:0/2372279766 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d809e7a0 con 0x7f82d809d400 2024-09-15T21:37:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.826+0000 7f82dd037640 1 --2- 172.21.15.17:0/2372279766 >> v2:172.21.15.17:3300/0 conn(0x7f82d8134d70 0x7f82d813cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dc836640 1 --2- 172.21.15.17:0/2372279766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d809d7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009e50 tx=0x7f82c001c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dee1ee0c0cde7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 1 -- 172.21.15.17:0/2372279766 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a55b0 con 0x7f82d809d400 2024-09-15T21:37:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: dump: 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.844 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 1 -- 172.21.15.17:0/2372279766 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c00a5750 con 0x7f82d809d400 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 5 monclient: authenticate success, global_id 234173 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 1 -- 172.21.15.17:0/2372279766 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a5b20 con 0x7f82d809d400 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 4 set_mon_vals no callback set 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82d77fe640 10 monclient: dump: 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 10 monclient: shutdown 2024-09-15T21:37:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 -- 172.21.15.17:0/2372279766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 msgr2=0x7f82d809d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 --2- 172.21.15.17:0/2372279766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d809d7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009e50 tx=0x7f82c001c8f0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 -- 172.21.15.17:0/2372279766 shutdown_connections 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 --2- 172.21.15.17:0/2372279766 >> v2:172.21.15.17:3300/0 conn(0x7f82d8134d70 0x7f82d813cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 --2- 172.21.15.17:0/2372279766 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d809d7e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 -- 172.21.15.17:0/2372279766 >> 172.21.15.17:0/2372279766 conn(0x7f82d80a66a0 msgr2=0x7f82d80a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 -- 172.21.15.17:0/2372279766 shutdown_connections 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 -- 172.21.15.17:0/2372279766 wait complete. 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 10 monclient: build_initial_monmap 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 10 monclient: monmap: 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 Processor -- start 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 1 -- start start 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.830+0000 7f82dea9a640 10 monclient: init 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d813e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): picked mon.noname-c con 0x7f82d809d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82d809dd20 0x7f82d813e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): picked mon.noname-b con 0x7f82d809dd20 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82d8134d70 0x7f82d813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): picked mon.noname-a con 0x7f82d8134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82d809fba0 con 0x7f82d8134d70 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82d809fa20 con 0x7f82d809dd20 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f82d813d820 con 0x7f82d809d400 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient(hunting): authenticate will time out at 2175.232102s 2024-09-15T21:37:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d813e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82d8134d70 0x7f82d813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d813e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57540/0 (socket says 172.21.15.17:57540) 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 1 -- 172.21.15.17:0/3465028189 learned_addr learned my addr 172.21.15.17:0/3465028189 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): get_auth_request con 0x7f82d809d400 auth_method 0 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): get_auth_request con 0x7f82d8134d70 auth_method 0 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d7fff640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82d809dd20 0x7f82d813e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d7fff640 10 monclient(hunting): get_auth_request con 0x7f82d809dd20 auth_method 0 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient(hunting): handle_auth_done global_id 204567 payload 824 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 -- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 msgr2=0x7f82d813e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d813e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 -- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82d809dd20 msgr2=0x7f82d813e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82d809dd20 0x7f82d813e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient: _finish_hunting 0 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 monclient: found mon.noname-a 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 -- 172.21.15.17:0/3465028189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d813fac0 con 0x7f82d8134d70 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d813e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dd037640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82d8134d70 0x7f82d813ead0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82d000e930 tx=0x7f82d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0017070 con 0x7f82d8134d70 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: dump: 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: _finish_auth 0 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: _check_auth_tickets 2024-09-15T21:37:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d0004020 con 0x7f82d8134d70 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00043f0 con 0x7f82d8134d70 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 5 monclient: authenticate success, global_id 204567 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _renew_subs 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d813b550 con 0x7f82d8134d70 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d6ffd640 4 set_mon_vals no callback set 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 10 monclient: dump: 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _renew_subs 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d809d0f0 con 0x7f82d8134d70 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d813dc00 con 0x7f82d8134d70 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f82d000c940 con 0x7f82d8134d70 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f82bc03b9c0 0x7f82bc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f82d00503f0 con 0x7f82d8134d70 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f82bc03b9c0 0x7f82bc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 10 monclient: get_auth_request con 0x7f82bc03b9c0 auth_method 0 2024-09-15T21:37:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.834+0000 7f82dc836640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f82bc03b9c0 0x7f82bc03de80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f82c0000f90 tx=0x7f82c00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.838+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f82d0003b40 con 0x7f82d8134d70 2024-09-15T21:37:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.838+0000 7f82d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:31.838+0000 7f82d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:32.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.002+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f82d813b2d0 con 0x7f82bc03b9c0 2024-09-15T21:37:32.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.002+0000 7f82d57fa640 1 -- 172.21.15.17:0/3465028189 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f82d813b2d0 con 0x7f82bc03b9c0 2024-09-15T21:37:32.010 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f82bc03b9c0 msgr2=0x7f82bc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f82bc03b9c0 0x7f82bc03de80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f82c0000f90 tx=0x7f82c00a7070 comp rx=0 tx=0).stop 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 10 monclient: shutdown 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82d8134d70 msgr2=0x7f82d813ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82d8134d70 0x7f82d813ead0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82d000e930 tx=0x7f82d000edf0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 shutdown_connections 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f82bc03b9c0 0x7f82bc03de80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82d8134d70 0x7f82d813ead0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82d809dd20 0x7f82d813e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 --2- 172.21.15.17:0/3465028189 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f82d809d400 0x7f82d813e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 >> 172.21.15.17:0/3465028189 conn(0x7f82d80a66a0 msgr2=0x7f82d80a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 shutdown_connections 2024-09-15T21:37:32.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.006+0000 7f82dea9a640 1 -- 172.21.15.17:0/3465028189 wait complete. 2024-09-15T21:37:32.028 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:37:32.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient: get_monmap_and_config 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient: monmap: 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:32.158827+0000 2024-09-15T21:37:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:32.158827+0000 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:32.160 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 Processor -- start 2024-09-15T21:37:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 -- start start 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient: init 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe7440a0800 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7440a12f0 0x7fe74413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): picked mon.noname-b con 0x7fe7440a12f0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe74413b0b0 0x7fe74413d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): picked mon.noname-a con 0x7fe74413b0b0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7440a1d30 con 0x7fe7440a0800 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe7440a2010 con 0x7fe74413b0b0 2024-09-15T21:37:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe7440a25d0 con 0x7fe7440a12f0 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe74993a640 10 monclient(hunting): authenticate will time out at 2175.554343s 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe742ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7440a0800 0x7fe7440a0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45462/0 (socket says 172.21.15.17:45462) 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe742ffd640 1 -- 172.21.15.17:0/708524859 learned_addr learned my addr 172.21.15.17:0/708524859 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 1 --2- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7440a12f0 0x7fe74413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 1 --2- 172.21.15.17:0/708524859 >> v2:172.21.15.17:3300/0 conn(0x7fe74413b0b0 0x7fe74413d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): get_auth_request con 0x7fe7440a12f0 auth_method 0 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): get_auth_request con 0x7fe74413b0b0 auth_method 0 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe741ffb640 1 -- 172.21.15.17:0/708524859 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576247285 0 0) 0x7fe7440a1d30 con 0x7fe7440a0800 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe741ffb640 10 monclient(hunting): _negotiate 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe741ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe741ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe741ffb640 10 monclient(hunting): my global_id is 204570 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe741ffb640 1 -- 172.21.15.17:0/708524859 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe72c003530 con 0x7fe7440a0800 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient(hunting): handle_auth_done global_id 204573 payload 274 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 1 -- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7440a12f0 msgr2=0x7fe74413ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 1 --2- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7440a12f0 0x7fe74413ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 1 -- 172.21.15.17:0/708524859 >> v1:172.21.15.17:6789/0 conn(0x7fe7440a0800 legacy=0x7fe7440a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 1 monclient: found mon.noname-a 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.154+0000 7fe7437fe640 1 -- 172.21.15.17:0/708524859 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74406f410 con 0x7fe74413b0b0 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 1 --2- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74413b0b0 0x7fe74413d4a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009dc0 tx=0x7fe73c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e49043b5e6e2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 1 -- 172.21.15.17:0/708524859 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0b2070 con 0x7fe74413b0b0 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: dump: 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 1 -- 172.21.15.17:0/708524859 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73c0a57e0 con 0x7fe74413b0b0 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 5 monclient: authenticate success, global_id 204573 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 1 -- 172.21.15.17:0/708524859 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0a5bb0 con 0x7fe74413b0b0 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 4 set_mon_vals no callback set 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe741ffb640 10 monclient: dump: 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: shutdown 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74413b0b0 msgr2=0x7fe74413d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 --2- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74413b0b0 0x7fe74413d4a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009dc0 tx=0x7fe73c01c810 comp rx=0 tx=0).stop 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- 172.21.15.17:0/708524859 shutdown_connections 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 --2- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74413b0b0 0x7fe74413d4a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 --2- 172.21.15.17:0/708524859 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe7440a12f0 0x7fe74413ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- 172.21.15.17:0/708524859 >> 172.21.15.17:0/708524859 conn(0x7fe7440a6820 msgr2=0x7fe7440a2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- 172.21.15.17:0/708524859 shutdown_connections 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- 172.21.15.17:0/708524859 wait complete. 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: monmap: 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 Processor -- start 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- start start 2024-09-15T21:37:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: init 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7440a0800 0x7fe7441cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): picked mon.noname-b con 0x7fe7440a0800 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7440a12f0 0x7fe744132820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): picked mon.noname-a con 0x7fe7440a12f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe74413b0b0 0x7fe744070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): picked mon.noname-c con 0x7fe74413b0b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe744140380 con 0x7fe7440a12f0 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe744140200 con 0x7fe7440a0800 2024-09-15T21:37:32.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fe744140500 con 0x7fe74413b0b0 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe74993a640 10 monclient(hunting): authenticate will time out at 2175.559615s 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7440a12f0 0x7fe744132820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe74413b0b0 0x7fe744070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7440a12f0 0x7fe744132820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35538/0 (socket says 172.21.15.17:35538) 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe74413b0b0 0x7fe744070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57556/0 (socket says 172.21.15.17:57556) 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 1 -- 172.21.15.17:0/3366046594 learned_addr learned my addr 172.21.15.17:0/3366046594 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): get_auth_request con 0x7fe7440a12f0 auth_method 0 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): get_auth_request con 0x7fe74413b0b0 auth_method 0 2024-09-15T21:37:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe7437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.158+0000 7fe742ffd640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7440a0800 0x7fe7441cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 10 monclient(hunting): get_auth_request con 0x7fe7440a0800 auth_method 0 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 10 monclient(hunting): handle_auth_done global_id 204576 payload 824 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 -- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe74413b0b0 msgr2=0x7fe744070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fe74413b0b0 0x7fe744070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 -- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7440a0800 msgr2=0x7fe7441cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7440a0800 0x7fe7441cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 monclient: found mon.noname-a 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 -- 172.21.15.17:0/3366046594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe744136490 con 0x7fe7440a12f0 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7427fc640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7440a12f0 0x7fe744132820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e930 tx=0x7fe73800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738017070 con 0x7fe7440a12f0 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: dump: 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73800ca00 con 0x7fe7440a12f0 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 5 monclient: authenticate success, global_id 204576 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 10 monclient: _renew_subs 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74411cbf0 con 0x7fe7440a12f0 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800cdb0 con 0x7fe7440a12f0 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: dump: 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7417fa640 4 set_mon_vals no callback set 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 10 monclient: _renew_subs 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7440721b0 con 0x7fe7440a12f0 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7440a0be0 con 0x7fe7440a12f0 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fe73801a070 con 0x7fe7440a12f0 2024-09-15T21:37:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe73003b9c0 0x7fe73003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fe738050740 con 0x7fe7440a12f0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7fe730044920 0x7fe730046d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 --> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe730047450 con 0x7fe730044920 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7fe738057050 con 0x7fe7440a12f0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 10 monclient: handle_get_version_reply finishing 1 version 261 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7437fe640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7fe730044920 0x7fe730046d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7437fe640 10 monclient: get_auth_request con 0x7fe730044920 auth_method 0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe73003b9c0 0x7fe73003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 10 monclient: get_auth_request con 0x7fe73003b9c0 auth_method 0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe7437fe640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7fe730044920 0x7fe730046d80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe742ffd640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe73003b9c0 0x7fe73003de80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe734007a70 tx=0x7fe734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.162+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== osd.2 v2:172.21.15.17:6800/671096025 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe730047450 con 0x7fe730044920 2024-09-15T21:37:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 --> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe74411c450 con 0x7fe730044920 2024-09-15T21:37:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe72bfff640 1 -- 172.21.15.17:0/3366046594 <== osd.2 v2:172.21.15.17:6800/671096025 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe74411c450 con 0x7fe730044920 2024-09-15T21:37:32.186 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:32.186 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:37:32.186 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7fe730044920 msgr2=0x7fe730046d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:6800/671096025,v1:172.21.15.17:6801/671096025] conn(0x7fe730044920 0x7fe730046d80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe73003b9c0 msgr2=0x7fe73003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fe73003b9c0 0x7fe73003de80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe734007a70 tx=0x7fe734009040 comp rx=0 tx=0).stop 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 10 monclient: shutdown 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7440a12f0 msgr2=0x7fe744132820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 --2- 172.21.15.17:0/3366046594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7440a12f0 0x7fe744132820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e930 tx=0x7fe73800edf0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe7437fe640 1 -- 172.21.15.17:0/3366046594 reap_dead start 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 shutdown_connections 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 >> 172.21.15.17:0/3366046594 conn(0x7fe7440a6820 msgr2=0x7fe7441357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 shutdown_connections 2024-09-15T21:37:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.178+0000 7fe74993a640 1 -- 172.21.15.17:0/3366046594 wait complete. 2024-09-15T21:37:32.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:37:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient: get_monmap_and_config 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient: monmap: 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:32.314377+0000 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:32.314377+0000 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:32.316 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 Processor -- start 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 -- start start 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient: init 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f501378c0 0x7f9f50137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): picked mon.noname-b con 0x7f9f501378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9f501381e0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f500a5210 0x7f9f500a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): picked mon.noname-a con 0x7f9f500a5210 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f500a1d00 con 0x7f9f501381e0 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9f500a1fe0 con 0x7f9f500a5210 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f500a22c0 con 0x7f9f501378c0 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f5730c640 10 monclient(hunting): authenticate will time out at 2175.709765s 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f500a5210 0x7f9f500a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f501378c0 0x7f9f50137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f500a5210 0x7f9f500a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35548/0 (socket says 172.21.15.17:35548) 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 -- 172.21.15.17:0/2706449553 learned_addr learned my addr 172.21.15.17:0/2706449553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): get_auth_request con 0x7f9f500a5210 auth_method 0 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): get_auth_request con 0x7f9f501378c0 auth_method 0 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient(hunting): handle_auth_done global_id 204579 payload 274 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 -- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f501378c0 msgr2=0x7f9f50137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 --2- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f501378c0 0x7f9f50137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 -- 172.21.15.17:0/2706449553 >> v1:172.21.15.119:6789/0 conn(0x7f9f501381e0 legacy=0x7f9f500a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 monclient: found mon.noname-a 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f558a9640 1 -- 172.21.15.17:0/2706449553 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f500a5c00 con 0x7f9f500a5210 2024-09-15T21:37:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.310+0000 7f9f550a8640 1 --2- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f501378c0 0x7f9f50137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f558a9640 1 --2- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500a5210 0x7f9f500a56c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009870 tx=0x7f9f4001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a46aa45c1906d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 1 -- 172.21.15.17:0/2706449553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400b0070 con 0x7f9f500a5210 2024-09-15T21:37:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: dump: 2024-09-15T21:37:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 1 -- 172.21.15.17:0/2706449553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f400ab600 con 0x7f9f500a5210 2024-09-15T21:37:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 1 -- 172.21.15.17:0/2706449553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400ab9d0 con 0x7f9f500a5210 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 5 monclient: authenticate success, global_id 204579 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 4 set_mon_vals no callback set 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4ffff640 10 monclient: dump: 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: shutdown 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500a5210 msgr2=0x7f9f500a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 --2- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500a5210 0x7f9f500a56c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009870 tx=0x7f9f4001c420 comp rx=0 tx=0).stop 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- 172.21.15.17:0/2706449553 shutdown_connections 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 --2- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f500a5210 0x7f9f500a56c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 --2- 172.21.15.17:0/2706449553 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f501378c0 0x7f9f50137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- 172.21.15.17:0/2706449553 >> 172.21.15.17:0/2706449553 conn(0x7f9f500aac40 msgr2=0x7f9f500a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- 172.21.15.17:0/2706449553 shutdown_connections 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- 172.21.15.17:0/2706449553 wait complete. 2024-09-15T21:37:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: monmap: 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 Processor -- start 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- start start 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: init 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 0x7f9f501cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): picked mon.noname-c con 0x7f9f500a5210 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f501378c0 0x7f9f501369b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): picked mon.noname-a con 0x7f9f501378c0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f501381e0 0x7f9f5013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): picked mon.noname-b con 0x7f9f501381e0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501425c0 con 0x7f9f501378c0 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501386d0 con 0x7f9f501381e0 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f9f50142740 con 0x7f9f500a5210 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient(hunting): authenticate will time out at 2175.714913s 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 0x7f9f501cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 0x7f9f501cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57576/0 (socket says 172.21.15.17:57576) 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 1 -- 172.21.15.17:0/1474828755 learned_addr learned my addr 172.21.15.17:0/1474828755 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): get_auth_request con 0x7f9f500a5210 auth_method 0 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f501378c0 0x7f9f501369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): get_auth_request con 0x7f9f501378c0 auth_method 0 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f558a9640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f501381e0 0x7f9f5013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f558a9640 10 monclient(hunting): get_auth_request con 0x7f9f501381e0 auth_method 0 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f558a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f558a9640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f558a9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient(hunting): handle_auth_done global_id 204582 payload 824 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 -- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 msgr2=0x7f9f501cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 0x7f9f501cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 -- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f501381e0 msgr2=0x7f9f5013cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f501381e0 0x7f9f5013cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 monclient: found mon.noname-a 2024-09-15T21:37:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 -- 172.21.15.17:0/1474828755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5013e630 con 0x7f9f501378c0 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f550a8640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 0x7f9f501cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f548a7640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f501378c0 0x7f9f501369b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400c390 tx=0x7f9f4400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44017070 con 0x7f9f501378c0 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: dump: 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f440052d0 con 0x7f9f501378c0 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44003800 con 0x7f9f501378c0 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4dffb640 10 monclient: dump: 2024-09-15T21:37:32.336 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f4f7fe640 4 set_mon_vals no callback set 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 5 monclient: authenticate success, global_id 204582 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: _renew_subs 2024-09-15T21:37:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.314+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501d0840 con 0x7f9f501378c0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 10 monclient: _renew_subs 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f50121060 con 0x7f9f501378c0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f500a56c0 con 0x7f9f501378c0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f9f4401a070 con 0x7f9f501378c0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f4dffb640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3c03b9c0 0x7f9f3c03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f9f4404f8b0 con 0x7f9f501378c0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f550a8640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3c03b9c0 0x7f9f3c03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f550a8640 10 monclient: get_auth_request con 0x7f9f3c03b9c0 auth_method 0 2024-09-15T21:37:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.318+0000 7f9f550a8640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3c03b9c0 0x7f9f3c03de80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009bd0 tx=0x7f9f4800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.322+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f9f501d0840 con 0x7f9f501378c0 2024-09-15T21:37:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.322+0000 7f9f4dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:32.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.322+0000 7f9f4dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.486+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f9f500a1270 con 0x7f9f3c03b9c0 2024-09-15T21:37:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.486+0000 7f9f4dffb640 1 -- 172.21.15.17:0/1474828755 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9f500a1270 con 0x7f9f3c03b9c0 2024-09-15T21:37:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:37:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3c03b9c0 msgr2=0x7f9f3c03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3c03b9c0 0x7f9f3c03de80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009bd0 tx=0x7f9f4800b040 comp rx=0 tx=0).stop 2024-09-15T21:37:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 10 monclient: shutdown 2024-09-15T21:37:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f501378c0 msgr2=0x7f9f501369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f501378c0 0x7f9f501369b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400c390 tx=0x7f9f4400c850 comp rx=0 tx=0).stop 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 shutdown_connections 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f9f3c03b9c0 0x7f9f3c03de80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f501381e0 0x7f9f5013cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f501378c0 0x7f9f501369b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 --2- 172.21.15.17:0/1474828755 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f9f500a5210 0x7f9f501cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 >> 172.21.15.17:0/1474828755 conn(0x7f9f500aac40 msgr2=0x7f9f5009da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 shutdown_connections 2024-09-15T21:37:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.490+0000 7f9f5730c640 1 -- 172.21.15.17:0/1474828755 wait complete. 2024-09-15T21:37:32.510 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T21:37:32.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:37:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.630+0000 7f4a221cb640 10 monclient: get_monmap_and_config 2024-09-15T21:37:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.630+0000 7f4a221cb640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.630+0000 7f4a221cb640 10 monclient: monmap: 2024-09-15T21:37:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:32.636108+0000 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:32.636108+0000 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:32.637 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.630+0000 7f4a221cb640 1 Processor -- start 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.630+0000 7f4a221cb640 1 -- start start 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.630+0000 7f4a221cb640 10 monclient: init 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4a1c0a4b00 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a1c0a2c60 0x7f4a1c0a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): picked mon.noname-a con 0x7f4a1c0a2c60 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): picked mon.noname-b con 0x7f4a1c13cd10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a1c0a5ea0 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c0a1ce0 con 0x7f4a1c0a2c60 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c0a1fc0 con 0x7f4a1c13cd10 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient(hunting): authenticate will time out at 2176.032328s 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4a1c0a4b00 0x7f4a1c0a4ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45472/0 (socket says 172.21.15.17:45472) 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a1c0a2c60 0x7f4a1c0a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1b7fe640 1 -- 172.21.15.17:0/619776180 learned_addr learned my addr 172.21.15.17:0/619776180 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): get_auth_request con 0x7f4a1c13cd10 auth_method 0 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): get_auth_request con 0x7f4a1c0a2c60 auth_method 0 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient(hunting): handle_auth_done global_id 204585 payload 274 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 -- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 msgr2=0x7f4a1c13f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 --2- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c13f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 -- 172.21.15.17:0/619776180 >> v1:172.21.15.17:6789/0 conn(0x7f4a1c0a4b00 legacy=0x7f4a1c0a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 monclient: found mon.noname-a 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 -- 172.21.15.17:0/619776180 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c0a22a0 con 0x7f4a1c0a2c60 2024-09-15T21:37:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1bfff640 1 --2- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1affd640 1 --2- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a2c60 0x7f4a1c0a55f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a100097c0 tx=0x7f4a1001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539176ae97c8f642 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 1 -- 172.21.15.17:0/619776180 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100b0070 con 0x7f4a1c0a2c60 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: dump: 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 1 -- 172.21.15.17:0/619776180 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a100ab600 con 0x7f4a1c0a2c60 2024-09-15T21:37:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 5 monclient: authenticate success, global_id 204585 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 1 -- 172.21.15.17:0/619776180 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100ab9d0 con 0x7f4a1c0a2c60 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 4 set_mon_vals no callback set 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a1a7fc640 10 monclient: dump: 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 10 monclient: shutdown 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a2c60 msgr2=0x7f4a1c0a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 --2- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a2c60 0x7f4a1c0a55f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a100097c0 tx=0x7f4a1001c360 comp rx=0 tx=0).stop 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- 172.21.15.17:0/619776180 shutdown_connections 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 --2- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 --2- 172.21.15.17:0/619776180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a2c60 0x7f4a1c0a55f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- 172.21.15.17:0/619776180 >> 172.21.15.17:0/619776180 conn(0x7f4a1c0aabc0 msgr2=0x7f4a1c0a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- 172.21.15.17:0/619776180 shutdown_connections 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.634+0000 7f4a221cb640 1 -- 172.21.15.17:0/619776180 wait complete. 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: monmap: 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:32.652 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 Processor -- start 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- start start 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: init 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _add_conns ranks=[1,0,2] 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0a2c60 0x7f4a1c120cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): picked mon.noname-b con 0x7f4a1c0a2c60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a4b00 0x7f4a1c137b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): picked mon.noname-a con 0x7f4a1c0a4b00 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c1cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): picked mon.noname-c con 0x7f4a1c13cd10 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1422d0 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c142150 con 0x7f4a1c0a2c60 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c142450 con 0x7f4a1c13cd10 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient(hunting): authenticate will time out at 2176.037107s 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c1cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a4b00 0x7f4a1c137b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c1cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57586/0 (socket says 172.21.15.17:57586) 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 1 -- 172.21.15.17:0/2957641201 learned_addr learned my addr 172.21.15.17:0/2957641201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): get_auth_request con 0x7f4a1c13cd10 auth_method 0 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): get_auth_request con 0x7f4a1c0a4b00 auth_method 0 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0a2c60 0x7f4a1c120cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 10 monclient(hunting): get_auth_request con 0x7f4a1c0a2c60 auth_method 0 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient(hunting): handle_auth_done global_id 204591 payload 824 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 -- 172.21.15.17:0/2957641201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 msgr2=0x7f4a1c1cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c1cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 -- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0a2c60 msgr2=0x7f4a1c120cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0a2c60 0x7f4a1c120cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 monclient: found mon.noname-a 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 -- 172.21.15.17:0/2957641201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c0a12f0 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1bfff640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4a1c13cd10 0x7f4a1c1cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1affd640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a4b00 0x7f4a1c137b60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a100ab000 tx=0x7f4a10019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100b8020 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: dump: 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.657 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a100b4410 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a100b0040 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: dump: 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a19ffb640 4 set_mon_vals no callback set 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 5 monclient: authenticate success, global_id 204591 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _renew_subs 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c13f2e0 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _renew_subs 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c121c50 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a1c0a4ee0 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4a100bb020 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a0803b9c0 0x7f4a0803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4a100ea1c0 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7f4a08044920 0x7f4a08046d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 --> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a100ab600 con 0x7f4a08044920 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7f4a100f1050 con 0x7f4a1c0a4b00 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a03fff640 10 monclient: handle_get_version_reply finishing 1 version 261 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a0803b9c0 0x7f4a0803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.638+0000 7f4a1b7fe640 10 monclient: get_auth_request con 0x7f4a0803b9c0 auth_method 0 2024-09-15T21:37:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.642+0000 7f4a1bfff640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7f4a08044920 0x7f4a08046d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.642+0000 7f4a1bfff640 10 monclient: get_auth_request con 0x7f4a08044920 auth_method 0 2024-09-15T21:37:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.642+0000 7f4a1b7fe640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a0803b9c0 0x7f4a0803de80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c007fd0 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.642+0000 7f4a1bfff640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7f4a08044920 0x7f4a08046d80 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.642+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== osd.3 v2:172.21.15.119:6800/3353277098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a100ab600 con 0x7f4a08044920 2024-09-15T21:37:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.654+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 --> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a1c1380a0 con 0x7f4a08044920 2024-09-15T21:37:32.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a03fff640 1 -- 172.21.15.17:0/2957641201 <== osd.3 v2:172.21.15.119:6800/3353277098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a1c1380a0 con 0x7f4a08044920 2024-09-15T21:37:32.662 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:32.662 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:37:32.662 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:32.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7f4a08044920 msgr2=0x7f4a08046d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6800/3353277098,v1:172.21.15.119:6801/3353277098] conn(0x7f4a08044920 0x7f4a08046d80 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a0803b9c0 msgr2=0x7f4a0803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4a0803b9c0 0x7f4a0803de80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c007fd0 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 10 monclient: shutdown 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a4b00 msgr2=0x7f4a1c137b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 --2- 172.21.15.17:0/2957641201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1c0a4b00 0x7f4a1c137b60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a100ab000 tx=0x7f4a10019100 comp rx=0 tx=0).stop 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a1bfff640 1 -- 172.21.15.17:0/2957641201 reap_dead start 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 shutdown_connections 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 >> 172.21.15.17:0/2957641201 conn(0x7f4a1c0aabc0 msgr2=0x7f4a1c13e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 shutdown_connections 2024-09-15T21:37:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.658+0000 7f4a221cb640 1 -- 172.21.15.17:0/2957641201 wait complete. 2024-09-15T21:37:32.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T21:37:32.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.782+0000 7fef685fd640 10 monclient: get_monmap_and_config 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.782+0000 7fef685fd640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.782+0000 7fef685fd640 10 monclient: monmap: 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:32.790050+0000 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:32.790050+0000 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:32.791 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:32.792 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:32.792 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 Processor -- start 2024-09-15T21:37:32.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 -- start start 2024-09-15T21:37:32.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient: init 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef600a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): picked mon.noname-b con 0x7fef600a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fef6009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef600a11b0 0x7fef60135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): picked mon.noname-c con 0x7fef600a11b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600a3130 con 0x7fef6009ffc0 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fef60073df0 con 0x7fef600a11b0 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fef600a1bf0 con 0x7fef600a0890 2024-09-15T21:37:32.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef685fd640 10 monclient(hunting): authenticate will time out at 2176.185001s 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef600a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef600a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57590/0 (socket says 172.21.15.17:57590) 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 -- 172.21.15.17:0/4174598720 learned_addr learned my addr 172.21.15.17:0/4174598720 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): get_auth_request con 0x7fef600a0890 auth_method 0 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66b9a640 1 --2- 172.21.15.17:0/4174598720 >> v2:172.21.15.119:3300/0 conn(0x7fef600a11b0 0x7fef60135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66b9a640 10 monclient(hunting): get_auth_request con 0x7fef600a11b0 auth_method 0 2024-09-15T21:37:32.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66b9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66b9a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66b9a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient(hunting): handle_auth_done global_id 234197 payload 274 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 -- 172.21.15.17:0/4174598720 >> v2:172.21.15.119:3300/0 conn(0x7fef600a11b0 msgr2=0x7fef60135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 --2- 172.21.15.17:0/4174598720 >> v2:172.21.15.119:3300/0 conn(0x7fef600a11b0 0x7fef60135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 -- 172.21.15.17:0/4174598720 >> v1:172.21.15.119:6789/0 conn(0x7fef6009ffc0 legacy=0x7fef60135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 monclient: found mon.noname-b 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:32.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.786+0000 7fef66399640 1 -- 172.21.15.17:0/4174598720 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef600a15e0 con 0x7fef600a0890 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 1 --2- 172.21.15.17:0/4174598720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef600a0c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef600a1870 tx=0x7fef5c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1c3a2431ebf823 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 1 -- 172.21.15.17:0/4174598720 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0b0070 con 0x7fef600a0890 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: dump: 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.803 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 1 -- 172.21.15.17:0/4174598720 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c0ab600 con 0x7fef600a0890 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 5 monclient: authenticate success, global_id 234197 2024-09-15T21:37:32.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 4 set_mon_vals no callback set 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 1 -- 172.21.15.17:0/4174598720 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0ab9d0 con 0x7fef600a0890 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65397640 10 monclient: dump: 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.805 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: shutdown 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/4174598720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 msgr2=0x7fef600a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 --2- 172.21.15.17:0/4174598720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef600a0c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef600a1870 tx=0x7fef5c01c400 comp rx=0 tx=0).stop 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/4174598720 shutdown_connections 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 --2- 172.21.15.17:0/4174598720 >> v2:172.21.15.119:3300/0 conn(0x7fef600a11b0 0x7fef60135d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 --2- 172.21.15.17:0/4174598720 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef600a0c70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/4174598720 >> 172.21.15.17:0/4174598720 conn(0x7fef600a66a0 msgr2=0x7fef600a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/4174598720 shutdown_connections 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/4174598720 wait complete. 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: build_initial_monmap 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: monmap: 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 Processor -- start 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- start start 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: init 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef6009ffc0 0x7fef6011e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): picked mon.noname-a con 0x7fef6009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef601351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): picked mon.noname-c con 0x7fef600a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef600a11b0 0x7fef6011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): picked mon.noname-b con 0x7fef600a11b0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fef60140510 con 0x7fef6009ffc0 2024-09-15T21:37:32.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef601364d0 con 0x7fef600a11b0 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fef60140690 con 0x7fef600a0890 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient(hunting): authenticate will time out at 2176.190725s 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef601351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef6009ffc0 0x7fef6011e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef601351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57602/0 (socket says 172.21.15.17:57602) 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 -- 172.21.15.17:0/1751768399 learned_addr learned my addr 172.21.15.17:0/1751768399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): get_auth_request con 0x7fef600a0890 auth_method 0 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): get_auth_request con 0x7fef6009ffc0 auth_method 0 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66b9a640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef600a11b0 0x7fef6011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66b9a640 10 monclient(hunting): get_auth_request con 0x7fef600a11b0 auth_method 0 2024-09-15T21:37:32.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66b9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66b9a640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66b9a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient(hunting): handle_auth_done global_id 234200 payload 824 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 -- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef600a11b0 msgr2=0x7fef6011c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef600a11b0 0x7fef6011c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 -- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef6009ffc0 msgr2=0x7fef6011e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef6009ffc0 0x7fef6011e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient: _finish_hunting 0 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 monclient: found mon.noname-c 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 -- 172.21.15.17:0/1751768399 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6011fb80 con 0x7fef600a0890 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66b9a640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef600a11b0 0x7fef6011c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:32.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef66399640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef6009ffc0 0x7fef6011e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef65b98640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef601351d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b440 tx=0x7fef5000b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50015020 con 0x7fef600a0890 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: dump: 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.812 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50011890 con 0x7fef600a0890 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50011c80 con 0x7fef600a0890 2024-09-15T21:37:32.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 5 monclient: authenticate success, global_id 234200 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: _renew_subs 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef60135e90 con 0x7fef600a0890 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef56ffd640 10 monclient: dump: 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:32.814 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef64b96640 4 set_mon_vals no callback set 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: _renew_subs 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.790+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601c5520 con 0x7fef600a0890 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef685fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef685fd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:32.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef685fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:32.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fef50018020 con 0x7fef600a0890 2024-09-15T21:37:32.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6011f800 con 0x7fef600a0890 2024-09-15T21:37:32.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef56ffd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fef4403b9c0 0x7fef4403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:32.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fef50050160 con 0x7fef600a0890 2024-09-15T21:37:32.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef66399640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fef4403b9c0 0x7fef4403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:32.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef66399640 10 monclient: get_auth_request con 0x7fef4403b9c0 auth_method 0 2024-09-15T21:37:32.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.794+0000 7fef66399640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fef4403b9c0 0x7fef4403de80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0ad000 tx=0x7fef5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:32.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.798+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fef5001f070 con 0x7fef600a0890 2024-09-15T21:37:32.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.798+0000 7fef56ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:32.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.798+0000 7fef56ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:32.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.970+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fef6006f9b0 con 0x7fef4403b9c0 2024-09-15T21:37:32.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.970+0000 7fef56ffd640 1 -- 172.21.15.17:0/1751768399 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fef6006f9b0 con 0x7fef4403b9c0 2024-09-15T21:37:32.977 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fef4403b9c0 msgr2=0x7fef4403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fef4403b9c0 0x7fef4403de80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0ad000 tx=0x7fef5c008040 comp rx=0 tx=0).stop 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 10 monclient: shutdown 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 msgr2=0x7fef601351d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef601351d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b440 tx=0x7fef5000b900 comp rx=0 tx=0).stop 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 shutdown_connections 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fef4403b9c0 0x7fef4403de80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef600a11b0 0x7fef6011c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fef600a0890 0x7fef601351d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 --2- 172.21.15.17:0/1751768399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef6009ffc0 0x7fef6011e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:32.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 >> 172.21.15.17:0/1751768399 conn(0x7fef600a66a0 msgr2=0x7fef600a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:32.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 shutdown_connections 2024-09-15T21:37:32.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:32.974+0000 7fef685fd640 1 -- 172.21.15.17:0/1751768399 wait complete. 2024-09-15T21:37:32.994 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T21:37:32.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:37:33.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient: get_monmap_and_config 2024-09-15T21:37:33.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient: monmap: 2024-09-15T21:37:33.120 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.120 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:33.119931+0000 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:33.119931+0000 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:33.121 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 1 Processor -- start 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 1 -- start start 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient: init 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e2809fdd0 0x7f5e280a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient(hunting): picked mon.noname-a con 0x7f5e2809fdd0 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e28074660 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e28139600 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2809df80 con 0x7f5e28074660 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2809e260 con 0x7f5e28139600 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2809e8b0 con 0x7f5e2809fdd0 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2fbdb640 10 monclient(hunting): authenticate will time out at 2176.515280s 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e2809fdd0 0x7f5e280a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d176640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5e28074660 0x7f5e280a0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45480/0 (socket says 172.21.15.17:45480) 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d176640 1 -- 172.21.15.17:0/4100883972 learned_addr learned my addr 172.21.15.17:0/4100883972 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 1 --2- 172.21.15.17:0/4100883972 >> v2:172.21.15.17:3300/0 conn(0x7f5e2809fdd0 0x7f5e280a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35580/0 (socket says 172.21.15.17:35580) 2024-09-15T21:37:33.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): get_auth_request con 0x7f5e2809fdd0 auth_method 0 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient(hunting): handle_auth_done global_id 204597 payload 274 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 1 -- 172.21.15.17:0/4100883972 >> v1:172.21.15.119:6789/0 conn(0x7f5e28139600 legacy=0x7f5e2813b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 1 -- 172.21.15.17:0/4100883972 >> v1:172.21.15.17:6789/0 conn(0x7f5e28074660 legacy=0x7f5e280a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 1 monclient: found mon.noname-a 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2d977640 1 -- 172.21.15.17:0/4100883972 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2809e540 con 0x7f5e2809fdd0 2024-09-15T21:37:33.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2c975640 1 -- 172.21.15.17:0/4100883972 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827342298 0 0) 0x7f5e2809df80 con 0x7f5e28074660 2024-09-15T21:37:33.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.114+0000 7f5e2c975640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:37:33.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2d977640 1 --2- 172.21.15.17:0/4100883972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2809fdd0 0x7f5e280a0800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0097c0 tx=0x7f5e1c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5832b491736441 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 1 -- 172.21.15.17:0/4100883972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0b1070 con 0x7f5e2809fdd0 2024-09-15T21:37:33.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:33.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: dump: 2024-09-15T21:37:33.131 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 1 -- 172.21.15.17:0/4100883972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0ac640 con 0x7f5e2809fdd0 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 5 monclient: authenticate success, global_id 204597 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 1 -- 172.21.15.17:0/4100883972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0aca10 con 0x7f5e2809fdd0 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 4 set_mon_vals no callback set 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2c975640 10 monclient: dump: 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.133 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: shutdown 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/4100883972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2809fdd0 msgr2=0x7f5e280a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 --2- 172.21.15.17:0/4100883972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2809fdd0 0x7f5e280a0800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0097c0 tx=0x7f5e1c01c480 comp rx=0 tx=0).stop 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/4100883972 shutdown_connections 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 --2- 172.21.15.17:0/4100883972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2809fdd0 0x7f5e280a0800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/4100883972 >> 172.21.15.17:0/4100883972 conn(0x7f5e280a6820 msgr2=0x7f5e280a2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/4100883972 shutdown_connections 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/4100883972 wait complete. 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: monmap: 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 Processor -- start 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- start start 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: init 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient: _add_conns ranks=[0,2,1] 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e28074660 0x7f5e28138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): picked mon.noname-a con 0x7f5e28074660 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2809fdd0 0x7f5e28138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): picked mon.noname-c con 0x7f5e2809fdd0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e28139600 0x7f5e28120fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): picked mon.noname-b con 0x7f5e28139600 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5e281403c0 con 0x7f5e28074660 2024-09-15T21:37:33.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28140240 con 0x7f5e28139600 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f5e28140540 con 0x7f5e2809fdd0 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.118+0000 7f5e2fbdb640 10 monclient(hunting): authenticate will time out at 2176.519996s 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e28074660 0x7f5e28138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2809fdd0 0x7f5e28138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e28074660 0x7f5e28138450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35586/0 (socket says 172.21.15.17:35586) 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 -- 172.21.15.17:0/1842159721 learned_addr learned my addr 172.21.15.17:0/1842159721 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): get_auth_request con 0x7f5e28074660 auth_method 0 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): get_auth_request con 0x7f5e2809fdd0 auth_method 0 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e28139600 0x7f5e28120fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 10 monclient(hunting): get_auth_request con 0x7f5e28139600 auth_method 0 2024-09-15T21:37:33.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient(hunting): handle_auth_done global_id 204603 payload 824 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 -- 172.21.15.17:0/1842159721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2809fdd0 msgr2=0x7f5e28138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f5e2809fdd0 0x7f5e28138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 -- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e28139600 msgr2=0x7f5e28120fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e28139600 0x7f5e28120fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 monclient: found mon.noname-a 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 -- 172.21.15.17:0/1842159721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e28139220 con 0x7f5e28074660 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d977640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e28074660 0x7f5e28138450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0af040 tx=0x7f5e1c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0b1050 con 0x7f5e28074660 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: dump: 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.140 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0bd070 con 0x7f5e28074660 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0b6a50 con 0x7f5e28074660 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 5 monclient: authenticate success, global_id 204603 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 10 monclient: _renew_subs 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e28124dc0 con 0x7f5e28074660 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1bfff640 4 set_mon_vals no callback set 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: dump: 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 10 monclient: _renew_subs 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e28124ae0 con 0x7f5e28074660 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e280a0800 con 0x7f5e28074660 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f5e1c0ac4e0 con 0x7f5e28074660 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9f0 0x7f5e0c03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f5e1c0eae30 con 0x7f5e28074660 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f5e0c044950 0x7f5e0c046db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 --> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e1c0a6ea0 con 0x7f5e0c044950 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7f5e1c0ed580 con 0x7f5e28074660 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e1a7fc640 10 monclient: handle_get_version_reply finishing 1 version 261 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9f0 0x7f5e0c03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 10 monclient: get_auth_request con 0x7f5e0c03b9f0 auth_method 0 2024-09-15T21:37:33.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f5e0c044950 0x7f5e0c046db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 10 monclient: get_auth_request con 0x7f5e0c044950 auth_method 0 2024-09-15T21:37:33.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2d176640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9f0 0x7f5e0c03deb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007fd0 tx=0x7f5e24007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.122+0000 7f5e2e178640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f5e0c044950 0x7f5e0c046db0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.126+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== osd.4 v2:172.21.15.119:6816/2086984234 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e1c0a6ea0 con 0x7f5e0c044950 2024-09-15T21:37:33.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 --> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5e281214e0 con 0x7f5e0c044950 2024-09-15T21:37:33.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e1a7fc640 1 -- 172.21.15.17:0/1842159721 <== osd.4 v2:172.21.15.119:6816/2086984234 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5e281214e0 con 0x7f5e0c044950 2024-09-15T21:37:33.145 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:33.145 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:37:33.145 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f5e0c044950 msgr2=0x7f5e0c046db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6816/2086984234,v1:172.21.15.119:6817/2086984234] conn(0x7f5e0c044950 0x7f5e0c046db0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9f0 msgr2=0x7f5e0c03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f5e0c03b9f0 0x7f5e0c03deb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007fd0 tx=0x7f5e24007ed0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 10 monclient: shutdown 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e28074660 msgr2=0x7f5e28138450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 --2- 172.21.15.17:0/1842159721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e28074660 0x7f5e28138450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0af040 tx=0x7f5e1c01acc0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2e178640 1 -- 172.21.15.17:0/1842159721 reap_dead start 2024-09-15T21:37:33.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 shutdown_connections 2024-09-15T21:37:33.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 >> 172.21.15.17:0/1842159721 conn(0x7f5e280a6820 msgr2=0x7f5e280a4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 shutdown_connections 2024-09-15T21:37:33.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.138+0000 7f5e2fbdb640 1 -- 172.21.15.17:0/1842159721 wait complete. 2024-09-15T21:37:33.159 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T21:37:33.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.258+0000 7f2517c5a640 10 monclient: get_monmap_and_config 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.258+0000 7f2517c5a640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.258+0000 7f2517c5a640 10 monclient: monmap: 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:33.265278+0000 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:33.265278+0000 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:33.267 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.258+0000 7f2517c5a640 1 Processor -- start 2024-09-15T21:37:33.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.258+0000 7f2517c5a640 1 -- start start 2024-09-15T21:37:33.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.258+0000 7f2517c5a640 10 monclient: init 2024-09-15T21:37:33.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25100a0890 0x7f25100a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient(hunting): picked mon.noname-c con 0x7f25100a0890 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f251009ffc0 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f25100a11b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2510136490 con 0x7f25100a11b0 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25100a3130 con 0x7f251009ffc0 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2510073df0 con 0x7f25100a0890 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 10 monclient(hunting): authenticate will time out at 2176.660568s 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f25100a11b0 0x7f2510135d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45490/0 (socket says 172.21.15.17:45490) 2024-09-15T21:37:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 -- 172.21.15.17:0/3885910473 learned_addr learned my addr 172.21.15.17:0/3885910473 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 1 --2- 172.21.15.17:0/3885910473 >> v2:172.21.15.119:3300/0 conn(0x7f25100a0890 0x7f25100a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): get_auth_request con 0x7f25100a0890 auth_method 0 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71370033 0 0) 0x7f2510136490 con 0x7f25100a11b0 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): _negotiate 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): my global_id is 204606 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24fc003530 con 0x7f25100a11b0 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 965260636 0 0) 0x7f24fc003530 con 0x7f25100a11b0 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 >> v2:172.21.15.119:3300/0 conn(0x7f25100a0890 msgr2=0x7f25100a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 --2- 172.21.15.17:0/3885910473 >> v2:172.21.15.119:3300/0 conn(0x7f25100a0890 0x7f25100a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 >> v1:172.21.15.119:6789/0 conn(0x7f251009ffc0 legacy=0x7f2510135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 monclient: found mon.noname-a-legacy 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.17:6789/0 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25100a1bf0 con 0x7f25100a11b0 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2885950372 0 0) 0x7f250c001700 con 0x7f25100a11b0 2024-09-15T21:37:33.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: dump: 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] but i'm connected to v1:172.21.15.17:6789/0, reconnecting 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 >> v1:172.21.15.17:6789/0 conn(0x7f25100a11b0 legacy=0x7f2510135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc003a60 0x7f24fc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): picked mon.b con 0x7f24fc003a60 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc004810 0x7f24fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): picked mon.c con 0x7f24fc004810 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24fc0053d0 0x7f24fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): picked mon.a con 0x7f24fc0053d0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2510073df0 con 0x7f24fc0053d0 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc00c240 con 0x7f24fc003a60 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc00c440 con 0x7f24fc004810 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25149f4640 10 monclient(hunting): _finish_auth 0 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24fc0053d0 0x7f24fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc004810 0x7f24fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): get_auth_request con 0x7f24fc0053d0 auth_method 0 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): get_auth_request con 0x7f24fc004810 auth_method 0 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25151f5640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient(hunting): handle_auth_done global_id 204606 payload 293 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 -- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc004810 msgr2=0x7f24fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc004810 0x7f24fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc003a60 0x7f24fc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 -- 172.21.15.17:0/3885910473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc003a60 msgr2=0x7f24fc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc003a60 0x7f24fc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 monclient: found mon.a 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 -- 172.21.15.17:0/3885910473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc003530 con 0x7f24fc0053d0 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 5 monclient: authenticate success, global_id 204606 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f2517c5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 10 monclient: get_auth_request con 0x7f24fc003a60 auth_method 0 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25159f6640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc003a60 0x7f24fc003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 -- 172.21.15.17:0/3885910473 reap_dead start 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.262+0000 7f25161f7640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24fc0053d0 0x7f24fc00b9d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f250c001060 tx=0x7f250c01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9954a3f522606e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c007d20 con 0x7f24fc0053d0 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: dump: 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.280 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250c0a9070 con 0x7f24fc0053d0 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 1 -- 172.21.15.17:0/3885910473 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c005d60 con 0x7f24fc0053d0 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 4 set_mon_vals no callback set 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25149f4640 10 monclient: dump: 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: shutdown 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24fc0053d0 msgr2=0x7f24fc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24fc0053d0 0x7f24fc00b9d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f250c001060 tx=0x7f250c01cb80 comp rx=0 tx=0).stop 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3885910473 shutdown_connections 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3885910473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24fc0053d0 0x7f24fc00b9d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3885910473 >> 172.21.15.17:0/3885910473 conn(0x7f25100a66a0 msgr2=0x7f25100a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3885910473 shutdown_connections 2024-09-15T21:37:33.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3885910473 wait complete. 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: monmap: 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:33.283 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 Processor -- start 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- start start 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: init 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 0x7f2510133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): picked mon.noname-c con 0x7f24fc0053d0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2510134510 0x7f251013c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): picked mon.noname-b con 0x7f2510134510 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25101348f0 0x7f251013cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): picked mon.noname-a con 0x7f25101348f0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f25100a1000 con 0x7f25101348f0 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25100a0e80 con 0x7f2510134510 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f2510135670 con 0x7f24fc0053d0 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient(hunting): authenticate will time out at 2176.666221s 2024-09-15T21:37:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 0x7f2510133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25101348f0 0x7f251013cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 0x7f2510133fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57630/0 (socket says 172.21.15.17:57630) 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 1 -- 172.21.15.17:0/3517877007 learned_addr learned my addr 172.21.15.17:0/3517877007 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): get_auth_request con 0x7f24fc0053d0 auth_method 0 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): get_auth_request con 0x7f25101348f0 auth_method 0 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25151f5640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2510134510 0x7f251013c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25151f5640 10 monclient(hunting): get_auth_request con 0x7f2510134510 auth_method 0 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25151f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25151f5640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25151f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient(hunting): handle_auth_done global_id 204609 payload 824 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 -- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 msgr2=0x7f2510133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 0x7f2510133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 -- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2510134510 msgr2=0x7f251013c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2510134510 0x7f251013c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 monclient: found mon.noname-a 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 -- 172.21.15.17:0/3517877007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251013e8e0 con 0x7f25101348f0 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25151f5640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2510134510 0x7f251013c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25159f6640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 0x7f2510133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25161f7640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25101348f0 0x7f251013cc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2510074910 tx=0x7f24fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0a5810 con 0x7f25101348f0 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: dump: 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.291 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250c0b2070 con 0x7f25101348f0 2024-09-15T21:37:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0a5bd0 con 0x7f25101348f0 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 10 monclient: dump: 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 5 monclient: authenticate success, global_id 204609 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25037fe640 4 set_mon_vals no callback set 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _renew_subs 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251011ca40 con 0x7f25101348f0 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _renew_subs 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251011dee0 con 0x7f25101348f0 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251009d0f0 con 0x7f25101348f0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.266+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f250c0047f0 con 0x7f25101348f0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.270+0000 7f25027fc640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f24f403b9c0 0x7f24f403de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.270+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f250c0e8eb0 con 0x7f25101348f0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.270+0000 7f25159f6640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f24f403b9c0 0x7f24f403de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.270+0000 7f25159f6640 10 monclient: get_auth_request con 0x7f24f403b9c0 auth_method 0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.270+0000 7f25159f6640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f24f403b9c0 0x7f24f403de80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f25080094c0 tx=0x7f250800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.274+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f250c016930 con 0x7f25101348f0 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.274+0000 7f25027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.274+0000 7f25027fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:33.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.438+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f2510135a00 con 0x7f24f403b9c0 2024-09-15T21:37:33.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.438+0000 7f25027fc640 1 -- 172.21.15.17:0/3517877007 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2510135a00 con 0x7f24f403b9c0 2024-09-15T21:37:33.444 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f24f403b9c0 msgr2=0x7f24f403de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f24f403b9c0 0x7f24f403de80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f25080094c0 tx=0x7f250800b040 comp rx=0 tx=0).stop 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 10 monclient: shutdown 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25101348f0 msgr2=0x7f251013cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25101348f0 0x7f251013cc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2510074910 tx=0x7f24fc005840 comp rx=0 tx=0).stop 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 shutdown_connections 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f24f403b9c0 0x7f24f403de80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25101348f0 0x7f251013cc50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2510134510 0x7f251013c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 --2- 172.21.15.17:0/3517877007 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f24fc0053d0 0x7f2510133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 >> 172.21.15.17:0/3517877007 conn(0x7f25100a66a0 msgr2=0x7f2510135c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 shutdown_connections 2024-09-15T21:37:33.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.442+0000 7f2517c5a640 1 -- 172.21.15.17:0/3517877007 wait complete. 2024-09-15T21:37:33.462 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T21:37:33.462 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient: get_monmap_and_config 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient: monmap: 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:33.579650+0000 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:33.579650+0000 2024-09-15T21:37:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:33.581 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 Processor -- start 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 -- start start 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient: init 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec0a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): picked mon.noname-b con 0x7f63ec0a0800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f63ec0a1120 0x7f63ec1355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): picked mon.noname-a con 0x7f63ec0a1120 addr v2:172.21.15.17:3300/0 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63ec135b20 0x7f63ec135f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): picked mon.noname-c con 0x7f63ec135b20 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec0a1d30 con 0x7f63ec0a1120 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec0a2010 con 0x7f63ec135b20 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec0a25d0 con 0x7f63ec0a0800 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63f29b0640 10 monclient(hunting): authenticate will time out at 2176.975025s 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec0a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec0a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57636/0 (socket says 172.21.15.17:57636) 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 -- 172.21.15.17:0/4110333404 learned_addr learned my addr 172.21.15.17:0/4110333404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): get_auth_request con 0x7f63ec0a0800 auth_method 0 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63eb7fe640 1 --2- 172.21.15.17:0/4110333404 >> v2:172.21.15.17:3300/0 conn(0x7f63ec0a1120 0x7f63ec1355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63eb7fe640 10 monclient(hunting): get_auth_request con 0x7f63ec0a1120 auth_method 0 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient(hunting): handle_auth_done global_id 234209 payload 274 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 -- 172.21.15.17:0/4110333404 >> v2:172.21.15.119:3300/0 conn(0x7f63ec135b20 msgr2=0x7f63ec135f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 --2- 172.21.15.17:0/4110333404 >> v2:172.21.15.119:3300/0 conn(0x7f63ec135b20 0x7f63ec135f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 -- 172.21.15.17:0/4110333404 >> v2:172.21.15.17:3300/0 conn(0x7f63ec0a1120 msgr2=0x7f63ec1355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 --2- 172.21.15.17:0/4110333404 >> v2:172.21.15.17:3300/0 conn(0x7f63ec0a1120 0x7f63ec1355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 monclient: found mon.noname-b 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63ebfff640 1 -- 172.21.15.17:0/4110333404 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec06f410 con 0x7f63ec0a0800 2024-09-15T21:37:33.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.574+0000 7f63eb7fe640 1 --2- 172.21.15.17:0/4110333404 >> v2:172.21.15.17:3300/0 conn(0x7f63ec0a1120 0x7f63ec1355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63ebfff640 1 --2- 172.21.15.17:0/4110333404 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec0a0be0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0a23b0 tx=0x7f63e401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b586689d40659d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 1 -- 172.21.15.17:0/4110333404 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e40a5750 con 0x7f63ec0a0800 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: dump: 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.592 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 1 -- 172.21.15.17:0/4110333404 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e40a58f0 con 0x7f63ec0a0800 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 5 monclient: authenticate success, global_id 234209 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:33.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 1 -- 172.21.15.17:0/4110333404 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e40a5cc0 con 0x7f63ec0a0800 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 4 set_mon_vals no callback set 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63eaffd640 10 monclient: dump: 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: shutdown 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- 172.21.15.17:0/4110333404 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 msgr2=0x7f63ec0a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- 172.21.15.17:0/4110333404 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec0a0be0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0a23b0 tx=0x7f63e401c8b0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- 172.21.15.17:0/4110333404 shutdown_connections 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- 172.21.15.17:0/4110333404 >> v2:172.21.15.119:3300/0 conn(0x7f63ec135b20 0x7f63ec135f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- 172.21.15.17:0/4110333404 >> v2:172.21.15.17:3300/0 conn(0x7f63ec0a1120 0x7f63ec1355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- 172.21.15.17:0/4110333404 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec0a0be0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- 172.21.15.17:0/4110333404 >> 172.21.15.17:0/4110333404 conn(0x7f63ec0a6820 msgr2=0x7f63ec0a2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- 172.21.15.17:0/4110333404 shutdown_connections 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- 172.21.15.17:0/4110333404 wait complete. 2024-09-15T21:37:33.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: monmap: 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 Processor -- start 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- start start 2024-09-15T21:37:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: init 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient: _add_conns ranks=[2,1,0] 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec11c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): picked mon.noname-c con 0x7f63ec0a0800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63ec0a1120 0x7f63ec13a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): picked mon.noname-b con 0x7f63ec0a1120 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63ec135b20 0x7f63ec1cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): picked mon.noname-a con 0x7f63ec135b20 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec0a01e0 con 0x7f63ec135b20 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec0a0060 con 0x7f63ec0a1120 2024-09-15T21:37:33.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec09d300 con 0x7f63ec0a0800 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.578+0000 7f63f29b0640 10 monclient(hunting): authenticate will time out at 2176.979982s 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec11c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec11c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57638/0 (socket says 172.21.15.17:57638) 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 -- 172.21.15.17:0/3847456852 learned_addr learned my addr 172.21.15.17:0/3847456852 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): get_auth_request con 0x7f63ec0a0800 auth_method 0 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63ec135b20 0x7f63ec1cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): get_auth_request con 0x7f63ec135b20 auth_method 0 2024-09-15T21:37:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63ec0a1120 0x7f63ec13a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 10 monclient(hunting): get_auth_request con 0x7f63ec0a1120 auth_method 0 2024-09-15T21:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient(hunting): handle_auth_done global_id 234212 payload 824 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 -- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63ec0a1120 msgr2=0x7f63ec13a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63ec0a1120 0x7f63ec13a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 -- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63ec135b20 msgr2=0x7f63ec1cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63ec135b20 0x7f63ec1cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 monclient: found mon.noname-c 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 -- 172.21.15.17:0/3847456852 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec11d340 con 0x7f63ec0a0800 2024-09-15T21:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63ec135b20 0x7f63ec1cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ebfff640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec11c9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f63e401af60 tx=0x7f63e4019650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e40bc020 con 0x7f63ec0a0800 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: dump: 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e40b8670 con 0x7f63ec0a0800 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e40b8a60 con 0x7f63ec0a0800 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: dump: 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 5 monclient: authenticate success, global_id 234212 2024-09-15T21:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 10 monclient: _renew_subs 2024-09-15T21:37:33.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:33.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63ea7fc640 4 set_mon_vals no callback set 2024-09-15T21:37:33.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec13b5a0 con 0x7f63ec0a0800 2024-09-15T21:37:33.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 10 monclient: _renew_subs 2024-09-15T21:37:33.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec11c450 con 0x7f63ec0a0800 2024-09-15T21:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63ec11e2d0 con 0x7f63ec0a0800 2024-09-15T21:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f63e40bf020 con 0x7f63ec0a0800 2024-09-15T21:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f63d003b9c0 0x7f63d003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f63e40eb540 con 0x7f63ec0a0800 2024-09-15T21:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f63d0044920 0x7f63d0046d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63d0047450 con 0x7f63d0044920 2024-09-15T21:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7f63e40f2050 con 0x7f63ec0a0800 2024-09-15T21:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63e8ff9640 10 monclient: handle_get_version_reply finishing 1 version 261 2024-09-15T21:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f63d003b9c0 0x7f63d003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 10 monclient: get_auth_request con 0x7f63d003b9c0 auth_method 0 2024-09-15T21:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f63d0044920 0x7f63d0046d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 10 monclient: get_auth_request con 0x7f63d0044920 auth_method 0 2024-09-15T21:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63eb7fe640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f63d003b9c0 0x7f63d003de80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0744d0 tx=0x7f63d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.582+0000 7f63f0f4d640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f63d0044920 0x7f63d0046d80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.586+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== osd.5 v2:172.21.15.119:6808/444741843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63d0047450 con 0x7f63d0044920 2024-09-15T21:37:33.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 --> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f63ec11de30 con 0x7f63d0044920 2024-09-15T21:37:33.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63e8ff9640 1 -- 172.21.15.17:0/3847456852 <== osd.5 v2:172.21.15.119:6808/444741843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f63ec11de30 con 0x7f63d0044920 2024-09-15T21:37:33.617 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-15T21:37:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:37:33.618 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-15T21:37:33.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f63d0044920 msgr2=0x7f63d0046d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6808/444741843,v1:172.21.15.119:6809/444741843] conn(0x7f63d0044920 0x7f63d0046d80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f63d003b9c0 msgr2=0x7f63d003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f63d003b9c0 0x7f63d003de80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0744d0 tx=0x7f63d4008040 comp rx=0 tx=0).stop 2024-09-15T21:37:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 10 monclient: shutdown 2024-09-15T21:37:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 msgr2=0x7f63ec11c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 --2- 172.21.15.17:0/3847456852 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f63ec0a0800 0x7f63ec11c9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f63e401af60 tx=0x7f63e4019650 comp rx=0 tx=0).stop 2024-09-15T21:37:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f0f4d640 1 -- 172.21.15.17:0/3847456852 reap_dead start 2024-09-15T21:37:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 shutdown_connections 2024-09-15T21:37:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 >> 172.21.15.17:0/3847456852 conn(0x7f63ec0a6820 msgr2=0x7f63ec13bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 shutdown_connections 2024-09-15T21:37:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.598+0000 7f63f29b0640 1 -- 172.21.15.17:0/3847456852 wait complete. 2024-09-15T21:37:33.623 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.730+0000 7fce14ec2640 10 monclient: get_monmap_and_config 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.730+0000 7fce14ec2640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.730+0000 7fce14ec2640 10 monclient: monmap: 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:33.737794+0000 2024-09-15T21:37:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:33.737794+0000 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:33.739 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 1 Processor -- start 2024-09-15T21:37:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 1 -- start start 2024-09-15T21:37:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient: init 2024-09-15T21:37:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009d400 0x7fce1009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient(hunting): picked mon.noname-b con 0x7fce1009d400 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fce1009dd20 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fce10134d70 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce100a3130 con 0x7fce1009dd20 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce1009e7a0 con 0x7fce10134d70 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fce100a1bc0 con 0x7fce1009d400 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient(hunting): authenticate will time out at 2177.133055s 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009d400 0x7fce1009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0dd9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fce1009dd20 0x7fce10134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45504/0 (socket says 172.21.15.17:45504) 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0dd9b640 1 -- 172.21.15.17:0/1434931692 learned_addr learned my addr 172.21.15.17:0/1434931692 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): get_auth_request con 0x7fce1009d400 auth_method 0 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient(hunting): handle_auth_done global_id 234215 payload 274 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 1 -- 172.21.15.17:0/1434931692 >> v1:172.21.15.119:6789/0 conn(0x7fce10134d70 legacy=0x7fce1013cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 1 -- 172.21.15.17:0/1434931692 >> v1:172.21.15.17:6789/0 conn(0x7fce1009dd20 legacy=0x7fce10134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 1 monclient: found mon.noname-b 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 1 -- 172.21.15.17:0/1434931692 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100a0320 con 0x7fce1009d400 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 1 -- 172.21.15.17:0/1434931692 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 748526110 0 0) 0x7fce100a3130 con 0x7fce1009dd20 2024-09-15T21:37:33.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-15T21:37:33.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0e59c640 1 --2- 172.21.15.17:0/1434931692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009d400 0x7fce1009d7e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80097f0 tx=0x7fcdf801e5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c94229df08a30d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 1 -- 172.21.15.17:0/1434931692 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80b2070 con 0x7fce1009d400 2024-09-15T21:37:33.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: dump: 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 5 monclient: authenticate success, global_id 234215 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 1 -- 172.21.15.17:0/1434931692 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf80ad600 con 0x7fce1009d400 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce14ec2640 4 set_mon_vals no callback set 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 1 -- 172.21.15.17:0/1434931692 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80ad9d0 con 0x7fce1009d400 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.734+0000 7fce0d59a640 10 monclient: dump: 2024-09-15T21:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: shutdown 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1434931692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009d400 msgr2=0x7fce1009d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1434931692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009d400 0x7fce1009d7e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80097f0 tx=0x7fcdf801e5b0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1434931692 shutdown_connections 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1434931692 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009d400 0x7fce1009d7e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1434931692 >> 172.21.15.17:0/1434931692 conn(0x7fce100a66a0 msgr2=0x7fce100a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1434931692 shutdown_connections 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1434931692 wait complete. 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: build_initial_monmap 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: monmap: 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 Processor -- start 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- start start 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: init 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce1009d400 0x7fce101d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): picked mon.noname-b con 0x7fce1009d400 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009dd20 0x7fce10133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): picked mon.noname-c con 0x7fce1009dd20 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 0x7fce1013a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): picked mon.noname-a con 0x7fce10134d70 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fce100a10a0 con 0x7fce10134d70 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fce100a3490 con 0x7fce1009d400 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7fce100a1220 con 0x7fce1009dd20 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient(hunting): authenticate will time out at 2177.137894s 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009dd20 0x7fce10133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 0x7fce1013a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 0x7fce1013a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35636/0 (socket says 172.21.15.17:35636) 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009dd20 0x7fce10133ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57642/0 (socket says 172.21.15.17:57642) 2024-09-15T21:37:33.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 -- 172.21.15.17:0/1467799865 learned_addr learned my addr 172.21.15.17:0/1467799865 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): get_auth_request con 0x7fce10134d70 auth_method 0 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): get_auth_request con 0x7fce1009dd20 auth_method 0 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:33.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0e59c640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce1009d400 0x7fce101d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0e59c640 10 monclient(hunting): get_auth_request con 0x7fce1009d400 auth_method 0 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient(hunting): handle_auth_done global_id 204621 payload 824 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 -- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009dd20 msgr2=0x7fce10133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009dd20 0x7fce10133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 -- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce1009d400 msgr2=0x7fce101d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce1009d400 0x7fce101d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient: _finish_hunting 0 2024-09-15T21:37:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 monclient: found mon.noname-a 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 -- 172.21.15.17:0/1467799865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce101d1040 con 0x7fce10134d70 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0ed9d640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 0x7fce1013a750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce0800c500 tx=0x7fce0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08017070 con 0x7fce10134d70 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: dump: 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce080052d0 con 0x7fce10134d70 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08003820 con 0x7fce10134d70 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce06ffd640 10 monclient: dump: 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce0cd99640 4 set_mon_vals no callback set 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 5 monclient: authenticate success, global_id 204621 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _renew_subs 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1013db10 con 0x7fce10134d70 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _renew_subs 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce100a2250 con 0x7fce10134d70 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.738+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1009d0f0 con 0x7fce10134d70 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.742+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7fce0801a070 con 0x7fce10134d70 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.742+0000 7fce06ffd640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcdf403b9f0 0x7fcdf403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.742+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7fce0804f980 con 0x7fce10134d70 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.742+0000 7fce0e59c640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcdf403b9f0 0x7fcdf403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.742+0000 7fce0e59c640 10 monclient: get_auth_request con 0x7fcdf403b9f0 auth_method 0 2024-09-15T21:37:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.742+0000 7fce0e59c640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcdf403b9f0 0x7fcdf403deb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80af000 tx=0x7fcdf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:33.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.746+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7fce1013db10 con 0x7fce10134d70 2024-09-15T21:37:33.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.746+0000 7fce06ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:33.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.746+0000 7fce06ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:33.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.906+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fce100a1680 con 0x7fcdf403b9f0 2024-09-15T21:37:33.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce06ffd640 1 -- 172.21.15.17:0/1467799865 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fce100a1680 con 0x7fcdf403b9f0 2024-09-15T21:37:33.915 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T21:37:33.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcdf403b9f0 msgr2=0x7fcdf403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcdf403b9f0 0x7fcdf403deb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80af000 tx=0x7fcdf8008040 comp rx=0 tx=0).stop 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 10 monclient: shutdown 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 msgr2=0x7fce1013a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 0x7fce1013a750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce0800c500 tx=0x7fce0800c9c0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 shutdown_connections 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7fcdf403b9f0 0x7fcdf403deb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce10134d70 0x7fce1013a750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7fce1009dd20 0x7fce10133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 --2- 172.21.15.17:0/1467799865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce1009d400 0x7fce101d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:33.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 >> 172.21.15.17:0/1467799865 conn(0x7fce100a66a0 msgr2=0x7fce100732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:33.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 shutdown_connections 2024-09-15T21:37:33.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:33.910+0000 7fce14ec2640 1 -- 172.21.15.17:0/1467799865 wait complete. 2024-09-15T21:37:33.932 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.042+0000 7f3a5951d640 10 monclient: get_monmap_and_config 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.042+0000 7f3a5951d640 10 monclient: build_initial_monmap 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.042+0000 7f3a5951d640 10 monclient: monmap: 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:34.049776+0000 2024-09-15T21:37:34.050 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:34.049776+0000 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:34.051 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 Processor -- start 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 -- start start 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient: init 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3a541378c0 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a541383b0 0x7f3a5413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): picked mon.noname-c con 0x7f3a541383b0 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a5413d2c0 0x7f3a5413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): picked mon.noname-b con 0x7f3a5413d2c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a540a1a20 con 0x7f3a541378c0 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3a540a1d00 con 0x7f3a541383b0 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3a540a1fe0 con 0x7f3a5413d2c0 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a5951d640 10 monclient(hunting): authenticate will time out at 2177.445507s 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a5413d2c0 0x7f3a5413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a52ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3a541378c0 0x7f3a54137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45518/0 (socket says 172.21.15.17:45518) 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): get_auth_request con 0x7f3a5413d2c0 auth_method 0 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a52ffd640 1 -- 172.21.15.17:0/622122804 learned_addr learned my addr 172.21.15.17:0/622122804 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:34.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient(hunting): handle_auth_done global_id 234221 payload 274 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 1 -- 172.21.15.17:0/622122804 >> v2:172.21.15.119:3300/0 conn(0x7f3a541383b0 msgr2=0x7f3a5413cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 1 --2- 172.21.15.17:0/622122804 >> v2:172.21.15.119:3300/0 conn(0x7f3a541383b0 0x7f3a5413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 1 -- 172.21.15.17:0/622122804 >> v1:172.21.15.17:6789/0 conn(0x7f3a541378c0 legacy=0x7f3a54137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient: _finish_hunting 0 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 1 monclient: found mon.noname-b 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:34.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.046+0000 7f3a537fe640 1 -- 172.21.15.17:0/622122804 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a540a22c0 con 0x7f3a5413d2c0 2024-09-15T21:37:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a537fe640 1 --2- 172.21.15.17:0/622122804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a5413d2c0 0x7f3a5413f6b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009dc0 tx=0x7f3a4c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720c7ab55a681632 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 1 -- 172.21.15.17:0/622122804 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c0b1070 con 0x7f3a5413d2c0 2024-09-15T21:37:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: dump: 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: _finish_auth 0 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: _check_auth_tickets 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 5 monclient: authenticate success, global_id 234221 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 1 -- 172.21.15.17:0/622122804 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4c0a4db0 con 0x7f3a5413d2c0 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 4 set_mon_vals no callback set 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 1 -- 172.21.15.17:0/622122804 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c0ac660 con 0x7f3a5413d2c0 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a51ffb640 10 monclient: dump: 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:34.062 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: shutdown 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/622122804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a5413d2c0 msgr2=0x7f3a5413f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 --2- 172.21.15.17:0/622122804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a5413d2c0 0x7f3a5413f6b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009dc0 tx=0x7f3a4c01c4f0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/622122804 shutdown_connections 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 --2- 172.21.15.17:0/622122804 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a5413d2c0 0x7f3a5413f6b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 --2- 172.21.15.17:0/622122804 >> v2:172.21.15.119:3300/0 conn(0x7f3a541383b0 0x7f3a5413cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/622122804 >> 172.21.15.17:0/622122804 conn(0x7f3a540aac40 msgr2=0x7f3a540a6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/622122804 shutdown_connections 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/622122804 wait complete. 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: build_initial_monmap 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: monmap: 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 Processor -- start 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- start start 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: init 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: _add_conns ranks=[2,0,1] 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 0x7f3a541d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): picked mon.noname-c con 0x7f3a541378c0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a541383b0 0x7f3a541d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): picked mon.noname-a con 0x7f3a541383b0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5413d2c0 0x7f3a5409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): picked mon.noname-b con 0x7f3a5413d2c0 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54143130 con 0x7f3a541383b0 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54142fb0 con 0x7f3a5413d2c0 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f3a541432b0 con 0x7f3a541378c0 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient(hunting): authenticate will time out at 2177.450514s 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 0x7f3a541d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:34.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 0x7f3a541d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:57660/0 (socket says 172.21.15.17:57660) 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 -- 172.21.15.17:0/2330932834 learned_addr learned my addr 172.21.15.17:0/2330932834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): get_auth_request con 0x7f3a541378c0 auth_method 0 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a541383b0 0x7f3a541d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): get_auth_request con 0x7f3a541383b0 auth_method 0 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:34.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient(hunting): handle_auth_done global_id 234224 payload 824 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a537fe640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5413d2c0 0x7f3a5409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 -- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5413d2c0 msgr2=0x7f3a5409dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5413d2c0 0x7f3a5409dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 -- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a541383b0 msgr2=0x7f3a541d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a541383b0 0x7f3a541d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient: _finish_hunting 0 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 monclient: found mon.noname-c 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:34.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 -- 172.21.15.17:0/2330932834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5409f770 con 0x7f3a541378c0 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a527fc640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a541383b0 0x7f3a541d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a537fe640 10 monclient: get_auth_request con 0x7f3a5413d2c0 auth_method 0 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a537fe640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5413d2c0 0x7f3a5409dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a52ffd640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 0x7f3a541d0890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400c4b0 tx=0x7f3a4400c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44017070 con 0x7f3a541378c0 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: dump: 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.069 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: _finish_auth 0 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: _check_auth_tickets 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a44005d10 con 0x7f3a541378c0 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a440072b0 con 0x7f3a541378c0 2024-09-15T21:37:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 5 monclient: authenticate success, global_id 234224 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a3b7fe640 10 monclient: dump: 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:34.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a517fa640 4 set_mon_vals no callback set 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: _renew_subs 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a54120e40 con 0x7f3a541378c0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: _renew_subs 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.050+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a54120750 con 0x7f3a541378c0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a5951d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a5951d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a5951d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a54137ca0 con 0x7f3a541378c0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f3a4401a070 con 0x7f3a541378c0 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a3b7fe640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a4003b9c0 0x7f3a4003de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:34.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f3a4404f8e0 con 0x7f3a541378c0 2024-09-15T21:37:34.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a527fc640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a4003b9c0 0x7f3a4003de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:34.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a527fc640 10 monclient: get_auth_request con 0x7f3a4003b9c0 auth_method 0 2024-09-15T21:37:34.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.054+0000 7f3a527fc640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a4003b9c0 0x7f3a4003de80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009d60 tx=0x7f3a48009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:34.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.058+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f3a540a1a20 con 0x7f3a541378c0 2024-09-15T21:37:34.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.058+0000 7f3a3b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:34.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.058+0000 7f3a3b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:34.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.230+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a54121590 con 0x7f3a4003b9c0 2024-09-15T21:37:34.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a3b7fe640 1 -- 172.21.15.17:0/2330932834 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+77978 (secure 0 0 0) 0x7f3a480099b0 con 0x7f3a4003b9c0 2024-09-15T21:37:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:37:34.240 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a4003b9c0 msgr2=0x7f3a4003de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a4003b9c0 0x7f3a4003de80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009d60 tx=0x7f3a48009110 comp rx=0 tx=0).stop 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 10 monclient: shutdown 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 msgr2=0x7f3a541d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 0x7f3a541d0890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400c4b0 tx=0x7f3a4400c970 comp rx=0 tx=0).stop 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 shutdown_connections 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f3a4003b9c0 0x7f3a4003de80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5413d2c0 0x7f3a5409dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a541383b0 0x7f3a541d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 --2- 172.21.15.17:0/2330932834 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f3a541378c0 0x7f3a541d0890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 >> 172.21.15.17:0/2330932834 conn(0x7f3a540aac40 msgr2=0x7f3a5406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 shutdown_connections 2024-09-15T21:37:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:34.234+0000 7f3a5951d640 1 -- 172.21.15.17:0/2330932834 wait complete. 2024-09-15T21:37:34.258 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":856,"stamp":"2024-09-15T21:37:33.531591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":215504,"kb_used_data":20056,"kb_used_omap":9,"kb_used_meta":195318,"kb_avail":566015536,"statfs":{"total":579820584960,"available":579599908864,"internally_reserved":0,"allocated":20537344,"data_stored":17290996,"data_compressed":1361822,"data_compressed_allocated":2695168,"data_compressed_original":6922240,"omap_allocated":9567,"internal_metadata":200006305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003813"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":302,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373667+0000","last_change":"2024-09-15T21:23:54.324070+0000","last_active":"2024-09-15T21:34:21.373667+0000","last_peered":"2024-09-15T21:34:21.373667+0000","last_clean":"2024-09-15T21:34:21.373667+0000","last_became_active":"2024-09-15T21:23:54.320552+0000","last_became_peered":"2024-09-15T21:23:54.320552+0000","last_unstale":"2024-09-15T21:34:21.373667+0000","last_undegraded":"2024-09-15T21:34:21.373667+0000","last_fullsized":"2024-09-15T21:34:21.373667+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:55.900283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256773+0000","last_change":"2024-09-15T21:23:54.301950+0000","last_active":"2024-09-15T21:34:46.256773+0000","last_peered":"2024-09-15T21:34:46.256773+0000","last_clean":"2024-09-15T21:34:46.256773+0000","last_became_active":"2024-09-15T21:23:54.301596+0000","last_became_peered":"2024-09-15T21:23:54.301596+0000","last_unstale":"2024-09-15T21:34:46.256773+0000","last_undegraded":"2024-09-15T21:34:46.256773+0000","last_fullsized":"2024-09-15T21:34:46.256773+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:07.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.251301+0000","last_change":"2024-09-15T21:23:54.314550+0000","last_active":"2024-09-15T21:34:25.251301+0000","last_peered":"2024-09-15T21:34:25.251301+0000","last_clean":"2024-09-15T21:34:25.251301+0000","last_became_active":"2024-09-15T21:23:54.313525+0000","last_became_peered":"2024-09-15T21:23:54.313525+0000","last_unstale":"2024-09-15T21:34:25.251301+0000","last_undegraded":"2024-09-15T21:34:25.251301+0000","last_fullsized":"2024-09-15T21:34:25.251301+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:21:52.088930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":174,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.256725+0000","last_change":"2024-09-15T21:23:54.314288+0000","last_active":"2024-09-15T21:34:46.256725+0000","last_peered":"2024-09-15T21:34:46.256725+0000","last_clean":"2024-09-15T21:34:46.256725+0000","last_became_active":"2024-09-15T21:23:54.313839+0000","last_became_peered":"2024-09-15T21:23:54.313839+0000","last_unstale":"2024-09-15T21:34:46.256725+0000","last_undegraded":"2024-09-15T21:34:46.256725+0000","last_fullsized":"2024-09-15T21:34:46.256725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:09.847621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":231,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:51.257506+0000","last_change":"2024-09-15T21:23:55.902295+0000","last_active":"2024-09-15T21:34:51.257506+0000","last_peered":"2024-09-15T21:34:51.257506+0000","last_clean":"2024-09-15T21:34:51.257506+0000","last_became_active":"2024-09-15T21:23:55.901252+0000","last_became_peered":"2024-09-15T21:23:55.901252+0000","last_unstale":"2024-09-15T21:34:51.257506+0000","last_undegraded":"2024-09-15T21:34:51.257506+0000","last_fullsized":"2024-09-15T21:34:51.257506+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:55.224562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":288,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373691+0000","last_change":"2024-09-15T21:23:54.323472+0000","last_active":"2024-09-15T21:34:21.373691+0000","last_peered":"2024-09-15T21:34:21.373691+0000","last_clean":"2024-09-15T21:34:21.373691+0000","last_became_active":"2024-09-15T21:23:54.318732+0000","last_became_peered":"2024-09-15T21:23:54.318732+0000","last_unstale":"2024-09-15T21:34:21.373691+0000","last_undegraded":"2024-09-15T21:34:21.373691+0000","last_fullsized":"2024-09-15T21:34:21.373691+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:32:38.556252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258088+0000","last_change":"2024-09-15T21:23:54.326082+0000","last_active":"2024-09-15T21:34:46.258088+0000","last_peered":"2024-09-15T21:34:46.258088+0000","last_clean":"2024-09-15T21:34:46.258088+0000","last_became_active":"2024-09-15T21:23:54.325406+0000","last_became_peered":"2024-09-15T21:23:54.325406+0000","last_unstale":"2024-09-15T21:34:46.258088+0000","last_undegraded":"2024-09-15T21:34:46.258088+0000","last_fullsized":"2024-09-15T21:34:46.258088+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:51:57.618330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380779+0000","last_change":"2024-09-15T21:23:54.301728+0000","last_active":"2024-09-15T21:33:57.380779+0000","last_peered":"2024-09-15T21:33:57.380779+0000","last_clean":"2024-09-15T21:33:57.380779+0000","last_became_active":"2024-09-15T21:23:54.300781+0000","last_became_peered":"2024-09-15T21:23:54.300781+0000","last_unstale":"2024-09-15T21:33:57.380779+0000","last_undegraded":"2024-09-15T21:33:57.380779+0000","last_fullsized":"2024-09-15T21:33:57.380779+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:02.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:06.250727+0000","last_change":"2024-09-15T21:23:54.308826+0000","last_active":"2024-09-15T21:34:06.250727+0000","last_peered":"2024-09-15T21:34:06.250727+0000","last_clean":"2024-09-15T21:34:06.250727+0000","last_became_active":"2024-09-15T21:23:54.307998+0000","last_became_peered":"2024-09-15T21:23:54.307998+0000","last_unstale":"2024-09-15T21:34:06.250727+0000","last_undegraded":"2024-09-15T21:34:06.250727+0000","last_fullsized":"2024-09-15T21:34:06.250727+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:32.478200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":216,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:31.255681+0000","last_change":"2024-09-15T21:23:54.311062+0000","last_active":"2024-09-15T21:34:31.255681+0000","last_peered":"2024-09-15T21:34:31.255681+0000","last_clean":"2024-09-15T21:34:31.255681+0000","last_became_active":"2024-09-15T21:23:54.310738+0000","last_became_peered":"2024-09-15T21:23:54.310738+0000","last_unstale":"2024-09-15T21:34:31.255681+0000","last_undegraded":"2024-09-15T21:34:31.255681+0000","last_fullsized":"2024-09-15T21:34:31.255681+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:17.678319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":232,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.253623+0000","last_change":"2024-09-15T21:23:54.312726+0000","last_active":"2024-09-15T21:34:21.253623+0000","last_peered":"2024-09-15T21:34:21.253623+0000","last_clean":"2024-09-15T21:34:21.253623+0000","last_became_active":"2024-09-15T21:23:54.311958+0000","last_became_peered":"2024-09-15T21:23:54.311958+0000","last_unstale":"2024-09-15T21:34:21.253623+0000","last_undegraded":"2024-09-15T21:34:21.253623+0000","last_fullsized":"2024-09-15T21:34:21.253623+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:15.944561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384735+0000","last_change":"2024-09-15T21:23:54.310984+0000","last_active":"2024-09-15T21:34:22.384735+0000","last_peered":"2024-09-15T21:34:22.384735+0000","last_clean":"2024-09-15T21:34:22.384735+0000","last_became_active":"2024-09-15T21:23:54.309483+0000","last_became_peered":"2024-09-15T21:23:54.309483+0000","last_unstale":"2024-09-15T21:34:22.384735+0000","last_undegraded":"2024-09-15T21:34:22.384735+0000","last_fullsized":"2024-09-15T21:34:22.384735+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:26.416656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384680+0000","last_change":"2024-09-15T21:23:54.313402+0000","last_active":"2024-09-15T21:34:22.384680+0000","last_peered":"2024-09-15T21:34:22.384680+0000","last_clean":"2024-09-15T21:34:22.384680+0000","last_became_active":"2024-09-15T21:23:54.312375+0000","last_became_peered":"2024-09-15T21:23:54.312375+0000","last_unstale":"2024-09-15T21:34:22.384680+0000","last_undegraded":"2024-09-15T21:34:22.384680+0000","last_fullsized":"2024-09-15T21:34:22.384680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:29.568805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":304,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.373629+0000","last_change":"2024-09-15T21:23:54.319437+0000","last_active":"2024-09-15T21:34:21.373629+0000","last_peered":"2024-09-15T21:34:21.373629+0000","last_clean":"2024-09-15T21:34:21.373629+0000","last_became_active":"2024-09-15T21:23:54.318494+0000","last_became_peered":"2024-09-15T21:23:54.318494+0000","last_unstale":"2024-09-15T21:34:21.373629+0000","last_undegraded":"2024-09-15T21:34:21.373629+0000","last_fullsized":"2024-09-15T21:34:21.373629+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:44:21.410329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250527+0000","last_change":"2024-09-15T21:23:55.306377+0000","last_active":"2024-09-15T21:34:20.250527+0000","last_peered":"2024-09-15T21:34:20.250527+0000","last_clean":"2024-09-15T21:34:20.250527+0000","last_became_active":"2024-09-15T21:23:55.305236+0000","last_became_peered":"2024-09-15T21:23:55.305236+0000","last_unstale":"2024-09-15T21:34:20.250527+0000","last_undegraded":"2024-09-15T21:34:20.250527+0000","last_fullsized":"2024-09-15T21:34:20.250527+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:48.783154+0000","last_clean_scrub_stamp":"2024-09-15T21:21:48.783154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:30.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:57.380708+0000","last_change":"2024-09-15T21:23:54.307012+0000","last_active":"2024-09-15T21:33:57.380708+0000","last_peered":"2024-09-15T21:33:57.380708+0000","last_clean":"2024-09-15T21:33:57.380708+0000","last_became_active":"2024-09-15T21:23:54.305792+0000","last_became_peered":"2024-09-15T21:23:54.305792+0000","last_unstale":"2024-09-15T21:33:57.380708+0000","last_undegraded":"2024-09-15T21:33:57.380708+0000","last_fullsized":"2024-09-15T21:33:57.380708+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:21:47.774527+0000","last_clean_scrub_stamp":"2024-09-15T21:21:47.774527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:33:47.178490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":533,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:01.251383+0000","last_change":"2024-09-15T21:23:54.325895+0000","last_active":"2024-09-15T21:34:01.251383+0000","last_peered":"2024-09-15T21:34:01.251383+0000","last_clean":"2024-09-15T21:34:01.251383+0000","last_became_active":"2024-09-15T21:23:54.325552+0000","last_became_peered":"2024-09-15T21:23:54.325552+0000","last_unstale":"2024-09-15T21:34:01.251383+0000","last_undegraded":"2024-09-15T21:34:01.251383+0000","last_fullsized":"2024-09-15T21:34:01.251383+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:31.232299+0000","last_clean_scrub_stamp":"2024-09-15T21:17:31.232299+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:57.451100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":451,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252971+0000","last_change":"2024-09-15T21:23:54.326302+0000","last_active":"2024-09-15T21:34:21.252971+0000","last_peered":"2024-09-15T21:34:21.252971+0000","last_clean":"2024-09-15T21:34:21.252971+0000","last_became_active":"2024-09-15T21:23:54.316513+0000","last_became_peered":"2024-09-15T21:23:54.316513+0000","last_unstale":"2024-09-15T21:34:21.252971+0000","last_undegraded":"2024-09-15T21:34:21.252971+0000","last_fullsized":"2024-09-15T21:34:21.252971+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:21.773590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:33:51.249994+0000","last_change":"2024-09-15T21:23:54.308012+0000","last_active":"2024-09-15T21:33:51.249994+0000","last_peered":"2024-09-15T21:33:51.249994+0000","last_clean":"2024-09-15T21:33:51.249994+0000","last_became_active":"2024-09-15T21:23:54.307605+0000","last_became_peered":"2024-09-15T21:23:54.307605+0000","last_unstale":"2024-09-15T21:33:51.249994+0000","last_undegraded":"2024-09-15T21:33:51.249994+0000","last_fullsized":"2024-09-15T21:33:51.249994+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:36.197287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.250456+0000","last_change":"2024-09-15T21:23:54.308207+0000","last_active":"2024-09-15T21:34:20.250456+0000","last_peered":"2024-09-15T21:34:20.250456+0000","last_clean":"2024-09-15T21:34:20.250456+0000","last_became_active":"2024-09-15T21:23:54.307364+0000","last_became_peered":"2024-09-15T21:23:54.307364+0000","last_unstale":"2024-09-15T21:34:20.250456+0000","last_undegraded":"2024-09-15T21:34:20.250456+0000","last_fullsized":"2024-09-15T21:34:20.250456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:31.761043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":461,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.252911+0000","last_change":"2024-09-15T21:23:54.322590+0000","last_active":"2024-09-15T21:34:21.252911+0000","last_peered":"2024-09-15T21:34:21.252911+0000","last_clean":"2024-09-15T21:34:21.252911+0000","last_became_active":"2024-09-15T21:23:54.314394+0000","last_became_peered":"2024-09-15T21:23:54.314394+0000","last_unstale":"2024-09-15T21:34:21.252911+0000","last_undegraded":"2024-09-15T21:34:21.252911+0000","last_fullsized":"2024-09-15T21:34:21.252911+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:54.033640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":439,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.384782+0000","last_change":"2024-09-15T21:23:55.901581+0000","last_active":"2024-09-15T21:34:22.384782+0000","last_peered":"2024-09-15T21:34:22.384782+0000","last_clean":"2024-09-15T21:34:22.384782+0000","last_became_active":"2024-09-15T21:23:55.900571+0000","last_became_peered":"2024-09-15T21:23:55.900571+0000","last_unstale":"2024-09-15T21:34:22.384782+0000","last_undegraded":"2024-09-15T21:34:22.384782+0000","last_fullsized":"2024-09-15T21:34:22.384782+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:22.834885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:26.254894+0000","last_change":"2024-09-15T21:23:54.311459+0000","last_active":"2024-09-15T21:34:26.254894+0000","last_peered":"2024-09-15T21:34:26.254894+0000","last_clean":"2024-09-15T21:34:26.254894+0000","last_became_active":"2024-09-15T21:23:54.310902+0000","last_became_peered":"2024-09-15T21:23:54.310902+0000","last_unstale":"2024-09-15T21:34:26.254894+0000","last_undegraded":"2024-09-15T21:34:26.254894+0000","last_fullsized":"2024-09-15T21:34:26.254894+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:32.505577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:47.388767+0000","last_change":"2024-09-15T21:23:54.295908+0000","last_active":"2024-09-15T21:34:47.388767+0000","last_peered":"2024-09-15T21:34:47.388767+0000","last_clean":"2024-09-15T21:34:47.388767+0000","last_became_active":"2024-09-15T21:23:54.294899+0000","last_became_peered":"2024-09-15T21:23:54.294899+0000","last_unstale":"2024-09-15T21:34:47.388767+0000","last_undegraded":"2024-09-15T21:34:47.388767+0000","last_fullsized":"2024-09-15T21:34:47.388767+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:02.582888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":177,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:34:46.258125+0000","last_change":"2024-09-15T21:23:54.302593+0000","last_active":"2024-09-15T21:34:46.258125+0000","last_peered":"2024-09-15T21:34:46.258125+0000","last_clean":"2024-09-15T21:34:46.258125+0000","last_became_active":"2024-09-15T21:23:54.301715+0000","last_became_peered":"2024-09-15T21:23:54.301715+0000","last_unstale":"2024-09-15T21:34:46.258125+0000","last_undegraded":"2024-09-15T21:34:46.258125+0000","last_fullsized":"2024-09-15T21:34:46.258125+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:17:38.270426+0000","last_clean_scrub_stamp":"2024-09-15T21:17:38.270426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:51.716170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607797,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38676,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333164,"statfs":{"total":96636764160,"available":96597159936,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640499,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34516,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337324,"statfs":{"total":96636764160,"available":96601419776,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640498,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34512,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337328,"statfs":{"total":96636764160,"available":96601423872,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640499,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640499,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38672,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333168,"statfs":{"total":96636764160,"available":96597164032,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640500,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34564,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337276,"statfs":{"total":96636764160,"available":96601370624,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:37:34.262 INFO:tasks.ceph:pgid 10.4 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.262 INFO:tasks.ceph:pgid 9.7 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.262 INFO:tasks.ceph:pgid 10.5 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 9.6 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 10.6 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 9.5 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 10.7 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 9.4 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 10.0 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 9.3 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 10.1 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.263 INFO:tasks.ceph:pgid 9.2 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 10.2 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 9.1 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 10.3 last_scrub_stamp 2024-09-15T21:21:48.783154+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 9.0 last_scrub_stamp 2024-09-15T21:21:47.774527+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:17:31.232299+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=31, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.264 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.265 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.265 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.265 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.265 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.265 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T21:17:38.270426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=17, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=37, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:37:34.265 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:37:54.267 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:37:54.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient: get_monmap_and_config 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient: build_initial_monmap 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient: monmap: 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:54.404298+0000 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:54.404298+0000 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:54.406 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 Processor -- start 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 -- start start 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient: init 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 0x7f4ce00a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): picked mon.noname-b con 0x7f4ce00a0890 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4ce009ffc0 0x7f4ce00a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): picked mon.noname-c con 0x7f4ce009ffc0 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4ce0135670 addr v1:172.21.15.119:6789/0 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce00a3130 con 0x7f4ce0135670 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce00a16f0 con 0x7f4ce009ffc0 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce0073df0 con 0x7f4ce00a0890 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.398+0000 7f4ce6500640 10 monclient(hunting): authenticate will time out at 2197.799769s 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 0x7f4ce00a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 0x7f4ce00a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36720/0 (socket says 172.21.15.17:36720) 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 -- 172.21.15.17:0/2235321336 learned_addr learned my addr 172.21.15.17:0/2235321336 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): get_auth_request con 0x7f4ce00a0890 auth_method 0 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 1 --2- 172.21.15.17:0/2235321336 >> v2:172.21.15.119:3300/0 conn(0x7f4ce009ffc0 0x7f4ce00a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f4ce009ffc0 auth_method 0 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient(hunting): handle_auth_done global_id 234227 payload 274 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 -- 172.21.15.17:0/2235321336 >> v2:172.21.15.119:3300/0 conn(0x7f4ce009ffc0 msgr2=0x7f4ce00a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 --2- 172.21.15.17:0/2235321336 >> v2:172.21.15.119:3300/0 conn(0x7f4ce009ffc0 0x7f4ce00a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 -- 172.21.15.17:0/2235321336 >> v1:172.21.15.119:6789/0 conn(0x7f4ce0135670 legacy=0x7f4ce0135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient: _finish_hunting 0 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 monclient: found mon.noname-b 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 -- 172.21.15.17:0/2235321336 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce00a1bf0 con 0x7f4ce00a0890 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdffff640 1 --2- 172.21.15.17:0/2235321336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 0x7f4ce00a0c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00a1870 tx=0x7f4cd001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02293901b852014 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:54.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 1 -- 172.21.15.17:0/2235321336 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00b1070 con 0x7f4ce00a0890 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: dump: 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 1 -- 172.21.15.17:0/2235321336 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd00ac640 con 0x7f4ce00a0890 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 5 monclient: authenticate success, global_id 234227 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 1 -- 172.21.15.17:0/2235321336 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00aca10 con 0x7f4ce00a0890 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 4 set_mon_vals no callback set 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4cdeffd640 10 monclient: dump: 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 10 monclient: shutdown 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 -- 172.21.15.17:0/2235321336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 msgr2=0x7f4ce00a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 --2- 172.21.15.17:0/2235321336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 0x7f4ce00a0c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00a1870 tx=0x7f4cd001c400 comp rx=0 tx=0).stop 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 -- 172.21.15.17:0/2235321336 shutdown_connections 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 --2- 172.21.15.17:0/2235321336 >> v2:172.21.15.119:3300/0 conn(0x7f4ce009ffc0 0x7f4ce00a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 --2- 172.21.15.17:0/2235321336 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce00a0890 0x7f4ce00a0c70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 -- 172.21.15.17:0/2235321336 >> 172.21.15.17:0/2235321336 conn(0x7f4ce00a66a0 msgr2=0x7f4ce00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 -- 172.21.15.17:0/2235321336 shutdown_connections 2024-09-15T21:37:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.402+0000 7f4ce6500640 1 -- 172.21.15.17:0/2235321336 wait complete. 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: build_initial_monmap 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: monmap: 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 Processor -- start 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- start start 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: init 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _add_conns ranks=[0,1,2] 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 0x7f4ce013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): picked mon.noname-a con 0x7f4ce009ffc0 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00a0890 0x7f4ce009d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): picked mon.noname-b con 0x7f4ce00a0890 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 0x7f4ce01346a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): picked mon.noname-c con 0x7f4ce0135670 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce0140690 con 0x7f4ce009ffc0 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce0140510 con 0x7f4ce00a0890 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce0140810 con 0x7f4ce0135670 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient(hunting): authenticate will time out at 2197.805139s 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 0x7f4ce01346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 0x7f4ce013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 0x7f4ce013c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58782/0 (socket says 172.21.15.17:58782) 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 0x7f4ce01346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3301/0 says I am v2:172.21.15.17:36722/0 (socket says 172.21.15.17:36722) 2024-09-15T21:37:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 -- 172.21.15.17:0/667647458 learned_addr learned my addr 172.21.15.17:0/667647458 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): get_auth_request con 0x7f4ce009ffc0 auth_method 0 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): get_auth_request con 0x7f4ce0135670 auth_method 0 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00a0890 0x7f4ce009d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f4ce00a0890 auth_method 0 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient(hunting): handle_auth_done global_id 204630 payload 824 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 -- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 msgr2=0x7f4ce01346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 0x7f4ce01346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 -- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00a0890 msgr2=0x7f4ce009d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00a0890 0x7f4ce009d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient: _finish_hunting 0 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 monclient: found mon.noname-a 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.17:3300/0 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 -- 172.21.15.17:0/667647458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce00a5730 con 0x7f4ce009ffc0 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00a0890 0x7f4ce009d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce4a9d640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 0x7f4ce01346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdffff640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 0x7f4ce013c620 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c10 tx=0x7f4cd00191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00b9030 con 0x7f4ce009ffc0 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: dump: 2024-09-15T21:37:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: _finish_auth 0 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: _check_auth_tickets 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd00b5410 con 0x7f4ce009ffc0 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 5 monclient: authenticate success, global_id 204630 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _renew_subs 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce0072800 con 0x7f4ce009ffc0 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00b1040 con 0x7f4ce009ffc0 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 10 monclient: dump: 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cde7fc640 4 set_mon_vals no callback set 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _renew_subs 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce0134be0 con 0x7f4ce009ffc0 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.17:3300/0 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce0135aa0 con 0x7f4ce009ffc0 2024-09-15T21:37:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f4cd00bc070 con 0x7f4ce009ffc0 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ccc03b9c0 0x7f4ccc03de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f4cd00ea050 con 0x7f4ce009ffc0 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ccc03b9c0 0x7f4ccc03de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.406+0000 7f4cdf7fe640 10 monclient: get_auth_request con 0x7f4ccc03b9c0 auth_method 0 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.410+0000 7f4cdf7fe640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ccc03b9c0 0x7f4ccc03de80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4ce009ea00 tx=0x7f4cd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.410+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f4cd00c3070 con 0x7f4ce009ffc0 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.410+0000 7f4cdcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.410+0000 7f4cdcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:54.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.582+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 --> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ce009d140 con 0x7f4ccc03b9c0 2024-09-15T21:37:54.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4cdcff9640 1 -- 172.21.15.17:0/667647458 <== mgr.4105 v2:172.21.15.119:6824/1238844171 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+77995 (secure 0 0 0) 0x7f4cd4011a10 con 0x7f4ccc03b9c0 2024-09-15T21:37:54.592 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-15T21:37:54.593 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-15T21:37:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ccc03b9c0 msgr2=0x7f4ccc03de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ccc03b9c0 0x7f4ccc03de80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4ce009ea00 tx=0x7f4cd4008040 comp rx=0 tx=0).stop 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 10 monclient: shutdown 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 msgr2=0x7f4ce013c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 0x7f4ce013c620 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c10 tx=0x7f4cd00191a0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 shutdown_connections 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f4ccc03b9c0 0x7f4ccc03de80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f4ce0135670 0x7f4ce01346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00a0890 0x7f4ce009d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 --2- 172.21.15.17:0/667647458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ce009ffc0 0x7f4ce013c620 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 >> 172.21.15.17:0/667647458 conn(0x7f4ce00a66a0 msgr2=0x7f4ce00a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 shutdown_connections 2024-09-15T21:37:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.590+0000 7f4ce6500640 1 -- 172.21.15.17:0/667647458 wait complete. 2024-09-15T21:37:54.611 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":866,"stamp":"2024-09-15T21:37:53.538839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":215480,"kb_used_data":20056,"kb_used_omap":9,"kb_used_meta":195318,"kb_avail":566015560,"statfs":{"total":579820584960,"available":579599933440,"internally_reserved":0,"allocated":20537344,"data_stored":17290996,"data_compressed":1361822,"data_compressed_allocated":2695168,"data_compressed_original":6922240,"omap_allocated":9567,"internal_metadata":200006305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004298"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":310,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:33.952588+0000","last_change":"2024-09-15T21:37:33.952588+0000","last_active":"2024-09-15T21:37:33.952588+0000","last_peered":"2024-09-15T21:37:33.952588+0000","last_clean":"2024-09-15T21:37:33.952588+0000","last_became_active":"2024-09-15T21:23:54.320552+0000","last_became_peered":"2024-09-15T21:23:54.320552+0000","last_unstale":"2024-09-15T21:37:33.952588+0000","last_undegraded":"2024-09-15T21:37:33.952588+0000","last_fullsized":"2024-09-15T21:37:33.952588+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:33.952525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:33.952525+0000","last_clean_scrub_stamp":"2024-09-15T21:37:33.952525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:41.643845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":182,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:37.454088+0000","last_change":"2024-09-15T21:37:37.454088+0000","last_active":"2024-09-15T21:37:37.454088+0000","last_peered":"2024-09-15T21:37:37.454088+0000","last_clean":"2024-09-15T21:37:37.454088+0000","last_became_active":"2024-09-15T21:23:54.301596+0000","last_became_peered":"2024-09-15T21:23:54.301596+0000","last_unstale":"2024-09-15T21:37:37.454088+0000","last_undegraded":"2024-09-15T21:37:37.454088+0000","last_fullsized":"2024-09-15T21:37:37.454088+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:37.453907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:37.453907+0000","last_clean_scrub_stamp":"2024-09-15T21:37:37.453907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:13:53.318328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":296,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:34.205199+0000","last_change":"2024-09-15T21:37:34.205199+0000","last_active":"2024-09-15T21:37:34.205199+0000","last_peered":"2024-09-15T21:37:34.205199+0000","last_clean":"2024-09-15T21:37:34.205199+0000","last_became_active":"2024-09-15T21:23:54.313525+0000","last_became_peered":"2024-09-15T21:23:54.313525+0000","last_unstale":"2024-09-15T21:37:34.205199+0000","last_undegraded":"2024-09-15T21:37:34.205199+0000","last_fullsized":"2024-09-15T21:37:34.205199+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:34.205039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:34.205039+0000","last_clean_scrub_stamp":"2024-09-15T21:37:34.205039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:31.718974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":182,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:34.442050+0000","last_change":"2024-09-15T21:37:34.442050+0000","last_active":"2024-09-15T21:37:34.442050+0000","last_peered":"2024-09-15T21:37:34.442050+0000","last_clean":"2024-09-15T21:37:34.442050+0000","last_became_active":"2024-09-15T21:23:54.313839+0000","last_became_peered":"2024-09-15T21:23:54.313839+0000","last_unstale":"2024-09-15T21:37:34.442050+0000","last_undegraded":"2024-09-15T21:37:34.442050+0000","last_fullsized":"2024-09-15T21:37:34.442050+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:34.441879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:34.441879+0000","last_clean_scrub_stamp":"2024-09-15T21:37:34.441879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:09.445913+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":239,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:38.409134+0000","last_change":"2024-09-15T21:37:38.409134+0000","last_active":"2024-09-15T21:37:38.409134+0000","last_peered":"2024-09-15T21:37:38.409134+0000","last_clean":"2024-09-15T21:37:38.409134+0000","last_became_active":"2024-09-15T21:23:55.901252+0000","last_became_peered":"2024-09-15T21:23:55.901252+0000","last_unstale":"2024-09-15T21:37:38.409134+0000","last_undegraded":"2024-09-15T21:37:38.409134+0000","last_fullsized":"2024-09-15T21:37:38.409134+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:38.408972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:38.408972+0000","last_clean_scrub_stamp":"2024-09-15T21:37:38.408972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:11.048605+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":296,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:31.936109+0000","last_change":"2024-09-15T21:37:31.936109+0000","last_active":"2024-09-15T21:37:31.936109+0000","last_peered":"2024-09-15T21:37:31.936109+0000","last_clean":"2024-09-15T21:37:31.936109+0000","last_became_active":"2024-09-15T21:23:54.318732+0000","last_became_peered":"2024-09-15T21:23:54.318732+0000","last_unstale":"2024-09-15T21:37:31.936109+0000","last_undegraded":"2024-09-15T21:37:31.936109+0000","last_fullsized":"2024-09-15T21:37:31.936109+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:31.936015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:31.936015+0000","last_clean_scrub_stamp":"2024-09-15T21:37:31.936015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:06.711707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":186,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:38.088269+0000","last_change":"2024-09-15T21:37:38.088269+0000","last_active":"2024-09-15T21:37:38.088269+0000","last_peered":"2024-09-15T21:37:38.088269+0000","last_clean":"2024-09-15T21:37:38.088269+0000","last_became_active":"2024-09-15T21:23:54.325406+0000","last_became_peered":"2024-09-15T21:23:54.325406+0000","last_unstale":"2024-09-15T21:37:38.088269+0000","last_undegraded":"2024-09-15T21:37:38.088269+0000","last_fullsized":"2024-09-15T21:37:38.088269+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:38.088115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:38.088115+0000","last_clean_scrub_stamp":"2024-09-15T21:37:38.088115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:03:35.111414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":211,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:33.981626+0000","last_change":"2024-09-15T21:37:33.981626+0000","last_active":"2024-09-15T21:37:33.981626+0000","last_peered":"2024-09-15T21:37:33.981626+0000","last_clean":"2024-09-15T21:37:33.981626+0000","last_became_active":"2024-09-15T21:23:54.300781+0000","last_became_peered":"2024-09-15T21:23:54.300781+0000","last_unstale":"2024-09-15T21:37:33.981626+0000","last_undegraded":"2024-09-15T21:37:33.981626+0000","last_fullsized":"2024-09-15T21:37:33.981626+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:33.981522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:33.981522+0000","last_clean_scrub_stamp":"2024-09-15T21:37:33.981522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:55.342363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":221,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:39.383054+0000","last_change":"2024-09-15T21:37:39.383054+0000","last_active":"2024-09-15T21:37:39.383054+0000","last_peered":"2024-09-15T21:37:39.383054+0000","last_clean":"2024-09-15T21:37:39.383054+0000","last_became_active":"2024-09-15T21:23:54.307998+0000","last_became_peered":"2024-09-15T21:23:54.307998+0000","last_unstale":"2024-09-15T21:37:39.383054+0000","last_undegraded":"2024-09-15T21:37:39.383054+0000","last_fullsized":"2024-09-15T21:37:39.383054+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:39.382880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:39.382880+0000","last_clean_scrub_stamp":"2024-09-15T21:37:39.382880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:00.697967+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:37.132248+0000","last_change":"2024-09-15T21:37:37.132248+0000","last_active":"2024-09-15T21:37:37.132248+0000","last_peered":"2024-09-15T21:37:37.132248+0000","last_clean":"2024-09-15T21:37:37.132248+0000","last_became_active":"2024-09-15T21:23:54.310738+0000","last_became_peered":"2024-09-15T21:23:54.310738+0000","last_unstale":"2024-09-15T21:37:37.132248+0000","last_undegraded":"2024-09-15T21:37:37.132248+0000","last_fullsized":"2024-09-15T21:37:37.132248+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:37.132074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:37.132074+0000","last_clean_scrub_stamp":"2024-09-15T21:37:37.132074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:11:08.161003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":240,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:34.175633+0000","last_change":"2024-09-15T21:37:34.175633+0000","last_active":"2024-09-15T21:37:34.175633+0000","last_peered":"2024-09-15T21:37:34.175633+0000","last_clean":"2024-09-15T21:37:34.175633+0000","last_became_active":"2024-09-15T21:23:54.311958+0000","last_became_peered":"2024-09-15T21:23:54.311958+0000","last_unstale":"2024-09-15T21:37:34.175633+0000","last_undegraded":"2024-09-15T21:37:34.175633+0000","last_fullsized":"2024-09-15T21:37:34.175633+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:34.175449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:34.175449+0000","last_clean_scrub_stamp":"2024-09-15T21:37:34.175449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:02.670441+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:37.002395+0000","last_change":"2024-09-15T21:37:37.002395+0000","last_active":"2024-09-15T21:37:37.002395+0000","last_peered":"2024-09-15T21:37:37.002395+0000","last_clean":"2024-09-15T21:37:37.002395+0000","last_became_active":"2024-09-15T21:23:54.309483+0000","last_became_peered":"2024-09-15T21:23:54.309483+0000","last_unstale":"2024-09-15T21:37:37.002395+0000","last_undegraded":"2024-09-15T21:37:37.002395+0000","last_fullsized":"2024-09-15T21:37:37.002395+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:37.002209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:37.002209+0000","last_clean_scrub_stamp":"2024-09-15T21:37:37.002209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:43.006570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":186,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:38.045022+0000","last_change":"2024-09-15T21:37:38.045022+0000","last_active":"2024-09-15T21:37:38.045022+0000","last_peered":"2024-09-15T21:37:38.045022+0000","last_clean":"2024-09-15T21:37:38.045022+0000","last_became_active":"2024-09-15T21:23:54.312375+0000","last_became_peered":"2024-09-15T21:23:54.312375+0000","last_unstale":"2024-09-15T21:37:38.045022+0000","last_undegraded":"2024-09-15T21:37:38.045022+0000","last_fullsized":"2024-09-15T21:37:38.045022+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:38.044856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:38.044856+0000","last_clean_scrub_stamp":"2024-09-15T21:37:38.044856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:23:59.077707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":312,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:32.929679+0000","last_change":"2024-09-15T21:37:32.929679+0000","last_active":"2024-09-15T21:37:32.929679+0000","last_peered":"2024-09-15T21:37:32.929679+0000","last_clean":"2024-09-15T21:37:32.929679+0000","last_became_active":"2024-09-15T21:23:54.318494+0000","last_became_peered":"2024-09-15T21:23:54.318494+0000","last_unstale":"2024-09-15T21:37:32.929679+0000","last_undegraded":"2024-09-15T21:37:32.929679+0000","last_fullsized":"2024-09-15T21:37:32.929679+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:32.929489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:32.929489+0000","last_clean_scrub_stamp":"2024-09-15T21:37:32.929489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:15.271808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:35.241036+0000","last_change":"2024-09-15T21:37:35.241036+0000","last_active":"2024-09-15T21:37:35.241036+0000","last_peered":"2024-09-15T21:37:35.241036+0000","last_clean":"2024-09-15T21:37:35.241036+0000","last_became_active":"2024-09-15T21:23:55.305236+0000","last_became_peered":"2024-09-15T21:23:55.305236+0000","last_unstale":"2024-09-15T21:37:35.241036+0000","last_undegraded":"2024-09-15T21:37:35.241036+0000","last_fullsized":"2024-09-15T21:37:35.241036+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:35.240836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:35.240836+0000","last_clean_scrub_stamp":"2024-09-15T21:37:35.240836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:56.428035+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:36.018903+0000","last_change":"2024-09-15T21:37:36.018903+0000","last_active":"2024-09-15T21:37:36.018903+0000","last_peered":"2024-09-15T21:37:36.018903+0000","last_clean":"2024-09-15T21:37:36.018903+0000","last_became_active":"2024-09-15T21:23:54.305792+0000","last_became_peered":"2024-09-15T21:23:54.305792+0000","last_unstale":"2024-09-15T21:37:36.018903+0000","last_undegraded":"2024-09-15T21:37:36.018903+0000","last_fullsized":"2024-09-15T21:37:36.018903+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:36.018723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:36.018723+0000","last_clean_scrub_stamp":"2024-09-15T21:37:36.018723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:24.660397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"261'74","reported_seq":543,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:33.117426+0000","last_change":"2024-09-15T21:37:33.117426+0000","last_active":"2024-09-15T21:37:33.117426+0000","last_peered":"2024-09-15T21:37:33.117426+0000","last_clean":"2024-09-15T21:37:33.117426+0000","last_became_active":"2024-09-15T21:23:54.325552+0000","last_became_peered":"2024-09-15T21:23:54.325552+0000","last_unstale":"2024-09-15T21:37:33.117426+0000","last_undegraded":"2024-09-15T21:37:33.117426+0000","last_fullsized":"2024-09-15T21:37:33.117426+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'74","last_scrub_stamp":"2024-09-15T21:37:33.117261+0000","last_deep_scrub":"261'74","last_deep_scrub_stamp":"2024-09-15T21:37:33.117261+0000","last_clean_scrub_stamp":"2024-09-15T21:37:33.117261+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:03.198813+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"261'2","reported_seq":460,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:36.436618+0000","last_change":"2024-09-15T21:37:36.436618+0000","last_active":"2024-09-15T21:37:36.436618+0000","last_peered":"2024-09-15T21:37:36.436618+0000","last_clean":"2024-09-15T21:37:36.436618+0000","last_became_active":"2024-09-15T21:23:54.316513+0000","last_became_peered":"2024-09-15T21:23:54.316513+0000","last_unstale":"2024-09-15T21:37:36.436618+0000","last_undegraded":"2024-09-15T21:37:36.436618+0000","last_fullsized":"2024-09-15T21:37:36.436618+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-09-15T21:37:36.436435+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-09-15T21:37:36.436435+0000","last_clean_scrub_stamp":"2024-09-15T21:37:36.436435+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:14.824818+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001029542,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:34.125828+0000","last_change":"2024-09-15T21:37:34.125695+0000","last_active":"2024-09-15T21:37:34.125828+0000","last_peered":"2024-09-15T21:37:34.125828+0000","last_clean":"2024-09-15T21:37:34.125828+0000","last_became_active":"2024-09-15T21:23:54.307605+0000","last_became_peered":"2024-09-15T21:23:54.307605+0000","last_unstale":"2024-09-15T21:37:34.125828+0000","last_undegraded":"2024-09-15T21:37:34.125828+0000","last_fullsized":"2024-09-15T21:37:34.125828+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:34.125536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:34.125536+0000","last_clean_scrub_stamp":"2024-09-15T21:37:34.125536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:19.281563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00090021299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":186,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:33.220443+0000","last_change":"2024-09-15T21:37:33.220443+0000","last_active":"2024-09-15T21:37:33.220443+0000","last_peered":"2024-09-15T21:37:33.220443+0000","last_clean":"2024-09-15T21:37:33.220443+0000","last_became_active":"2024-09-15T21:23:54.307364+0000","last_became_peered":"2024-09-15T21:23:54.307364+0000","last_unstale":"2024-09-15T21:37:33.220443+0000","last_undegraded":"2024-09-15T21:37:33.220443+0000","last_fullsized":"2024-09-15T21:37:33.220443+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:33.220367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:33.220367+0000","last_clean_scrub_stamp":"2024-09-15T21:37:33.220367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:28.788860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012860930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"261'3","reported_seq":470,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:35.427167+0000","last_change":"2024-09-15T21:37:35.427167+0000","last_active":"2024-09-15T21:37:35.427167+0000","last_peered":"2024-09-15T21:37:35.427167+0000","last_clean":"2024-09-15T21:37:35.427167+0000","last_became_active":"2024-09-15T21:23:54.314394+0000","last_became_peered":"2024-09-15T21:23:54.314394+0000","last_unstale":"2024-09-15T21:37:35.427167+0000","last_undegraded":"2024-09-15T21:37:35.427167+0000","last_fullsized":"2024-09-15T21:37:35.427167+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-09-15T21:37:35.427000+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-09-15T21:37:35.427000+0000","last_clean_scrub_stamp":"2024-09-15T21:37:35.427000+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:37.181508+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0019575920000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":447,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:32.941345+0000","last_change":"2024-09-15T21:37:32.941345+0000","last_active":"2024-09-15T21:37:32.941345+0000","last_peered":"2024-09-15T21:37:32.941345+0000","last_clean":"2024-09-15T21:37:32.941345+0000","last_became_active":"2024-09-15T21:23:55.900571+0000","last_became_peered":"2024-09-15T21:23:55.900571+0000","last_unstale":"2024-09-15T21:37:32.941345+0000","last_undegraded":"2024-09-15T21:37:32.941345+0000","last_fullsized":"2024-09-15T21:37:32.941345+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:32.941253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:32.941253+0000","last_clean_scrub_stamp":"2024-09-15T21:37:32.941253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:32.565146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001331406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:35.101514+0000","last_change":"2024-09-15T21:37:35.101514+0000","last_active":"2024-09-15T21:37:35.101514+0000","last_peered":"2024-09-15T21:37:35.101514+0000","last_clean":"2024-09-15T21:37:35.101514+0000","last_became_active":"2024-09-15T21:23:54.310902+0000","last_became_peered":"2024-09-15T21:23:54.310902+0000","last_unstale":"2024-09-15T21:37:35.101514+0000","last_undegraded":"2024-09-15T21:37:35.101514+0000","last_fullsized":"2024-09-15T21:37:35.101514+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:35.101320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:35.101320+0000","last_clean_scrub_stamp":"2024-09-15T21:37:35.101320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:30.543423+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010374360000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:34.989786+0000","last_change":"2024-09-15T21:37:34.989786+0000","last_active":"2024-09-15T21:37:34.989786+0000","last_peered":"2024-09-15T21:37:34.989786+0000","last_clean":"2024-09-15T21:37:34.989786+0000","last_became_active":"2024-09-15T21:23:54.294899+0000","last_became_peered":"2024-09-15T21:23:54.294899+0000","last_unstale":"2024-09-15T21:37:34.989786+0000","last_undegraded":"2024-09-15T21:37:34.989786+0000","last_fullsized":"2024-09-15T21:37:34.989786+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:34.989627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:34.989627+0000","last_clean_scrub_stamp":"2024-09-15T21:37:34.989627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:15.630461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013886599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-15T21:37:36.120278+0000","last_change":"2024-09-15T21:37:36.120278+0000","last_active":"2024-09-15T21:37:36.120278+0000","last_peered":"2024-09-15T21:37:36.120278+0000","last_clean":"2024-09-15T21:37:36.120278+0000","last_became_active":"2024-09-15T21:23:54.301715+0000","last_became_peered":"2024-09-15T21:23:54.301715+0000","last_unstale":"2024-09-15T21:37:36.120278+0000","last_undegraded":"2024-09-15T21:37:36.120278+0000","last_fullsized":"2024-09-15T21:37:36.120278+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:37:36.120083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:37:36.120083+0000","last_clean_scrub_stamp":"2024-09-15T21:37:36.120083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:26.482351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015518109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11642,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607801,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38672,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333168,"statfs":{"total":96636764160,"available":96597164032,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640503,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34512,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337328,"statfs":{"total":96636764160,"available":96601423872,"internally_reserved":0,"allocated":3407872,"data_stored":2705536,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640502,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34508,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337332,"statfs":{"total":96636764160,"available":96601427968,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640503,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34560,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337280,"statfs":{"total":96636764160,"available":96601374720,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640503,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38668,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333172,"statfs":{"total":96636764160,"available":96597168128,"internally_reserved":0,"allocated":3403776,"data_stored":2705517,"data_compressed":225030,"data_compressed_allocated":438272,"data_compressed_original":983040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34560,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337280,"statfs":{"total":96636764160,"available":96601374720,"internally_reserved":0,"allocated":3457024,"data_stored":3234445,"data_compressed":230851,"data_compressed_allocated":471040,"data_compressed_original":1495040,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5821,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:37:54.613 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient: get_monmap_and_config 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient: build_initial_monmap 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient: monmap: 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:37:54.738782+0000 2024-09-15T21:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:37:54.738782+0000 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:0: v1:172.21.15.17:6789/0 mon.noname-a-legacy 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:1: v1:172.21.15.119:6789/0 mon.noname-c-legacy 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-b 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:3: v2:172.21.15.17:3300/0 mon.noname-a 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:4: v2:172.21.15.119:3300/0 mon.noname-c 2024-09-15T21:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 Processor -- start 2024-09-15T21:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 -- start start 2024-09-15T21:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient: init 2024-09-15T21:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-09-15T21:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b480a4a90 0x7f1b480a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): picked mon.noname-c con 0x7f1b480a4a90 addr v2:172.21.15.119:3300/0 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1b480a53b0 addr v1:172.21.15.17:6789/0 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a5800 0x7f1b48137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): picked mon.noname-b con 0x7f1b480a5800 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b480a2c30 con 0x7f1b480a53b0 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1b480a0f00 con 0x7f1b480a4a90 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1b480a19b0 con 0x7f1b480a5800 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4e4a8640 10 monclient(hunting): authenticate will time out at 2198.134152s 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b477fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1b480a53b0 0x7f1b48137850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55964/0 (socket says 172.21.15.17:55964) 2024-09-15T21:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b477fe640 1 -- 172.21.15.17:0/740419508 learned_addr learned my addr 172.21.15.17:0/740419508 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 1 --2- 172.21.15.17:0/740419508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a5800 0x7f1b48137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): get_auth_request con 0x7f1b480a5800 auth_method 0 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 1 --2- 172.21.15.17:0/740419508 >> v2:172.21.15.119:3300/0 conn(0x7f1b480a4a90 0x7f1b480a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): get_auth_request con 0x7f1b480a4a90 auth_method 0 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b46ffd640 1 -- 172.21.15.17:0/740419508 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 762520137 0 0) 0x7f1b3c001120 con 0x7f1b480a53b0 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b46ffd640 10 monclient(hunting): _negotiate 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b46ffd640 10 monclient(hunting): my global_id is 204633 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b46ffd640 1 -- 172.21.15.17:0/740419508 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b30003530 con 0x7f1b480a53b0 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_done global_id 234233 payload 274 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 1 -- 172.21.15.17:0/740419508 >> v2:172.21.15.119:3300/0 conn(0x7f1b480a4a90 msgr2=0x7f1b480a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 1 --2- 172.21.15.17:0/740419508 >> v2:172.21.15.119:3300/0 conn(0x7f1b480a4a90 0x7f1b480a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 1 -- 172.21.15.17:0/740419508 >> v1:172.21.15.17:6789/0 conn(0x7f1b480a53b0 legacy=0x7f1b48137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient: _finish_hunting 0 2024-09-15T21:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 1 monclient: found mon.noname-b 2024-09-15T21:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.734+0000 7f1b4ca45640 1 -- 172.21.15.17:0/740419508 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b480a1c90 con 0x7f1b480a5800 2024-09-15T21:37:54.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 1 --2- 172.21.15.17:0/740419508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a5800 0x7f1b48137f60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009dc0 tx=0x7f1b4001c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a45eabd875502be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 1 -- 172.21.15.17:0/740419508 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400a66f0 con 0x7f1b480a5800 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: dump: 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.751 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: _finish_auth 0 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: _check_auth_tickets 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 1 -- 172.21.15.17:0/740419508 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b400a6890 con 0x7f1b480a5800 2024-09-15T21:37:54.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 5 monclient: authenticate success, global_id 234233 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 1 -- 172.21.15.17:0/740419508 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400a6c60 con 0x7f1b480a5800 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: get_monmap_and_config success 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 4 set_mon_vals no callback set 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b46ffd640 10 monclient: dump: 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.753 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: shutdown 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/740419508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a5800 msgr2=0x7f1b48137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/740419508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a5800 0x7f1b48137f60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009dc0 tx=0x7f1b4001c810 comp rx=0 tx=0).stop 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/740419508 shutdown_connections 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/740419508 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a5800 0x7f1b48137f60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/740419508 >> v2:172.21.15.119:3300/0 conn(0x7f1b480a4a90 0x7f1b480a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/740419508 >> 172.21.15.17:0/740419508 conn(0x7f1b480aabb0 msgr2=0x7f1b480a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:54.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/740419508 shutdown_connections 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/740419508 wait complete. 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: build_initial_monmap 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: monmap: 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 0 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 0.000000 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:created 0.000000 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 0 (unknown) 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.noname-a 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.noname-b 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.noname-c 2024-09-15T21:37:54.755 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 Processor -- start 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- start start 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: init 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: _reopen_session rank -1 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient: _add_conns ranks=[1,2,0] 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b480a4a90 0x7f1b4813dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): picked mon.noname-b con 0x7f1b480a4a90 addr [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a53b0 0x7f1b481206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): picked mon.noname-c con 0x7f1b480a53b0 addr [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 0x7f1b48137550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): picked mon.noname-a con 0x7f1b480a5800 addr [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48138620 con 0x7f1b480a5800 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481384a0 con 0x7f1b480a4a90 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): start opening mon connection 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 1 -- --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_getmap magic: 0 -- 0x7f1b48142550 con 0x7f1b480a53b0 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): _renew_subs 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4e4a8640 10 monclient(hunting): authenticate will time out at 2198.139400s 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 0x7f1b48137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 --2- >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a53b0 0x7f1b481206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 0x7f1b48137550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58786/0 (socket says 172.21.15.17:58786) 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 1 -- 172.21.15.17:0/4172126769 learned_addr learned my addr 172.21.15.17:0/4172126769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): get_auth_request con 0x7f1b480a53b0 auth_method 0 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): get_auth_request con 0x7f1b480a5800 auth_method 0 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b47fff640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b480a4a90 0x7f1b4813dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b47fff640 10 monclient(hunting): get_auth_request con 0x7f1b480a4a90 auth_method 0 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient(hunting): handle_auth_done global_id 234236 payload 824 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 -- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b480a4a90 msgr2=0x7f1b4813dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b480a4a90 0x7f1b4813dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 -- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 msgr2=0x7f1b48137550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 0x7f1b48137550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient: _finish_hunting 0 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 monclient: found mon.noname-c 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b477fe640 1 -- 172.21.15.17:0/4172126769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4809fcc0 con 0x7f1b480a53b0 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.738+0000 7f1b4ca45640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 0x7f1b48137550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b477fe640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a53b0 0x7f1b481206e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c730 tx=0x7f1b3c00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c017070 con 0x7f1b480a53b0 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: dump: 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.760 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: _finish_auth 0 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: _check_auth_tickets 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 5 monclient: authenticate success, global_id 234236 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 monclient: _renew_subs 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3c0058c0 con 0x7f1b480a53b0 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b480a1280 con 0x7f1b480a53b0 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: handle_config config(0 keys) 2024-09-15T21:37:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c005cd0 con 0x7f1b480a53b0 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 10 monclient: dump: 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:epoch 1 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:fsid bbfd2e88-7522-4bd5-ae16-c665761d9528 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:last_changed 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:created 2024-09-15T21:16:49.663269+0000 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:min_mon_release 19 (squid) 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:election_strategy: 1 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-15T21:37:54.762 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 monclient: _renew_subs 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b45ffb640 4 set_mon_vals no callback set 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b48121c00 con 0x7f1b480a53b0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b480a4e70 con 0x7f1b480a53b0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 4 ==== mgrmap(e 5) ==== 48867+0+0 (secure 0 0 0) 0x7f1b3c01a070 con 0x7f1b480a53b0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b3403b9f0 0x7f1b3403deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 5 ==== osd_map(261..261 src has 236..261) ==== 7657+0+0 (secure 0 0 0) 0x7f1b3c04fbc0 con 0x7f1b480a53b0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b47fff640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b3403b9f0 0x7f1b3403deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b47fff640 10 monclient: get_auth_request con 0x7f1b3403b9f0 auth_method 0 2024-09-15T21:37:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.742+0000 7f1b47fff640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b3403b9f0 0x7f1b3403deb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009af0 tx=0x7f1b38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:37:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.746+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187702 (secure 0 0 0) 0x7f1b4809fcc0 con 0x7f1b480a53b0 2024-09-15T21:37:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.746+0000 7f1b44ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:37:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.746+0000 7f1b44ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T21:37:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.918+0000 7f1b4e4a8640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T21:37:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.918+0000 7f1b4e4a8640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T21:37:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.918+0000 7f1b4e4a8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.17:3301/0 2024-09-15T21:37:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.918+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 --> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1b48069200 con 0x7f1b480a53b0 2024-09-15T21:37:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1b3c055080 con 0x7f1b480a53b0 2024-09-15T21:37:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b44ff9640 10 monclient: handle_config config(1 keys) 2024-09-15T21:37:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b45ffb640 4 set_mon_vals no callback set 2024-09-15T21:37:54.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b45ffb640 10 set_mon_vals mon_health_to_clog = false 2024-09-15T21:37:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b44ff9640 1 -- 172.21.15.17:0/4172126769 <== mon.2 v2:172.21.15.17:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f1b480a2c30 con 0x7f1b480a53b0 2024-09-15T21:37:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b44ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T21:37:54.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b44ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T21:37:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b3403b9f0 msgr2=0x7f1b3403deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b3403b9f0 0x7f1b3403deb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009af0 tx=0x7f1b38008040 comp rx=0 tx=0).stop 2024-09-15T21:37:54.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 10 monclient: shutdown 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a53b0 msgr2=0x7f1b481206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a53b0 0x7f1b481206e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c730 tx=0x7f1b3c00cbf0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 shutdown_connections 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:6824/1238844171,v1:172.21.15.119:6825/1238844171] conn(0x7f1b3403b9f0 0x7f1b3403deb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b480a5800 0x7f1b48137550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] conn(0x7f1b480a53b0 0x7f1b481206e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 --2- 172.21.15.17:0/4172126769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b480a4a90 0x7f1b4813dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 >> 172.21.15.17:0/4172126769 conn(0x7f1b480aabb0 msgr2=0x7f1b4809e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 shutdown_connections 2024-09-15T21:37:54.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-15T21:37:54.926+0000 7f1b4e4a8640 1 -- 172.21.15.17:0/4172126769 wait complete. 2024-09-15T21:37:54.951 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:37:54.951 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:37:54.952 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:37:54.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.038 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:37:55.038 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:37:55.039 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.113 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:37:55.113 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:37:55.114 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.196 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:37:55.196 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T21:37:55.197 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.277 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T21:37:55.278 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T21:37:55.278 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.367 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T21:37:55.367 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T21:37:55.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.454 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T21:37:55.454 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:37:55.454 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:37:55.455 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.484 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:37:55.484 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:37:55.484 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:37:55.485 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.505 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:37:55.505 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:37:55.505 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.567 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:37:55.567 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:37:55.568 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:37:55.593 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:37:55.594 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:37:55.594 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(AUTH_BAD_CAPS\)' | head -n 1 2024-09-15T21:37:55.632 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:37:55.632 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:37:55.913 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:37:55.914 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:37:56.095 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:37:56.096 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:37:56.270 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:37:56.271 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T21:37:56.454 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:37:56.454 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T21:37:56.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:37:56.663 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T21:37:56.854 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:37:56.855 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/data/mon.a.tgz 2024-09-15T21:37:56.855 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-15T21:37:56.864 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.ppSKNzm4ne 2024-09-15T21:37:56.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ppSKNzm4ne 2024-09-15T21:37:57.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.ppSKNzm4ne 2024-09-15T21:37:57.135 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.ppSKNzm4ne is 933KB 2024-09-15T21:37:57.201 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.ppSKNzm4ne 2024-09-15T21:37:57.206 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/data/mon.c.tgz 2024-09-15T21:37:57.207 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-15T21:37:57.255 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.ujNq21Xv8w 2024-09-15T21:37:57.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ujNq21Xv8w 2024-09-15T21:37:57.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.ujNq21Xv8w 2024-09-15T21:37:57.531 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.ujNq21Xv8w is 934KB 2024-09-15T21:37:57.597 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.ujNq21Xv8w 2024-09-15T21:37:57.611 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/data/mon.b.tgz 2024-09-15T21:37:57.611 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-15T21:37:57.617 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.YKAh09S1Z4 2024-09-15T21:37:57.617 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YKAh09S1Z4 2024-09-15T21:37:57.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.YKAh09S1Z4 2024-09-15T21:37:57.875 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.YKAh09S1Z4 is 932KB 2024-09-15T21:37:57.942 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.YKAh09S1Z4 2024-09-15T21:37:57.947 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:37:57.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:37:57.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:37:58.009 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:37:58.051 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:37:58.052 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:37:58.053 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/remote/smithi017/crash 2024-09-15T21:37:58.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:37:58.068 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/remote/smithi119/crash 2024-09-15T21:37:58.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:37:58.109 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:37:58.109 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:37:58.116 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:37:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29774.log 2024-09-15T21:37:58.142 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35144.log 2024-09-15T21:37:58.142 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-09-15T21:37:58.144 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35681.log 2024-09-15T21:37:58.144 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.29774.log.gz 2024-09-15T21:37:58.144 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34962.log 2024-09-15T21:37:58.144 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34529.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35144.log.gz 2024-09-15T21:37:58.144 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36715.log 2024-09-15T21:37:58.149 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-09-15T21:37:58.149 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38644.log 2024-09-15T21:37:58.149 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35681.log.gz 2024-09-15T21:37:58.149 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38774.log 2024-09-15T21:37:58.149 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34962.log.gz 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32440.log 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38774.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36715.log.gz 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38339.log 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38644.log.gz 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38774.log.gz 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28419.log 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38339.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32440.log.gz 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29304.log 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29330.log 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.38339.log.gz 2024-09-15T21:37:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36533.log 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28419.log.gz 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29930.log 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29304.log.gz 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29330.log.gz 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33907.log 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34780.log 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36533.log.gz 2024-09-15T21:37:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38391.log 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34780.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.29930.log.gz 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36429.log 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33907.log.gz 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30391.log 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34780.log.gz 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31583.log 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38391.log.gz 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30391.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36429.log.gzgzip 2024-09-15T21:37:58.164 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27988.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34477.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31583.log.gz 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30391.log.gz 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32414.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34910.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27988.log.gz 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31848.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32414.log.gz 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31874.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.34477.log.gz 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29670.log 2024-09-15T21:37:58.165 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31874.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31848.log.gz 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30903.log 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34910.log.gz 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31874.log.gz 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35944.log 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29670.log: /var/log/ceph/ceph-client.admin.30903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30799.log 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35944.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.29670.log.gz 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29696.log 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30903.log.gz 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35944.log.gz 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36845.log 2024-09-15T21:37:58.166 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38313.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30799.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34832.log 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29696.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36845.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38027.log 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36845.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36767.log 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28733.log 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36767.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38027.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34832.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36767.log.gz 2024-09-15T21:37:58.167 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.28733.log: --verbose -- /var/log/ceph/ceph-client.admin.31822.log 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31661.log 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31822.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28733.log.gz 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38800.log 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31661.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31822.log.gz 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33394.log 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38800.log.gz 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.0.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31661.log.gz 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35040.log 2024-09-15T21:37:58.168 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29982.log 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35196.log 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33394.log.gz 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29982.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35040.log.gz 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34243.log 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29982.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35196.log.gz 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35551.log 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31323.log 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34243.log: /var/log/ceph/ceph-client.admin.35551.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2024-09-15T21:37:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37382.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30591.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.35904.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31282.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.bar.35904.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30591.log.gz 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29478.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.bar.35904.log.gz 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25170.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31282.log: /var/log/ceph/ceph-client.admin.29478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29304.log 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.25170.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.31282.log.gz 2024-09-15T21:37:58.391 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30319.log 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29478.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25170.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33576.log 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32204.log 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29304.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33841.log 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30319.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34997.log 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33841.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33576.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32828.log 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32204.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34997.log: gzip -5 89.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33841.log.gz 2024-09-15T21:37:58.392 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.33311.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30391.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34997.log.gz 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31550.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30391.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32828.log.gz 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35503.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33311.log.gz 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31550.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30391.log.gz 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30247.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33552.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35503.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31550.log.gz 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29889.log 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33552.log: gzip 88.3% 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.30247.log.gz 2024-09-15T21:37:58.393 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- replaced with /var/log/ceph/ceph-client.admin.35503.log.gz -- 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.28914.log 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31670.log 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33552.log.gz 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29889.log.gz 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.31670.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29863.log 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28914.log.gz 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28733.log 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31670.log.gz/var/log/ceph/ceph-client.admin.29863.log: 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31746.log 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-09-15T21:37:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29112.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.29863.log.gz 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35468.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31746.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28733.log.gz 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28296.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35468.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31746.log.gz 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33158.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29112.log.gz 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30972.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35468.log.gz 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25242.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33158.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28296.log.gz 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35389.log 2024-09-15T21:37:58.395 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33158.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.25242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29010.log 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30972.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34249.log 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.25242.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29010.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35389.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31380.log 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29010.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34877.log 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34249.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34321.log 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30659.log.gz 2024-09-15T21:37:58.396 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34853.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34877.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31380.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32903.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34877.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34925.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34321.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28523.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34853.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.36129.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32903.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33792.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34925.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.bar.36129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33005.log 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28523.log.gz 2024-09-15T21:37:58.397 INFO:teuthology.orchestra.run.smithi119.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.bar.36129.log.gz 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30469.log 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.33005.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29232.log 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33792.log.gz 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30469.log: -- /var/log/ceph/ceph-client.admin.32636.log 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29232.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33005.log.gz 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35341.log 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32636.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30469.log.gz 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29232.log.gz 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28571.log 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32732.log 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32636.log.gz 2024-09-15T21:37:58.398 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.28571.log: --verbose -- /var/log/ceph/ceph-client.admin.30996.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35341.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28851.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28571.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32732.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35437.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30996.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33691.log: -- /var/log/ceph/ceph-client.admin.34685.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28851.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33134.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29184.log 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35437.log.gz 2024-09-15T21:37:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32321.log 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34685.log.gz 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.35860.log 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29184.log: /var/log/ceph/ceph-client.admin.32321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35165.log 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33134.log.gz 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29184.log.gz 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.bar.35860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32321.log.gz 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29697.log 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28962.log 88.8% 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.bar.35860.log.gz 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28938.log 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29697.log: /var/log/ceph/ceph-client.admin.28962.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35165.log.gz 2024-09-15T21:37:58.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33287.log 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29697.log.gz 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33715.log 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28962.log.gz 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35069.log 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28938.log.gz 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.33715.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32421.log 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33287.log.gz 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34225.log 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33715.log.gz 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35141.log 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35069.log.gz 2024-09-15T21:37:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31162.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32421.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.35141.log: -- /var/log/ceph/ceph-client.admin.31893.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31162.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34225.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34201.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35141.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34589.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31162.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29280.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34589.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34201.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31989.log 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34589.log.gz 2024-09-15T21:37:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30367.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32446.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29280.log.gz 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34949.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30367.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31989.log.gz 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32109.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30367.log.gz 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30103.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32446.log.gz 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29256.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34949.log.gz 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29401.log 2024-09-15T21:37:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30103.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32109.log.gz 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29672.log 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30103.log.gz 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29401.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.29256.log.gz -5 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33086.log 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29672.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29401.log.gz 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32660.log 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33086.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29672.log.gz 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32061.log 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35812.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33086.log.gz 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31138.log 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.32660.log.gz 2024-09-15T21:37:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35317.log 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33407.log 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32061.log.gz 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29034.log 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31138.log.gz 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.35948.log 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35317.log.gz 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30759.log 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33407.log.gz 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28454.log 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.bar.35948.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29034.log.gz 2024-09-15T21:37:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30011.log 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28454.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30759.log.gz 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31720.log 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30011.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28454.log.gz 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34973.log 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31720.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.30011.log.gz -- 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.foo.35665.log 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31965.log 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.bar.35948.log.gz 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31720.log.gz 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35045.log 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34973.log.gz 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29427.log/var/log/ceph/ceph-client.admin.31965.log: 2024-09-15T21:37:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30541.log 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31965.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30079.log 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35045.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34297.log 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29427.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32371.log 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30079.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30541.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.foo.35665.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29526.log 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30079.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32371.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34297.log.gz 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32395.log 2024-09-15T21:37:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29526.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.32371.log.gz -- 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.29647.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28807.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32395.log: /var/log/ceph/ceph-client.admin.29647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29526.log.gz 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31478.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28807.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32395.log.gz 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29647.log.gz 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28432.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28807.log.gz 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.31478.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33263.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32930.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28432.log: /var/log/ceph/ceph-client.admin.33263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28274.log 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31478.log.gz 2024-09-15T21:37:58.408 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28432.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30417.log 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33263.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28274.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32930.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32299.log 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31452.log 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28274.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28200.log 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30417.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31452.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32299.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.32179.log 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34057.log 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31452.log.gz 2024-09-15T21:37:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32179.log: /var/log/ceph/ceph-client.admin.30565.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28200.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29136.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30565.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34057.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32179.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29987.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.29136.log: --verbose -- /var/log/ceph/ceph-client.admin.31941.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30565.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29502.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29136.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35293.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31941.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29987.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29328.log 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31941.log.gz 2024-09-15T21:37:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31090.log 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29502.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31502.log 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35293.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29088.log 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31090.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29328.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29550.log 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31622.log 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31502.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31090.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30733.log 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29088.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31622.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29550.log.gz 2024-09-15T21:37:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32277.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30733.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31622.log.gz 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30033.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32277.log: /var/log/ceph/ceph-client.admin.31404.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29623.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30733.log.gz 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31404.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30033.log.gz 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32277.log.gz 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35365.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31404.log.gz 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28340.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29623.log: /var/log/ceph/ceph-client.admin.35365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28318.log 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29623.log.gz 2024-09-15T21:37:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28340.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35365.log.gz 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32756.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30922.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28340.log.gz 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28318.log.gz 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29160.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30922.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32756.log.gz 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35269.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.29160.log: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30922.log.gz 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25218.log 2024-09-15T21:37:58.413 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29160.log.gz 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35269.log.gz 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.35836.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31598.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.25218.log: /var/log/ceph/ceph-client.bar.35836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34733.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31598.log: 88.8% -- replaced with /var/log/ceph/ceph-client.bar.35836.log.gz 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.25218.log.gz 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.35621.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30685.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31598.log.gz 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34733.log.gz 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32957.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34709.log 2024-09-15T21:37:58.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30685.log: 87.9% -- replaced with /var/log/ceph/ceph-client.foo.35621.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30805.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28621.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32957.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30685.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.30805.log: /var/log/ceph/ceph-client.admin.25146.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34709.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28595.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30805.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28621.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.25146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30897.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31574.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28595.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.25146.log.gz 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33110.log 2024-09-15T21:37:58.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33913.log 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28595.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30897.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30223.log 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33913.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31574.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31792.log 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29058.log 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30223.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33110.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33913.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32611.log 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30223.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32562.log 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31792.log.gz 2024-09-15T21:37:58.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33239.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29058.log.gz 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33865.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32611.log.gz 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32345.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32562.log.gz 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34177.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33239.log.gz 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32981.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33865.log.gz 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31186.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34177.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32345.log.gz 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.35599.log 2024-09-15T21:37:58.417 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34177.log.gz 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32708.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32981.log.gz 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32852.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31186.log.gz 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34637.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.foo.35599.log.gz/var/log/ceph/ceph-client.admin.32708.log: 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33359.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32852.log.gz 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34805.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6%/var/log/ceph/ceph-client.admin.34637.log: -- replaced with /var/log/ceph/ceph-client.admin.32708.log.gz 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31044.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33359.log: /var/log/ceph/ceph-client.admin.34805.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34637.log.gz 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29352.log 2024-09-15T21:37:58.418 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33359.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:gzip 88.0%/var/log/ceph/ceph-client.admin.31044.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.34805.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29722.log 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30443.log 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31044.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34490.log 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29722.log: /var/log/ceph/ceph-client.admin.30443.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29352.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30517.log 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30443.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34490.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29722.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32155.log 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33961.log 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34490.log.gz 2024-09-15T21:37:58.419 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32155.log: /var/log/ceph/ceph-client.admin.32876.log 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30517.log.gz 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31234.log 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32155.log.gz 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33961.log.gz 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30827.log 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34661.log 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29599.log 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32876.log.gz 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31234.log.gz 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34466.log 2024-09-15T21:37:58.420 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30827.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29599.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34661.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28225.log 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.34466.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35021.log 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29599.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30057.log 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35021.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31917.log 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34466.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28225.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.30057.log: /var/log/ceph/ceph-client.admin.35245.log 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35021.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31917.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30057.log.gz 2024-09-15T21:37:58.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35245.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31917.log.gz 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28386.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28707.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.x.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35245.log.gz 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.35575.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28829.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28386.log.gz 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31068.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1%/var/log/ceph/ceph-client.admin.28829.log: -- replaced with /var/log/ceph/ceph-client.admin.28707.log.gz 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30151.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.foo.35575.log.gz 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25194.log 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28829.log.gz 2024-09-15T21:37:58.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30151.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31068.log.gz 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.25194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32229.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30151.log.gz 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30873.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5%/var/log/ceph/ceph-client.admin.32229.log: -- replaced with /var/log/ceph/ceph-client.admin.25194.log.gz 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32538.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30873.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28152.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32229.log.gz 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35117.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30873.log.gz 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32780.log 2024-09-15T21:37:58.423 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32538.log.gz 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28176.log 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35117.log: /var/log/ceph/ceph-client.admin.32780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31306.log 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.28152.log.gz 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35117.log.gz 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28176.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32780.log.gz 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28176.log.gz 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28872.log 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31819.log 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29963.log 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31306.log.gz 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35093.log 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28872.log.gz 2024-09-15T21:37:58.424 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5%gzip/var/log/ceph/ceph-client.admin.29963.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31258.log 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31819.log.gz 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28986.log 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29963.log.gz 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33767.log 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31258.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35093.log.gz 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28986.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31258.log.gz 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28986.log.gz 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28249.log 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28499.log 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34033.log 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33767.log.gz/var/log/ceph/ceph-client.admin.28499.log: 2024-09-15T21:37:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30127.log 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28249.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2%/var/log/ceph/ceph-client.admin.34033.log: -- replaced with /var/log/ceph/ceph-client.admin.28499.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34033.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36216.log 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34153.log 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34538.log 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30127.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28478.log 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36216.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34538.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34153.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34538.log.gz 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bazar.35492.log 2024-09-15T21:37:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30495.log 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.bazar.35492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32253.log 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28478.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr: 86.5%/var/log/ceph/ceph-client.admin.30495.log: -- replaced with /var/log/ceph/ceph-client.bazar.35492.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33479.log 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30495.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33528.log 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32253.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.33479.log: /var/log/ceph/ceph-client.admin.33431.log 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33383.log 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32253.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33479.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33431.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33528.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33431.log.gz 2024-09-15T21:37:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28763.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32085.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35413.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33383.log.gz 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33335.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32085.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28763.log.gz 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30709.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8%/var/log/ceph/ceph-client.admin.33212.log: -- replaced with /var/log/ceph/ceph-client.admin.35413.log.gz 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32085.log.gz 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33335.log.gz 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31845.log 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-09-15T21:37:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34418.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30709.log: /var/log/ceph/ceph-client.admin.31845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34129.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1%/var/log/ceph/ceph-client.admin.34418.log: -- replaced with /var/log/ceph/ceph-client.admin.30709.log.gz 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29208.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31845.log.gz 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34081.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34418.log.gz 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33625.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34129.log.gz 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29575.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29208.log.gz/var/log/ceph/ceph-client.admin.34081.log: 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34369.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30271.log 2024-09-15T21:37:58.429 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34081.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2%/var/log/ceph/ceph-client.admin.34369.log: -- replaced with /var/log/ceph/ceph-client.admin.33625.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29575.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34369.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33889.log 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30271.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31114.log 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30271.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32684.log 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36240.log 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31114.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33889.log.gz 2024-09-15T21:37:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29452.log 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32684.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31114.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36240.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32684.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36240.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28362.log 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30175.log 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28362.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31356.log 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29452.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28362.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33985.log 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31356.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30175.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34613.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31356.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-09-15T21:37:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33032.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34273.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33985.log.gz 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32013.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30343.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33032.log.gz 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32035.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% 91.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34273.log.gz 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31646.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30343.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32013.log.gz 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36104.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30343.log.gz 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36142.log 2024-09-15T21:37:58.432 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32035.log.gz 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.36104.log: --verbose -- /var/log/ceph/ceph-client.admin.33817.log 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31646.log.gz 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32131.log 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31020.log 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36142.log.gz 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31524.log 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.36104.log.gz 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33817.log.gz 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34514.log 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32131.log.gz 2024-09-15T21:37:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30635.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31020.log.gz 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28408.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31524.log.gz 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36191.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34514.log.gz 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33455.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28408.log: /var/log/ceph/ceph-client.admin.36191.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35777.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30635.log.gz 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28408.log.gz 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32587.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36191.log.gz 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30199.log 2024-09-15T21:37:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35777.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33455.log.gz 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35777.log.gz 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28547.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32587.log.gz 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25266.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28893.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30199.log.gz 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28547.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gz 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33937.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34901.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28547.log.gz 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.25266.log: /var/log/ceph/ceph-client.admin.33937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.35551.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34901.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29915.log 2024-09-15T21:37:58.435 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33937.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25266.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30781.log 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34901.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34009.log 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.foo.35551.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30613.log 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29915.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34829.log 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30781.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32512.log 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34009.log.gz 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34829.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.30613.log.gz -5 2024-09-15T21:37:58.436 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.foo.35801.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35213.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34829.log.gz 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35801.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.foo.35801.log.gz 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32512.log.gz 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33059.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34393.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33059.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35213.log.gz 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29939.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34393.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.33059.log.gz -- 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.34105.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28785.log 2024-09-15T21:37:58.437 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34393.log.gz 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34781.log 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29939.log.gz 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34105.log: /var/log/ceph/ceph-client.admin.28785.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34105.log.gz 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28785.log.gz 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30849.log 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30947.log 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30849.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34781.log.gz 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32804.log 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30849.log.gz 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33601.log 2024-09-15T21:37:58.438 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30947.log.gz 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31210.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.32804.log: /var/log/ceph/ceph-client.admin.33601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30295.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31210.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32804.log.gz 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33601.log.gz 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31210.log.gz -5 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31694.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.35643.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.30295.log: /var/log/ceph/ceph-client.admin.31694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29376.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.foo.35643.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31694.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30295.log.gz 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.foo.35643.log.gz 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33185.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28651.log 2024-09-15T21:37:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29839.log 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33185.log: /var/log/ceph/ceph-client.admin.28651.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29376.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33185.log.gz 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28651.log.gz 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31770.log 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29747.log 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29839.log.gz 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.29747.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31770.log.gz 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29747.log.gz 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.35926.log 2024-09-15T21:37:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35189.log: /var/log/ceph/ceph-client.admin.34562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28677.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.bar.35926.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.bar.35926.log.gz 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34345.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.28677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33743.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31332.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28677.log.gz 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33743.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34345.log.gz 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33743.log.gz 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.35882.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35527.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31332.log: /var/log/ceph/ceph-client.bar.35882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31428.log 2024-09-15T21:37:58.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35527.log: 89.1% 87.6% -- replaced with /var/log/ceph/ceph-client.bar.35882.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31332.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35527.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34442.log 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34757.log 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31869.log 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31428.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.34757.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34442.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34757.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29815.log 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.31869.log: /var/log/ceph/ceph-client.admin.29815.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31869.log.gz 2024-09-15T21:37:58.442 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29815.log.gz 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31323.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34243.log.gz 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31427.log 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35551.log.gz 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32821.log 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31427.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37382.log.gz 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31323.log.gz 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33725.log 2024-09-15T21:37:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32821.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31427.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34321.log 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32821.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36689.log 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28523.log 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33725.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34321.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28159.log 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28523.log: /var/log/ceph/ceph-client.admin.30183.log 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36689.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30157.log 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28523.log.gz 2024-09-15T21:37:58.456 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.28159.log.gz 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33987.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31123.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30183.log.gz 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30157.log.gz 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34138.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36062.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34138.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.33987.log.gz 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32716.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34138.log.gz 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31123.log.gz 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29512.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36062.log: /var/log/ceph/ceph-client.admin.32716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34675.log 2024-09-15T21:37:58.457 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36062.log.gz 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32716.log.gz 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31557.log 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29512.log.gz 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32082.log 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34675.log.gz 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27828.log 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29408.log 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31557.log.gz 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33134.log 2024-09-15T21:37:58.458 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.32082.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29408.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27828.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38235.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28575.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29408.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33134.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35248.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36741.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38235.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28575.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29956.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36819.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35248.log.gz 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31926.log 2024-09-15T21:37:58.459 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36741.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29956.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34044.log 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36819.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38722.log 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31926.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31149.log 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34044.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32847.log 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38722.log.gz 2024-09-15T21:37:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37434.log: gzip 89.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.31149.log.gz --verbose 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.36793.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32847.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30131.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32847.log.gz 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30772.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36793.log.gz 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37460.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30131.log.gz 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37460.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30772.log.gz 2024-09-15T21:37:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36585.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37460.log.gz 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25128.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33472.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36585.log: /var/log/ceph/ceph-client.admin.25128.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35430.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33472.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36585.log.gz 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.25128.log.gz 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35430.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33472.log.gz 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33855.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29278.log 2024-09-15T21:37:58.462 INFO:teuthology.orchestra.run.smithi017.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.35430.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28341.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37538.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28341.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33855.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30935.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28679.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.29278.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28341.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32228.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37538.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34425.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30935.log.gz 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32163.log 2024-09-15T21:37:58.463 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28679.log.gz 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37590.log 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32163.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32228.log.gz 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.34425.log.gz --verbose 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.28863.log 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37590.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32163.log.gz 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28549.log 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28863.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37590.log.gz 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.28863.log.gz 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29590.log 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.28549.log.gz 2024-09-15T21:37:58.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28133.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32925.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29590.log.gz 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31900.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37793.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28133.log.gz 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32925.log.gz 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35733.log 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31900.log.gz 2024-09-15T21:37:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34936.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.37793.log.gz 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35733.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35629.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33264.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34936.log: /var/log/ceph/ceph-client.admin.35629.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35733.log.gz 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29356.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36009.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35629.log.gz 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34936.log.gz 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29151.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33264.log.gz 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30079.log 2024-09-15T21:37:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33003.log 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29356.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29151.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36009.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37330.log 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36245.log 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.29151.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30079.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37330.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33003.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34503.log 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32977.log 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36245.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37330.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28367.log 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34503.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36245.log.gz 2024-09-15T21:37:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29878.log 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28367.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34503.log.gz 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32977.log.gz 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38592.log 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29878.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.28367.log.gz 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33368.log 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25011.log 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29878.log.gz 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38592.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gz 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35118.log 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.38592.log.gz 2024-09-15T21:37:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.25011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39034.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33368.log.gz 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34884.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.25011.log.gz 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35118.log.gz 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35785.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39034.log.gz 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30417.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35785.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36088.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34884.log.gz 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34165.log 2024-09-15T21:37:58.469 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35785.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34165.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30417.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31796.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36088.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36897.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34165.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35759.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30851.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35759.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31796.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38696.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36897.log.gz 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30694.log 2024-09-15T21:37:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30851.log: 90.2%/var/log/ceph/ceph-client.admin.38696.log: -- replaced with /var/log/ceph/ceph-client.admin.35759.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37923.log 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32518.log 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38696.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.30851.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30694.log: /var/log/ceph/ceph-client.admin.37923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32030.log 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32518.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30694.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30991.log 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.37923.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34217.log 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32518.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34451.log 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32030.log.gz 2024-09-15T21:37:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.34217.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28601.log 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30991.log.gz 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35352.log 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34217.log.gz 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28967.log 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34451.log.gz 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37252.log 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.28601.log.gz 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35014.log 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35352.log.gz 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28967.log.gz 2024-09-15T21:37:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28445.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37252.log.gz 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31201.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35014.log.gz 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35274.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31765.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28445.log.gz 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30313.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33829.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35274.log.gz 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30313.log: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.31201.log.gz 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34988.log 2024-09-15T21:37:58.473 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31765.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32743.log 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33829.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30313.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37131.log 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32743.log: -- replaced with /var/log/ceph/ceph-client.admin.33829.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34988.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29019.log 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32743.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37131.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30261.log 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37131.log.gz 2024-09-15T21:37:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33029.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29019.log.gz 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32284.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30446.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30261.log.gz 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34754.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33029.log.gz 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27899.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30446.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32284.log.gz 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35889.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30446.log.gz 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28289.log 2024-09-15T21:37:58.475 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34754.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.35889.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28497.log 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27899.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37871.log 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35889.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31453.log 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28289.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32258.log 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28497.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38261.log 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37871.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28705.log 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31453.log.gz 2024-09-15T21:37:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33160.log 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38131.log 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr: 91.4% 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32258.log.gz 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38261.log.gz/var/log/ceph/ceph-client.admin.33160.log: 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29644.log 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.28705.log.gz 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37408.log 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38131.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33160.log.gz 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37278.log 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31017.log 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38131.log.gz 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9%/var/log/ceph/ceph-client.admin.37278.log: -- replaced with /var/log/ceph/ceph-client.admin.29644.log.gz 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37408.log.gz -5 2024-09-15T21:37:58.477 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36140.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.37278.log.gz 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31017.log.gz 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30287.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37512.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36140.log.gz 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38443.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39060.log 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37512.log: -- replaced with /var/log/ceph/ceph-client.admin.30287.log.gz 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38443.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37512.log.gz 2024-09-15T21:37:58.478 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.38443.log.gz 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32492.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37819.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37975.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39060.log.gz 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33420.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32492.log.gz 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37975.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37819.log.gz 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37975.log.gz 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31175.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37897.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35655.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0%/var/log/ceph/ceph-client.admin.37897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30825.log 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33420.log.gz 2024-09-15T21:37:58.479 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31175.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35655.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37897.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35482.log 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35655.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29722.log 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29618.log 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30825.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37616.log 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35482.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28393.log 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29722.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29618.log: /var/log/ceph/ceph-client.admin.37616.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37616.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.29618.log.gz 2024-09-15T21:37:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30564.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29434.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28759.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8%/var/log/ceph/ceph-client.admin.29434.log: -- replaced with /var/log/ceph/ceph-client.admin.28393.log.gz 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36481.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30564.log.gz 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28759.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.29434.log.gz 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28759.log.gz 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37564.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29071.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36507.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36481.log.gz 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33055.log 2024-09-15T21:37:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36507.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.37564.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29071.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36036.log 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27982.log 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36507.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30590.log 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34269.log 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.33055.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30590.log: 88.7% 91.3% -- replaced with /var/log/ceph/ceph-client.admin.36036.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27982.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37225.log 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27995.log 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30590.log.gz 2024-09-15T21:37:58.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33316.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34269.log.gz 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28156.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37225.log.gz 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32056.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0%/var/log/ceph/ceph-client.admin.33316.log: -- replaced with /var/log/ceph/ceph-client.admin.27995.log.gz 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35456.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.32056.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30720.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0%/var/log/ceph/ceph-client.admin.35456.log: -- replaced with /var/log/ceph/ceph-client.admin.28156.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33316.log.gz 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32056.log.gz 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28315.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.30720.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38982.log 2024-09-15T21:37:58.483 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.35456.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35577.log 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38982.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.30720.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31739.log 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28315.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32622.log 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35577.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38982.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31739.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.35577.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31739.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29382.log 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36325.log 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32622.log.gz 2024-09-15T21:37:58.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34295.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27996.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8%/var/log/ceph/ceph-client.admin.36325.log: -- replaced with /var/log/ceph/ceph-client.admin.29382.log.gz 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34295.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36325.log.gz 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34295.log.gz 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30616.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27993.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32004.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27996.log.gz 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27993.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30616.log.gz 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27993.log.gz 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37714.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33081.log 2024-09-15T21:37:58.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36219.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32004.log.gz 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33081.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37714.log.gz 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33081.log.gz 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33581.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38826.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30235.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33581.log: /var/log/ceph/ceph-client.admin.38826.log: 88.2% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36219.log.gz 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33581.log.gz 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38826.log.gz 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35326.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32336.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35378.log 2024-09-15T21:37:58.486 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30235.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32336.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35326.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32336.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36377.log 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37079.log 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35378.log: /var/log/ceph/ceph-client.admin.36377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27687.log 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37079.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35378.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36377.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37079.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36455.log 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31069.log 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33803.log 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27687.log.gz 2024-09-15T21:37:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31069.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36455.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31069.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37741.log 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33803.log: /var/log/ceph/ceph-client.admin.37741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34073.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33803.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37741.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33446.log 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29045.log 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36351.log 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1%/var/log/ceph/ceph-client.admin.29045.log: -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33446.log.gz 2024-09-15T21:37:58.488 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29045.log.gz 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30105.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36351.log: /var/log/ceph/ceph-client.admin.30105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32362.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36351.log.gz/var/log/ceph/ceph-client.admin.39162.log: 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32466.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.30105.log.gz 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30961.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32362.log.gz 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32466.log: /var/log/ceph/ceph-client.admin.30961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31687.log 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36403.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32466.log.gz 2024-09-15T21:37:58.489 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30961.log.gz 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36663.log 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28941.log 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31687.log: /var/log/ceph/ceph-client.admin.36663.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28941.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31687.log.gz 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36663.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37486.log 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.28941.log.gz 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33751.log 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38930.log 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33751.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37486.log.gz 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33751.log.gz 2024-09-15T21:37:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21408.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36559.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.tmp-client.admin.21408.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21408.log.gz 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38930.log.gz 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37304.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34649.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36559.log.gz 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39086.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39086.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37304.log.gz 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39086.log.gz 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-09-15T21:37:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28162.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33212.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr: 90.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38878.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29097.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28162.log.gz 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34558.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.38878.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.32899.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29097.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-09-15T21:37:58.492 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.29097.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35811.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32899.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32544.log 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28915.log 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39112.log 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32544.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29125.log 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39112.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28915.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29125.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39112.log.gz 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28653.log 2024-09-15T21:37:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.audit.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.29125.log.gz 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29564.log 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29564.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28653.log.gz 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29564.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29460.log 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38748.log 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29460.log.gz 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38538.log 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38748.log.gz 2024-09-15T21:37:58.494 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36923.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36192.log.gz 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.38538.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32873.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29800.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38538.log.gz 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32873.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36923.log.gz 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29486.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28471.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32873.log.gz 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35404.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29800.log.gz 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29486.log.gz 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28072.log 2024-09-15T21:37:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28471.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37105.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35404.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28837.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29904.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28072.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30339.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37105.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29904.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.28837.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29904.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27853.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37160.log 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30339.log.gz 2024-09-15T21:37:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31505.log 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37160.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27853.log.gz 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37160.log.gz -5 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31375.log 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32795.log 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38852.log 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31505.log.gz 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32795.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37053.log 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31375.log.gz 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38852.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32795.log.gz 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38852.log.gz 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38956.log 2024-09-15T21:37:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31531.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31253.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37053.log.gz 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31531.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38956.log.gz 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31097.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38001.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31531.log.gz 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36637.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31253.log.gz 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38001.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31097.log.gz 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33959.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38001.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34806.log 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36637.log: 2024-09-15T21:37:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33959.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36637.log.gz 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33550.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33959.log.gz 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32134.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34623.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33550.log: /var/log/ceph/ceph-client.admin.32134.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34806.log.gz 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33550.log.gz 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32134.log.gz 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33342.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37845.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31349.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34623.log.gz 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31401.log 2024-09-15T21:37:58.499 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.33342.log.gz 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31349.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37845.log.gz 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38183.log 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31349.log.gz 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30746.log 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31401.log: /var/log/ceph/ceph-client.admin.38183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31609.log 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0%/var/log/ceph/ceph-client.admin.30746.log: -- replaced with /var/log/ceph/ceph-client.admin.31401.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38183.log.gz 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30538.log 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30746.log.gz/var/log/ceph/ceph-client.admin.31609.log: 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33673.log 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33498.log 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31609.log.gz 2024-09-15T21:37:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33673.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30538.log.gz 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30668.log 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28993.log 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33673.log.gz/var/log/ceph/ceph-client.admin.30668.log: 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.33498.log.gz 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30668.log.gz 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37356.log 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27925.log 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32310.log 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.28993.log.gz 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27925.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37356.log.gz 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32951.log 2024-09-15T21:37:58.501 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27925.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33646.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32951.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32310.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33777.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32951.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31635.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33646.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36299.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33777.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37027.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31635.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35222.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.31635.log.gz 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30209.log 2024-09-15T21:37:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37027.log: /var/log/ceph/ceph-client.admin.35222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38469.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30209.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37027.log.gz 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.35222.log.gz 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33699.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30209.log.gz 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32388.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38469.log: /var/log/ceph/ceph-client.admin.33699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28627.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32388.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33699.log.gz 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38469.log.gz 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32388.log.gz 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38079.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34858.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28627.log: /var/log/ceph/ceph-client.admin.38079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37949.log 2024-09-15T21:37:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34858.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28627.log.gz 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38079.log.gz 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34858.log.gz -5 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.28785.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36273.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30877.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28785.log: /var/log/ceph/ceph-client.admin.36273.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37949.log.gz 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28785.log.gz 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30877.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.36273.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38618.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8%/var/log/ceph/ceph-client.admin.35837.log: -- replaced with /var/log/ceph/ceph-client.admin.30877.log.gz 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29252.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5%/var/log/ceph/ceph-client.admin.38618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-09-15T21:37:58.504 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29252.log: 89.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35707.log 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38618.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30642.log 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.29252.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28889.log 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30642.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35707.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30642.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33524.log 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32651.log 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31043.log 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28889.log.gz 2024-09-15T21:37:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32651.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33524.log.gz 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29186.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32651.log.gz 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36949.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28811.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31043.log.gz 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36949.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29186.log.gz 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36975.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28811.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36949.log.gz 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38053.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.28811.log.gz 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35603.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32596.log 2024-09-15T21:37:58.506 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8%/var/log/ceph/ceph-client.admin.38053.log: -- replaced with /var/log/ceph/ceph-client.admin.36975.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35603.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.38053.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35603.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31952.log 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30511.log 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34018.log 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32596.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30511.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31952.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30511.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38566.log 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29826.log 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27873.log 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34018.log.gz 2024-09-15T21:37:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29826.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38566.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.29826.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30013.log 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29852.log 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27873.log: /var/log/ceph/ceph-client.admin.30013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33881.log 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27873.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29852.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30013.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29852.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31227.log 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33107.log 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29538.log 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33881.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33107.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31227.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33107.log.gz 2024-09-15T21:37:58.508 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38105.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38670.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32570.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9%/var/log/ceph/ceph-client.admin.38670.log: -- replaced with /var/log/ceph/ceph-client.admin.29538.log.gz 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38105.log.gz 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38670.log.gz 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25202.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32108.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.25202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29748.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1%/var/log/ceph/ceph-client.admin.32108.log: -- replaced with /var/log/ceph/ceph-client.admin.32570.log.gz 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25202.log.gz 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32108.log.gz 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35170.log 2024-09-15T21:37:58.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36611.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35170.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37001.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29748.log.gz 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36611.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35170.log.gz 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36611.log.gz 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27997.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27951.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28163.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37001.log.gz 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27951.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27997.log.gz 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27951.log.gz 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33238.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27803.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28163.log: /var/log/ceph/ceph-client.admin.33238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31713.log 2024-09-15T21:37:58.510 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28163.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.27803.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33238.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27803.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31479.log 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32769.log 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30365.log 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31713.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32769.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31479.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32769.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33933.log 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.30365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31978.log 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35300.log 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.30365.log.gz 2024-09-15T21:37:58.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31978.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33933.log.gz 2024-09-15T21:37:58.512 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31978.log.gz 2024-09-15T21:37:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35092.log 2024-09-15T21:37:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35300.log: /var/log/ceph/ceph-client.admin.35092.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35300.log.gz 2024-09-15T21:37:58.512 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35092.log.gz 2024-09-15T21:37:58.525 INFO:teuthology.orchestra.run.smithi119.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:37:58.776 INFO:teuthology.orchestra.run.smithi017.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:37:58.807 INFO:teuthology.orchestra.run.smithi119.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:37:59.147 INFO:teuthology.orchestra.run.smithi119.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T21:37:59.245 INFO:teuthology.orchestra.run.smithi017.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:37:59.287 INFO:teuthology.orchestra.run.smithi119.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T21:37:59.288 INFO:teuthology.orchestra.run.smithi017.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:37:59.576 INFO:teuthology.orchestra.run.smithi119.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T21:37:59.577 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-15T21:37:59.577 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m1.422s 2024-09-15T21:37:59.577 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m4.264s 2024-09-15T21:37:59.577 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.279s 2024-09-15T21:37:59.660 INFO:teuthology.orchestra.run.smithi017.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:37:59.924 INFO:teuthology.orchestra.run.smithi017.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:37:59.925 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-15T21:37:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m1.807s 2024-09-15T21:37:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:user 0m6.258s 2024-09-15T21:37:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m0.299s 2024-09-15T21:37:59.926 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:37:59.926 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/remote/smithi017/log 2024-09-15T21:37:59.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:38:02.892 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/remote/smithi119/log 2024-09-15T21:38:02.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:38:03.697 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:38:03.708 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:38:03.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:38:03.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:38:03.773 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:38:03.781 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:38:03.781 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:38:03.783 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:38:03.792 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:38:03.793 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2024-09-15T21:38:03.793 INFO:teuthology.orchestra.run.smithi017.stdout:*hv01.front.sepi 67.205.162.81 3 u 18 64 377 0.092 -2.777 0.406 2024-09-15T21:38:03.793 INFO:teuthology.orchestra.run.smithi017.stdout:+hv02.front.sepi 63.231.80.2 3 u 66 64 377 0.088 -1.618 0.699 2024-09-15T21:38:03.793 INFO:teuthology.orchestra.run.smithi017.stdout:+hv03.front.sepi 74.6.168.72 3 u 15 64 377 0.096 +0.379 0.418 2024-09-15T21:38:03.793 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-15T21:38:03.796 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:38:03.796 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2024-09-15T21:38:03.796 INFO:teuthology.orchestra.run.smithi119.stdout:*hv01.front.sepi 67.205.162.81 3 u 18 64 377 0.078 -9.447 0.991 2024-09-15T21:38:03.796 INFO:teuthology.orchestra.run.smithi119.stdout:+hv02.front.sepi 63.231.80.2 3 u 2 64 377 0.090 -7.069 0.496 2024-09-15T21:38:03.796 INFO:teuthology.orchestra.run.smithi119.stdout:+hv03.front.sepi 74.6.168.72 3 u 13 64 377 0.127 -5.970 0.473 2024-09-15T21:38:03.796 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-15T21:38:03.797 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:38:03.805 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:38:03.806 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:38:03.814 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:38:03.823 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:38:03.831 INFO:teuthology.task.internal:Duration was 1854.153450 seconds 2024-09-15T21:38:03.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:38:03.841 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:38:03.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:38:03.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:38:03.882 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:38:03.882 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-09-15T21:38:03.883 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:38:03.932 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-09-15T21:38:03.932 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:38:03.942 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:38:03.942 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:38:03.975 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:38:04.012 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:38:04.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:38:04.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:38:04.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:38:04.122 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:38:04.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:38:04.133 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:38:04.143 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:38:04.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:38:04.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:38:04.185 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-09-15T21:38:04.196 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-09-15T21:38:04.212 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:38:04.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:38:04.258 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:38:04.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:38:04.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:38:04.276 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:38:04.276 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/remote/smithi017 2024-09-15T21:38:04.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:38:04.324 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905675/remote/smithi119 2024-09-15T21:38:04.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:38:04.343 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:38:04.343 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:38:04.364 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:38:04.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:38:04.398 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:38:04.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:38:04.407 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:38:04.408 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:38:04.412 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:38:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:38 /home/ubuntu/cephtest 2024-09-15T21:38:04.436 INFO:teuthology.orchestra.run.smithi119.stdout: 655474 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:38 /home/ubuntu/cephtest 2024-09-15T21:38:04.437 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:38:04.448 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:38:04.469 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} duration: 1854.1534502506256 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:38:04.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:38:04.560 INFO:teuthology.run:pass